Data storage device and method for using a secondary rendering engine for prioritized data

Abstract

To provide a data storage device and method for using a secondary rendering engine for prioritized data.SOLUTION: One or more processors in a data storage device are configured to receive requests for non-private data and private data stored in a memory, transmit the non-private data to a primary rendering engine within a host in order to render the non-private data on a display device, and transmit, in response to a user being authorized to access the private data, the private data to a secondary rendering engine external to the host in order to render the private data on the display device. By transmitting the private data to the secondary rendering engine instead of the primary rendering engine within the host, the private data is rendered inaccessible by the host.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] Virtual reality (VR) and augmented reality (AR) applications are becoming more common, and the number of devices using such applications is expected to increase. Virtual reality devices typically take the form of goggles with built-in screens and lenses that display a digitally created world, and the user's entire field of view is of the digitally created world, not the "real world" outside the goggles. In contrast, augmented reality devices can present a display overlaid on the user's view of the "real world." Augmented reality devices can take the form of goggles, but can also take the form of an additional device, such as a computing device (e.g., a mobile phone or tablet) with a camera and a display screen that displays real objects detected by the camera and an overlay generated by the mobile device. [Brief explanation of the drawings]

[0002] [Figure 1A] FIG. 2 is a block diagram of a data storage device according to an embodiment. [Figure 1B] FIG. 2 is a block diagram illustrating a storage module of one embodiment. [Figure 1C] FIG. 1 is a block diagram illustrating a hierarchical storage system of one embodiment. [Figure 2A] 1B is a block diagram illustrating components of a controller of the data storage device illustrated in FIG. 1A according to one embodiment. [Figure 2B] 1B is a block diagram illustrating components of the data storage device illustrated in FIG. 1A, according to one embodiment. [Figure 3] FIG. 2 is a block diagram of a host and a data storage device of one embodiment. [Figure 4] 1 is an illustration of an exemplary architecture of one embodiment. [Figure 5]1 is a flowchart of an embodiment method for using a secondary rendering engine on prioritized data. [Figure 6] 1 is an illustration of an exemplary architecture of one embodiment. [Figure 7] 1 is a flowchart of an embodiment method for executing an urgent read command. DETAILED DESCRIPTION OF THE INVENTION

[0003] The following embodiments generally relate to data storage devices and methods for using a secondary rendering engine for prioritized data. In one embodiment, a data storage device is provided that includes a memory and one or more processors. The one or more processors are configured, individually or in combination, to: transmit unrestricted data stored in the memory to a host, the host including a primary rendering engine configured to render the unrestricted data on a display device, in response to receiving information from the host identifying the unrestricted data; receive information from the host regarding a user; determine based on the information about the user whether the user is authorized to access the restricted data stored in the memory; and, in response to determining that the user is authorized to access the restricted data, transmit the restricted data to a secondary rendering engine configured to render the restricted data as an overlay to the rendered unrestricted data on the display device; wherein by transmitting the restricted data to the secondary rendering engine instead of the primary rendering engine in the host, the restricted data is not exposed to the host.

[0004] In some embodiments the display device comprises a virtual reality display device.

[0005] In some embodiments, the display device comprises an augmented reality display device.

[0006] In some embodiments, the secondary rendering engine is located in a data storage device.

[0007] In some embodiments, the secondary rendering engine is located external to the data storage device.

[0008] In some embodiments, the unrestricted data and the restricted data are stored in different partitions in memory.

[0009] In some embodiments, the secondary rendering engine comprises a graphics processing unit (GPU) and a frame buffer.

[0010] In some embodiments, the one or more processors, individually or in combination, are further configured to identify, directly or indirectly from information received from the host, the non-restricted data to read from the memory.

[0011] In some embodiments, the one or more processors, individually or in combination, are further configured to read the restricted data from the memory using an emergency read command.

[0012] In some embodiments, the one or more processors, individually or in combination, are further configured to store the restricted data in a host memory buffer.

[0013] In some embodiments, the memory comprises a three-dimensional memory.

[0014] In another embodiment, a method is provided that is executed in a data storage device that includes a memory and that is in communication with a host that includes a primary rendering engine, the method including: receiving a request for non-private data and private data stored in the memory; sending the non-private data to a primary rendering engine within the host for rendering the non-private data on a display device; and, in response to a user being authorized to access the private data, sending the private data to a secondary rendering engine external to the host for rendering the private data on the display device, wherein by sending the private data to the secondary rendering engine instead of the primary rendering engine within the host, the private data is inaccessible by the host.

[0015] In some embodiments, the secondary rendering engine is further configured to receive an identification of a particular slot within the video buffer of the primary rendering engine in which the rendered private data is to be stored.

[0016] In some embodiments, the secondary rendering engine is further configured to receive an identification of the rendering format.

[0017] In some embodiments, the secondary rendering engine is further configured to interleave the rendered private data with the rendered non-private data.

[0018] In some embodiments, the secondary rendering engine is further configured to binary add the rendered private data with the rendered non-private data.

[0019] In some embodiments, the secondary rendering engine is further configured to receive the rendered non-private data from the primary rendering engine, modify the rendered non-private data with the rendered private data, and provide the results of the modifications to the display device.

[0020] In some embodiments, the secondary rendering engine is further configured to output the rendered private data as key value (KV) pairs or in logical block address (LBA) format.

[0021] In some embodiments, the display device comprises a virtual reality display device or an augmented reality display device.

[0022] In another embodiment, a data storage device comprising: a memory; and means for providing private data stored in the memory to a secondary rendering engine external to a host including a primary rendering engine within an augmented reality / virtual reality environment in response to a user being authenticated to access the private data.

[0023] Other embodiments are possible, each of which can be used alone or together in combination. Accordingly, various embodiments will now be described with reference to the accompanying drawings.

[0024] Embodiment

[0025] The following embodiments relate to data storage devices (DSDs). As used herein, "data storage device" refers to a non-volatile device that stores data. Examples of DSDs include, but are not limited to, hard disk drives (HDDs), solid state drives (SSDs), tape drives, hybrid drives, etc. Details of exemplary DSDs are provided below.

[0026] Examples of data storage devices suitable for use in implementing aspects of these embodiments are shown in FIGS. 1A-1C. Note that these are merely examples and other implementations may be used. FIG. 1A is a block diagram illustrating a data storage device 100 according to one embodiment. Referring to FIG. 1A, the data storage device 100 in this example includes a controller 102 coupled to nonvolatile memory, which may be comprised of one or more nonvolatile memory dies 104. As used herein, the term die refers to a collection of nonvolatile memory cells and associated circuitry for managing the physical operation of these nonvolatile memory cells formed on a single semiconductor substrate. The controller 102 interfaces with a host system and transmits command sequences for read, program, and erase operations to the nonvolatile memory dies 104. Also, as used herein, the phrases "in communication with" or "coupled to" can mean in direct communication with / coupled to one or more components, or indirect communication with / coupled through one or more components, which may or may not be shown or described herein. The communication / coupling can be wired or wireless.

[0027] The controller 102 (which may be a non-volatile memory controller (e.g., flash, resistive random-access memory (ReRAM), phase-change memory (PCM), or magnetoresistive random-access memory (MRAM) controller)) may include one or more components, individually or in combination, configured to perform certain functions, including but not limited to those described herein and illustrated in the flowcharts. For example, as shown in FIG. 2A , the controller 102 may include one or more processors 138, individually or in combination, configured to perform functions, including but not limited to those described herein and illustrated in the flowcharts, by executing computer-readable program code stored in one or more non-transitory memories 139 within the controller 102 and / or external to the controller 102 (e.g., in random access memory (RAM) 116 or read-only memory (ROM) 118). As another example, one or more components may include circuits such as, but not limited to, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.

[0028] In one exemplary embodiment, the non-volatile memory controller 102 is a device that manages data stored in non-volatile memory and communicates with a host, such as a computer or electronic device, using any suitable operating system. The non-volatile memory controller 102 can have a variety of functionality in addition to the specific functionality described herein. For example, the non-volatile memory controller can format the non-volatile memory to ensure that the memory is operating properly, map out bad non-volatile memory cells, and allocate spare cells to replace future failed cells. Some of the spare cells can be used to hold firmware (and / or other metadata used for housekeeping and tracking) to operate the non-volatile memory controller and implement other features. In operation, when the host needs to read data from or write data to the non-volatile memory, the host can communicate with the non-volatile memory controller. If the host provides a logical address to which data is to be read / written, the non-volatile memory controller can translate the logical address received from the host into a physical address within the non-volatile memory. The non-volatile memory controller may also perform various memory management functions such as, but not limited to, wear leveling (distributing writes to avoid wearing out particular blocks of memory that would otherwise be written to repeatedly) and garbage collection (moving only valid pages of data to a new block after the block becomes full so that the full block can be erased and reused).

[0029] The non-volatile memory die 104 may include any suitable non-volatile storage medium, including resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), phase change memory (PCM), NAND flash memory cells, and / or NOR flash memory cells. The memory cells may take the form of solid-state (e.g., flash) memory cells and may be single-time programmable, multiple-time programmable, or multi-time programmable. The memory cells may also be single-level cells (SLC), multi-level cells (MLC) (e.g., dual-level cells, triple-level cells (TLC), quad-level cells (QLC), etc.), or may use other memory cell level technologies now known or later developed. The memory cells may also be fabricated in two or three dimensions.

[0030] The interface between the controller 102 and the non-volatile memory die 104 may be any suitable flash interface, such as toggle mode 200, 400, or 800. In one embodiment, data storage device 100 may be a card-based system, such as a secure digital (SD) or micro secure digital (micro-SD) card. In an alternative embodiment, data storage device 100 may be part of an embedded data storage device.

[0031] 1A, data storage device 100 (sometimes referred to herein as a storage module) includes a single channel between controller 102 and non-volatile memory die 104, although the subject matter described herein is not limited to having a single memory channel. For example, in some architectures (such as those shown in FIGS. 1B and 1C), two, four, eight, or more memory channels may exist between the controller and the memory device, depending on the capabilities of the controller. In any of the embodiments described herein, even when a single channel is shown in the figures, more than a single channel may exist between the controller and the memory die.

[0032] 1B illustrates a storage module 200 including multiple non-volatile data storage devices 100. Accordingly, the storage module 200 may include a storage controller 202, which interfaces with a host and a data storage system 204, which includes multiple data storage devices 100. The interface between the storage controller 202 and the data storage devices 100 may be a bus interface, such as a serial advanced technology attachment (SATA), a peripheral component interconnect express (PCIe) interface, a double-data-rate (DDR) interface, or a serial attached small scale compute interface (SAS / SCSI). In one embodiment, the storage module 200 may be a solid state drive (SSD) or a non-volatile dual in-line memory module (NVDIMM), such as those found in server PCs or portable computing devices, such as laptop computers and tablet computers.

[0033] FIG. 1C is a block diagram illustrating a hierarchical storage system. The hierarchical storage system 250 includes multiple storage controllers 202, each controlling a respective data storage device 204. A host system 252 may access memory in the storage system 250 through a bus interface. In one embodiment, the bus interface may be a Non-Volatile Memory Express (NVMe) or Fibre Channel over Ethernet (FCoE) interface. In one embodiment, the system illustrated in FIG. 1C may be a rack-mountable mass storage system accessible by multiple host computers, such as might be found in a data center or other location where mass storage is needed.

[0034] Referring again to FIG. 2A , the controller 102 in this example also includes a front-end module 108 that interfaces with the host, a back-end module 110 that interfaces with one or more non-volatile memory dies 104, and various other components or modules, such as, but not limited to, a buffer manager / bus controller module that manages buffers in RAM 116 and controls internal bus arbitration for the controller 102. The modules may include one or more processors or components, as discussed above. ROM 118 may store system boot code. While illustrated in FIG. 2A as being located separately from the controller 102, in other embodiments, one or both of the RAM 116 and the ROM 118 may be located within the controller 102. In still other embodiments, portions of the RAM 116 and the ROM 118 may be located both within and outside the controller 102.

[0035] The front-end module 108 includes a host interface 120 and a physical layer interface (PHY) 122 that provide an electrical interface with a host or next-level storage controller. The choice of host interface 120 type may depend on the type of memory being used. Examples of host interfaces 120 include, but are not limited to, SATA, SATA Express, Serially Attached Small Computer System Interface (SAS), Fibre Channel, Universal Serial Bus (USB), PCIe, and NVMe. The host interface 120 typically facilitates the transfer of data, control signals, and timing signals.

[0036] The back-end module 110 includes an error correction code (ECC) engine 124 that encodes data bytes received from the host and decodes and corrects errors in data bytes read from the non-volatile memory. A command sequencer 126 generates command sequences, such as program and erase command sequences, sent to the non-volatile memory die 104. A redundant array of independent drives (RAID) module 128 manages the generation of RAID parity and the recovery of failed data. RAID parity can be used as an additional level of integrity protection for data being written to the memory device 104. In some cases, the RAID module 128 may be part of the ECC engine 124. A memory interface 130 provides command sequences to and receives status information from the non-volatile memory die 104. In one embodiment, the memory interface 130 may be a double data rate (DDR) interface, such as a toggle mode 200, 400, or 800 interface. The controller 102 in this example also includes a media management layer 137 and a flash control layer 132 that controls the overall operation of the backend module 110 .

[0037] Data storage device 100 also includes other separate components 140, such as an external electrical interface, external RAM, resistors, capacitors, or other components that may interface with controller 102. In alternative embodiments, one or more of physical layer interface 122, RAID module 128, media management layer 138, and buffer management / bus controller are optional components not required in controller 102.

[0038] FIG. 2B is a block diagram illustrating the components of the non-volatile memory die 104 in more detail. The non-volatile memory die 104 includes peripheral circuitry 141 and non-volatile memory array 142. The non-volatile memory array 142 includes non-volatile memory cells used to store data. The non-volatile memory cells may be any suitable non-volatile memory cells, including ReRAM, MRAM, PCM, NAND flash memory cells, and / or NOR flash memory cells in two-dimensional and / or three-dimensional configurations. The non-volatile memory die 104 also includes a data cache 156 for caching data and address decoders 148 and 150. The peripheral circuitry 141, in this example, includes a state machine 152 that provides state information to the controller 102. The peripheral circuitry 141 may also include one or more components, individually or in combination, configured to perform certain functions, including, but not limited to, the functions described and illustrated in the flowcharts herein. 2B, memory die 104 may include one or more processors 168, individually or in combination, configured to execute computer-readable program code stored in one or more non-transitory memories 169, in memory array 142, or stored external to memory die 104. As another example, one or more components may include circuits such as, but not limited to, logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.

[0039] In addition to or instead of one or more processors 138 (or, more generally, components) in the controller 102 and one or more processors 168 (or, more generally, components) in the memory die 104, the data storage device 100 may include another set of one or more processors (or, more generally, components). Generally, one or more processors (or, more generally, components) in the data storage device 100, regardless of where they are located and how many are present, individually or in combination, may be configured to perform various functions, including, but not limited to, the functions described herein and illustrated in the flowcharts. For example, one or more processors (or components) may be located in the controller 102, the memory device 104, and / or elsewhere in the data storage device 100. Also, different processors (or components) or combinations of processors (or components) may be used to perform different functions. Furthermore, a means for performing a function may be implemented using a controller that includes one or more components (e.g., a processor or other component as described above).

[0040] Referring again to FIG. 2A , flash control layer 132 (referred to herein as flash translation layer (FTL)) handles flash errors and interfaces with the host. In particular, the FTL, which may be an algorithm in firmware, is responsible for the internals of memory management, translating writes from the host into writes into memory 104. The FTL may be needed because memory 104 may have limited endurance, may only be written in multiples of pages, and / or may not be written to unless erased as a block. The FTL understands these potential limitations of memory 104, which may not be visible to the host. Thus, the FTL attempts to translate writes from the host into writes into memory 104.

[0041] The FTL may include a logical-to-physical address (L2P) map (sometimes referred to herein as a table or data structure) and allocated cache memory. In this manner, the FTL translates logical block addresses (LBAs) from the host into physical addresses within memory 104. The FTL may include other features, including, but not limited to, power-off recovery (so that the FTL's data structures can be recovered in the event of a sudden power loss) and wear leveling (so that wear across memory blocks is uniform to prevent excessive wear in any particular block that may result in a greater chance of failure).

[0042] Referring again to the drawings, FIG. 3 is a block diagram of a host 300 and a data storage device 100 in one embodiment. The host 300 can take any suitable form, including, but not limited to, a computer, a mobile phone, a tablet, a wearable device, a digital video recorder, a surveillance system, etc. The host 300 (here, a computing device) in this embodiment comprises one or more processors 330 and one or more memories 340. In one embodiment, computer-readable program code stored in the one or more memories 340 configures the one or more processors 330 to perform the operations described herein as being performed by the host 300. Accordingly, actions performed by the host 300 may be referred to herein as being performed by an application (computer-readable program code) executing on the host 300. For example, the host 300 may be configured to send data (e.g., initially stored in the host's memory 340) to the data storage device 100 for storage in the data storage device's memory 104.

[0043] As mentioned above, virtual reality (VR) and augmented reality (AR) applications are becoming more common, and the number of devices using such applications is expected to increase. Virtual reality devices typically take the form of goggles with built-in screens and lenses that display a digitally created world, and the user's entire field of view is of the digitally created world, not the "real world" outside the goggles. In contrast, augmented reality devices can present a display overlaid on the user's view of the "real world." Augmented reality devices can take the form of goggles, but can also take the form of an additional device, such as a computing device (e.g., a mobile phone or tablet) with a camera and a display screen that displays real objects detected by the camera and an overlay on the display generated by the computing device.

[0044] The data used in the virtual reality or augmented reality display can be private data retrieved from a data storage device according to the user's identity and / or other information. As used herein, "private data" refers to data that is relatively more restricted than non-private data. Accordingly, the terms restricted data and unrestricted data are sometimes used herein. Private data can have multiple degrees of privacy. For example, consider a hospital environment where a user is a member of the medical staff (e.g., a doctor, nurse, security guard, etc.) and wears augmented goggles that overlay a display of patient-related data when the user is in a patient's room, near the patient, etc., while viewing the patient. The data displayed may depend on the patient's identity and / or the user's identity. For example, if the user is a doctor, the doctor can see certain statistics about the patient's medical history, notes from other doctors, and the patient's current condition. If the user is a nurse, the nurse can see when the patient last received treatment, the next scheduled examination time, and any known allergies, but perhaps not the level of private data that a doctor can see. If the user is a security officer, the security officer may be able to see if the patient has any associated security risks, such as a history of violence or potential external threats, but perhaps not the level of private data that doctors and nurses can see. Additionally, if the patient is a security officer, the data shown to hospital staff may be limited.

[0045] Data storage devices used with virtual reality and augmented reality applications are typically general-purpose devices and are not customized for the needs of a particular segment. Therefore, these data storage devices may not provide the necessary level of privacy. For example, a host using an AR / VR device may include several external active applications, and any given application may not want its private information displayed as augmented / virtual content (e.g., text and / or graphical information overlays associated with visual data) shared / exposed to different applications at the host in a native digital format that may be relatively easy to view and copy. While some conventional data storage devices can limit access to data according to user restrictions, such restrictions may not provide adequate privacy protection in certain situations.

[0046] The following embodiments can be used to address this issue by providing a direct interface between a data storage device that stores private data and a secondary rendering engine that presents the private data in an augmented / virtual environment. Through this direct interface, prioritized and / or protected data can be passed according to the current user's restrictions and / or request priorities. In this way, the native private data in digital form is not exposed to applications running on the host, thus enhancing user privacy and protecting sensitive information against potential malware. Furthermore, these embodiments can improve the use of data storage devices with augmented reality and virtual reality devices, enabling a competitive edge in a growing market.

[0047] In one embodiment, a secondary rendering engine is introduced into an augmented reality and virtual reality device to display data to the user with a direct link to the data storage device. This data can be presented in an overlay mode (as in an augmented reality device) or as part of the virtual reality application. The data can be associated with the user's identity, and data prioritization can depend on the user, application, location, and / or other factors (e.g., environmental factors). This data can be transferred to the secondary rendering engine, bypassing the host, so that the native digital data is not exposed to the host and other applications running on it.

[0048] Referring again to the drawings, Figure 4 illustrates an exemplary architecture of one embodiment. As shown in Figure 4, in this embodiment, data storage device 100 communicates with host 300 having a primary rendering engine 320 (implemented by one or more processors within host 300) that renders output (e.g., audio, video, text, haptics, etc.) on a user's augmented reality (AR) / virtual reality (VR) display 410. In this example, data storage device 100 includes storage controller 102 (having a secondary rendering control engine 430 and a readout module 440), memory die 104, and secondary rendering engine 420, which may include a graphics processing unit (GPU), a frame buffer, and a small machine learning / computation engine that performs tasks associated with private data (without exposing it to host 300). Secondary rendering engine control 430 and / or readout module 440 may be provided by one or more processors of the controller, individually or in combination, executing instruction code stored in memory of data storage device 100. Alternatively, secondary rendering engine control 430 may be provided purely in hardware. While secondary rendering engine 420 is part of data storage device 100 in this example, it should be understood that secondary rendering engine 420 may be located in a different location, such as within host 300, within user AR / VR display device 410, or elsewhere. Also, while data storage device 100, host 300, and user AR / VR display device 410 are shown as separate devices in FIG. 4, it should be understood that one or more of these devices may be integrated with each other or with other devices. For example, data storage device 100 may take the form of a memory card that is inserted into a slot in user AR / VR display device 410 or host 300.

[0049] In this embodiment, the primary rendering engine 320 in the host 300 retrieves non-private data (e.g., from the data storage device 100 or elsewhere) and renders that data via the user's AR / VR display 410. For example, in a hospital context, the non-private data might be a patient's first-level identification (e.g., name, age, reason the patient is in the hospital, etc.). In addition to this non-private data, the AR / VR application in the host 300 may wish to overlay private data (e.g., the patient's criminal background, more detailed medical data, family information, etc.) stored in the memory 104 of the data storage device 100 (e.g., in a different partition from the non-private data). However, in this embodiment, it is desirable to avoid providing a raw / native uncompressed digital version of the private data to the application on the host 300. Thus, in this embodiment, the secondary rendering engine control 430 and the secondary rendering engine 420 are used to retrieve the private data stored in the memory 104 and render it on the user's AR / VR display device 410. This provides an isolated secondary channel for the private data, so that the uncompressed digital version of the private data is not exposed to applications running on host 300. While the rendered output may be accessible, that output, being in an analog format (e.g., video display data), is more difficult to capture and use by other applications within host 300 compared to native uncompressed digital data.

[0050] FIG. 5 is a method flowchart 500 illustrating an exemplary use of the secondary rendering engine 420. It should be understood that these are merely examples and other implementations may be used. As shown in FIG. 5, the host 300 initiates the secondary rendering engine control 430 to operate with a selected user and environment configuration when an associated application is started on the host 300 (operation 510). For example, the host 300 may provide user identification, authentication, and / or authorization information to the secondary rendering engine control 430. In this manner, the secondary rendering engine 430 provides access to private data in a manner tightly coupled with the user's authorization level. Using that information, the secondary rendering engine control 430 can determine whether the user is authorized to access private data of subjects in the current venue. If the user is not authorized, access to the private data will be prevented.

[0051] However, if the user is authorized, the secondary rendering engine control 430 detects information that can be used to identify private data to retrieve from memory (e.g., private data related to the current situation / scene) (operation 520). For example, the detection can be receiving information from the host 300 (e.g., an index number, a patient's name, age, and a patient ID number), which directly or indirectly enables the secondary rendering engine control 430 to retrieve the relevant private data from memory 104 (e.g., using the read module 440). As another example, the secondary rendering engine control 430 can detect the context of the current scene from the user AR / VR display device 410 and / or the camera / microphone of the host 300 (e.g., identifying the patient using facial or voice recognition). Thus, the secondary rendering engine control 430 can detect information related to the current situation based on the user, the environment, what appears on-screen from the primary rendering engine 320, etc.

[0052] Regardless of how the subject / context is identified, the secondary rendering engine control 430 can allow the private data to be accessed from memory 104 (e.g., using a read module) and provide the private data to the secondary rendering engine 420. In this example, the secondary rendering engine 420 is the authorization / authentication layer and the data retrieval element that is translating the information (e.g., index) provided by the host 300 into the specific data that needs to be retrieved / generated accordingly. In one embodiment, the secondary rendering engine control 430 does not control the operation of the secondary rendering engine 420, although such control is possible in other embodiments.

[0053] Secondary rendering engine control 430 then uses secondary rendering engine 420 to render the associated data on user AR / VR display device 410 (act 530). This can be done in any suitable manner. For example, secondary rendering engine 420 may include a separate video buffer that stores the data to be overlaid with the primary video buffer of primary rendering engine 320. Host 300 may provide secondary rendering engine 420 (e.g., directly or via data storage device 100) with information regarding the particular slot within the primary video buffer of primary rendering engine 320 where the overlaid private data should be stored. Thus, in addition to being the gatekeeper to the private data, secondary rendering engine control 430 can provide secondary rendering engine 420 with the format of the data and how to display or play the data. However, in other embodiments, secondary rendering engine control 430 simply acts as a gatekeeper to the private data stored in memory 104 of data storage device 100. In other examples, the private data is interleaved with or added to the non-private data in binary units and provided to a specific slot in the video buffer of the primary rendering engine 320 or written to a specific buffer based on host synchronization.

[0054] As another example, the primary rendering engine 320 can provide its video output to the secondary rendering engine 420, which can then modify the video output with the overlaid private data. In this example, the native / raw, uncompressed digital version of the private data is not exposed to the host 300 and applications running thereon. The secondary rendering engine 420 can also output the rendered private data in a common format (e.g., key-value (KV) pair or logical block address (LBA) format). The host 300 can communicate the target format to the secondary rendering engine 420 and / or secondary rendering engine control 420, or the format can be pre-configured (e.g., at the factory, during an initial user setup phase, etc.). The rendered private data can take any suitable form, such as, but not limited to, images, text, video, audio, bitmaps, haptics, etc.

[0055] As described above, in one embodiment, the controller 102 (e.g., using the read module 440) reads private data from the memory 104 after permission by the secondary rendering engine control 430. Due to the real-time nature of AR / VR environments, it may be desirable to treat the read of private data as an urgent read to help ensure that the private data is rendered in a timely manner via the user's AR / VR display device 410 (e.g., to meet real-time latency requirements of, e.g., 5 milliseconds, while minimizing impact on the primary workload). This can be done, for example, by bypassing the normal read host interface and using a different type of interface, such as, but not limited to, an NVMe weighted round-robin urgent queue. Note that this type of urgent read command may be used in other contexts, such as, but not limited to, games.

[0056] Special handling of this type of read command may include hardware and / or software interrupts to facilitate the fetching of private data. Additionally, there may be a special memory space allocation for the host 300 to collect urgent data, separate from the main queue where the main workload data is fetched. The memory may be allocated either in the storage controller's volatile memory (e.g., SRAM or DRAM), in the host memory buffer (HMB) (e.g., host DRAM) of the host 300, or in the non-volatile memory 100 of the data storage device 100.

[0057]

[0023] Figure 6 illustrates the architecture of one embodiment. As shown in Figure 6, in this embodiment, the data storage device's controller 102 communicates with the host 300, which has a host memory buffer (HMB) 600. For example, the Non-Volatile Memory Express (NVMe) specification stipulates that the host memory buffer 600 is allocated for exclusive use by the data storage device's controller 102, data in the host memory buffer 600 is not actively modified or accessed by the host 300 (i.e., the data is guaranteed to be valid), and the host 300 is obligated to notify the data storage device's controller 102 before any operation that could result in data loss (e.g., in the event of a power loss or if the host 300 may need the buffer) (in such cases, the host 300 has the controller 102 acknowledge the operation before data is lost). An HMB descriptor list on the host 300 can maintain a listing of entries associated with host data buffers for exclusive use by the controller 102. During initialization, the host software may provide the HMB descriptor list to the data storage device 100 for exclusive use by the controller 102 .

[0058] The controller 102 in this example includes an emergency read control module 620 and a read module 440. The emergency read control module 620 and / or the read module 440, individually or in combination, may be provided by one or more processors of the controller executing instruction code stored in memory of the data storage device 100. Alternatively, one or both may be provided purely in hardware. The architecture may provide an emergency read memory allocation 610. The emergency read memory may be allocated either in the host HMB 600 or in volatile or non-volatile memory in the data storage device 100. The location may also depend on the urgency and intensity of the primary workload, as determined by the emergency read control 620.

[0059] FIG. 7 is a flowchart 700 of one embodiment method for executing an urgent read command. As shown in FIG. 7, the urgent read control module 620 detects an urgent read from within the main workload (operation 710). Next, the urgent read control module 620 allocates memory to contain the urgent read data (operation 720). Next, the urgent read control module 620 either suspends the current main workload or issues a read-ahead (operation 730). Finally, the urgent read is performed from the non-volatile memory die 104, and the data is fetched into the allocated memory (operation 740). Thus, in this example, upon detecting the urgent read, the urgent read control 620 allocates a memory buffer outside of the main workflow queue buffer to contain the data. Next, it suspends the main workflow or issues a read-ahead command synchronized with the current main workflow. Once fetched, the urgent read data is placed in the allocated memory buffer for access by the host.

[0060] Finally, as noted above, any suitable type of memory may be used. Semiconductor memory devices include volatile memory devices such as dynamic random access memory ("DRAM"), static random access memory ("SRAM") devices, non-volatile memory devices such as resistive random access memory ("ReRAM"), electrically erasable programmable read only memory ("EEPROM"), flash memory (which may be considered a subset of EEPROM), ferroelectric random access memory ("FRAM"), magnetoresistive random access memory ("MRAM"), and other semiconductor elements capable of storing information. Each type of memory device may have a different configuration. For example, flash memory devices may be configured in a NAND or NOR configuration.

[0061] Memory devices may be formed from passive and / or active elements in any combination. By way of non-limiting example, passive semiconductor memory elements include ReRAM device elements, which in some embodiments include a resistive switching storage element such as an anti-fuse, a phase change material, and optionally a steering element such as a diode. By way of further non-limiting example, active semiconductor memory elements include EEPROM and flash memory device elements, which in some embodiments include elements that include charge storage regions such as floating gates, conductive nanoparticles, or charge storage dielectric materials.

[0062] The memory elements may be configured such that the memory elements are connected in series or such that each element is individually accessible. As a non-limiting example, flash memory devices in a NAND configuration (NAND memory) typically include memory elements connected in series. A NAND memory array may be configured such that the array is made up of multiple strings of memory, each consisting of multiple memory elements that share a single bit line and are accessed as a group. Alternatively, the memory elements may be configured such that each element is individually accessible, e.g., a NOR memory array. NAND and NOR memory configurations are examples, and memory elements may be configured alternatively.

[0063] The semiconductor memory elements located within and / or above the substrate may be arranged in two or three dimensions, such as a two-dimensional memory structure, a three-dimensional memory structure, or the like.

[0064] In a two-dimensional memory structure, semiconductor memory elements are arranged in a single plane or a single memory device level. Typically, in a two-dimensional memory structure, the memory elements are arranged in a plane (e.g., an xz-direction plane) that extends substantially parallel to a major surface of a substrate that supports the memory elements. The substrate may be a wafer on or within which layers of memory elements are formed, or may be a carrier substrate to which the memory elements are attached after they are formed. As a non-limiting example, the substrate may include a semiconductor such as silicon.

[0065] The memory elements may be arranged in an ordered array, such as multiple rows and / or columns, in a single memory device level. However, the memory elements may be arranged in a non-regular or non-orthogonal configuration. The memory elements may each have two or more electrodes or contact lines, such as bit lines and word lines.

[0066] A three-dimensional memory array is one in which memory elements are arranged to occupy multiple planes or multiple memory device levels, thereby forming a three-dimensional (i.e., x, y, and z directions, where the y direction is substantially perpendicular to the major surface of the substrate and the x and z directions are substantially parallel to the major surface of the substrate) structure.

[0067] As a non-limiting example, a three-dimensional memory structure may be arranged vertically as a stack of multiple two-dimensional memory device levels. As another non-limiting example, a three-dimensional memory array may be arranged as multiple vertical columns (e.g., columns extending substantially perpendicular to the major surface of the substrate, i.e., in the y-direction), with each column having multiple memory elements in each column. The columns may be arranged in a two-dimensional configuration, e.g., in the xz plane, resulting in a three-dimensional arrangement of memory elements having elements on multiple vertically stacked memory planes. Other configurations of three-dimensional memory elements may also make up a three-dimensional memory array.

[0068] As a non-limiting example, in a three-dimensional NAND memory array, memory elements may be coupled together to form NAND strings within a single horizontal (e.g., xz) memory device level. Alternatively, memory elements may be coupled together to form vertical NAND strings that traverse multiple horizontal memory device levels. Other three-dimensional configurations are possible, in which some NAND strings contain memory elements at a single memory level and other strings contain memory elements across multiple memory levels. Three-dimensional memory arrays may also be designed in NOR and ReRAM configurations.

[0069] Typically, in a monolithic three-dimensional memory array, one or more memory device levels are formed above a single substrate. Optionally, the monolithic three-dimensional memory array may also have one or more memory layers at least partially within the single substrate. As a non-limiting example, the substrate may include a semiconductor such as silicon. In a monolithic three-dimensional array, the layers making up each memory device level of the array are typically formed on layers of the memory device level below the array. However, layers of adjacent memory device levels of a monolithic three-dimensional memory array may be shared or may have intervening layers between the memory device levels.

[0070] Similarly, two-dimensional arrays may then be formed separately and then packaged together to form a non-monolithic memory device having multiple memory layers. For example, a non-monolithic stacked memory may be constructed by forming memory levels on separate substrates and then stacking the memory levels on top of each other. The substrate may be thinned or removed from the memory device levels before stacking, but the resulting memory array is not a monolithic three-dimensional memory array because the memory device levels are first formed across separate substrates. Furthermore, multiple two-dimensional or three-dimensional memory arrays (monolithic or non-monolithic) may be formed on separate chips and then packaged together to form a stacked-chip memory device.

[0071] Associated circuitry is typically required for operation of and communication with the memory elements. As a non-limiting example, a memory device may have circuitry used to control and drive the memory elements to accomplish functions such as programming and reading. This associated circuitry may be on the same substrate as the memory elements and / or on a separate substrate. For example, a controller for memory read and write operations may be located on a separate controller chip and / or on the same substrate as the memory elements.

[0072] Those skilled in the art will appreciate that the present invention is not limited to the two-dimensional and three-dimensional structures described, but rather encompasses all relevant memory structures within the spirit and scope of the present invention as described herein and as understood by those skilled in the art.

[0073] The above detailed description is intended to be understood as an illustration of selected forms that the invention may take, and not as a definition of the invention. It is only the following claims, including all equivalents, that are intended to define the scope of the claimed invention. Finally, it should be noted that any aspect of any of the embodiments described herein may be used alone or in combination with each other.

Claims

1. 1. A data storage device comprising: Memory and and one or more processors, wherein the one or more processors individually or in combination: transmitting the unrestricted data stored in the memory to a host, the host including a primary rendering engine configured to render the unrestricted data on a display device, in response to receiving information from the host identifying the unrestricted data; receiving information about a user from the host; determining whether the user is authorized to access restricted data stored in the memory based on the information about the user; A data storage device configured to, in response to determining that the user is authorized to access the restricted data, send the restricted data to a secondary rendering engine configured to render the restricted data as an overlay to the rendered unrestricted data on the display device, wherein by sending the restricted data to the secondary rendering engine instead of the primary rendering engine within the host, the restricted data is not exposed to the host.

2. The data storage device of claim 1 , wherein the display device comprises a virtual reality display device.

3. The data storage device of claim 1 , wherein the display device comprises an augmented reality display device.

4. The data storage device of claim 1 , wherein the secondary rendering engine is located within the data storage device.

5. The data storage device of claim 1 , wherein the secondary rendering engine is located external to the data storage device.

6. The data storage device of claim 1 , wherein the non-restricted data and the restricted data are stored in different partitions within the memory.

7. The data storage device of claim 1 , wherein the secondary rendering engine comprises a graphics processing unit (GPU) and a frame buffer.

8. 10. The data storage device of claim 1, wherein the one or more processors, individually or in combination, are further configured to identify the non-restricted data to read from the memory directly or indirectly from information received from the host.

9. The data storage device of claim 1 , wherein the one or more processors, individually or in combination, are further configured to read the restricted data from the memory using an emergency read command.

10. The data storage device of claim 1 , wherein the one or more processors, individually or in combination, are further configured to store the restricted data in a host memory buffer.

11. 10. The data storage system of claim 1, wherein the memory comprises a three-dimensional memory.

12. 1. A method comprising:

1. A data storage device having a memory and in communication with a host including a primary rendering engine, comprising: receiving a request for non-private data and private data stored in the memory; transmitting the non-private data to the primary rendering engine in the host for rendering the non-private data on a display device; In response to a user being authorized to access the private data, sending the private data to a secondary rendering engine external to the host for rendering the private data on the display device, wherein by sending the private data to the secondary rendering engine instead of the primary rendering engine within the host, the private data is inaccessible by the host.

13. 13. The method of claim 12, wherein the secondary rendering engine is further configured to receive an identification of a particular slot within a video buffer of the primary rendering engine where the rendered private data is to be stored.

14. The method of claim 12 , wherein the secondary rendering engine is further configured to receive an identification of a rendering format.

15. The method of claim 12 , wherein the secondary rendering engine is further configured to interleave the rendered private data with the rendered non-private data.

16. The method of claim 12 , wherein the secondary rendering engine is further configured to binary-wise add the rendered private data with the rendered non-private data.

17. 13. The method of claim 12, wherein the secondary rendering engine is further configured to receive the rendered non-private data from the primary rendering engine, modify the rendered non-private data with the rendered private data, and provide the results of the modification to the display device.

18. 10. The data storage device of claim 1, wherein the secondary rendering engine is further configured to output the rendered private data as a key-value (KV) pair or in a logical block address (LBA) format.

19. The method of claim 12 , wherein the display device comprises a virtual reality display device or an augmented reality display device.

20. 1. A data storage device comprising: Memory and and means for providing private data stored in the memory to a secondary rendering engine external to a host including a primary rendering engine within an augmented reality / virtual reality environment in response to a user being authenticated to access the private data.

Images