Processors, methods, systems, and instructions to support live migration of protected containers

Abstract

Processors, methods, systems, and instructions are disclosed for supporting live migration of protected containers. The processor includes a decode unit to decode an instruction to indicate a page of the protected container memory and a storage location outside of the protected container memory. The execution unit is responsive to the instruction to ensure that there is no writable reference to a page of the protected container memory while the page has a write-protected state. The execution unit is to encrypt a copy of a page of the protected container memory. The execution unit is to store the encrypted copy of the page to a storage location outside of the protected container memory after it has been ensured that there is no writable reference. The execution unit is to leave a page of the protected container memory in a write-protected state after the encrypted copy has been stored to the storage location, the page being still valid and readable.

Description

[0001] This application is a divisional application filed again for the divisional application 202010877325.3. The divisional application 202010877325.3 is a divisional application of the invention patent application with PCT international application number PCT / US2016 / 034519, international application date May 27, 2016, application number 201680030029.1 entering the Chinese national phase, and entitled "Processor, method, system and instructions for supporting live migration of protected containers". Background Art Technical Field

[0002] Embodiments described herein generally relate to processors. Specifically, embodiments described herein generally relate to processors having an architecture that supports enclaves and other protected containers. Technical Background

[0004] Desktop computers, laptop computers, smart phones, servers, routers and other network elements, as well as various other types of computer systems, are often used to process secret or confidential information. Several representative examples of such secret or confidential information include, but are not limited to: passwords, account information, financial information, information during financial transactions, confidential company data, enterprise rights management information, personal calendars, personal contacts, medical information, other personal information, etc. In general, it is desirable to protect such secret or confidential information from inspection, tampering, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present invention may be best understood by referring to the following description and accompanying drawings which illustrate embodiments thereof. In the drawings:

[0006] Figure 1 is a block diagram of a computing environment in which a protected container can be migrated from a source computer system to a destination computer system.

[0007] Figure 2 is a block diagram of a first example embodiment of a software environment in which live migration of a protected container of a virtual machine may be performed.

[0008] Figure 3 is a block diagram of a second example embodiment of a software environment in which migration of a protected container of an operating system container may be performed.

[0009] Figure 4 is a block flow diagram of an embodiment of a method of migrating a protected container from a source computer system to a destination computer system.

[0010] Figure 5is a block flow diagram of an embodiment of a method of write-protecting pages of a protected container memory and storing encrypted copies of the write-protected pages outside the protected container memory while keeping the write-protected pages valid and readable in the protected container memory.

[0011] Figure 6 is a block diagram of an embodiment of a processor operable to execute an embodiment having a set of one or more instructions for supporting live migration of protected containers.

[0012] Figure 7 is a block diagram of an embodiment of a processor operable to execute an embodiment of a protected container page write protect instruction.

[0013] Figure 8 is a block diagram of an embodiment of a processor operable to execute an embodiment of encrypt protected container pages and store encrypted copies outside protected container memory instructions.

[0014] Fig. 9 is a block diagram of an embodiment of a processor operable to execute embodiments of protected container page unwrite protection and encrypted page copy invalidation instructions.

[0015] Fig. 10A is a block diagram illustrating an embodiment of an in-order pipeline and an embodiment of a register renaming out-of-order issue / execution pipeline.

[0016] Fig. 10B is a block diagram of an embodiment of a processor core including a front end unit coupled to an execution engine unit, and both the front end unit and the execution engine unit are coupled to a memory unit.

[0017] Fig.11A is a block diagram of an embodiment of a single processor core along with its connection to the on-die interconnect network and along with its local subset of the level 2 (L2) cache.

[0018] Fig. 11B yes Fig.11A A block diagram of an embodiment of an expanded view of a portion of a processor core.

[0019] Fig.12 is a block diagram of an embodiment of a processor that may have more than one core, may have an integrated memory controller, and may have integrated graphics.

[0020] Fig.13 is a block diagram of a first embodiment of a computer architecture.

[0021] Fig.14 is a block diagram of a second embodiment of a computer architecture.

[0022] Fig.15is a block diagram of a third embodiment of a computer architecture.

[0023] Fig.16 is a block diagram of a fourth embodiment of a computer architecture.

[0024] Fig.17 is a block diagram of using a software instruction converter to convert binary instructions in a source instruction set into binary instructions in a target instruction set according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] Disclosed herein are instructions for performing operations useful during the real-time migration of a protected container, a processor for executing the instructions, a method performed by the processor when processing or executing the instructions, and a system incorporating one or more processors for processing or executing the instructions. Although the instructions are described primarily in conjunction with the real-time migration of protected containers, it is to be understood that the instructions are not limited to such uses but have general utility and can be optionally used for other uses that are completely unrelated to the real-time migration of protected containers. In the following description, many specific details are set forth (e.g., specific instruction operations, data structures and their contents, processor configurations, microarchitecture details, sequences of operations, etc.). However, embodiments may be practiced without these specific details. In other instances, well-known circuits, structures, and techniques are not shown in detail to avoid obscuring the understanding of this specification.

[0026] Figure 1 1 is a block diagram of a computing environment in which a protected container 107 may be migrated from a source computer system 100 to a destination computer system 121. The source computer system and the destination computer system may represent desktop computer systems, laptop computer systems, smart phones, servers, network elements, or other types of computer systems. As shown, the source computer system and the destination computer system may be coupled to each other or otherwise communicate with each other via one or more intermediate networks 120. In one aspect, the source computer system and the destination computer system may be coupled via the Internet (e.g., the "cloud"). Alternatively, the source computer system and the destination computer system may be more directly coupled via one or more local wired or wireless links.

[0027] The source computer system includes at least one processor 112, conventional memory 102, and protected container memory 106. Conventional memory and protected container may represent different portions of system memory that may include one or more types of physical memory (e.g., dynamic random access memory (DRAM), flash memory, etc.). Conventional memory and protected container memory may have different levels of protection or security that are partially performed by the logic of the processor. Conventional memory may represent a portion of system memory of a type that is typically used to store applications, data, etc. As shown, conventional memory may store privileged system software 103 (e.g., a virtual machine monitor, one or more operating systems, etc.). Conventional memory may also store one or more user-level applications (e.g., a network management application, a database application, an email application, a spreadsheet application, etc.). In one aspect, the source computer system may represent a so-called "open" system that does not typically significantly restrict user selection of system software and user-level applications that may be loaded onto the system.

[0028] The protected container memory 106 may have a higher level of protection and / or security than the conventional memory 102. The higher level of protection and / or security may be performed, controlled, or otherwise provided at least in part by the hardware and / or other on-die logic of the processor. In some embodiments, the protected container memory may represent a portion of the processor reserved memory reserved specifically for the processor, while the conventional memory may not be part of the processor reserved memory. By way of example, the processor may have one or more range registers corresponding to the protected container memory. The range registers may be used to store data associated with the range of the protected container memory, and may be queried when an attempt is made to access the protected container memory as part of providing protection and / or security. On the one hand, the data or range may be stored in the range registers by the basic input / output system (BIOS) during boot.

[0029] A protected container 107 (e.g., one or more pages 108 of a protected container) may be stored in a protected container memory 106. The processor may have an instruction set 113, which includes instructions for interacting with a protected container (e.g., creating, destroying, entering, exiting, managing paging, performing security operations, etc.). Some of the instructions may be privileged instructions that can be executed by privileged software (e.g., software 103) rather than by non-privileged or user-level software. Other instructions may be non-privileged or user-level instructions. As an example, a protected container may be created for a protected container utilizing a domain 105 (such as, for example, a virtual machine module, an application module, etc.). For example, privileged system software may create a protected container. A protected container may generally be opaque to privileged software (e.g., privileged software may not be able to see the code and / or data within the protected container), but may be managed by privileged software (e.g., through privileged instructions of an instruction set).

[0030] Protected container utilization domains may store secret or confidential data in protected containers. Protected containers may help provide confidentiality to secret or confidential information in protected containers and, in some cases, optionally provide one or more other protections (e.g., integrity protection, replay protection, etc.). Confidentiality generally involves preventing data disclosure. Integrity generally involves ensuring that there is no data tampering. At least some hardware logic of the processor may help provide such confidentiality and / or other protections. In various embodiments, a protected container may represent a secure enclave, a container executed by hardware, a hardware-managed execution environment, a hardware-managed isolated execution area, a secure and / or dedicated memory area to be used by an application, or other protected container. In some embodiments, a protected container may represent Software Guard Extensions ( In some embodiments, the protected container memory may represent a page operable to store one or more pages of a running or executing secure enclave. An SGX enclave page cache (EPC), although the scope of the invention is not limited in this respect.

[0031] In some embodiments, a protected container page metadata structure (PCPMS) 109 may be used to store metadata (e.g., security metadata, access control metadata, etc.) for the protected container 107. As shown, in some embodiments, the PCPMS may optionally be stored in the protected container memory, although this is not required. In some embodiments, the PCPMS may store such metadata for each page stored in the protected container memory. In one aspect, the PCPMS may be structured to have different entries for different corresponding pages in the protected container memory, although other ways of structuring the PCPMS are possible (e.g., other types of tables, data structures, etc.). Each entry may store metadata for a corresponding page. Depending on the particular implementation, examples of suitable types of metadata for protected container pages include, but are not limited to, information indicating whether a page is valid or invalid, information indicating a protected container to which a protected container page belongs, information indicating a virtual address (through which the protected container page is allowed to be accessed), information indicating read / write / execute permissions for the protected container page, and the like, and various combinations thereof. Alternatively, optionally, in different embodiments, less metadata, additional metadata, or other metadata combinations may be stored. The scope of the present invention is not limited to any known type of metadata to be stored in PCPMS. For some embodiments, an example suitable for PCPMS is SGX Enclave Page Cache Map (EPCM), although the scope of the invention is not limited in this respect.

[0032] like Figure 1 As shown in , optionally, the protected container (e.g., its pages) can be stored in the protected container memory. Likewise, optionally, the PCPMS can be stored in the protected container memory. Additionally or alternatively, optionally, the protected container (e.g., its pages) and / or the PCPMS can be stored in the protected container storage device on the die of the processor. For example, the processor can have one or more caches 115 for storing protected memory pages and / or PCPMS. By way of example, one or more dedicated caches can be used, dedicated portions of one or more caches can be used, or a combination thereof. As another option, in addition to the cache, the processor can have another type of dedicated storage device for storing such pages or structures. Combinations of both off-die memory and (multiple) on-die caches or other storage devices are also suitable.

[0033] Refer again Figure 1, the processor may also have protected container logic 116. The protected container logic may include hardware, firmware, or a combination thereof for executing instructions and / or otherwise supporting protected containers (e.g., controlling access to protected containers). The protected container logic includes access control logic 117. The access control logic is operable to implement, control, or otherwise provide access control to protected container memory and data of protected container memory when it resides on the die of the processor (e.g., residing in (multiple) caches, registers, other structures, etc.). Different types of access control logic 117 are suitable in different embodiments, depending on the specific implementation.

[0034] In some embodiments, depending on the particular implementation, the access control logic may include a memory management unit (MMU) and / or a page miss handler (PMH) unit operable to control access to protected containers and / or protected container memory in part by querying page tables, range registers, PCPMS 109, etc., or a combination thereof. In some embodiments, the access control logic may include logic operable to control access to code and / or data of protected containers while the code and / or data reside within the processor. For example, the logic may be operable to control access to the code and / or data while the code and / or data is stored in an unencrypted form or otherwise resides within a cache, register, or other structure or component within the processor during runtime for computation. In one aspect, the logic may be operable to allow authorized access from within the code of a protected container to the code and / or data of the same protected container (whether stored in protected container memory or resident on the die of the processor), but may prevent unauthorized access to the code and / or data of the protected container (whether stored in protected container memory or resident on the die of the processor) by code outside the protected container.

[0035] The protected container logic may also include a cryptographic unit 118. The cryptographic unit may be operable to perform encryption and decryption. In some embodiments, the cryptographic unit may automatically encrypt the code and / or data of the protected container before the code and / or data is stored outside the processor (e.g., in system memory) (such as, for example, during a write to system memory, during an eviction of a cache line holding the protected container code and / or data, etc.). This may help prevent the code and / or data from being viewed (e.g., to help provide data confidentiality). The cryptographic unit may also be operable to decrypt the encrypted code and / or data of the enclave page as it is received into the processor (e.g., from system memory).

[0036] In some embodiments, optionally, the cryptographic unit is further operable to cryptographically provide integrity protection and / or authentication to the code and / or data of the protected container. For example, in some embodiments, the cryptographic unit may automatically calculate a message authentication code or other authentication or integrity check data for the code and / or data of the protected container before the code and / or data is stored outside the processor (e.g., stored in the system memory). Optionally, the cryptographic unit is further operable to use such authentication or integrity check data to authenticate or ensure the integrity of the code and / or data when the code and / or data of the protected container page is received into the processor (e.g., from the system memory). This may help allow authentication or integrity checking of the data to help detect any tampering or alteration of the data. The logic may be operable to detect integrity violations of the protected container page and prevent access to the tampered code / data (when detected).

[0037] In one aspect, such cryptographic operations may be performed automatically and autonomously by the cryptographic unit and transparently to the software (e.g., as opposed to the software having to execute multiple instructions of the software cryptographic algorithm). In some embodiments, the cryptographic unit may optionally perform such cryptographic operations selectively on code and / or data of protected containers and generally not on code and / or data of regular pages that do not belong to the protected containers.

[0038] In some embodiments, the protected container logic 116 may optionally include logic for generating and using version information associated with the code and / or data of the protected container. For example, optionally, when a page of a protected container is stored outside of the processor (e.g., stored in system memory), the page may be assigned version information (e.g., a unique version value, a version counter, etc.). Optionally, the protected container logic may include logic for checking such version information when the code and / or data (e.g., pages) of the protected container are reloaded. In some embodiments, the protected container logic may only allow loading of protected container pages that are indicated as legal or valid by version information (e.g., only the last evicted version). This may help prevent replay of protected container code and / or data.

[0039] To further illustrate certain concepts, certain types of protection or security have been described above. However, it is to be understood that the type or level of protection or security may vary from implementation to implementation, depending on the specific implementation, environment, security needs, cost versus security tradeoffs, etc. The embodiments described herein may be used in conjunction with protected containers having different levels of security or protection. The scope of the present invention is not limited to any known type or level of protection or security.

[0040] Refer again Figure 1In some embodiments, the privileged system software 103 of the source computer system 110 may include a protected container live migration module 104, which is operable to control or manage the live migration of the protected container 107 from the source computer system to the destination computer system 121. The term "real-time" means that the migration can be partially performed while the protected container is running on the source computer system. As further shown, in some embodiments, the processor includes one or more instructions 114 for supporting the live migration of the protected container (e.g., the protected container 107). As mentioned above, these instructions are not limited to being used only for the live migration of the protected container, but can also be optionally used for various other purposes according to the creative talent of the programmer. In some embodiments, the one or more instructions can be privileged level instructions that can only be executed at one or more privileged execution levels, but not at a user execution level or a non-privileged execution level. The one or more non-privileged execution levels are higher than the user execution privilege level used by user-level applications (e.g., word processing applications, email applications, spreadsheets, etc.). In some embodiments, the protected container live migration module 104 can use the one or more instructions 114. Furthermore, in some embodiments, the processor may include logic 119 operable to execute the one or more instructions 114 for supporting live migration of protected containers. Advantageously, the instructions (multiple) and logic for executing the instructions may help improve the efficiency of executing live migration of protected containers, although the instructions may alternatively be used for other purposes.

[0041] In some embodiments, the source computer system may include a key manager protected container 110 operable to manage one or more capable or migratable keys (e.g., a key hierarchy) that may correspond to the protected container 107. As shown, the key manager protected container may also optionally be stored in a protected container memory. The key manager protected container may represent a trusted entity that controls or manages keys and allows the keys to be virtualized and migrated from the source computer system to the destination computer system in conjunction with migrating the protected container from the source computer system to the destination computer system. In some embodiments, the key manager protected container may represent an architecture protection container. In an SGX implementation example, a suitable example of a key manager protected container is a migration engine (MigE), although the scope of the invention is not limited thereto. In other embodiments, optionally, the keys that can be migrated can be implemented in different ways, such as, for example, stored in a special type of page in a protected container, controlled or managed by a different type of trusted entity, etc.

[0042] Depending on the particular implementation, one or more other structures 111 may also optionally be used in conjunction with the protected container. For example, in some embodiments, there may be a structure for holding the one or more migratable or portable keys (e.g., a key hierarchy). For example, in In an SGX implementation example, there may be an SGX domain control structure (SDCS) for storing migratable platform SGX keys, counters, and domain states. As another example, in some embodiments, one or more version pages may be optionally included for storing version information of protected container pages. For example, in In an SGX implementation embodiment, there may be one or more version array pages operable to store a version array of pages in protected container memory. For example, as will be further explained below, there may be embodiments of VA pages for pages stored from the EPC and invalidated in the EPC and VAX pages for pages stored from the EPC and retained in the EPC in a write-protected, valid, and readable state according to embodiments. As yet another example, In an SGX implementation example, there may be a paged password metadata (PCMD) structure operable to store cryptographic metadata associated with a paged-out page, and a page metadata structure (PGMD) operable to store metadata related to a page. It will be understood that, optionally, in different implementations, data and / or metadata associated with a protected container may be partitioned or combined in different ways, and the scope of the present invention is not limited to any known way of partitioning or combining data and / or metadata.

[0043] After the migration is complete, the destination computer system may have a migrated protected container 122. A simplified version of the destination computer system is shown, although it is to be understood that the destination computer system may alternatively be similar or identical to the source computer system. In general, it may be desirable to migrate protected containers in real time for a variety of different reasons, and the scope of the present invention is not limited to any known reasons. On the one hand, protected containers may be migrated in conjunction with load balancing. For example, a virtual machine or other protected container utilization domain running on a source server in a data center or a cloud computing environment may use a protected container, and the domain and the protected container may be migrated from the source server to the destination server to balance the workload on the source server and the destination server. In other embodiments, the protected container may be migrated for other reasons such as, for example, to relocate a workload from a source computer system to be repaired, maintained, or upgraded, to relocate a workload from a running desktop computer to a portable computer, and so on.

[0044] Figure 2205-N。 The computer system includes a virtual machine monitor (VMM) 203 and a first VM 205-1 to an Nth VM 205-N. The VMM may represent a host program (usually in firmware) operable to provide virtualization management or control to allow the computer system to support the VM. Representatively, the VMM may manage one or more processors, memory, and other resources of the computer system and allocate resources associated therewith to the VM. Each VM may represent a virtual or software implementation of a machine (e.g., a physical machine) that simulates the execution of a program. Each VM may support the execution of a guest operating system (OS). As shown, the first VM may have a first guest operating system (OS) 224-1, and the Nth VM may have an Nth guest OS 224-N. The operating system may be either multiple instances of the same OS or different OSes.

[0045] In the example shown, the Nth VM will utilize protected container 207. In some embodiments, the VMM may include a protected container implementation migration module 204 for controlling or managing the migration of protected containers out of the computer system. As described elsewhere herein, one or more processors (not shown) of the computer system may have one or more instructions for supporting live migration and logic for executing the instructions. In some embodiments, the first VM may include a key manager protected container 210. In some embodiments, the key manager protected container may control or manage a set of one or more per-VM or VM-specific keys (e.g., a key hierarchy), each of which corresponds to a different VM in the VM having a corresponding protected container.

[0046] Figure 33 is a block diagram of a second example embodiment of a software environment in which a protected container 307 of an operating system container 305 may be migrated. A computer system includes an operating system (OS) 303, an OS container 305 having a protected container 307, and an OS control service 325. The OS 303 may represent a kernel and may provide container-based virtualization or OS virtualization. The OS container may represent an application within the OS that represents a virtualization layer similar to a guest virtual machine. In some embodiments, the OS may include a protected container implementation migration module 304 for controlling or managing the migration of a protected container out of the computer system. As previously described, as described elsewhere herein, one or more processors (not shown) of the computer system may have one or more instructions for supporting live migration of a protected container and logic for executing the instructions. Similarly, in some embodiments, the OS control service may include a key manager protected container 310. In some embodiments, the key manager protected container may control or manage a set of one or more per-OS container or OS container specific keys (e.g., a key hierarchy), each of which corresponds to a different OS container having a corresponding protected container in the OS container.

[0047] Figure 4 4 is a block flow diagram of an embodiment of a method 430 of migrating a protected container from a source computer system to a destination computer system. In some embodiments, the method may be controlled or managed by cooperating protected container live migration modules on the source computer system and the destination computer system.

[0048] At block 431, while an application or domain (e.g., VM, OS container, etc.) using the protected container is running on the source computer system, a copy of the pages of the protected container may be stored from the protected container memory of the source computer system to an encrypted copy in the conventional memory of the destination computer system. In some embodiments, the operation at block 431 may optionally be performed using Figure 5 or a portion thereof, including any of the variations mentioned for the method, although the scope of the invention is not limited thereto.

[0049] In some embodiments, each page may be write-protected in the protected container memory of the source computer system before a copy (e.g., an encrypted copy) of the page is stored from the protected container memory of the source computer system to an external location (e.g., stored in the conventional memory of the source computer system). In some embodiments, before a copy (e.g., an encrypted copy) of each write-protected page is stored from the protected container memory of the source computer system to the external location, a corresponding processor of the source computer system may ensure or verify that there are no writable references to the write-protected page. In some embodiments, after a copy of each write-protected page has been stored from the protected container memory of the source computer system to the external location, the write-protected page may remain write-protected but valid and readable in the protected container memory of the source computer system. That is, in some embodiments, a copy of the page may exist simultaneously in the protected container memory of the source computer system (e.g., EPC) and also as an encrypted copy outside the protected container memory (e.g., in the conventional memory of the source computer system or the destination computer system). Advantageously, this may allow a running application or domain of the source computer system to read data from the page while the protected container live migration module is simultaneously working to migrate the page to the destination computer.

[0050] In some embodiments, in order to reduce the amount of downtime required to achieve a full migration of a protected container, most to substantially as many pages as possible can be copied from the protected container memory to the conventional memory of the destination storage location while the application or domain (e.g., VM or OS container) is running on the source system. These pages can be at least conceptually considered as a write working set and a non-write working set. Pages in the write working set tend to be written during the migration window or time span in which the application or domain is running. In contrast, pages in the non-write working set tend not to be written or are unlikely to be written during the migration window or time span in which the application or domain is running. Typically, most to substantially all of the pages in the non-write working set and (possibly) some of the pages in the write working set (e.g., pages that have not been written after the pages have been copied from the protected container memory) may be copied from the protected container memory to an external location while the application or domain is running. Furthermore, the write-protected pages copied from the protected container memory may still be read even after they have been copied from the protected container memory, because the pages are write-protected but are still valid (unless a subsequent write to the page has been detected) and readable. This may allow an application or domain to read from these pages while migration of these pages is in progress.

[0051] On the one hand, assuming that all protected container pages in the protected container page are not in the write working set, the protected container live migration module can iterate the protected container page one or more times to copy the protected container page from the protected container memory. If an attempted write to a write-protected page is detected, the speculative copy of the page stored outside the protected container memory can be invalidated. Otherwise, the page can be kept in the protected container in a write-protected but valid and readable state. After the first iteration, the protected container live migration module may iterate the remaining uncopied protected container pages one or more times until the number of such remaining uncopied pages becomes sufficiently small (e.g., reduced to below a threshold number or ratio) or according to some other desired criteria. Typically, after several iterations, the group of remaining uncopied protected container pages should be roughly aggregated into the write working set of pages that tend to be written during the migration window or time frame. Advantageously, write-protecting the protected container pages and allowing the protected container pages to be valid and readable in the protected container memory can help reduce the application or domain downtime required to implement the live migration of the protected container. Instead of invalidating the page, the page can remain readable in the protected container memory. Effectively, almost all pages outside the written working set can be copied from the protected container memory, rather than just those pages outside the larger set of pages representing the working set (e.g., the larger set of pages additionally includes those pages that the application tends to read during the migration window). This can help reduce the number of pages that need to be copied after the application is deactivated or stopped, which can help reduce downtime.

[0052] Refer again Figure 4 , at block 432, execution of the application that is using the protected container on the source computer system may be stopped. In various embodiments, the application may optionally be a VM or OS container, although this is not required.

[0053] At box 433, after the application or domain that is using the protected container has stopped running, copies of any remaining unreplicated pages (and optionally, any remaining unreplicated special pages) can be copied from the protected container memory of the source computer system to encrypted copies in the regular memory of the destination computer system. In some embodiments, one or more special pages can optionally be used, although the scope of the invention is not limited in this regard. As an example, one or more special pages can optionally be used to store keys that can be migrated. As another example, one or more special pages can optionally be used for version information of pages of the protected container. For example, in In an SGX implementation embodiment, SDCS may be used to store keys that can be migrated, and one or more version array pages (VAX) may be used to store version information for pages that are written out of protected container memory but remain in a write-protected, valid, and readable state. In other embodiments, a single special page may be used to store both keys that can be migrated and version information, or keys that can be migrated and / or version information may be stored instead in the protected container page itself, or alternatively, such information may be stored in other data structures, etc.

[0054] At block 434, an application or domain that will use the protected container on the destination computer system may begin running. In various embodiments, the application may optionally be a VM or OS container, although this is not required.

[0055] In block 435, the encrypted pages (and optionally, the encrypted special pages) may be loaded from the regular memory of the destination computer system into unencrypted pages in the protected container memory of the destination computer system. For example, a protected container may be created and initialized in the protected container memory of the destination computer system, and then the pages may be loaded into the protected container. In some embodiments, the special pages may optionally be loaded into the protected container memory before the regular pages are loaded into the protected container memory, although the scope of the invention is not limited in this regard.

[0056] Figure 5 5 is a block flow diagram of an embodiment of a method 538 for write-protecting pages of protected container memory and storing encrypted copies of the write-protected pages outside of the protected container memory while keeping the write-protected pages valid and readable in the protected container memory. In some embodiments, the method may be controlled or managed by a protected container live migration module of a source computer system while migrating a protected container to a destination computer system, although the scope of the invention is not limited in this regard. For example, in some embodiments, the method may optionally be performed at a Figure 4 The method may be performed at block 431 of the embodiment of the present invention, although the scope of the present invention is not limited in this regard. Figure 1 Alternatively, the features and components described for computer system 100 may be used in the method, but such features and components are not required. In some embodiments, one or more instructions of the instruction set of the processor (e.g., Figure 1 Directive 114 and / or Figure 6 Instructions 614) are used to perform the method, and the one or more instructions are operable to support live migration of protected containers.

[0057] At box 539, the pages of the protected container memory may be write protected. The pages of the protected container memory may be stored in either secure storage within the protected container memory in system memory or may be cached or otherwise stored in a processor's cache or other on-die storage location in secure storage operable to keep the pages secure therein. In some embodiments, each page may be write protected in response to the execution or implementation of a single instruction of the instruction set. In some embodiments, each page may be write protected by configuring a write protection indication in a protected container page metadata structure (e.g., PCPMS109) to indicate that the page is write protected. For example, in In an example of an SGX implementation, enclave pages may be write protected by configuring (e.g., setting) a write protect (WP) bit in an enclave page cache map (EPCM). In some embodiments, while a page is write protected, the page may be valid and readable. In some embodiments, each write protected page may also be made read-only in a paging structure (e.g., an extended paging table), although the scope of the invention is not limited in this regard. In some embodiments, modifications to the paging structure may optionally be made outside the bounds of an instruction for modifying a write protection indication in a PCPMS. In various aspects, from an overall algorithmic perspective, pages may be write protected one at a time or in batches, or all pages in a protected container may be write protected at once.

[0058] At block 540, the processor may ensure or verify that there are no writable references to the write-protected page. In different embodiments, this may be done in different ways. In some embodiments, this may be implemented using a TLB tracking mechanism. The TLB may cache translations from virtual addresses associated with the page to physical addresses. Permissions associated with obtaining those permissions (e.g., read and write permissions) may also be cached in the TLB. These permissions cached in the TLB reflect the permissions at the time the translation was performed when the page table walk was performed. Based on the memory access request, if the MMU finds a translation in the TLB, it may bypass the page table lookup and use the translation and permissions cached in the TLB. That is, the MMU may use permissions from the TLB (which may be outdated) instead of looking up permissions in the page table and / or checking permissions in the PCPMS (e.g., EPCM). In the case where the page has permissions in the TLB indicating that the page is writable, the thread may write to the page even after the page has been write-protected (e.g., as described at block 539). To enhance security, the method may ensure that such cached permissions are flushed from the TLB (multiple) TLBs. This can be done before making a copy of the write-protected page. In different embodiments, this can be done in different ways. As an example, an epoch counter can be used to determine when a thread can have access to such a TLB mapping and when such a TLB mapping must be flushed (e.g., a thread must have observed write protection on the page). Once all threads have observed write protection on the page, it can be ensured that there are no longer writable references to the write-protected page. In some embodiments, after In an SGX implementation example, the ETRACK instruction and associated mechanisms may optionally be used to ensure that write mappings to migrated pages are removed from the TLB before the pages are written out to main memory. By way of example, the ETRACK instruction may be used to configure the microarchitectural tracker logic to detect when all logical processors executing in an enclave have exited the enclave and, therefore, when all TLB entries have been evicted when the ETRACK instruction is executed (e.g., TLB entries created during enclave execution may be evicted upon exiting the enclave).

[0059] At block 541, encrypted copies of write-protected pages of the protected container memory may be generated. In some embodiments, the encrypted copy of each write-protected page may be generated in response to execution or implementation of a single instruction of the instruction set. In some embodiments, a cryptographic unit of the processor (e.g., cryptographic unit 118) may be used, which may be used to encrypt the protected container pages as they are written out of the processor to anywhere in system memory, including to the protected container memory as well as to conventional system memory outside of the protected container memory. In some embodiments, encrypted protected container pages may optionally be stored in conventional memory using different encryption relative to the protected container memory, although this is not required.

[0060] At box 542, an encrypted copy of the write-protected page of the protected container memory (e.g., the encrypted copy generated at box 540) can be stored outside the protected container memory (e.g., to conventional memory of the same computer system or otherwise in non-protected container memory). In some embodiments, the encrypted copy can be stored in conventional memory while the corresponding write-protected page remains valid and readable in the protected container memory. In some embodiments, after the encrypted copy has been stored outside the protected container memory, an application or domain using the associated protected container can be allowed to read from the write-protected page in the protected container memory. Another possible alternative would be to invalidate the page while the application is running (e.g., rather than write-protecting the page and allowing the page to remain valid and readable), although this may have potential disadvantages, such as not allowing the page to be read and / or reducing the overall number of pages that can be stored outside the protected container memory. In some embodiments, an encrypted copy of each write-protected page can be stored outside the protected container memory (e.g., in conventional memory) in response to the execution or implementation of a single instruction of the instruction set. In some embodiments, the encrypted copy may be stored outside of the protected container memory only after ensuring or verifying that there are no writable references to the write-protected pages (eg, after block 541 ).

[0061] At block 543, a determination may be made as to whether a write attempt has been detected to a write-protected page of the protected container memory. If no such attempted write has been detected (i.e., a "no" determination at block 543), the method may proceed to block 545. This may allow the page to remain write-protected but valid and readable in the protected container memory while there is only one true non-dirty copy of the page, thereby allowing security to be maintained.

[0062] Conversely, if such a write attempt to a write-protected page has been detected (i.e., at box 543, "yes" is the determination result), the method can proceed to box 544. Representatively, such an attempt may be detected by the logic of the processor (e.g., in response to an extended page table violation), and in response to this situation, the logic may indicate an error. At box 544, the write-protected page can be unwrite-protected (e.g., the page can be made writable), and any encrypted copies outside the protected container memory (e.g., in the conventional memory of the source computer system or in the conventional memory of the destination computer system) can be invalidated. Invalidating copies outside the protected container memory can help ensure security, such as by ensuring that there is only one true copy of the page (e.g., there is no divergence between the contents of the encrypted copy and the contents of the page in the protected container memory). In some embodiments, in response to the execution or implementation of a single instruction of the instruction set, the page can be unwrite-protected and the encrypted copy outside the protected container memory can be invalidated. In some embodiments, each page may be write-protected by configuring a write protection indication in a protected container page metadata structure (e.g., PCPMS 109) to indicate that the page is not write-protected. In an example of an SGX technology implementation, each page may be write-protected by configuring (e.g., flushing) a write-protect (WP) bit in an enclave page cache map (EPCM). In some embodiments, the page may also be made readable and writable in a paging structure (e.g., an extended paging table), although the scope of the invention is not limited in this regard. In some embodiments, modifications to the paging structure may be made outside the bounds of an instruction for modifying a write-protect indication in a PCPMS.

[0063] The method may proceed to box 545 from box 543 or box 544. At box 545, a determination may be made as to whether to repeat the method. If the method is to be repeated (i.e., at box 545, "yes" is the determination result), the method may return to box 539. Alternatively, the method may end. In different embodiments, the determination of whether to repeat the method may be performed in different ways. For example, the method may be performed a predetermined or configurable number of times (e.g., the determination may involve a loop counter with a threshold value). As another example, the determination may involve determining whether additional pages are still write-protected (e.g., at box 539) and stored in the immediately preceding iteration, whether additional pages greater than the number of pages whose write protection was revoked (e.g., at box 544) were write-protected (e.g., at box 539) and stored in the immediately preceding iteration, and so on. Alternatively, optionally, the method may be performed only once, and optionally, box 545 may be omitted.

[0064] It will be understood that this is only an illustrative example of a method. Other methods may include a subset of the displayed frames. For example, an alternative method may include only frame 539. Another alternative method may include frames 540, 541, and 542. Yet another frame may include only frame 544. Various other combinations of frames are also contemplated. Moreover, additional operations may be added to the method, optionally. In addition, some operations or frames may overlap (e.g., frames 540 and 541 may overlap) or may be performed in different orders (e.g., frame 541 may be performed before frame 540, frame 543 may be performed continuously within the entire method, etc.).

[0065] Figure 6 612 is a block diagram of an embodiment of a processor 612 operable to execute an embodiment of a set of one or more instructions 614 for supporting live migration of protected containers. As previously mentioned, instructions may also be used for other purposes besides supporting live migration of protected containers. In some embodiments, the processor may be a general-purpose processor (e.g., a general-purpose microprocessor or central processing unit (CPU) of the type used in a desktop computer, laptop computer, or other computer). Alternatively, the processor may be a special-purpose processor. Examples of suitable special-purpose processors include, but are not limited to: cryptographic processors, network processors, communication processors, coprocessors, embedded processors, digital signal processors (DSPs), and controllers (e.g., microcontrollers). The processor may have any of the following architectures: various complex instruction set computing (CISC) architectures, reduced instruction set computing (RISC) architectures, very long instruction word (VLIW) architectures, hybrid architectures, other types of architectures, or a combination of different architectures (e.g., different cores may have different architectures).

[0066] The set of one or more instructions 614 for supporting live migration of protected containers may be instructions of an instruction set of a processor. Instructions may represent macroinstructions, machine code instructions, or assembly language instructions. In the illustrated example embodiment, the instructions may include four different instructions, although the scope of the present invention is not limited thereto. Specifically, the instructions include a protected container page write protection instruction 650, a protected container page encryption and storage of encrypted copies outside of protected container memory instruction 651, an optional protected container page write protection release and page copy invalidation instruction 652, and an optional protected container version array page creation instruction 653.

[0067] In other embodiments, less than or more than four instructions may optionally be used. For example, in alternative embodiments, any single instruction of these instructions may optionally be included, and other instructions may optionally be omitted. As an example, optionally, only the protected container page write protection instruction may be included. As another example, optionally, only the protected container page encryption and store encrypted copy outside the protected container memory instruction may be included. As yet another example, optionally, only the protected container page write protection instruction and the protected container page encryption and store encrypted copy outside the protected container memory instruction may be included. Moreover, optionally, one or more other instructions may be added, along with one or more of the four instructions shown.

[0068] In still other embodiments, the functionality of these four instructions may be assigned in different ways. As a specific example, optionally, the operation of the protected container write protection instruction 650 and the operation of the protected container page encryption and storage of the encrypted copy outside the protected container memory instruction 651 may be combined into a single instruction. As another specific example, optionally, the encryption operation of the protected container page encryption and storage of the encrypted copy outside the protected container memory instruction 651 may instead be assigned to and executed by the protected container write protection instruction 614. As yet another specific example, optionally, the operation of the protected container version array page creation instruction 653 and either the protected container page write protection instruction 650 or the protected container page encryption and storage of the encrypted copy outside the protected container memory instruction 651 may be combined into a single instruction. These are just a few illustrative examples. Other variations will be apparent to those skilled in the art who benefit from this disclosure.

[0069] Refer again Figure 6, the processor includes a decoding unit or decoder 654. The decoding unit can receive any of the one or more instructions 614 and decode it. The decoding unit can output one or more relatively low-level instructions or control signals (e.g., one or more microinstructions, microoperations, microcode entry points, decoded instructions or control signals, etc.) that reflect, represent and / or are derived from the received relatively high-level instructions. In some embodiments, the decoding unit may include one or more input structures (e.g., (multiple) ports, (multiple) interconnections, interfaces) for receiving instructions, instruction recognition and decoding logic coupled thereto to recognize and decode the received instructions, and one or more output structures (e.g., (multiple) ports, (multiple) interconnections, interfaces) coupled thereto to output (multiple) lower-level instructions or (multiple) control signals. The decoding unit can be implemented using a variety of different mechanisms, including but not limited to: microcode read-only memory (ROM), lookup table, hardware implementation, programmable logic array (PLA), and other mechanisms suitable for implementing the decoding unit.

[0070] The processor also includes a set of registers 667 (e.g., general purpose registers). Each of the registers may represent an on-die storage location operable to store data. A register may represent an architecturally visible or architectural register visible to software and / or a programmer, and / or a register indicated by an instruction of the processor's instruction set for identifying operands. These architectural registers contrast with other non-architectural registers (e.g., temporary registers, reorder buffers, retirement registers, etc.) in a given microarchitecture. Registers may be implemented in different microarchitectures and are not limited to any particular type of design. Examples of suitable types of registers include, but are not limited to, dedicated physical registers, dynamically allocated physical registers using register renaming, and combinations thereof. In some embodiments, registers may be used to store input and / or output data associated with an instruction. On the one hand, registers may include registers for Figures 7 to 9 Any of the general purpose registers shown and described for any of the figures and may have any of the described inputs and / or outputs mentioned for any of the figures.

[0071] Execution unit 655 is coupled to decode unit 654 and register 667. The execution unit may receive the one or more decoded or otherwise converted instructions or control signals representing and / or derived from any of the decoded instructions (e.g., any of instructions 614). The execution unit is operable to perform one or more operations 668 in response to the decoded instruction and / or as a result of the decoded instruction (e.g., in response to one or more instructions or control signals decoded from the instruction) to implement the operation of the instruction. As shown, the execution unit may be coupled to or otherwise communicate with other logic of processor 616 and / or pages or structures in memory 606 to implement the operation of the specific instruction being executed. In some embodiments, the execution unit may be for Figures 7 to 9 Any of the execution units shown and described for any of the figures and may perform any of the operations described for any of the figures.

[0072] An execution unit and / or processor may include specific or particular logic (e.g., transistors, integrated circuit systems, or other hardware that may be combined with firmware (e.g., instructions stored in non-volatile memory) and / or software) operable to perform such operations in response to instructions and / or as a result of instructions (e.g., in response to one or more instructions or control signals decoded from the instructions). In some embodiments, an execution unit may include one or more input structures (e.g., port(s), interconnect(s), interfaces) for receiving source data, circuitry or logic coupled to the one or more input structures for receiving and processing source data, and one or more output structures (e.g., port(s), interconnect(s), interfaces) coupled to the circuitry or logic for implementing operations.

[0073] To avoid obscuring the description, a relatively simple processor has been shown and described. However, the processor may optionally include other processor components. For example, various different embodiments may include a processor for FIG. 10A to FIG. 12 Various different combinations and configurations of components shown and described for any of the processors. All of the components of the processor may be coupled together to allow them to function as intended.

[0074] Figure 7 is a block diagram of an embodiment of a processor 712 operable to execute an embodiment of a protected container page write protection instruction. The processor includes a decode unit 754, an execution unit 755, and (optionally) a set of registers 767 (e.g., general purpose registers). Unless otherwise specified or otherwise clearly apparent, these components may optionally have Figure 6To avoid obscuring the description, the different and / or additional features will be described primarily without repeating all (optionally) common features. In addition, in some embodiments, the processor 712 may be configured to: Figure 1 The processor 712 may be used in a similar or different computer system.

[0075] The decode unit 754 may receive a protected container page write protection instruction 750. In some embodiments, the protected container page write protection instruction may be a privileged instruction that is executable only at a privileged execution level and not at a non-privileged or user execution level. For example, the protected container page write protection instruction may be executable only at a ring 0 privilege level, such as by an operating system, a virtual machine monitor (VMM), or other privileged system software and not by user-level application software. In some embodiments, the instruction may indicate additional instruction-specifying information 774, although this is not required. For example, in In an SGX implementation example, the instruction may be a privileged ENCLS instruction that may be executed at ring 0 privilege level, and the general register EAX may be implicitly indicated as having a leaf function index value of "28h" (which is used to indicate an EPC page write protection operation), although the scope of the invention is not limited in this regard. Although the instruction may be used in a variety of different ways unrelated to the migration of protected containers, in one aspect, such privileged software may use the protected container page write protection instruction to write-protect protected container pages in conjunction with live migration of the protected container before storing an encrypted copy of the page outside of the protected container memory.

[0076] In some embodiments, a protected container page write protection instruction may indicate a protected container page 708. In different embodiments, the page may be indicated in different ways. In some embodiments, the instruction may explicitly specify (e.g., via one or more fields or bit sets) or otherwise indicate (e.g., implicitly indicate) a register (e.g., one of the general registers 767) for storing the effective address or other indication 775 of the protected container page. As an example, optionally, the instruction may have a register designation field for specifying a register that will have an effective address for indicating the protected container page. As another example, optionally, the instruction may implicitly or impliedly indicate an implicit register that will have an effective address for indicating the protected container page. When the instruction is received, it may be understood that the implicit register is also used to find the effective address, even though it is not explicitly stated. As a specific example, in In an SGX implementation example, the implicit general register RCX may store the effective address of an enclave page cache (EPC) page.

[0077] The processor is operable to combine the effective address with other address information to obtain the address of the protection container page. For example, data segment (DS) segmentation can be used to create a linear or virtual address. As shown, in some embodiments, the protected container memory 706 can optionally be in a system memory 770 coupled to the processor. Alternatively, the protected container memory can optionally be one or more caches or other on-die storage devices of the processor. As a specific example, in In an SGX implementation example, the protected container memory may be an enclave page cache (EPC).

[0078] The execution unit 755 is coupled to the decode unit 754 and the optional general register 767. The execution unit is operable to write-protect the indicated protected container page 708 of the protected container memory 706 (which may be in either system memory or on-die cache or other on-die storage device) in response to the protected container page write protection instruction. In some embodiments, the execution unit may have a write protection unit that is operable to write-protect the indicated page by configuring a write protection indicator 777 (which corresponds to the indicated page) so that the indicated page is write-protected. Different types of write protection indicators are suitable for different embodiments. In some embodiments, the write protection indicator may be implemented as one or more bits in the protected container memory, in an access control protected data structure in a processor reserved memory, in an access control protected register or other structure of the processor, etc. As shown, in some embodiments, the write protection indicator may optionally be included in a protected container page security metadata structure (PSPMS) 709, which is used to store metadata (e.g., security and access control metadata) of the page in the protected container memory. In some embodiments, the PSPMS may have a different write protection indicator for each respective different page in the protected container memory. According to one possible convention, the write protection indicator may be a single bit that may be configured to have a first binary value (e.g., set to binary one) to indicate that the respective indicated protected container page 708 is not write protected or a second different binary value (e.g., flushed to binary zero) to indicate that the respective indicated protected container page is write protected. As a specific example, in In an SGX implementation example, the execution unit may write-protect the page by setting a write-protect (WP) bit in an enclave page cache map (EPCM) to indicate that the page of the enclave page cache (EPC) and the non-management fields in the EPCM are write-protected, although the scope of the invention is not limited in this regard. Representatively, when the WP bit of the EPCM is set, if a write access is attempted to the page, a page miss handler (PMH) unit and / or a translation lookaside buffer (TLB) may indicate an error (e.g., a write protection error, a page fault, etc.).

[0079] In some embodiments, the instruction may optionally explicitly specify or otherwise indicate a metadata structure 711 to be used to store metadata 778 for the indicated protected container page 708 of the protected container memory. In different embodiments, the metadata structure may be indicated in different ways. In some embodiments, the instruction 750 may explicitly specify (e.g., via one or more fields or bit sets) or otherwise indicate (e.g., implicitly indicate) a register (e.g., one of the general registers 767) to be used to store the effective address or other indication 776 of the metadata structure. As an example, optionally, the instruction may have a register designation field for specifying a register that will have an effective address for indicating the metadata structure. As another example, optionally, the instruction may implicitly or impliedly indicate an implicit register that will have an effective address for indicating the metadata structure. As a specific example, in In an SGX implementation example, an implicit general register RBX may store an effective address of a page metadata (PGMD) structure, although the scope of the invention is not limited in this regard. In some embodiments, other data structures (e.g., PCPMS) may optionally be used to store metadata. The execution unit is operable, in response to an instruction, to store metadata 778 related to the indicated protected container page in a metadata structure 711. As shown, the execution unit may have an optional metadata storage unit 782 for storing metadata in a metadata structure. Alternatively, in other embodiments, such storage of metadata may optionally be omitted (e.g., may not be required, may be performed by another instruction, etc.).

[0080] In some embodiments, the execution unit is optionally operable to perform one or more security or verification checks before write-protecting the protected container page. In some embodiments, the execution unit may include a security check unit 783 for checking or verifying that the migratable key structure 779 has control over the indicated protected container page 708, the migratable key structure having the migratable key 780. For example, in In an SGX implementation example, the execution unit may be operable to determine that a current SGX domain control structure (SDCS) has control over the protected container page, the current SDCS may have SGX keys, counters, and cryptographic metadata that can be migrated, although the scope of the invention is not limited in this regard. Alternatively, in some embodiments, such security or verification checks may be omitted (e.g., may not be required, may be performed by another instruction, etc.).

[0081] Figure 8 8 is a block diagram of an embodiment of a processor 812 operable to execute an embodiment of instructions 851 for encrypting protected container pages and storing encrypted copies outside protected container memory. The processor includes a decode unit 854, an execution unit 855, and (optionally) a set of registers 867 (e.g., general purpose registers). Unless otherwise specified or otherwise clearly apparent, these components may optionally have Figure 6 To avoid obscuring the description, the different and / or additional features will be described primarily without repeating all (optionally) common features. In addition, in some embodiments, the processor 812 may be configured to: Figure 1 The processor 812 may be used in a similar or different computer system.

[0082] Decode unit 854 may receive instruction 851. In some embodiments, the instruction may be a privileged instruction that can only be executed at a privileged execution level and not at a non-privileged or user execution level. In some embodiments, the instruction may indicate additional instruction specific information 874, although this is not required. For example, in In an SGX implementation example, the instruction may be a privileged ENCLS instruction, and the general register EAX may be implicitly indicated as having a leaf function index value of "2Ah" (which is used to indicate operations of storing encrypted pages from the EPC and making pages readable in the EPC), although the scope of the invention is not limited in this regard. Although the instruction may be used in a general manner, in one aspect, such privileged software may use the instruction in conjunction with live migration of a protected container while the protected container is running to store an encrypted copy of a write-protected page (e.g., a page that is write-protected by a previous write-protect instruction as disclosed herein) outside of the protected container memory (e.g., to regular memory).

[0083] In some embodiments, the instruction may indicate a write-protected page 808 of the protected container memory 806. In different embodiments, the write-protected page may be indicated in different ways. In some embodiments, the instruction may explicitly specify (e.g., via one or more fields or bit sets) or otherwise indicate (e.g., implicitly indicate) a register (e.g., one of the general registers 867) for storing the effective address or other indication 875 of the write-protected page 808. As a specific example, in In an SGX implementation example, an implicit general register RCX can store the effective address of a write-protected EPC page to be stored outside the EPC (e.g., stored in conventional memory). As shown, in some embodiments, the protected container memory 806 can optionally be in a system memory 870 coupled to the processor (e.g., in a hardware reserved portion of the system memory). Alternatively, the protected container memory 806 can optionally be one or more caches or other on-die storage devices of the processor. Combinations are also possible. As a specific example, in In an SGX implementation example, the protected container memory may be an enclave page cache (EPC).

[0084] The execution unit 855 is coupled to the decoding unit and the optional general register 867. The execution unit is responsive to the protected container page write protection instruction and is operable to ensure that there are no execution writable references to the write-protected pages of the protected container memory while the pages of the protected container memory have a write-protected state by hardware. As shown, the execution unit may include a writable reference tracker unit 890 that may be coupled to the TLB tracking logic 891. The writable reference tracker logic is operable to communicate with the TLB tracking logic to ensure that there are no writable references to the write-protected pages of the protected container memory. Optionally, this may be performed as described elsewhere herein or by other means. In some embodiments, optionally, an ETRACK instruction and associated mechanisms may be used to ensure that the write mapping for the migrated page is deleted from the TLB before the page is written out to the main memory. By way of example, the ETRACK instruction may be used to configure the microarchitectural tracker logic to detect when all logical processors executing in an enclave at the time the ETRACK instruction is executed have exited the enclave and therefore when all TLB entries have been evicted (e.g., TLB entries created during execution of the enclave may be evicted upon exiting the enclave).

[0085] The execution unit is further operable to encrypt a copy of the indicated write-protected page 808 of the protected container memory in response to the protected container page write protection instruction. As shown, the execution unit may include an encryption unit 818 (e.g., which may be a Figure 1part of the cryptographic unit 118).

[0086] The execution unit is further operable to: store an encrypted copy 887 of the write-protected page of the protected container memory to a destination location 886 outside of the protected container memory 806 after ensuring that there are no writable references to the page of the protected container memory executed by the hardware. In some embodiments, the destination location may be in conventional memory such as, for example, memory used to store user-level applications (e.g., an Internet browser, a database application, a word processing application, etc.). In some embodiments, the write-protected page in the protected container memory may be in processor reserved memory, and the encryption key may be stored outside of the processor reserved memory. In some embodiments, the instruction may explicitly specify or otherwise indicate an indication 885 of the destination storage location (such as, for example, by having a designated or implicit register for storing such an indication). The execution unit is further operable to leave the write-protected page 808 in the protected container memory 806 in a write-protected state (which is still valid and readable (e.g., as opposed to being invalidated)) after the encrypted copy 887 of the write-protected page has been stored to the indicated destination location 886 outside the protected container memory (e.g., in conventional memory that is non-processor reserved memory). Allowing the write-protected page to remain valid and readable may provide advantages as described elsewhere herein, such as, for example, allowing the page to be read, reducing downtime after a live migration, etc.

[0087] In some embodiments, the execution unit is optionally operable to store version information 889 for write-protected pages 808 stored outside of the protected container memory. For example, the execution unit may include a page version storage unit 892 for storing version information 889. In some embodiments, the instruction may indicate a version page 888 for storing version information for pages in the protected container memory. Alternatively, in lieu of a version page, another structure may be used to store version information. In some embodiments, the instruction may explicitly specify or otherwise indicate a register (e.g., one of the general registers 867) for storing an effective address or other indication 876 of a version page 888 or version structure. As a specific example, in In an SGX implementation example, an implicit general register RDX may store the effective address of a VAX page slot, although the scope of the invention is not limited in this regard. In one aspect, a VAX page may represent a dedicated type of version array page for storing version array information for write-protected pages stored outside the EPC and is distinct from a VA page for storing version array information for invalid pages stored outside the EPC. In other embodiments, version information may be stored in other types of pages in other structures (e.g., protected structures in memory, protected structures on a die, etc.). Version information may help prevent replay of encrypted pages. Alternatively, for a given implementation, it may not be intended or desirable to provide such protection against replay, and such version information may optionally be omitted.

[0088] In some embodiments, optionally, the execution unit is operable to perform one or more security or verification checks before storing the encrypted copy 887 in a location outside of the protected container memory. As shown, the execution unit may include a security check unit 883. In some embodiments, the security check unit is operable to check or verify that the key structure 879 capable of migration has control over the write-protected container pages 808 to be stored outside of the protected container memory, the key structure capable of migration having the key 880 capable of migration. In some embodiments, optionally, this may also include checking or verifying that the key structure capable of migration has control over the version pages 888 or other version storage structures to be used to store version information. For example, in In an SGX implementation example, this may include determining that the current SGX domain control structure (SDCS) has control over write-protected pages and VAX pages, which may have SGX keys, counters, and cryptographic metadata that can be migrated, although the scope of the invention is not limited in this regard. Alternatively, in some embodiments, such security or verification checks may be omitted (e.g., may not be required, may be performed by another instruction, etc.).

[0089] In some embodiments, the instruction may optionally specify or otherwise indicate a metadata structure 811 to be used to store metadata 878 for storing out of the write-protected page 808. In some embodiments, the instruction 851 may specify or otherwise indicate a register (e.g., one of the general registers 867) to be used to store the effective address or other indication 884 of the metadata structure 811. As a specific example, in In an SGX implementation example, the instruction may indicate a (PCMD) structure, although the scope of the invention is not limited in this regard. The execution unit is operable to store metadata 878 related to storing the write-protected page in the metadata structure in response to the instruction. As shown, the execution unit may include an optional metadata storage unit 882. By way of example, the metadata storage unit is operable to store metadata such as, for example, a page type, a read-write-execute permission state, a suspended state, a modified state, etc. corresponding to the indicated page, and various combinations thereof. Such metadata may be used to ensure metadata integrity when the page is reloaded (e.g., into a migrated protected container). Alternatively, in other embodiments, such storage of metadata may optionally be omitted (e.g., may not be required, may be performed by another instruction, etc.).

[0090] Fig. 9 9 is a block diagram of an embodiment of a processor 912 operable to execute an embodiment of the protected container page write protection removal and encrypted page copy invalidation instruction 952. The processor includes a decode unit 954, an execution unit 955, and (optionally) a set of registers 967 (e.g., general purpose registers). Unless otherwise specified or otherwise clearly apparent, these components may optionally have Figure 6 To avoid obscuring the description, the different and / or additional features will be described primarily without repeating all (optionally) common features. In addition, in some embodiments, the processor 912 may be configured to: Figure 1 The processor 912 may be used in a similar or different computer system.

[0091] Decode unit 954 may receive instruction 952. In some embodiments, the instruction may be a privileged instruction that can only be executed at a privileged execution level and not at a non-privileged or user execution level. In some embodiments, the instruction may indicate additional instruction specific information 974, although this is not required. For example, in In an SGX implementation example, the instruction may be a privileged ENCLS instruction, and the general register EAX may be implicitly indicated as having a leaf function index value of "29h" (which is used to indicate an EPC page write protection removal operation), although the scope of the invention is not limited thereto. Although the instruction may be used in a general manner, on the one hand, such privileged software may use the instruction in conjunction with live migration of a protected container while the protected container is running to resolve an error on a write-protected page (e.g., after an attempted write to the write-protected page).

[0092] In some embodiments, the instruction may indicate a write-protected page 908 of a protected container. In some embodiments, the instruction may explicitly specify or otherwise indicate a register (e.g., one of the general registers 967) for storing an effective address or other indication 975 of the write-protected page 908. As a specific example, in In an SGX implementation example, an implicit general register RCX may store the effective address of a write-protected EPC page. As shown, in some embodiments, the protected container memory 906 may optionally be in a system memory 970 coupled to the processor (e.g., in a hardware reserved portion of the system memory). Alternatively, the protected container memory 906 may optionally be one or more caches or other on-die storage devices of the processor. Combinations of this approach are also suitable. As a specific example, in In an SGX implementation example, the protected container memory may be an enclave page cache (EPC).

[0093] The execution unit 955 is coupled to the decoding unit and the optional general register 967. The execution unit is operative to remove write protection from the indicated page 908 of the protected container memory (which may be in either system memory or on-die cache or other on-die storage device) in response to the protected container page write protection instruction. On the one hand, this can leave the page in a valid and usable state, in which both reading and writing are allowed. In some embodiments, the execution unit may have a write protection removal unit 993, which is operative to remove write protection from the page by configuring the write protection indicator 977 (which corresponds to the indicated page 908) to indicate that the page is not write-protected. The same type of write protection indicator mentioned above for the write protection indicator is suitable. In some embodiments, the same write protection indicator can be used for two instructions. The write protection removal instruction can basically perform the indicator configuration opposite to the write protection instruction. As shown, in some embodiments, the write protection indicator may optionally be included in a protected container page security metadata structure (PSPMS) 909, which is used to store metadata (e.g., security and access control metadata) for a page in protected container memory. In an SGX implementation example, the execution unit may unwrite-protect the page by flushing the write-protect (WP) bit in the enclave page cache map (EPCM) to indicate that the enclave page cache (EPC) is a write-protected enclave page and the unmanaged field in the EPCM is not write-protected, although the scope of the invention is not limited in this regard.

[0094] In some embodiments, optionally, the execution unit is operable to invalidate a copy of a write-protected page of the protected container memory that is outside the protected container memory. In some embodiments, a version 989 corresponding to the write-protected page 908 may be used. In some embodiments, the execution unit is operable to change the version corresponding to the page in response to an instruction. As shown, the execution unit may include a version change unit 994 for changing the version of the write-protected page. As shown, optionally, the version may be stored in the version page 988. Alternatively, optionally, another protected data structure in the memory or a protected structure on the die may be used to store the version. By way of example, the execution unit may, for example, invalidate the entry by flushing the entry corresponding to the write-protected page in the migration version array page or otherwise changing the version so that the version will not match.

[0095] In some embodiments, the instruction may explicitly specify or otherwise indicate a register (e.g., one of the general registers 967) for storing the effective address or other indication 976 of the migration version array slot of the version 989 of the write-protected page to be changed or invalidated. As a specific example, in In an SGX implementation example, the implicit general register RDX may store the effective address of the migration version array slot, although the scope of the invention is not limited thereto. In other embodiments, the version information may be stored in other types of pages in other structures (e.g., protected structures in memory, protected structures on die, etc.).

[0096] Advantageously, this can help provide additional security because there can only be one true copy or version of a write-protected page. When a write-protected page becomes dirty or modified, other copies of the write-protected page can be invalidated so that the dirty or modified copy of the page is the only true copy or version of the page. This can also help prevent replay of encrypted pages. Alternatively, for a given implementation, it may not be intended or desirable to provide such protection, and optionally, such invalidation of copies can be omitted. Furthermore, in addition to using version information, other ways of invalidating copies of a page outside of a protected container are contemplated.

[0097] In some embodiments, optionally, the execution unit is operable to perform one or more security or verification checks before invalidating the copy of the page outside the protected container. As shown, the execution unit may include a security check unit 983. In some embodiments, the security check unit is operable to check or verify that the correct write-protected page is invalidated. For example, the execution unit is operable to compare the version of the page with the version of the invalidated page to ensure that the correct version information is flushed or otherwise changed. Optionally, if the page is invalid or if the version does not match the value stored in the version page, the instruction may fail. Alternatively, in some embodiments, such a security or verification check may be omitted (e.g., may not be required, may be performed by another instruction, etc.).

[0098] In some embodiments, the processor's decoder is optionally further operable to create instructions (e.g., Figure 6 In some embodiments, the protected container version page or structure creation instruction may explicitly specify or otherwise indicate a register (e.g., a general register) for storing an effective address or other indication of a page in the protected container memory. In some embodiments, the page may be an invalid page available for use with a new page. The processor may also include an execution unit that is operable, in response to the instruction, to create a version page or structure for holding version information. In some embodiments, a version page may be created in the protected container memory at an indicated page in the protected container memory (e.g., at an indicated effective address). In some embodiments, this may include zeroing the page. In some embodiments, the version page may be a special type of page dedicated to storing version information for a page, an encrypted copy of which is stored outside the protected container memory while the page remains stored in the protected container memory in a valid and readable state. In some embodiments, the created page may be an empty page, to which version information may then be stored (e.g., according to different instructions). In some embodiments, the execution unit in response to the instruction may also initialize or configure one or more entries, fields, or other portions of a protected container page metadata structure (PCPMS), such as, for example, an EPCM. By way of example, this may include indicating that the page is a versioned type page, setting the page to be valid, and configuring read-write-execute permissions for the page.

[0099] Exemplary Core Architectures, Processors, and Computer Architectures

[0100] Processor cores can be implemented in different ways, for different purposes and in different processors. For example, implementations of such cores may include: 1) a general-purpose in-order core intended for general-purpose computing; 2) a high-performance out-of-order core intended for general-purpose computing; 3) a dedicated core intended primarily for graphics and / or scientific (throughput) computing. Implementations of different processors may include: 1) a CPU including one or more general-purpose in-order cores intended for general-purpose computing and / or one or more general-purpose out-of-order cores intended for general-purpose computing; and 2) a coprocessor including one or more dedicated cores intended primarily for graphics and / or science (throughput). Such different processors lead to different computer system architectures, which may include: 1) a coprocessor on a separate chip from the CPU; 2) a coprocessor on a separate die in the same package as the CPU; 3) a coprocessor on the same die as the CPU (in this case, such a coprocessor is sometimes referred to as dedicated logic, such as integrated graphics and / or scientific (throughput) logic or dedicated cores); and 4) a system on a chip, which may include the described CPU (sometimes referred to as (multiple) application cores or (multiple) application processors), the above coprocessors, and additional functions on the same die. An exemplary core architecture is described next, followed by a description of exemplary processor and computer architectures.

[0101] Exemplary Core Architecture

[0102] In-order and out-of-order core block diagram

[0103] Fig. 10A is a block diagram illustrating both an exemplary in-order pipeline and an exemplary register renaming out-of-order issue / execution pipeline according to embodiments of the present invention. Fig. 10B is a block diagram illustrating both an exemplary embodiment of an in-order architecture core and an exemplary register renaming out-of-order issue / execution architecture core to be included in a processor in accordance with embodiments of the present invention. Fig. 10A and Fig. 10B The solid line boxes in show the in-order pipeline and in-order core, while the optional addition of dashed line boxes shows the register renaming out-of-order issue / execution pipeline and core. The out-of-order aspect will be described assuming that the in-order aspect is a subset of the out-of-order aspect.

[0104] exist Fig. 10A , the processor pipeline 1000 includes an acquisition stage 1002, a length decoding stage 1004, a decoding stage 1006, an allocation stage 1008, a renaming stage 1010, a scheduling (also known as dispatch or issue) stage 1012, a register read / memory read stage 1014, an execution stage 1016, a write back / memory write stage 1018, an exception handling stage 1022, and a commit stage 1024.

[0105] Fig. 10BA processor core 1090 is shown, the processor core comprising a front end unit 1030 coupled to an execution engine unit 1050, and both the execution engine unit and the front end unit are coupled to a memory unit 1070. The core 1090 may be a reduced instruction set computing (RISC) core, a complex instruction set computing (CISC) core, a very long instruction word (VLIW) core, or a hybrid or alternative core type. As yet another option, the core 1090 may be a special purpose core, such as, for example, a network or communication core, a compression engine, a coprocessor core, a general purpose computing graphics processing unit (GPGPU) core, a graphics core, etc.

[0106] The front end unit 1030 includes a branch prediction unit 1032 coupled to an instruction cache unit 1034, which is coupled to an instruction translation lookaside buffer (TLB) 1036, which is coupled to an instruction fetch unit 1038, which is coupled to a decode unit 1040. The decode unit 1040 (or decoder) can decode the instruction and generate one or more micro-operations, microcode entry points, microinstructions, other instructions or other control signals decoded from the original instruction, or otherwise reflecting the original instruction or derived from the original instruction as output. The decode unit 1040 can be implemented using a variety of different mechanisms. Examples of suitable mechanisms include, but are not limited to: a lookup table, a hardware implementation, a programmable logic array (PLA), a microcode read-only memory (ROM), etc. In one embodiment, the core 1090 includes a microcode ROM or other medium that stores microcode for certain macroinstructions (e.g., in the decode unit 1040 or within the front end unit 1030). Decode unit 1040 is coupled to rename / allocator unit 1052 in execution engine unit 1050 .

[0107] The execution engine unit 1050 includes a rename / allocator unit 1052 coupled to a retirement unit 1054 and a set of one or more scheduler units 1056. The (multiple) scheduler units 1056 represent any number of different schedulers, including reservation stations, central instruction windows, etc. The (multiple) scheduler units 1056 are coupled to (multiple) physical register file units 1058. The (multiple) physical register units 1058 each represent one or more physical register files, where different physical register files store one or more different data types, such as scalar integers, scalar floating points, packed integers, packed floating points, vector integers, vector floating points, status (e.g., an instruction pointer as the address of the next instruction to be executed), etc. In one embodiment, the (multiple) physical register file units 1058 include vector register units, write mask register units, and scalar register units. These register units can provide architectural vector registers, vector mask registers, and general purpose registers. The physical register file(s) unit(s) 1058 overlap with the retirement unit 1054 to illustrate various ways in which register renaming and out-of-order execution may be implemented (e.g., using reorder buffer(s) and retirement register file(s); using future file(s), history buffer(s) and retirement register file(s); using register maps and register pools; etc.). The retirement unit 1054 and the physical register file(s) unit(s) 1058 are coupled to the execution cluster(s) 1060. The execution cluster(s) 1060 include a set of one or more execution units 1062 and a set of one or more memory access units 1064. The execution units 1062 may perform various operations (e.g., shifts, additions, subtractions, multiplications) and on various types of data (e.g., scalar floating point, packed integer, packed floating point, vector integer, vector floating point). While some embodiments may include multiple execution units dedicated to a particular function or set of functions, other embodiments may include only one execution unit or include multiple execution units that all perform all functions. Scheduler unit(s) 1056, physical register file(s) units 1058, and execution cluster(s) 1060 are shown as possibly plural because some embodiments create separate pipelines for certain types of data / operations (e.g., scalar integer pipelines, scalar floating point / packed integer / packed floating point / vector integer / vector floating point pipelines, and / or memory access pipelines, each of which has its own scheduler unit, physical register file(s), and / or execution cluster, and in the case of separate memory access pipelines, certain embodiments are implemented in which only the execution cluster of that pipeline has memory access unit(s) 1064). It should also be understood that where separate pipelines are used, one or more of these pipelines may not be issue / execution pipelines, and the rest are in-order pipelines.

[0108] The set of memory access units 1064 are coupled to a memory unit 1070, which includes a data TLB unit 1072 coupled to a data cache unit 1074, which is coupled to a level 2 (L2) cache unit 1076. In an exemplary embodiment, the memory access unit 1064 may include a load unit, a store address unit, and a store data unit, each coupled to the data TLB unit 1072 in the memory unit 1070. The instruction cache unit 1034 is further coupled to a level 2 (L2) cache unit 1076 in the memory unit 1070. The L2 cache unit 1076 is coupled to one or more other levels of cache and ultimately to the main memory.

[0109] By way of example, an exemplary register renaming out-of-order issue / execute core architecture may implement pipeline 1000 as follows: 1) instruction fetch 1038 performs fetch and length decode stages 1002 and 1004; 2) decode unit 1040 performs decode stage 1006; 3) rename / allocator unit 1052 performs allocate stage 1008 and rename stage 1010; 4) (multiple) scheduler units 1056 perform schedule stage 1012; 5) (multiple) physical register file units 1058 and memory units 1070 perform register read / memory read stage 1014; execution cluster 1060 implements execute stage 1016; 6) memory unit 1070 and (multiple) physical register file units 1058 perform write back / memory write stage 1018; 7) various units may be involved in exception handling stage 1022; and 8) retirement unit 1054 and (multiple) physical register file units 1058 perform commit stage 1024.

[0110] Core 1090 may support one or more instruction sets (e.g., the x86 instruction set (with some extensions that newer versions have added); the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California; the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California (with optional additional extensions, such as NEON)), including the instruction(s) described herein. In one embodiment, core 1090 includes logic to support packed data instruction set extensions (e.g., AVX1, AVX2), thereby allowing operations used by many multimedia applications to be performed using packed data.

[0111] It should be understood that a core may support multithreading (executing two or more operations or sets of threads in parallel), and may accomplish multithreading in a variety of ways, including time-division multithreading, simultaneous multithreading (where a single physical core provides a logical core for each of the threads that the physical core is simultaneously multithreading), or a combination thereof (e.g., time-division fetch and decode and subsequent operations such as Hyper-Threading (Simultaneous Multithreading).

[0112] Although register renaming is described in the context of out-of-order execution, it should be understood that register renaming can be used in an in-order architecture. Although the illustrated embodiment of the processor also includes separate instruction and data cache units 1034 / 1074 and a shared L2 cache unit 1076, alternative embodiments may have a single internal cache for both instructions and data, such as a level 1 (L1) internal cache, or multiple levels of internal caches. In some embodiments, the system may include a combination of internal caches and external caches external to the core and / or processor. Alternatively, all caches may be external to the core and / or processor.

[0113] Certain Exemplary In-Order Core Architectures

[0114] Fig.11A and Fig. 11B A block diagram of a more specific exemplary in-order core architecture is shown, where the core is one of multiple logic blocks in a chip (including other cores of the same type and / or different types). Depending on the application, the logic blocks communicate through a high-bandwidth interconnect network (e.g., a ring network) with some fixed-function logic, memory I / O interfaces, and other necessary I / O logic.

[0115] Fig.11A 1 is a block diagram of a single processor core according to an embodiment of the present invention, together with its connection to the on-die interconnect network 1102 and together with its local subset of the level 2 (L2) cache 1104. In one embodiment, the instruction decoder 1100 supports the x86 instruction set with a packed data instruction set extension. The L1 cache 1106 allows low latency access to cache memory in the scalar and vector units. Although in one embodiment (to simplify the design), the scalar unit 1108 and the vector unit 1110 use separate register sets (scalar registers 11112 and vector registers 1114, respectively), and data transferred between them is written to memory and then read back from the level 1 (L1) cache 1106, alternative embodiments of the present invention may use different approaches (e.g., using a single register set or including a communication path that allows data to be transferred between two register files without being written and read back).

[0116] The local subset of L2 cache 1104 is part of the global L2 cache, which is divided into multiple separate local subsets, one for each processor core. Each processor core has a direct access path to its own local subset of L2 cache 1104. Data read by the processor core is stored in its L2 cache subset 1104 and can be quickly accessed, accessing its local L2 cache subset in parallel with other processor cores. Data written by the processor core is stored in its own L2 cache subset 1104 and, if necessary, flushed from other subsets. The ring network ensures the coherence of shared data. The ring network is bidirectional, allowing media such as processor cores, L2 caches and other logic blocks to communicate with each other within the chip. Each ring data path is 1012 bits wide in each direction.

[0117] Fig. 11B According to an embodiment of the present invention Fig.11A An expanded view of a portion of a processor core in FIG. Fig. 11B The L1 data cache 1106A portion of the L1 cache 1104 is included, as well as more details about the vector unit 1110 and vector registers 1114. Specifically, the vector unit 1110 is a 16-bit wide vector processing unit (VPU) (see the 16-bit wide ALU 1128) that executes one or more of integer, single-precision floating point, and double-precision floating point instructions. The VPU supports mixing register inputs with a mixing unit 1120, digital conversion with conversion units 1122A and 1122B, and copying memory inputs with a copy unit 1124. Write mask registers 1126 allow predictive result vector writes.

[0118] Processor with integrated memory controller and graphics

[0119] Fig.12 is a block diagram of a processor 1200 that may have more than one core, may have an integrated memory controller, and may have integrated graphics in accordance with an embodiment of the present invention. Fig.12 The solid line box in the figure shows a processor 1200 having a single core 1202A, a system agent 1210, a set of one or more bus controller units 1216, while the optional addition of the dashed line box shows an alternative processor 1200 having multiple cores 1202A to 1202N, a set of one or more integrated memory controller units 1214 in the system agent unit 1210, and dedicated logic 1208.

[0120] Thus, different implementations of processor 1200 may include: 1) a CPU with dedicated logic 1208 as integrated graphics and / or scientific (throughput) logic (which may include one or more cores) and cores 1202A to 1202N as one or more general purpose cores (e.g., general purpose in-order cores, general purpose out-of-order cores, combinations of both); 2) a coprocessor, where cores 1202A to 1202N are a large number of dedicated cores intended primarily for graphics and / or scientific (throughput); and 3) a coprocessor, where cores 1202A to 1202N are a large number of general purpose in-order cores. Thus, processor 1200 may be a general purpose processor, a coprocessor, or a dedicated processor, such as a network or communication processor, a compression engine, a graphics processor, a GPGPU (general purpose graphics processing unit), a high throughput integrated many-core (MIC) coprocessor (including 30 or more cores), an embedded processor, etc. The processor may be implemented on one or more chips. Processor 1200 may be part of one or more substrates and / or may be implemented on one or more substrates using any of a variety of processing technologies such as, for example, BiCMOS, CMOS, or NMOS.

[0121] The memory hierarchy includes one or more levels of cache within the core, a set of one or more shared cache units 1206, and external memory (not shown) coupled to the set of integrated memory controller units 1214. The set of shared cache units 1206 may include one or more mid-level caches, such as level 2 (L2), level 3 (L3), level 4 (L4), or other levels of cache, a last-minute cache (LLC), and / or combinations thereof. While in one embodiment, a ring-based interconnect unit 1212 interconnects the integrated graphics logic 1208, the set of shared cache units 1206, and the system agent unit 1210 / (multiple) integrated memory controller units 1214, alternative embodiments may use any number of known techniques for interconnecting such units. In one embodiment, coherency is maintained between the one or more cache units 1206 and the cores 1202A to 1202N.

[0122] In some embodiments, one or more of the cores 1202A to 1202N are capable of multithreading. System agent 1210 includes those components that coordinate and operate cores 1202A to 1202N. System agent unit 1210 may include, for example, a power control unit (PCU) and a display unit. The PCU may be or include logic and components required to regulate the power state of cores 1202A to 1202N and integrated graphics logic 1208. The display unit is used to drive one or more externally connected displays.

[0123] The cores 1202A-1202N may be homogeneous or heterogeneous with respect to the architectural instruction set; that is, two or more of the cores 1202A-1202N may be capable of executing the same instruction set, while other cores may be capable of executing only a subset of the instruction set or a different instruction set.

[0124] Exemplary Computer Architecture

[0125] Figures 13 to 17 is a block diagram of an exemplary computer architecture. Other system designs and configurations known in the art for laptop computers, desktop computers, handheld PCs, personal digital assistants, engineering workstations, servers, network devices, network hubs, switches, embedded processors, digital signal processors (DSPs), graphics devices, video game devices, set-top boxes, microcontrollers, cellular phones, portable media players, handheld devices, and various other electronic devices are also suitable. In general, a wide variety of systems or electronic devices that can incorporate the processors and / or other execution logic disclosed herein are generally suitable.

[0126] Now refer to Fig.13 , a block diagram of a system 1300 according to one embodiment of the present invention is shown. The system 1300 may include one or more processors 1310, 1315 coupled to a controller hub 1320. In one embodiment, the controller hub 1320 includes a graphics memory controller hub (GMCH) 1390 and an input / output hub (IOH) 1350 (which may be on separate chips); the GMCH 1390 includes memory and a graphics controller, to which a memory 1340 and a coprocessor 1345 are coupled; and the IOH 1350 couples an input / output (I / O) device 1360 to the GMCH 1390. Alternatively, one or both of the memory and the graphics controller are integrated within a processor (as described herein), and the memory 1340 and the coprocessor 1345 are directly coupled to the processor 1310 and the controller hub 1320 in a single chip through the IOH 1350.

[0127] Fig.13 The optional nature of the additional processor 1315 is indicated by dashed lines in FIG. 13. Each processor 1310, 1315 may include one or more processing cores described herein and may be some version of the processor 1200.

[0128] The memory 1340 may be, for example, a dynamic random access memory (DRAM), a phase change memory (PCM), or a combination of both. For at least one embodiment, the controller hub 1320 communicates with the processor(s) 1310, 1315 via a multi-drop bus (e.g., a front side bus (FSB), a point-to-point interface such as a Quick Path Interconnect (QPI), or a similar connection 1395).

[0129] In one embodiment, the coprocessor 1345 is a special purpose processor, such as a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc. In one embodiment, the controller hub 1320 may include an integrated graphics accelerator.

[0130] There may be various differences between the physical resources 1310 , 1315 in terms of a range of metrics including architectural, microarchitectural, thermal, power consumption characteristics, and the like.

[0131] In one embodiment, processor 1310 executes instructions that control general types of data processing operations. Coprocessor instructions may be embedded in the instructions. Processor 1310 identifies these coprocessor instructions as being of a type that should be executed by an attached coprocessor 1345. Accordingly, processor 1310 issues these coprocessor instructions (or control signals representing coprocessor instructions) on a coprocessor bus or other interconnect to coprocessor 1345. Coprocessor(s) 1345 accept and execute the received coprocessor instructions.

[0132] Now refer to Fig.14 , a block diagram of a first more specific exemplary system 1400 according to an embodiment of the present invention is shown. Fig.14 As shown, multiprocessor system 1400 is a point-to-point interconnect system and includes a first processor 1470 and a second processor 1480 coupled via a point-to-point interconnect 1450. Processors 1470 and 1480 may each be a version of processor 1200. In one embodiment of the present invention, processors 1470 and 1480 are processors 1310 and 1315, respectively, and coprocessor 1438 is coprocessor 1345. In another embodiment, processors 1470 and 1480 are processors 1310 and coprocessor 1345, respectively.

[0133] Processors 1470 and 1480 are shown as including integrated memory controller (IMC) units 1472 and 1482, respectively. Processor 1470 also includes point-to-point (PP) interfaces 1476 and 1478 as part of its bus controller unit; similarly, second processor 1480 includes PP interfaces 1486 and 1488. Processors 1470, 1480 can exchange information via point-to-point (PP) interface 1450 using PP interface circuits 1478, 1488. Fig.14 As shown, IMCs 1472 and 1482 couple the processors to corresponding memories (ie, memory 1432 and memory 1434), which may be portions of main memory locally attached to the corresponding processors.

[0134] Processors 1470, 1480 may each exchange information with chipset 1490 via separate PP interfaces 1452, 1454 using point-to-point interface circuits 1476, 1494, 1486, 1498. Alternatively, chipset 1490 may exchange information with coprocessor 1438 via high performance interface 1439. In one embodiment, coprocessor 1438 is a special purpose processor, such as a high throughput MIC processor, a network or communication processor, a compression engine, a graphics processor, a GPGPU, an embedded processor, etc.

[0135] A shared cache (not shown) may be included in either processor or external to both processors but connected to the processors via the PP interconnect such that if the processors are placed in a low power mode, local cache information of either or both processors may be stored in the shared cache.

[0136] Chipset 1490 may be coupled to first bus 1416 via interface 1496. In one embodiment, first bus 1416 may be a peripheral component interconnect (PCI) bus, or a bus such as a PCI Express bus or another third generation I / O interconnect bus, although the scope of the invention is not limited in this regard.

[0137] like Fig.14 As shown, different I / O devices 1414 may be coupled to the first bus 1416 along with a bus bridge 1418 which may couple the first bus 1416 to a second bus 1420. In one embodiment, one or more additional processors 1415 (e.g., a coprocessor, a high throughput MIC processor, a GPGPU, an accelerator (e.g., a graphics accelerator or digital signal processing (DSP) unit), a field programmable gate array, or any other processor) are coupled to the first bus 1416. In one embodiment, the second bus 1420 may be a low pin count (LPC) bus. In one embodiment, various devices may be coupled to the second bus 1420 including, for example, a keyboard and / or mouse 1422, communication devices 1427, and a storage unit 1428 (e.g., a disk drive or other mass storage device) which may include instructions / code and data 1430. Further, an audio I / O 1424 may be coupled to the second bus 1420. Note that other architectures are possible. For example, instead of Fig.14 Instead of a point-to-point architecture, the system may implement a multi-drop bus or other such architecture.

[0138] Now refer to Fig.15 , a block diagram of a second more specific exemplary system 1500 according to an embodiment of the present invention is shown. Fig.14 and Fig.15The same elements in the drawings have the same reference numerals and have been changed from Fig.15 Omitted Fig.14 to avoid using Fig.15 Other aspects are vague.

[0139] Fig.15 It is shown that processors 1470, 1480 may include integrated memory and I / O control logic ("CL") 1472 and 1482, respectively. Thus, CL 1472, 1482 includes an integrated memory controller unit and includes I / O control logic. Fig.15 It is shown that not only the memory 1432, 1434 is coupled to the CL 1472, 1482, but also the I / O device 1514 is coupled to the control logic 1472, 1482. The legacy I / O device 1515 is coupled to the chipset 1490.

[0140] Now refer to Fig.16 , shows a block diagram of a SoC 1600 according to an embodiment of the present invention. Fig.12 Similar components in FIG. 1 have the same reference numbers. In addition, dashed boxes are optional features for more advanced SoCs. Fig.16 , interconnect unit(s) 1602 are coupled to: application processor 1610, which includes a set of one or more cores 152A to 152N and (multiple) shared cache units 1206; system agent unit 1210; bus controller unit(s) 1216; integrated memory controller unit(s) 1214; a set of one or more coprocessors 1620, which may include integrated graphics logic, image processor, audio processor, and video processor; static random access memory (SRAM) unit 1630; direct memory access (DMA) unit 1632; and display unit 1640, which is used to couple to one or more external displays. In one embodiment, coprocessor(s) 1620 are special purpose processors, such as, for example, network or communication processors, compression engines, GPGPUs, high throughput MIC processors, embedded processors, etc.

[0141] Embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of such implementations. Embodiments of the present invention may be implemented as a computer program or program code executed on a programmable system comprising at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0142] like Fig.14Program codes such as the shown code 1430 can be applied to input instructions to perform functions described herein and generate output information. The output information can be applied to one or more output devices in a known manner. For the purpose of this application, the processing system includes any system with a processor, such as, for example: a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0143] The program code can be implemented in a high-level program or object-oriented programming language to communicate with the processing system. If desired, the program code can also be implemented in an assembly or machine language. In fact, the scope of the mechanism described herein is not limited to any specific programming language. In any case, the language can be a compiled or interpreted language.

[0144] One or more aspects of at least one embodiment may be implemented by representative instructions stored on a machine-readable medium, which represent various logic within a processor, and which, when read by a machine, causes the machine to fabricate logic for performing the techniques described herein. Such representations, known as "IP cores," may be stored on a tangible machine-readable medium and provided to various customers or manufacturing facilities to be loaded into a fabrication machine that actually fabricates the logic or processor.

[0145] Such machine-readable storage media may include, but are not limited to: a non-transitory tangible arrangement of items manufactured or formed by a machine or device, including storage media such as a hard disk; any other type of disk, including floppy disks, optical disks, compact disk-read only memory (CD-ROM), compact disk-rewritable (CD-RW), and magneto-optical disks; semiconductor devices, such as read-only memory (ROM); random access memory (RAM), such as dynamic random access memory (DRAM), static random access memory (SRAM); erasable programmable read-only memory (EPROM); flash memory; electrically erasable programmable read-only memory (EEPROM); phase change memory (PCM); magnetic or optical cards; or any other type of medium suitable for storing electronic instructions.

[0146] Therefore, embodiments of the present invention also include non-transitory tangible machine-readable media containing instructions or containing design data (e.g., hardware description language (HDL)), which define the structures, circuits, devices, processors and / or system features described herein. Such embodiments may also be referred to as program products.

[0147] Emulation (including binary conversion, code deformation, etc.)

[0148] In some cases, an instruction converter may be used to convert instructions from a source instruction set to a target instruction set. For example, the instruction converter may convert (e.g., using static binary translation, dynamic binary translation including dynamic compilation), deform, simulate, or otherwise convert instructions to one or more other instructions to be processed by the core. The instruction converter may be implemented in software, hardware, firmware, or a combination thereof. The instruction converter may be on the processor, off the processor, or partially on the processor and partially off the processor.

[0149] Fig.17 1 is a block diagram comparing the use of a software instruction converter for converting binary instructions in a source instruction set into binary instructions in a target instruction set according to an embodiment of the present invention. In the illustrated embodiment, the instruction converter is a software instruction converter, however, alternatively, the instruction converter may be implemented in software, firmware, hardware, or various combinations thereof. Fig.17 It is shown that an x86 compiler 1704 can be used to compile a program in a high-level language 1702 to generate x86 binary code 1706, which can be natively executed by a processor 1716 having at least one x86 instruction set core. The processor 1716 having at least one x86 instruction set core represents any processor that can perform substantially the same functions as an Intel processor having at least one x86 instruction set core by compatibly executing or otherwise processing the following: (1) a substantial portion of the instruction set of the Intel x86 instruction set core, or (2) an object code version of an application or other software that is targeted to run on an Intel processor having at least one x86 instruction set core to achieve substantially the same results as an Intel processor having at least one x86 instruction set core. The x86 compiler 1704 represents a compiler that can be used to generate x86 binary code 1706 (e.g., object code) that can be executed on a processor having at least one x86 instruction set core 1716 with or without additional link processing. Similarly, Fig.17A program of a high-level language 1702 is shown that can be compiled using an alternative instruction set compiler 1708 to generate an alternative instruction set binary code 1710 that can be natively executed by a processor that does not have at least one x86 instruction set core 1714 (e.g., a processor that has multiple cores that execute the MIPS instruction set of MIPS Technologies, Inc. of Sunnyvale, California and / or execute the ARM instruction set of ARM Holdings, Inc. of Sunnyvale, California). Instruction converter 1712 is used to convert x86 binary code 1706 to code that can be natively executed by a processor that does not have an x86 instruction set core 1714. This converted code is unlikely to be the same as the alternative instruction set binary code 1710 because it is difficult to make an instruction converter that can achieve this; however, the converted code will complete the general operation and be composed of instructions from the alternative instruction set. Therefore, instruction converter 1712 represents software, firmware, hardware, or a combination thereof that allows a processor or other electronic device that does not have an x86 instruction set processor or core to execute x86 binary code 1706 through emulation, simulation, or any other process.

[0150] Optionally, for Figure 5 and Figure 6 The components, features and details described in any one of the above may also be applicable to Figures 7 to 9 In addition, optionally, the components, features, and details described for any of the devices may also apply to any method that may be performed by and / or with such a device in an embodiment. Any of the processors described herein may include in the present invention (for example, Figures 13 to 16 ) disclosed in any of the computer systems described herein. In some embodiments, the computer system may include dynamic random access memory (DRAM). Alternatively, the computer system may include some type of volatile memory or flash memory that does not require refreshing. The instructions disclosed herein may be executed using any processor shown herein on any of the systems shown herein having any of the microarchitectures shown herein.

[0151] In the specification and claims, the terms "coupled" and "connected" and their derivatives may be used. These terms are not intended to be synonymous with each other. Rather, in an embodiment, "connected" may be used to indicate that two or more elements are in direct physical and / or electrical contact. "Coupled" may mean that two or more elements are in direct physical and / or electrical contact with each other. However, "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. For example, an execution unit may be coupled to a register and / or a decoding unit through one or more intermediate components. In the accompanying drawings, arrows are used to illustrate connections and couplings.

[0152] The term "and / or" may have been used. As used herein, the term "and / or" means one or the other or both (eg, A and / or B means A or B or both A and B).

[0153] In the above description, many specific details have been set forth in order to provide a thorough understanding of the embodiments. However, other embodiments may be practiced without some of these specific details. The scope of the present invention is not determined by the specific embodiments provided above, but by the claims below. In other cases, known circuits, structures, devices, and operations have been shown in block diagram form and / or without detail to avoid obscuring the understanding of the specification. Where deemed appropriate, reference numbers or endpoint portions of reference numbers have been repeated in the accompanying drawings to indicate corresponding or similar elements, which may have similar or identical characteristics, unless otherwise noted or clearly apparent.

[0154] Certain operations may be performed by hardware components or may be embodied in machine-executable or circuit-executable instructions, which may be used to cause and / or result in a machine, circuit, or hardware component (e.g., a processor, a portion of a processor, a circuit, etc.) being programmed with instructions to perform these operations. Alternatively, operations may also be performed by a combination of hardware and software. A processor, machine, circuit, or hardware, which may include specific or concrete circuitry or other logic (e.g., hardware that may be combined with firmware and / or software), may be operable to execute and / or process instructions and store results in response to the instructions.

[0155] Some embodiments include articles (e.g., computer program products) that include machine-readable media. The medium may include a mechanism for providing (e.g., storing) information in a machine-readable form. The machine-readable medium may provide or have stored thereon instructions or sequences of instructions that, if and / or when executed by a machine, are operable to cause the machine to perform and / or cause the machine to perform one or more operations, methods, or techniques disclosed herein.

[0156] In some embodiments, the machine-readable medium may include a non-transitory machine-readable storage medium. For example, the non-transitory machine-readable storage medium may include a floppy disk, an optical storage medium, an optical disc, an optical data storage device, a CD-ROM, a magnetic disk, a magneto-optical disc, a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a random access memory (RAM), a static RAM (SRAM), a dynamic RAM (DRAM), a flash memory, a phase change memory, a phase change data storage material, a non-volatile memory, a non-volatile data storage device, a non-transitory memory, a non-transitory data storage device, etc. The non-transitory machine-readable storage medium does not consist of a transient propagation signal. In some embodiments, the storage medium may include a tangible medium, and the tangible medium includes a solid substance.

[0157] Examples of suitable machines include, but are not limited to, general purpose processors, special purpose processors, digital logic circuits, integrated circuits, etc. Other examples of suitable machines include: computer systems or other electronic devices including processors, digital logic circuits, or integrated circuits. Examples of such computing systems or electronic devices include, but are not limited to, desktop computers, laptop computers, notebook computers, tablet computers, netbooks, smart phones, cellular phones, servers, network devices (e.g., routers and switches), mobile Internet devices (MIDs), media players, smart televisions, nettops, set-top boxes, and video game controllers.

[0158] Throughout this specification, references to "one embodiment," "an embodiment," "one or more embodiments," "some embodiments," for example, indicate that a particular feature may be included but is not necessarily required in the practice of the invention. Similarly, in the specification, various features are sometimes grouped together in a single embodiment, figure, or description thereof to simplify the disclosure and aid in understanding the various inventive aspects. However, this method of disclosure is not to be interpreted as reflecting an intention that the invention requires more features than expressly stated in each claim. On the contrary, as reflected in the claims below, inventive aspects lie in less than all of the features of a single disclosed embodiment. Accordingly, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim independently representing a separate embodiment of the invention.

[0159] Example Embodiments

[0160] The following examples relate to further embodiments. The details in the examples may be used anywhere in one or more embodiments.

[0161] Example 1 is a processor, the processor comprising a decoding unit, the decoding unit being used to decode an instruction. The instruction is used to indicate a page of a protected container memory and to indicate a storage location outside the protected container memory. The processor also includes an execution unit coupled to the decoding unit. The execution unit is used in response to the instruction to ensure that the page of the protected container memory has a write-protected state while no writable permission for the page of the protected container memory is cached in the processor. The execution unit is also used to encrypt a copy of the page of the protected container memory. The execution unit is further used to: after ensuring that there is no writable reference to the page of the protected container memory, store the encrypted copy of the page to the indicated storage location outside the protected container memory. The execution unit is also used to leave the page of the protected container memory in the write-protected state after the encrypted copy of the page has been stored in the indicated storage location outside the protected container memory, and the page will also be valid and readable.

[0162] Example 2 includes the processor of Example 1, wherein the decoding unit is to decode the instruction, the instruction being to indicate the page of the protected container memory that already has the write-protected state.

[0163] Example 3 includes the processor of Example 1, wherein the execution unit is responsive to the instruction to write-protect the indicated page of the protected container memory.

[0164] Example 4 includes the processor of Example 1, wherein the decoding unit is to decode the instruction, the instruction is to indicate the page of the protected container memory to be located in the processor reserved memory, and the instruction is to indicate the storage location to be located outside the processor reserved memory.

[0165] Example 5 includes the processor of Example 1, wherein the execution unit is to ensure that there are no writable references to the page by ensuring that the page of the protected container memory is deleted from the translation lookaside buffer.

[0166] Example 6 includes the processor of any of Examples 1 to 5, wherein the execution unit is responsive to the instruction to store a version of the page having the write-protected state in the protected container memory.

[0167] Example 7 includes a processor as described in any of Examples 1 to 5, wherein the execution unit is responsive to the instruction to determine that: before the encrypted copy of the page is stored in the indicated storage location, a migratable key structure has control over the page of the protected container memory, and the migratable key structure will have one or more migratable keys.

[0168] Example 8 includes a processor as described in any one of Examples 1 to 5, wherein the decoding unit is used to decode the instruction, the instruction is used to indicate a page metadata structure. The execution unit is responsive to the instruction to store metadata corresponding to the indicated page in the page metadata structure. The metadata will include multiple of the following items corresponding to the indicated page in the page metadata structure: page type, modification status, read permission status, write permission status, and execute permission status.

[0169] Example 9 includes the processor of any of Examples 1 to 5, wherein the decode unit is to decode the instruction to indicate the page in the protected container memory to be the page of the enclave page in the enclave page cache.

[0170] Example 10 includes the processor of any of Examples 1 to 5, wherein the decode unit is to decode the instruction to have an implicit general register having an indication of the page of the protected container memory.

[0171] Example 11 includes a processor as described in any of Examples 1 to 5, wherein the decoding unit is to decode the instruction that is to be a privileged level instruction.

[0172] Example 12 is a method of executing one to three machine instructions in a processor for performing operations, the method comprising: write-protecting a page of a protected container memory; ensuring that writable permissions for the page of the protected container memory are not cached in the processor; and encrypting a copy of the page of the protected container memory. The operation also includes: after ensuring that there are no writable references to the write-protected page of the protected container memory, storing the encrypted copy of the page of the protected container memory to a storage location outside the protected container memory; and after storing the encrypted copy of the page of the protected container memory to the storage location outside the protected container memory, leaving the write-protected page of the protected container memory in a valid and readable state.

[0173] Example 13 includes the method of Example 12, further comprising: after said storing the encrypted copy of the page to the storage location, reading the write-protected page.

[0174] Example 14 includes a method as described in Example 12, wherein write-protecting the page includes configuring a write-protect indication in a protected container page metadata structure to indicate that the page is write-protected, wherein the protected container page metadata structure stores security metadata for the write-protected page.

[0175] Example 15 includes the method of Example 14, wherein the configuring the write protection indication in the protected container page metadata structure comprises setting a write protection bit in an enclave page cache map.

[0176] Example 16 includes the method as described in Example 12, further comprising: detecting an attempted write to the write-protected page of the protected container memory; unwrite-protecting the page of the protected container memory; and invalidating the encrypted copy of the page stored in the storage location outside the protected container memory.

[0177] Example 17 includes a method as described in Example 12, wherein the write protection is performed in response to executing a first machine instruction among the machine instructions, and wherein the encryption, the ensuring, the storing and the retaining operations are performed in response to executing a second machine instruction among the machine instructions.

[0178] Example 18 is a system for processing instructions, the system comprising: an interconnect; and a processor coupled to the interconnect. The processor is configured to receive an instruction indicating a page of a protected container memory and indicating a storage location outside the protected container memory. The processor is configured to ensure, in response to the instruction, that there are no writable references to the page of the protected container memory while the page of the protected container memory has a write-protected state; and to encrypt a copy of the page of the protected container memory. The processor is further configured to: store an encrypted copy of the page to the indicated storage location outside the protected container memory after ensuring that there are no writable references to the page of the protected container memory; and after storing the encrypted copy of the page to the indicated storage location outside the protected container memory, leave the page of the protected container memory in the write-protected state, and the page will also be valid and readable. The system also includes a dynamic random access memory (DRAM) coupled to the interconnect.

[0179] Example 19 includes the system of Example 18, wherein the processor is to receive the instruction indicating the page of the protected container memory that already has the write-protected state.

[0180] Example 20 is an article of manufacture comprising a non-transitory machine-readable storage medium. The non-transitory machine-readable storage medium stores one to three machine instructions that, if executed by a machine, will cause the machine to perform operations including: write-protecting a page of a protected container memory; and ensuring that there are no writable references to the write-protected page of the protected container memory. The operations also include: encrypting a copy of the page of the protected container memory; and after ensuring that there are no writable references to the write-protected page of the protected container memory, storing the encrypted copy of the page of the protected container memory to a storage location outside the protected container memory. The operations also include leaving the write-protected page of the protected container memory in a valid and readable state after storing the encrypted copy of the page of the protected container memory to the storage location outside the protected container memory.

[0181] Example 21 includes an article as described in Example 18, wherein the non-transitory machine-readable storage medium further stores one or two machine instructions that, if executed by a machine, cause the machine to perform operations including: after detecting an attempted write to the write-protected page of the protected container memory, unwrite-protecting the page of the protected container memory; and invalidating the encrypted copy of the page stored in the storage location outside the protected container memory.

[0182] Example 22 is a processor, the processor comprising a decoding unit, the decoding unit being configured to decode a protected container page write protection instruction. The instruction is configured to indicate a page of a protected container memory. The processor further comprises an execution unit coupled to the decoding unit. The execution unit is configured to write-protect the indicated page of the protected container memory in response to the protected container page write protection instruction.

[0183] Example 23 includes a processor as described in Example 22, wherein the execution unit is used to write-protect the indicated page by configuring a write protection indicator corresponding to the indicated page in a protected container page metadata structure used to store metadata of the indicated page.

[0184] Example 24 includes the processor of Example 23, wherein the execution unit is to write protect the indicated page by configuring a write protection bit in an enclave page cache map.

[0185] Example 25 includes a processor as described in any of Examples 22 to 24, wherein the execution unit is responsive to the instruction to determine that: before the page is write-protected, the migratable key structure has control over the page of the protected container memory, and the migratable key structure will have one or more migratable keys.

[0186] Example 26 includes the processor of any of Examples 22 to 24, wherein the decode unit is to decode the instruction to have an implicit register having an effective address of the page of the protected container memory executed by hardware.

[0187] Example 27 is a processor, the processor comprising a decoding unit, the decoding unit being configured to decode a protected container page write-protection removal and copy invalidation instruction. The instruction is configured to indicate a page of a protected container memory. The processor further comprises an execution unit coupled to the decoding unit. The execution unit is configured to, in response to the instruction, remove write protection from the indicated page of the protected container memory and invalidate any copy of the page of the protected container memory that is to be located outside the protected container memory.

[0188] Example 28 includes the processor of Example 27, wherein the decoding unit is to decode the instruction, the instruction is to indicate version information, and wherein the execution unit is to invalidate the any copy of the page by changing the indicated version information.

[0189] Example 29 includes the processor of Example 28, wherein the version information is to be stored in the protected container memory.

[0190] Example 30 includes a processor as described in any of Examples 27 to 29, wherein the execution unit is used to remove write protection from the indicated page by configuring the write protection indicator in a protected container page metadata structure in response to the instruction, and the protected container page metadata structure is used to store security metadata of pages of the protected container memory.

[0191] Example 31 includes a processor as described in any of Examples 1 to 11, the processor further comprising: an optional branch prediction unit for predicting branches; and an optional instruction prefetch unit coupled to the branch prediction unit, the instruction prefetch unit for prefetching instructions including the instruction. Optionally, the processor may also include an optional level 1 (L1) instruction cache coupled to the instruction prefetch unit, the L1 instruction cache for storing instructions; an optional L1 data cache for storing data; and an optional level 2 (L2) cache for storing data and instructions. The processor may also optionally include an instruction fetch unit, the instruction fetch unit coupled to the decode unit, the L1 instruction cache, and the L2 cache to fetch the instruction from one of the L1 instruction cache and the L2 cache in some cases and provide the instruction to the decode unit. The processor may also optionally include: a register renaming unit for renaming registers; an optional scheduler for scheduling one or more operations that have been decoded from the instruction for execution; and an optional commit unit for committing the execution result of the instruction.

[0192] Example 32 is a processor or other device that is used to perform or is operable to perform a method as described in any of Examples 12 to 17.

[0193] Example 33 is a processor or other device comprising means for performing the method as described in any of Examples 12 to 17.

[0194] Example 34 is an article of manufacture comprising an optionally non-transitory machine-readable medium that optionally stores or otherwise provides instructions that, if and / or when executed by a processor, computer system, electronic device, or other machine, are operable to cause the machine to perform a method as described in any of Examples 12 to 17.

[0195] Example 35 is a processor or other apparatus substantially as described herein.

[0196] Example 36 is a processor or other apparatus operable to perform any of the methods substantially as described herein.

[0197] Example 37 is a processor or other apparatus for executing (eg, having components for executing or being operable to execute) any instructions substantially as described herein.

[0198] Example 38 is a computer system or other electronic device, comprising a processor having a decoding unit, the decoding unit being used to decode instructions of a first instruction set. The processor also has one or more execution units. The electronic device also includes a storage device coupled to the processor. The storage device is used to store a first instruction, which may be any instruction in the instructions substantially as disclosed herein, and the first instruction will belong to a second instruction set. The storage device is also used to store instructions for converting the first instruction into one or more instructions of the first instruction set. The one or more instructions of the first instruction set will cause the processor to emulate the first instruction when executed by the processor.

Claims

1. A processor, comprising: a decoding unit configured to decode an instruction indicating a page in a protected container memory and indicating a storage location outside of the protected container memory; as well as an execution unit, the execution unit being coupled to the decoding unit, the execution unit being responsive to the instruction to: ensuring that no writable permission for the page of the protected container memory is cached in the processor while the page of the protected container memory is in a write-protected state; encrypting a copy of the page of the protected container memory; storing an encrypted copy of the page to an indicated storage location outside of the protected container memory after ensuring that no writable references to the page of the protected container memory exist; as well as After the encrypted copy of the page has been stored to the indicated storage location outside of the protected container memory, the page of the protected container memory is left in the write-protected state, the page will also be valid and readable.

2. The processor of claim 1, wherein: The decoding unit is configured to decode the instruction, wherein the instruction indicates the page of the protected container memory that has the write protection state.

3. The processor of claim 1, wherein: The execution unit is responsive to the instruction to write-protect the indicated page of the protected container memory.

4. A method of executing one to three machine instructions in a processor to perform operations comprising: Write-protecting pages in protected container memory; ensuring that writable permissions for the page of the protected container memory are not cached in the processor; encrypting a copy of the page of the protected container memory; storing the encrypted copy of the page of the protected container memory to a storage location outside of the protected container memory after ensuring that there are no writable references to the write-protected page of the protected container memory; as well as Subsequent to the storing the encrypted copy of the page of the protected container memory to the storage location outside of the protected container memory, the write-protected page of the protected container memory is left in a valid and readable state.

5. The method of claim 4, further comprising: Subsequent to the storing the encrypted copy of the page to the storage location, the write-protected page is read.

6. The method of claim 4, wherein: The operation of write-protecting the page includes configuring a write protection indication in a protected container page metadata structure to indicate that the page is write-protected, wherein the protected container page metadata structure stores security metadata for the write-protected page.

7. A system for processing instructions, the system comprising: interconnection; a processor coupled to the interconnect, the processor configured to receive an instruction indicating a page of a protected container memory and indicating a storage location outside of the protected container memory, the processor configured to, in response to the instruction: ensuring that no writable references to the page of the protected container memory exist while the page of the protected container memory has a write-protected state; encrypting a copy of the page of the protected container memory; storing an encrypted copy of the page to an indicated storage location outside of the protected container memory after ensuring that no writable references to the page of the protected container memory exist; as well as After the encrypted copy of the page has been stored to the indicated storage location outside of the protected container memory, leaving the page of the protected container memory in the write-protected state, the page will also be valid and readable; and A dynamic random access memory (DRAM) is coupled to the interconnect.

8. The system of claim 7, wherein: The processor is configured to receive the instruction, wherein the instruction indicates the page of the protected container memory that has the write-protected state.

9. An article of manufacture comprising a non-transitory machine-readable storage medium storing one to three machine instructions that, if executed by a machine, will cause the machine to perform operations comprising: Write-protecting pages in protected container memory; ensuring that there are no writable references to the write-protected pages of the protected container memory; encrypting a copy of the page of the protected container memory; Subsequent to said ensuring that there are no writable references to said write-protected page of said protected container memory, storing said encrypted copy of said page of said protected container memory to a storage location external to said protected container memory; and Subsequent to the storing the encrypted copy of the page of the protected container memory to the storage location outside of the protected container memory, the write-protected page of the protected container memory is left in a valid and readable state.

10. The article of claim 9, wherein: The non-transitory machine-readable storage medium further stores one or two machine instructions that, if executed by a machine, cause the machine to perform operations including: After detecting an attempted write to the write-protected page of the protected container memory, unwrite-protecting the page of the protected container memory; as well as The encrypted copy of the page stored in the storage location outside of the protected container memory is invalidated.

Images