Occupancy state-based operations in processor cache

By maintaining occupancy state information in the processor cache of the computer system and optimizing write and read operations, the problem of low occupancy state management efficiency in the prior art is solved, and the efficiency and performance of memory operation are improved.

CN118939447APending Publication Date: 2024-11-12NOKIA NETWORKS OY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410553262.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-09
Filing Date
2024-05-07
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing computer systems are difficult to efficiently manage occupied state information in processor cache, resulting in low memory operation efficiency, especially when writing and reading operations, it is necessary to scan multiplexes for empty or full cache lines, increasing clock cycles Number and power consumption.

Method used

By keeping occupied state information, including full-bit and empty-bit in the cache, memory operations are performed based on this information, avoiding scanning each path to determine whether the set is full or empty, thereby optimizing write and read operations.

Benefits of technology

Reduces the number of clock cycles and power consumption required for memory block writing and reading, improves processor performance and efficiency, especially avoids eviction processes during writes and avoids searching every way during reading.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939447A_ABST
    Figure CN118939447A_ABST
Patent Text Reader

Abstract

Various example embodiments are presented herein for supporting operation of a processor cache. Various example embodiments for supporting operation of a processor cache may be configured to support operation of an N-way set association cache based on occupancy state information. Various example embodiments for supporting operation of an N-way set associated cache may be configured to support operation of an N-way set associated cache based on occupancy state information, wherein the occupancy state information may be used to support more efficient memory operations on the N-way set associated cache. Various example embodiments for supporting operations of an N-way set-associated cache based on occupancy state information may be configured to maintain occupancy state information for a set of the N-way set-associated cache to support more efficient memory operations on the N-way set-associated cache, including write operations and read operations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Various example embodiments relate generally to computer systems and, more particularly, but not exclusively, to cache memories associated with processors of computer systems. Background Art

[0002] In various contexts, computer systems utilize various types of processors to perform various functions. In various contexts, processors utilize various types of caches to perform various functions. Summary of the invention

[0003] In at least some example embodiments, an apparatus includes a cache, the cache including a set having multiple ways, wherein the cache is configured to: maintain occupancy status information for the set, the occupancy status information indicating occupancy of the set; and perform a memory operation based on the occupancy status information. In at least some example embodiments, the occupancy status information includes: at least one of information indicating whether the set is full, or information indicating whether the set is empty. In at least some example embodiments, the occupancy status information includes at least one of the following items: a full-bit indicating whether the set is full, an empty-bit indicating whether the set is empty, a value indicating the number of ways in the set that are full, or a value indicating the number of ways in the set that are empty. In at least some example embodiments, the occupancy status information includes: information indicating whether the set is full. In at least some example embodiments, the information indicating whether the set is full includes at least one of the following items: a full-bit set in a manner indicating whether the set is full, a value indicating the number of ways in the set that are occupied, or a value indicating the number of ways in the set that are empty. In at least some example embodiments, the memory operation is a write operation, wherein the cache is configured to: initiate an eviction process for the set based on determining that the occupancy status information indicates that the set is full, without scanning multiple ways of the set to find an empty cache line in which the memory block is stored. In at least some example embodiments, the occupancy status information includes: information indicating whether the set is empty. In at least some example embodiments, the information indicating whether the set is empty includes at least one of the following items: an empty bit set in a manner indicating whether the set is empty, a value indicating the number of empty ways in the set, or a value indicating the number of occupied ways in the set. In at least some example embodiments, the memory operation is a read operation, wherein the cache is configured to select a next set of the cache to search for the memory block based on determining that the occupancy status information indicates that the set is empty, without scanning multiple ways of the set to find the memory block. In at least some example embodiments, the cache includes: a data array, a tag array, and an occupancy status information array, wherein the occupancy status information is maintained in the occupancy status information array. In at least some example embodiments, the occupancy status information includes: a bit indicating the occupancy of the set. In at least some example embodiments, the bits indicating occupancy of the set are determined based on a corresponding plurality of occupancy status bits for the ways of the set. In at least some example embodiments, the occupancy status bits are stored in a corresponding plurality of entries of a tag array of the cache. In at least some example embodiments, the occupancy status bits indicate whether the corresponding way is empty, or indicate whether the corresponding way is full. In at least some example embodiments, the memory operation is a write operation for the memory block.In at least some example embodiments, to perform a write operation for a memory block, the cache is configured to: determine, based on a memory block address of the memory block, that the set is a default set for the memory block; determine, based on the occupancy status information, whether the set is full; and perform a write operation for the memory block based on whether the set is full. In at least some example embodiments, to perform a write operation for a memory block, the cache is configured to: determine, based on the occupancy status information, that the set is not full; select an empty cache line in the set from the set; and store the memory block in the empty cache line in the set. In at least some example embodiments, to perform a write operation for a memory block, the cache is configured to: determine, based on the occupancy status information, that the set is full; evict the cache line of the set from the set; and store the memory block in the cache line of the set. In at least some example embodiments, the memory operation is a read operation for a memory block. In at least some example embodiments, in order to perform a read operation for a memory block, the cache is configured to: determine, based on a memory block address of the memory block, that the set is a default set of the memory block; determine, based on occupancy status information, whether the set is empty; and perform a read operation for the memory block based on whether the set is empty.

[0004] In at least some example embodiments, a method includes: for a cache including a set having multiple ways, maintaining occupancy status information, the occupancy status information indicating occupancy of the set; and performing a memory operation based on the occupancy status information. In at least some example embodiments, the occupancy status information includes: at least one of information indicating whether the set is full, or information indicating whether the set is empty. In at least some example embodiments, the occupancy status information includes at least one of the following items: a full-bit indicating whether the set is full, an empty bit indicating whether the set is empty, a value indicating the number of ways in the set that are full, or a value indicating the number of ways in the set that are empty. In at least some example embodiments, the occupancy status information includes: information indicating whether the set is full. In at least some example embodiments, the information indicating whether the set is full includes at least one of the following items: a full-bit set in a manner indicating whether the set is full, a value indicating the number of ways in the set that are occupied, or a value indicating the number of ways in the set that are empty. In at least some example embodiments, the memory operation is a write operation, and the method includes: based on determining that the occupancy status information indicates that the set is full, initiating an eviction process for the set without scanning multiple ways of the set to find an empty cache line in which a memory block is stored. In at least some example embodiments, the occupancy status information includes: information indicating whether the set is empty. In at least some example embodiments, the information indicating whether the set is empty includes at least one of the following items: an empty bit set in a manner indicating whether the set is empty, a value indicating the number of ways in the set that are empty, or a value indicating the number of ways in the set that are occupied. In at least some example embodiments, the memory operation is a read operation, and the method includes: based on determining that the occupancy status information indicates that the set is empty, selecting a next set of the cache to search for the memory block without scanning multiple ways of the set to find the memory block. In at least some example embodiments, the cache includes: a data array, a tag array, and an occupancy status information array, wherein the occupancy status information is maintained in the occupancy status information array. In at least some example embodiments, the occupancy status information includes: a bit indicating occupancy of the set. In at least some example embodiments, the bit indicating occupancy of the set is determined based on a corresponding plurality of occupancy status bits for a way of the set. In at least some example embodiments, the occupancy status bits are stored in a corresponding plurality of entries of a tag array of a cache. In at least some example embodiments, the occupancy status bits indicate whether the corresponding way is empty or whether the corresponding way is full. In at least some example embodiments, the memory operation is a write operation for a memory block. In at least some example embodiments, performing a write operation for a memory block includes: determining that the set is a default set for the memory block based on a memory block address of the memory block; determining whether the set is full based on the occupancy status information; and performing a write operation for the memory block based on whether the set is full.In at least some example embodiments, performing a write operation for a memory block includes: determining that a set is not full based on occupancy status information; selecting a cache line in the set that is empty from the set; and storing the memory block in the cache line in the set that is empty. In at least some example embodiments, performing a write operation for a memory block includes: determining that the set is full based on occupancy status information; evicting a cache line of the set from the set; and storing the memory block in the cache line of the set. In at least some example embodiments, the memory operation is a read operation for a memory block. In at least some example embodiments, performing a read operation for a memory block includes: determining that the set is a default set for the memory block based on a memory block address of the memory block; determining whether the set is empty based on the occupancy status information; and performing a read operation for the memory block based on whether the set is empty. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The teachings herein may be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:

[0006] Figure 1 An example embodiment of a computing system including a processor and a memory for illustrating an instruction pipeline supported by the processor is depicted;

[0007] Figure 2 Depicted as Figure 1 An example embodiment of a processor of a computing system;

[0008] Figure 3 An example embodiment of a multi-core processor including multiple cores and multiple levels of cache is depicted;

[0009] Figure 4 An example embodiment of an N-way set associative cache for use in a processor is depicted;

[0010] Figure 5 An example embodiment of field partitioning of addresses of memory blocks in a memory is depicted;

[0011] Figure 6 Depicting example embodiments of P-tag, P-index, and P-offset mappings of addresses of memory blocks to be stored in a cache based on write operations;

[0012] Figure 7 An example embodiment of a set associative cache prior to storing a memory block that does not support occupancy state information is depicted;

[0013] Figure 8 An example embodiment of a set associative cache prior to storing a memory block supporting occupancy state information is depicted;

[0014] Fig. 9 depicts an example embodiment of logic circuitry configured to determine a full-bit to be used as occupied status information for a write operation;

[0015] Fig.10 Depicting an example embodiment of a P-tag, P-index, and P-offset mapping of an address of a memory block to be read from a cache based on a read operation;

[0016] Fig.11 depicts an example embodiment of a set associative cache prior to reading a memory block that does not support occupancy state information;

[0017] Fig.12 depicts an example embodiment of a set associative cache prior to reading a memory block supporting occupancy status information;

[0018] Fig.13 depicts an example embodiment of logic circuitry configured to determine empty bits to be used as occupied status information for a read operation;

[0019] Fig.14 depicts an example embodiment of a set associative cache including occupancy state information;

[0020] Fig.15 An example embodiment of an N-way set associative cache configured to support use of occupancy state information is depicted;

[0021] Fig.16 An example embodiment of a method for storing memory blocks in a set associative cache supporting occupancy state information is depicted;

[0022] Fig.17 Depicted with Fig.16 An example embodiment of a method for finding an empty cache line in a set associative cache used in conjunction with the method of;

[0023] Fig.18 An example embodiment of a method for reading a memory block from a set associative cache supporting occupancy state information is depicted;

[0024] Fig.19 Depicted with Fig.18 An example embodiment of a method for finding an empty cache line in a set associative cache used in conjunction with the method of;

[0025] Fig. 20 An example embodiment of a method for operating a cache using occupancy status information is depicted; and

[0026] Fig.21An example embodiment of a computer suitable for use in performing the various functions presented herein is depicted.

[0027] To facilitate understanding, identical reference numerals have been used herein, where possible, to designate identical elements that are common to the various figures. DETAILED DESCRIPTION

[0028] Various example embodiments for supporting the operation of a processor cache are presented herein. Various example embodiments for supporting the operation of a processor cache may be configured to support the operation of an N-way set associative cache based on occupancy state information. Various example embodiments for supporting the operation of an N-way set associative cache may be configured to support the operation of an N-way set associative cache based on occupancy state information, wherein the occupancy state information may be used to support more efficient memory operations on the N-way set associative cache. Various example embodiments for supporting the operation of an N-way set associative cache based on occupancy state information may be configured to maintain occupancy state information of a set of N-way set associative caches to support more efficient memory operations on the set of N-way set associative caches, such as by preventing a way-by-way search to find an empty way in the set for a write operation, wherein the occupancy state information of the set indicates that the set is full; or by preventing a way-by-way search to find a data element stored in the set for a read operation, wherein the occupancy state information of the set indicates that the set is empty.

[0029] In at least some example embodiments, an N-way set associative cache may be configured to support memory operations for a memory block based on the occupancy state information by maintaining occupancy state information for a set and performing memory operations for the memory block based on the occupancy state information. The memory operation may be a write operation or a read operation. The occupancy state information for the set may include various types of occupancy state information, which may be used to support more efficient memory operations, which may vary for different memory operations. For example, the occupancy state information for the set may include information indicating whether the set is full (e.g., a full-bit that may be set in a manner indicating whether the set is full, an indication of the number of ways currently occupied in the set, or an indication of the number of ways currently empty in the set, etc., and various combinations thereof). For example, the occupancy state information for the set may include information indicating whether the set is empty (e.g., an empty bit that may be set in a manner indicating whether the set is empty, an indication of the number of ways currently empty in the set, an indication of the number of ways currently occupied in the set, etc., and various combinations thereof). It should be appreciated that various other types of occupancy state information may be supported for enabling more efficient memory operations on an N-way set associative cache.

[0030] In at least some example embodiments, during a write operation for writing a memory block to a set of an N-way set associative cache, the N-way set associative cache may be configured to check occupancy status information for the set before scanning the set for the availability of empty ways that can be used to store the memory block, to determine whether the set of the cache is full, thereby requiring the use of an eviction process to write the memory block to the set. Assume that the cache or processor employs a mechanism that predetermines that the memory block to be stored does not exist in the set. For example, the set occupancy information for the set that can be checked to determine whether the set of the cache is full may include: a full-bit that can be set in a manner indicating whether the set is full, an indication of the number of ways currently occupied in the set, an indication of the number of ways currently empty in the set, and the like, and various combinations thereof. It should be appreciated that this enables the N-way set associative cache to determine whether an eviction process needs to be performed in order to write the memory block to the set without first having to scan each way of the set to determine the availability of empty ways that can be used to store the memory block, thereby reducing the number of clock cycles and power consumption required to store the memory block.

[0031] In at least some example embodiments, during a read operation for reading a memory block into a set of an N-way set associative cache, the N-way set associative cache may be configured to check occupancy status information for the set before scanning the set to find a way in which the memory block may be stored to determine whether the set of the cache is empty, thereby eliminating the need to search each way to determine whether the memory block is stored in the set. For example, the set occupancy information for the set that may be checked to determine whether the set of the cache is full may include: an empty bit that may be set in a manner indicating whether the set is empty, an indication of the number of ways in the set that are currently empty, an indication of the number of ways in the set that are currently occupied, and the like, and various combinations thereof. It should be appreciated that this enables the N-way set associative cache to determine whether the memory block should be searched in the set without having to scan each way of the set to determine whether the memory block is stored in the set, thereby reducing the number of clock cycles and power consumption required to read the memory block.

[0032] It should be appreciated that advantages or potential advantages of these and various other example embodiments, as well as example embodiments for supporting operation of a processor cache based on occupancy state information, may be further understood by reference to the various figures, which will be further discussed below.

[0033] Figure 1 An example embodiment of a computing system including a processor and a memory is depicted for illustrating an instruction pipeline supported by the processor.

[0034] Computing system 100 includes processor 110 and memory 120. Processor 110 includes instruction cache (IC) 111, micro-operation cache (UC) 112, and branch predictor (BP) 113. High-level stages in the pipeline supported by processor 110 include fetch stage 130, decode stage 140, and execute stage 150.

[0035] In the processor 110, the format and encoding of the instructions in the program to be executed by the processor 110 are determined by the instruction set architecture (ISA) of the processor 110. For example, some well-known ISAs include x86 / x86-64, IA-32 / IA-64, MIPS, ARM, etc.; however, due to its complexity, the microarchitecture of the processor is generally unable to execute ISA instructions in its native form. The ISA is designed to provide complex operations, which in turn also keeps the program compact, that is, reduces the program's footprint in memory. It should be noted that the optimal footprint of the program in memory is particularly important for the optimal use of the IC. Most ISAs provide variable-length instructions, which further increases the complexity of execution. Therefore, at the microarchitecture level of the processor, instructions are represented by simpler micro-operations of fixed length (commonly referred to as "cro-ops" or "UOPs"). An ISA instruction is decomposed into one or more fixed-length UOPs. UOPs perform basic operations on data stored in one or more registers, including transferring data between registers or between registers and external buses, performing arithmetic and logical operations on registers, etc. For example, for an ISA instruction that adds register to memory to add the value in register X to the value in memory location M, the instruction is decomposed into a sequence of three independent UOPs as follows: (1) load from M to register Y; (2) add Y to X; and (3) store X to M.

[0036] In the processor 110, the execution of a program is based on a pipeline, which, as indicated above, includes a fetch stage 130, a decode stage 140, and an execute stage 150. The fetch stage 130 retrieves a block of instructions of a program from the IC 111 or the memory 120. The IC 111 is located on the processor 110 board. The size of the IC 111 is typically much smaller than the memory 120 (e.g., 32kB, 64kB, 128kB, etc.), and is therefore much faster than the memory 120. The IC 111 caches blocks of instructions fetched from the memory 120 in units called "IC lines" (or more generally, cache lines). If a set of instructions is fetched repeatedly, these instructions may be available in the IC 111, so a hit in the IC 111 reduces the time to fetch instructions (compared to fetching instructions from the memory 120). The IC 111 is independent of the syntax and semantics of the instructions, and the IC line caches all instructions in a fixed address range in the memory block, i.e., the memory 120. The typical size of an IC row is 64B, but it should be understood that other sizes can be supported. Only when an IC row is not found in IC 111, the processor 110 extracts an instruction block from memory 120. In IC 111, a memory block is identified by the first memory address in the memory block. In the decoding stage 140, the instructions extracted during the extraction stage 130 are dynamically decoded by the processor 110 as the native UOPs of the instruction. This dynamic decoding also provides a clearer separation of the "stable" and "standardized" ISA from the underlying micro-architecture of the processor 110, and the processor 110 can freely define its own UOP set. As a result, programs written for ISA can run on different micro-architectures that support the ISA. This enables program compatibility between processors of different generations to be easily achieved. For example, different micro-architectures can support the same ISA, but each micro-architecture can define its own native UOP set. The execution stage 150 executes the UOPs supplied by the decoding stage 140.

[0037] In processor 110, extraction stage 130 and decoding stage 140 are usually expensive in terms of clock cycles and power consumption. Therefore, many modern processors implement another instruction cache, usually referred to as micro-operation cache (UC) or decoding stream buffer (DSB), which stores decoded UOPs. This is illustrated as UC 112 of processor 110. When processor 110 needs to execute instructions and its decoded UOPs are already present in UC 112, then UC 112 can directly supply UOPs to execution stage 150. The size of UC 112 is usually much smaller than IC 111 and memory 120 (e.g., 1.5kB, 2kB, 3kB, etc.), and therefore much faster than IC 111 and memory 120 (usually operated at the clock speed of processor 110). A hit in UC 112 eliminates extraction stage 130 and decoding stage 140 (both are expensive), thereby improving the performance and power budget of processor 110. The instruction is fetched and decoded only if the instruction misses in UC 112, otherwise the fetch stage 130 and decode stage 140 may be powered down. Note that although the instructions are not shown in the UC 112 for clarity, the instructions are not shown in the UC 112 for clarity. Figure 1 Although omitted in the MCU, some processors may use a component called a trace cache (TC) in place of the UC, where the TC is simpler than the UC because the TC is a single large block that contains all the instructions or micro-ops for control flow.

[0038] In the processor 110, the UC 112 stores the UOPs received from the decoding stage 140 in blocks of smaller size, but in the order of execution. This means that each branch (conditional or unconditional) causes the processor 110 to start with a new UC row, even if the current IC row has not been filled. This simple rule allows high bandwidth to be extracted from the UC 112, because once there is a hit in the UC 112, the entire UC row can be supplied to the execution stage 150 without worrying about changes in the execution sequence in the middle of the UC row. In this article, unless otherwise specified, the address of the instruction in the memory is referred to as the instruction pointer (IP). The UC row is identified by the IP of the parent instruction of the first UOP in the UC row; except that there is no correlation between the UOPs in the UC row and their corresponding parent instructions, it should be noted that since the entire UC row is supplied to the execution stage 150, this correlation is not required. Therefore, the UOPs in the UC row cannot usually be found by the IP of their parent instruction.

[0039] In the processor 110, the BP 113 is configured to predict the outcome of a conditional branch instruction when fetching instructions from the memory 120, IC 111, or UC 112. A program may include a branch instruction that changes the sequence of instructions executed by the processor 110. Branch instructions are generally of two types: a one-way unconditional branch instruction and a two-way conditional branch instruction. An unconditional branch instruction always jumps to a different location in the program memory where the program code branch is stored. A conditional branch instruction may be (1) "not taken" and continue to execute the first branch of the code immediately following the conditional branch instruction, or (2) "taken" and jumps to a different location in the program memory where the second branch of the code is stored. The outcome of the conditional branch instruction depends on certain conditions, such as a predicate variable. It is not determined whether a conditional branch will be taken or not taken until the condition has been calculated and the conditional branch has passed through the execute stage 150 in the instruction pipeline. This means that the processor 110 must wait until the conditional branch instruction has passed through the execute stage 150 before the next instruction can enter the fetch stage 130 in the pipeline, which essentially stalls the pipeline. To solve this problem, the front end of processor 110 attempts to guess whether a conditional branch is more likely to be taken or not taken. The branch guessed most likely is then fetched and speculatively executed. BP 113 is a logic circuit system that enables the front end of processor 110 to predict the outcome of a conditional branch instruction. If it is later detected that the guess was wrong, the speculatively executed, or partially executed, instruction will be discarded and the pipeline restarted with the correct branch, thereby incurring a delay known as a "branch misprediction penalty".

[0040] It should be understood that processors typically use component stages to implement each of the three levels of the instruction pipeline. As a result, the pipeline of a processor may consist of many stages (e.g., 20 or more stages). Figure 2 An example of a processor is presented to illustrate the stages used to implement portions of an instruction pipeline.

[0041] Figure 2 Depicted as Figure 1 An example embodiment of a processor of a computing system.

[0042] The processor 200 may include a front end and a back end. It should be noted that although the details of the front end are illustrated, the details of the back end are omitted for the sake of clarity.

[0043] Processor 200 includes a level 1 (L1) instruction cache (L1-IC) 210, an instruction fetch unit (IFU) 220, a branch prediction unit (BPU) 230, an instruction length decoder (ILD) 240, an instruction queue (IQ) 250, an instruction decoder (ID) 260, a UOP cache (UC) 270, and an instruction decode queue (IDQ) 280. It should be appreciated that IFU 220 and BPU 230 may be viewed as forming a fetch stage, while ILD 240, IQ 250, ID 260, and IDQ 280 may be viewed as forming a decode stage.

[0044] L1-IC 210 is a cache that is part of the cache hierarchy of processor 200 and, in general, it may be further understood by considering the cache hierarchy of a processor and the cache arrangement of the caches.

[0045] In general, a cache is a smaller, faster memory, closer to the processor, which stores copies of program instructions or program data from frequently accessed memory locations to reduce the average cost (time or energy) of access. Program instructions or program data are stored in a cache by a continuous block of memory locations commonly referred to as cache lines, wherein each cache line is indexed in the cache by the first memory address in the cache line. Caches benefit from the temporal and spatial locality of memory access patterns in programs, wherein spatial locality refers to the use of relatively close memory locations (e.g., within a cache line), and temporal locality refers to the reuse of specific cache lines within a relatively small duration. Many processors use multi-level caches. For example, common processor architectures may utilize at least three levels (L) of cache, which are commonly referred to as L1, L2, and L3. L1 cache is the smallest and closest to the processor core, and is therefore faster than caches at other levels. Typically, the L1 cache is divided into two parts: an L1 instruction cache (e.g., 32 kB, 64 kB, etc., although other sizes may be used) that only stores program instructions, and an L1 data cache (e.g., 32 kB, 64 kB, etc., although other sizes may be used) that only stores program data. The L2 cache (e.g., 256 kB, 512 kB, etc., although other sizes may be used) and the L3 cache (e.g., 2 MB, 4 MB, etc., although other sizes may be used) are subsequent levels, which are typically unified caches (meaning that they store both program instructions and program data). For the processor core in a multi-core processor, the L3 cache is typically public, and is therefore located outside the processor core. It should be appreciated that cache size and access latency grow according to level. If the cache line corresponding to the memory address being sought is missing in the L1 cache, the processor performs a search in the caches of subsequent levels (e.g., L2 cache, then L3 cache, etc.). If the memory address is missing in all available cache levels, the processor can access the main memory to retrieve the instruction or data at the memory address. Therefore, the main memory is accessed only when the memory address is missing in all caches. Once the missing block is located, it is placed into a cache line in the L1 cache.

[0046] In general, caches are usually organized as set associative arrays, which can be imagined as MxN matrices. The cache is divided into M sets, and each set contains N cache lines. In order to put a memory block into the cache, its address is usually divided into three fields: tag, index, offset. First, the memory block is mapped to the set according to the "index bit" derived from the address of the memory block. Then, the memory block is placed in the cache line in the set, and the "tag" is stored in the cache line. The tag consists of bits in the address of the memory block (except the index bit), which can be distinguished between cache lines sharing the same set. The offset field refers to any address within the cache line. The offset field consists of several least significant bits of the address of the memory block, and the number of bits depends on the size of the cache line. For example, if the cache line size is 64B, the 6 least significant bits of the address of the memory block can be used as offset bits. As previously noted, the term "IP" is used to identify the memory address of an instruction, and the three fields of IP (which are used to map instruction blocks into cache) are referred to as IP-tag, IP-index, and IP-offset, respectively. In a typical cache, if all cache lines in a set are occupied when attempting to store a new memory block, the existing cache lines in the set will be evicted (the replacement strategy selects the cache lines to be evicted), making way for the new memory block. When the evicted cache line is accessed later, it will result in a miss in the cache, and therefore it will be necessary to bring it back to the cache from the memory hierarchy. This miss is referred to as a conflict miss, and repeated conflict misses due to conflicts between cache lines sharing the same set are referred to as jitter. If a cache line is evicted due to capacity overflow (i.e., there are no more unused cache lines in the entire cache) and the evicted cache line is accessed again, a miss in the cache will result. This miss is referred to as a capacity miss. Capacity misses are extremely rare and most of the time they are due to thrashing.

[0047] The IFU 220 is responsible for feeding the processor with instructions to be executed and, therefore, is the first component to process instructions. The IFU 220 primarily includes the logic required to calculate the next fetch address and then fetch instructions from the L1-IC 210. The IFU 220 fetches instructions from the L1-IC 210 in the form of a raw byte stream.

[0048] The BPU 230 is configured to predict the next fetch address of the IFU 220, because otherwise the branch instruction introduces a significant additional level of complexity in the fetch instruction stream, because the correct fetch address cannot be calculated before the branch instruction itself is executed. By default, the processor processes instructions in sequence. This sequential execution can be interrupted by control instructions (e.g., conditional branches, unconditional branches, subroutine calls, and subroutine returns, etc.) to start executing a sequence of instructions starting from a new address (target address). For example, JE (jump if equal) is an example of a conditional branch instruction in x86, which depends on the equality of two variables (data elements). Conditional branches are data-dependent (e.g., the value of the data is used as a condition), and only branches to the target address when the condition is true. Unconditional branch instructions always branch to the target address. For example, in x86, instructions (such as CALL, RET, and JUMP) are examples of unconditional branches for subroutine calls, subroutine returns, and unconditional branches, respectively. Any control instruction other than a conditional branch instruction switches the execution sequence to the target address specified in the instruction. In this article, the target instruction sequence of control instructions is collectively referred to as control blocks. The execution of a program can be regarded as a sequence of executing certain control blocks. In this article, the execution order of control blocks in a program is referred to as control flow (i.e., flow of control). Conditional branches (e.g., JE) can generate multiple control flows in a program because each such branch is a fork, and the execution can go to any way on the fork based on the condition of the fork. Control instructions introduce significant additional complexity when extracting instruction streams because the correct extraction address after the control instruction cannot be calculated until the back end executes the control instruction itself. For this reason, the front end of a high-performance processor (specifically, BPU 230) predicts the next extraction address and speculatively extracts from the predicted address. There are two parts in this prediction. The first part is to predict the direction of the branch taken by the control instruction, that is, whether to take to the target sequence or not. The second part is to predict the target address of the branch. Once the direction of the branch is predicted, the memory address of the control instruction and its predicted target address are stored in a branch target buffer (BTB), a cache organized similarly to the set associative array described in the context of L1-IC 210.

[0049] ILD 240 provides a pre-decoding stage. ILD 240 separates the raw byte stream from IFU 220 into valid instruction sequences and passes them to IQ 250. For example, as noted above, the length of an x86 instruction can vary between 1B and 15B and can reside in any byte address in the program memory, so the raw byte stream needs to be split into variable-length instructions. Decoding the lengths of several instructions per cycle adds a degree of complexity because the starting address must be determined speculatively. That is, the fact that the starting address of the second instruction is not known before the length of the first instruction is calculated requires serialization of the length decoding process, and parallelization requires determining the length of each instruction before decoding the instruction. ILD 240 provides complex logic based on many parallel and speculative calculations to help achieve this parallelization (although this is at the expense of increased power consumption).

[0050] IQ 250 queues instructions for the instruction decode stage. After ILD 240 separates the instructions from the raw byte stream, IQ 250 queues the instructions for use by ID 260 in the instruction decode stage.

[0051] ID 260 provides the instruction decoding stage (it may also be referred to as the dynamic conversion stage). In this stage, instructions are read from IQ 250 and converted into the UOPs of subsequent functional equivalence. This conversion is performed by one of the several decoders in the set of decoder 261, and the set of decoder 261 includes complex decoder 261-C and three simple decoders 261-S1 to 261-S3 (although it should be understood that less or more instruction decoders may be used). In this article, the ID 260 including the set of decoder 261 may also be referred to as the microinstruction conversion engine (MITE). The resulting UOP is passed to IDQ 280 by ID 260, and then the UOP can enter the back end of processor 200 by IDQ 280. For example, in a processor based on x86, a simple instruction may be converted into one to four UOPs, and a complex instruction may be converted into five or more UOPs. It should be understood that for processors based on other ISAs, instructions may be converted into the UOPs of other quantities.

[0052] In general, UC 270 is a UOP cache that is configured to cache UOPs of instructions previously decoded by MITE, thereby avoiding the need for MITE to re-decode instructions previously decoded by MITE in order to obtain the associated UOP (i.e., avoiding L1-IC->IFU->ILD->IQ->MITE decoding path). This type of cache may be referred to as L0 instruction cache (L0-IC), which may store instruction blocks decoded into UOPs in units of UC lines. The UOP cache benefits from the temporal locality of control flow in the program, so previously executed instructions are executed again. Before extracting the instruction address from L1-IC 210, the instruction address is first looked up in L0-IC. If there is a corresponding UC line in L0-IC (meaning "hit"), the associated UOP is directly supplied to IDQ 280 for further execution, thereby completely avoiding the L1-IC->IFU->ILD->IQ->MITE decoding path. If there is no corresponding UC row in L0-IC (meaning "miss"), the instruction will go through the entire complex decode cycle through the L1-IC->IFU->ILD->IQ->MITE decode path. The ability to avoid the L1-IC->IFU->ILD->IQ->MITE decode path in this way provides a significant advantage because the decoding process from instructions to UOPs (especially for high-performance processors) can be expensive in terms of circuit systems, power consumption, and time, especially when a single complex instruction can perform several operations. It should be appreciated that since the back end of the processor can execute several UOPs per clock cycle (e.g., six UOPs per cycle), the rate at which UOPs are supplied from the front end of the processor 200 to the back end of the processor 200 is a key element of performance, which can be achieved through a high hit rate in the UC 270.

[0053] IDQ 280 queues UOPs to be provided to the back end of processor 200. UOPs queued by IDQ 280 may include UOPs decoded by ID 260 (MITE), and UOPs delivered from UC 270.

[0054] Although details are omitted for clarity, the back end of the processor 200 may include various elements, such as a reorder buffer (e.g., configured to receive UOPs from the front end of the processor 200), a unified reservation station having a set of ports configured to direct UOPs to ports of various element chains), various elements or element chains configured to support the execution of UOPs, etc., and various combinations thereof.

[0055] Figure 3 An example embodiment of a multi-core processor including multiple cores and multiple levels of cache is depicted.

[0056] The multi-core processor 300 includes four cores (denoted as core 0, core 1, core 2, and core 3), and three levels of cache (denoted using L1, L2, and L3 indicators). In the multi-core processor 300, each core includes a CPU (illustratively including a micro-operation cache (UC)), and L1 and L2 caches (illustratively including an L1 instruction cache, an L1 data cache, and an L2 cache). In the multi-core processor 300, the four cores share an L3 cache.

[0057] Generally, the cores are configured to operate as a processor (e.g., similar to the sole core of a single-core processor). It should be appreciated that each core has its own pipeline that independently fetches, decodes, and executes instructions (e.g., following Figure 1 The concept pipeline, which can be like Figure 2 Thus, in this document, the term "processor" may refer to a single core of a single-core processor, a core of a multi-core processor, or a combination of multiple cores of a multi-core processor.

[0058] In general, a cache is a smaller, faster memory, closer to the processor core, that stores copies of program instructions or program data from frequently used memory locations to reduce the average cost (e.g., time and / or energy) of operating the processor core. Program instructions or program data are stored in a cache by blocks of contiguous memory locations called cache lines, where each cache line is indexed in the cache by the first memory address in the cache line. Caches benefit from the temporal and spatial locality of memory access patterns in programs. Spatial locality refers to the use of relatively close memory locations (e.g., within a cache line). Temporal locality refers to the repeated use of a particular cache line for a relatively short duration.

[0059] In a multi-core processor, the levels of cache are typically arranged in a hierarchy as discussed below (although it should be appreciated that other arrangements are possible). The L1 cache and L2 cache are each specific to a processor core of the processor (i.e., each processor core has its own L1 cache and L2 cache associated with it), while the processor's L3 cache is common to all processor cores in the processor. For each processor core, the L1 cache is the smallest cache and is closest to the processor core and is therefore faster than the other cache levels. For each processor core, the L1 cache is split into two caches, as follows: an L1 instruction cache (e.g., 32KB in size, 64KB in size, or any other suitable size) that stores program instructions, and an L1 data cache (e.g., 32KB in size, 64KB in size, or any other suitable size) that stores program data. The L1 instruction cache may correspond to Figure 1 IC in. L2 cache (e.g., 256KB in size, 512KB in size, or any other suitable size) and L3 cache (e.g., 2MB in size, 4MB in size, or any other suitable size) are subsequent levels of cache, which are usually unified caches (meaning that the cache stores both program instructions and program data). For each processor core, the L2 cache is farther away from the processor core than the L1 cache. As noted above, the L3 cache of a processor is common to all processor cores in the processor. The size and access latency grow according to the level. If the cache line corresponding to the memory address being sought is missing in the L1 cache, the processor performs a search in the caches of subsequent levels. Only when the memory address is missing in all caches, the main memory is accessed. Ultimately, the missing block is read into the cache line in the L1 cache. The UC is located inside the core. It should be appreciated that the operation of the IC and UC in a processor can be further understood by first considering an example embodiment of the logical organization of the caches in the processor.

[0060] Figure 4 An example embodiment of an N-way set associative cache for use in a processor is depicted.

[0061] exist Figure 4 In the embodiment of the present invention, N-way set associative cache 400 can be an instruction cache or a data cache that stores memory blocks in cache lines. In this article, unless otherwise specified, the address or instruction or data in the memory is referred to as a pointer, and the first address stored in the cache line is referred to as the pointer of the cache line or the corresponding memory block.

[0062] like Figure 4As shown in , N-way set associative cache 400 includes two main building blocks: a data array 410 , and a tag array 420 .

[0063] The data array 410 stores cache lines, and the tag array 420 is used to match pointers to data array entries. The data array 410 is logically organized into groups of S sets. Each set consists of N cache lines. The number of cache lines in a set is called the "degree of associativity" of the cache. It should be noted that a cache with an associativity of N is an N-way associative cache, where each way is a cache line. A memory block is first mapped into a set Si through its pointer, and then placed into any cache line Nj in the set Si. In order to map a memory block into the N-way set associative cache 400, the pointer is partitioned into three fields, such as Figure 5 As shown in the figure.

[0064] Figure 5 An example embodiment of field partitioning of an address of a memory block in a memory is depicted. As previously noted, the term "pointer" may be used to refer to an address of a memory block in a memory. Figure 5 As illustrated in , in order to map a memory block into a cache, a pointer 500 is partitioned into the following fields: P-Tag, P-Index, and P-Offset. Here, the leading "P" is shorthand for "pointer".

[0065] The P-offset field (which may also be referred to as a block offset, or more generally as an offset) includes the K least significant bits of the pointer 500, which are used to identify which bytes within the cache line are to be accessed. Assuming that the size of the cache line block is Q bytes, then K in the P-offset field = log 2 (Q) bits. In this document, unless otherwise specified, these K bits are denoted as P-offset.

[0066] The P-index field (which may also be more generally referred to as an index) includes the next M least significant bits of the pointer 500, which are used to identify the set Si in the cache. For a cache consisting of S sets, M=log 2 (S) bits. In this document, unless otherwise specified, these M bits are denoted as P-index.

[0067] The P-Tag field includes the remaining bits of pointer 500. Herein, these bits are labeled as P-Tag unless otherwise noted.

[0068] Different memory blocks can map to the same set Si in the cache (they have the same P-index due to the M bits overlapping), so a mechanism is needed to map the P-index back to a pointer. The tag array is used for this purpose. The tag array has the same logical organization as the data array (same number of sets S and associativity N). For each cache line in the data array, the tag array holds some metadata: the P-tag bits and the state of the cache line (valid, etc.).

[0069] To look up a pointer, the P-index portion is used to access the set Si in the data array and the tag array, but in order to know if a cache line within that set corresponds to a given pointer, the P-tag bits must match to an entry in the set Si in the tag array. If the P-tag bits of the j-th entry in set Si match, then the correct data is located in the j-th cache line of the corresponding data array in set Si (this is called a "cache hit"). If no P-tag in set Si matches that in the tag array, then the requested cache line does not reside in the cache (this is a "cache miss"). In the case of a cache miss, a request may be made to a higher level of the memory hierarchy, and the processor will wait for the cache line to be installed in the cache before being able to continue accessing it.

[0070] As an example, consider an 8-way associative cache with 64 sets and a cache line size of 64B. Each cache line will then hold a memory block of size 64B. Here K=6, and M=6. If the processor attempts to access an instruction at IP 0xf045 (tag=0x1e, index=0x1, offset=0x5), the processor looks for a cache line in set 1 with tag 0x1e. If the cache line is found, the fifth byte in that cache line is retrieved.

[0071] Access to the tag array and data array can occur serially or in parallel. Figure 4 In , the entire set is read from the data array while accessing the tag array. This address is compared with the P-tag entry to determine in which cache line of the set the data that needs to be accessed resides. This information is fed to a multiplexer (way multiplexer) at the output of the data array, which selects one of the cache lines (or ways) of the set. Finally, the offset portion of the address is used to extract the appropriate bytes from the selected cache line (a process known as data alignment).

[0072] The number of bits in the P-Offset field determines the size of the cache line, i.e., the size of a cache line is log 2(Number of bits in the P-Offset field). A set in a cache is selected based on the P-index, and the cache lines within the set (i.e., the ways in the set) are marked with a P-tag. Figure 4 In the example of , the P-offset is 4 bits, the P-index is 4 bits, and the P-tag is 8 bits, so for the exemplary cache line, P-tag = 0xFF, and P-index = 0x0. Therefore, the cache line is marked as 0xFF in set 0. Obviously, all instructions with a cache line share the same P-tag and P-index.

[0073] Generally speaking, Figure 4 The design of the exemplary cache enables simple and efficient cache implementation in hardware, and therefore, the design is the basis of most caches in processors. For example, when a memory block is to be stored in an N-way set associative cache, the memory block is first mapped to a set Si based on the P-index bits of the pointer to the memory block. The operations can then be grouped into the following steps: (1) Look up each of the N entries (ways or cache lines) in the tag array Si to see if any entry is empty (unused); if an empty entry is found, the memory block is stored in that entry, otherwise the method proceeds to step (2); (2) if all N entries of the set Si are in use, a second pass through the entries in the tag array set Si is performed to select an entry for eviction; (3) the selected entry is evicted; and (4) the memory block is stored in the selected entry (meaning that the entry in the tag array stores the P-tag of the memory block, and the memory block is stored in the corresponding entry in the data array). Note that if none of the N entries in set Si is empty, then step (1) above is a significant waste of clock cycles and power, since it requires that each entry in the tag array Si be looked up, but none of them are ultimately found to be empty. Furthermore, this waste of cycles and power is multiplied for each miss as cache lines are constantly "thrashing" due to "conflict misses".

[0074] In general, when Q frequently accessed memory blocks are mapped to the same set Si and the cache associativity N is less than Q, the cache will suffer a conflict miss. In this case, one of the valid cache lines in the set Si needs to be evicted to accommodate the updated memory block. When the processor needs the evicted memory block again, it will be a miss and will need to be retrieved back to the cache. In order to make room for the memory block again, another cache line may need to be evicted, and the pattern continues. This pattern is called cache line jitter. For example, when the pool of packet buffers is recycled to process all incoming packets, jitter is frequently encountered in high-bandwidth packet processing in routers (e.g., each new packet always misses in the cache until the packet buffer is stored in the cache, and the set in the cache to which the packet buffer is mapped is likely to be full). This is again a significant waste of clock cycles and power, because this step requires that each entry in the tag array Si is checked, but it is eventually found that no entry is empty.

[0075] Various example embodiments may be configured to support more efficient operation of an N-way set associative cache based on use of occupancy state information. Various example embodiments for supporting operation of an N-way set associative cache based on occupancy state information may be configured to maintain occupancy state information for a set of the N-way set associative cache to support more efficient memory operations on the set of the N-way set associative cache.

[0076] Various example embodiments for supporting the operation of an N-way set associative cache based on occupancy state information may be configured to support memory operations for a memory block based on the occupancy state information by maintaining occupancy state information for a set, and performing memory operations for a memory block based on the occupancy state information. The memory operation may be a write operation or a read operation. The occupancy state information for the set may include various types of occupancy state information that may be used to support more efficient memory operations, which may vary for different memory operations. For example, the occupancy state information for the set may include information indicating whether the set is full (e.g., a full-bit that may be set in a manner indicating whether the set is full, an indication of the number of ways currently occupied in the set, or an indication of the number of ways currently empty in the set, etc., and various combinations thereof). For example, the occupancy state information for the set may include information indicating whether the set is empty (e.g., an empty bit that may be set in a manner indicating whether the set is empty, an indication of the number of ways currently empty in the set, an indication of the number of ways currently occupied in the set, etc., and various combinations thereof). It should be appreciated that various other types of occupancy status information may be supported to enable more efficient memory operations on an N-way set associative cache.

[0077] Various example embodiments for supporting operation of an N-way set associative cache based on occupancy status information may be configured to support write operations to a memory block based on the occupancy status information. The occupancy status information maintained by the N-way set associative cache of a set may indicate whether at least one of the ways of the set is empty, and may be used by the N-way set associative cache to support more efficient writing of the memory block to the set (e.g., by checking the occupancy status information before scanning each way of the set to find an empty cache line, so that an eviction process can be initiated when the set is full without having to first scan each way of the set to determine that the set is full). The occupancy status information for the set may be a full-bit (F-bit), which may be set in a manner indicating whether the set is full (e.g., F-bit=1) or not full (e.g., F-bit=0), although it should be understood that other types of occupancy status information may also or alternatively be maintained and used for the set of the N-way set associative cache (e.g., a counter indicating the number of ways that are full, a counter indicating the number of ways that are empty, etc., and various combinations thereof). The occupancy status information may be used to prevent a way-by-way search to find an empty way of the set for a write operation, wherein the occupancy status information of the set indicates that the set is full. It should be appreciated that at least some of these embodiments may be described with reference to Figures 6 to 9 can be further understood by examples.

[0078] Various example embodiments for supporting the operation of an N-way set associative cache based on occupancy status information may be configured to support read operations for a memory block based on the occupancy status information. The occupancy status information maintained by the N-way set associative cache of a set may indicate whether at least one of the ways of the set is occupied, and may be used by the N-way set associative cache to support more efficient reading of memory blocks from the set (e.g., by checking the occupancy status information before scanning the set to find ways that may be storing the memory block, thereby determining whether the set of the cache is empty, so that it is not necessary to search each way to determine whether the memory block is stored in the set). The occupancy status information for the set may be an empty bit (E-bit), which may be set in a manner indicating whether the set is empty (e.g., E-bit=1) or non-empty (e.g., E-bit=0), although it should be understood that other types of occupancy status information may also or alternatively be maintained and used for the set of the N-way set associative cache (e.g., a counter indicating the number of ways that are empty, a counter indicating the number of ways that are occupied, etc., and various combinations thereof). The occupancy status information may be used to prevent a path-by-path search to find a data element stored in the set for a read operation, wherein the occupancy status information of the set indicates that the set is empty. It should be appreciated that at least some of these embodiments may be described by reference to Figures 10 to 13 examples to further understand.

[0079] It should be understood that by referring to Figures 14 to 20 With reference to the example embodiments of the present invention, it may be further appreciated that occupancy status information may be used to support more efficient memory operations of an N-way set associative cache.

[0080] Figure 6 An example embodiment of a P-tag, P-index, and P-offset mapping of an address of a memory block to be stored in a cache based on a write operation is depicted. In this example, assume that a 16-bit memory block with an address of 1011011010010101 needs to be stored in an 8-way set associative cache (N=8). In addition, assume that the size of the cache line is 64B and there are 32 sets in the 8-way set associative cache. Note that the bit positions are from right to left, starting from zero to higher. In addition, assume that the P-tag, P-index, and P-offset mapping of the address of the memory block is as follows: Figure 6 As shown in Figure 6 As shown in the memory address map 600 of FIG. 1 , bits 0 to 5 (P-offset) are used to index the offset into the 64B cache line, bits 6 to 10 (P-index) are used to index one of the 32 sets in the cache, and bits 11 to 15 are used as P-tags. The set is determined by P-index bits 11010, which maps to set 26. Storing a memory block into an 8-way set associative cache is for situations where an 8-way set associative cache does not maintain occupancy status information (such as information about Figure 7 ), and for the case where occupancy status information is maintained for an 8-way set associative cache (such as Figure 8 presented).

[0081] Figure 7 An example embodiment of a set associative cache prior to storing a memory block that does not support occupancy state information is depicted. Figure 7 In the set associative cache 700 of FIG. 7 , only a few sets are shown (for clarity), and “T” means that the tag bits from the address of the memory block are used for the index mode. Figure 6As illustrated in the memory address map 600 of FIG. 1 , bits 0 to 5 (P-offset) are used to index an offset in a 64B cache line, bits 6 to 10 (P-index) are used to index one of the 32 sets in the cache, and bits 11 to 15 are used as P-tags, thus determining that P-index bits 11010 map to set 26, such that set associative cache 700 will store the memory block in one of the ways of set 26. In set associative cache 700 (without occupancy status information), set associative cache 700 will have to scan each of the eight ways of set 26 to determine that none of the eight ways are empty, and therefore set associative cache 700 will have to evict one of the cache lines from one of the ways of set 26 to make room to store the memory block.

[0082] Figure 8 An example embodiment of a set associative cache prior to storing a memory block supporting occupancy state information is depicted. Figure 8 In the set associative cache 800 of FIG. 8 , only a few sets are shown (for clarity), and “T” means that the tag bits from the address of the memory block are used for the index mode. Figure 6 As shown in the memory address map 600 of FIG. 8 , bits 0 to 5 (P-offset) are used to index the offset in the 64B cache line, bits 6 to 10 (P-index) are used to index one of the 32 sets in the cache, and bits 11 to 15 are used as P-tags, so it is determined that P-index bits 11010 map to set 26, so that the set associative cache 800 will store the memory block in one of the ways of set 26. Figure 8 As depicted in , the set associative cache 800 includes occupancy status information 810 in the form of a full-bit (F-bit) array maintained for each set, wherein the F-bit maintained for a given set indicates whether the given set is full (i.e., whether all ways of the given set are full). Figure 8In the example of , the F-bits for sets 1 and 22 are set to "0" because these sets are not full (i.e., at least one way of each set is empty), and the F-bit for set 26 is set to "1" because this set is full (i.e., each way of the set is occupied). In the set associative cache 800, using the occupancy status information 810, the set associative cache 800 will not have to scan each of the eight ways of set 26 in order to determine that set 26 is full; instead, the set associative cache 800 can initiate an eviction process after determining that set 26 is full using the occupancy status information 810 (i.e., the F-bit for set 26 = 1) (thereby saving clock cycles and processor resources). The set associative cache 800 will evict one of the cache lines from one of the ways of set 26 based on the eviction process to make room for storing a memory block. It should be appreciated that various types of logic circuits can be used to determine the F-bit for each set, examples of which are related to Fig. 9 is presented.

[0083] Fig. 9 An example embodiment of a logic circuit configured to determine a full-bit to be used as occupancy status information for a write operation is depicted. Fig. 9 As depicted in , logic circuit 900 may be used for a particular set of set associative caches to automatically turn off and on the F-bit based on the empty-bit from each cache line in the set. Fig. 9 , the empty-bit from each cache line in the set can be input to a NOR gate such that the F-bit is set to OFF whenever at least one of the empty-bits of the cache line is set to true (i.e., empty-bit=1), and the F-bit is set to ON when all of the empty-bits of the cache line are set to false (i.e., empty-bit=0). It should be appreciated that logic circuit 900 is merely one implementation of a logic circuit system for setting the F-bit for a set. For example, in at least some example embodiments (which are omitted for clarity), the F-bit for the set can be automatically turned off and on by inputting the full-bit from each cache line in the set to an AND gate (i.e., when all cache lines are full, the set is indicated as full and the F-bit=1).

[0084] Fig.10An example embodiment of a P-tag, P-index, and P-offset mapping of an address of a memory block read from a cache based on a read operation is depicted. In this example, assume that a 16-bit memory block with an address of 1011011100010101 needs to be read from an 8-way set associative cache (N=8). In addition, assume that the size of the cache line is 64B and there are 32 sets in the 8-way set associative cache. Note that the bit positions are from right to left, starting from zero to higher. In addition, assume that the P-tag, P-index, and P-offset mapping of the address of the memory block is as follows: Fig.10 As shown in Fig.10 As shown in the memory address map 1000 of FIG. 1 , bits 0 to 5 (P-offset) are used to index the offset into the 64B cache line, bits 6 to 10 (P-index) are used to index one of the 32 sets in the cache, and bits 11 to 15 are used as P-tags. The set is determined by P-index bits 11100 which map to set 28. Reading a memory block from an 8-way set associative cache is for the case where the 8-way set associative cache does not maintain occupancy status information (such as information about Fig.11 ), and for the case where occupancy status information is maintained for an 8-way set associative cache (such as Fig.12 presented).

[0085] Fig.11 An example embodiment of a set associative cache prior to reading a memory block that does not support occupancy status information is depicted. Fig.11 In the set associative cache 1100, only a few sets are shown (for clarity), and "T" refers to the tag bit of the address from the memory block. Fig.10 As shown in the memory address map 1000 of , bits 0 to 5 (P-offset) are used to index the offset in the 64B cache line, bits 6 to 10 (P-index) are used to index one of the 32 sets in the cache, and bits 11 to 15 are used as P-tags, so it is determined that P-index bits 11100 map to set 28, so that the set associative cache 1100 will attempt to read the memory block from one of the ways of set 28. In the absence of occupancy status information, the set associative cache 1100 will have to scan each of the eight ways of set 28 to determine that none of the eight ways are full, and therefore the memory block cannot be accessed from any way of set 28, and the set associative cache 1100 will need to obtain the memory block from the next level of the memory hierarchy.

[0086] Fig.12 An example embodiment of a set associative cache prior to reading a memory block supporting occupancy status information is depicted. Fig.12 In the set associative cache 1200, only a few sets are shown (for clarity), and "T" refers to the tag bit of the address from the memory block. Fig.10 As shown in the memory address map 1000 of FIG. 1 , bits 0 to 5 (P-offset) are used to index the offset in the 64B cache line, bits 6 to 10 (P-index) are used to index one of the 32 sets in the cache, and bits 11 to 15 are used as P-tags, so it is determined that P-index bits 11100 map to set 28, so that the set associative cache 1200 will attempt to read a memory block from one of the ways in set 28. Fig.12 As depicted in , the set associative cache 1200 includes occupancy status information 1210 in the form of an array of empty bits (E-bits) maintained for each set, wherein the E-bit maintained for a given set indicates whether the given set is empty (i.e., whether all ways of the given set are empty). Fig.12 In the example, the E-bits for sets 1 and 22 are set to "0" because these sets are not empty (i.e., at least one way of each set is full), and the E-bit for set 29 is set to "1" because this set is empty (i.e., all ways of the set are not occupied). In the set associative cache 1200, using the occupancy status information 1210, the set associative cache 1200 will not have to scan each of the eight ways of set 26 in order to determine that set 28 is empty; instead, the set associative cache 1200 can initiate the retrieval of a memory block from the next level of the memory hierarchy after determining that set 28 is full using the occupancy status information 1210 (i.e., the E-bit for set 28 = 1) (thereby saving clock cycles and processor resources). It should be appreciated that various types of logic circuits can be used to determine the E-bits for each set, examples of which are with respect to Fig.13 is presented.

[0087] Fig.13 An example embodiment of a logic circuit configured to determine an empty bit to be used as occupied status information for a read operation is depicted. Fig.13 As depicted in , logic circuit 1300 may be used for a particular set of set associative caches to automatically turn off and on the E-bit based on the empty-bit from each cache line in the set. Fig.13, the empty-bit from each cache line in the set can be input to an AND gate such that the E-bit is set to OFF as long as at least one of the empty-bits of the cache line is set to false (i.e., empty-bit=0), and the E-bit is set to ON when all the empty-bits of the cache line are set to true (i.e., empty-bit=1). It should be understood that logic circuit 1300 is merely one implementation of a logic circuit system for setting the E-bit for a set. For example, in at least some example embodiments (which are omitted for clarity), the E-bit of the set can be automatically turned off and on by inputting the full-bit from each cache line in the set to a NOR gate (i.e., when all cache lines are empty, the set is indicated as empty and the E-bit=1).

[0088] It should be understood that although the main focus is on the occupancy status information including full-bit (such as Figures 6 to 9 and is presented) or empty bits (such as Figures 10 to 13 The example embodiments presented are presented in conjunction with the example embodiments of the present invention, but various combinations of such occupancy state information may be maintained and used to support, various other types of occupancy state information may be maintained and used to support memory operations, etc., and various combinations thereof. Fig.14 FIG. 1 is a diagram showing an exemplary embodiment of the present invention.

[0089] Fig.14 An example embodiment of a set associative cache including occupancy state information is depicted.

[0090] like Fig.14 As depicted in Fig.14 In the set associative cache 1400, only a few sets are shown (for clarity), and "T" means that the tag bits from the address of the memory block are used for the index mode. It should be noted that for clarity, the set associative cache 1400 is assumed to be a 4-way set associative cache. It should be understood that more generally, an N-way set associative cache may include fewer or more ways, and may include fewer or more sets. The intersection of sets and ways represents cache lines that are configured to store memory blocks, and cache lines that may currently be storing memory blocks (as indicated by "T=", and therefore can be considered full) or may not currently be storing memory blocks (as indicated by "EMPTY", and therefore can be considered empty).

[0091] like Fig.14As depicted in , the set associative cache 1400 includes occupancy state information 1410, which, for each set, includes corresponding occupancy state information for the corresponding set (for a given set, it may also be referred to as set occupancy state information for the set). The occupancy state information maintained for a set indicates the occupancy of the set and may include one or more types of occupancy state information. For example, the occupancy state information maintained for a set may include one or more of the following items: a full-bit (F-bit) indicating whether the set is full (i.e., all ways are full), an empty bit (E-bit) indicating whether the set is empty (i.e., all ways of the set are empty), a full counter (FC) value indicating the number of occupied ways in the set, an empty counter (EC) value indicating the number of empty ways in the set, a utilization counter (UC) value that may indicate the number of empty ways in the set or the number of full ways in the set, etc., and various combinations thereof). It should be appreciated that the set associative cache 1400 may maintain various other types of occupancy state information.

[0092] like Fig.14 As depicted in , for clarity, the set associative cache 1400 is depicted as maintaining occupancy status information, which includes an F-bit, an E-bit, and a UC value indicating the number of ways occupied. As illustrated with respect to set 1, two of the four ways are occupied, so the E-bit is set to zero, the F-bit is set to zero, and the UC value is set to 2, because two of the ways are storing memory blocks. As illustrated with respect to set 22, all four ways are occupied, so the E-bit is set to 0, the F-bit is set to 1, and the UC value is set to 4, because all four ways are storing memory blocks. As illustrated with respect to set 28, all four ways are empty, so the E-bit is set to 1, the F-bit is set to 0, and the UC value is set to 0, because no way is storing memory blocks.

[0093] It should be appreciated that although primarily presented with respect to example embodiments in which the same type of occupancy status information is maintained for each set, in at least some example embodiments, different types of set occupancy status information may be maintained for different sets.

[0094] Fig.15 An example embodiment configured to support an N-way set associative cache using occupancy state information is depicted.

[0095] like Fig.15As illustrated in , the N-way set associative cache 1500 includes two main building blocks: a data array 1510 and a tag array 1520. The N-way set associative cache 1500 also includes an occupancy status information (OSI) array 1530, which is configured to maintain occupancy status information that can be used to improve memory operations performed on the N-way set associative cache 1500.

[0096] In an N-way set associative cache 1500, a data array 1510 stores cache lines, and a tag array 1520 is used to match pointers to data array entries. The data array 410 is logically organized into groups of S sets. Each set consists of N cache lines. The number of cache lines in a set is called the "degree of associativity" of the cache. It should be noted that a cache with an associativity of N is an N-way associative cache, where each way is a cache line. A memory block is first mapped into a set Si via its pointer, and then placed into any cache line Nj in the set Si.

[0097] In the N-way set associative cache 1500, to look up a pointer, the P-index portion is used to access a set Si in both the data array 1510 and the tag array 1520. Selection of Si is accomplished by the address decoder circuitry, which activates access to Si. The OSI array 1530 maintains occupancy status information for each set (e.g., one or more of the F-bit, the E-bit, the occupancy counter value, etc., and various combinations thereof). For a memory operation to be performed on a set, before scanning the cache lines of the set to perform the memory operation, the OSI array 1530 is accessed to determine the occupancy status information for the set, because the occupancy status information may provide an indication that the cache lines of the set do not need to be scanned (e.g., initiating an eviction operation without scanning the cache lines of the set during a write operation if the occupancy status information for the set indicates that the set is full; or initiating a retrieval operation at the next level of the memory hierarchy during a read operation without scanning the cache lines of the set, where the occupancy status information for the set indicates that the set is empty).

[0098] Fig.16 An example embodiment of a method for storing a memory block in a set associative cache supporting occupancy status information is depicted. It should be appreciated that although primarily presented herein as being performed serially, at least a portion of the functionality of method 1600 may be performed simultaneously or in parallel. Fig.16 It should be appreciated that although primarily presented with respect to an example embodiment in which the occupancy status information is a full-bit (F-bit) indicating whether a set is full, various other types of occupancy status information may be present in Fig.16 1600 is used in the context of method 1600 of . At block 1601, method 1600 starts. At block 1610, a memory block to be written to the cache is obtained, and then method 1600 proceeds to block 1620. At block 1620, a set of memory blocks is determined based on the memory block addresses of the memory blocks, and then method 1600 proceeds to block 1630. At block 1630, the F-bit of the set is read to determine whether the set is full, and then method 1600 proceeds to block 1640. At block 1640, a determination is made as to whether the F-bit indicates that the set is full. If the F-bit indicates that the set is not full, method 1600 proceeds to block 1650. If the F-bit indicates that the set is full, method 1600 proceeds to block 1660. At block 1650, a cache line in the set is selected, and then method 1600 proceeds to block 1670. At block 1660, the cache line is evicted from the set, and the method 1600 proceeds to block 1670. At block 1670, the memory block is stored in the cache line (i.e., the empty cache line from block 1650, or the evicted cache line from block 1660). At block 1680, the tag portion of the memory block address is stored in the cache line. At block 1690, the empty bit of the cache line is set to zero (and the F-bit for the set will be automatically set based on the empty bit of the cache line of the set). At block 1699, the method 1600 ends.

[0099] Fig.17 Depicted with Fig.16 An example embodiment of a method for finding an empty cache line in a set associative cache used in conjunction with a method of . It should be understood that although primarily presented herein as being performed serially, at least a portion of the functionality of method 1700 may be performed simultaneously or in a sequential manner. Fig.17 It should be understood that Fig.17 The method 1700 may be used to provide Fig.16At block 1701, method 1700 begins. At block 1710, a set in the cache is selected, and method 1700 proceeds to block 1720. At block 1720, the first cache line in the set is selected, and method 1700 proceeds to block 1730. At block 1730, a determination is made as to whether the selected cache line of the selected set is empty. If the selected cache line of the selected set is empty, method 1700 proceeds to block 1740. If the selected cache line of the selected set is not empty, method 1700 proceeds to block 1750. At block 1740, the cache line is returned as empty, and method 1700 proceeds to block 1799, where method 1700 ends. At block 1750, it is determined whether more cache lines are available (not yet checked to see if they are candidates for storing the memory block) because the currently selected cache line is not empty and therefore cannot be used to store the memory block. If more cache lines are available in the selected set, the method 1700 proceeds to block 1760. If no more cache lines are available in the selected set, the method 1700 proceeds to block 1770. At block 1760, the next cache line in the set is selected (checked to see if the cache line is empty), and then the method 1700 returns to block 1730 (to see if the newly selected cache line is empty). At block 1770, a statement is made that there are no empty cache lines in the set, and then the method 1700 proceeds to block 1799, where the method 1700 ends. At block 1799, the method 1700 ends.

[0100] Fig.18 An example embodiment of a method for reading a memory block from a set associative cache supporting occupancy state information is depicted. It should be appreciated that although primarily presented herein as being performed serially, at least a portion of the functionality of method 1800 may be performed simultaneously or in parallel. Fig.18 It should be appreciated that although primarily presented with respect to an example embodiment in which the occupancy status information is an empty bit (E-bit) indicating whether a set is empty, various other types of occupancy status information may be present in Fig.181800 is used in the context of method 1800 of . At block 1801, method 1800 starts. At block 1810, a memory block address of a memory block to be read is obtained, and then method 1800 proceeds to block 1820. At block 1820, a set of memory blocks is determined based on the memory block address of the memory block, and then method 1800 proceeds to block 1830. At block 1830, the E-bit of the set is read to determine whether the set is empty, and then method 1800 proceeds to block 1840. At block 1840, a determination is made as to whether the E-bit indicates that the set is empty. If the E-bit indicates that the set is not empty, method 1800 proceeds to block 1850. If the E-bit indicates that the set is empty, method 1800 proceeds to block 1890. At block 1850, the cache lines of the set are searched to determine whether a matching cache line is stored in the set (i.e., whether the memory block is stored in the cache lines of the set). At block 1860, a determination is made as to whether a matching cache line has been identified in the set based on the search of the set. If a matching cache line has been identified in the set, the method 1800 proceeds to block 1870. If a matching cache line has not been identified in the set, the method 1800 proceeds to block 1880. At block 1870, a hit is declared for the memory block in the cache, and the method 1800 proceeds to block 1899, where the method 1800 ends. At block 1880, a miss is declared for the memory block in the cache, and the method 1800 proceeds to block 1899, where the method 1800 ends. At block 1890, a lookup of a memory block in the next level of the memory hierarchy is initiated based on a determination that the set is empty (without searching every cache line of the set), and the method 1800 proceeds to block 1899, where the method 1800 ends.

[0101] Fig.19 Depicted with Fig.18 An example embodiment of a method for finding an empty cache line in a set associative cache used in conjunction with a method of . It should be understood that although primarily presented herein as being performed serially, at least a portion of the functionality of method 1900 may be performed simultaneously or in a sequential manner. Fig.19 It should be understood that Fig.19 The method 1900 may be used to provide Fig.18At block 1901, method 1900 begins. At block 1910, a set to be searched and a tag of a memory block address to be searched in the identification to be searched are identified, and then method 1900 proceeds to block 1920. At block 1920, a first cache line in the identification to be searched is selected, and then method 1900 proceeds to block 1930. At block 1930, a determination is made as to whether the selected cache line is empty. If the selected cache line is not empty, method 1900 proceeds to block 1940. If the selected cache line is empty, method 1900 proceeds to block 1960. At block 1940, a determination is made as to whether a tag from the memory address matches a tag of the selected cache line. If the tag from the memory address matches a tag of the selected cache line, method 1900 proceeds to block 1950. If the tag from the memory address does not match the tag of the selected cache line, the method 1900 proceeds to block 1960. At block 1950, a hit is declared for the memory block in the selected cache line (because a matching tag is identified), and the method 1900 then proceeds to block 1999, where the method 1900 ends. At block 1960, it is determined whether there are more cache lines in the set (that can be used to search for the memory block). If there are more cache lines in the set, the method 1900 proceeds to block 1970. If there are no more cache lines in the set, the method 1900 proceeds to block 1980. At block 1970, the next cache line in the set is selected (for searching for the memory block), and the method 1900 then returns to block 1930. At block 1980, a miss is declared for the memory block in the set (because all cache lines of the set have been searched without identifying a matching tag), and the method 1900 then proceeds to block 1999, where the method 1900 ends. At block 1999, the method 1900 ends.

[0102] Fig. 20 An example embodiment of a method for operating a cache based on the use of occupancy status information is depicted. It should be appreciated that although primarily presented herein as being performed serially, at least a portion of the functionality of method 2000 may be performed simultaneously or in parallel. Fig. 20 At block 2001, method 2000 begins. At block 2010, for a cache including a set with multiple ways, occupancy status information is maintained, the occupancy status information indicating occupancy of the set. At block 2020, memory operations are performed based on the occupancy status information. At block 2099, method 2000 ends.

[0103] Various example embodiments of supporting the operation of a processor cache based on occupancy status information as presented herein may be configured to be applied to various types of caches that may be implemented within a processor or otherwise operated in association with a processor, such as an instruction cache (IC) of a processor, a micro-operation cache (UC) of a processor, a data cache (DC) of a processor, a unified cache of a processor that may host both instructions and data, a branch target buffer (BTB) associated with a branch predictor of a processor, and the like.

[0104] The various example embodiments for supporting operation of a processor cache based on occupancy status information as presented herein may be configured for use within various types of processors that may utilize a cache, such as a complex instruction set computer (CISC) processor, a reduced instruction set computer (RISC) processor, or any other type of processor or other device that may utilize a cache.

[0105] Various example embodiments for supporting operation of a processor cache based on occupancy status information may provide various advantages or potential advantages. For example, various example embodiments for supporting operation of a processor cache based on occupancy status information may be configured to support more efficient operation of a set associative cache by using occupancy status information to avoid the need to search cache lines of a set of a set associative cache under certain conditions. A set associative cache (e.g., where during a write operation, the occupancy status information is used to determine that the set is full; or where during a read operation, the occupancy status information is used to determine that the set is empty).

[0106] Various example embodiments for supporting operation of a processor cache based on occupancy status information may provide various other advantages or potential advantages.

[0107] Fig.21 An example embodiment of a computer suitable for use in performing the various functions presented herein is depicted.

[0108] The computer 2100 includes a processor 2102 (e.g., a central processing unit (CPU), a processor, a processor core of a processor, a subset of a processor core of a processor, a collection of processor cores of a processor, etc.) and a memory 2104 (e.g., a random access memory (RAM), a read-only memory (ROM), etc.). In at least some example embodiments, the computer 2100 may include: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the computer to perform various functions presented herein.

[0109] The computer 2100 may also include a cooperating element 2105. The cooperating element 2105 may be a hardware device. The cooperating element 2105 may be a process that may be loaded into the memory 2104 and executed by the processor 2102 to implement the various functions presented herein (in this case, for example, the cooperating element 2105 (including associated data structures) may be stored on a non-transitory computer-readable medium, such as a storage device, or other suitable type of storage element (e.g., a magnetic drive, an optical drive, etc.)).

[0110] The computer 2100 may also include one or more input / output devices 2106. The input / output devices 2106 may include user input devices (e.g., keyboard, keypad, mouse, microphone, camera, etc.), user output devices (e.g., display, speaker, etc.), one or more network communication devices or elements (e.g., input port, output port, receiver, transmitter, transceiver, etc.), one or more storage devices (e.g., tape drive, floppy disk drive, hard disk drive, optical disk drive, etc.), etc., and various combinations thereof.

[0111] It should be appreciated that the computer 2100 may represent a general architecture and functionality suitable for implementing the functional elements described herein, portions of the functional elements described herein, etc., and various combinations thereof. For example, the computer 2100 may provide a general architecture and functionality suitable for implementing one or more elements presented herein.

[0112] It should be appreciated that the computer 2100 may also represent the general architecture and functionality in which various example embodiments may be implemented (eg, where the processor 2102 includes a set associative cache configured to support the use of occupancy state information).

[0113] It should be appreciated that at least some of the functionality presented herein may be implemented in software (e.g., via implementation of software on one or more processors for execution on a general-purpose computer (e.g., by execution by one or more processors to provide a special-purpose computer, etc.) and / or may be implemented in hardware (e.g., using a general-purpose computer, one or more application-specific integrated circuits, and / or any other hardware equivalents).

[0114] It should be understood that at least some of the functions presented herein can be implemented in hardware, for example, as circuit systems that cooperate with a processor to perform various functions. Portions of the functions / elements described herein can be implemented as computer program products, wherein computer instructions adjust the operation of the computer when processed by the computer so that the methods and / or techniques described herein are called or otherwise provided. Instructions for calling various methods can be stored in fixed or removable media (e.g., non-transient computer-readable media), sent via a data stream in a broadcast or other signal-bearing medium, and / or stored in a memory within a computing device that operates according to the instructions.

[0115] It should be understood that the term "non-transitory" as used herein is a limitation on the medium itself (ie, tangible, not a signal), and not on data storage persistence (eg, RAM vs. ROM).

[0116] It should be understood that, as used herein, “at least one of ” and “at least one of: ” and similar expressions (wherein a list of two or more elements is connected by “and” or “or”) mean at least any one element, or at least any two or more elements, or at least all elements.

[0117] It should be understood that, as used herein, the term "or" refers to a non-exclusive "or" unless otherwise indicated (eg, with use of "or . . . . ," or "or in the alternative").

[0118] It will be appreciated that although various embodiments incorporating the teachings presented herein have been shown and described in detail, those skilled in the art can readily devise many other varied embodiments that still incorporate these teachings.

Claims

1. A memory device, include: A cache comprising a set having multiple ways, wherein the cache is configured to: maintaining occupancy status information for the set, the occupancy status information indicating occupancy of the set; and Based on the occupancy status information, a memory operation is performed.

2. The device according to claim 1, wherein the occupancy status information include: At least one of information indicating whether the set is full, or information indicating whether the set is empty.

3. The apparatus according to claim 1, wherein the occupancy status information comprises at least one of the following items: a full bit indicating whether the set is full, an empty bit indicating whether the set is empty, a value indicating the number of full paths in the set, or a value indicating the number of empty paths in the set.

4. An apparatus according to any one of claims 1 to 3, wherein the memory operation is a write operation, and wherein the cache is configured to: based on determining that the occupancy status information indicates that the set is full, initiate an eviction process for the set without scanning the multiple ways of the set to find an empty cache line in which a memory block is stored.

5. An apparatus according to any one of claims 1 to 3, wherein the memory operation is a read operation, and wherein the cache is configured to: based on determining that the occupancy status information indicates that the set is empty, select a next set of the cache to search for a memory block without scanning the multiple ways of the set to find the memory block.

6. The apparatus according to any one of claims 1 to 3, wherein the cache include: A data array, a tag array, and an occupancy status information array, wherein the occupancy status information is maintained in the occupancy status information array.

7. The device according to any one of claims 1 to 3, wherein the occupancy status information include: bits indicating occupancy of the set, wherein the bits indicating occupancy of the set are determined based on a corresponding plurality of occupancy status bits for the ways of the set. 8 . The apparatus of claim 7 , wherein the occupancy status bits are stored in corresponding multiple entries of a tag array of the cache, wherein the occupancy status bits indicate whether a corresponding way is empty or whether a corresponding way is full.

9. The apparatus according to any one of claims 1 to 3, wherein the memory operation is a write operation to a memory block.

10. The device according to claim 9, in, To perform the write operation to the memory block, the cache is configured to: determining, based on a memory block address of the memory block, that the set is a default set for the memory block; Based on the occupancy status information, determining whether the set is full; as well as Based on whether the set is full, the write operation for the memory block is performed.

11. The device according to claim 9, in, To perform the write operation to the memory block, the cache is configured to: Based on the occupancy status information, determining that the set is not full; selecting from the set a cache line that is empty in the set; and The memory block is stored in the cache line that is empty in the set.

12. The device according to claim 9, in, To perform the write operation to the memory block, the cache is configured to: Based on the occupancy status information, determining that the set is full; evicting a cache line of the set from the set; as well as The memory block is stored in the cache line of the set.

13. The apparatus according to any one of claims 1 to 3, wherein the memory operation is a read operation on a memory block.

14. The device according to claim 13, in, To perform the read operation for the memory block, the cache is configured to: determining, based on a memory block address of the memory block, that the set is a default set for the memory block; Based on the occupancy status information, determining whether the set is empty; as well as Based on whether the set is empty, the read operation for the memory block is performed.

15. A method for storing a memory, include: For a cache including a set having multiple ways, maintaining occupancy status information, the occupancy status information indicating occupancy of the set; as well as Based on the occupancy status information, a memory operation is performed.