Store coherence verification related to disabling page transformations

Abstract

A system and method for invalidating page translation entries are described. A processing element may apply a delay to a drain cycle of a store order queue (SRQ) of the processing element. The processing element may drain the SRQ under the delayed drain cycle. The processing element may receive a translation lookaside buffer invalidate (TLBI) instruction from an interconnect connecting the multiple processing elements. The TLBI instruction may be an instruction to invalidate a translation lookaside buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame. The TLBI instruction may be broadcast by another processing element. Applying a delay to the drain cycle of the SRQ may reduce a difference between the drain cycle of the SRQ and the invalidation cycle associated with the TLBI.

Description

Technical Field

[0001] The present invention relates to execution in a processor, more specifically to data processing, and even more specifically to cache coherence and page translation invalidation in a multi-core processor, a microprocessor, or a multi-processor system.

Background Art

[0002] In one example, a computing system can access an address in physical memory using virtual memory without having to consider the exact location of the address in the physical memory. The mapping between the virtual memory address and the physical memory address can be maintained and stored as a page table. In one example, when a program accesses a virtual memory address, address translation using the page table is performed to identify the physical memory address referred to by the accessed virtual memory address. Then, the data stored at this identified physical memory address can be read from the physical memory address.

[0003] In a multiprocessor system containing multiple processing elements (e.g., a system with multiple processors or a processor with multiple cores), all cores can share a page table. To improve the efficiency of accessing translations within the page table, each processing element (processor or core) may maintain its own translation lookaside buffer (TLB). Each TLB may be a cache representing a portion of the page table. A TLB may contain multiple entries from the page table, and each TLB entry may contain a mapping from a virtual address to a physical address. In one example, the TLB entries may be maintained such that a portion of the total available memory covered by the TLB includes the most recently accessed portion, the most commonly accessed portion, or the most likely to be accessed portion of the total available memory. When data is moved in or out of physical memory (e.g., a new process is started or a context switch is performed), the entries in the TLB must be updated to reflect the presence of the new data, and TLB entries related to data removed from system memory must be invalidated. Since each core maintains its own TLB, cores must communicate with each other to maintain cache coherency. [Overview of the project]

[0004] This summary of the disclosure is provided to aid in understanding computer processing systems and methods that disable page conversion entries and maintain cache coherence, and is not intended to limit the disclosure or the invention. This disclosure is directed to those skilled in the art. It should be understood that various aspects and features of this disclosure may be used advantageously, in some cases alone, and in other cases in combination with other aspects and features of this disclosure. Accordingly, modifications and changes may be made to the memory system, architectural structure, and operating method to achieve different effects.

[0005] In several examples, methods for invalidating page conversion entries in a data processing system are generally described. The data processing system may include multiple processing elements. The method may include applying a delay to the drain cycle of the processing element's store reorder queue (SRQ). The method may further include draining the SRQ under the delayed drain cycle. The method may further include receiving a translation lookaside buffer invalidation (TLBI) instruction from an interconnect connecting the multiple processing elements. The TLBI instruction may be an instruction that invalidates a translation lookaside buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame. The TLBI instruction may be broadcast by another processing element. By applying a delay to the SRQ drain cycle, the difference between the SRQ drain cycle and the invalidation cycle associated with the TLBI may be reduced.

[0006] In some examples, a computing system configured to invalidate page translation entries in a data processing system is commonly described. The computing system may include a first processing element, a second processing element, and an interconnect connected to the first and second processing elements. The first processing element may be configured to broadcast a Translation Lookaside Buffer Invalidation (TLBI) instruction over the interconnect. The TLBI instruction may be an instruction that invalidates a Translation Lookaside Buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame. The second processing element may be configured to apply a delay to the drain cycle of the second processing element's Store Reorder Queue (SRQ). The second processing element may be further configured to drain the SRQ under the delayed drain cycle. The second processing element may be further configured to receive a TLBI instruction from the interconnect. By applying a delay to the SRQ drain cycle, the difference between the SRQ drain cycle and the invalidation cycle associated with the TLBI is reduced.

[0007] In some examples, a processing element configured to invalidate page conversion entries in a data processing system is commonly described. The processing element may include a processor pipeline having one or more load / store units (LSUs) configured to execute load and store instructions. One or more LSUs may be configured to apply a delay to the drain cycle of the processing element's store reorder queue (SRQ). One or more LSUs may be further configured to drain the SRQ under the delayed drain cycle. One or more LSUs may be configured to receive translation lookaside buffer invalidation (TLBI) instructions from an interconnect connecting multiple processing elements. A TLBI instruction may be an instruction that invalidates a translation lookaside buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame. A TLBI instruction is broadcast by another processing element. By applying a delay to the SRQ drain cycle, the difference between the SRQ drain cycle and the invalidation cycle associated with the TLBI is reduced.

[0008] Further features and the structure and operation of various embodiments will be described in detail below with reference to the attached drawings. [Brief explanation of the drawing]

[0009] Various aspects, features, and embodiments of processors, processor systems, or methods or combinations thereof for processing data will be better understood in conjunction with the accompanying drawings. Each embodiment is provided in the drawings for illustrative purposes of aspects, features, or various embodiments or combinations thereof of processors, processor systems, and methods for managing and processing data, but the claims should not be limited to these illustrated exact systems, embodiments, methods, processes, or devices or combinations thereof, and these illustrated features or processes or both may be used alone or in combination with other features or processes or both. Elements with reference numerals are numbered according to the drawing in which they are introduced and will generally (but not necessarily) be referenced by the same number in subsequent drawings. Similar reference numerals in the drawings generally (but not necessarily) represent similar parts in exemplary embodiments of the present invention. [Figure 1] This figure shows a general computing system or data processing system according to one embodiment. [Figure 2] This is a block diagram of a processor according to one embodiment. [Figure 3] This figure shows an example of an implementation of store coherence verification related to the invalidation of page conversion in one embodiment. [Figure 4] This figure shows an example of another implementation of store coherence verification related to page conversion invalidation in one embodiment. [Figure 5] This figure shows an example of another implementation of store coherence verification related to page conversion invalidation in one embodiment. [Figure 6] This figure shows an example flowchart of store coherence verification related to page conversion invalidation according to one embodiment. [Modes for carrying out the invention]

[0010] The following description is provided to illustrate the general principles of the present invention and does not limit the inventive concepts claimed herein. The following detailed description includes numerous details to provide an understanding of the processor, its architectural structure, and its operating method; however, it will be understood by those skilled in the art that numerous embodiments of the processor, its architectural structure, and operating method are implementable without these specific details. Furthermore, the claims and the present invention should not be limited to the embodiments, subassemblies, features, processes, methods, aspects, characteristics, or details specifically described and shown herein. Moreover, certain features described herein can be used in combination with other described features in each of the various possible combinations and substitutions.

[0011] Unless otherwise defined herein, all terms shall be given the broadest possible interpretation, including the meaning as understood by those skilled in the art, the meaning as defined in dictionaries, specialized books, or both, and the meaning implied herein.

[0012] The term "workload" for a processor refers to the number of instructions executed by that processor during a given period or at a specific moment in time.

[0013] A processor can process instructions by executing them in a series of small steps. In some cases, a processor may be pipelined to increase the number of instructions it processes (and therefore increase the processor's speed). Pipelining refers to the arrangement of separate stages within a processor, each stage executing one or more of the small steps required to execute an instruction. For example, a pipeline (in addition to other circuits) may be located in a part of the processor called a processor core. Some processors may have multiple processor cores (e.g., a multiprocessor system), and in some cases, each processor core may have multiple pipelines. When a processor core has multiple pipelines, groups of instructions (called issue groups) may be issued in parallel to multiple pipelines and executed in parallel by each pipeline. A pipeline may include multiple stages, such as a decode stage, a dispatch stage, and an execution stage. An execution stage may include various execution units that handle different types of operations specified by an instruction. For example, a load-store unit (LSU) is an execution unit that handles, for example, load instructions and store instructions.

[0014] In one example, the physical address of a store instruction to be executed may be stored in the LSU as an entry in a store reorder queue (SRQ). In one example, the SRQ may reside in the L1 data cache of a processor core. An SRQ entry may be a committed store instruction. A committed store instruction may be a store instruction completed by a processor or processing element, and the completion cannot be undone, and other processing elements are unaware of the completion until the SRQ is drained into memory (e.g., a Level 2 (L2) cache). When an SRQ or a committed store instruction is drained into memory (e.g., a Level 2 (L2) cache), the value stored or updated at the memory address specified by the committed store instruction becomes visible to all processors or processing elements in a multiprocessor system. For example, if the L2 cache is global memory accessible to all processing elements, draining an SRQ entry into the L2 cache means that the value updated by the store instruction of the drained SRQ entry becomes visible to all processing elements. In one or more exemplary embodiments, the L2 cache may be memory local to the processing element, while higher-level caches, such as a Level 3 (L3) cache, may be global memory accessible to multiple processing elements. SRQ entries can be drained from local memory to global memory.

[0015] In one example, the mapping between virtual and physical addresses may be invalidated in response to certain events. For instance, when data is moved in or out of physical memory (e.g., a new process is launched or a context switch is performed), the entries in the TLB need to be updated to reflect the presence of the new data, and the TLB entries associated with the data removed from system memory need to be invalidated. In one example, the instruction used to invalidate a TLB entry is sometimes called a TLB invalidate instruction (TLBI). When the mapping between virtual and physical addresses is invalidated, a TLBI instruction is issued to all cores to delete the TLB entry corresponding to the invalidated mapping. In one example, one core may be assigned the task of broadcasting TLBI instructions to other cores in a multiprocessor system.

[0016] In one example, the first processor may invalidate a specific TLB entry in its TLB. This specific TLB entry may map a specific virtual address to a specific physical address. The first processor may stall any process or instruction, or both, associated with this specific physical address (for example, a committed store instruction that writes to the specific physical address). The first processor may then broadcast a TLBI instruction on an interconnect accessible to all processors in the multiprocessor system. The second processor in the multiprocessor system receives the TLBI instruction from the interconnect and, in response, may drain the SRQ entry associated with the specific physical address into memory. After draining, the second processor may invalidate the entry in its TLB that contains the specific physical address. In response to the invalidation, the second processor may send an acknowledgment to the first processor indicating that it has completed invalidating the TLB. The first processor may wait for acknowledgments from the second processor and all other processors in the multiprocessor system before invalidating a specific TLB entry in its TLB.

[0017] However, in one example, the propagation time of a TLBI instruction may include the time required for the TLBI instruction to propagate from the first processing element to the interconnect, propagate within the interconnect, and propagate from the interconnect to the second processing element. During this propagation time, the second processor may be draining its own SRQ at its normal SRQ drain rate or drain cycle. Problems can arise if the SRQ entry associated with a specific physical address of the TLBI instruction is not drained by the second processing element before the second processing element sends an acknowledgment to the first processing element that broadcast the TLBI instruction. For example, SRQ logic or algorithms may be used to detect the completion of SRQ draining by the second processing element. If the SRQ logic is corrupted and makes a false positive about whether the SRQ has been completely drained, the second processing element may send an acknowledgment before completely draining the SRQ. As a result of the second processing element not completely draining the SRQ before sending an acknowledgment, the value of the specific physical address indicated by the TLBI instruction may not be up-to-date when all processing elements have completed invalidation in accordance with the broadcasted TLBI instruction. For example, after all processing elements have completed their respective invalidation, a processing element may be reading an old value from a specific physical address. Therefore, a conflicting condition exists between TLB entry invalidation (or page conversion), commitment within SRQ, and drain, and this conflicting condition may affect the visibility of store instruction results for all processing elements in a multiprocessor system.

[0018] The methods and systems described herein can widen the race condition window between a processor's TLB entry invalidation (e.g., TLBI) cycle and drain cycle in a multiprocessor system to test whether the processor's SRQ logic is functioning correctly. By widening this race condition window, the likelihood of an SRQ drain event (or SRQ cycle) overlapping with a TLBI invalidation cycle can be increased. As an example, a delay can be applied to the processor's SRQ drain cycle to slow down the SRQ drain. Traditionally, the drain cycle may be shorter than the invalidation cycle (e.g., SRQ drain may be faster than the propagation of TLBI instructions). Furthermore, the drain cycle can be unpredictable because the processing time of store instructions may be variable (e.g., because they depend on other threads and results from other processor cores). By applying a delay to the SRQ drain cycle, the processor has more opportunities and time to identify potential problems or corruptions in the SRQ logic. For example, without a delay, the processor's SRQ may drain relatively quickly, increasing the likelihood that the SRQ is empty when the processor receives a TLBI instruction from the interconnect. Because the SRQ logic indicates that the SRQ is empty, the processor may send an acknowledgment without verifying whether the SRQ is truly empty, making it impossible to test the SRQ logic. By adding time by delaying the drain cycle, the probability that the SRQ is not empty when the processor receives the TLBI instruction increases. This allows for testing to determine whether the SRQ logic is functioning correctly in order to verify cache coherence. If the SRQ is not empty, the processor is triggered by the SRQ logic, checks the SRQ before sending an acknowledgment, and drains the SRQ. If the processor is successfully triggered by the SRQ logic and drains the SRQ before sending an acknowledgment, the processor's cache coherence can be considered successful.If the processor is not properly triggered by the SRQ logic and the processor does not drain the SRQ before sending an acknowledgment, the processor's cache coherence can be considered a failure.

[0019] Figure 1 shows an information handling system 101. The information handling system 101 can be a simple example of a computer system capable of performing the computing operations described herein. The computer system 101 may include one or more processors 100 coupled to a host bus 102. The processors 100 may include, for example, commercially available microprocessors, custom processors, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), discrete logic, or any device in general for executing instructions. In one example, a processor 100This may be a multi-core processor including two or more processor cores. The Level 2 (L2) cache memory 104 may be coupled to the host bus 102. The I / O bridge (e.g., host-PCI bridge) 106 may be coupled to the main memory 108. The I / O bridge may include cache memory and main memory control functions and may provide bus control for handling transfers between the PCI bus 110, processor 100, L2 cache 104, main memory 108, and host bus 102. The main memory 108 may be coupled to the I / O bridge 106, as well as the host bus 102. Other types of memory, such as random access memory (RAM), or various volatile and / or non-volatile memory devices, or combinations thereof, may also be coupled to the host bus 102, the I / O bridge 106, or both. For example, memory devices coupled to the host bus 102 may include electrically erasable programmable ROM (EEPROM), flash programmable ROM (PROM), battery-backed RAM, and hard disk drives. Non-volatile memory devices coupled to the host bus 102 may be used to hold any non-volatile data, including executable firmware and executable programming instructions for causing the processor 100 to perform certain functions, such as those described herein. Devices used solely by the processor 100, such as I / O components 130, may be coupled to the PCI bus 110. A service processor interface and ISA access passthrough 112 may provide an interface between the PCI bus 110 and the PCI bus 114. In this way, the PCI bus 114 may be isolated from the PCI bus 110. Devices such as flash memory 118 are coupled to the PCI bus 114. In one implementation, the flash memory 118 may contain BIOS code incorporating processor-executable code necessary for various low-level system functions and system boot functions.

[0020] The PCI bus 114 may provide interfaces for various devices shared by the host processor 100 and the service processor 116, such as flash memory 118. The PCI-to-ISA bridge 135 provides bus control, Universal Serial Bus (USB) functionality 145, and power management functionality 155 for handling transfers between the PCI bus 114 and the ISA bus 140, and may also include other functional elements not shown, such as a real-time clock (RTC), DMA control, interrupt support, and system management bus support. Non-volatile RAM 120 may be connected to the ISA bus 140. The service processor 116 may include a bus 122 (e.g., a JTAG or I2C bus or both) for communicating with the processor 100 during the initialization step. The bus 122 may also be coupled to the L2 cache 104, I / O bridge 106, and main memory 108, providing a communication path between the processor, the service processor, the L2 cache, the host-to-PCI bridge, and main memory 108. The service processor 116 can also access system power resources to power off the information handling device 101.

[0021] Peripheral devices and input / output (I / O) devices can be connected to various interfaces coupled to the ISA bus 140 (e.g., parallel interface 162, serial interface 164, keyboard interface 168, mouse interface 170). Alternatively, a super I / O controller (not shown) connected to the ISA bus 140 can accommodate many I / O devices. Other interfaces that allow the processor 100 to communicate with external devices include, but are not limited to, serial interfaces such as RS-232, USB (Universal Serial Bus), Small Computer System Interface (SCSI), RS-309, or wireless communication interfaces such as Wi-Fi®, Bluetooth®, and Near Field Communication (NFC).

[0022] In one example, to connect computer system 101 to another computer system and copy files over a network, I / O component 130 may include a LAN card coupled to PCI bus 110. Similarly, to connect computer system 101 to an ISP using a telephone line connection and connect to the Internet, modem 175 is connected to serial port 164 and PCI-to-ISA bridge 135. FIG. 1 shows an information handling system using processor 100, but the information handling system can take many forms. For example, information handling system 101 may be in the form of a desktop, server, portable, laptop, notebook, or other form factor computer or data processing system. Information handling system 101 may also have other form factors such as a personal digital assistant (PDA), gaming device, ATM machine, cellular phone device, communication device, or other device including a processor and memory.

[0023] Figure 2 is a block diagram of a processor 200 according to one embodiment. The processor 200 may include at least a memory 202, an instruction cache 204, an instruction fetch unit 206, a branch predictor 208, and a processor pipeline or processing pipeline 210. The processor 200 may be contained within a computer processor or otherwise distributed within a computer system. Instructions and data can be stored in the memory 202, and the instruction cache 204 may access instructions in the memory 202 and store instructions to be fetched. The memory 202 may include any type of volatile or non-volatile memory, such as cache memory. The memory 202 and the instruction cache 204 may include multiple cache levels. A data cache (not shown) may be included in the processor 200. In one embodiment, the instruction cache 204 may be configured to provide instructions in an 8-way set associative structure. Alternatively, any other desired configuration and size may be employed. For example, the instruction cache 204 may be implemented as a fully associative, set-associative, or direct-mapped configuration.

[0024] FIG. 2 shows a simple example of an instruction fetch unit 206 and a processing pipeline 210. In various embodiments, the processor 200 may include multiple processing pipelines 210 and instruction fetch units 206. In one embodiment, the processing pipeline 210 includes a decode unit 20, an issue unit 22, an execution unit 24, and a write-back logic 26. In some examples, the instruction fetch unit 206 or the branch predictor 208 or both may also be part of the processing pipeline 210. The processing pipeline 210 may also include other features such as error checking and processing logic, a reorder buffer, one or more parallel paths through the processing pipeline 210, and other features currently known or later to be known in the art. Although FIG. 2 depicts a forward path through the processor 200, other feedback paths and signal paths may be included between elements of the processor 200.

[0025] Branch instructions (or "branch") can be unconditional branches, meaning that a branch occurs every time an instruction is encountered in a program, or conditional branches, meaning that a branch may or may not occur depending on a condition. The processor 200 may provide conditional branch instructions that, if a condition is met, allow the computer program to branch from a given instruction to a target instruction (thus skipping any intermediate instructions). If the condition is not met, the instruction following the branch instruction may be executed without branching to the target instruction. In many cases, the instruction to be executed following a conditional branch is not known for certain until the condition on which the branch depends is resolved. The branch predictor 208 may attempt to predict the outcome of a conditional branch instruction in the program before the branch instruction is executed. If the branch is incorrectly predicted, all speculative work done in the program from the point of encountering the branch onward must be discarded. For example, upon encountering a conditional branch instruction, the processor 200 may predict which instruction will be executed after the outcome of the branch condition is known. Instead of stalling the processing pipeline 210 when a conditional branch instruction is issued, the processor can continue issuing instructions from the next expected instruction.

[0026] A conditional branch allows control to be transferred to a target address depending on the result of the previous instruction. A conditional branch is either resolved or unresolved depending on whether the result of the previous instruction is known at the time the branch is executed. If the branch is resolved, it is known whether the branch will be executed or not. If the conditional branch is not executed, the sequential instruction stream immediately following the branch instruction is executed. If the conditional branch is executed, the instruction stream starting from the target address is executed.

[0027] The instruction fetch unit 206 fetches an instruction from the instruction cache 204 according to the instruction address so that the decode unit 20 can perform further processing. The decode unit 20 decodes the instruction and passes the decoded instruction, part of the instruction, or other decoded data to the issue unit 22. The decode unit 20 may also detect a branch instruction that was not predicted by the branch predictor 208. The issue unit 22 analyzes the instruction or other data and, based on the analysis, sends the decoded instruction, part of the instruction, or other data to one or more execution units in the execution unit 24. The execution unit 24 executes the instruction and determines whether the predicted branch direction is incorrect. The branch direction may be "taken". In this case, the subsequent instruction is fetched from the target address of the branch instruction. Conversely, the branch direction may be "not taken". In this case, the subsequent instruction is fetched from a memory location adjacent to the branch instruction. If a mispredicted branch instruction is detected, instructions following this mispredicted branch can be discarded from various units of the processor 200.

[0028] The execution unit 24 may include multiple execution units, such as a fixed-point execution unit, a floating-point execution unit, a load / store execution unit (or load / store unit denoted as LSU), and a vector multimedia execution unit. The execution unit 24 may also include a dedicated branch predictor for predicting the target of a multi-target branch. The write-back logic 26 writes the result of instruction execution back to the destination resource 220. The destination resource 220 can be any type of resource, including registers, cache memory, other memory, I / O circuits for communicating with other devices, other processing circuits, or any other type of destination for the executed instruction or data. One or more of the processor pipeline units may also provide the branch predictor 208 with information regarding the execution of a conditional branch instruction.

[0029] For example, an execution slice may be referred to as a set of data processing circuits or hardware units connected in series within a processor core. An execution slice may be a pipeline or a pipeline-like structure. Multiple execution slices may be used as part of simultaneous multi-threading within one of several processor cores in a multiprocessor system. In modern computer architectures, multiple execution units, such as LSUs, vector scalar units (VSUs), and arithmetic logic units (ALUs), may reside within an execution slice. An LSU typically includes one or more store queues, each tracking store instructions and having entries for holding store data, and one or more load queues, each tracking load instructions and having entries for holding load data.

[0030] In one embodiment, the processor 200 may perform branch prediction to speculatively fetch instructions following a conditional branch instruction. A branch predictor 208 is included to perform such branch prediction operation. In one embodiment, the instruction cache 204 may provide the branch predictor 208 with instructions for the instruction addresses to be fetched. This allows the branch predictor 208 to determine which branch target addresses to select to form a branch prediction. The branch predictor 208 may be coupled to various parts of the processing pipeline 210, such as the execution unit 24, the decode unit 20, and the reorder buffer, to determine whether the predicted branch direction is correct or incorrect.

[0031] To facilitate multithreading, instructions from different threads can be interleaved in some way at some point in the overall processor pipeline. One technique for interleaving instructions from different threads involves interleaving instructions cycle by cycle based on an interleaving rule. For example, instructions from different threads can be interleaved such that the processor executes the instruction from the first thread in the first clock cycle, then the instruction from the second thread in the second clock cycle, and then another instruction from the first thread in the third clock cycle. Some interleaving techniques assign a priority to each thread and interleave instructions from different threads based on that assigned priority. For example, if the first thread is assigned a higher priority than the second thread, the interleaving rule may require that instructions from the first thread, which has the higher priority, be included in the interleaved stream twice as often as instructions from the second thread, which has the lower priority. You can set up various different interleaving rules, such as rules designed to resolve threads of the same priority, or rules that periodically interleave instructions from relatively less important threads (for example, executing instructions from lower-priority threads every X cycles).

[0032] Priority-based thread interleaving allows for the allocation of processor resources based on assigned priorities. However, thread priorities may not account for processor events (such as branch prediction misses) that could affect a thread's ability to progress through the processor pipeline. Such events can impact the efficiency of processor resource allocation between different instruction threads in a multithreaded processor. For example, priority-based techniques that give higher priority to threads with fewer instructions in the pipeline's decode, rename, and instruction queue stages can be inefficient in reducing the number of mis-passed instructions resulting from branch prediction misses in the pipeline (e.g., mis-speculated instructions). These mis-passed instructions can occupy processor fetch bandwidth and other valuable resources, such as instruction queues and other functional units.

[0033] The efficiency, performance, or both of the processor 200 can be improved by reducing the number of incorrectly pathed instructions in the processing pipeline 210. For example, threads with a higher prediction miss rate can be delayed in the processing pipeline 210 (e.g., fetched more slowly by the instruction fetch unit), thereby reducing the number of incorrectly pathed instructions in the processing pipeline 210. Furthermore, the processing pipeline 210 can be tracked for multiple instructions following the first incomplete or unresolved branch instruction to prevent potentially incorrectly pathed instructions from being executed excessively.

[0034] In one embodiment, the processor 200 may be an SMT processor configured to perform multithreading. The processor 200 may collect instructions from one or more different threads using one or more instruction queues 212. An instruction fetch unit 206 may fetch instructions stored in an instruction cache 204 and fill the instruction queues 212 with these fetched instructions. The performance of the processor 200 may depend on how the instruction fetch unit 206 fills these instruction queues 212. The instruction fetch unit 206 may be configured to assign and manage priorities for different threads, determine which instructions or threads or both to fetch based on these priorities, and send these fetched instructions to the instruction queues 212. The processor 200 may further include a thread scheduler 214 configured to schedule instructions in the instruction queues 212 and distribute them to a processing pipeline 210. In one example, the processor 200 may be a multicore processor having two or more processor cores, each core may be configured to handle its own threads.

[0035] In one example, if the execution unit 24 is a load-store unit (LSU) 228, the circuit 230 may be embedded in or integrated into the LSU 228 to implement SRQ drain delay. The SRQ drain delay may, for example, delay or slow down the drain cycle of a store reorder queue (SRQ) 229 (or store queue) to memory (e.g., an L2 cache which may be part of the destination resource 220). In one example, the circuit 230 may be activated (e.g., switched on) or deactivated (e.g., switched off) by the processor 200. The activation and deactivation of the circuit 230 may be based on the operating state of the processor 200 or other processors or processor cores in a multiprocessor system or a combination thereof. For example, the processor 200 may activate the circuit 230 to apply a delay to the drain cycle of the SRQ 229 in the LSU 228. In another example, the processor 200 may deactivate the circuit 230 to prevent the application of a delay to the drain cycle of the SRQ 229 in the LSU 228. By applying a delay to the drain cycle of SRQ229, processor 200 can use additional time to verify whether processor 102's cache coherence (including load coherence and store coherence) is successful or unsuccessful. For example, this additional time increases the likelihood that committed store instructions remain in SRQ229. This allows processor 102 to test the SRQ logic used to detect an empty SRQ and trigger the sending of an acknowledgment signal. If the SRQ is always empty when a TLBI instruction arrives, the SRQ logic cannot be tested. If a TLBI instruction is received, the SRQ logic triggers a processing element, and the SRQ is drained before sending an acknowledgment signal, then processor 102's cache coherence can be considered successful. If a TLBI instruction is received, but the SRQ logic does not trigger the draining of the SRQ, and the processing element sends an acknowledgment signal without draining the SRQ, then processor 102's cache coherence can be considered unsuccessful.

[0036] Figure 3 shows an example implementation of store coherence verification related to page conversion invalidation in one embodiment. In one example, the processor 200 (see Figures 1 and 2) may include N processing elements such as processing elements 310, 320, and 340 (labeled as core 0, core 1, and core N). Although three processor cores are shown in Figure 3, the processor 200 may have additional processor cores. An interconnect 301 (e.g., a bus, mesh network, crossbar, etc.) may connect core 0, core 1, and core N, as well as the other cores of the processor 200. Core 0 may include a load-store unit (LSU) 312, a level 2 (L2) cache 316, and a TLB 318. The LSU 312 may include an SRQ 314. The TLB 318 may include multiple entries indicating the mapping between virtual memory addresses and physical memory addresses assigned to core 0. For example, TLB318 may contain entries labeled M1, M2, M3, and M4. Core 0 may contain a load-store unit (LSU) 312, a level 2 (L2) cache 316, and TLB318. LSU312 may contain SRQ314. Core 1 may contain LSU322, an L2 cache 326, and TLB328. LSU322 may contain SRQ324. TLB328 may contain multiple entries indicating the mapping between virtual memory addresses and physical memory addresses assigned to Core 1. Core N may contain LSU342, an L2 cache 346, and TLB348. LSU342 may contain SRQ344. TLB348 may contain multiple entries indicating the mapping between virtual memory addresses and physical memory addresses assigned to Core N. In one or more embodiments, the L2 caches 316, 326, and 346 may be individual memory banks of a global L2 cache accessible by core 0, core 1, and core N.

[0037] For example, in response to an event such as a context switch, core 0 may invalidate an entry in TLB 318 and broadcast a Trans-Look-Aside Buffer Invalidation (TLBI) instruction 302 onto interconnect 301. The TLBI instruction 302 may also be an instruction to other processing elements to invalidate one or more TLB entries in their respective TLBs corresponding to a specific virtual address (e.g., P3) or a specific physical address (e.g., F4) or both. For example, the TLBI instruction 302 may be an instruction to processing elements other than core 0 to invalidate a TLB entry that maps a virtual address to physical address F4.

[0038] The TLBI instruction 302 may propagate from core 0 to interconnect 301, then propagate within interconnect 301, and then propagate from interconnect 301 to processing elements such as core 1 and core N. Therefore, the propagation time of the TLBI instruction 302 may be the sum of the time required for the TLBI instruction 302 to propagate from core 0 through interconnect 301 to the receiving cores (e.g., core 1, core N). Note that the propagation time of the TLBI instruction 302 may differ between different processing elements based on the distance between the receiving core and the core that issued the TLBI instruction 302, and other factors such as various process fluctuations, core hardware capabilities, and interconnect traffic. In Figure 3, core 0 is shown as the processing element that issues the TLBI instruction, but other processing elements such as core 1 and core N may also be configured to issue TLBI instructions related to TLB entry invalidation in their respective TLBs.

[0039] Core 1 receives a TLBI instruction 302 from interconnect 301 and may accordingly drain SRQ324, or any remaining entries in SRQ324 of LSU322. Here, the SRQ entries being drained may be committed store instructions. Upon completely draining SRQ324, Core 1 may send an acknowledgment signal (ACK) 304 to Core 0 to notify Core 0 that SRQ324 has been drained. Upon sending the ACK 304, Core 1 may invalidate the TLB entries in TLB328 associated with the virtual address P3 or virtual address F4 or both, as indicated by the TLBI instruction 302. For example, Core 1 may invalidate entry M2 in TLB328 that maps virtual address P3 to virtual address F4. Core 0 may invalidate the TLB entries in TLB318 associated with virtual address P3 or virtual address F4 or both, upon receiving the ACK 304 from Core 1 and all other cores (e.g., ACK 306 from Core N). In one example, core 0 may wait for ACK signals from all cores before resuming normal operation. For instance, upon receiving ACK signals from all cores, core 0 may map page P3 to a different physical address and update TLB318 with the new mapping.

[0040] For example, core 1 may execute logic 327 to detect whether SRQ324 is empty. Depending on whether SRQ324 is empty, it may trigger an action to send ACK304 to core 0. However, if logic 327 is corrupted, core 1 may incorrectly detect that SRQ324 is empty even though it is not. If core 1 sends ACK304 to core 0 even though the SRQ is not empty, problems can arise if committed store instructions are not properly drained from SRQ324. For example, if a committed store instruction to store in F4 remains in SRQ324, but due to a corrupted logic 327, it cannot detect that this committed store instruction exists in SRQ324, core 1 may send ACK304 to core 0 and invalidate entry M2. As a result of this error, core 0 and other cores may not be able to see the value updated in F4 because the committed store instruction remaining in SRQ324 was not drained.

[0041] To reduce the possibility of SRQs not being properly drained, cores 0, 1, and N may implement a configurable delay circuit (e.g., circuit 230 shown in Figure 2) that applies a delay 330 to the drain cycle of SRQs 314, 324, and 344, respectively. In one example, the delay circuit may be integrated into LSUs 312, 322, and 342. The delay 330 may be a specific number of cycles added to the default drain cycle of an SRQ (e.g., SRQs 314, 324, and 344), so that SRQs 314, 324, and 344 may be drained at a slower rate depending on the application of the delay 330. Slowing down the SRQ drain cycle increases the likelihood that SRQ entries remain in the SRQ when a TLBI instruction is received. In other words, the delay 330 can provide an additional time for a receiving processing element (e.g., core 1 or core N receiving a TLBI instruction 302) to identify the SRQ entries associated with the TLBI instruction 302 and take appropriate action to improve the situation. For example, core 1 can identify the SRQ entry related to F4 in SRQ324 and drain this identified entry from SRQ324 to L2 cache 326. Depending on whether SRQ324 has been drained (for example, until SRQ324 is empty), core 1 may invalidate the TLB entry related to TLBI instruction 302 in TLB328.

[0042] Figure 4 shows an example of another implementation of store coherence verification related to page conversion invalidation in one embodiment. In the example shown in Figure 4, scenario 401 shows that core 1 processes the TLBI instruction 302 without applying delay 330, and scenario 402 shows that core 1 processes the TLBI instruction 302 with delay 330 applied. In scenario 401, when core 1 receives the TLBI instruction 302 from the interconnect, SRQ324 is empty, and SRQ entries E1, E2, and E3 have already been drained, for example, to the L2 cache 326. If SRQ entry E3 is related to the TLBI instruction 302 (e.g., a write to physical address F4), and SRQ entry E3 was drained before core 1 received the TLBI instruction 302, then E3 would be considered properly drained. However, if logic 327 (see Figure 3) is corrupted in scenario 401, then SRQ324 being empty may not be correct. If core 1 incorrectly determines that SRQ324 is empty, even though SRQ entry E3 exists within SRQ324, entry E3 may not be drained before core 1 sends ACK304 to interconnect 301.

[0043] In scenario 402, when core 1 receives the TLBI instruction 302 from the interconnect, the delay 330 is applied to the drain cycle of SRQ324, resulting in SRQ324 not being empty and SRQ entry E3 remaining in SRQ324. Core 1 may determine that SRQ entry E3 is associated with the TLBI instruction 302 and drain SRQ entry E3 from SRQ324 before sending ACK304 to the interconnect 301. Applying the delay 330 slows down the drain of SRQ324 and provides core 1 with additional time to identify the SRQ entry in SRQ324. As an example, in scenario 401, core 1 receives the SRQ 324Relying on logic 327 to indicate that it is empty, it may send ACK304 to interconnect 301 without checking if there are any entries remaining in SRQ324. Delaying the SRQ drain cycle makes it less likely that SRQ324 is empty. This can trigger core 1 to drain SRQ324 before sending ACK304.

[0044] In one example, the delay 330 may be a specific number of cycles added to the default drain cycles of the SRQ324. The number of cycles for the delay 330 may be proportional to the time required to propagate the TLBI instruction 302 among all cores in a multiprocessor system. In one example, the number of cycles for the delay 330 may be the product of the number of cycles required to drain each entry of the SRQ324 and the size of the SRQ324 (e.g., the number of allowed entries or the maximum number of entries in the SRQ324). For example, the default number of cycles required to drain each entry of the SRQ324 may be 2 (e.g., one SRQ entry is drained every 2 cycles), and the number of allowed entries in the SRQ324 may be 64. Thus, the number of cycles for the delay 330 may be any multiple of 64. In another example, circuit 230 (see Figure 2) may include a random number generator that generates random numbers between 1 and a multiple of the number of allowed entries in the SRQ324. The generated random number may be set as the number of cycles for the delay 330, the number of delay cycles for each SRQ entry, or both. For example, the delay 330 applied to the first SRQ entry may be a first number of cycles, and the delay 330 applied to the second SRQ entry may be a second number of cycles. In one example, the circuit 230 may include a linear feedback shift register (LFSR) capable of implementing random number generation. The number of cycles defining the delay 330 can be arbitrary and can be configured or programmed based on a desired implementation of the processor 102 (see Figure 2).

[0045] Figure 5 shows an example of another implementation of store coherence verification related to page conversion invalidation in one embodiment. In the example shown in Figure 5, the TLBI cycle 500 may span from time T0 to T3. The TLBI cycle 500 may include the propagation time of the TLBI instruction from the first processing element to the second processing element. The SRQ drain cycle 502 may span from time T0 to time T1. The SRQ drain cycle may be smaller than the TLB cycle. This may cause the SRQ to be drained at a faster pace than the propagation of the TLBI instruction. Therefore, if the SRQ is drained faster than the propagation of the TLBI instruction, it becomes more likely that the SRQ will be empty when the TLBI instruction arrives at the processing element. If the SRQ is empty when the TLBI arrives, the SRQ logic implemented to detect whether the SRQ is empty cannot be tested because the SRQ is already empty (e.g., there are no non-empty SRQs to detect).

[0046] Applying a delay of 330 to the SRQ drain cycle 502 allows the SRQ to drain at a slower pace under a delayed SRQ drain cycle spanning time T0 to T2. Therefore, the difference between the TLBI cycle 500 and the SRQ drain cycle 502 can be reduced. In other words, the amount of overlap between the SRQ drain cycle 502 and the TLBI cycle 500 can be increased depending on the application of delay 330. The additional time T1-T2 resulting from the delay 300 increases the likelihood that the SRQ holds more SRQ entries. This means that when a TLBI instruction arrives, the SRQ may not be empty, providing an opportunity to test the SRQ logic (e.g., implemented to detect empty SRQs in order to send acknowledgments). increase This is possible. Note that the delay 330 may be variable. Therefore, the SRQ drain cycle 502 may be delayed by different amounts depending on the desired implementation.

[0047] Figure 6 shows an example flowchart of store coherence verification related to page conversion invalidation according to one embodiment. Process 600 may include one or more operations, actions, or functions, as exemplified by one or more blocks 602, 604, or 606 or combinations thereof. Although shown as individual blocks in the figure, various blocks can be divided into additional blocks, combined into fewer blocks, deleted, executed in parallel, or executed in different orders, depending on the desired implementation.

[0048] Process 600 may begin in block 602. In block 602, a processing element may apply a delay to the drain cycle of its Store Reorder Queue (SRQ). This processing element may be one of several processing elements in a data processing system. Process 600 may proceed from block 602 to block 604. In block 604, a processing element may drain the SRQ under a delayed drain cycle. In one example, the delay may be proportional to the time required to propagate a TLBI instruction from one processing element to another via the interconnect. In another example, the delay may be based on the product of the number of cycles required to drain each entry in the SRQ and the size of the SRQ. In yet another example, the number of cycles for the delay may be based on a random number between 1 and a multiple of the size of the SRQ.

[0049] Process 600 may proceed from block 604 to block 606. In block 606, a processing element may receive a Trans-Look-Aside Buffer Invalidation (TLBI) instruction from an interconnect connecting multiple processing elements. The TLBI instruction may be an instruction that invalidates a Trans-Look-Aside Buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame. The TLBI instruction may be broadcast by another processing element connected to the interconnect. The difference between the drain cycle of the SRQ and the invalidation cycle associated with the TLBI may be reduced by applying a delay to the drain cycle of the SRQ.

[0050] In one example, upon receiving a TLBI instruction, the processing element may decide whether to send an acknowledgment signal or drain the SRQ. For example, the processing element may identify the SRQ entries in the SRQ associated with the TLB entries to be invalidated and drain the identified SRQ entries from the SRQ. In one example, upon completely draining the SRQ, the processing element may send an acknowledgment signal to the processing element that broadcast the TLBI instruction via the interconnect. In one example, upon receiving a TLBI instruction, the processing element may determine that the SRQ is empty and send an acknowledgment signal to the processing element that broadcast the TLBI instruction via the interconnect.

[0051] The present invention may be a system, method, or computer program product or combination thereof, integrated at any possible level of technical detail. The computer program product may include a computer-readable storage medium storing computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0052] A computer-readable storage medium can be a tangible device capable of holding and storing instructions used by an instruction execution device. Examples of computer-readable storage media include electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or appropriate combinations thereof. More specific examples of computer-readable storage media include portable computer diskettes, hard disks, RAM, ROM, erasable programmable ROM (EPROM or flash memory), static random access memory (SRAM), CD-ROMs, DVDs, memory sticks, floppy disks, mechanically encoded devices with instructions recorded on punch cards or grooved raised structures, and appropriate combinations thereof. The computer-readable storage media used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0053] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing device / processing device. Alternatively, they can be downloaded to an external computer or external storage device via a network (e.g., the Internet, LAN, WAN, or wireless network, or a combination thereof). The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers, or a combination thereof. A network adapter card or network interface within each computing device / processing device receives computer-readable program instructions from the network and transfers them for storage in a computer-readable storage medium within each computing device / processing device.

[0054] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions can be executed as a standalone software package, either entirely on the user's computer or partially on the user's computer. Alternatively, they can be executed partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including LANs and WANs, or it may be connected to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, electronic circuits, including, for example, programmable logic circuits, field-programmable gate arrays (FPGAs), and programmable logic arrays (PLAs), can execute computer-readable program instructions by utilizing state information of computer-readable program instructions in order to customize the electronic circuits for the purpose of performing aspects of the present invention.

[0055] Aspects of the present invention are described herein with reference to flowcharts or block diagrams, or both, of methods, apparatus (systems), and computer program products according to embodiments of the present invention. Each block in a flowchart or block diagram, or both, and combinations of blocks in a flowchart or block diagram, or both, are executable by computer-readable program instructions.

[0056] These computer-readable program instructions can be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing device for the production of a machine. This creates a means for these instructions, executed via the processor of such a computer or other programmable data processing device, to perform functions / operations identified in one or more blocks in a flowchart or block diagram, or both. These computer-readable program instructions can further be stored in a computer-readable storage medium that can be instructed to function in a particular manner for a computer, a programmable data processing device, or other device, or a combination thereof. Thus, the computer-readable storage medium containing the instructions constitutes a product containing instructions for performing functions / operations identified in one or more blocks in a flowchart or block diagram, or both.

[0057] Alternatively, a computer execution process may be generated by loading computer-readable program instructions into a computer, another programmable device, or other device, and having a series of operational steps executed on that computer, other programmable device, or other device. This ensures that the instructions executed on the computer, other programmable device, or other device perform functions / operations identified by one or more blocks in a flowchart, block diagram, or both.

[0058] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for performing a specific logical function. In some other implementations, the functions shown within a block may be executed in an order different from the order shown in each diagram. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may be executed in reverse order. Each block in a block diagram or flowchart, or both, and combinations of multiple blocks in a block diagram or flowchart, or both, can be executed by a dedicated hardware-based system that performs a specific function or operation, or executes a combination of dedicated hardware and computer instructions.

[0059] The terms used herein are for the sole purpose of describing specific embodiments and are not intended to limit the invention. Furthermore, in this specification and the appended claims, the singular forms “a,” “an,” and “the” include the plural form unless otherwise specified. In addition, where the terms “comprises” or “comprising” are used herein, they specify the existence of the described features, integers, steps, operations, elements, or components or combinations thereof, but do not preclude the existence or addition of one or more other features, integers, steps, operations, elements, components, or groups or combinations thereof.

[0060] Where the following claims contain means-plus-function elements or step-plus-function elements, all corresponding structures, materials, actions, and equivalents are intended to include any structures, materials, or actions for performing a function in combination with other specifically claimed claimed elements. The description of the present invention is presented for illustrative and explanatory purposes only and is not intended to be exhaustive or to limit the invention to the disclosed forms. Many changes and modifications will be apparent to those skilled in the art without departing from the scope of the invention. These embodiments have been selected and described to best illustrate the principles and practical applications of the invention and to enable others skilled in the art to understand the invention in various embodiments with various modifications suitable for their particular intended use.

Claims

1. A method for invalidating page conversion entries in a data processing system having multiple processing elements, Applying a delay to the drain cycle of the processing element's Store Order Queue (SRQ), Draining the SRQ under the delayed drain cycle, The process includes receiving a Trans-Look-Aside Buffer Invalidation (TLBI) instruction from an interconnect connecting the multiple processing elements, wherein the TLBI instruction invalidates a Trans-Look-Aside Buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame, and the TLBI instruction is broadcast by another processing element. A method in which the amount of overlap between the drain cycle of the SRQ and the invalidation cycle associated with the TLBI instruction is increased by applying the delay to the drain cycle of the SRQ.

2. Identifying the SRQ entry in the SRQ related to the TLB entry to be invalidated, The identified SRQ entry is drained from the SRQ, The method according to claim 1, further comprising:

3. The method according to claim 1, further comprising transmitting an acknowledgment signal to the other processing element via the interconnect in response to the complete draining of the SRQ.

4. The method according to claim 1, wherein the delay is proportional to the time required for the TLBI instruction to propagate from one processing element to the other processing element via the interconnect.

5. The method according to claim 1, wherein the delay is based on the product of the number of cycles required to drain each entry in the SRQ and the size of the SRQ.

6. The method according to claim 1, wherein the number of cycles of the delay is based on a random number between 1 and a multiple of the size of the SRQ.

7. The first processing element, The second processing element, An interconnect connected to the first processing element and the second processing element, A computer system comprising, The first processing element is configured to broadcast a Trans-Look-Aside Buffer Invalidation (TLBI) instruction on the interconnect, the TLBI instruction being an instruction that invalidates a Trans-Look-Aside Buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame, The second processing element described above is Applying a delay to the drain cycle of the store order queue (SRQ) of the second processing element, Draining the SRQ under the delayed drain cycle, The system is configured to receive the TLBI instruction from the interconnect and to perform the following actions: A computing system in which the amount of overlap between the drain cycle of the SRQ and the invalidation cycle associated with the TLBI instruction is increased by applying the delay to the drain cycle of the SRQ.

8. The second processing element described above is Identifying the SRQ entry in the SRQ related to the TLB entry to be invalidated, The identified SRQ entry is drained from the SRQ, A computing system according to claim 7, configured to perform the following:

9. The computing system according to claim 7, wherein the second processing element is configured to transmit an acknowledgment signal to the first processing element via the interconnect in response to the complete draining of the SRQ.

10. The computing system according to claim 7, wherein the delay is proportional to the time required for the TLBI instruction to propagate from the first processing element to the second processing element via the interconnect.

11. The computing system according to claim 7, wherein the delay is based on the product of the number of cycles required to drain each entry in the SRQ and the size of the SRQ.

12. The computing system according to claim 7, wherein the number of cycles of the delay is based on a random number between 1 and a multiple of the size of the SRQ.

13. The computing system according to claim 12, comprising a random number generator configured to generate the random numbers.

14. The computing system according to claim 13, wherein the random number generator is implemented by one or more linear feedback shift registers.

15. A computer program comprising program code adapted to perform the steps of any one of claims 1 to 6 when the program is executed on a computer.

16. A processing element including a processor pipeline that includes one or more load / store units (LSUs) configured to execute load and store instructions, wherein the one or more LSUs are Applying a delay to the drain cycle of the Store Order Queue (SRQ) of the aforementioned processing element, Draining the SRQ under the delayed drain cycle, It is configured to receive a Trans-Look-Aside Buffer Invalidation (TLBI) instruction from an interconnect connecting multiple processing elements, the TLBI instruction being an instruction that invalidates a Trans-Look-Aside Buffer (TLB) entry corresponding to at least one of a virtual memory page and a physical memory frame, and the TLBI instruction being broadcast by another processing element, A processing element that increases the amount of overlap between the drain cycle of the SRQ and the invalidation cycle associated with the TLBI instruction by applying the delay to the drain cycle of the SRQ.

17. The one or more LSUs mentioned above are Identifying the SRQ entry in the SRQ related to the TLB entry to be invalidated, The identified SRQ entry is drained from the SRQ, A processing element according to claim 16, configured to perform the following:

18. The processing element according to claim 16, wherein one or more LSUs are configured to transmit an acknowledgment signal to the other processing element via the interconnect in response to the complete draining of the SRQ.

19. The processing element according to claim 16, wherein the delay is proportional to the time required for the TLBI instruction to propagate from the other processing element to the processing element via the interconnect.

20. The processing element according to claim 16, wherein the delay is based on the product of the number of cycles required to drain each entry in the SRQ and the size of the SRQ.

21. The processing element according to claim 16, wherein the number of cycles of the delay is based on a random number between 1 and a multiple of the size of the SRQ.

Images