Virtual processor interrupt tracking

Abstract

An apparatus includes an interrupt distributor to distribute virtual interrupts to one or more physical processors, each virtual interrupt to be handled by one of a plurality of virtual processors that are able to be mapped to the one or more physical processors; and control circuitry to maintain virtual processor interrupt tracking information corresponding to a given virtual processor. The virtual processor interrupt tracking information includes a pending interrupt record that tracks which types of virtual interrupts are pending for the given virtual processor, and a pending interrupt status indication that is distinct from the pending interrupt record and that indicates a pending interrupt status of the given virtual processor. The pending interrupt status indication indicates whether a number of pending virtual interrupts for the given virtual processor is zero.

Description

Background Technology Technical Field

[0002] This technology relates to the field of data processing. More specifically, this technology relates to processing virtual interrupts that will be handled by a virtual processor. Technical Background

[0004] Virtualization allows multiple operating systems to run simultaneously on the same system hardware. Each operating system can run on a virtual processor (or virtual machine VM), which operates under the impression of having exclusive access to the system hardware's processor, peripherals, system memory, and input / output devices, but in reality, this virtual processor shares the hardware with other virtual processors. Physical hardware can include multiple physical processors that can be mapped to a particular virtual processor. A given virtual processor can frequently move from one physical processor to another. A given physical processor can have more than one virtual processor mapped to it at a given time, but can execute instructions from a specific virtual processor among those mapped to that physical processor. The virtual processor from which instructions are currently being executed is considered a resident virtual processor, while other virtual processors mapped to a given physical processor but whose instructions are not currently being executed are considered non-resident virtual processors. Virtualization can be particularly useful for enterprise server application spaces where a single physical server, including multiple physical processors, can be configured to support many virtual machines, even up to tens, hundreds, or thousands of virtual machines. Such virtualized servers can provide services such as web hosting, cloud services, databases, and storage management. Summary of the Invention

[0005] At least some examples provide an apparatus comprising: an interrupt assigner for assigning virtual interrupts to one or more physical processors, each virtual interrupt being processed by one of a plurality of virtual processors capable of being mapped to the one or more physical processors; and control circuitry for maintaining virtual processor interrupt tracking information corresponding to a given virtual processor, the virtual processor interrupt tracking information including: a pending interrupt record indicating which types of virtual interrupts are pending for the given virtual processor; and a pending interrupt status indication different from the pending interrupt record and indicating the pending interrupt status of the given virtual processor, the pending interrupt status indicating whether the number of pending virtual interrupts for the given virtual processor is zero.

[0006] At least some examples provide an apparatus comprising: means for assigning virtual interrupts to one or more physical processors, each virtual interrupt being handled by one of a plurality of virtual processors capable of being mapped to the one or more physical processors; and means for maintaining virtual processor interrupt tracking information corresponding to a given virtual processor, the virtual processor interrupt tracking information including: a pending interrupt record indicating which types of virtual interrupts are pending for the given virtual processor; and a pending interrupt status indication different from the pending interrupt record and indicating the pending interrupt status of the given virtual processor, the pending interrupt status indicating whether the number of pending virtual interrupts for the given virtual processor is zero.

[0007] At least some examples provide a method comprising: assigning virtual interrupts to one or more physical processors, each virtual interrupt being handled by one of a plurality of virtual processors capable of being mapped to the one or more physical processors; and maintaining virtual processor interrupt tracking information corresponding to a given virtual processor, the virtual processor interrupt tracking information comprising: a pending interrupt record indicating which types of virtual interrupts are pending for the given virtual processor; and a pending interrupt status indication different from the pending interrupt record and indicating the pending interrupt status of the given virtual processor, the pending interrupt status indicating whether the number of pending virtual interrupts for the given virtual processor is zero.

[0008] Further aspects, features, and advantages of this technology will become apparent from the following description, which is taken in conjunction with the accompanying drawings. Attached Figure Description

[0009] Figure 1 An example of a data processing system with multiple interrupt controllers that assign interrupts to physical processors is illustrated schematically.

[0010] Figure 2 An example of an interrupt controller is shown;

[0011] Figure 3 An example of a message-based interrupt is shown, where the address specified by the message is used to determine the interrupt type of the message-based interrupt;

[0012] Figure 4 An example of virtual processor interrupt trace information is shown;

[0013] Figure 5 Examples of pending interruption mapping, coarse pending mapping, and coarse enabled pending mapping are shown.

[0014] Figure 6 A finite state machine (FSM) is shown for controlling the transitions between states that indicate the pending interrupt state of a given virtual processor;

[0015] Figure 7 This is a flowchart illustrating the handling of a received pending interrupt for a non-resident virtual processor;

[0016] Figure 8 This is a flowchart illustrating a method in response to a search-triggered event of a given virtual processor; and

[0017] Figure 9 This is a flowchart illustrating a method for interrupting the search in response to the search triggering event. Detailed Implementation

[0018] The apparatus may have one or more physical processors, which may support two or more virtual processors that can be mapped to the physical processors. An interrupt allocator may be provided to assign virtual interrupts to appropriate physical processors. For example, the interrupt allocator may consider mapping information that specifies which physical processors a given virtual processor is mapped to. If, when a virtual interrupt is received targeting a particular virtual processor, and that virtual processor resides on that particular physical processor such that its instructions are being executed, the virtual interrupt may be routed to the relevant physical processor so that it can be processed by the virtual processor to which the virtual interrupt is targeted.

[0019] However, sometimes virtual interrupts may be received targeting a virtual processor that is currently in a non-resident state, such that the virtual interrupt is not currently executed when that virtual processor can be mapped to a specific physical processor. Therefore, it may be desirable to record some trace information that tracks received pending interrupts for a specific virtual processor, so that when that virtual processor later becomes resident, the trace information can be searched to identify pending interrupts that should be delivered to a new resident virtual processor, which can then handle these interrupts as if the virtual processor had already executed when the virtual interrupt was received. Therefore, control circuitry can be provided to maintain virtual processor interrupt trace information corresponding to a given virtual processor.

[0020] However, there may be numerous types of virtual interrupts that can be received for a given virtual processor, so searching for pending interrupt records indicating which types of virtual interrupts are pending for a given virtual processor can take a relatively long time. This can increase the latency associated with residency changes at a given physical processor when a previously executed virtual processor is made non-resident and a newly executed virtual processor is made resident, making it possible to perform a search for its pending interrupts. If the residency change operation is too slow, this can increase the downtime for a given physical processor that cannot perform useful processing for any virtual processor. This can be a specific problem in cloud servers or other systems in an enterprise space where the number of different virtual processors may be very large compared to the number of physical processors, making residency changes relatively frequent.

[0021] In the example below, the control circuitry maintains virtual processor interrupt trace information, which includes pending interrupt records indicating which types of virtual interrupts are pending for a given virtual processor, and also includes a pending interrupt status indication different from the pending interrupt records. This pending interrupt status indication indicates the pending interrupt status for the given virtual processor. The pending interrupt status at least indicates whether the number of pending virtual interrupts for the given virtual processor is zero. By recording some state information tracking whether the number of pending virtual interrupts is zero, this can accelerate residency changes by eliminating the need to search for pending interrupt records when it is known that there will be no pending virtual interrupts for the given virtual processor.

[0022] The device may include interrupt search circuitry that performs an interrupt search in response to a search trigger event associated with a given virtual processor. The interrupt search may include searching a pending interrupt record to identify pending interrupts that will be processed by the given virtual processor. This operation can be relatively slow because pending interrupt records may be stored, for example, in a memory system that may have some access latency, and in some examples, the interrupt search may need to examine multiple different cache lines in memory that record information about different types of virtual interrupts that may be pending. In response to a search trigger event, the interrupt search circuitry may suppress the execution of the interrupt search when a pending interrupt status indicator indicates that the number of pending interrupts to be processed by the given virtual processor is zero. Therefore, since it is not necessary to search through all the pending interrupt records, the search trigger event can be responded to more quickly, and thus any memory access latency associated with searching the pending interrupt record can be eliminated. This improves performance.

[0023] In some examples, the pending interrupt status can indicate whether the number of these pending virtual interrupts for a given virtual processor is zero, regardless of whether the pending virtual interrupts are enabled or disabled.

[0024] However, some systems support interrupt masking, where masking information can be used to define certain types of interrupts as disabled and others as enabled, such that disabled interrupts are types of interrupts that should not be delivered to the corresponding virtual processor. Therefore, in some cases, the pending interrupt state can indicate whether the number of enabled pending virtual interrupts for a given virtual processor is zero. For this example, any disabled pending virtual interrupts do not affect whether the pending interrupt state indicates that the number of enabled pending virtual interrupts for a given virtual processor is zero. In some cases, the interrupt search circuitry performing the interrupt search can specifically identify enabled pending interrupts (because disabled pending interrupts do not need to be delivered), and therefore, in response to a search trigger event, when the pending interrupt state indicates that the number of enabled pending interrupts to be processed by the given virtual processor is zero, the interrupt search can be suppressed to avoid delays in the search when only disabled interrupts that are pending for the given virtual processor exist.

[0025] In other examples, the pending interrupt status can distinguish between the presence of any pending virtual interrupts (enabled or disabled) for a given virtual processor and the presence of any enabled pending virtual interrupts (the status can still indicate that the number of enabled pending virtual interrupts is zero even if at least one disabled pending virtual interrupt still exists). For example, the pending interrupt status indication could be a status indicator that presents different states based on whether the pending interrupt record has no pending interrupts at all, or whether the pending interrupt record is filled but only has disabled pending interrupts, or whether at least one enabled pending interrupt exists. By providing different states based on the presence of disabled or enabled interrupts, this can further improve search time depending on whether the search is looking for all pending interrupts or only enabled interrupts.

[0026] Therefore, in some examples, the pending state indicator may indicate one of a plurality of states, and those states may include at least: a pending enable state indicating the existence of at least one enabled pending interrupt for a given virtual processor; and at least one other state indicating at least one of the following: no pending interrupt for a given virtual processor; and one or more disabled interrupts being the only pending interrupt for a given virtual processor.

[0027] More generally, a pending interrupt state indicator can indicate one of two or more states of a pending interrupt state machine associated with a given virtual processor. Control circuitry can trigger transitions between states of the pending interrupt state machine in response to an event associated with the given virtual processor (such as receiving a virtual interrupt targeted by the given virtual processor, or the given virtual processor becoming resident). The aforementioned pending enable state and at least one other state can be some of the states of the state machine, but the state machine can also have other states.

[0028] For example, in one implementation, the pending interrupt state may also indicate whether an interrupt search for a given virtual processor is in progress. For instance, the state machine may include a search state indicating whether the interrupt search circuitry is in the middle of an interrupt search for a given virtual processor. In some implementations, the state machine may also include a "search (received enable interrupt)" state, which indicates that an interrupt search is in progress and at least one enable pending interrupt has been received since the search began. These states can be used to determine whether the ongoing search will be sufficient to resolve another search trigger event upon receipt, or whether another interrupt search should be performed upon completion of the current search. When an enable pending interrupt for a given virtual processor is received while the pending state is indicated as "search," the control circuitry may update the pending interrupt state indication to switch the current state to the "search (enabled)" state.

[0029] This can be useful because another virtual interrupt might be received while an interrupt search is in progress, and if the interrupt search has already checked the portion of the pending interrupt log corresponding to that interrupt type, this other virtual interrupt might not be detected in the currently ongoing interrupt search. Therefore, if another search-triggered event occurs before the current search has completed, another search might be required. On the other hand, if no other enabled virtual interrupt has been received since the start of the search, another search might not be necessary in response to that other interrupt-triggered event, because if no enabled interrupt for a given virtual processor has been received since the start of the current search, the currently ongoing search (which started in response to an earlier interrupt search-triggered event) might be sufficient to handle that other search-triggered event. Therefore, providing a "search" state and a "search (enabled)" state can help avoid delays in interrupt searches where possible: another search-triggered event for a given virtual processor occurs while the previous search is still in progress and no other interrupt has been received since the start of that search.

[0030] An alternative method for handling another interrupt search trigger event received while a previous search is in progress could be to abort the current search in response to the other interrupt search trigger event and restart the search, thus taking into account any recently received virtual interruptions received after the previous search began. However, the inventors recognize that simply aborting an interrupt search upon the occurrence of another interrupt search trigger event may risk denying service to certain types of interruptions, leaving some interruptions indefinitely pending and rarely processed, which could be a problem for some applications running within a given virtual processor. For example, an interrupt search might involve sequentially examining different portions of a pending interrupt record to check which types of virtual interruptions are pending, so if the interrupt search is continuously aborted due to another interrupt search trigger event, the portion of the pending interrupt record examined later during the search process could track some pending interruptions that were not yet detected because the interrupt search was aborted before reaching that portion of the pending interrupt record. Such an abort mechanism would statistically tend to favor those interruptions recorded in the portion of the pending interrupt record that was searched earliest during the interrupt search process, which may be undesirable as it could lead to other interruptions being denied service.

[0031] Instead, in the example described in more detail below, instead of suspending the current interrupt search when another interrupt search trigger event is received while the current interrupt search is in progress, a completion and restart mechanism is provided, which allows the current interrupt search to complete, but restarts the interrupt search after completion. This is to ensure that the complete pending interrupt map can be checked in the current interrupt search, resulting in less bias towards interrupts recorded in the early parts of the pending interrupt map, thus mitigating the denial-of-service problem.

[0032] Therefore, in response to the detection that another search-triggered event for a given virtual processor has occurred during an interrupt search for that virtual processor, and that a new enabled interrupt has become pending for that virtual processor, the control circuitry can update the pending interrupt status indicator to indicate a re-search state. In other words, when a search-triggered event occurs and the pending interrupt status is the aforementioned "search (enabled)" state, the pending interrupt status indicator can be updated to switch to a re-search state. When the interrupt search is complete and the pending interrupt status indicator indicates a re-search state, the interrupt search circuitry can restart the interrupt search for the given virtual processor, allowing later interrupts to be considered subsequently. Compared to the alternative described above, which involves aborting the current search in response to another search-triggered event, this method reduces the risk of denial of service.

[0033] On the other hand, if a search-triggered event occurs while the pending interruption state indicates the "search" state, the current search can be completed without switching to the re-search state, thus eliminating the need to restart the search upon completion. This is because the "search" state (unlike the "search (enabled)" state) indicates that no other interruption has been received since the start of the currently ongoing search, and therefore performing another search in response to another search-triggered event would be redundant, as the result would be known from the current search. Therefore, latency can be reduced by avoiding performing another search in response to another search-triggered event received while the current search is in progress. This improves performance.

[0034] Virtual interrupts can be associated with corresponding memory addresses. For example, a virtual interrupt can be a message-based interrupt triggered by a received message received by an interrupt distributor. For example, a virtual interrupt can be a message-signaled interrupt according to the PCIExpress (PCIe) standard. The memory address associated with a given message-based interrupt can be an indication of the type of interrupt to be delivered to the target virtual processor. A pending interrupt log can indicate (explicitly or implicitly) the memory address corresponding to a pending virtual interrupt that will be processed by a given virtual processor. For example, a pending interrupt log can provide a bitmap with multiple bits, each bit indicating the presence of a pending interrupt for the corresponding memory address (i.e., the corresponding interrupt type). These bits can be stored within the memory address space and may require a relatively large number of cache lines to store the corresponding bit indications for each type of receivable virtual interrupt, and therefore may require a relatively long search time. The techniques described above can be particularly useful for reducing the latency associated with searching for this type of pending interrupt log.

[0035] In one example, the virtual processor interrupt trace information may also include a coarse enabled pending interrupt record that tracks, at a coarser granularity than a pending interrupt record, which tracks which groups of interrupt types include at least one interrupt type that will be handled by a given virtual processor. Therefore, while the coarse enabled pending interrupt record may be less precise, the search may be faster because each indication in the coarse enabled pending interrupt record may correspond to a larger set of interrupt types than each indication in the pending interrupt record. The coarse enabled pending interrupt record can specify the interrupt types that include those groups of enabled pending interrupts and ignore those groups that only include those of disabled pending interrupts. This allows for faster searching of enabled interrupts, which can be useful for residency changes of a given virtual processor. Interrupt search circuitry can suppress the search for pending interrupts corresponding to a portion of a set of interrupt types for which the coarse enabled pending interrupt record indicates that no enabled pending interrupts will be handled by a given virtual processor. Therefore, the coarse enabled pending interrupt record eliminates the need to search for address blocks within the pending interrupt record that do not correspond to any enabled pending interrupts, thus accelerating the interrupt search.

[0036] In addition to the coarse-grained enabled pending interrupt record, some examples also provide a coarse-grained pending interrupt record that tracks, at a coarser granularity, which groups of interrupt types include at least one address corresponding to a pending interrupt (enabled or disabled) that will be handled by a given virtual processor. Thus, while the coarse-grained enabled pending interrupt record tracks memory address blocks that include enabled pending interrupts, the coarse-grained pending interrupt record can record blocks that include either enabled or disabled pending interrupts. This can be useful if sometimes it is necessary to search for disabled pending interrupts in addition to searching for enabled ones, or to search for changes in the status indication of pending interrupts.

[0037] If it is considered unlikely that a search for disabled pending interruptions will be needed, other methods may not provide a coarse pending interruption record, but only a coarse pending interruption record with pending interruptions enabled.

[0038] The search-triggered event that causes an interrupt search can be a series of different types of events. For example, a search-triggered event could be a reset event that causes the processing system to be reset to a known state, in which case a search for interrupt trace information might be expected, indicating that some virtual interrupts had not been serviced prior to the reset. Alternatively, a search-triggered event could be a change in the operating mode of a physical processor or the execution of an explicit instruction requesting an interrupt search. However, in one example, a search-triggered event could be a virtual processor residency change event that indicates that a given virtual processor will become resided on one of the physical processors.

[0039] In some examples, interrupt trace information may be stored in the memory system. The control circuitry may have access to multiple sets of interrupt trace information, each set corresponding to a specific virtual processor. Since the number of virtual processors can be relatively large, the interrupt allocator and / or control circuitry may not be able to maintain sufficient local storage for interrupt trace information to store information about all virtual processors. Instead, a cache memory can be provided to cache at least a portion of the interrupt trace information for at least a subset of virtual processors, and this information can be obtained from the memory system if interrupt trace information for other virtual processors is needed, although this may be slower than if the information were available in the cache memory.

[0040] In one example, the control circuitry may maintain separate pending interrupt status indicators for the cache memory and memory. The cache pending interrupt status indicator may indicate the pending interrupt status associated with the cache portion of the pending interrupt record for a given virtual processor stored in the cache memory, while the memory pending interrupt status indicator may indicate the pending interrupt status associated with the memory-based portion of that pending interrupt record stored in the memory system for a given virtual processor. This memory-based portion may include the cache portion (because the cache portion may be a copy of the corresponding data stored in memory). However, sometimes the cache portion may only be a subset of the memory-based portion of the pending interrupt record. Additionally, for write-back caches, sometimes, if the cache portion is dirty, the cache portion of the pending interrupt record may differ from the corresponding memory-based portion of that pending interrupt record. By providing separate pending interrupt status indicators corresponding to the cache portion and the memory-based portion, this allows the interrupt search circuitry to make more targeted decisions about whether to search the cache memory and / or memory to discover potentially unresolved pending interrupts for a given virtual processor. For example, if it is known that there are no pending interrupts for a given virtual processor in the cache memory, cache lookups can be omitted and memory can be accessed directly without first going through the cache memory.

[0041] Figure 1An example of a data processing system 2 is shown, comprising multiple integrated circuits (chips) 4 communicating with each other. Each chip includes multiple physical processors (CPUs) 6 and has some on-chip memory 8 accessible by the processors on that chip. Each chip may also have one or more device interfaces 10 through which messages are received from a corresponding device communicating with that interface. For example, the device may be a peripheral device, an input / output controller, a network controller, etc. The number of physical processors 6 and device interfaces 10 on each chip may vary from chip to chip. It should be understood that... Figure 1 The specific number of chips shown is merely an example. Other examples may have only a single chip or may have more or fewer chips.

[0042] Each chip 4 has access to the shared memory 12, which can store data accessible by any of the chips 4. Figure 1 In the example, shared memory 12 is off-chip memory; however, in other examples, shared memory may reside on one of the chips 4 but be accessible by other chips 4. Data can be loaded from shared memory 12 into the on-chip memory 8 of a specific chip. Each physical processor 6 may also have a cache memory that caches data from the on-chip memory 8 or shared memory 12 of that chip 4. It should be understood that... Figure 1 It is a simplified diagram, and although not shown, it provides many other features of the data processing system 2.

[0043] In one example, data processing system 2 can be a virtualized server that can provide services such as web hosting, cloud services, database management, or storage management.

[0044] An interrupt controller is provided to handle interrupts that require interrupt handling on one or more physical processors. For example, an interrupt may be based on a message sent from a given device via device interface 10. Interrupt controllers are assigned such that each chip in chip 4 includes a corresponding general-purpose interrupt controller (GIC) 14, which is responsible for relaying interrupts to the physical processor 6 on the corresponding chip. The GICs 14 on each chip communicate with each other via interrupt interconnect 16. The respective chips 4 may also communicate with each other via memory interconnect 18, which also allows data to be read and written in shared memory 12. Although in Figure 1 In this example, interrupt interconnect 16 and memory interconnect 18 are shown as separate, but in other examples, these interconnects may be combined to form a shared interconnect for both interrupt and memory access.

[0045] Data processing system 2 is a virtualized processing system, and therefore capable of executing multiple virtual processors (or virtual machines), each of which can be mapped to different physical processors in the system, including processors on different chips 4. Some virtual processors may require hardware features available only on certain physical processors, which may limit which physical processors can have virtual processors mapped to them, while other virtual processors may not have such limitations. The decision regarding mapping virtual processors to physical processors can be made by a hypervisor or other software components running on processing system 2.

[0046] Figure 2 An example of a GIC 14 is shown, responsible for allocating interrupts to physical processors 6 on a given chip 4 and virtual processors running on those physical processors 6. GIC 14 includes an interrupt allocator 20, which is responsible for routing decisions about how to route received interrupt requests to the relevant physical processor capable of handling them. Interrupt requests received by allocator 20 may include a physical interrupt signal 22 asserted on a wired path of device 2. Interrupt requests may also include a device interrupt 24 received from the device via device interface 10. As the complexity of data processing systems increases, the increased number of interrupts on a chip can lead to routing congestion and management complexity (in cases where all interrupts on these chips are associated with physical wired paths). Therefore, for some systems, a more efficient way to manage interrupt handling may be to use message-based interrupts (such as Information Signal Interrupts (MSI) for device interfaces implemented using a PCI Express (RTM) bus). Message-based interrupts may be transactions specifying memory addresses, where the specific memory address indicates the type of interrupt being asserted. This allows a single wired communication channel to be shared among multiple interrupt types distinguished by the corresponding addresses of the messages. For a message-based interrupt received as device interrupt 24 from a given device interface 10, the Interrupt Translation Service (ITS) circuit 26 can translate the message into the appropriate interrupt type by looking up the address specified by the message in a table 32 that defines various interrupt types associated with different addresses. For example, the Interrupt Translation Service can generate an interrupt ID based on the message received from the device and can identify which specific target processor should handle the interrupt, which can be a physical processor or a virtual processor. A virtual processor can also be referred to as a "virtual processing element" or simply a VPE.

[0047] GIC 14 has memory access circuitry 18 that controls the reading of data from or writing data to shared memory 12 or the on-chip memory 8 of the corresponding chip 4. GIC 14 may also have internal memory 30 that can be used to cache various tables or other information read from on-chip memory 8 or shared memory 12. Internal memory 30 stores ITS mapping table 32, used by ITS 26 to map message-based interrupts to interrupt IDs and target physical or virtual processors. GIC memory 30 may also cache VPE configuration table 34, which provides virtual-to-physical processor mapping information indicating which physical processor in the entire system 2 a given virtual processor is mapped to. VPE configuration table 34 may be outdated and different from a set of corresponding information stored as shared virtual processor mapping information in shared memory 12, which is accessible by all GICs 14 in the system. For example, this shared information may be a set of Virtual Processor Tables (VPTs) 36, described in more detail below. Although the VPTs primarily reside in shared memory 12, some VPTs or portions of VPTs may optionally be cached in GIC memory 30.

[0048] Distributor 20 uses VPE configuration table 34 to determine how to route interrupts 22 and 24 to their respective destinations. If VPE configuration table 34 indicates that a specific VPE targeted by an interrupt is mapped to a CPU 6 under the responsibility of GIC 14, then the interrupt can be routed to the corresponding physical processor 6. If the target virtual processor is mapped to a CPU 6 on a different chip 4 under a different GIC 14, distributor 20 can route the interrupt to another interrupt controller 14 on another chip 4 via interrupt interconnect 16, and then that other interrupt controller can deliver the interrupt to the desired physical processor.

[0049] To distribute interrupt requests to the processors responsible for a given GIC 14, each physical processor 6 may have a corresponding reallocator 42 responsible for routing signals to that physical processor, and communicate with the reallocator to make more global decisions. The reallocator 42 may be responsible for controlling signal exchange with a specific physical processor and may communicate with the CPU interrupt interface 44 within the corresponding physical processor 6, which can be configured by a hypervisor running on the CPU to control the VPE running on the CPU to handle interrupts.

[0050] GIC 14 includes an interrupt search circuit 39 for performing a search for pending interrupts against a given VPE based on VPT 36, and a control circuit 38 for maintaining VPT 36 and controlling when and how the interrupt search circuit 39 performs the search. Although Figure 2The control circuit 38 and interrupt search circuit 39 are shown to be located within the interrupt distributor 20, but they can also be implemented elsewhere.

[0051] Figure 3 The interrupt translation functionality provided by ITS 26 is illustrated in more detail. The received device interrupt is message-based interrupt 24, which specifies address 50 and some data 52. ITS 26 looks up address 50 in ITS mapping table 32 to identify interrupt type 54, target virtual processor 56 to handle the interrupt, and one or more parameters 58 of the interrupt, which may be based on the data field 52 of message-based interrupt 24 or on information stored in ITS mapping table 32. For example, parameter 58 may indicate interrupt attributes, such as interrupt priority used to control whether the interrupt should preempt other pending interrupts with lower priority. Address 50 associated with message 24 may also be passed by ITS 26, or alternatively may be omitted from the signal passed to distributor 20. Thus, by using address 50 to identify the interrupt type, this allows a single communication channel to be shared among multiple different types of interrupts, which is more efficient in hardware than using dedicated wired channels for different types of interrupts.

[0052] Figure 4 An example of a virtual processor table (VPT) 36 is shown, which provides virtual processor interrupt trace information for a given virtual processor VPEx. Figure 4 As shown, interrupt tracking information may include: a pending interrupt record (also known as a pending interrupt map) 60, which tracks which types of interrupts are pending for a given virtual processor VPEx; a coarse pending map 62, which outlines which address blocks in the pending interrupt map 60 have the indicated pending interrupts; a coarse enabled pending map 64, which indicates which address blocks in the pending interrupt map 60 have enabled pending interrupts (excluding those pending disabled interrupts that may have been masked by interrupt masking information associated with the given virtual processor); and a pending interrupt status indication 66, which indicates one of a variety of finite state machine (FSM) states that track the pending interrupt status of the given virtual processor VPEx.

[0053] Figure 5 Examples of pending interrupt mapping 60, coarse pending mapping 62, and coarse enabled pending mapping 64 are shown. Pending interrupt mapping 60 tracks those pending interrupts that have not yet been delivered to the corresponding virtual processor VPEx. Pending interrupt mapping 60 includes multiple bit flags 68, each bit flag corresponding to a given interrupt type that can be indicated as interrupt type 54 by ITS 26 (or in other words, this can be considered as indicating the address 50 of a pending message-based interrupt). It should be understood that pending interrupt mapping 60 only indicates the existence of pending interrupts. The parameters 58 associated with these interrupts can be stored separately.

[0054] When a given virtual processor VPEx becomes resident on a specific CPU 6, it may be necessary to search the pending interrupt map 60 to identify any pending interrupts that may have been received while the virtual processor was not resident. The pending interrupt map 60 may be stored in shared memory 12 or on-chip memory 8, and reading and traversing the data in the various data blocks of the pending interrupt map 60 to identify which interrupts are pending may be relatively slow. To speed up such searches, a coarse pending map 62 and a coarse enabled pending map 64 may be provided. The coarse pending map 62 includes a plurality of bit flags 70, each bit flag corresponding to a block 72 of bit flags 68 within the pending interrupt map 60. Each coarse pending map bit flag 70 corresponds to the result of an OR reduction operation that combines all bit flags 68 within the corresponding block 72 of the pending interrupt map 60 using a logical OR operation, such that the bit flag 70 in the coarse pending map 62 indicates 1 if one or more indicated pending interrupts exist within the corresponding block 72. If all bit flags in the corresponding block 72 of the pending interrupt map 60 are 0, indicating that there are no pending interrupts within a set of corresponding interrupt types, then the coarse pending map bit flag 70 for that block 72 is 0.

[0055] Similarly, the coarse enable pending map includes multiple bit flags 74, each corresponding to a block 72 of the pending interrupt map 60. However, for the coarse enable pending map 64, the bit flags 74 indicate 1 if at least one enabled pending interrupt exists in the corresponding block 72, but indicate 0 if no enabled pending interrupt exists in the corresponding block 72, even if at least one disabled pending interrupt exists. For example, for Figure 5 In block 72-x, there is a pending interrupt within the block, so the coarse pending mapping flag 70-x is 1. However, if the pending interrupt is a disabled interrupt, the corresponding coarse enabled pending mapping flag 74-x will be 0. When a new pending interrupt is received, the allocator 20 can check whether the corresponding interrupt type is enabled or disabled, and then decide whether to set the coarse enabled pending mapping flag 74 for a set of corresponding interrupt types based on whether the interrupt is enabled or disabled. In contrast, the coarse pending mapping flag 70 can be set regardless of whether the received interrupt is enabled or disabled.

[0056] It should be understood that Figure 5 The specific encoding of 0 and 1 for the corresponding bit flags 70, 74, and 68 in the code is only one example, and other examples may use different encodings (e.g., swapping the meanings of 0 and 1).

[0057] Information in the coarse pending map 62 and the coarse enabled pending map 64 can be used to accelerate the search for the pending interrupt map 60. Some events (such as a VPE becoming residing on a specific physical CPU 6) can be considered as search-triggered events that trigger a search for the pending interrupt map 60 to identify pending interrupts to be delivered to the corresponding VPE. Any identified interrupts can be delivered to the corresponding CPU for that VPE, and then the "pending" flag 68 for the delivered interrupt type can be cleared in the pending interrupt map 60. If the search disables or enables pending interrupts, the interrupt search circuitry 39 can use each coarse pending map bit 70 to determine whether it is necessary to load from memory and search through the corresponding block 72 of the bit flag 68 in the pending interrupt map 60. Similarly, if only enabled pending interrupts are searched (excluding disabled pending interrupts), the interrupt search circuit 39 can use the coarse enabled pending map 64 to determine which blocks 72 of the pending interrupt map 60 need to be loaded and searched, and can exclude from this search any blocks 72 that the coarse enabled pending map 64 indicates do not contain any enabled interrupts. By avoiding the need to load and search some blocks of the pending interrupt map 60, this speeds up the interrupt search and thus reduces downtime during switching between which virtual processor resides on a particular physical processor 6.

[0058] The pending interrupt state 66 indicates one of several states for a given virtual processor VPEx, based on a finite state machine (FSM), such as... Figure 6 As shown in the example. For example, a pending interruption status indicator could be a status indicator that identifies a specific state among these states 80 to 90. In this example, the FSM transitions between the following set of states:

[0059] • Empty 80: Pending interrupt mapping 60 is completely empty and does not indicate any pending interrupts for a given virtual processor VPEx.

[0060] • Pending Disabled 82: Pending interrupt mapping 60 for VPEx indicates at least one pending interrupt, but all pending interrupts indicated in pending interrupt mapping 60 for VPEx are disabled interrupts.

[0061] • Pending Enable 84: Pending Interrupt Mapping 60 indicates that at least one pending interrupt is pending for a given virtual processor VPEx, which includes at least one enabled interrupt.

[0062] • Search 86: An interrupt search for a given virtual processor VPEx is in progress, but no additional enabled interrupts are added to the pending interrupt map 60 while the search is in progress.

[0063] • Search (Enabled) 88: An interrupt search for a given virtual processor VPEx is in progress, and during the search, another enabled interrupt is added to the pending interrupt map 60.

[0064] • Re-search 90: Another search-triggered event occurred while a previous search was being performed. If an interrupt enabled during the search was added to the pending interrupt map 60, there is a risk that the interrupt may be lost. Therefore, when the search is completed, the search should be restarted so that a new interrupt enabled can be considered.

[0065] Control circuitry 38 controls the transitions between FSM states to update the pending interrupt state 66 based on various events occurring for a given virtual processor VPEx. For example, when in an empty state 80, if a disabled interrupt is added to the pending interrupt map 60, the FSM state transitions from empty 80 to pending disabled 82. In contrast, if an enabled interrupt is added, the transition would be from empty 80 to pending enabled 84. If a search-triggered event (such as a given virtual processor VPEx becoming resident) is detected when the FSM state is empty 80, the interrupt search circuitry 39 does not need to actually perform a search, because it is already known from the pending interrupt state 66 that the pending interrupt map 60 will not indicate any pending interrupts.

[0066] When the current state is Pending Disabled 82, if the last pending interrupt is removed from pending interrupt map 60, leaving no pending interrupts for a given virtual processor VPEx, then state 66 transitions back to Empty 80. If an enabled interrupt is added to pending interrupt map 60 while in Pending Disabled 82, then state 66 transitions to Pending Enabled 84. If a search-triggered event occurs indicating that a search will be performed to detect whether any pending enabled interrupts for VPEx exist while the current pending interrupt state is Pending Disabled 82, then no search needs to be performed, as the Pending Disabled state already indicates that no enabled interrupts exist. Alternatively, although in Figure 6 Not shown, but if the system sometimes needs to search for pending disabled interrupts or pending enabled interrupts, the search trigger event can trigger a transition from pending disabled 82 to search 86, but this may not be necessary in a system that only needs to search for pending enabled interrupts.

[0067] If the current state is Pending Enabled 84 and the last pending interrupt has been removed from the pending interrupt map 60, making the pending interrupt map completely empty, then the state transitions back to Empty 80. If the last enabled pending interrupt has been removed but at least one disabled interrupt remains pending, then the control circuit 38 changes the pending interrupt state from Pending Enabled 84 to Pending Disabled 82. If a search trigger event is detected while the current state is Pending Enabled 84, the control circuit triggers the interrupt search circuit 39 to begin searching, and the pending interrupt state 66 switches to search state 86.

[0068] When the current state 66 of VPEx is search 86 and the interrupt search circuit 39 has completed its search, there are two possible transitions. During the interrupt search, any enabled pending interrupts can be identified based on the pending interrupt map 60 and can then be delivered for processing by the corresponding virtual processor VPEx, such that after the search is complete, no enabled pending interrupts remain because they have all been delivered. If there is any interrupt type block for which a coarse pending bit 70 is set, but the coarse enabled pending bit 74 is cleared, it can be determined that at least one pending disabled interrupt remains. If both the coarse pending bit 70 and the coarse enabled pending bit 74 are set for a given block, searching the pending interrupt map 60 for that block can identify which types are pending, and then the corresponding interrupt masking information can be used to distinguish between disabled and enabled interrupts, making it possible to determine whether any pending disabled interrupts remain. Therefore, if at least one disabled interrupt remains pending after the search is completed, the control circuit 38 switches state 66 back to pending disabled 82, while if it is determined after the search that the pending mapping is completely empty, the state returns to empty state 80.

[0069] If, while in the search state, another enabled interrupt is added to the pending interrupt map 60 during the search, control circuitry 38 switches the FSM state 66 to search (enabled) 88 for the corresponding virtual processor VPEx. If, while in the search (enabled) state 88, the currently ongoing search completes, control circuitry 38 switches the FSM state to pending enabled, because the interrupt added during the search will still be unresolved. In this case, there is no need to restart the search, as no other search-triggered event has occurred since the previous search was required.

[0070] On the other hand, if another search trigger event occurs while the search (enabled) state 88 is in, the FSM state 66 is switched to the re-search state 90. This indicates that another search should be triggered once the current search is complete. Therefore, if it is detected that the interrupt search circuit 39 has completed its search and the current state is re-search 90, the control circuit 38 triggers the interrupt search circuit 39 to restart its search and switches the FSM state back to the search state 86. If a late-arriving interrupt added midway through the search has been encountered, and another reason for the search interruption map has been generated, it cannot be assumed that the previously performed search will be sufficient to locate the late-arriving interrupt, and therefore a repeated search may be necessary. However, by allowing the initial search to complete, this means that potential high-priority interrupts can still be detected during the initial search and can still be recorded in some of the later-checked portions of the pending interruption map 60. However, if the initial search has been aborted when another search trigger event is received, these higher-priority interrupts may be at risk of being denied service if they have a lower priority than the newly added interrupts added midway through the earlier search. Therefore, the restart method reduces the risk of denying service to certain types of interrupts.

[0071] Therefore, by providing the finite state machine with transitions between states and recording the current state, which tracks whether a search is in progress and whether there are any pending enabled or disabled interrupts for a given virtual processor, this can help speed up the search, because if the current state is empty 80 or if the search is only for enabling interrupts and the current state is pending disabled 82, the search can be completely eliminated, and furthermore, by providing a coarse enabled pending map 64, this can be further accelerated by allowing some blocks 72 of the known pending interrupt map 60 to be omitted from the search if they do not contain any enabled pending interrupts.

[0072] Figure 4 The example shown contains virtual processor trace information for a single virtual processor; however, it should be understood that multiple sets of virtual processor trace information can be stored in memory designated for each virtual processor. It should also be understood that the specific format of the trace information shown in the example above is merely an example, and other formats may also be used.

[0073] The initialized VPT 36 for each virtual processor can be recorded within the shared memory system 12 and may be cached in the on-chip memory 8 or the interrupt controller memory 30. Typically, the interrupt controller memory 30 may only cache trace information about a specific subset of virtual processors (e.g., a certain number of recently encountered virtual processors). Therefore, when a device interrupt 24 is received that determines it should target a specific virtual processor VPEx, if the trace information 36 about VPEx is currently in the interrupt controller cache memory 30, the FSM state 66 and pending interrupt map 60 in its cache memory can be updated. However, if the received interrupt targets a virtual processor for which the interrupt controller has no cached information, the control circuitry 38 can control the memory access circuitry 18 to write information to memory, causing the pending interrupt to be recorded in the pending interrupt map 60 associated with the target virtual processor. Additionally, the control circuitry 38 can read the pending interrupt state 66 from the memory for the target virtual processor and then cause any necessary updates to be written to the pending interrupt state indication 66.

[0074] like Figure 4 As shown by the dashed lines, in some cases, in addition to the pending interrupt status indication 66 stored in memory in the VPT 36 for a given virtual processor VPEx, the control circuitry 38 of the given interrupt controller 14 may also maintain a separate cached pending interrupt status indication 67. The control circuitry 38 may maintain cached pending interrupt status indications 67 for a certain number of recently encountered virtual processors. For example, these virtual processors may include virtual processors currently residing on the CPU 6 under the interrupt controller's control, as well as one or more recently residing virtual processors. The cached pending interrupt status 67 may differ from the pending interrupt status 66 recorded in the VPT of the corresponding virtual processor in memory. For example, although a given virtual processor resides on a particular physical processor 6, it may also reside elsewhere, so it is not a problem that the VPT 36 in memory has expired relative to the cached pending interrupt status, provided that if the cached pending interrupt status 67 of the given virtual processor held in the interrupt controller memory 30 is refreshed, it is written back to the cached pending interrupt status 66 in memory. By maintaining separate cached pending interrupt status indicators 67 and memory pending interrupt status indicators 66, these indicators can each cycle repeatedly based on any event associated with adding an interrupt to the pending interrupt map 60 stored in the cache memory or memory. Figure 6The FSM state shown allows the interrupt search circuit 39 to decide whether a portion of VPT 36 stored in cache memory or memory needs to be searched as appropriate. Additionally, this allows for deferring memory write-back for performance reasons if it is not already needed.

[0075] It should be understood that Figure 6 The example of the finite state machine in the example is just one example, and other concrete implementations may add additional states or remove them. Figure 6 Some of the states shown. For example, if an implementation never needs to search for interrupts that include both disabled and enabled interrupts, but only supports searching for interrupts that are pending enabled interrupts, then it may not be necessary to distinguish between empty state 80 and pending disabled state 82; instead, these states can be combined into a single state. Therefore, Figure 6 The example does not exclude the possibility of additional or alternative states defined.

[0076] Figure 7 This is a flowchart illustrating the actions taken upon receiving a pending interrupt 24. ITS 26 determines that the pending interrupt is targeted at a given virtual processor VPEx that is currently in a non-resident state. If the target virtual processor is already in a resident state, it can be simply delivered to the corresponding processor 6, and it is not necessary to set the pending bit in the pending interrupt map 60.

[0077] At step 100, a pending interrupt is received targeting a VPEx that is currently in a non-resident state. At step 102, in response, control circuitry 38 triggers the setting of the bit in the pending interrupt map 60 corresponding to the interrupt type of the received interrupt. The mapping function used to map the interrupt type 54 of the received interrupt to a memory address in the memory system storing the corresponding bit in the pending interrupt map 60 can be hardwired (using fixed hardware mapping) or can be changed based on a programmable table accessible by control circuitry 38. At step 102, the update to the pending interrupt map 60 for VPEx can be performed in the cache memory 30 of interrupt controller 14, or in memories 8, 12, depending on whether relevant information about the target VPEx is currently cached.

[0078] At step 104, control circuitry 38 controls the setting of the corresponding bit 70 in the coarse pending map 62, wherein the set bit corresponds to a set of interrupt types including the type of the received interrupt. Similarly, the address mapping function can be used to select which memory address is written to the bit set at step 104, and this update can be performed within cache memory 30 or within memories 8 and 12.

[0079] At step 106, the control circuitry determines whether the received interrupt is enabled or disabled based on the interrupt masking information associated with VPEx. If the received interrupt type is indicated as disabled, at step 108, the control circuitry checks the current pending interrupt states 66 and 67 of the target virtual processor VPEx, and if the current interrupt state is empty 80, it updates it to pending disabled 82; otherwise, if the current state is any state other than empty 80, the current state is not changed.

[0080] If the pending interrupt type of the received interrupt is enabled, at step 110, control circuitry 38 triggers an update to bit 74 of the coarsely enabled pending mapping 64, corresponding to a set of interrupt types including the type of the received interrupt, to indicate that the bit includes at least one enabled pending interrupt within that set of types (if bit 74 has not already indicated this). At step 112, control circuitry also updates the pending interrupt status of the corresponding virtual processor VPEx. If the current status is empty 80 or pending disabled 82, the current status is switched to pending enabled 84. If the status is already pending enabled 84, it remains unchanged. If the current status is search 86, the status 66 of VPEx is updated to indicate search (enabled) status 88.

[0081] Figure 8 This is a flowchart illustrating the process executed when a search trigger event is detected. At step 150, control circuitry 38 detects that a given virtual processor VPEx has encountered a search trigger event. For example, a search trigger event could be that VPEx has been made to reside on one of the physical processors 6 under the responsibility of interrupt controller 14. Therefore, an enable interrupt needs to be searched. During the search, enable interrupts can be identified and delivered for processing by the virtual processor, and these enable interrupts can then be cleared from the recorded trace information 36 regarding that virtual processor VPEx. If a full system search is required, this can be a relatively time-consuming operation, so it is desirable to accelerate this operation if possible.

[0082] Therefore, at step 152, before triggering the search, the control circuit 38 checks the current finite state machine state 66 indicated by the pending interrupt state 66 of VPEx. If necessary, the current finite state machine state is loaded from memory if it is not already cached. If the current state is empty 80 or pending disabled 82, the search operation can be suppressed at step 154 ​​because it is known that there will be no pending enabled interrupts indicated in the pending interrupt map 60. Therefore, this eliminates the delay of fetching blocks of the pending interrupt map 60 from memory systems 8, 12 and checking those blocks to check the bits corresponding to enabled interrupts. Therefore, in this case, the control circuit does not invoke the interrupt search circuit 39, but simply returns a response indicating that there are no pending enabled interrupts.

[0083] If the current pending interruption state indicates that VPEx is currently in the search (enabled) state 88, this means that a previous search is still in progress, but at least one enabled interrupt has been received since the start of that search, making it possible that the enabled interrupt has not yet been detected in that search. Since another search trigger event exists, the interrupt received later should be considered. Therefore, at step 156, control circuitry 38 updates the pending interruption state 66 to indicate a re-search state 90, indicating that another search will be required once the current search is complete.

[0084] On the other hand, if the current state at step 152 is pending enable 84, it means that the previous search was not in progress, thus triggering a new search. At step 158, control circuit 38 updates pending interruption state 66 to indicate that VPEx is in search state 86, and at step 160, control circuit 38 controls interrupt search circuit 39 to start interrupt search, thereby recognizing the received enabled pending interrupt for VPEx.

[0085] Figure 9 Step 160 is shown in more detail. At step 162, a flag (used to track whether any disabled pending interrupts are found in the search) is initialized to a value of 0, indicating that no disabled pending interrupts have been detected so far. At step 164, the interrupt search circuit 39 reads bits 70 and 74 of the coarse pending map 62 and the coarse enabled pending map 64, which correspond to the current block 72 of the next pending interrupt map to be checked. At the first pass through step 164, the current block will be the first block of the pending interrupt map 60. At step 166, the control circuit checks whether the relevant coarse enabled pending map bit 74 for the current block is set, indicating the presence of at least one enabled interrupt indicated in the corresponding block 72 of the pending interrupt map 60. If so, at step 168, the current block of the pending interrupt map 60 is searched (if necessary, the block is loaded from memory if it is not already cached), and any pending enabled interrupts in the block are identified and delivered to the physical processor 6 executing the target virtual processor VPEx. Bit 68 of the pending map 60 corresponding to the delivered interrupt is cleared, and the corresponding coarse enabled pending map bit 74 is also cleared (because all enabled interrupts have been delivered, there are no more enabled pending interrupts in this block of pending map 60). Similarly, at step 172, if any pending interrupt in this block is a disabled interrupt, the disabled pending interrupt discovery flag is set to 1 if it has not already been set to 1, to indicate that at least one disabled interrupt has been found as pending during the search. If no disabled pending interrupt is found, the corresponding bit 70 in the coarse pending map 62 can be cleared.

[0086] Alternatively, if it is determined at step 166 that the relevant bit 74 of the coarsely enabled pending map 64 is set to 0, then at step 174, the search for block 72 is suppressed and there is no need to load or check the block from memory, thus accelerating the search. At step 176, the control circuit 38 or the interrupt search circuit 39 checks the coarsely enabled pending map bit 70 corresponding to the currently processed block 72, and if the coarsely enabled pending map bit is set to 1, the disabled pending interrupt discovery flag is set to 1 to indicate that at least one disabled pending interrupt has been found during the search. It should be noted that by providing a coarsely enabled pending map 62 different from the coarsely enabled pending map 64, this allows the identification of at least one disabled pending interrupt in the corresponding block 72 of the pending interrupt map 60 even if the block is not actually searched.

[0087] Whether the search for the current block is executed or suppressed, at step 178, the interrupt search circuit 39 then detects whether another block 72 of the pending interrupt map 60 is still pending search, and if so, at step 180, the next block to be searched becomes the current block for subsequent passes through steps 164 to 178.

[0088] Ultimately, once all blocks have been searched or it has been determined based on the coarse enabled pending map 64 that no blocks need to be searched, then at step 182, once all blocks 72 of the pending map 60 have been processed and there are no remaining blocks to search, it is determined that the search is complete, wherein any enabled pending interruptions found in the search are delivered.

[0089] return Figure 8 Once the search is complete, at step 190, control circuitry 38 determines the current pending interrupt state 66 of the target processor VPEx that has received its search trigger event. If the current state is "re-search" 90, then... Figure 8 The method returns to step 158 to update the state to search, and restarts another interrupted search at step 160. This means that, if needed, any enabled interruptions received since the start of a previous search can be identified and delivered.

[0090] If the current state is Search (Enabled) 88 at step 190, then at step 192, the current state switches to Pending Enabled 84. If the current state is Search, then at step 194, the Pending Interruption Discovery Flag is disabled (e.g., relative to...). Figure 9The process involves detecting whether any disabled pending interrupts remain, and if so, switching the current state to pending disabled 82 at step 196. If no disabled pending interrupts remain while the current state is searched and the search is complete, the method proceeds from step 194 to step 198, where the current state is updated to indicate empty 80, since there are no longer any pending interrupts for VPEx.

[0091] It should be understood that, in an alternative implementation as described above that does not distinguish between empty 80 and pending disabled state 82, the disabled pending interruption discovery flag is involved. Figure 8 and Figure 9 The steps may be unnecessary. It should also be understood that using a disabled pending interrupt discovery flag to track the presence of disabled pending interrupts is only one possible implementation, and other methods may also be used. For example, instead of setting a sticky flag that accumulates by processing the corresponding block 72 of the pending interrupt map 60, another approach could be to check the coarse pending map 62 independently of the search, and if any bit in the coarse pending map 62 is 1, this would indicate the presence of disabled pending interrupts.

[0092] In addition, although the interrupt search circuit 39 and the control circuit 38 are in Figure 2 In the example shown, it is a hardware circuit, but in other examples, some of the functionality of this circuit can alternatively be implemented by software (e.g., a hypervisor) executing on CPU 6. For example, while hardware circuit 38 may be provided to perform... Figures 7 to 9 The steps are for maintaining trace information 36 about a given VPEx (including maintaining / updating the pending interrupt record 60, coarse pending map 62, coarse enabled pending map 64, and pending interrupt status 66 as described above), but the interrupt search step (including deciding whether to search a given block 72 of the pending interrupt map 60 based on trace information 62, 64, 66) can be implemented in software. Therefore, in some cases, the CPU 6 will act as the interrupt circuit 39 and (partially) the control circuit 38.

[0093] Therefore, in summary, by using a state machine to track pending interrupt states as described above, this allows for the complete elimination of interrupt search when it is already known that no pending enable interrupts for a given virtual processor will exist. Additionally, by providing a coarse enable pending map 64, this allows for the elimination of searches for specific blocks 72 within the pending interrupt map 60, thus accelerating the search process. Either way, this improves performance by reducing the time spent processing the actual program within the virtual processor and decreasing the duration of dwell time when switching between virtual processors.

[0094] In this application, the phrase "configured as..." is used to mean that the elements of the device have a configuration capable of performing the defined operation. In this context, "configuration" means the arrangement or manner of interconnection of hardware or software. For example, the device may have dedicated hardware that provides the defined operation, or a processor or other processing device may be programmed to perform the function. "Configured as" does not mean that the elements of the device need to be changed in any way to provide the defined operation.

[0095] While exemplary embodiments of the invention have been described in detail herein with reference to the accompanying drawings, it should be understood that the invention is not limited to those precise embodiments, and various changes and modifications can be made therein by those skilled in the art without departing from the scope of the invention as defined by the appended claims.

Claims

1. An interrupt tracking device, comprising: An interrupt allocator is used to assign virtual interrupts to one or more physical processors, and each virtual interrupt will be handled by one of a plurality of virtual processors that can be mapped to the one or more physical processors; and Control circuitry, the control circuitry being configured to maintain virtual processor interrupt tracking information corresponding to a given virtual processor, the virtual processor interrupt tracking information including: Pending interrupt logs, which indicate which types of virtual interrupts are pending for the given virtual processor; and A pending interrupt status indication, distinct from the pending interrupt record, indicates the pending interrupt status of a given virtual processor. The pending interrupt status indication specifies whether the number of pending virtual interrupts for the given virtual processor is zero. The pending interrupt status indication indicates one of several states, including at least: A pending enable state, indicating the existence of at least one pending enable interrupt for the given virtual processor, and A pending disable state indicates that one or more disabled pending interrupts are the only pending interrupts for the given virtual processor.

2. The apparatus of claim 1, further comprising an interrupt search circuitry configured to perform an interrupt search in response to a search trigger event associated with the given virtual processor, the interrupt search comprising searching the pending interrupt records to identify pending interrupts to be processed by the given virtual processor.

3. The apparatus according to claim 2, wherein: In response to the search trigger event, when the pending interrupt status indicator indicates that the number of pending interrupts to be processed by the given virtual processor is zero, the interrupt search circuit is configured to suppress the execution of the interrupt search.

4. The apparatus of claim 1, wherein the pending interrupt state indicates whether the number of enabled pending virtual interrupts for the given virtual processor is zero.

5. The apparatus of claim 4, further comprising an interrupt search circuitry configured to perform an interrupt search in response to a search trigger event associated with the given virtual processor, the interrupt search comprising searching the pending interrupt records to identify enabled pending interrupts to be processed by the given virtual processor; wherein: In response to the search trigger event, when the pending interrupt status indicator indicates that the number of enabled pending interrupts to be processed by the given virtual processor is zero, the interrupt search circuit is configured to suppress the execution of the interrupt search.

6. The apparatus of claim 1, wherein each virtual interrupt is associated with a corresponding memory address specified by a received message that triggers the virtual interrupt; and The pending interrupt record indicates the memory address corresponding to a pending virtual interrupt that will be processed by the given virtual processor.

7. The apparatus of claim 1, wherein the virtual processor interrupt tracking information further comprises: A coarse-grained enabled pending interrupt record is used to track which groups of interrupt types include at least one interrupt type at a coarser granularity than the pending interrupt record, and the enabled pending interrupts of the at least one interrupt type will be handled by the given virtual processor.

8. The apparatus of claim 7, further comprising an interrupt search circuitry configured to perform an interrupt search in response to a search trigger event, the interrupt search comprising searching the pending interrupt records to identify enabled pending interrupts to be processed by the given virtual processor; wherein: During the interruption search, the interruption search circuit is configured to determine which parts of the pending interruption record need to be searched based on the coarsely enabled pending interruption record.

9. The apparatus of claim 8, wherein the interrupt search circuitry is configured to suppress a search for the pending interrupt record corresponding to a subset of a set of interrupt types, wherein, for the set of interrupt types, the coarsely enabled pending interrupt record indicates that there is no enabled pending interrupt to be processed by the given virtual processor.

10. The apparatus of claim 1, wherein the virtual processor interrupt tracking information further comprises: A coarse pending interrupt log, which tracks which groups of interrupt types include at least one interrupt type at a coarser granularity than the pending interrupt log, and which enable or disable pending interrupts for the at least one interrupt type will be handled by the given virtual processor.

11. The apparatus of claim 1, wherein the pending interrupt state indicator indicates one of a plurality of states of a pending interrupt state machine associated with the given virtual processor, and the control circuitry triggers a transition between the plurality of states of the pending interrupt state machine in response to an event associated with the given virtual processor.

12. The apparatus of claim 2, wherein the pending interrupt state indicates whether an interrupt search for the given virtual processor is in progress.

13. The apparatus according to claim 12, wherein: In response to the detection that: during the execution of the interrupt search for the given virtual processor, another search trigger event for the given virtual processor has occurred, and the new enabled interrupt has become pending for the given virtual processor since the start of the interrupt search, the control circuitry is configured to update the pending interrupt status indication to indicate a re-search status.

14. The apparatus of claim 13, wherein, in response to completing the interrupt search, when the pending interrupt state indication indicates the re-search state, the interrupt search circuitry is configured to restart the interrupt search for the given virtual processor.

15. The apparatus of claim 2, wherein the search triggering event includes a virtual processor residency change event, the virtual processor residency change event indicating that the given virtual processor will become resided on one of the physical processors.

16. The apparatus of claim 1, wherein the control circuit has access to multiple sets of interrupt tracking information stored in the memory system, each set of interrupt tracking information corresponding to a corresponding virtual processor.

17. The apparatus of claim 16, further comprising a cache memory for caching at least a portion of the interrupt trace information relating to at least a subset of the plurality of virtual processors; and The control circuit is configured to hold: A cache pending interrupt status indicator, the cache pending interrupt status indicator indicating the pending interrupt status associated with the cache portion of the cache record for the given virtual processor in the cache memory; and A memory pending interrupt status indication indicates the pending interrupt status associated with a memory-based portion of the memory system for the pending interrupt record for the given virtual processor.

18. An interrupt tracking device, comprising: A means for assigning virtual interrupts to one or more physical processors, each virtual interrupt being processed by one of a plurality of virtual processors capable of being mapped to the one or more physical processors; and A means for maintaining virtual processor interrupt tracking information corresponding to a given virtual processor, the virtual processor interrupt tracking information including: Pending interrupt logs, which indicate which types of virtual interrupts are pending for the given virtual processor; and A pending interrupt status indication, distinct from the pending interrupt record, indicates the pending interrupt status of a given virtual processor. The pending interrupt status indication specifies whether the number of pending virtual interrupts for the given virtual processor is zero. The pending interrupt status indication indicates one of several states, including at least: A pending enable state, indicating the existence of at least one pending enable interrupt for the given virtual processor, and A pending disable state indicates that one or more disabled pending interrupts are the only pending interrupts for the given virtual processor.

19. An interruption tracing method, comprising: Virtual interrupts are assigned to one or more physical processors, and each virtual interrupt is handled by one of a plurality of virtual processors that can be mapped to the one or more physical processors; as well as Maintain virtual processor interrupt trace information corresponding to a given virtual processor, the virtual processor interrupt trace information including: Pending interrupt logs, which indicate which types of virtual interrupts are pending for the given virtual processor; and A pending interrupt status indication, distinct from the pending interrupt record, indicates the pending interrupt status of a given virtual processor. The pending interrupt status indication specifies whether the number of pending virtual interrupts for the given virtual processor is zero. The pending interrupt status indication indicates one of several states, including at least: A pending enable state, indicating the existence of at least one pending enable interrupt for the given virtual processor, and A pending disable state indicates that one or more disabled pending interrupts are the only pending interrupts for the given virtual processor.

Images