Doorbell Physical Interrupt Control

Abstract

The doorbell physical interrupt control circuit (20) includes an interrupt detection circuit (22) that detects an incoming interrupt generated as a given virtual interrupt (having a given priority) for a given virtual interrupt processing context, and in response to the detection of the incoming interrupt by the interrupt detection circuit, determines whether the given priority of the given virtual interrupt is indicated by the doorbell enable priority configuration data (28) as being valid for doorbell physical interrupt generation, and if so, a doorbell physical interrupt generation circuit (24) that generates a doorbell physical interrupt to be processed in a given physical interrupt processing context. The doorbell physical interrupt indicates to the physical processor that processes an interrupt for a given physical interrupt processing context that the given virtual interrupt is pending for the given virtual interrupt processing context.

Description

Technical Field

[0001] This technology relates to the field of interrupt control.

[0002] In a data processing system, an interrupt controller has a job of detecting an interrupt generated by one or more interrupt sources and signaling the interrupt to a physical processor or virtual processor responsive to the processing of the interrupt.

[0003] At least some embodiments provide a doorbell physical interrupt control circuit comprising an interrupt detection circuit that detects an incoming interrupt generated as a given virtual interrupt of a given virtual interrupt processing context, the interrupt detection circuit for a given virtual interrupt having a given priority, the doorbell physical interrupt control circuit being responsive to the detection of the incoming interrupt by the interrupt detection circuit and determining whether a given priority of a given virtual interrupt is indicated as being valid for doorbell physical interrupt generation by doorbell enable priority configuration data, the doorbell enable priority configuration data generating a doorbell physical interrupt to be processed in a given physical interrupt processing context in response to determining that a given priority of a given virtual interrupt is indicated as being valid for doorbell physical interrupt generation, the doorbell physical interrupt including a physical interrupt indicating that a given virtual interrupt is pending for being processed in a given virtual interrupt processing context by a physical processor that processes interrupts for the given physical interrupt processing context.

[0004] At least some embodiments provide an interrupt controller comprising a virtual interrupt injection circuit that supports direct injection of virtual interrupts to virtual processors executing on a physical processor and the above-described doorbell physical interrupt control circuit.

[0005] At least some embodiments provide an apparatus comprising one or more physical processors, an interrupt controller for distributing virtual interrupts and physical interrupts to the one or more physical processors, and the doorbell physical interrupt control circuit described above.

[0006] At least some embodiments provide a computer-readable medium storing code for manufacturing the doorbell physical interrupt control circuit, the interrupt controller, or the apparatus described above. The computer-readable medium may be a non-transitory storage medium.

[0007] At least some embodiments provide a first method that detects an incoming interrupt occurring as a given virtual interrupt for a given virtual interrupt processing context, the given virtual interrupt having a given priority, and in response to detecting the incoming interrupt, determines whether the given priority of the given virtual interrupt is indicated as being valid for doorbell physical interrupt generation by doorbell enable priority configuration data indicating which virtual interrupt priorities are valid for doorbell physical interrupt generation, and the doorbell enable priority configuration data includes generating a doorbell physical interrupt to be processed in a given physical interrupt processing context in response to determining that the given priority of the given virtual interrupt is indicated as being valid for doorbell physical interrupt generation, the doorbell physical interrupt indicating to a physical processor processing an interrupt for the given physical interrupt processing context that a given virtual interrupt is pending for being processed in the given virtual interrupt processing context.

[0008] At least some embodiments provide a second method, the second method detecting that a first virtual interrupt of a first virtual interrupt processing context is supplied by a first virtual processor that processes virtual interrupts for the first virtual interrupt processing context, and in response to detecting this, determining whether a first priority of the first virtual interrupt is higher than a current priority associated with a process executed by a second virtual processor executing on a given physical processor. The second virtual processor processes virtual interrupts for a second virtual interrupt processing context, and in response to determining that the first priority is higher than the current priority, sets doorbell activation priority configuration data associated with the second virtual interrupt processing context. The doorbell activation priority configuration data indicates, for a given virtual interrupt processing context, which priorities of interrupts are enabled or disabled for the generation of doorbell physical interrupts to indicate that a virtual interrupt is pending, and includes software-programmable data for rescheduling a given physical processor to process the first virtual processor. The doorbell activation priority configuration data associated with the second virtual interrupt processing context is set to indicate that one or more priorities selected based on the first priority of the first virtual interrupt detected for the first virtual interrupt processing context are disabled for the generation of doorbell physical interrupts.

[0009] At least some embodiments provide a computer program that controls a computer to execute the second method described above when executed by the computer. The computer program may be stored in a storage medium. The storage medium may be a non-transitory storage medium.

[0010] Further aspects, features, and advantages of the present technology will become apparent from the following description of examples read in conjunction with the accompanying drawings.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

DETAILED DESCRIPTION OF THE INVENTION

[0012] A physical processor may execute software corresponding to several virtual processors, and each virtual processor (also well-known as a virtual machine) may have characteristics different from those of the host physical processor that actually executes the virtual processor, and may simulate the behavior of the corresponding physical processor. Thus, an interrupt may be signaled either as a physical interrupt to be processed by the physical processor or as a virtual interrupt to be processed by the virtual processor. For example, a physical interrupt may be processed by hypervisor software (software that manages the scheduling of virtual processors running on that physical processor), and a virtual interrupt may be processed by virtual processor software. Interrupt configuration data (e.g., set by hypervisor software) may define which events should be signaled as physical interrupts and which events should be signaled as virtual interrupts. One approach to signaling virtual interrupts may involve the hypervisor updating the interrupt queue of a given virtual processor to signal that there are pending virtual interrupts to be processed by that given virtual processor. Another approach is to use hardware to directly inject virtual interrupts into the queue corresponding to a given virtual processor without involving the hypervisor (except for constructing data that identifies the position of the queue for a given virtual processor to the hardware), which may help to improve performance. However, a problem arises as to what to do when a given virtual processor targeted by a virtual interrupt generated by an interrupt controller is not currently scheduled to execute on any physical processor when the hypervisor is not involved in the assignment of virtual interrupts to the interrupt queue of the virtual processor.Pending virtual interrupts may represent high-priority events that need to be processed quickly, but when the hardware simply updates the virtual interrupt waiting queue structure and signals the interrupt to a non-resident virtual processor that is not currently executing, it may not be well-known that it is desirable to reschedule which virtual processor is executing on the physical processor to enable the interrupt to be delivered. One approach to address this is to respond to the detection of an interrupt event to be signaled as a virtual interrupt by generating a physical interrupt called a "doorbell" physical interrupt, where the physical interrupt signals to the physical processor that there is a pending virtual interrupt for a given virtual processor that is not currently resident on the physical processor. The doorbell physical interrupt may prompt the hypervisor to reschedule execution on the physical processor so that a given virtual processor can execute and process the virtual interrupt.

[0013] However, a problem with systems that generate doorbell physical interrupts in response to the detection of pending virtual interrupts is that the doorbell physical interrupt may cause the processing of higher-priority interrupts within one virtual processor to be interrupted and preempted by a doorbell physical interrupt generated in response to a lower-priority interrupt detected for another virtual processor. Using a shared priority space between virtual and physical interrupts may seem attractive to solve this problem, but this is impractical as it would require changing all of the software of the virtual processors at the hypervisor level to cooperate in interrupt priority assignment, breaking the illusion that virtual processors are executing in separate isolated environments, which is important for many hypervisor designs.

[0014] In the embodiments described below, the doorbell physical interrupt control circuit includes an interrupt detection circuit that detects an incoming interrupt that occurs as a given virtual interrupt (having a given priority) for a given virtual interrupt processing context, and in response to the detection of the incoming interrupt by the interrupt detection circuit, determines whether a given priority of a given virtual interrupt is indicated as being enabled for doorbell physical interrupt generation according to doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation, and if so, a doorbell physical interrupt generation circuit that generates a doorbell physical interrupt to be processed in a given physical interrupt processing context. The doorbell physical interrupt indicates to a physical processor that processes an interrupt for a given physical interrupt processing context that a given virtual interrupt is pending for a given virtual interrupt processing context.

[0015] Accordingly, a circuit is provided for controlling whether a doorbell physical interrupt is generated in response to a given virtual interrupt. Whether a doorbell physical interrupt is generated depends on doorbell enable priority configuration data that specifies which priority level of virtual interrupts is enabled to generate the corresponding doorbell physical interrupt. Accordingly, a mechanism is provided to prevent a doorbell physical interrupt from being generated in response to a virtual interrupt of a given level of priority that is shown to be disabled by the doorbell enable priority configuration data, which may be useful, for example, to prevent a lower priority virtual interrupt for one virtual processor from causing a doorbell physical interrupt that could interrupt a higher priority process being executed by another virtual processor. This approach does not require any cooperation between the hypervisor software developer and the virtual processor software developer when setting the priority level for a particular interrupt and can avoid a shared priority space between physical and virtual interrupts.

[0016] Each interrupt processing context refers to a specific group of interrupts that can be prioritized relative to each other based on the software-defined priority assigned to each interrupt for that interrupt processing context. The software-defined priority for one interrupt processing context may be independent of the software-defined priority for another interrupt processing context.

[0017] In some implementations, each physical interrupt processing context may correspond to a specific physical processor, and each virtual interrupt processing context may correspond to a specific virtual processor. In some examples, the configuration information available to the doorbell physical interrupt control circuit (or the interrupt controller described below) may be able to identify the physical / virtual processor corresponding to a specific physical / virtual interrupt processing context.

[0018] However, in other implementations, the doorbell physical interrupt control circuit (or the interrupt controller described below) may not actually recognize which specific physical or virtual processor is handling the physical or virtual interrupt for a specific interrupt processing context. Thus, the physical / virtual interrupt processing context may be an abstraction that identifies a given set of interrupts, and different physical / virtual processors may be configurable to monitor a specific data structure in memory used to identify pending interrupts for a given (physical or virtual) interrupt processing context. The doorbell physical interrupt control circuit (or the interrupt controller) may not know which specific physical or virtual processor is monitoring a specific set of data structures for a given interrupt processing context. Instead, the doorbell physical interrupt control circuit (or the interrupt controller) may be able to identify the interrupt processing context based on, for example, one or more addresses that identify the data structure used to track the interrupts of that interrupt processing context.

[0019] Doorbell activation priority configuration data can be implemented in different ways. For example, the doorbell activation priority configuration data can be either software-maintained information set by software or hardware-maintained information automatically maintained by hardware without explicit software intervention (however, the hardware can still control the hardware implementation update of the doorbell activation priority configuration data based on the configuration data set by software). Also, the doorbell activation priority configuration data can be associated with a given virtual interrupt processing context in which a given virtual interrupt is detected, or can be associated with a physical processor that receives a doorbell physical interrupt when it is generated. Various embodiments of the doorbell activation priority configuration data are described below.

[0020] In some embodiments, the doorbell activation priority configuration data includes software-programmable doorbell activation priority configuration data associated with a given virtual interrupt processing context. The software-programmable doorbell activation priority configuration data indicates, for a given virtual interrupt processing context, which priorities of interrupts are enabled or disabled for doorbell physical interrupt generation. This approach can enable the hypervisor to set the software-programmable doorbell activation priority configuration data for the originating virtual processor when switching which virtual processors are resident on the physical processor, reducing the chance of higher-priority processing on the newly resident virtual processor being preempted due to lower-priority virtual interrupts for the originating virtual processor.

[0021] Thus, doorbell activation priority configuration data can be defined in relation to a given virtual interrupt processing context. Thus, different virtual interrupt processing contexts can have different settings in which virtual interrupt priorities are enabled for the generation of doorbell physical interrupts. Depending on the doorbell activation priority configuration data associated with those specific virtual interrupt processing contexts, a virtual interrupt of a given priority occurring for one virtual interrupt processing context may trigger the generation of a doorbell physical interrupt, while a virtual interrupt of the same priority occurring for another virtual interrupt processing context may not trigger the generation of a doorbell physical interrupt.

[0022] Thus, the doorbell physical interrupt generation circuit can be configured to select software-programmable doorbell activation priority configuration data from among two or more sets of doorbell activation priority configuration data associated with each virtual interrupt processing context, based on a context identification value associated with a given virtual interrupt processing context.

[0023] The context identification value used to select which set of software-programmable doorbell activation priority configuration data to use can include a virtual processor identifier indicating the virtual processor that processes the virtual interrupts for a given virtual interrupt processing context. Alternatively, the context identification value can include at least one address indicating a memory location of a virtual interrupt tracking structure for tracking pending virtual interrupts related to a given virtual interrupt processing context.

[0024] A set of software-programmable doorbell activation priority configuration data can be defined in one or more memory-based tables stored in a memory system. In this case, data from those tables can also be cached in a cache local to the doorbell physical interrupt control circuit, such that more recently accessed doorbell activation priority configuration data can be accessed more quickly than the underlying data in memory.

[0025] For example, each virtual interrupt processing context can have a corresponding set of memory-based interrupt tracking data structures used to indicate pending virtual interrupts for that virtual interrupt processing context, although one of these structures can also provide software-programmable doorbell activation priority configuration data for that virtual interrupt processing context (thus, each set of doorbell activation priority configuration data for different virtual interrupt processing contexts can be taken from different sets of interrupt tracking data structures).

[0026] Alternatively, a data structure (e.g., the doorbell configuration information can specify a physical interrupt identifier to use for the doorbell physical interrupt) that defines doorbell configuration information used to specify details of the doorbell physical interrupt generated in response to a virtual interrupt for a given virtual interrupt processing context can also specify the software-programmable doorbell activation priority configuration data for that virtual interrupt processing context.

[0027] Alternatively, multiple sets of software-programmable doorbell activation priority configuration data can be defined in a register storage device. For example, a set of memory-mapped configuration registers may be associated with the doorbell physical interrupt control circuit and can be configured by software on the processor by issuing memory read / write operations to the addresses mapped to those registers.

[0028] The software-programmable doorbell activation priority configuration data can be any information that can be set by software to enable the software to specify whether the virtual interrupt of a given priority should generate the corresponding doorbell physical interrupt within a given interrupt processing context. This can be represented in different ways.

[0029] In one embodiment, the software-programmable doorbell activation priority configuration data includes a threshold, and the doorbell physical interrupt generation circuit determines that a given priority is enabled for doorbell physical interrupt generation if the given priority has a higher priority than the doorbell physical interrupt generation threshold priority indicated by the threshold. The threshold can be an efficient way to encode the software-programmable doorbell activation priority configuration data to indicate whether each virtual interrupt priority should be treated as enabled or disabled for the purpose of generating a doorbell physical interrupt based on the comparison between a given priority and the threshold. Thus, priorities with a higher priority than the threshold priority represented by the threshold are considered enabled for doorbell physical interrupt generation, and priorities with a lower priority or the same priority as the threshold priority are considered disabled for doorbell physical interrupt generation.

[0030] The term "higher priority" is used to indicate an interrupt that is more important than a lower priority interrupt. That is, an interrupt with a higher priority can preempt a lower priority interrupt within the same interrupt processing context. Note that different encoding schemes can be used to encode the priority level for a given interrupt. In some embodiments, an interrupt assigned a priority level with a higher numerical value can be considered to have a higher priority than an interrupt assigned a priority level with a lower numerical value (e.g., priority 4 can be considered more important than priority 3). Other embodiments can use a lower numerical value to indicate an interrupt with a higher priority (e.g., priority 3 is a higher priority than priority 4). Thus, the term "higher priority" refers to the level of importance of an interrupt, rather than the magnitude of the numerical value used to represent the priority level.

[0031] Note that the threshold priority can be represented by a threshold in different ways. In some embodiments, the threshold can specify the threshold priority itself, and in other embodiments, the threshold can specify a priority one level higher than the threshold priority (in which case, a comparison value greater than or equal to between a given priority and the threshold can indicate whether the given priority is a higher priority than the threshold priority, or, if a lower numerical value encodes a higher priority than a higher numerical value, the comparison value less than or equal to can be used). Also, the threshold can have an encoding that indirectly represents the threshold level rather than directly indicating it (e.g., selecting from among a number of predetermined thresholds). Thus, generally, the threshold priority can be specified in different ways, and a wide range of different types of comparisons can exist that can be used to evaluate whether a given priority is a higher priority than the threshold priority indicated (explicitly or implicitly) by a threshold.

[0032] When the priority of the doorbell physical interrupt generation threshold is higher than a given priority, the doorbell physical interrupt generation circuit can suppress the generation of the doorbell physical interrupt even when a given virtual interrupt is signaled as pending in a given virtual interrupt processing context. Therefore, the doorbell physical interrupt generation threshold priority does not control whether a virtual interrupt is signaled as pending at all, but is a control specific to the generation of the doorbell physical interrupt in response to a virtual interrupt of a given virtual interrupt priority. When a virtual interrupt is signaled as pending but the corresponding doorbell physical interrupt is not generated, the corresponding virtual processor may not be prompted to be scheduled by the hypervisor, so the processing of that virtual interrupt may be delayed. However, ultimately, when the virtual processor becomes resident on a given physical processor again, the pending virtual interrupt can be processed by that virtual processor.

[0033] In some embodiments, for a physical interrupt, a physical interrupt mask threshold priority for comparing with the priority of a given physical interrupt can be defined to determine whether to signal the given physical interrupt to a physical processor that processes the interrupt for the corresponding physical interrupt processing context. Therefore, some embodiments can provide the ability to suppress, even slightly, the signaling of a physical interrupt having a priority lower than or equal to the physical interrupt mask threshold priority to the physical processor (even if they are pending). In such a system, the physical interrupt mask threshold priority is a threshold defined separately from the doorbell physical interrupt generation threshold priority. The reason they are different is that the doorbell physical interrupt generation threshold priority is compared with the priority of the virtual interrupt to determine whether to signal the doorbell physical interrupt to the physical processor, while the physical interrupt mask threshold priority is compared with the priority of the physical interrupt to determine whether to signal that physical interrupt to the physical processor.

[0034] The threshold associated with a given virtual interrupt processing context can be settable to specify a priority other than the priority associated with the current point of the program flow reached by the virtual processor that processes the interrupts of the given virtual interrupt processing context, as the doorbell physical interrupt generation threshold priority. This can ensure that, as long as a sufficiently high interrupt does not occur for a given virtual interrupt processing context, the hypervisor uses the threshold of the given virtual interrupt processing context to prevent another virtual processor that processes high-priority interrupts for different virtual interrupt processing contexts from receiving interrupts. Thus, the priority defined as the threshold can be useful because it can depend greatly on what is being executed by other virtual processors rather than the current point of the program flow reached by the virtual processor corresponding to the given virtual interrupt processing context.

[0035] The priority threshold is not the only way to define doorbell enablement priority configuration data for the purpose of doorbell physical interrupt generation to indicate which priorities are enabled or disabled.

[0036] In another example, software-programmable doorbell enablement priority configuration data includes a set of priority indicators, each corresponding to a respective priority and indicating whether that priority is enabled or disabled for doorbell physical interrupt generation. For example, the doorbell enablement priority configuration data can be a bitmap where each bit corresponds to a given priority level (or group of priority levels) and indicates whether that priority level or group of priority levels is enabled for doorbell physical interrupt generation (such that virtual interrupts at that priority level trigger the generation of doorbell physical interrupts) or disabled for doorbell physical interrupt generation (such that virtual interrupts at that priority level do not trigger the generation of doorbell physical interrupts). This can allow for finer control over which priority levels cause the generation of doorbell physical interrupts.

[0037] In some embodiments, the doorbell activation priority configuration data includes a current virtual interrupt priority indication indicating the virtual interrupt priority of the current processing on a resident virtual processor that resides on a physical processor that is signaled when a doorbell physical interrupt is generated.

[0038] The physical processor may have hardware circuit logic that maintains a current virtual interrupt priority indication for tracking the virtual interrupt priority of the current processing on a resident virtual processor that currently resides on the physical processor. A resident virtual processor is a virtual processor that is currently assigned to that physical processor. A resident virtual processor does not necessarily have to be currently executing any instructions (e.g., during a system call to the hypervisor, for example, the instructions can be executed from the hypervisor, but the resident virtual processor that made the system call can still be considered to be resident).

[0039] Generally, the current priority execution indication of the processing executed on the resident virtual processor can be useful for the hypervisor to make scheduling decisions. Thus, some hardware circuit logic can be provided within the physical processor to respond to events indicating a change in virtual interrupt priority (such as when a virtual interrupt occurs or when returning to the previous processing after completion of the virtual interrupt processing) by updating the current virtual interrupt priority indication according to the virtual interrupt priority of the processing being executed after the event.

[0040] In some embodiments, the in - flight virtual interrupt priority indication for the physical processor that will receive the doorbell physical interrupt can be used by the doorbell physical generation circuit as doorbell enable priority configuration data to determine whether to generate the doorbell physical interrupt. Thus, in response to the detection of an incoming interrupt, the doorbell physical interrupt generation circuit, when generated, determines whether the given priority of a given virtual interrupt is high enough to preempt the current processing on the resident virtual processor based on the in - flight virtual interrupt priority indication for the physical processor to which the doorbell physical interrupt will be signaled. And the in - flight virtual interrupt priority indication can suppress the generation of the doorbell physical interrupt in response to determining that the given priority of a given virtual interrupt is not high enough to preempt the current processing on the resident virtual processor. By considering the virtual priority of the processing currently being performed on the resident virtual processor in the physical processor that receives the doorbell physical interrupt, this can lower the likelihood that the doorbell physical interrupt generation circuit will generate a doorbell physical interrupt that results in a lower - priority virtual interrupt for a non - resident virtual processor that causes an interrupt to a higher - priority processing on the resident virtual processor.

[0041] The different approaches discussed for implementing the doorbell enable priority configuration data can be implemented separately or in combination.

[0042] Accordingly, in some embodiments, the doorbell enablement priority configuration data includes software-programmable doorbell enablement priority configuration data but does not include virtual interrupt priority indications in execution. This may provide a more easily scalable approach for handling different numbers of physical processors within the system (the amount of configuration data considered by the doorbell physical interrupt control circuit may be independent of the number of physical processors provided). Also, it enables software to set doorbell enablement priority configuration data for a particular virtual interrupt processing context based on any particular knowledge of the current processing on the corresponding virtual processor. For example, if it is known that a particular virtual processor has no processing to perform, the doorbell enablement priority configuration data for the virtual interrupt processing context corresponding to that particular virtual processor may be set to disable a larger subset of virtual interrupt priorities for doorbell physical interrupt generation, reducing the likelihood that doorbell physical interrupts will be generated in response to virtual interrupts occurring for that virtual processor.

[0043] As further described below, the doorbell activation priority configuration data includes software-programmable doorbell activation priority configuration data, but in some embodiments that do not include the virtual interrupt priority indication during execution, the virtual interrupt priority indication during execution is still maintained by a given physical processor and can be used by the hardware to control reprogramming of the software-programmable doorbell activation priority configuration data for one or more virtual processors. Thus, even if the virtual interrupt priority indication during execution is not directly considered by the doorbell physical interrupt control circuit (e.g., to improve scalability by not requiring the virtual interrupt priority indication during execution of each physical processor to be routed to the doorbell physical interrupt control circuit), the virtual interrupt priority indication during execution can still indirectly affect doorbell physical interrupt generation.

[0044] In other embodiments, the doorbell activation priority configuration data may include the virtual interrupt priority indication during execution, but may not include the software-programmable doorbell activation priority configuration data. This may enable the doorbell physical interrupt generation to more accurately respond to changes in the priority associated with the current process executed by the resident virtual processor in a given physical processor, reducing the likelihood that the current process will be preempted by a lower-priority virtual interrupt that occurs for a non-resident virtual processor.

[0045] Other embodiments may consider both software-programmable doorbell enablement priority configuration data associated with a given virtual interrupt processing context and a running virtual interrupt priority indication associated with a physical processor that receives a doorbell physical interrupt when generated. In this case, the doorbell physical interrupt is generated in response to an incoming interrupt in response to a determination that a given priority of a given virtual interrupt is of sufficient priority to preempt the current processing (of the resident virtual processor interrupts on the physical processor that will receive the doorbell physical interrupt) as indicated by the running virtual interrupt priority indication, and that software-programmable data associated with the given virtual interrupt processing context indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation. By considering both types of doorbell enablement priority configuration data, more accurate control of doorbell physical interrupt generation is possible (e.g., when setting software-programmable doorbell enablement priority configuration data, both reacting to changes in the execution virtual priority in a given physical processor, which may be difficult to track using any software-specific knowledge of the operating state of the virtual processor that may be used by the hypervisor, and using software-programmable doorbell enablement priority configuration data associated with a specific virtual interrupt processing context), reducing the probability of interrupting higher-priority processing due to doorbell physical interrupts generated in response to lower-priority virtual interrupts.

[0046] There are several ways to distribute the doorbell physical interrupt to the physical processor. For example, the doorbell physical interrupt generation circuit can distribute the doorbell physical interrupt to the physical processor by asserting a physical interrupt signal on an interrupt bus or by updating at least one memory-based physical interrupt tracking structure to indicate that the doorbell physical interrupt is pending. Which physical processor receives the doorbell physical interrupt can be defined by doorbell physical interrupt configuration information (which can be configured independently of the virtual processing context and doorbell enable priority configuration data), or can be unknown to the doorbell physical interrupt control circuit (when updates to the memory-based tracking structure are used to signal pending doorbell physical interrupts, the doorbell physical interrupt control circuit may not know which physical processors are monitoring those tracking structures). The doorbell physical interrupt can have its own physical priority level, which can be independent of the priority associated with a given virtual interrupt that caused the physical interrupt to be generated.

[0047] The doorbell physical interrupt control circuit can be implemented in various different parts of the data processing system.

[0048] In one embodiment, the doorbell physical interrupt control circuit may also be part of an interrupt controller that also includes a virtual interrupt injection circuit that supports direct injection of virtual interrupts into virtual processors running on a physical processor. An interrupt controller that supports direct injection of virtual interrupts has hardware that can signal an interrupt to a virtual processor without the intervention of a hypervisor when signaling an interrupt. For example, the hardware of the interrupt controller can issue a memory access request to update a virtual interrupt queue monitored by the virtual processor, which can indicate that a virtual interrupt is pending in the virtual interrupt queue. When actually signaling the virtual interrupt, there is no involvement of the hypervisor, but this does not exclude the hypervisor from being involved in setting configuration information that controls how the hardware injects the interrupt into the virtual processor (for example, the hypervisor can set the address of the virtual interrupt queue used for interrupts of the virtual processor). By providing an interrupt controller with support for direct injection of virtual interrupts to the doorbell physical interrupt control circuit, this helps to avoid the problem that the interrupt controller causes preemption of higher-priority virtual interrupts due to detection of lower-priority virtual interrupts by the doorbell physical interrupt.

[0049] Alternatively, the doorbell physical interrupt control circuit may also be provided in other parts of the data processing device other than the interrupt controller. For example, the doorbell physical interrupt control circuit can be part of the physical processor to determine whether to respond by detecting a virtual interrupt injected by the interrupt controller and asserting a doorbell physical interrupt to its physical processor. Also, the doorbell physical interrupt control circuit may be a dedicated doorbell physical interrupt generation unit separate from both the physical processor and the interrupt controller.

[0050] Thus, generally, the apparatus may include one or more physical processors, an interrupt controller that distributes virtual interrupts and physical interrupts to the one or more physical processors, and a doorbell physical interrupt control circuit. The doorbell physical interrupt control circuit may be configured by the interrupt controller, or by at least one of the one or more physical processors, or by a dedicated doorbell physical interrupt generation unit.

[0051] Each physical processor can handle physical interrupts as having a higher priority than virtual interrupts, regardless of the priority specified by software for the physical interrupt or virtual interrupt. Generally, since hypervisor events are often considered more important than events targeting a specific virtual machine, by default, it may be useful to handle physical interrupts as having a higher priority than any virtual interrupt (in extreme cases, it is conceivable that the lowest-priority physical interrupt has a higher priority than the highest-priority virtual interrupt). This avoids the need for coordination between the hypervisor software developer and each virtual processor software, but can lead to the problem that a lower-priority virtual interrupt for one virtual processor may preempt the processing of a higher-priority virtual interrupt by another virtual processor, potentially generating a high-priority doorbell physical interrupt. This problem can be addressed by providing doorbell enable priority configuration data used to determine whether to signal a doorbell physical interrupt as described above. Thus, the above-described technique can be particularly useful in a system that treats physical interrupts as having a higher priority than virtual interrupts, regardless of the priority level of the interrupt specified by software.

[0052] In an implementation that includes doorbell activation priority configuration data, which is software-programmable doorbell activation priority configuration data associated with a given virtual interrupt processing context (indicating which interrupt priorities are enabled or disabled for doorbell physical interrupt generation for a given virtual interrupt processing context), when determining whether to generate a doorbell physical interrupt, if it is desired to also consider the current virtual priority in the physical processor that receives the doorbell physical interrupt, another approach can be that when an event indicating a change in the current virtual interrupt priority occurs in the physical processor, the software-programmable doorbell activation priority configuration data can be reprogrammed. This can be a way to avoid the physical routing cost of signals passing the current virtual priority indication of each physical processor to the doorbell physical interrupt control circuit while still enabling doorbell interrupt generation to be controlled based on the current virtual priority of the physical processor.

[0053] Accordingly, a given physical processor can include a doorbell activation priority configuration data reprogramming circuit for detecting an event indicating a change in the current virtual interrupt priority associated with the given physical processor, which indicates the virtual interrupt priority associated with the current processing by a resident virtual processor resident on the given physical processor, and in response to detecting the event indicating a change in the current virtual interrupt priority, reprogramming a target set of software-programmable doorbell activation priority configuration data for one or more virtual interrupt processing contexts based on the updated value of the current virtual interrupt priority after the event.

[0054] The doorbell activation priority configuration data reprogramming circuit can identify a target set of software-programmable doorbell activation priority configuration data based on software-programmable control information. For example, the software-programmable control information can be any information that enables the hardware to identify an address (an address corresponding to the software-programmable doorbell activation priority configuration data referred to for one or more virtual processors) of a memory location updated based on the current virtual interrupt priority after an event indicating a change in the virtual interrupt priority during execution. For example, the software-programmable control information can directly specify those addresses or can specify information (such as a list of virtual processor identifiers or virtual interrupt processing context identifiers) that can be looked up in another data structure to obtain the address of a location in memory updated with the updated current virtual interrupt priority. The hypervisor software can set the software-programmable control information, for example, when rescheduling which virtual processors are resident on a physical processor.

[0055] A corresponding method is provided, which, in response to detecting that a first virtual interrupt of a first virtual interrupt processing context is supplied by a first virtual processor that processes virtual interrupts for the first virtual interrupt processing context, · determining whether a first priority of the first virtual interrupt is higher than a current priority associated with a process executed by a second virtual processor executing on a given physical processor, where the second virtual processor processes virtual interrupts for a second virtual interrupt processing context, and determining whether the first priority of the first virtual interrupt is higher than the current priority associated with the process executed by the second virtual processor executing on the given physical processor, ● in response to determining that the first priority is higher than the current priority, ● Configuring doorbell activation priority configuration data associated with a second virtual interrupt processing context, wherein the doorbell activation priority configuration data indicates, for a given virtual interrupt processing context, which priority levels of interrupts are enabled or disabled for the generation of doorbell physical interrupts in order to indicate that a virtual interrupt is pending, and includes software-programmable data; configuring doorbell activation priority configuration data associated with a second virtual interrupt processing context; ○ Rescheduling a given physical processor to process a first virtual processor; ○ The doorbell activation priority configuration data associated with the second virtual interrupt processing context is set to indicate that one or more priority levels selected based on a first priority level of a first virtual interrupt detected for the first virtual interrupt processing context are disabled for the generation of doorbell physical interrupts. This method can be executed by software for configuring doorbell activation priority configuration data. This method is based on an implementation form in which software-programmable doorbell activation priority configuration data is set corresponding to a given virtual interrupt processing context.

[0056] For example, this method can be executed by hypervisor software that is responsive to scheduling regarding which virtual processor to execute on a given physical processor. Therefore, the hardware of the data processing device may not have dedicated circuit logic for executing the steps of this method. Rather, the hardware provides, for example, an architectural function for responding to doorbell activation priority configuration data set by software by including the above-described doorbell physical interrupt control circuit.

[0057] In some embodiments, based on the first priority of the first virtual interrupt, one or more priorities selected to be disabled for generating the doorbell physical interrupt may include priorities lower than the first priority or equal to the first priority. Thus, this ensures that only virtual interrupts with a priority higher than the first priority (which caused the preemption of the second virtual processor) will cause the doorbell physical interrupt to be generated when subsequent virtual interrupts are detected for the second virtual interrupt processing context processed by the second virtual processor. This approach can be assumed if it can be considered that interrupts of a given priority in one virtual interrupt processing context are as important as interrupts of the same priority in another virtual interrupt processing context (this can be a default assumption since a particular virtual processor may use the priority levels in a particular way not recognized by the hypervisor).

[0058] In other embodiments, when the hypervisor recognizes the relative importance of interrupts of a given priority on one virtual processor and interrupts of a given priority on another virtual processor, there can be a conversion between the priority levels of the respective virtual processes. Thus, one or more priorities for which the generation of the doorbell physical interrupt is disabled can be the converted priorities within the second virtual interrupt processing context that are considered to be lower than or equal to the first priority of the first virtual interrupt within the first virtual interrupt processing context. Therefore, when the hypervisor recognizes otherwise that interrupts of some priorities higher than the first priority in the second virtual interrupt processing context should be prioritized earlier than interrupts of the first priority in the first virtual interrupt processing context, the priority selected based on the first priority does not necessarily have to be exclusively lower than the first priority.

[0059] Here, specific embodiments will be described with reference to the figures. It will be understood that the claims are not limited to these embodiments.

[0060] FIG. 1 schematically shows an embodiment of a data processing system 2 (e.g., a system-on-chip) comprising a plurality of processors 4 (e.g., a Central Processing Unit (CPU)). In this embodiment, three processors 4 are shown, but it will be understood that the number of processors can be changed. The processors communicate with each other and with a shared memory 8 via a cache coherent interconnect 6 that supports a coherent protocol for maintaining cache coherence of data cached in the private cache of each processor 4.

[0061] The interrupt controller 10 receives incoming interrupt signals from the connected peripheral devices 14 and transfers them to the processors 4. In some cases, the processors also generate interrupts for other processors, known as Inter-Processor Interrupts (IPIs), so that the processors 4 themselves can also function as interrupt sources. The peripheral devices 14 can signal interrupts to the interrupt controller 10 via dedicated lines 11 or by reusing existing I / O mechanisms 12 such as memory-mapped I / O (Input / Output) write operations. The latter is typically known as Message-Signalled Interrupt (MSI). The job of the interrupt controller is to prioritize interrupts according to a configuration executed by software and ensure that higher-priority interrupts are presented to the processors ahead of lower-priority interrupts. In modern multiprocessor systems with two or more processors, the interrupt controller may also route interrupts to one or more specified processors and handle reprogramming of the routing configuration without losing interrupt signals.

[0062] Therefore, the interrupt controller needs to communicate with a processor that can transfer interrupts. As shown in FIG. 1, one way to achieve this is to use a dedicated communication protocol and an interrupt communication bus 16 within the system, which is specifically designed to carry interrupt signals from the interrupt controller 10 to the processor 4. The interrupt communication bus 16 is completely separate from the cache coherent interconnect 6 that is used to transfer memory transactions among the processor 4, the memory 8, and the interrupt controller 10 (the interrupt controller 10 can still have an interface to the cache coherent interconnect 6 to enable programming of configuration information that controls how the memory transactions issued by the processor 4 affect the transfer of interrupts from the interrupt controller 10 to the processor 4).

[0063] Another approach for distributing interrupts to processors is that the existing cache coherent mechanism supported by the cache coherent interconnect 6 is utilized for distributing interrupts from the interrupt controller 10 to the processors 4, and the interrupt configuration and state can be represented by a memory-based table shared by the processors and the interrupt controller. By storing an interrupt tracking data structure (e.g., a queue of pending interrupts and other configuration states such as interrupt priorities and enable states for different interrupt identifiers) in memory and using the cache coherent protocol of the cache coherent interconnect 6 to ensure consistency between the views of the shared tracking data structure seen by the interrupt controller 10 and the processors 4, the interrupt tracking data structure is shared between the interrupt controller 10 and the processors 4, which means that when the state of an interrupt changes as a result of a message or signal received by the interrupt controller 10, the interrupt controller can simply update the tracking data structure stored in memory to cause the interrupt to be delivered to the correct processor. This can be done without the need for a dedicated interrupt bus 16 by using the existing cache coherent interconnect mechanism supported by the interconnect 6. The processor 4 that monitors the tracking structure for updates can ensure that the address being monitored is held in its cache in an "exclusive" coherent state, which causes any write to that address by the interrupt controller to cause the cache coherent interconnect 6 to snoop the processor's cache, detect it, and trigger an invalidation that causes the data for that address to be requester-returned to the processor's cache in an exclusive state, whereby the processor can inspect the updated data and use it to signal an exception to the processor, but the processor can more closely inspect the updated tracking structure to determine how to handle any interrupts signaled as pending.This coherent-based approach can scale chip designs to different numbers of processors and interrupts, reducing the design effort associated with scaling, since it does not require wiring expansion of the dedicated interrupt distribution bus 16 and can scale by changing the size or number of the data structures stored in memory.

[0064] In the following examples, either the interrupt distribution bus 16 or a memory write via the cache coherent interconnect 6 can be used to distribute virtual or physical interrupts to the processor 4. It is not essential to use the same approach for both physical and virtual interrupts. For example, physical interrupts can be distributed via the interrupt communication bus 16 and virtual interrupts can be distributed using the interconnect 6. Alternatively, both physical and virtual interrupts can use the same type of distribution mechanism, but can be distinguished by the physical signals exchanged via the interrupt communication bus or by updating different sets of interrupt tracking structures (a set of physical interrupt tracking structures for a given physical interrupt processing context that a given processor 4 can be configured to monitor updates for, or a set of virtual interrupt tracking structures for a given virtual interrupt processing context that a given virtual processor executing on a given physical processor can be configured to monitor updates for). When using memory writes via the cache coherent interconnect 6 to distribute both virtual and physical interrupts, the interrupt communication bus 16 can be omitted.

[0065] The device 2 may include a doorbell physical interrupt control circuit 20 for controlling the generation of a doorbell physical interrupt that notifies the processor that there is a pending virtual interrupt to be processed by a virtual processor that is not currently resident on the processor. As shown by the dotted line in FIG. 1, the doorbell physical interrupt control circuit 20 may be implemented at different locations within the data processing system 2. For example, the doorbell physical interrupt control circuit 20 may be designed to respond to a virtual interrupt caused by an interrupt controller by generating a doorbell physical interrupt to be signaled to the processor 4, either as a stand-alone doorbell physical interrupt unit or as part of each physical processor 4, and may be provided within the interrupt controller 20. These are all alternative locations for the doorbell physical interrupt control circuit 20, and thus it will be understood that in a given system, the doorbell physical interrupt control circuit 20 is not provided at all of these locations.

[0066] FIG. 2 shows the doorbell physical interrupt control circuit 20 in more detail. The doorbell physical interrupt control circuit includes an interrupt detection circuit 22 that detects the occurrence of an incoming interrupt to be signaled as a virtual interrupt. When the doorbell physical interrupt control circuit 20 is provided within the interrupt controller 10, the interrupt detection circuit 22 may detect the occurrence of the interrupt based on a signal asserted on the interrupt wire 11 or using a message that signals an interrupt represented by a memory write request issued on the I / O bus 12. When the doorbell interrupt control circuit 20 is provided as a stand-alone unit or within the processor 4, the interrupt detection circuit 22 may detect the virtual interrupt based on a physical signal asserted on the interrupt communication bus 16 by the interrupt controller 10 (in an embodiment where a coherence mechanism is used to deliver interrupts from the interrupt controller 10 to the processor as described above), or based on an update made by the interrupt controller 10 to a memory address allocated to represent interrupt tracking data.

[0067] The doorbell physical interrupt control circuit 20 also includes a doorbell physical interrupt generation circuit 24 that, in response to the interrupt detection circuit 22, determines whether a doorbell physical interrupt should be signaled and detects the incoming interrupt to be signaled as a virtual interrupt, and, if so, generates a doorbell physical interrupt and signals it to the physical processor.

[0068] When the doorbell physical interrupt control circuit 20 is part of the interrupt controller 10, the interrupt controller may also have a virtual interrupt injection circuit 26 that, in response to the detection of an interrupt generated by the peripheral device 14 or other source of interrupt events, injects a virtual interrupt into the virtual interrupt queue associated with the corresponding virtual processor. Thus, the interrupt controller supports the direct injection of virtual interrupts, in which case hardware is provided that directly updates the memory structure representing the queue of virtual interrupts for a given virtual processor without the need for hypervisor involvement to update that queue. The virtual interrupt injection circuit 26 is shown in FIG. 2 as part of the doorbell physical interrupt control circuit 20, but may also be provided in other parts of the interrupt controller 10. The virtual interrupt injection circuit 26 is not provided in the doorbell physical interrupt control circuit 20 when the doorbell physical interrupt control circuit 20 is implemented within the processor 4 or as a stand-alone unit.

[0069] The doorbell physical interrupt generation circuit has access to the doorbell enable priority configuration data 28 and the doorbell configuration data 30. In this embodiment, both data sets are software programmable structures that can be programmed by software executed on the processor 4, for example, by a hypervisor that manages the execution of virtual processors (as will be described later with respect to FIGS. 9-12, other embodiments may use hardware maintained information maintained by a physical processor as doorbell enable priority configuration data). In the embodiment of FIG. 2, the doorbell enable priority configuration data 28 and the doorbell configuration data 30 can be maintained in a register of the doorbell physical interrupt control circuit 20 or in a memory-based data structure accessible by issuing a memory request. In the case of a memory-based structure, information from the memory-based data structure can also be cached in a cache local to the doorbell physical interrupt control circuit 20. Memory management permissions defined in the processor 4 using a page table can be used to control the right to update the doorbell enable priority configuration data 28 and the doorbell configuration data 30, for example, restricting the permission to set the doorbell enable priority configuration data to the hypervisor and denying access to the virtual processor itself.

[0070] The doorbell configuration data 30 can define information used to define the doorbell physical interrupt to be generated when a given virtual interrupt is detected. For example, the doorbell configuration data 30 can specify the physical interrupt number of the doorbell physical interrupt and can specify information identifying the physical interrupt processing context in which the doorbell physical interrupt is generated (for example, this information can identify a specific physical processor, an identifier of a node on the interrupt communication bus 16 to which the doorbell physical interrupt is delivered, or the base address of a memory-based interrupt tracking structure that is updated to queue the doorbell physical interrupt).

[0071] In the embodiment of FIG. 2, the doorbell activation priority configuration data 28 comprises several sets of software-programmable data, each set corresponding to a respective virtual interrupt processing context (e.g., a corresponding virtual processor), and providing data that defines, for each respective virtual interrupt priority, whether each priority is enabled or disabled for the generation of a doorbell physical interrupt. The doorbell activation priority configuration data 28 has an encoding that allows for the definition that a first subset of interrupt priorities is to be enabled for the generation of a doorbell physical interrupt, and a second subset of interrupt priorities is to be defined as being disabled for the generation of a doorbell physical interrupt. Since there are multiple sets of doorbell activation priority configuration data 28, each corresponding to a different virtual interrupt processing context, different virtual interrupt processing contexts may have different settings regarding which interrupt priorities are enabled / disabled for the generation of a doorbell physical interrupt. The doorbell physical interrupt generation circuit 24 selects which set of the doorbell activation priority configuration data 28 to select based on the context identifier associated with the virtual interrupt processing context in which an incoming interrupt is detected by the interrupt detection circuit 22. For example, the context identifier may be a virtual processor identifier or the address of a tracking structure used to represent a pending interrupt for the virtual interrupt processing context.

[0072] The doorbell activation priority configuration data 28 can be represented in different ways. For example, in one implementation form, for each set of doorbell activation priority configuration data, a threshold value that identifies a priority threshold used to determine whether a doorbell physical interrupt should be generated in response to a specific virtual interrupt can be provided. A priority higher than the priority threshold may be considered valid for doorbell physical interrupt generation, and a priority lower than the priority threshold (and a priority equal to the priority threshold) may be considered invalid for doorbell physical interrupt generation. Therefore, when an incoming interrupt corresponding to a given virtual interrupt is detected by the interrupt detection circuit 22, the priority of the given virtual interrupt can be compared with the threshold value of the corresponding virtual interrupt processing context (or a threshold value derived from the threshold value), and the comparison result can be used to determine whether to trigger a doorbell physical interrupt.

[0073] In another approach, the doorbell activation priority configuration data 28 for a given virtual interrupt processing context can specify a set of indicators (e.g., a bitmap) where each indicator corresponds to a given priority level and specifies whether an interrupt of that priority should be enabled or disabled for doorbell physical interrupt generation.

[0074] The doorbell activation priority configuration data 28 is useful for addressing issues related to preemption of high-priority interrupts that can occur in a system that supports direct injection of virtual interrupts. This problem is illustrated in FIG. 3. For example, consider two separate virtual processors vPE A and vPE B, where vPE A has virtual interrupts INT0 and INT2, and vPE B has virtual interrupt INT1. Further, consider that INT0 has a higher priority than INT1, and INT1 has a higher priority than INT2 (a smaller number indicates a higher priority). All physical interrupts are considered to have a higher priority than all virtual interrupts (since hypervisor events may be considered more important than events processed by a given virtual processor).

[0075] Consider the following sequence of events on a single physical processor (pPE) 4 shown in FIG. 3. 1. vPE A is executing in a non-interrupt context, and the interrupt controller (or doorbell physical interrupt control circuit 20) is configured to generate a physical doorbell when INT1 is asserted for vPE B. 2. INT1, which generates a physical doorbell to the hypervisor that preempts the execution of vPE A, is asserted. 3. The hypervisor schedules vPE B to handle the more important interrupt INT1 and configures the doorbell to be generated when a virtual interrupt (e.g., INT0 or INT2) is asserted to be processed by vPE A. 4. vPE B begins executing the interrupt handling logic for INT1. 5. INT2, which generates a physical doorbell to the hypervisor that preempts the execution of vPE B, is asserted.

[0076] Thus, the lower-priority interrupt INT2 causes preemption of the vPE that was handling the higher-priority interrupt INT1.

[0077] This problem can be addressed by providing doorbell enablement priority configuration data 28, because this enables the hypervisor, when configuring the doorbell for vPE A, to define which priority of virtual interrupts for vPE A should generate the doorbell physical interrupt, for example, by defining that one or more lower priorities are to be disabled for doorbell physical interrupt generation.

[0078] FIG. 4 illustrates a method for processing an incoming interrupt. In step 100, an incoming interrupt is detected, which is generated as a given virtual interrupt for a given virtual interrupt processing context. The given virtual interrupt has a given priority. In step 106, the doorbell physical interrupt generation circuit 24 determines whether doorbell enable priority configuration data 28 (which is doorbell enable priority configuration data for a given virtual interrupt processing context in the embodiment of FIG. 2) indicates that the given priority is valid for doorbell physical interrupt generation. If it is indicated that the given priority is valid for doorbell physical interrupt generation, in step 108, the doorbell physical interrupt generation circuit 24 generates a doorbell physical interrupt to be processed in a given physical interrupt processing context. Doorbell configuration data 30 can be used to determine which physical interrupt processing context should be used to generate other information about the doorbell physical interrupt, such as the doorbell physical interrupt and its interrupt identifier. The doorbell physical interrupt identifies that a given virtual interrupt is pending for a given virtual interrupt processing context to be processed by the physical processor that is processing the interrupt of the given physical interrupt processing context. For example, the doorbell physical interrupt may reschedule with the hypervisor which virtual processor is executing on the physical processor, such that the virtual processor that processes the interrupt for a given virtual interrupt processing context can process the given virtual interrupt. In step 106, if the doorbell physical interrupt generation circuit 24 determines that the given priority is disabled for doorbell physical interrupt generation, in step 110, the doorbell physical interrupt is not generated.

[0079] In some embodiments, the priority mask may also be supported to define a threshold priority for determining whether to signal a pending physical interrupt to the corresponding physical processor that processes the corresponding physical interrupt for the corresponding physical interrupt processing context, as compared to the priority of the pending physical interrupt. In such embodiments, when masking based on the priority mask is enabled, in step 108, a further check may also be performed to determine whether the priority of the generated doorbell physical interrupt is higher than the threshold physical interrupt priority represented by the priority mask. Note that the threshold priority defined by the priority mask is a threshold separate from the threshold defined by the doorbell activation priority configuration data 28 in embodiments that use a threshold to implement doorbell activation priority configuration data. This is because the (physical interrupt) priority mask is a control applied to physical interrupts (generally not just doorbell interrupts), and the priority mask threshold is compared with the priority of the physical interrupt to determine whether the physical interrupt should be signaled to processor 4. In contrast, when the doorbell activation priority configuration data is defined using a threshold, the threshold is compared with the priority of the virtual interrupt to determine whether a doorbell physical interrupt should be generated.

[0080] FIG. 5 is a flowchart showing a method of configuring the doorbell activation priority configuration data 28 in the embodiment of FIG. 2. This method is executed by software executed on processor 4. For example, the method may be executed by hypervisor software. It will be understood that the hardware of device 2 does not have any hardware circuitry for executing this method.

[0081] In step 150 of FIG. 5, the hypervisor detects that a first virtual interrupt has occurred for a first virtual interrupt processing context in which interrupts are processed by a first virtual processor. For example, the hypervisor can detect the first virtual interrupt because the first virtual interrupt causes the physical processor executing the first virtual processor to deliver a doorbell physical interrupt, which causes the hypervisor to execute. The first virtual interrupt has a first priority. In response to the detection of the first virtual interrupt, in step 152, the hypervisor detects whether the first priority is higher than the current priority associated with processing on a second virtual processor that was resident on a given physical processor (executing prior to the interrupt to process the hypervisor) and is processing virtual interrupts for a second virtual interrupt processing context. If the first priority is not higher than the current priority of processing on the second virtual processor, in step 158, the hypervisor returns the processing to the second virtual processor, and the second virtual processor continues processing the task with the current priority.

[0082] On the one hand, when the first priority is higher than the current priority, in step 154, the hypervisor configures the doorbell configuration data 30 to define a doorbell physical interrupt generated in response to a virtual interrupt detected for the second virtual interrupt processing context (such that when the interrupt is processed by the second virtual processor, it is not resident on the physical processor). The hypervisor also sets the doorbell enable priority configuration data 28 associated with the second virtual interrupt processing context such that one or more priorities selected based on the first priority (of the first virtual interrupt detected for the first virtual interrupt processing context) are indicated as being disabled for the generation of the doorbell physical interrupt. Therefore, note that the selection of the priority of the second virtual interrupt processing context for which the doorbell physical interrupt is disabled depends on the first priority of the virtual interrupt detected for the first virtual interrupt processing context (rather than the current priority of the previously processed processing by the second virtual processor that processes the interrupt for the second virtual interrupt processing context). This is because the purpose of disabling the doorbell physical interrupt for the second virtual interrupt processing context is to protect the first virtual interrupt processing context from interrupts based on interrupts that are not more important than the interrupts being processed by the first virtual interrupt processing context. That is, when the first virtual interrupt preempts processing on the second virtual processor, any virtual interrupt for the second virtual processor that is considered less important than the first virtual interrupt should preferably not cause the doorbell physical interrupt to be generated because the doorbell physical interrupt would interrupt higher-priority processing currently scheduled on a given physical processor using the first virtual processor.

[0083] In some embodiments, the priority at which doorbell physical interrupt generation is disabled for the second virtual interrupt processing context may simply be a priority that is less than or equal to the first priority. This approach can assume that a given priority within the first virtual interrupt processing context may be considered to have the same importance as the same priority within the second virtual interrupt processing context.

[0084] In other embodiments, when the hypervisor recognizes the mapping between the relative importance of the priority levels of the first interrupt processing context and the priority levels of the second interrupt processing context, the hypervisor may select, as the priority at which to disable the generation of the doorbell physical interrupt, the priority of the second virtual interrupt processing context that maps to a priority that is less than or equal to the first priority in the first virtual interrupt processing context.

[0085] In step 156, when configuring the doorbell interrupt and setting the doorbell enable priority configuration data 28 associated with the second virtual interrupt processing context, the hypervisor re-schedules which virtual processor is being executed on a given physical processor, such that the first virtual processor is processed instead of the second virtual processor. Here, the first virtual processor may process the first virtual interrupt detected in step 150.

[0086] FIG. 6 is a ladder diagram showing the same scenario as FIG. 3, but here, when a virtual interrupt having an interrupt priority indicated by the doorbell enable priority configuration data 28 (where the corresponding doorbell physical interrupt is configured using the doorbell configuration data 30) is detected, the doorbell enable priority configuration data 28 is applied to suppress the generation of the doorbell physical interrupt, although the virtual interrupt is disabled for doorbell physical interrupt generation.

[0087] Thus, in FIG. 6 1. vPE A is executed in a non-interrupt context, and the interrupt controller (or doorbell physical interrupt control circuit 20) is configured to generate a physical doorbell when INT1 is asserted for vPE B. 2. INT1 that generates a physical doorbell to the hypervisor that preempts the execution of vPE A is asserted. 3. The hypervisor schedules vPE B to process the more important interrupt INT1 and configures the doorbell to be generated when a virtual interrupt is asserted to be processed by vPE A. At this time, the hypervisor also sets the doorbell enable priority configuration data 28 to indicate that priorities below INT1 are disabled for the generation of doorbell physical interrupts. 4. vPE B starts executing the interrupt handling logic for INT1. 5. INT2 is asserted, but unlike in FIG. 3, the doorbell enable priority configuration data 28 specifies that interrupt priority 2 is disabled for the generation of doorbell physical interrupts, so no doorbell physical interrupt is generated. Thus, vPE B is not interrupted and can continue to process the higher-priority interrupt INT1. In contrast, if the interrupt asserted for vPE A is the higher-priority interrupt INT0, a doorbell physical interrupt is still generated.

[0088] FIG. 7 shows a first embodiment of an interrupt control configuration table that can be used to define priority configuration data 28 with doorbell labels. Interrupts generated by interrupt source 14 specify a device identifier (ID) that identifies the interrupt source and an event ID that can be used to identify the type of event that occurred (to select the interrupt identifier of the interrupt signaled in response to that event). The device ID is used to select an Interrupt Translation Table (ITT) base address 200 from among a number of base addresses defined for different devices (interrupt sources). Thus, different ITT structures can be used for different interrupt sources, which is useful because different interrupt sources can reference different events that need to be mapped to different interrupts using the same event ID.

[0089] The event ID is used to select an entry from the ITT 202 accessed using the base address 200, and the selected ITT entry provides a number of information regarding the interrupt to be generated, including, for example, the following. ● The interrupt ID of the interrupt generated in response to the event, ● An indication of whether the interrupt should be signaled as a virtual interrupt or as a physical interrupt, ● When the interrupt is signaled as a virtual interrupt, the virtual processor ID (vPE ID) indicates the virtual processor (virtual interrupt processing context) that supplies the virtual interrupt, and when the doorbell interrupt is configured for this virtual interrupt, the doorbell interrupt ID is used for the doorbell physical interrupt generated when this virtual interrupt occurs. ● When an interrupt is signaled as a physical interrupt, the collection ID (or interrupt redistribution device ID that identifies the node of the integrated circuit to which the interrupt is assigned by the interrupt controller) used to select an entry from the collection table 206 that maps each collection ID to the identifier of the corresponding physical processor. Thus, each collection ID corresponds to the collection of interrupt IDs assigned to the same physical processor. This helps with the physical routing of interrupts in embodiments that use the interrupt distribution bus 16 for interrupt distribution. In embodiments that use memory writes and coherence mechanisms to spread interrupts via the cache coherent interconnect 6, the collection table 206 may instead provide the base address of a data structure used to track the physical interrupts of a given physical interrupt processing context in which the physical interrupt occurs.

[0090] In the case of a virtual interrupt, the virtual processor ID selects an entry from the virtual processor table 204 that provides information including the following. ● The base address of the interrupt reservation table of the virtual processor to be injected with the pending virtual interrupt. ● In one embodiment, the target physical processor ID (or alternatively the interrupt redistributor ID) that identifies the physical processor or circuit node to which the doorbell interrupt is to be assigned using the interrupt distribution bus 16 (alternatively, the vPE table 204 may identify the base address of a data structure used to track the physical interrupts for a given physical interrupt processing context in which the doorbell physical interrupt is to be caused). ● Doorbell enable priority configuration data 28 (virtual interrupt processing context) for the corresponding virtual processor.

[0091] Thus, in this example, the doorbell enable priority configuration data 28 for a given virtual interrupt processing context is provided by an entry in the virtual processor table 204 corresponding to the virtual processor that processes the interrupts of the given virtual interrupt processing context.

[0092] It should be understood that the table structures and information provided in each table as shown in FIG. 7 are merely examples, and other embodiments may compile this information in different ways or provide other types of information in the table. Tables 202, 204, and 206 may be memory-based table structures that can be set by software to control interrupt routing.

[0093] FIG. 8 shows a second embodiment of a data structure that can be used to control the generation of doorbell physical interrupts. This embodiment can be used when the doorbell physical interrupt control circuit 20 is implemented in the processor 4 or as a stand-alone unit separate from the interrupt controller 10. In this embodiment, the interrupt controller 10 distributes virtual and physical interrupts by issuing a memory write request on the cache coherent interconnect 6 to update the corresponding memory-based interrupt tracking structures 250, 252, whereby the interconnect snoops the caches of one or more processors 4 that hold the corresponding address in an exclusive state in the cache, and the processor 4 can detect the invalidation of the cache data triggered by the snoop, whereby the processor 4 can know about the pending interrupts. Thus, when a virtual interrupt needs to be signaled, the interrupt controller 10 writes to the set of interrupt tracking structures 250 for a given virtual interrupt processing context in which the virtual interrupt occurs. The interrupt detection circuit 22 of the doorbell physical interrupt control circuit 20 detects pending virtual interrupts, for example, based on a snoop generated by the coherent interconnect 6, which is that the doorbell physical interrupt is configured using the doorbell configuration data 30, and when the doorbell enable priority configuration data 28 for a given virtual interrupt processing context specifies that a given priority of a given virtual interrupt is enabled for doorbell physical interrupt generation, causes the doorbell physical interrupt generation circuit 24 to generate a doorbell physical interrupt. When a doorbell physical interrupt is generated, the doorbell physical interrupt generation circuit 24 writes to the set of interrupt tracking structures 252 of the corresponding physical interrupt processing context, signaling that the doorbell physical interrupt is pending. Thus, a physical processor 4 that monitors the address in that structure 252 can detect the physical interrupt, which can prompt the hypervisor in that physical processor to reschedule virtual processor execution to process the original virtual interrupt signaled by the interrupt controller 10.

[0094] FIG. 8 shows two embodiments of how doorbell activation priority configuration data 28 can be provided to the doorbell physical interrupt control circuit 20. In one embodiment, the doorbell activation priority configuration data 28 can be provided within a set of interrupt tracking structures 250 of a given virtual interrupt processing context. For example, an interrupt descriptor table that defines information (e.g., valid status, priority) regarding each interrupt identifier of the corresponding virtual interrupt processing context can include a value that specifies a doorbell interrupt generation priority threshold, or can also specify an indicator of whether each priority is valid / invalid.

[0095] In another embodiment, the doorbell activation priority configuration data 28 can be provided within a part of a doorbell configuration data structure 30 that is used to define other information for generating a doorbell physical interrupt. For example, the doorbell configuration data structure 30 can include an address 260 of a virtual interrupt processing structure 250 for the virtual interrupt processing context in which a doorbell interrupt is to be generated, a physical interrupt number 262 of the doorbell interrupt, and a physical interrupt tracking structure address 264 that identifies a physical interrupt tracking structure 252 in which the doorbell interrupt is to be signaled as pending. Thus, when a virtual interrupt is detected from the interrupt tracking structure 250 corresponding to the address 260 specified by a given entry of the doorbell configuration data structure 30, the priority of the virtual interrupt is compared with the doorbell available priority configuration data 28 within that entry to determine whether the priority is available for doorbell physical interrupt generation, and if available, the doorbell physical interrupt having the physical identifier 262 is signaled by updating the tracking structure 252 represented by the address 264.

[0096] There may also exist other ways in which the doorbell activation priority configuration data 28 can be provided. For example, the doorbell activation priority configuration data 28 can be defined in a further data structure stored in memory, separate from both the virtual interrupt tracking structure 250 and the doorbell configuration data structure 30. Also, the doorbell activation priority configuration data 28 can be stored in the memory map register of the doorbell physical interrupt control circuit 20.

[0097] FIG. 9 shows another embodiment of the system 2 that is the same as FIG. 1, but in this embodiment, each physical processor 4 also maintains an execution virtual interrupt priority indication 300 indicating the virtual interrupt priority of the current processing on the resident virtual processor resident on that physical processor. In this embodiment, the execution virtual interrupt priority indication 300 maintained by each physical processor can be disclosed to the doorbell interrupt control circuit 20 for use in determining whether to generate a doorbell physical interrupt that is signaled to the corresponding physical processor.

[0098] FIG. 10 shows an example of hypervisor-maintained information indicating whether an in-execution virtual interrupt priority indication for a given physical processor should be used to determine whether to generate a doorbell physical interrupt for that given physical processor. This information is accessible to the doorbell interrupt control circuit 20 and can be set by hypervisor software running on one of the physical processors 4 (e.g., by writing to a memory address assigned to a structure providing hypervisor-maintained information). As shown in FIG. 10, for each virtual processor (or each virtual interrupt processing context), the doorbell physical interrupt control circuit 20 has access to a corresponding in-execution priority control indicator (e.g., a flag) 302 set by the hypervisor to indicate whether the generation of the doorbell physical interrupt should depend on the in-execution virtual interrupt priority indication 300 of the physical processor to which the doorbell physical interrupt would be signaled if the doorbell physical interrupt were generated. The hypervisor may set the flag 302, for example, so that the resident virtual processor does not have its execution priority control flag 302 set, and non-resident virtual processors for which doorbell generation should not depend on the in-execution virtual interrupt priority indication 300 also do not have their execution priority control flags 302 set, but non-resident virtual processors for which the in-execution virtual interrupt priority indication 300 should be considered when generating a doorbell interrupt may have their execution priority control flags 302 set.

[0099] Accordingly, when a virtual processor (or virtual interrupt processing context) for which a virtual interrupt is to be signaled as pending has the corresponding execution priority control flag 302 set, when a physical processor to which a doorbell interrupt is to be sent is identified (e.g., based on a collection table 206 looked up based on the doorbell interrupt identifier shown in FIG. 7), the execution virtual interrupt priority indication 300 of that physical processor is checked to identify the priority of the execution virtual interrupt in that physical processor. The doorbell interrupt control circuit 20 uses the virtual priority in execution to determine whether the virtual interrupt has a high enough priority to justify generating a doorbell physical interrupt. For example, the doorbell interrupt control circuit 20 may assume that both resident and non-resident virtual processors that process incoming interrupts have the same level of priority set, and thus, the virtual interrupt to be signaled may generate a doorbell physical interrupt if it has a higher priority than the priority indicated by the virtual interrupt in execution in the physical processor to which the doorbell physical interrupt would be sent if generated.

[0100] As shown by the dotted line in FIG. 10, in some embodiments, in an embodiment that uses the virtual priority indication in execution to control the generation of doorbell physical interrupts, it is not necessary to provide software-specified doorbell enablement priority configuration data 28 (provided for each virtual interrupt processing context).

[0101] Other embodiments may still provide doorbell enablement priority configuration data 28 for each virtual interrupt processing context and determine whether to generate a doorbell physical interrupt based on both the software-specified doorbell enablement priority configuration data 28 for a given virtual interrupt processing context in which a virtual interrupt is to be signaled as pending and the current virtual interrupt priority of the corresponding physical processor to which the doorbell physical interrupt is to be sent. In that case, the doorbell physical interrupt is suppressed if either the software-specified doorbell enablement priority configuration data 28 for a given virtual interrupt processing context or the current virtual interrupt priority of the corresponding physical processor indicates that the virtual interrupt to be signaled as pending in a given virtual interrupt processing context has an insufficient priority to justify generation of the doorbell interrupt. As described above for the configuration data 28 itself, the current priority control flag 302 may be maintained in various structures, such as an interrupt tracking structure 250, or a doorbell configuration data structure 30.

[0102] Figure 11 illustrates a method. Steps 100, 108, and 110 are the same as in Figure 4. In this embodiment, in response to a received interrupt being detected at step 100, at step 306, an optional check is included to confirm whether software-specified doorbell activation priority configuration data 28 designated for a given virtual interrupt processing context indicates that a given priority is valid for doorbell physical interrupt generation. This is similar to step 106 of Figure 4 in the embodiment of Figure 2 that provides software-specified doorbell activation priority configuration data 28. If the software-specified doorbell activation priority configuration data 28 indicates that a given priority is invalid for doorbell physical interrupt generation (e.g., if a given priority does not exceed a threshold indicated by the software-specified doorbell activation priority configuration data 28), at step 110, the doorbell physical interrupt is not generated. When the software-specified doorbell activation priority configuration data 28 indicates that a given priority is enabled for doorbell physical interrupt generation, the method proceeds to step 310. Also, in embodiments that do not support software-specified doorbell activation priority configuration data 28, step 306 is omitted and the method proceeds directly from step 100 to step 310.

[0103] At step 310, the doorbell interrupt control circuit 20 determines whether an execution priority control flag 302 of a given virtual interrupt processing context is set by hypervisor software. If not, the method proceeds to step 108 and a doorbell physical interrupt is generated in the same manner as described for step 108 of Figure 4.

[0104] In operation 310, when doorbell interrupt control circuit 20 determines that an execution priority control flag 302 for a given virtual interrupt processing context is set by hypervisor software, in operation 312, doorbell interrupt control circuit 20 obtains an execution virtual interrupt priority indication 300 of physical processor 4 that is signaled when a doorbell physical interrupt is generated. Doorbell interrupt control circuit 20 determines whether a given priority of a given virtual interrupt is sufficient to preempt the current processing on the resident virtual processor based on the priority (of the current processing on the resident virtual processor residing on that physical processor) indicated by the currently executing virtual interrupt priority indication 300. Some implementations may assume that the resident virtual processor and the virtual processor to process a given virtual interrupt have the same virtual interrupt priority, and thus, doorbell interrupt control circuit 20 may determine that a given priority is sufficient to preempt the current processing when the given priority is higher than the priority indicated by the currently executing virtual interrupt priority indication 300. Other embodiments may support a conversion of priority levels indicating how to map between the priority level of the resident virtual processor and the priority level of the virtual processor to process a given virtual interrupt using software-configurable mapping information.

[0105] In implementations that implement both checks in operations 306 and 310 / 312, FIG. 11 shows that these are first performed continuously in operation 306 and then, when passing through operation 306, are performed in operation 310 / 312, although other embodiments may perform the checks in the reverse order or in parallel.

[0106] FIG. 12 is a ladder diagram showing a scenario in which the ability to consider the execution virtual priority can be utilized when determining whether to generate a doorbell physical interrupt. In this embodiment, for simplicity of explanation, each virtual processor vPE A, vPE B is assumed to have a virtual interrupt priority set such that the priority level X of the virtual processor vPE A can be regarded as equivalent to the same priority level X of the virtual processor vPE B. In this embodiment, the physical processor (pPE) is currently processing an instruction from the resident virtual processor (vPE A). The current process corresponds to a virtual interrupt of priority level 32 (in this embodiment, a lower numerical value of interrupt priority is considered to have a higher priority than a higher numerical value). The incoming interrupt is detected by the interrupt controller 10, and the interrupt controller 10 signals a virtual interrupt having a priority level 30 to be pending on the resident virtual processor vPE A. This preempts the previous process having a priority level 32, and thus the virtual interrupt priority indication 300 being executed on the physical processor pPE is updated to indicate that the virtual interrupt being executed currently has a priority of 30.

[0107] Another incoming interrupt is then detected by the interrupt controller 10, and this interrupt will be signaled to the non-resident virtual processor vPE B as a virtual interrupt with a priority level 31. The doorbell physical interrupt control circuit 20 determines that the priority control flag 302 being executed for vPE B is set, and thus compares the virtual interrupt priority 31 of the virtual interrupt for vPE B with the priority 30 indicated by the virtual interrupt priority being executed of the physical processor that will receive the doorbell interrupt. Since the virtual interrupt priority 31 of the virtual interrupt to be signaled to vPE B is lower than the virtual priority 30 being executed of the physical processor, the doorbell physical interrupt is not generated. Thereby, based on the lower-priority interrupt for the non-resident virtual processor vPE B, interruption of the higher-priority processing in the pPE 4 is avoided. Instead, vPE B will either process the interrupt when it next becomes resident, or when vPE A finishes processing the interrupt with priority 30, vPE A can return to processing the interrupt with priority 32, at which time, since the virtual interrupt priority being executed for the physical processor becomes 32 again, the doorbell physical interrupt control circuit 20 can detect that the interrupt for vPE B is of a priority sufficient to justify generating a doorbell interrupt.

[0108] FIG. 13 shows a further embodiment for enabling the doorbell physical interrupt control circuit 20 to take into account the virtual interrupt priority indication 300 during execution of the physical processor without directly exposing the virtual interrupt priority indication 300 during execution to the doorbell physical interrupt control circuit 20. Instead, the virtual interrupt processing circuit 350 within a given physical processor 4 that maintains the virtual interrupt priority indication 300 during execution may also have a doorbell activation priority configuration reprogramming circuit 360 that updates the software-defined doorbell activation priority configuration data 28 for one or more virtual interrupt processing contexts such that the corresponding entries of the doorbell activation priority configuration data 28 can track the most recent virtual interrupt priority during execution in that physical processor 4. Thus, software may initially set the doorbell activation priority configuration data 28, for example, based on the method of FIG. 5, but the reprogramming circuit 360 can also update this data 28 in hardware. This approach can avoid the need to route the virtual interrupt priority indication 300 during execution of each physical processor to the doorbell physical interrupt control circuit 20 in an implementation where the doorbell physical interrupt control circuit 20 is centrally provided separately from the physical processor 4. The doorbell activation priority configuration reprogramming circuit 360 can access reprogramming control information 370 set by the hypervisor software, which specifies information for identifying a target set of doorbell activation priority configuration data 28 that is updated when the virtual processor executing on that physical processor changes. For example, the reprogramming control information 370 can be a list of addresses updated with the updated value for the virtual priority during execution, or it can be a list of vPE identifiers that can be used, for example, to look up the vPE table 204 to find the doorbell activation priority configuration data. The hypervisor can update the list 370 of information when rescheduling the virtual processors on the physical processor.

[0109] Therefore, in this embodiment, the control of whether to generate a doorbell physical interrupt is, for the virtual interrupt processing context in which a given virtual interrupt is to be signaled as pending, based on the doorbell enabling priority configuration data 28 set by software, and can be as shown in FIG. 4. As shown in FIG. 11, it is not necessary to consider the execution virtual priority of the physical processor that receives the doorbell physical interrupt.

[0110] However, when there is an event indicating a change in the virtual interrupt priority during execution in the resident virtual processor being processed by a given physical processor, and the doorbell enabling priority configuration data 28 is one of the items of the doorbell enabling priority configuration data 28 specified by the reprogramming control information 370 to be updated in response to such an event, the software-set doorbell enabling priority configuration data 28 can be modified in hardware by the doorbell enabling priority reprogramming circuit 360. Therefore, in a scenario similar to that of FIG. 12, the change in the execution virtual priority in the physical processor 4 due to vPE A switching to process a higher-priority virtual interrupt can be reflected in the doorbell enabling priority configuration data 28.

[0111] FIG. 14 shows a method executed by the virtual interrupt processing circuit 350 in a given physical processor 4. In step 400, an event indicating a change in the virtual interrupt priority during execution associated with the current processing on the resident virtual processor resident on a given physical processor 4 is detected. For example, the event can be taking a higher-priority pending virtual interrupt or returning to lower-priority processing after processing a previously taken virtual interrupt. In step 402, the virtual interrupt processing circuit 350 of the physical processor 4 updates the execution virtual interrupt priority indication 300 to account for the change in execution priority following the event detected in step 400.

[0112] In operation 404, the virtual interrupt processing circuit 350 determines whether the hypervisor has set reprogramming control information 370 (e.g., a data structure in memory at an address set in a configuration register of the virtual interrupt processing circuit 350) indicating doorbell enabling priority configuration data 28 to be updated in response to an event indicating a change in the virtual interrupt priority during execution. If no valid reprogramming control information 370 is set, the method returns to operation 400 and waits for another event.

[0113] However, if valid reprogramming control information 370 is set, in operation 406, the doorbell enabling priority configuration reprogramming circuit 360 reprograms the doorbell enabling priority configuration data 28 of the virtual interrupt processing context corresponding to one or more other virtual processors (other than the resident virtual processor) based on the updated virtual interrupt priority indication 300 during execution. The data to be updated is specified using the hypervisor-set reprogramming control information 370.

[0114] For example, the threshold set in the doorbell enabling priority configuration data 28 can be set corresponding to the new value of the virtual interrupt priority indication 300 during execution resulting from the update in operation 402. Again, this provides a way to help the doorbell physical interrupt control circuit 20 determine the priority of the processing currently being executed in the physical processor and whether to preempt that processing by delivering a doorbell physical interrupt in response to a given virtual interrupt at a given priority level.

[0115] The concepts described herein may be embodied in computer-readable code for the manufacture of an apparatus that implements the described concepts. For example, the computer-readable code may be used in one or more stages of a semiconductor design and manufacturing process, including an Electronic Design Atomation (EDA) stage, to manufacture an integrated circuit comprising an apparatus that implements the concepts. The computer-readable code described above may additionally or alternatively enable the definition, modeling, simulation, verification, and / or testing of an apparatus that implements the concepts described herein.

[0116] For example, computer-readable code for manufacturing an apparatus that implements the concepts described herein may be embodied in code that defines a Hardware Description Language (HDL) representation of the concepts. For example, the code may define a Register-Transfer-Level (RTL) abstraction of one or more logic circuits that define an apparatus that implements the concepts. The code may use an intermediate representation such as Verilog, SystemVerilog, Chisel, or Very High-Speed Integrated Circuit Hardware Description Language (VHDL), as well as FIRRTL, to define an HDL representation of one or more logic circuits that implement the apparatus. The computer-readable code may provide a definition of the concepts that implement the concepts using a system-level modeling language such as SystemC and SystemVerilog, or other behavioral representations of the concepts that can be interpreted by a computer to enable simulation, functional and / or formal verification, and testing of the concepts.

[0117] Additionally or alternatively, the computer-readable code may define a low-level description of integrated circuit elements that embody the concepts described herein, such as one or more netlists or integrated circuit layout definitions, including representations such as GDSII. One or more netlists or other computer-readable representations of integrated circuit elements may be generated by applying one or more logic synthesis processes to an RTL representation to generate definitions for use in manufacturing an apparatus embodying the present invention. Alternatively or additionally, one or more logic synthesis processes may generate a bitstream for loading onto a Field Programmable Gate Array (FPGA) to configure the FPGA to embody the described concepts. The FPGA may be deployed for purposes of proof-of-concept and testing prior to manufacture in an integrated circuit, or the FPGA may be deployed directly in a product.

[0118] The computer-readable code may include a mixture of code representations for manufacturing an apparatus, including, for example, one or more mixtures of RTL representations, netlist representations, or other computer-readable definitions used in semiconductor design and manufacturing processes for manufacturing an apparatus embodying the present invention. Alternatively or additionally, the concepts may be defined in combination with computer-readable definitions used in semiconductor design and manufacturing processes for manufacturing an apparatus and computer-readable code that defines instructions to be executed by the defined apparatus once manufactured.

[0119] Such computer-readable code may be disposed on any well-known transient computer-readable medium (such as a wired or wireless transmission of the code over a network) or non-transient computer-readable medium such as a semiconductor, magnetic disk, or optical disk. An integrated circuit manufactured using the computer-readable code comprises components such as one or more of a central processing unit, a graphics processing unit, a neural processing unit, a digital signal processor, or other components that individually or collectively embody the concepts.

[0120] In this application, the term "configured to..." is used to mean that an element of a device has a configuration capable of performing a defined operation. In this context, "configuration" means a way of arranging or interconnecting hardware or software. For example, a device may have dedicated hardware that provides a defined operation, or a processor or other processing device may be programmed to perform a function. "Configured to" does not mean that some change must be made to the device element to provide the defined operation.

[0121] Exemplary embodiments of the present invention are described in detail herein with reference to the accompanying drawings, but it is to be understood that the present invention is not limited to these exact embodiments, and that various changes and modifications can be made by those skilled in the art without departing from the scope of the present invention as defined by the appended claims.

Claims

1. A doorbell physical interrupt control circuit, An interrupt detection circuit that detects an incoming interrupt to be generated as a given virtual interrupt for a given virtual interrupt processing context, wherein the given virtual interrupt has a given priority, the interrupt detection circuit; A doorbell physical interrupt generation circuit that responds to the detection of the incoming interrupt by the interrupt detection circuit, The given priority of the given virtual interrupt is determined by doorbell activation priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation, whether it is indicated as enabled for doorbell physical interrupt generation, In response to determining that the doorbell activation priority configuration data indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, a doorbell physical interrupt to be processed in a given physical interrupt processing context is generated, and the doorbell physical interrupt indicates that the given virtual interrupt is pending to be processed in the given virtual interrupt processing context, a doorbell physical interrupt generation circuit including a physical interrupt, which is shown to a physical processor that processes interrupts for the given physical interrupt processing context. A doorbell physical interrupt control circuit comprising:

2. The doorbell activation priority configuration data includes software-programmable doorbell activation priority configuration data associated with the given virtual interrupt processing context, and the software-programmable doorbell activation priority configuration data indicates which priorities of interrupts are enabled or disabled for doorbell physical interrupt generation for the given virtual interrupt processing context. The doorbell physical interrupt control circuit according to claim 1.

3. The doorbell physical interrupt generation circuit is configured to select the software programmable doorbell activation priority configuration data from a plurality of sets of software programmable doorbell activation priority configuration data associated with respective virtual interrupt processing contexts, based on a context identification value associated with the given virtual interrupt processing context. The doorbell physical interrupt control circuit according to claim 2.

4. The context identification value is a virtual processor identifier indicating a virtual processor that processes a virtual interrupt for the given virtual interrupt processing context, and at least one address indicating a memory location of a virtual interrupt tracking structure for tracking pending virtual interrupts for the given virtual interrupt processing context. The doorbell physical interrupt control circuit according to claim 3, including one of them.

5. The plurality of sets of software programmable doorbell activation priority configuration data are at least one memory base table stored in a memory system, and a register storage device. The doorbell physical interrupt control circuit according to any one of claims 3 or 4, defined in one of them.

6. The software programmable doorbell activation priority configuration data includes a threshold value, and the doorbell physical interrupt generation circuit is configured to determine that the given priority is valid for doorbell physical interrupt generation when the given priority has a higher priority than the doorbell physical interrupt generation threshold priority indicated by the threshold value. The doorbell physical interrupt control circuit according to any one of claims 2 to 5.

7. When the doorbell physical interrupt generation threshold priority is higher than the given priority, the doorbell physical interrupt generation circuit is configured to suppress the generation of the doorbell physical interrupt even when the given virtual interrupt is signaled as pending in the given virtual interrupt processing context. The doorbell physical interrupt control circuit according to claim 6.

8. The doorbell physical interrupt generation threshold priority is defined separately from the physical interrupt mask threshold priority to determine whether to signal the given physical interrupt to the physical processor that processes the interrupt of the physical interrupt processing context corresponding to the given physical interrupt, when compared with the priority of the given physical interrupt. The doorbell physical interrupt control circuit according to any one of claims 6 or 7.

9. The threshold value is configurable to specify a priority other than the priority associated with the current point of the program flow reached by the virtual processor that processes the interrupt for the given virtual interrupt processing context, as the doorbell physical interrupt generation threshold priority. The doorbell physical interrupt control circuit according to any one of claims 6 to 8.

10. The software-programmable doorbell activation priority configuration data includes a set of priority indicators corresponding to respective priorities and indicating whether the priority is activated or deactivated for doorbell physical interrupt generation. The doorbell physical interrupt control circuit according to any one of claims 2 to 5.

11. The doorbell activation priority configuration data includes an indication of the current virtual interrupt priority of the processing on the resident virtual processor resident on the physical processor that is signaled when the doorbell physical interrupt is generated. The doorbell physical interrupt control circuit according to any one of claims 1 to 10.

12. In response to the detection of the incoming call interrupt, the doorbell physical interrupt generation circuit Based on the current virtual interrupt priority indication for the physical processor that is signaled when the doorbell physical interrupt is generated, determines whether the given priority of the given virtual interrupt is sufficient to preempt the current processing on the resident virtual processor, Configured to suppress the generation of the doorbell physical interrupt in response to determining that the given priority of the given virtual interrupt is insufficient to preempt the current processing on the resident virtual processor, as indicated by the current virtual interrupt priority indication. The doorbell physical interrupt control circuit according to claim 11.

13. The doorbell activation priority configuration data also includes software programmable data associated with the given virtual interrupt processing context, and the software programmable data indicates, for the given virtual interrupt processing context, which interrupt priorities are enabled or disabled for doorbell physical interrupt generation, The active virtual interrupt priority indication indicates that the given priority of the given virtual interrupt is a high enough priority to preempt the current process, and in response to determining that the software programmable data associated with the given virtual interrupt processing context indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, the doorbell physical interrupt is generated, according to any one of claims 11 or 12, The doorbell physical interrupt control circuit described in the item.

14. The doorbell physical interrupt generation circuit distributes the doorbell physical interrupt to Asserting a physical interrupt signal on the interrupt bus; Updating at least one memory-based physical interrupt tracking structure to indicate that the doorbell physical interrupt is pending, and is configured to be distributed to the physical processor by one of the above, according to any one of claims 1 to 13 The doorbell physical interrupt control circuit described in the item.

15. An interrupt controller, A virtual interrupt injection circuit that supports direct injection of virtual interrupts into a virtual processor running on a physical processor; An interrupt controller comprising the doorbell physical interrupt control circuit according to any one of claims 1 to 14.

16. An apparatus, One or more physical processors; An interrupt controller that distributes virtual interrupts and physical interrupts to the one or more physical processors; An apparatus comprising the doorbell physical interrupt control circuit according to any one of claims 1 to 14.

17. The doorbell physical interrupt control circuit is The interrupt controller; At least one of the one or more physical processors; The apparatus according to claim 16, which is constituted by one of a dedicated doorbell physical interrupt generation unit separate from the one or more physical processors and the interrupt controller.

18. The apparatus according to any one of claims 16 or 17, wherein each physical processor is configured to handle physical interrupts as having a higher priority than virtual interrupts, regardless of the priority specified by software for the physical interrupt or the virtual interrupt.

19. The doorbell activation priority configuration data includes software-programmable doorbell activation priority configuration data associated with the given virtual interrupt processing context, and the software-programmable doorbell activation priority configuration data indicates, for the given virtual interrupt processing context, which priority levels of interrupts are enabled or disabled for doorbell physical interrupt generation, a given physical processor includes a doorbell activation priority configuration data reprogramming circuit, detects an event indicating a change in the current virtual interrupt priority associated with the given physical processor, the current virtual interrupt priority indicating the virtual interrupt priority associated with the current processing by a resident virtual processor resident on the given physical processor, and in response to detecting the event indicating the change in the current virtual interrupt priority, reprograms a target set of software-programmable doorbell activation priority configuration data for one or more virtual interrupt processing contexts based on an updated value for the current virtual interrupt priority after the event. The apparatus according to any one of claims 16 to 18.

20. The apparatus according to claim 19, wherein the doorbell activation priority configuration data reprogramming circuit is configured to identify the target set of software-programmable doorbell activation priority configuration data based on software-programmable control information.

21. A computer-readable medium storing code for manufacturing a doorbell physical interrupt control circuit according to any one of claims 1 to 14, an interrupt controller according to claim 15, or an apparatus according to any one of claims 16 to 20.

22. A method, comprising: Detecting an incoming interrupt that should occur as a given virtual interrupt for a given virtual interrupt processing context, wherein the given virtual interrupt has a given priority, and detecting an incoming interrupt that should occur as a given virtual interrupt for a given virtual interrupt processing context. In response to detecting the incoming interrupt Determining whether the given priority of the given virtual interrupt is indicated to be enabled for doorbell physical interrupt generation by doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation. In response to determining that the doorbell enable priority configuration data indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, generating a doorbell physical interrupt to be processed in a given physical interrupt processing context, the doorbell physical interrupt including an interrupt indicating that the given virtual interrupt is pending for processing in the given virtual interrupt processing context to a physical processor that processes interrupts for the given physical interrupt processing context.

23. A method comprising: In response to detecting that a first virtual interrupt of a first virtual interrupt processing context should be supplied by a first virtual processor that processes virtual interrupts for the first virtual interrupt processing context, Determining whether a first priority of the first virtual interrupt is higher than a current priority associated with processing performed by a second virtual processor executing on a given physical processor, the second virtual processor processing virtual interrupts for a second virtual interrupt processing context, and determining whether the first priority of the first virtual interrupt is higher than a current priority associated with processing performed by a second virtual processor executing on a given physical processor. In response to determining that the first priority is higher than the current priority Setting doorbell activation priority configuration data associated with the second virtual interrupt processing context, the doorbell activation priority configuration data including software-programmable data indicating which priority levels of interrupts are enabled or disabled for generating a doorbell physical interrupt for a given virtual interrupt processing context to indicate that a virtual interrupt is pending, and setting the doorbell activation priority configuration data associated with the second virtual interrupt processing context, rescheduling the given physical processor to process the first virtual processor, A method, wherein the doorbell activation priority configuration data associated with the second virtual interrupt processing context is set to indicate one or more priority levels selected based on a first priority level of a first virtual interrupt detected for the first virtual interrupt processing context as being disabled for generating a doorbell physical interrupt. Claim 24 A computer program that controls a computer to execute the method according to claim 23 when executed by the computer.

Images