Doorbell physical interrupt control

Abstract

A doorbell physical interrupt control circuit (20) comprises: an interrupt detection circuit (22) for detecting an incoming interrupt to be induced as a given virtual interrupt (with a given priority) for a given virtual interrupt handling context; and doorbell physical interrupt generation circuitry (24) for, in response to detection of the incoming interrupt by the interrupt detection circuitry, determining whether the given priority of the given virtual interrupt is indicated by doorbell enabling priority configuration data (28) as being enabled for doorbell physical interrupt generation, and if being enabled, generating a doorbell physical interrupt for the doorbell physical interrupt. If so, a doorbell physical interrupt to be processed in a given physical interrupt handling context is generated. The doorbell physical interrupt indicates to a physical processor that handles an interrupt for the given physical interrupt handling context that the given virtual interrupt is pending for the given virtual interrupt handling context.

Description

[0001] The present technology relates to the field of interrupt control.

[0002] In a data processing system, the job of an interrupt controller is to detect interrupts arising from one or more interrupt sources and to signal the interrupt to a physical or virtual processor that is responsible for handling the interrupt.

[0003] At least some examples provide a doorbell physical interrupt control circuit, which includes: an interrupt detection circuit, which is used to detect an incoming interrupt that will be triggered as a given virtual interrupt for a given virtual interrupt handling context, the given virtual interrupt having a given priority; and a doorbell physical interrupt generation circuit, which, in response to the detection of the incoming interrupt by the interrupt detection circuit: determines whether the given priority of the given virtual interrupt is indicated by doorbell enable priority configuration data as enabled for doorbell physical interrupt generation, the doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation; and in response to determining that the doorbell enable priority configuration data indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, generates a doorbell physical interrupt to be processed in a given physical interrupt handling context, the doorbell physical interrupt including a physical interrupt for indicating to a physical processor that handles the interrupt for the given physical interrupt handling context that the given virtual interrupt is to be handled in the given virtual interrupt handling context.

[0004] At least some examples provide an interrupt controller including: a virtual interrupt injection circuit for supporting injection of virtual interrupts directly into a virtual processor executing on a physical processor; and the doorbell physical interrupt control circuit described above.

[0005] At least some examples provide an apparatus comprising: one or more physical processors; an interrupt controller for allocating virtual interrupts and physical interrupts to the one or more physical processors; and the doorbell physical interrupt control circuit described above.

[0006] At least some examples provide a computer-readable medium for storing code for manufacturing the doorbell physical interrupt control circuit, interrupt controller or device described above. The computer-readable medium may be a non-transitory storage medium.

[0007] At least some examples provide a first method, which includes: detecting an incoming interrupt that will be triggered as a given virtual interrupt for a given virtual interrupt handling context, the given virtual interrupt having a given priority; and in response to the detection of the incoming interrupt: determining whether the given priority of the given virtual interrupt is indicated by the doorbell enable priority configuration data as enabled for doorbell physical interrupt generation, the doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation; and in response to determining that the doorbell enable priority configuration data indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, generating a doorbell physical interrupt to be processed in a given physical interrupt handling context, the doorbell physical interrupt including a physical interrupt for indicating to a physical processor that handles the interrupt for the given physical interrupt handling context that the given virtual interrupt is to be handled in the given virtual interrupt handling context.

[0008] At least some examples provide a second method comprising: in response to detecting that a first virtual interrupt of a first virtual interrupt handling context is to be serviced by a first virtual processor that handles a virtual interrupt for the first virtual interrupt handling context: determining whether a first priority of the first virtual interrupt is higher than a current priority associated with processing performed by a second virtual processor executing on a given physical processor that handles a virtual interrupt for a second virtual interrupt handling context; and in response to determining that the first priority is higher than the current priority: setting doorbell enable priority configuration data associated with the second virtual interrupt handling context, the doorbell enable priority configuration data comprising software programmable data that indicates, for the given virtual interrupt handling context, which priorities of the interrupt are enabled or disabled for generation of a doorbell physical interrupt, the doorbell physical interrupt being used to indicate that a virtual interrupt is pending; and rescheduling the given physical processor to handle the first virtual processor; wherein the doorbell enable priority configuration data associated with the second virtual interrupt handling context is set to indicate that one or more priorities selected based on the first priority of the first virtual interrupt detected for the first virtual interrupt handling context are disabled for generation of the doorbell physical interrupt.

[0009] At least some examples provide a computer program that, when executed by a computer, controls the computer to perform the second method described above. The computer program may be stored on a storage medium. The storage medium may be a non-transitory storage medium.

[0010] Additional aspects, features, and advantages of the present technology will be apparent from the following description of examples read in conjunction with the accompanying drawings, in which:

[0011] Figure 1An example of an apparatus having one or more processors, an interrupt controller, and a doorbell physical interrupt control circuit is illustrated;

[0012] Figure 2 The doorbell physical interrupt control circuit is illustrated;

[0013] Figure 3 is a ladder diagram illustrating the problems that arise when a doorbell physical interrupt is generated to indicate to a physical processor that there is a pending virtual interrupt;

[0014] Figure 4 is a flow chart illustrating a method of using doorbell enable priority configuration data to determine whether to generate a doorbell physical interrupt in response to a given virtual interrupt;

[0015] Figure 5 is a flow chart illustrating a method performed by software (e.g., a hypervisor) to configure doorbell enablement priority configuration data;

[0016] Figure 6 This is an example Figure 4 The method is applied to Figure 3 Ladder diagram of an example of the situation shown;

[0017] Figure 7 and Figure 8 Some examples of data structures used in interrupt control that can be used to specify doorbell enable priority configuration data are shown;

[0018] Fig. 9 Instantiating maintaining a running virtual interrupt priority indication at a physical processor;

[0019] Fig.10 illustrates information maintained by a hypervisor for indicating whether a running virtual interrupt priority indication for a given physical processor should be used to determine whether to generate a doorbell physical interrupt for the given physical processor;

[0020] Fig.11 A method for controlling whether to generate a doorbell physical interrupt according to a running virtual interrupt priority indication is illustrated;

[0021] Fig.12 This is an example application Fig.11 A ladder diagram of an example of the method;

[0022] Fig.13 An example is illustrated in which the processor has a doorbell enable priority configuration reprogramming circuit; and

[0023] Fig.14A method is illustrated in which a doorbell enable priority configuration reprogramming circuit reprograms doorbell enable priority configuration information of one or more other virtual processors in response to an event indicating a change in a running virtual interrupt handler of a resident virtual processor at a given physical processor.

[0024] A physical processor may execute software corresponding to a plurality of virtual processors, wherein each virtual processor (also referred to as a virtual machine) may simulate the behavior of a corresponding physical processor, which may have characteristics different from those of a main physical processor that actually executes the virtual processor. Thus, an interrupt may be signaled as a physical interrupt to be handled by a physical processor or as a virtual interrupt to be handled by a virtual processor. For example, a physical interrupt may be handled by hypervisor software (software that manages the scheduling of virtual processors executed on the physical processor), while a virtual interrupt may be handled by virtual processor software. Interrupt configuration data (e.g., set by hypervisor software) may define which events will be signaled using physical interrupts and which events will be signaled using virtual interrupts. One method of signaling a virtual interrupt may be to involve the hypervisor in updating the interrupt queue of a given virtual processor to signal the presence of a pending virtual interrupt to be handled by a given virtual processor. Another method is to use hardware to inject a virtual interrupt directly into a queue corresponding to a given virtual processor without hypervisor involvement (except for configuring data identifying the location of the queue of a given virtual processor to the hardware), which may help improve performance. However, if the hypervisor is not involved in allocating virtual interrupts to the interrupt queues of virtual processors, there is a problem in what to do when a given virtual processor for which a virtual interrupt is caused by an interrupt controller is not currently scheduled for execution on any physical processor. A pending virtual interrupt may represent a high priority event that needs to be processed immediately, but if the hardware simply updates the virtual interrupt queue structure to signal the interrupt to a non-resident virtual processor that is not currently executing, it may not be known which virtual processor is expected to be rescheduled to execute on a physical processor to allow the interrupt to be serviced. One method of solving this problem may be to respond to the detection of an interrupt event that will be signaled as a virtual interrupt by causing a physical interrupt known as a "doorbell" physical interrupt that signals the physical processor that there is a pending virtual interrupt for a given virtual processor that is not currently resident on a physical processor. The doorbell physical interrupt may prompt the hypervisor to reschedule execution on a physical processor so that a given virtual processor can execute and handle the virtual interrupt.

[0025] However, a problem with a system that generates a doorbell physical interrupt in response to the detection of a pending virtual interrupt is that sometimes the doorbell physical interrupt may cause the processing of a higher priority interrupt in one virtual processor to be interrupted and preempted by a doorbell physical interrupt raised in response to a lower priority interrupt detected for another virtual processor. While it may seem attractive to use a shared priority space between virtual and physical interrupts to solve this problem, it may be impractical because it would require changes to all software of the virtual processors and would require coordination at the hypervisor level for interrupt priority assignments, and this would break the illusion of the virtual processors that they are running in separate isolated environments, which is important to many hypervisor designs.

[0026] In the example discussed below, the doorbell physical interrupt control circuit includes: an interrupt detection circuit for detecting an incoming interrupt to be raised as a given virtual interrupt (with a given priority) for a given virtual interrupt handling context; and a doorbell physical interrupt generation circuit for, in response to the interrupt detection circuit detecting the incoming interrupt: determining whether the given priority of the given virtual interrupt is indicated by doorbell enable priority configuration data as enabled for doorbell physical interrupt generation, and if enabled, generating a doorbell physical interrupt to be processed in a given physical interrupt handling context, the doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation. The doorbell physical interrupt indicates to a physical processor handling an interrupt for the given physical interrupt handling context that the given virtual interrupt is pending for the given virtual interrupt handling context.

[0027] Thus, a circuit is provided for controlling whether a doorbell physical interrupt is generated in response to a given virtual interrupt. Whether the doorbell physical interrupt is generated depends on doorbell enable priority configuration data, which specifies which priority levels of virtual interrupts are enabled to cause the corresponding doorbell physical interrupt to be generated. Thus, a mechanism is provided to prevent the generation of a doorbell physical interrupt in response to a virtual interrupt of a given priority level indicated as disabled by the doorbell enable priority configuration data, which is useful, for example, to prevent a lower priority virtual interrupt for one virtual processor from causing a doorbell physical interrupt that may interrupt a higher priority process being executed by another virtual processor. The method does not require any collaboration between the developer of the hypervisor software and the developer of the virtual processor software in setting the priority of a particular interrupt - a shared priority space between physical interrupts and virtual interrupts can be avoided.

[0028] Each interrupt handling context refers to a certain group of interrupts that can be prioritized relative to each other based on a software-defined priority assigned to each interrupt for that interrupt handling context. The software-defined priority of one interrupt handling context can be independent of the software-defined priority of another interrupt handling context.

[0029] In some implementations, each physical interrupt handling context may correspond to a specific physical processor, and each virtual interrupt handling context may correspond to a specific virtual processor. In some examples, configuration information available to the doorbell physical interrupt control circuit (or interrupt controller described below) may identify the physical / virtual processor corresponding to a specific physical / virtual interrupt handling context.

[0030] However, for other specific implementations, the doorbell physical interrupt control circuit (or interrupt controller described below) may not actually know which specific physical or virtual processor is handling the physical or virtual interrupt for a specific interrupt handling context. Therefore, the physical / virtual interrupt handling context can be an abstract concept that identifies a given set of interrupts, and different physical / virtual processors can be configured to monitor a specific data structure in memory for identifying pending interrupts for a given (physical or virtual) interrupt handling context. The doorbell physical interrupt control circuit (or interrupt controller) may not know which specific physical or virtual processor is monitoring a specific set of data structures for a given interrupt handling context. Instead, the doorbell physical interrupt control circuit (or interrupt controller) can identify the interrupt handling context based on, for example, one or more addresses, which identify a data structure for tracking interrupts for the interrupt handling context.

[0031] The doorbell enable priority configuration data may be implemented in different ways. For example, the doorbell enable priority configuration data may be software-maintained information set by software, or hardware-maintained information automatically maintained by hardware without explicit software intervention (nevertheless, the hardware may still control hardware-implemented updates to the doorbell enable priority configuration data based on the configuration data set by software). In addition, the doorbell enable priority configuration data may be associated with a given virtual interrupt handling context for which a given virtual interrupt is detected, or may be associated with a physical processor that will receive a doorbell physical interrupt if a doorbell physical interrupt is generated. Various examples of doorbell enable priority configuration data are described below.

[0032] In some examples, the doorbell enable priority configuration data includes software programmable doorbell enable priority configuration data associated with a given virtual interrupt handling context. The software programmable doorbell enable priority configuration data indicates, for a given virtual interrupt handling context, which priorities of interrupts are enabled or disabled for doorbell physical interrupt generation. The method may allow a hypervisor to set the software programmable doorbell enable priority configuration data of an outgoing virtual processor when switching which virtual processor resides on a physical processor to reduce the chance of preempting a higher priority processing on a newly resident virtual processor due to a lower priority virtual interrupt for the outgoing virtual processor.

[0033] Thus, doorbell enablement priority configuration data may be defined in association with a given virtual interrupt handling context. Thus, different virtual interrupt handling contexts may have different settings for which virtual interrupt priorities will be enabled for the generation of a doorbell physical interrupt. A virtual interrupt of a given priority raised for one virtual interrupt handling context may trigger the generation of a doorbell physical interrupt, while a virtual interrupt of the same priority raised for another virtual interrupt handling context may not, depending on the doorbell enablement priority configuration data associated with those particular virtual interrupt handling contexts.

[0034] Thus, the doorbell physical interrupt generation circuit may be configured to select software programmable doorbell enable priority configuration data from among two or more sets of software programmable doorbell enable priority configuration data associated with a given virtual interrupt handling context based on a context identification value associated with the corresponding virtual interrupt handling context.

[0035] The context identification value for selecting which set of software programmable doorbell enablement priority configuration data to use may include a virtual processor identifier indicating a virtual processor that handles a virtual interrupt for a given virtual interrupt handling context. Alternatively, the context identification value may include at least one address indicating a memory location of a virtual interrupt tracking structure for tracking pending virtual interrupts for a given virtual interrupt handling context.

[0036] These groups of software programmable doorbell enable priority configuration data may be defined in one or more memory-based tables stored in a memory system. In this case, data from those tables may also be cached in a cache local to the doorbell physical interrupt control circuitry so that more recently accessed doorbell enable priority configuration data may be accessed more quickly than from the underlying data in memory.

[0037] For example, each virtual interrupt handling context may have a corresponding set of memory-based interrupt tracking data structures for indicating pending virtual interrupts for the virtual interrupt handling context, and one of these structures may also provide software programmable doorbell enable priority configuration data for the virtual interrupt handling context (thus, each set of doorbell enable priority configuration data for a different virtual interrupt handling context may be obtained from a different set of interrupt tracking data structures).

[0038] Alternatively, a data structure defining doorbell configuration information that specifies details of a doorbell physical interrupt to be generated in response to a virtual interrupt for a virtual interrupt handling context (e.g., the doorbell configuration information may specify a physical interrupt identifier to be used for the doorbell physical interrupt) may also specify software programmable doorbell enable priority configuration data for the virtual interrupt handling context.

[0039] Alternatively, multiple sets of software programmable doorbell enable priority configuration data may be defined in register memory. For example, a set of memory mapped configuration registers may be associated with the doorbell physical interrupt control circuit and may be configurable by software on the processor by issuing memory read / write operations to addresses mapped to those registers.

[0040] The software programmable doorbell enable priority configuration data may be any information settable by software to allow the software to specify, for each interrupt priority level in a given interrupt handling context, whether a virtual interrupt of that priority level should cause a corresponding doorbell physical interrupt to be generated. This may be implemented in different ways.

[0041] In one example, the software programmable doorbell enable priority configuration data includes a threshold value, and when the given priority has a priority level higher than the doorbell physical interrupt generation threshold priority level indicated by the threshold value, the doorbell physical interrupt generation circuit determines that the given priority level is enabled for doorbell physical interrupt generation. The threshold value can be an effective way to encode the software programmable doorbell enable priority configuration data to indicate whether each virtual interrupt priority level should be considered enabled or disabled for the purpose of generating a doorbell physical interrupt based on a comparison between the given priority level and the threshold. Thus, a priority level higher than the threshold priority level indicated by the threshold value is considered enabled for doorbell physical interrupt generation, while a priority level lower than or equal to the threshold value priority level is considered disabled for doorbell physical interrupt generation.

[0042] The term "higher priority" is used to indicate an interrupt that is more important than a lower priority interrupt. That is, an interrupt with a higher priority may preempt an interrupt with a lower priority within the same interrupt handling context. It should be noted that different encoding schemes may be used to encode the priority of a given interrupt. In some examples, an interrupt assigned a priority level with a higher numerical value may be considered to have a higher priority than an interrupt assigned a priority level with a lower numerical value (e.g., priority 4 may be considered to be more important than priority 3). Other examples may use a lower numerical value to indicate an interrupt with a higher priority (e.g., priority 3 is higher than priority 4). Therefore, the term "higher priority" refers to the level of importance of the interrupt, rather than the size of the numerical value used to represent the priority level.

[0043] Note that a threshold priority may be represented by a threshold value in different ways—in some examples, the threshold value may specify the threshold priority itself, but in other examples, the threshold value may specify a priority that is one level higher than the threshold priority (in which case a greater than or equal to comparison between a given priority and the threshold value may indicate whether the given priority has a higher priority than the threshold priority, or alternatively, a less than or equal to comparison may be used where a lower numerical value encodes a higher priority than a higher numerical value). Furthermore, a threshold value may not directly indicate a threshold level, but may have an encoding that indirectly indicates the threshold level (e.g., selecting from among several predetermined thresholds). Thus, in general, threshold priorities may be specified in different ways, and there may be a wide range of different kinds of comparisons that may be used to evaluate whether a given priority has a higher priority than the threshold priority indicated by a threshold value (explicitly or implicitly).

[0044] When the priority of the doorbell physical interrupt generation threshold priority is higher than the given priority, the doorbell physical interrupt generation circuit may suppress the generation of the doorbell physical interrupt, even when the given virtual interrupt is signaled as pending in the given virtual interrupt handling context. Therefore, the doorbell physical interrupt generation threshold priority is specific to the control of generating the doorbell physical interrupt in response to the virtual interrupt of the given virtual interrupt priority, and is not at all a control of whether the virtual interrupt is signaled as pending. If the virtual interrupt is signaled as pending, but does not cause the generation of the corresponding doorbell physical interrupt, the handling of the virtual interrupt may be delayed because the corresponding virtual processor may not be prompted to be scheduled by the hypervisor, but eventually when the virtual processor becomes resident on the given physical processor again, the pending virtual interrupt may be handled by the virtual processor.

[0045] In some examples, for physical interrupts, a physical interrupt mask threshold priority may be defined, which is used to compare with the priority of a given physical interrupt to determine whether to signal a given physical interrupt to a physical processor that handles an interrupt for a corresponding physical interrupt handling context. Thus, some examples may provide the ability to completely suppress signaling of physical interrupts with a priority lower than or equal to the physical interrupt mask threshold priority to a physical processor (even if these physical interrupts are pending). In such systems, the physical interrupt mask threshold priority is a threshold defined separately from the doorbell physical interrupt generation threshold priority. These are different because the doorbell physical interrupt generation threshold priority is compared with the priority of the virtual interrupt to determine whether to signal the doorbell physical interrupt to the physical processor, while the physical interrupt mask threshold priority is compared with the priority of the physical interrupt to determine whether to signal the physical interrupt to the physical processor.

[0046] The threshold value associated with a given virtual interrupt handling context may be capable of being set to specify a priority for generating a threshold priority for a doorbell physical interrupt in addition to the priority associated with the current point in program flow reached by the virtual processor handling the interrupt for the given virtual interrupt handling context. This is useful because the hypervisor may use the threshold value for a given virtual interrupt handling context to ensure that another virtual processor handling a high priority interrupt for a different virtual interrupt handling context is not interrupted unless a sufficiently high interrupt occurs for the given virtual interrupt handling context, and therefore, the priority that will be defined as the threshold value may depend more on what is being executed in the other virtual processor than on the current point in program flow reached by the virtual processor corresponding to the given virtual interrupt handling context.

[0047] Priority thresholds are not the only way to define doorbell enablement priority configuration data to indicate which priorities are enabled or disabled for purposes of doorbell physical interrupt generation.

[0048] In another example, the software programmable doorbell enable priority configuration data includes a set of priority indicators, each priority indicator corresponding to a respective priority and indicating whether the priority is enabled or disabled for doorbell physical interrupt generation. For example, the doorbell enable priority configuration data may be a bitmap, where each bit corresponds to a given priority level (or a group of priority levels) and indicates whether the priority level or the group of priority levels is enabled for doorbell physical interrupt generation (so that a virtual interrupt of the priority level will trigger the generation of a doorbell physical interrupt) or disabled for doorbell physical interrupt generation (so that a virtual interrupt of the priority level will not trigger the generation of a doorbell physical interrupt). This may allow for more fine-grained control over which priorities will cause the generation of a doorbell physical interrupt.

[0049] In some examples, the doorbell enable priority configuration data includes a running virtual interrupt priority indication that indicates a virtual interrupt priority of a current process on a resident virtual processor resident on a physical processor to which a doorbell physical interrupt is to be signaled if a doorbell physical interrupt is generated.

[0050] The physical processor may have a hardware circuit logic component that maintains a running virtual interrupt priority indication to track the virtual interrupt priority of the current processing on the resident virtual processor currently resident on the physical processor. The resident virtual processor is the virtual processor currently assigned to the physical processor. The resident virtual processor does not need to be currently executing any instructions (e.g., during a system call to a hypervisor, for example, instructions may be executed from the hypervisor, but the resident virtual processor making the system call may still be considered resident).

[0051] Generally speaking, a running indication of the current priority of a process executing on a resident virtual processor may be useful for the hypervisor to make scheduling decisions. Therefore, some hardware circuit logic components may be provided in the physical processor to respond to events indicating a change in virtual interrupt priority (such as taking a virtual interrupt or returning to a previous process after completing handling of a virtual interrupt) by updating the running virtual interrupt priority indication according to the virtual interrupt priority of the process being executed after the event.

[0052] In some examples, the running virtual interrupt priority indication of the physical processor that will receive the doorbell physical interrupt can be used by the doorbell physical generation circuit as doorbell enable priority configuration data, which is used to determine whether to generate a doorbell physical interrupt. Therefore, in response to the detection of an incoming interrupt, the doorbell physical interrupt generation circuit can: determine whether the priority of a given virtual interrupt is sufficient to preempt the current process on the resident virtual processor based on the running virtual interrupt priority indication of the physical processor to which the doorbell physical interrupt will be signaled in the event of the doorbell physical interrupt being generated; and suppress the generation of the doorbell physical interrupt in response to determining that the running virtual interrupt priority indication indicates that the priority of the given virtual interrupt is insufficient to preempt the current process on the resident virtual processor. By taking into account the virtual priority of the process currently being processed on the resident virtual processor at the physical processor that will receive the doorbell physical interrupt, this can make it less likely that the doorbell physical interrupt control circuit will generate the following doorbell physical interrupt, which causes a lower priority virtual interrupt for a non-resident virtual processor to cause an interrupt of a higher priority process on the resident virtual processor.

[0053] The different methods discussed for implementing doorbell enablement priority configuration data may be implemented individually or in combination.

[0054] Thus, in some examples, the doorbell enable priority configuration data includes software programmable doorbell enable priority configuration data, but does not include an indication of the running virtual interrupt priority. This may provide a method that may be more easily scaled to handle different numbers of physical processors in a system (the amount of configuration data to be considered by the doorbell physical interrupt control circuitry may be independent of the number of physical processors provided). Furthermore, this allows software to set the doorbell enable priority configuration data for a particular virtual interrupt handling context based on any particular knowledge of the current processing on the corresponding virtual processor. For example, if it is known that a particular virtual processor has no processing to execute, the doorbell enable priority configuration data for the virtual interrupt handling context corresponding to that particular virtual processor may be set to disable doorbell physical interrupt generation for a larger subset of virtual interrupt priorities, thereby reducing the chances of generating a doorbell physical interrupt in response to a virtual interrupt raised for that virtual processor.

[0055] As discussed further below, in some examples where the doorbell enable priority configuration data includes software programmable doorbell enable priority configuration data but does not include a running virtual interrupt priority indication, the running virtual interrupt priority indication may still be maintained by a given physical processor and used by hardware to control reprogramming of the software programmable doorbell enable priority configuration data for one or more virtual processors. Thus, even if the doorbell physical interrupt control circuitry does not directly consider the running virtual interrupt priority indication (e.g., to improve scalability by not requiring the running virtual interrupt priority indication of each physical processor to be routed to the doorbell physical interrupt control circuitry), the running virtual interrupt priority indication may still indirectly affect doorbell physical interrupt generation.

[0056] For other examples, the doorbell enable priority configuration data may include a running virtual interrupt priority indication, but not include software programmable doorbell enable priority configuration data. This may allow doorbell physical interrupt generation to more accurately respond to changes in the priority associated with a current process executed by a resident virtual processor at a given physical processor to reduce the likelihood that the current process is preempted by a lower priority virtual interrupt raised for a non-resident virtual processor.

[0057] Other examples may consider both software programmable doorbell enable priority configuration data associated with a given virtual interrupt handling context and a running virtual interrupt priority indication associated with a physical processor that will receive the doorbell physical interrupt if the doorbell physical interrupt is generated. In this case, a doorbell physical interrupt may be generated in response to an incoming interrupt in response to: determining that the running virtual interrupt priority indication indicates that the priority of a given virtual interrupt is sufficient to preempt the current processing (of a resident virtual processor interrupt on the physical processor that will receive the doorbell physical interrupt), and software programmable data associated with a given virtual interrupt handling context indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation. By considering both types of doorbell enable priority configuration data, more precise control of doorbell physical interrupt generation is possible (e.g., both taking into account any software-specific knowledge of the operating state of the virtual processor that may be used by the hypervisor in setting the software programmable doorbell enable priority configuration data, and reacting to changes in the running virtual priority at a given physical processor, which may be difficult to track using the software programmable doorbell enable priority configuration data associated with a particular virtual interrupt handling context), thereby reducing the probability of interrupting a higher priority process due to a doorbell physical interrupt generated in response to a lower priority virtual interrupt.

[0058] There are a variety of ways in which a doorbell physical interrupt may be assigned to a physical processor. For example, the doorbell physical interrupt generation circuitry may assign a doorbell physical interrupt to a physical processor by asserting a physical interrupt signal on an interrupt bus, or updating at least one memory-based physical interrupt tracking structure to indicate that a doorbell physical interrupt is pending. Which physical processor receives the doorbell physical interrupt may be defined by the doorbell physical interrupt configuration information (which may be configured independently of the virtual handling context and the doorbell enable priority configuration data), or may be unknown to the doorbell physical interrupt control circuitry (if updates to memory-based tracking structures are used to signal a pending doorbell physical interrupt, the doorbell physical interrupt control circuitry may not know which physical processor is monitoring those tracking structures). The doorbell physical interrupt may have its own physical priority level, which may be independent of a given priority level associated with a given virtual interrupt that caused the physical interrupt to be generated.

[0059] The doorbell physical interrupt control circuit can be implemented at various different parts of the data processing system.

[0060] In one example, the doorbell physical interrupt control circuit can be part of an interrupt controller that also includes a virtual interrupt injection circuit that is used to support direct injection of virtual interrupts into a virtual processor executing on a physical processor. An interrupt controller that supports direct injection of virtual interrupts has hardware that can signal interrupts to a virtual processor without hypervisor intervention when signaling interrupts. For example, the hardware of the interrupt controller can issue a memory access request to update a virtual interrupt queue monitored by a virtual processor to indicate in the virtual interrupt queue that a virtual interrupt is pending. Although the hypervisor is not involved in actually signaling the virtual interrupt, this does not exclude the hypervisor from participating in setting configuration information that controls how the hardware injects interrupts into the virtual processor (for example, the hypervisor can set the address of the virtual interrupt queue that will be used for the interrupt of the virtual processor). By providing a doorbell physical interrupt control circuit to an interrupt controller that supports direct injection of virtual interrupts, this helps the interrupt controller avoid the problem of a doorbell physical interrupt preempting a higher priority virtual interrupt due to the detection of a lower priority virtual interrupt.

[0061] Alternatively, the doorbell physical interrupt control circuit may be provided in a portion of the data processing device other than the interrupt controller. For example, the doorbell physical interrupt control circuit may be a portion of the physical processor for detecting a virtual interrupt injected by the interrupt controller and determining whether to respond by asserting a doorbell physical interrupt to the physical processor. Furthermore, the doorbell physical interrupt control circuit may be a dedicated doorbell physical interrupt generation unit separate from both the physical processor and the interrupt controller.

[0062] Thus, in general, an apparatus may include one or more physical processors, an interrupt controller for allocating virtual interrupts and physical interrupts to the one or more physical processors, and a doorbell physical interrupt control circuit. The doorbell physical interrupt control circuit may be composed of the interrupt controller, at least one of the one or more physical processors, or a dedicated doorbell physical interrupt generation unit.

[0063] Each physical processor may treat a physical interrupt as having a higher priority than a virtual interrupt, regardless of the priority specified by software for the physical interrupt or virtual interrupt. Because in general hypervisor events can often be considered more important than events for a specific virtual machine, it is useful to default to treating physical interrupts as all having a higher priority than any virtual interrupt (in extreme cases, the lowest priority physical interrupt is considered to have a higher priority than the highest priority virtual interrupt). This avoids the need for collaboration between the developer of the hypervisor software and the corresponding virtual processor software, but may cause the following problem: a lower priority virtual interrupt for one virtual processor may cause a high priority doorbell physical interrupt to be generated, which may preempt the processing of a higher priority virtual interrupt by another virtual processor. This problem can be solved by providing doorbell enable priority configuration data for determining whether to signal a doorbell physical interrupt as discussed above. Therefore, the techniques discussed above may be particularly useful in systems where physical interrupts are treated as having a higher priority than virtual interrupts, regardless of the priority levels specified by software for those interrupts.

[0064] In a specific implementation where the doorbell enable priority configuration data includes software programmable doorbell enable priority configuration data associated with a given virtual interrupt handling context (the software programmable doorbell enable priority configuration data indicates which priorities of interrupts are enabled or disabled for doorbell physical interrupt generation for the given virtual interrupt handling context), if it is desired to also consider the running virtual priority at the physical processor that will receive the doorbell physical interrupt when determining whether to generate a doorbell physical interrupt, another method may be to reprogram the software programmable doorbell enable priority configuration data when an event indicating a change in the running virtual interrupt priority occurs at the physical processor. This may be a way to avoid the physical routing cost of a signal that conveys the running virtual priority indication of each physical processor to the doorbell physical interrupt control circuit, while still allowing control of doorbell interrupt generation based on the running virtual priority of the physical processor.

[0065] Thus, a given physical processor may include a doorbell enable priority configuration data reprogramming circuit that is used to: detect an event indicating a change in a running virtual interrupt priority associated with the given physical processor, the running virtual interrupt priority indicating a virtual interrupt priority associated with a current processing of a resident virtual processor residing on the given physical processor; and in response to detecting an event indicating a change in the running virtual interrupt priority, reprogram a set of target software programmable doorbell enable priority configuration data for one or more virtual interrupt handling contexts based on an updated value of the running virtual interrupt priority after the event.

[0066] The doorbell enable priority configuration data reprogramming circuitry may identify a set of target software programmable doorbell enable priority configuration data based on the software programmable control information. For example, the software programmable control information may be any information that allows the hardware to identify addresses corresponding to memory locations (corresponding to those addresses of the software programmable doorbell enable priority configuration data mentioned for one or more virtual processors) that will be updated based on updating the running virtual interrupt priority following an event indicating a change in the running virtual interrupt priority. For example, the software programmable control information may specify those addresses directly, or may specify information that may be looked up in another data structure to obtain the addresses of the locations in memory that will be updated along with updating the running virtual interrupt priority (such as a list of virtual processor identifiers or virtual interrupt handling context identifiers). The hypervisor software may set the software programmable control information, for example, when rescheduling which virtual processor resides on a physical processor.

[0067] A corresponding method is provided, the method comprising: in response to detecting that a first virtual interrupt of a first virtual interrupt handling context is to be serviced by a first virtual processor handling a virtual interrupt for the first virtual interrupt handling context:

[0068] determining whether a first priority level of the first virtual interrupt is higher than a current priority level associated with a process executed by a second virtual processor executing on the given physical processor, the second virtual processor handling a virtual interrupt for a second virtual interrupt handling context;

[0069] as well as

[0070] In response to determining that the first priority is higher than the current priority:

[0071] o setting doorbell enable priority configuration data associated with the second virtual interrupt handling context, the doorbell enable priority configuration data comprising software programmable data indicating, for the given virtual interrupt handling context, which priorities of interrupts are enabled or disabled for generation of a doorbell physical interrupt, the doorbell physical interrupt being used to indicate that a virtual interrupt is pending; and

[0072] o rescheduling the given physical processor to process the first virtual processor;

[0073] The doorbell enable priority configuration data associated with the second virtual interrupt handling context is set to indicate that one or more priorities selected based on the first priority of the first virtual interrupt detected for the first virtual interrupt handling context are disabled for the generation of the doorbell physical interrupt. The method may be performed by software for setting the doorbell enable priority configuration data. The method is based on a specific implementation of setting software programmable doorbell enable priority configuration data corresponding to a given virtual interrupt handling context.

[0074] For example, the method may be performed by hypervisor software that is responsible for scheduling which virtual processor to execute on a given physical processor. Thus, the hardware of the data processing apparatus may not have any circuit logic components dedicated to performing the steps of the method. Instead, the hardware provides the architectural functionality for responding to the doorbell enable priority configuration data set by software, for example by including the doorbell physical interrupt control circuit described above.

[0075] In some examples, the one or more priorities selected to be disabled for doorbell physical interrupt generation based on the first priority level of the first virtual interrupt may include priorities that are lower than or equal to the first priority level. Thus, when a subsequent virtual interrupt is detected for a second virtual interrupt handling context handled by a second virtual processor, only virtual interrupts with a priority level higher than the first priority level (which caused the second virtual processor to be preempted) will subsequently cause the doorbell physical interrupt to be generated. The method may assume that an interrupt of a given priority level in one virtual interrupt handling context may be considered equally important as an interrupt of the same priority level in another virtual interrupt handling context (this may be a default assumption because the hypervisor may not be aware of the particular way in which a particular virtual processor is using priority levels).

[0076] In other examples, if the hypervisor knows the relative importance of an interrupt of a given priority on one virtual processor to an interrupt of a given priority on another virtual processor, there may be a transition between priority levels of the corresponding virtual processes, and therefore, one or more priorities for which the doorbell physical interrupt is disabled may be those transition priorities in the second virtual interrupt handling context that are considered to be lower in priority than or equal in priority to the first priority of the first virtual interrupt in the first virtual interrupt handling context. Thus, if the hypervisor otherwise knows that some priority interrupts in the second virtual interrupt handling context that are higher than the first priority should take precedence over the first priority interrupt in the first virtual interrupt handling context, the priorities selected based on the first priority may not necessarily all be exclusively priorities lower than the first priority.

[0077] Specific examples will now be described with reference to the accompanying drawings. It should be understood that the claims are not limited to these examples.

[0078] Figure 1 An example of a data processing system 2 (e.g., a system on a chip) including a plurality of processors 4 (e.g., central processing units, CPUs) is schematically illustrated. In this example, three processors 4 are shown, but it should be understood that the number of processors may vary. The processors communicate with each other and with a shared memory 8 via a cache coherent interconnect 6 that supports a coherence protocol to maintain cache coherence of data cached in a dedicated cache of each processor 4.

[0079] An interrupt controller 10 is provided to receive incoming interrupt signals from connected peripheral devices 14 and forward the incoming interrupt signals to the processor 4. In some cases, the processor also generates interrupts for other processors (known as inter-processor interrupts (IPIs)), so the processor 4 itself may also act as an interrupt source. The peripheral device 14 may signal the interrupt to the interrupt controller 10 via a dedicated wire 11 or by reusing an existing I / O (input / output) mechanism 12 (such as a memory-mapped I / O write operation). The latter is generally referred to as a message-signaled interrupt (MSI). The job of the interrupt controller is to prioritize interrupts according to a configuration performed by software to ensure that higher priority interrupts are presented to the processor in preference to lower priority interrupts. On modern multi-processor systems with more than a single processor, the interrupt controller may also have to route interrupts to one or more designated processors and handle reprogramming of the routing configuration without losing interrupt signals.

[0080] Therefore, the interrupt controller needs to be able to communicate with the processor to which it can forward interrupts. Figure 1 As shown, one way to achieve this is to use a dedicated communication protocol and interrupt communication bus 16 in the system that is specifically designed to carry interrupt signals from the interrupt controller 10 to the processor 4. The interrupt communication bus 16 is completely separate from the cache coherent interconnect 6 used to transmit memory transactions between the processor 4, the memory 8 and the interrupt controller 10 (the interrupt controller 10 may still have an interface to the cache coherent interconnect 6 to allow memory transactions issued by the processor 4 to program configuration information that controls how the interrupt controller 10 handles forwarding of interrupts to the processor 4).

[0081] Another method for distributing interrupts to processors may be to utilize the existing cache coherence mechanism supported by the cache coherent interconnect 6 to distribute interrupts from the interrupt controller 10 to the processor 4, and to represent the interrupt configuration and status through a memory-based table shared by the processor and the interrupt controller. Storing the interrupt tracking data structure (e.g., a queue of pending interrupts, and other configuration states (such as interrupt priority and enable status for different interrupt identifiers)) in memory and using the cache coherence protocol of the cache coherent interconnect 6 to ensure consistency between the views of the shared tracking data structure seen by the interrupt controller 10 and the processor 4 means that when an interrupt changes state due to a message or signal received by the interrupt controller 10, the interrupt controller can cause the interrupt to be delivered to the correct processor by simply updating the tracking data structure stored in memory. This can be done without the need for a dedicated interrupt bus 16 by using the existing cache coherence interconnect mechanism supported by the interconnect 6. The processor 4 can ensure that the address to be monitored is maintained in its cache in an "exclusive" coherency state for updating the monitoring tracking structure, which ensures that any write to the address by the interrupt controller will cause the cache coherent interconnect 6 to snoop the processor's cache, resulting in an invalidation, which can be detected and used as a hint to re-request the data for the address to be returned to the processor's cache in an exclusive state, so that the processor can check the updated data and use the updated data to signal an exception to the processor, which can cause software to more closely check the updated tracking structure to determine how to handle any interrupts that are signaled as pending. This consistency-based approach can reduce the design effort involved in scaling chip designs according to different numbers of processors and interrupts, because scaling can be achieved by changing the size or number of structures storing data in memory without the need to expand the wiring of the dedicated interrupt dispatch bus 16.

[0082] The following examples may use the interrupt dispatch bus 16 or memory writes via the cache coherent interconnect 6 to dispatch virtual or physical interrupts to the processor 4. It is not necessary to use the same method for both physical and virtual interrupts. For example, physical interrupts may be dispatched via the interrupt communication bus 16, while virtual interrupts may be dispatched using the interconnect 6. Alternatively, both physical and virtual interrupts may use the same type of dispatch mechanism, but may be distinguished by physical signals exchanged on the interrupt communication bus or by updating different sets of interrupt tracking structures (a set of physical interrupt tracking structures for a given physical interrupt handling context that a given processor 4 may be configured to monitor for updates; or a set of virtual interrupt tracking structures for a given virtual interrupt handling context that a given virtual processor executing on a given physical processor may be configured to monitor for updates). If both virtual and physical interrupts are dispatched using memory writes via the cache coherent interconnect 6, the interrupt communication bus 16 may be omitted.

[0083] The apparatus 2 may include a doorbell physical interrupt control circuit 20 for controlling the generation of a doorbell physical interrupt that signals to a processor that there is a pending virtual interrupt to be handled by a virtual processor that is not currently resident on the processor. Figure 1 As shown by the dotted lines in , the doorbell physical interrupt control circuit 20 can be implemented at different locations in the data processing system 2. For example, the doorbell physical interrupt control circuit 20 can be set within the interrupt controller 20; set as an independent doorbell physical interrupt unit that is designed to respond to the virtual interrupt caused by the interrupt controller by generating a doorbell physical interrupt that will signal the processor 4; or set as part of each physical processor 4. These are all alternative locations for the doorbell physical interrupt control circuit 20, and it should be understood that in a given system, the doorbell physical interrupt control circuit 20 will not be set at all of these locations.

[0084] Figure 2The doorbell physical interrupt control circuit 20 is shown in more detail. The doorbell physical interrupt control circuit includes an interrupt detection circuit 22 that detects the occurrence of an incoming interrupt to be signaled as a virtual interrupt. In the case where the doorbell physical interrupt control circuit 20 is provided within the interrupt controller 10, the interrupt detection circuit 22 may detect the occurrence of an interrupt based on a signal asserted on the interrupt wire 11 or an interrupt signaled using a message represented by a memory write request issued on the I / O bus 12. If the doorbell interrupt control circuit 20 is provided as a separate unit or provided within the processor 4, the interrupt detection circuit 22 may detect a virtual interrupt based on a physical signal asserted by the interrupt controller 10 on the interrupt communication bus 16 or based on an update made by the interrupt controller 10 to a memory address allocated for representing interrupt tracking data (in an example where a coherence mechanism is used to deliver interrupts from the interrupt controller 10 to the processor, as discussed above).

[0085] The doorbell physical interrupt control circuit 20 also includes a doorbell physical interrupt generation circuit 24, which is used to respond to the interrupt detection circuit 22 detecting an incoming interrupt to be signaled as a virtual interrupt by determining whether a doorbell physical interrupt should be signaled, and if so, causing a doorbell physical interrupt and signaling it to the physical processor.

[0086] If the doorbell physical interrupt control circuit 20 is part of the interrupt controller 10, the interrupt controller may also have a virtual interrupt injection circuit 26 that responds to detecting an interrupt caused by a peripheral device 14 or other interrupt event source to inject a virtual interrupt into a virtual interrupt queue associated with the corresponding virtual processor. Thus, the interrupt controller will support direct injection of virtual interrupts, where hardware is provided to directly update a memory structure representing a virtual interrupt queue for a given virtual processor without requiring the hypervisor to participate in updating the queue. Although the virtual interrupt injection circuit 26 is in Figure 2 4 is shown as a part of the doorbell physical interrupt control circuit 20, but it can also be provided at other parts of the interrupt controller 10. If the doorbell physical interrupt control circuit 20 is implemented in the processor 4 or as an independent unit, the virtual interrupt injection circuit 26 will not be provided at the doorbell physical interrupt control circuit 20.

[0087] The doorbell physical interrupt generation circuitry has access to the doorbell enable priority configuration data 28 and the doorbell configuration data 30. In this example, both sets of data are software programmable structures that can be programmed by software executing on the processor 4, such as by a hypervisor that manages the execution of virtual processors (as described later for Figures 9 to 12As mentioned, other examples may use hardware maintained information maintained by a physical processor as doorbell enable priority configuration data). Figure 2 In the example of , the doorbell enable priority configuration data 28 and the doorbell configuration data 30 may be maintained in registers of the doorbell physical interrupt control circuit 20, or in a memory-based data structure that can be accessed by issuing a memory request. In the case of a memory-based structure, information from the memory-based data structure may also be cached in a cache local to the doorbell physical interrupt control circuit 20. The right to update the doorbell enable priority configuration data 28 and the doorbell configuration data 30 may be controlled using memory management permissions defined at the processor 4 using a page table, such as limiting permissions to set the doorbell enable priority configuration data to the hypervisor and denying access to the virtual processor itself.

[0088] The doorbell configuration data 30 may define information for defining a doorbell physical interrupt that should be generated when a given virtual interrupt is detected. For example, the doorbell configuration data 30 may specify a physical interrupt number for the doorbell physical interrupt, and may specify information identifying a physical interrupt handling context in which the doorbell physical interrupt will be generated (e.g., the information may identify a specific physical processor, an identifier of a node on the interrupt communication bus 16 to which the doorbell physical interrupt will be delivered, or a base address of a memory-based interrupt tracking structure that will be updated to enqueue the doorbell physical interrupt).

[0089] exist Figure 2 In the example of , the doorbell enable priority configuration data 28 includes multiple sets of software programmable data, each set corresponding to a corresponding virtual interrupt handling context (e.g., a corresponding virtual processor), and provides data defining, for the corresponding virtual interrupt priority, whether each priority is enabled for generation of a doorbell physical interrupt or disabled for generation of a doorbell physical interrupt. The doorbell enable priority configuration data 28 has an encoding that allows a first subset of interrupt priorities to be defined as enabled for generation of a doorbell physical interrupt, while a second subset of interrupt priorities is defined as disabled for generation of a doorbell physical interrupt. Since there are multiple sets of doorbell enable priority configuration data 28, each set corresponding to a different virtual interrupt handling context, different virtual interrupt handling contexts may have different settings regarding which interrupt priorities are enabled / disabled for generation of a doorbell physical interrupt. The doorbell physical interrupt generation circuit 24 selects which set of doorbell enable priority configuration data 28 based on a context identifier associated with the virtual interrupt handling context in which the incoming interrupt is detected by the interrupt detection circuit 22. For example, the context identifier may be a virtual processor identifier or an address of a tracking structure for indicating pending interrupts for the virtual interrupt handling context.

[0090] The doorbell enable priority configuration data 28 may be represented in different ways. For example, one implementation may provide, for each set of doorbell enable priority configuration data, a threshold value identifying a priority threshold that will be used to determine whether a doorbell physical interrupt should be generated in response to a particular virtual interrupt. Priorities above the priority threshold may be considered enabled for doorbell physical interrupt generation, while priorities below the priority threshold (and priorities equal to the priority threshold) may be considered disabled for doorbell physical interrupt generation. Thus, when the interrupt detection circuit 22 detects an incoming interrupt corresponding to a given virtual interrupt, the priority of the given virtual interrupt may be compared to the threshold value (or a threshold derived from the threshold value) for the corresponding virtual interrupt handling context, and the comparison result may be used to decide whether to trigger a doorbell physical interrupt.

[0091] In another approach, the doorbell enable priority configuration data 28 for a given virtual interrupt handling context may specify a set of indicators (e.g., a bitmap), where each indicator corresponds to a given priority level and specifies whether interrupts of that priority should be considered enabled or disabled for doorbell physical interrupt generation.

[0092] The doorbell enable priority configuration data 28 helps to solve the preemption problem of high priority interrupts that may occur in systems that support direct injection of virtual interrupts. Figure 3 Consider, for example, two separate virtual processors vPE A and vPE B, where vPE A has virtual interrupts INT0 and INT2, and vPE B has a virtual interrupt INT1. Further consider that INT0 has a higher priority than INT1, which in turn has a higher priority than INT2 (lower numbers are higher priorities). All physical interrupts are considered to have a higher priority than all virtual interrupts (because hypervisor events can be considered more important than events to be handled by a given virtual processor).

[0093] consider Figure 3 The following sequence of events on a single physical processor (pPE) 4 is shown:

[0094] 1. vPE A is running in a non-interrupt context and the interrupt controller (or doorbell physical interrupt control circuit 20) is configured to generate a physical doorbell when INT1 is asserted for vPE B.

[0095] 2. INT1 is asserted, generating a physical doorbell to the hypervisor, which preempts the execution of vPE A.

[0096] 3. The hypervisor schedules vPE B to handle the more important interrupt INT1 and configures the doorbell to be generated if a virtual interrupt (eg, INT0 or INT2) is asserted to be handled by the vPE.

[0097] 4. vPE B starts running the interrupt processing logic for INT1.

[0098] 5. INT2 is asserted, generating a physical doorbell to the hypervisor, which preempts execution of vPE B.

[0099] Therefore, the low priority interrupt INT2 has caused preemption of the vPE that was processing the higher priority interrupt INT1.

[0100] This problem may be addressed by providing doorbell enable priority configuration data 28, as this allows the hypervisor to define which priorities of virtual interrupts for vPE A should cause doorbell physical interrupts to be generated when configuring the doorbell for vPE A, e.g. defining one or more lower priorities as disabled for doorbell physical interrupt generation.

[0101] Figure 4 A method of handling an incoming interrupt is illustrated. At step 100, an incoming interrupt is detected, which is to be raised as a given virtual interrupt for a given virtual interrupt handling context. The given virtual interrupt has a given priority. At step 106, the doorbell physical interrupt generation circuit 24 determines the doorbell enable priority configuration data 28 (in Figure 2 In an example of , it is the doorbell enable priority configuration data of a given virtual interrupt handling context) whether the given priority is indicated to be enabled for doorbell physical interrupt generation. If the given priority is indicated to be enabled for doorbell physical interrupt generation, then at step 108, the doorbell physical interrupt generation circuit 24 generates a doorbell physical interrupt to be processed in the given physical interrupt handling context. The doorbell configuration data 30 can be used to determine which physical interrupt handling context should be used to cause the doorbell physical interrupt and other information about the doorbell physical interrupt, such as its interrupt identifier. The doorbell physical interrupt identifies to the physical processor that is handling the interrupt for the given physical interrupt handling context that the given virtual interrupt is to be handled in the given virtual interrupt handling context. For example, the doorbell physical interrupt can cause the hypervisor to reschedule which virtual processor is executing on the physical processor so that the virtual processor that handles the interrupt for the given virtual interrupt handling context can service the given virtual interrupt. If, at step 106, the doorbell physical interrupt generation circuit 24 determines that the given priority is disabled for doorbell physical interrupt generation, then at step 110, the doorbell physical interrupt is not generated.

[0102] In some examples, a priority mask may also be supported to define a threshold priority that is used to compare with the priority of a pending physical interrupt to determine whether to signal the pending physical interrupt to the corresponding physical processor that handles the physical interrupt for the corresponding physical interrupt handling context. In such examples, if masking based on the priority mask is enabled, a further check may also be performed at step 108 to determine whether the priority of the generated doorbell physical interrupt is higher than the threshold physical interrupt priority represented by the priority mask. Note that in the example where a threshold is used to implement the doorbell enable priority configuration data, the threshold priority defined by the priority mask will be a threshold separate from the threshold defined by the doorbell enable priority configuration data 28. This is because the (physical interrupt) priority mask will be a control imposed on physical interrupts (generally, not just doorbell interrupts), and the priority mask threshold will be compared with the priority of the physical interrupt to determine whether the physical interrupt should be signaled to the processor 4. Conversely, if a threshold is used to define the doorbell enable priority configuration data, the threshold is compared with the priority of the virtual interrupt to determine whether the doorbell physical interrupt should be generated.

[0103] Figure 5 It shows the configuration Figure 2 Flowchart of a method for configuring the doorbell enable priority configuration data 28 in an example of . The method is performed by software executed on the processor 4. For example, the method may be performed by a hypervisor software. It should be understood that the hardware of the device 2 does not have any hardware circuit logic components for performing the method.

[0104] exist Figure 5 At step 150, the hypervisor detects that a first virtual interrupt has been raised for a first virtual interrupt handling context, and the interrupt for the first virtual interrupt handling context is handled by the first virtual processor. For example, the hypervisor may detect the first virtual interrupt because the first virtual interrupt has caused a doorbell physical interrupt to be delivered to a physical processor that is executing the first virtual processor, and this caused the hypervisor to be executed. The first virtual interrupt has a first priority. In response to the detection of the first virtual interrupt, at step 152, the hypervisor detects whether the first priority is higher than the current priority associated with processing on a second virtual processor, which resides on a given physical processor (executing before the interrupt to handle the hypervisor) and is handling a virtual interrupt for the second virtual interrupt handling context. If the first priority is not higher than the current priority of the processing on the second virtual processor, then at step 158, the hypervisor returns processing to the second virtual processor, and the second virtual processor continues to process the task with the current priority.

[0105] On the other hand, if the first priority is higher than the current priority, then at step 154, the hypervisor configures the doorbell configuration data 30 to define a doorbell physical interrupt that is to be generated in response to the detected virtual interrupt for the second virtual interrupt handling context (the interrupt for which is handled by the second virtual processor that is now made not resident on the physical processor). The hypervisor also sets the doorbell enable priority configuration data 28 associated with the second virtual interrupt handling context to indicate that one or more priorities selected based on the first priority (of the first virtual interrupt detected for the first virtual interrupt handling context) are disabled for the generation of the doorbell physical interrupt. Therefore, it is noted that the selection of the priority for the second virtual interrupt handling context for which the doorbell physical interrupt is to be disabled depends on the first priority of the virtual interrupt detected for the first virtual interrupt handling context (rather than on the current priority of the processing that was previously being processed by the second virtual processor that handled the interrupt for the second virtual interrupt handling context). This is because the purpose of disabling the doorbell physical interrupt for the second virtual interrupt handling context is to protect the first virtual interrupt handling context from interrupts based on interrupts that are less important than the interrupt being handled by the first virtual interrupt handling context. That is, when a first virtual interrupt preempts processing on a second virtual processor, it can be assumed that any virtual interrupt to the second virtual processor that is deemed to be less important than the first virtual interrupt should preferably not cause a doorbell physical interrupt to be generated, as this would interrupt the higher priority processing that will now be scheduled on the given physical processor using the first virtual processor.

[0106] In some examples, the priority generated for the disabled doorbell physical interrupt for the second virtual interrupt handling context may simply be a priority lower than or equal to the first priority. The method may assume that a given priority in the first virtual interrupt handling context may be considered to have equal importance to the same priority in the second virtual interrupt handling context.

[0107] In other examples, if the hypervisor knows the mapping between the relative importance of the priority levels of the first interrupt handling context and the priority levels of the second interrupt handling context, the hypervisor may select those priorities of the second virtual interrupt handling context as priorities to be disabled for generation of the doorbell physical interrupt, which priorities of the second virtual interrupt handling context mapped into the first virtual interrupt handling context will be lower than or equal to the priorities of the first priorities.

[0108] At step 156, after the doorbell interrupt is configured and the doorbell enable priority configuration data 28 associated with the second virtual interrupt handling context is set, the hypervisor controls the given physical processor to reschedule which virtual processor is executing so that the first virtual processor is now processed instead of the second virtual processor. The first virtual processor can now service the first virtual interrupt detected at step 150.

[0109] Figure 6 is shown with Figure 3 , but now the doorbell enable priority configuration data 28 is applied to suppress the generation of the doorbell physical interrupt when a virtual interrupt with a certain interrupt priority is detected (for which a corresponding doorbell physical interrupt has been configured using the doorbell configuration data 30), and the interrupt priority is indicated by the doorbell enable priority configuration data 28 as being disabled for the doorbell physical interrupt generation.

[0110] Therefore, in Figure 6 middle:

[0111] 1. vPE A is running in a non-interrupt context and the interrupt controller (or doorbell physical interrupt control circuit 20) is configured to generate a physical doorbell when INT1 is asserted for vPE B.

[0112] 2. INT1 is asserted, generating a physical doorbell to the hypervisor, which preempts the execution of vPE A.

[0113] 3. The hypervisor schedules vPE B to handle the more important interrupt INT1, and configures the doorbell to be generated if a virtual interrupt is asserted to be handled by vPE A. At this point, the hypervisor also sets the doorbell enable priority configuration data 28 to indicate that priorities lower than or equal to INT1 are disabled for the generation of doorbell physical interrupts.

[0114] 4. vPE B starts running the interrupt processing logic for INT1.

[0115] 5.INT2 is asserted, but with Figure 3 Unlike in FIG. 1 , since the doorbell enable priority configuration data 28 specifies that interrupt priority 2 is disabled for the generation of the doorbell physical interrupt, the doorbell physical interrupt is not generated, and thus vPE B is not interrupted and can continue to process the higher priority interrupt INT1. On the contrary, if the interrupt asserted for vPE A is the higher priority interrupt INT0, the doorbell physical interrupt will still be generated.

[0116] Figure 7A first example of an interrupt control configuration table that may be used to define doorbell tag priority configuration data 28 is shown. An interrupt raised by an interrupt source 14 specifies a device identifier (ID) identifying the interrupt source and an event ID identifying the type of event that has occurred (the event ID may be used to select an interrupt identifier for an interrupt to be signaled in response to the event). The device ID is used to select an interrupt translation table (ITT) base address 200 from among a plurality of base addresses defined for different devices (interrupt sources). Thus, different ITT structures may be used for different interrupt sources, which is useful because different interrupt sources may use the same event ID to refer to different events that need to be mapped to different interrupts.

[0117] The event ID is used to select an entry from the ITT 202 that is accessed using the base address 200, and the selected ITT entry provides a number of pieces of information about the interrupt that will be generated, including, for example:

[0118] The interrupt ID of the interrupt that will be generated in response to the event;

[0119] An indication of whether the interrupt is signaled as a virtual interrupt or a physical interrupt;

[0120] In case the interrupt is to be signaled as a virtual interrupt, the virtual processor ID indicating the virtual processor that will service the virtual interrupt (virtual interrupt handling context)

[0121] (vPE ID), and if a doorbell interrupt has been configured for this virtual interrupt, the doorbell interrupt ID that will be used for the doorbell physical interrupt that will be generated when this virtual interrupt is raised;

[0122] In the case where the interrupt is to be signaled as a physical interrupt, a set ID for selecting an entry from the set table 206 that maps each set ID to an identifier of a corresponding physical processor (or alternatively, an interrupt redistributor ID that identifies the node of the integrated circuit to which the interrupt is to be distributed by the interrupt controller). Thus, each set ID corresponds to a set of interrupt IDs that are to be distributed to the same physical processor. In embodiments where the interrupt distribution bus 16 is used to distribute interrupts, this facilitates the physical routing of interrupts. In embodiments where memory writes and coherence mechanisms are used to distribute interrupts via the cache coherent interconnect 6, the set table 206 may alternatively provide a base address of a data structure that is used to track the physical interrupt for a given physical interrupt handling context in which a physical interrupt is to be raised.

[0123] For a virtual interrupt, the virtual processor ID selects an entry from the virtual processor table 204 that provides information including:

[0124] The base address of the interrupt pending table of the virtual processor, into which the pending virtual interrupts will be injected;

[0125] · A target physical processor ID (or alternatively, an interrupt redistributor ID), which, in an embodiment using the interrupt dispatch bus 16, identifies the physical processor or circuit node to which the doorbell interrupt will be dispatched (alternatively, the vPE table 204 may identify the base address of a data structure that is used to track physical interrupts for a given physical interrupt handling context in which the doorbell physical interrupt will be raised).

[0126] Doorbell enable priority configuration data 28 corresponding to the virtual processor (virtual interrupt handling context).

[0127] Thus, in this example, the doorbell enable priority configuration data 28 for a given virtual interrupt handling context is provided by an entry in the virtual processor table 204 that corresponds to the virtual processor that handles the interrupt for the given virtual interrupt handling context.

[0128] It should be understood that Figure 7 The table structure shown and the information provided in each table is only one example, and other examples may organize the information differently or provide other types of information in the tables.Tables 202, 204, 206 may be memory-based table structures that may be set by software to control the routing of interrupts.

[0129] Figure 8A second example of a data structure that may be used to control the generation of a doorbell physical interrupt is shown. This example may be used when the doorbell physical interrupt control circuit 20 is implemented at the processor 4 or as a separate unit from the interrupt controller 10. In this example, the interrupt controller 10 allocates virtual interrupts and physical interrupts by updating corresponding memory-based interrupt tracking structures 250, 252 by issuing a memory write request on the cache coherent interconnect 6, which causes the interconnect to snoop the cache of one or more processors 4 that hold the corresponding address in its cache in an exclusive state, and the invalidation of the cache data triggered by the snoop can be detected by the processor 4, so that the processor 4 can be informed of the pending interrupt. Therefore, when a virtual interrupt needs to be signaled, the interrupt controller 10 writes to a set of interrupt tracking structures 250 for a given virtual interrupt handling context in which the virtual interrupt will be caused. The interrupt detection circuit 22 of the doorbell physical interrupt control circuit 20 detects a pending virtual interrupt, for example based on a snoop generated by the coherent interconnect 6, and if the doorbell physical interrupt has been configured using the doorbell configuration data 30, and the doorbell enable priority configuration data 28 of the given virtual interrupt handling context specifies that a given priority of a given virtual interrupt is enabled for doorbell physical interrupt generation, then this causes the doorbell physical interrupt generation circuit 24 to generate a doorbell physical interrupt. If the doorbell physical interrupt is generated, the doorbell physical interrupt generation circuit 24 writes to a set of interrupt tracking structures 252 for the corresponding physical interrupt handling context to signal that the doorbell physical interrupt is pending. Thus, the physical processor 4 monitoring the address in the structure 252 can detect the physical interrupt, and this can prompt the hypervisor at the physical processor to reschedule the virtual processor execution to handle the initial virtual interrupt signaled by the interrupt controller 10.

[0130] Figure 8 Two examples of how the doorbell enable priority configuration data 28 may be provided to the doorbell physical interrupt control circuit 20 are shown. In one example, the doorbell enable priority configuration data 28 may be provided within a set of interrupt tracking structures 250 for a given virtual interrupt handling context. For example, an interrupt descriptor table defining information about each interrupt identifier for a corresponding virtual interrupt handling context (e.g., enablement status, priority) may also include a value specifying a doorbell interrupt generation priority threshold, or an indicator specifying whether each priority is enabled / disabled.

[0131] In another example, the doorbell enable priority configuration data 28 may be provided within a portion of the doorbell configuration data structure 30 that is used to define other information for generating a doorbell physical interrupt. For example, the doorbell configuration data structure 30 may include an address 260 of a virtual interrupt handling structure 250 for a virtual interrupt handling context for which a doorbell interrupt is to be generated, a physical interrupt number 262 for the doorbell interrupt, and a physical interrupt tracking structure address 264 that identifies a physical interrupt tracking structure 252 in which the doorbell interrupt is to be signaled as pending. Thus, when a virtual interrupt is detected from an interrupt tracking structure 250 corresponding to an address 260 specified by a given entry of the doorbell configuration data structure 30, the priority of the virtual interrupt is compared to the doorbell enable priority configuration data 28 in that entry to determine whether the priority is enabled for doorbell physical interrupt generation, and if so, a doorbell physical interrupt having a physical identifier 262 is signaled by updating the tracking structure 252 represented by the address 264.

[0132] There may also be other ways of providing the doorbell enable priority configuration data 28. For example, the doorbell enable priority configuration data 28 may be defined in another data structure stored in memory, separate from the virtual interrupt tracking structure 250 and the doorbell configuration data structure 30. In addition, the doorbell enable priority configuration data 28 may be stored in a memory mapped register of the doorbell physical interrupt control circuit 20.

[0133] Fig. 9 Another example of system 2 is shown, which is Figure 1 , but in this example, each physical processor 4 also maintains a running virtual interrupt priority indication 300, which indicates the virtual interrupt priority of the current process on the resident virtual processor resident on the physical processor. In this example, the running virtual interrupt priority indication 300 maintained by each physical processor can be exposed to the doorbell interrupt control circuit 20 for determining whether to generate a doorbell physical interrupt signaled to the corresponding physical processor.

[0134] Fig.10 An example of hypervisor-maintained information for indicating whether a running virtual interrupt priority indication for a given physical processor should be used to determine whether a doorbell physical interrupt is generated for the given physical processor is illustrated. This information is accessible to the doorbell interrupt control circuit 20 and can be set by hypervisor software executing on one of the physical processors 4 (e.g., by writing to a memory address assigned to a structure that provides hypervisor-maintained information). Fig.10As shown, for each virtual processor (or each virtual interrupt handling context), the doorbell physical interrupt control circuit 20 can access a corresponding running priority control indicator (e.g., a flag) 302, which is set by the hypervisor to indicate whether the generation of the doorbell physical interrupt should depend on the running virtual interrupt priority indication 300 of the physical processor to which the doorbell physical interrupt will be signaled in the event of the doorbell physical interrupt being generated. The hypervisor can set the flag 302 so that, for example, a resident virtual processor does not set its running priority control flag 302, and a non-resident virtual processor for which doorbell generation should not depend on the running virtual interrupt priority indication 300 also does not set its running priority control flag 302, but a non-resident virtual processor for which the running virtual interrupt priority indication 300 should be considered when generating a doorbell interrupt can set its running priority control flag 302.

[0135] Thus, if the virtual processor (or virtual interrupt handling context) for which a virtual interrupt is to be signaled as pending has the corresponding run priority control flag 302 set, then once the physical processor to which the doorbell interrupt is to be sent is identified (e.g., based on the example of Figure 7 The doorbell interrupt control circuit 20 uses the running virtual priority to determine whether the priority of the virtual interrupt is sufficient to justify the generation of a doorbell physical interrupt. For example, the doorbell interrupt control circuit 20 may assume that both the resident virtual processor and the non-resident virtual processor that will handle the incoming interrupt have priority levels that are set equally, and therefore, if the virtual interrupt to be signaled has a priority higher than the priority indicated by the running virtual interrupt at the physical processor to which the doorbell physical interrupt will be sent if a doorbell physical interrupt is generated, a doorbell physical interrupt may be generated.

[0136] like Fig.10 As shown by the dashed line in , in some examples, in an implementation scheme that uses a running virtual priority indication to control the generation of a doorbell physical interrupt, it is not necessary to provide software-specified doorbell enable priority configuration data 28 (provided per virtual interrupt handling context).

[0137] Still other examples may provide doorbell enable priority configuration data 28 per virtual interrupt handling context, and determine whether to generate a doorbell physical interrupt based on both the software-specified doorbell enable priority configuration data 28 for a given virtual interrupt handling context in which a virtual interrupt is to be signaled as pending and the running virtual interrupt priority of the corresponding physical processor to which the doorbell physical interrupt is to be sent—in which case, if the software-specified doorbell enable priority configuration data 28 in a given virtual interrupt handling context or the running virtual interrupt priority of the corresponding physical processor indicates that the priority of the virtual interrupt to be signaled as pending in the given virtual interrupt handling context is insufficient to justify the generation of a doorbell interrupt, then the doorbell physical interrupt is suppressed. As previously described with respect to the configuration data 28 itself, the running priority control flag 302 may be maintained in various structures, for example, in the interrupt tracking structure 250 or in the doorbell configuration data structure 30.

[0138] Fig.11 A method is illustrated. Steps 100, 108 and 110 are similar to Figure 4 In this example, in response to detecting an incoming interrupt at step 100, at step 306, an optional check is included to check whether the software-specified doorbell enable priority configuration data 28 specified for a given virtual interrupt handling context indicates that a given priority is enabled for doorbell physical interrupt generation. This is similar to providing software-specified doorbell enable priority configuration data 28. Figure 2 In the example Figure 4 If the software-specified doorbell enable priority configuration data 28 indicates that the given priority is disabled for doorbell physical interrupt generation (e.g., the given priority does not exceed the threshold indicated by the software-specified doorbell enable priority configuration data 28), then at step 110, no doorbell physical interrupt is generated. If the software-specified doorbell enable priority configuration data 28 indicates that the given priority is enabled for doorbell physical interrupt generation, then the method proceeds to step 310. Furthermore, in embodiments that do not support software-specified doorbell enable priority configuration data 28, step 306 is omitted and the method proceeds directly from step 100 to step 310.

[0139] At step 310, the doorbell interrupt control circuit 20 determines whether the run priority control flag 302 for a given virtual interrupt handling context is set by the hypervisor software. If not, the method proceeds to step 108, where the run priority control flag 302 for a given virtual interrupt handling context is set by the hypervisor software. Figure 4 The doorbell physical interrupt is generated in the same manner as discussed in step 108 .

[0140] If the doorbell interrupt control circuit 20 determines at step 310 that the running priority control flag 302 for the given virtual interrupt handling context has been set by the hypervisor software, then at step 312, the doorbell interrupt control circuit 20 obtains the running virtual interrupt priority indication 300 of the physical processor 4 to which the doorbell physical interrupt will be signaled if the doorbell physical interrupt is generated. The doorbell interrupt control circuit 20 determines whether the given priority of the given virtual interrupt is sufficient to preempt the current process on the resident virtual processor based on the priority (of the current process on the resident virtual processor residing on the physical processor) indicated by the running virtual interrupt priority indication 300. Some specific implementations may assume that the resident virtual processor and the virtual processor that will handle the given virtual interrupt have equal virtual interrupt priorities, and therefore, when the priority of the given priority is higher than the priority indicated by the running virtual interrupt priority indication 300, the doorbell interrupt control circuit 20 may determine that the given priority is sufficient to preempt the current process. Other examples may support translating priority levels using mapping information that may be set by software to indicate how to map between the priority level of a resident virtual processor and the priority level of a virtual processor that will handle a given virtual interrupt.

[0141] In a specific implementation that implements both the checks at step 306 and 310 / 312, although Fig.11 These checks are shown to be completed sequentially, with step 306 being completed first, and if step 306 passes, steps 310 / 312 are then completed, but other examples may perform these checks in the reverse order or in parallel.

[0142] Fig.12 is a ladder diagram illustrating a situation in which the ability to consider running virtual priorities can be utilized when deciding whether to generate a doorbell physical interrupt. For simplicity, this example assumes that each virtual processor vPE A, vPE B sets a virtual interrupt priority so that a priority level X for virtual processor vPE A can be considered equivalent to the same priority level X for virtual processor vPE B. In this example, the physical processor (pPE) is currently processing an instruction from a resident virtual processor (vPE). The current processing corresponds to a virtual interrupt with a priority level of 32 (in this example, a lower numerical value of the interrupt priority is considered to have a higher priority than a higher numerical value). An incoming interrupt is detected by the interrupt controller 10, which causes a virtual interrupt with a priority level of 30 to be signaled to the resident virtual processor vPE A as pending. This preempts the previous processing with a priority level of 32, and therefore, the running virtual interrupt priority indication 300 on the physical processor pPE is updated to indicate that the running virtual interrupt priority is now a priority level of 30.

[0143] Another incoming interrupt is then detected by the interrupt controller 10 and is signaled to the non-resident virtual processor vPE B as a virtual interrupt with a priority level of 31. The doorbell physical interrupt control circuit 20 determines that the running priority control flag 302 is set for vPE B and, therefore, compares the virtual interrupt priority 31 of the virtual interrupt for vPE B with the priority 30 indicated by the running virtual interrupt priority of the physical processor that will receive the doorbell interrupt. Since the virtual interrupt priority 31 of the virtual interrupt to be signaled to vPE B is lower in priority than the running virtual priority 30 of the physical processor, the doorbell physical interrupt is not generated. This avoids interrupting higher priority processing at pPE 4 based on a lower priority interrupt for the non-resident virtual processor vPE B. Alternatively, vPE B may service the interrupt the next time it becomes the resident virtual processor, or when vPE A completes servicing the interrupt with priority 30, vPE A may return to processing the interrupt with priority 32, and at this point the physical processor's running virtual interrupt priority becomes 32 again, and therefore, the doorbell physical interrupt control circuit 20 can now detect whether the priority of the interrupt to vPE B is sufficient to justify generating a doorbell interrupt.

[0144] Fig.13 Another example is shown for allowing the doorbell physical interrupt control circuit 20 to take into account the running virtual interrupt priority indication 300 of the physical processor without directly exposing the running virtual interrupt priority indication 300 to the doorbell physical interrupt control circuit 20. In contrast, the virtual interrupt handling circuit 350 that maintains the running virtual interrupt priority indication 300 within a given physical processor 4 may also have a doorbell enable priority configuration reprogramming circuit 360 that updates the software-defined doorbell enable priority configuration data 28 of one or more virtual interrupt handling contexts so that the corresponding entry of the doorbell enable priority configuration data 28 can track the latest running virtual interrupt priority at the physical processor 4. Thus, although the software may, for example, based on Figure 5The method of initially setting the doorbell enable priority configuration data 28, but the reprogramming circuit 360 can also update the data 28 in hardware. In a specific implementation where the doorbell physical interrupt control circuit 20 is provided centrally separately from the physical processor 4, the method can avoid the need to route the running virtual interrupt priority indication 300 of each physical processor to the doorbell physical interrupt control circuit 20. The doorbell enable priority configuration reprogramming circuit 360 can access reprogramming control information 370 set by the hypervisor software, which specifies information for identifying a set of target doorbell enable priority configuration data 28 to be updated when the running virtual processor at the physical processor changes. For example, the reprogramming control information 370 can be a list of addresses to be updated along with the updated value of the running virtual priority, or can be a list of vPE identifiers that can be used to look up the vPE table 204 to find the doorbell enable priority configuration data. The hypervisor can update the list of information 370 when rescheduling the virtual processor on the physical processor.

[0145] Therefore, in this example, the control of whether to generate a doorbell physical interrupt can be as follows Figure 4 The example is shown based on the doorbell enable priority configuration data 28 set by software for the virtual interrupt handling context for which a given virtual interrupt will be signaled as pending. It is not necessary to consider the running virtual priority of the physical processor that will receive the doorbell physical interrupt, such as Fig.11 shown.

[0146] However, the doorbell enable priority configuration data 28 set by software may be modified in hardware by the doorbell enable priority configuration reprogramming circuit 360 when there is an event indicating a change in the priority of the running virtual interrupts at the resident virtual processor being processed by the given physical processor, and the doorbell enable priority configuration data 28 is one of the doorbell enable priority configuration data 28 items designated by the reprogramming control information 370 as to be updated in response to such an event. Fig.12 In the case of , the change in the running virtual priority at the physical processor 4 due to vPE A switching to handle higher priority virtual interrupts can be reflected in the doorbell enable priority configuration data 28.

[0147] Fig.14A method performed by a virtual interrupt handling circuit 350 at a given physical processor 4 is shown. At step 400, an event is detected that indicates a change in the running virtual interrupt priority level associated with a current process on a resident virtual processor resident on the given physical processor 4. For example, the event may be the receipt of a pending virtual interrupt of a higher priority or a return to a lower priority process after processing a previously accepted virtual interrupt. At step 402, the virtual interrupt handling circuit 350 of the physical processor 4 updates the running virtual interrupt priority indication 300 to take into account the change in the running priority level after the event detected at step 400.

[0148] At step 404, the virtual interrupt handling circuit 350 determines whether the hypervisor has set reprogramming control information 370 (e.g., a data structure in memory located at an address set in a configuration register of the virtual interrupt handling circuit 350) indicating the doorbell enable priority configuration data 28 to be updated in response to an event indicating a change in the priority of the running virtual interrupt. If valid reprogramming control information 370 has not been set, the method returns to step 400 to wait for another event.

[0149] However, if valid reprogramming control information 370 has been set, then at step 406, the doorbell enable priority configuration reprogramming circuit 360 reprograms the doorbell enable priority configuration data 28 corresponding to the virtual interrupt handling context of one or more other virtual processors (except the resident virtual processor) based on the updated running virtual interrupt priority indication 300. The updated data is identified using the reprogramming control information 370 set by the hypervisor.

[0150] For example, the threshold set for the doorbell enable priority configuration data 28 may be set corresponding to the new value of the running virtual interrupt priority indication 300 resulting from the update in step 402. Again, this provides a way to allow the doorbell physical interrupt control circuitry 20 to determine the priority of a process currently executing at a physical processor to help decide whether to preempt that process by delivering a doorbell physical interrupt in response to a given virtual interrupt of a given priority level.

[0151] The concepts described herein may be embodied in computer readable code for manufacturing devices embodying the described concepts. For example, the computer readable code may be used at one or more stages of a semiconductor design and manufacturing process, including an electronic design automation (EDA) stage, to manufacture integrated circuits including devices embodying the concepts. The computer readable code may additionally or alternatively enable the definition, modeling, simulation, verification, and / or testing of devices embodying the concepts described herein.

[0152] For example, computer readable code for making an apparatus embodying the concepts described herein may be embodied in code defining a hardware description language (HDL) representation of the concepts. For example, the code may define a register-transfer-level (RTL) abstraction of one or more logic circuits for defining an apparatus embodying the concepts. The code may define an HDL representation of one or more logic circuits embodying the apparatus in Verilog, SystemVerilog, Chisel, or VHDL (Very High Speed ​​Integrated Circuit Hardware Description Language), as well as intermediate representations such as FIRRTL. The computer readable code may provide definitions of concepts embodying concepts or other behavioral representations of concepts using system-level modeling languages ​​such as SystemC and SystemVerilog that may be interpreted by a computer to enable simulation, functional and / or formal verification, and testing of the concepts.

[0153] Additionally or alternatively, the computer readable code may define a low-level description of an integrated circuit component embodying the concepts described herein, such as one or more netlists or integrated circuit layout definitions, including representations such as GDSII. One or more netlists or other computer readable representations of integrated circuit components may be generated by applying one or more logic synthesis processes to the RTL representation to generate definitions for manufacturing devices embodying the present invention. Alternatively or additionally, one or more logic synthesis processes may generate from the computer readable code a bitstream to be loaded into a field programmable gate array (FPGA) to configure the FPGA to embody the described concepts. The FPGA may be deployed for the purpose of verifying and testing concepts prior to manufacturing the integrated circuit, or the FPGA may be deployed directly in a product.

[0154] The computer readable code may include a mixture of code representations for making a device, such as a mixture including one or more of an RTL representation, a netlist representation, or another computer readable definition used in a semiconductor design and manufacturing process to make a device embodying the invention. Alternatively or additionally, concepts may be defined in a combination of a computer readable definition used in a semiconductor design and manufacturing process to make a device and computer readable code defining instructions to be executed by the defined device once manufactured.

[0155] Such computer readable code may be provided in any known transient computer readable medium (such as wired or wireless transmission of the code over a network) or non-transitory computer readable medium such as a semiconductor, a magnetic disk, or an optical disk. An integrated circuit fabricated using the computer readable code may include components such as one or more of the following: a central processing unit, a graphics processing unit, a neural processing unit, a digital signal processor, or other components that embody the concept individually or collectively.

[0156] In this application, the words "configured to..." are used to mean that elements of a device have a configuration that enables the performance of a defined operation. In this context, "configuration" means the arrangement or manner of interconnection of hardware or software. For example, the device may have dedicated hardware that provides the defined operation, or a processor or other processing device may be programmed to perform the function. "Configured to" does not mean that the device elements need to be changed in any way in order to provide the defined operation.

[0157] Although exemplary embodiments of the present invention have been described in detail herein with reference to the accompanying drawings, it should be understood that the invention is not limited to those precise embodiments and that various changes and modifications may be effected therein by those skilled in the art without departing from the scope of the invention as defined in the appended claims.

Claims

1. A doorbell physical interruption control circuit, the doorbell physical interruption control circuit comprising: interrupt detection circuitry for detecting an incoming interrupt to be raised as a given virtual interrupt for a given virtual interrupt handling context, the given virtual interrupt having a given priority; and a doorbell physical interrupt generation circuit, the doorbell physical interrupt generation circuit being responsive to detection of the incoming interrupt by the interrupt detection circuit: determining whether the given priority level of the given virtual interrupt is indicated by doorbell enable priority configuration data as being enabled for doorbell physical interrupt generation, the doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation; as well as In response to determining that the doorbell enable priority configuration data indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, a doorbell physical interrupt is generated to be processed in a given physical interrupt handling context, the doorbell physical interrupt comprising a physical interrupt for indicating to a physical processor handling an interrupt for the given physical interrupt handling context that the given virtual interrupt is to be handled in the given virtual interrupt handling context.

2. The doorbell physical interrupt control circuit of claim 1 , wherein the doorbell enable priority configuration data comprises software programmable doorbell enable priority configuration data associated with the given virtual interrupt handling context, the software programmable doorbell enable priority configuration data indicating which interrupt priorities are enabled or disabled for doorbell physical interrupt generation for the given virtual interrupt handling context.

3. The doorbell physical interrupt control circuit according to claim 2, wherein the doorbell physical interrupt generation circuit is configured to select the software programmable doorbell enable priority configuration data from multiple sets of software programmable doorbell enable priority configuration data associated with the corresponding virtual interrupt handling context based on the context identification value associated with the given virtual interrupt handling context.

4. The doorbell physical interrupt control circuit according to claim 3, wherein the context identification value comprises one of the following: a virtual processor identifier indicating a virtual processor that handles a virtual interrupt for the given virtual interrupt handling context; and At least one address indicating a memory location of a virtual interrupt tracking structure for tracking pending virtual interrupts for the given virtual interrupt handling context.

5. The doorbell physical interrupt control circuit according to any one of claims 3 and 4, wherein the plurality of sets of software programmable doorbell enable priority configuration data are defined in one of the following: at least one memory-based table stored in the memory system; and Register storage device.

6. A doorbell physical interrupt control circuit according to any one of claims 2 to 5, wherein the software programmable doorbell enable priority configuration data includes a threshold value, and the doorbell physical interrupt generation circuit is configured to determine that the given priority is enabled for doorbell physical interrupt generation when the given priority has a priority higher than the doorbell physical interrupt generation threshold priority indicated by the threshold value.

7. The doorbell physical interruption control circuit according to claim 6, wherein: When the doorbell physical interrupt generation threshold priority level has a priority level higher than the given priority level, the doorbell physical interrupt generation circuit is configured to suppress generation of the doorbell physical interrupt even when the given virtual interrupt is signaled as pending in the given virtual interrupt handling context.

8. The doorbell physical interrupt control circuit according to any one of claims 6 and 7, wherein the doorbell physical interrupt generation threshold priority is defined separately from the physical interrupt mask threshold priority, and the physical interrupt mask threshold priority is used to compare with the priority of a given physical interrupt to determine whether to signal the given physical interrupt to the physical processor that handles the interrupt for the physical interrupt handling context corresponding to the given physical interrupt.

9. A doorbell physical interrupt control circuit according to any one of claims 6 to 8, wherein the threshold value can be set to specify a priority level other than the priority level associated with the current point in the program flow reached by the virtual processor handling the interrupt for the given virtual interrupt handling context as generating the threshold priority level for the doorbell physical interrupt.

10. The doorbell physical interrupt control circuit according to any one of claims 2 to 5, wherein the software programmable doorbell enable priority configuration data includes a set of priority indicators, each priority indicator corresponding to a respective priority and indicating whether the priority is enabled or disabled for doorbell physical interrupt generation.

11. A doorbell physical interrupt control circuit according to any preceding claim, wherein the doorbell enable priority configuration data includes a running virtual interrupt priority indication, the running virtual interrupt priority indication indicating the virtual interrupt priority of a current process residing on a resident virtual processor on the physical processor to which the doorbell physical interrupt is to be signaled if the doorbell physical interrupt is generated.

12. The doorbell physical interrupt control circuit of claim 11 , wherein in response to the detection of the incoming interrupt, the doorbell physical interrupt generation circuit is configured to: determining, based on the running virtual interrupt priority indication of the physical processor to which the doorbell physical interrupt is to be signaled if the doorbell physical interrupt is generated, whether the given priority level of the given virtual interrupt is of sufficient priority to preempt the current processing on the resident virtual processor; Responsive to determining that the running virtual interrupt priority indication indicates that the given priority level of the given virtual interrupt is insufficiently prioritized to preempt the current processing on the resident virtual processor, suppressing generation of the doorbell physical interrupt.

13. The doorbell physical interrupt control circuit according to any one of claims 11 and 12, wherein the doorbell enable priority configuration data further comprises software programmable data associated with the given virtual interrupt handling context, the software programmable data indicating which interrupt priorities are enabled or disabled for doorbell physical interrupt generation for the given virtual interrupt handling context; In response to determining that the running virtual interrupt priority indication indicates that the given priority level of the given virtual interrupt is sufficient to preempt the current processing and the software programmable data associated with the given virtual interrupt handling context indicates that the given priority level of the given virtual interrupt is enabled for doorbell physical interrupt generation, the doorbell physical interrupt is generated.

14. The doorbell physical interrupt control circuit according to any preceding claim, wherein the doorbell physical interrupt generation circuit is configured to distribute the doorbell physical interrupt to the physical processor by one of the following: asserting a physical interrupt signal on an interrupt bus; and At least one memory-based physical interrupt tracking structure is updated to indicate that the doorbell physical interrupt is pending.

15. An interrupt controller, the interrupt controller comprising: A virtual interrupt injection circuit, the virtual interrupt injection circuit is used to support the injection of virtual interrupts directly into a virtual processor executing on a physical processor; and A doorbell physical interrupt control circuit as claimed in any preceding claim.

16. A device, comprising: One or more physical processors; an interrupt controller, the interrupt controller being used to allocate virtual interrupts and physical interrupts to the one or more physical processors; and A doorbell physical interruption control circuit according to any one of claims 1 to 14.

17. The device of claim 16, wherein the doorbell physical interrupt control circuit is composed of one of the following: the interrupt controller; at least one physical processor of the one or more physical processors; and A dedicated doorbell physical interrupt generation unit is separate from the one or more physical processors and the interrupt controller.

18. The apparatus of any one of claims 16 and 17, wherein each physical processor is configured to treat a physical interrupt as having a higher priority than a virtual interrupt, regardless of a priority specified by software for the physical interrupt or the virtual interrupt.

19. An apparatus according to any one of claims 16 to 18, wherein the doorbell enable priority configuration data comprises software programmable doorbell enable priority configuration data associated with the given virtual interrupt handling context, the software programmable doorbell enable priority configuration data indicating which interrupt priorities are enabled or disabled for doorbell physical interrupt generation for the given virtual interrupt handling context; and A given physical processor includes a doorbell enable priority configuration data reprogramming circuit, the doorbell enable priority configuration data reprogramming circuit being configured to: detecting an event indicative of a change in a running virtual interrupt priority level associated with the given physical processor, the running virtual interrupt priority level indicative of a virtual interrupt priority level associated with a current process of a resident virtual processor resident on the given physical processor; and In response to detecting the event indicating a change in the running virtual interrupt priority, reprogramming a set of target software programmable doorbell enable priority configuration data for one or more virtual interrupt handling contexts based on an updated value of the running virtual interrupt priority after the event.

20. The apparatus of claim 19, wherein the doorbell enable priority configuration data reprogramming circuit is configured to identify the set of target software programmable doorbell enable priority configuration data based on software programmable control information.

21. A computer-readable medium for storing code for manufacturing the doorbell physical interrupt control circuit according to any one of claims 1 to 14, the interrupt controller according to claim 15, or the device according to any one of claims 16 to 20.

22. A method comprising: detecting an incoming interrupt to be raised as a given virtual interrupt for a given virtual interrupt handling context, the given virtual interrupt having a given priority; as well as In response to detecting the incoming interrupt: determining whether the given priority level of the given virtual interrupt is indicated by the doorbell enable priority configuration data as being enabled for doorbell physical interrupt generation, the doorbell enable priority configuration data indicating which virtual interrupt priorities are enabled for doorbell physical interrupt generation; as well as In response to determining that the doorbell enable priority configuration data indicates that the given priority of the given virtual interrupt is enabled for doorbell physical interrupt generation, a doorbell physical interrupt is generated to be processed in a given physical interrupt handling context, the doorbell physical interrupt comprising a physical interrupt for indicating to a physical processor handling an interrupt for the given physical interrupt handling context that the given virtual interrupt is to be handled in the given virtual interrupt handling context.

23. A method comprising: In response to detecting that a first virtual interrupt of a first virtual interrupt handling context is to be serviced by a first virtual processor that handles a virtual interrupt for the first virtual interrupt handling context: determining whether a first priority level of the first virtual interrupt is higher than a current priority level associated with a process executed by a second virtual processor executing on the given physical processor, the second virtual processor handling a virtual interrupt for a second virtual interrupt handling context; and In response to determining that the first priority is higher than the current priority: setting doorbell enable priority configuration data associated with the second virtual interrupt handling context, the doorbell enable priority configuration data comprising software programmable data indicating, for the given virtual interrupt handling context, which priorities of interrupts are enabled or disabled for generation of a doorbell physical interrupt, the doorbell physical interrupt being used to indicate that a virtual interrupt is pending; as well as rescheduling the given physical processor to process the first virtual processor; wherein the doorbell enable priority configuration data associated with the second virtual interrupt handling context is configured to indicate that generation of the doorbell physical interrupt is disabled based on one or more priorities selected based on the first priority of the first virtual interrupt detected for the first virtual interrupt handling context.

24. A computer program, which, when executed by a computer, controls the computer to perform the method according to claim 23.

Images