A PCIe device and a method and system for dynamically allocating global interrupt resources thereof

Through the linked list, the global interrupt resources are managed and the actual interrupt resource links are maintained for each Function, and the interrupt resource vector is dynamically allocated, which solves the problem of wasted storage resources in PCIe devices, improves resource utilization efficiency and supports rapid virtual machine migration and recycling.

CN117519970BActive Publication Date: 2024-11-05CHENGDU BEIZHONG NETCORE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311466797.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-03
Publication Date
2024-11-05
Estimated Expiration
2043-11-03

AI Technical Summary

Technical Problem

The existing PCIe device interrupt resource allocation method leads to wasting of device storage resources, especially when the number of functions is large and the number of interrupt vectors is limited in each function.

Method used

The global interrupt resource is managed using linked lists, and a practical interrupt resource link is maintained for each function. The interrupt resource vector is dynamically allocated according to the working status and actual needs of the Function to realize the dynamic allocation of global interrupt resources.

Benefits of technology

It effectively improves resource utilization efficiency, avoids resource waste, and quickly updates interrupt resources during virtual machine migration and recycling, ensuring that Functions in non-working states do not occupy interrupt resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117519970B_ABST
    Figure CN117519970B_ABST
Patent Text Reader

Abstract

The present invention discloses a PCIe device and a global interrupt resource dynamic allocation method and system thereof. The method comprises the following steps: constructing a free linked list for the PCIe device, which is used to manage the global interrupt vectors reserved by the PCIe device for its functions; constructing a function linked list for each of the N functions supported by the PCIe device, which is used to manage the interrupt vectors actually used by the corresponding functions; initializing the free linked list and the N function linked lists, and dynamically allocating the interrupt vectors according to the working state of the functions and the actual use needs of the interrupt resources; the dynamic allocation method proposed by the present invention adopts a linked list to manage the global interrupt resources, and maintains an actually used interrupt resource linked list for each function, that is, allocating the corresponding interrupt resource vector according to the actual use needs of the functions, thus avoiding waste of resources and improving resource utilization efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of network storage, and in particular relates to a PCIe device and a global interrupt resource dynamic allocation method and system thereof. Background Art

[0002] In the PCIe bus, the MSI and MSI-X interrupt mechanisms use memory write requests to submit interrupt requests to the processor. The basic principles of the MSI and MSI-X mechanisms are the same, where the MSI terminal mechanism can support up to 32 interrupt requests and requires that the interrupt vectors must be continuous, while the MSI-X interrupt mechanism can support more interrupt requests and does not require the interrupt vectors to be continuous (that is, they can be allocated at will). Compared with the MSI mechanism, the MSI-X interrupt is more widely used.

[0003] In the MSI-X Capability structure, each interrupt request uses an independent Message Address field and Message Data field, so the interrupt controller can allocate interrupt resources to the device more reasonably. MSI-XCapability uses an array to store the Message Address field and the Message Data field. This array is called the MSI-X Table (abbreviated as MSIXT) and is stored in the BAR space of the PCIe device. The MSI-X Table consists of multiple entries (each Function supports up to 2048 entries), and each entry corresponds to an interrupt request. When a PCIe device submits an interrupt request, it writes the Message Data to the Message Address in the MSI-X Table to form a memory write request.

[0004] When there are many PCIe functions and each function is designed according to the maximum number of interrupt vectors specified in the protocol, the PCIe device will consume a large amount of storage resources to store the interrupt vector table. However, in actual applications, the number of interrupt vectors that each function may use is limited and closely related to the business, resulting in a waste of device storage resources. Summary of the invention

[0005] In order to solve the problem that the existing PCIe device interrupt resource allocation technology causes waste of device storage resources, the present invention provides a PCIe device and its global interrupt resource dynamic allocation method and system. The present invention adopts a linked list to manage global interrupt resources, and maintains an actually used interrupt resource linked list for each Function, thereby realizing dynamic allocation of global interrupt resources.

[0006] The present invention is achieved through the following technical solutions:

[0007] A PCIe device and a method and system for dynamically allocating global interrupt resources thereof, the dynamic allocation method comprising:

[0008] Construct a free linked list for the PCIe device, wherein the free linked list is used to manage the global interrupt vectors reserved by the PCIe device for its Function;

[0009] Constructing a function linked list for each of the N functions supported by the PCIe device, wherein the function linked list is used to manage the interrupt vector actually used by the corresponding function;

[0010] The free linked list and the N function linked lists are initialized, and the interrupt vectors are dynamically allocated according to the working status of the Function and the actual use needs of the interrupt resources.

[0011] At present, PCIe devices usually adopt a fixed allocation method for interrupt resources, but in actual applications, Function may not need to use all reserved interrupt vectors, and sometimes some Functions are not in use, resulting in a large amount of storage resource waste. The dynamic allocation method proposed in the present invention adopts a linked list to manage global interrupt resources, and maintains an interrupt resource linked list actually used for each Function, that is, the corresponding interrupt resource vector is allocated according to the actual use needs of the Function, which does not cause resource waste and improves resource utilization efficiency.

[0012] As a preferred embodiment, the dynamic allocation method of the present invention further includes:

[0013] During initialization, all global interrupt vectors reserved by the PCIe device for its Function are hung on the free linked list, and the N function linked lists are empty.

[0014] As a preferred embodiment, the dynamic allocation method of the present invention further includes:

[0015] When a function in the PCIe device needs to use interrupt resources, the required interrupt vector is allocated to the function linked list corresponding to the function. At the same time, for each interrupt vector initialized by the function, an interrupt vector on the free linked list is taken down and hung on the function linked list corresponding to the function, and the information is filled into the corresponding field of the corresponding MSI-X Table Entry.

[0016] As a preferred embodiment, the dynamic allocation method of the present invention further includes:

[0017] When a certain Function in the PCIe device needs to be migrated, all the interrupt vectors existing on the function linked list corresponding to the Function are removed and attached to the function linked list corresponding to the migration target.

[0018] As a preferred embodiment, the dynamic allocation method of the present invention further includes:

[0019] When a certain Function in the PCIe device needs to increase interrupt resources, an interrupt vector is picked up from the free linked list and attached to the end of the function linked list corresponding to the Function.

[0020] As a preferred embodiment, the dynamic allocation method of the present invention further includes:

[0021] When a certain Function in the PCIe device needs to be recycled, all the existing interrupt vectors on the function linked list corresponding to the Function are removed and hung at the end of the free linked list.

[0022] As a preferred embodiment, the dynamic allocation method of the present invention further includes:

[0023] The Function in the non-working state in the PCIe device does not occupy interrupt resources.

[0024] In the process of resource allocation, the present invention only needs to modify the chain head or chain tail to realize the real-time addition, migration or recovery of Function interrupt resources. The operation is simple and convenient, and the interrupt resources can be quickly updated when the virtual machine is migrated and recovered. In addition, the Function in the non-working state will not occupy the interrupt resources, thereby maximizing the use of limited global interrupt resources.

[0025] In a second aspect, the present invention proposes a PCIe device global interrupt resource dynamic allocation system, the dynamic allocation system comprising:

[0026] A linked list construction unit, wherein the linked list construction unit constructs a free linked list for the PCIe device, and the free linked list is used to manage the global interrupt vectors reserved by the PCIe device for its Function; the linked list construction unit also constructs a function linked list for each of the N Functions supported by the PCIe device, and the function linked list is used to manage the interrupt vectors actually used by the corresponding Function;

[0027] A resource control unit, the resource control unit is used to initialize the free list and the N function lists and dynamically allocate interrupt vectors according to the working status of the Function and the actual use needs of the interrupt resources.

[0028] In a third aspect, the present invention proposes a PCIe device, wherein the PCIe device uses the dynamic allocation method of the present invention to dynamically allocate interrupt resources to Functions thereon.

[0029] In a fourth aspect, the present invention proposes an interrupt request method based on the PCIe device of the present invention, the interrupt request method comprising:

[0030] First, find the head of the function list corresponding to Function;

[0031] After that, find the corresponding node of the function linked list according to the Function global interrupt vector number, and use the MSI-X Table ID marked by the node to address the required MSI-X Table;

[0032] Then, get the Message Address information from the MSI-X Table;

[0033] Finally, write the Message Data structure to the Message Address to interrupt the write packet.

[0034] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0035] 1. The PCIe device interrupt resource allocation technology proposed in the present invention allocates corresponding interrupt resource vectors to each Function in the PCIe device according to the interrupt resources actually used by it, that is, the maximum available interrupt resources of each Function in the PCIe device are not limited, as long as the total number of interrupt resources actually used by all Functions does not exceed the global resources, so that there is no need to reserve interrupt resources according to the number of interrupt resources that may be used by the Function, thereby reducing the occupation of PCIe device memory resources; at the same time, more interrupt resources can be allocated to applications with a large number of interrupt requirements, and fewer interrupt resources can be allocated to applications with a small number of interrupt requirements, thereby avoiding waste of resources and effectively improving resource utilization efficiency;

[0036] 2. The PCIe device interrupt resource allocation technology proposed in the present invention only needs to modify the head or tail of the linked list to realize the real-time addition, migration or recovery of interrupt resources during the interrupt resource allocation process. The operation is simple and convenient, and the interrupt resources can be quickly updated when the virtual machine is migrated or recovered; and the Function in the non-working state will not occupy the interrupt resources, thereby maximizing the use of limited global interrupt resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0038] Figure 1 A schematic diagram of the result of fixed allocation of interrupt vector resources adopted by existing PCIe devices.

[0039] Figure 2 The figure is a flow chart of a dynamic allocation method according to an embodiment of the present invention.

[0040] Figure 3 The figure is a principle block diagram of a dynamic allocation system according to an embodiment of the present invention.

[0041] Figure 4 This is a schematic diagram of the results of constructing and initializing a linked list according to an embodiment of the present invention.

[0042] Figure 5 It is a schematic diagram of the result of dynamically allocating interrupt vector resources according to an embodiment of the present invention.

[0043] Figure 6 The figure is a schematic diagram of the migration, recycling and addition operations of interrupt vector resources according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0045] Embodiment 1:

[0046] Currently, PCIe devices usually allocate fixed interrupt vector resources for each Function to store MSIXT mapped in the BAR space, such as Figure 1 As shown, suppose a PCIe device supports 1024 functions and numbers them, reserves 4 interrupt vector resources for even-numbered functions, and reserves 8 interrupt vector resources for odd-numbered functions, then a total of 6144 interrupt vectors need to be reserved globally. If the interrupt resource storage space reserved for each function is large, when the number of functions is large, a large amount of storage resources in the PCIe device will be consumed. In actual applications, the function may not need to use all the reserved interrupt vectors, and sometimes some functions are not in use, which will cause a lot of storage resources to be wasted; if the interrupt resource storage space reserved for each function is small, it may not meet certain business scenarios; in addition, this fixed allocation method is subject to many restrictions when performing virtual machine migration, and the operation is complicated, resulting in low efficiency of migration recovery and update. For example, Figure 1As shown in , an even-numbered Function cannot use more than 4 interrupt vectors. If it is migrated to an odd-numbered position, 4 interrupt vectors will be wasted. An odd-numbered Function cannot use more than 8 interrupt vectors and can only be migrated to an odd-numbered position, otherwise the interrupt vectors will not be enough. In view of this, this embodiment proposes a method for dynamic allocation of global interrupt resources of a PCIe device. The dynamic allocation structure proposed in this embodiment uses a linked list to manage global interrupt resources, and maintains an actually used interrupt resource linked list for each Function, thereby realizing dynamic allocation of global interrupt resources. It should be noted that Function refers to a relatively independent functional logic inside a PCIe device, and each Function has a completely independent configuration space.

[0047] Specific as Figure 2 As shown, the dynamic allocation method proposed in this embodiment includes:

[0048] Build a free list (FreeList) for the PCIe device, which is used to manage the global interrupt vectors (i.e. all reserved interrupt vectors) reserved by the PCIe device for its Function;

[0049] Build a function list for each of the N functions supported by the PCIe device. The function list is used to manage the interrupt vectors actually used by the corresponding functions.

[0050] Initialize the free list and function list and dynamically allocate interrupt vectors according to the Function working status and the actual use of interrupt resources.

[0051] Furthermore, the dynamic allocation method proposed in this embodiment also includes:

[0052] During initialization, all global interrupt vectors reserved by the PCIe device for its Function are hung on the free list, and the N function lists are empty;

[0053] When a Function needs to use interrupt resources, the required interrupt vector is allocated to the function linked list corresponding to the Function. At the same time, for each interrupt vector initialized by the Function, an interrupt vector on the free linked list is taken down and hung on the function linked list corresponding to the Function, and the information is filled in the corresponding field of the corresponding MSI-X Table Entry.

[0054] Furthermore, the dynamic allocation method proposed in this embodiment also includes:

[0055] When a Function needs to be migrated, it is only necessary to remove all interrupt vectors on the function linked list corresponding to the Function and directly attach them to the function linked list corresponding to the migration target.

[0056] Furthermore, the dynamic allocation method proposed in this embodiment also includes:

[0057] When a Function needs to add new interrupt resources, an interrupt vector is taken from the free list and attached to the end of the function list corresponding to the Function.

[0058] Furthermore, the dynamic allocation method proposed in this embodiment also includes:

[0059] When a function needs to be recycled, it is only necessary to remove all interrupt vectors on the function linked list corresponding to the function and directly hang them on the end of the free linked list. The recycled interrupt resources can be allocated to any other function.

[0060] The method proposed in this embodiment does not limit the maximum available interrupt resources of each Function in the PCIe device. As long as the total number of interrupt resources actually used by all Functions does not exceed the global resources, there is no need to reserve global interrupt resources according to the interrupt resources that may be used by the Function, thereby reducing the occupation of PCIe device memory resources. At the same time, more interrupt resources can be allocated to applications with a large number of interrupt requirements. Conversely, fewer interrupt resources can be allocated to applications with a small number of interrupt requirements, thereby realizing dynamic allocation of global interrupt resources, which will not cause waste of resources and effectively improve resource utilization efficiency.

[0061] In addition, in the resource allocation process, the method proposed in this embodiment only needs to modify the chain head or chain tail to achieve real-time addition, migration or recovery of interrupt resources. The operation is simple and convenient, and can quickly update interrupt resources when virtual machines are migrated and recovered. Functions in a non-working state will not occupy interrupt resources, thereby maximizing the use of limited global interrupt resources.

[0062] Based on the same technical concept as above, this embodiment also proposes a PCIe device global interrupt resource dynamic allocation system, specifically as follows: Figure 3 As shown, the system includes:

[0063] A linked list construction unit, which constructs a free linked list for the PCIe device, and the free linked list is used to manage the global interrupt vectors (that is, all reserved interrupt vectors) reserved by the PCIe device for its Function; the linked list construction unit also constructs a function linked list for each of the N Functions supported by the PCIe device, and the function linked list is used to manage the interrupt vectors actually used by the corresponding Function.

[0064] The resource control unit is used to initialize the free list and the function list and dynamically allocate interrupt vectors according to the working status of the Function and the actual use of interrupt resources.

[0065] This embodiment also provides a PCIe device, which uses the above method to dynamically allocate interrupt resources to the Function on it.

[0066] When the PCIe device sends an MSI-X interrupt request, it first finds the corresponding function list chain head, then finds the corresponding function list node according to the Funicon global interrupt vector number, and uses the MSI-X TableID marked by the node to address the required MSI-X Table, and then obtains the Message Address information; writes Message Data to MessageAddress to construct an interrupt write packet. It should be noted that the global interrupt vector number is the number of the reserved global interrupt vector resources, usually starting from 0 and numbered in sequence, for example Figure 1 As shown in , there are 6144 global interrupt vectors, and the global interrupt vector numbers are 0-6143. The function list initializes the interrupt vector table according to the local interrupt vector. For example, the function list 1 (Function1 List) initializes the local interrupt vector 4. Since the starting global interrupt vector number of Function1 is 4, the local interrupt vector 4 corresponds to the global interrupt vector number 4+4=8. The global interrupt vector number 8 is passed in the list. When sending an interrupt, you only need to find the 4th node (value 8) from the head of the function list 1 to get the information required for sending the interrupt from the position 8 of the global interrupt vector table.

[0067] Embodiment 2:

[0068] This embodiment takes a PCIe device as an example to explain in detail the interrupt resource dynamic allocation method and system proposed in the above embodiment. The details are as follows.

[0069] The PCIe device supports 1024 functions and reserves 4096 interrupt vectors globally. Initially, all 4096 global interrupt vectors are hung on the FreeList. Figure 4 shown.

[0070] Each time an interrupt vector is initialized for a function of the PCIe device, an interrupt vector ID on the FreeList is taken and hung on the corresponding FunctionList, and the Message Address and other information are filled into the corresponding fields of the corresponding MSI-XTable Entry. Figure 5 When the PCIe device needs to send an MSI-X interrupt, it must first find the corresponding FunctionList linked list header, then find the Mth node of the corresponding linked list according to the Function local interrupt vector number M, and use the MSIXT ID marked by the Mth node to address the required MSI-X Table, and then obtain information such as the Message Address to construct an interrupt write packet.

[0071] by Figure 5 Take the initialized linked list as an example to perform operations such as adding, migrating, and recycling. Figure 6 shown.

[0072] When FunctionN in the PCIe device needs to add an interrupt vector, that is, initialize a new interrupt vector for FunctionN, first pick a node from the FreeList and hang it at the end of the FunctionN List chain; then fill the Message Address and other information into the corresponding fields of the corresponding MSI-X Table Entry, such as Figure 6 Indicated by the green arrow.

[0073] When Function1 in the PCIe device needs to be recycled, you only need to remove the entire Function1 List chain (that is, all nodes on the chain) and directly hang it to the end of the FreeList chain. The recycled interrupt resources can be allocated to any other function. This operation only requires modifying the end of the FreeList chain, that is, hanging the Function1 List chain head to the end of the FreeList chain, such as Figure 6 Indicated by the red arrow.

[0074] When you need to migrate Function0 to FunctionX, you only need to remove the entire Function0 List chain and directly attach it to FunctionX List to generate FunctionX List. This operation only requires modifying the FunctionXList chain head, that is, filling in the original Function0 List chain head, such as Figure 6 Indicated by the blue arrow.

[0075] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0076] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0078] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0079] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for dynamically allocating global interrupt resources of a PCIe device, characterized in that: The dynamic allocation method comprises: Construct a free linked list for the PCIe device, wherein the free linked list is used to manage the global interrupt vectors reserved by the PCIe device for its Function; Constructing a function linked list for each of the N functions supported by the PCIe device, wherein the function linked list is used to manage the interrupt vector actually used by the corresponding function; The free linked list and the N function linked lists are initialized, and the interrupt vectors are dynamically allocated according to the working status of the Function and the actual use needs of the interrupt resources.

2. A method for dynamically allocating global interrupt resources of a PCIe device according to claim 1, characterized in that: The dynamic allocation method further includes: During initialization, all global interrupt vectors reserved by the PCIe device for its Function are hung on the free linked list, and the N function linked lists are empty.

3. A method for dynamically allocating global interrupt resources of a PCIe device according to claim 2, characterized in that: The dynamic allocation method further includes: When a function in the PCIe device needs to use interrupt resources, the required interrupt vector is allocated to the function linked list corresponding to the function. At the same time, for each interrupt vector initialized by the function, an interrupt vector on the free linked list is taken down and hung on the function linked list corresponding to the function, and the information is filled into the corresponding field of the corresponding MSI-X Table Entry.

4. A method for dynamically allocating global interrupt resources of a PCIe device according to claim 1, characterized in that: The dynamic allocation method further includes: When a certain Function in the PCIe device needs to be migrated, all the existing interrupt vectors on the function linked list corresponding to the Function are removed and attached to the function linked list corresponding to the migration target.

5. A method for dynamically allocating global interrupt resources of a PCIe device according to claim 1, characterized in that: The dynamic allocation method further includes: When a certain Function in the PCIe device needs to increase interrupt resources, an interrupt vector is picked up from the free linked list and attached to the end of the function linked list corresponding to the Function.

6. A method for dynamically allocating global interrupt resources of a PCIe device according to claim 1, characterized in that: The dynamic allocation method further includes: When a certain Function in the PCIe device needs to be recycled, all the existing interrupt vectors on the function linked list corresponding to the Function are removed and hung at the end of the free linked list.

7. A method for dynamically allocating global interrupt resources of a PCIe device according to claim 1, characterized in that: The dynamic allocation method further includes: The Function in the non-working state in the PCIe device does not occupy interrupt resources.

8. A PCIe device global interrupt resource dynamic allocation system, characterized in that: The dynamic allocation system comprises: A linked list construction unit, wherein the linked list construction unit constructs a free linked list for the PCIe device, and the free linked list is used to manage the global interrupt vectors reserved by the PCIe device for its Function; the linked list construction unit also constructs a function linked list for each of the N Functions supported by the PCIe device, and the function linked list is used to manage the interrupt vectors actually used by the corresponding Function; A resource control unit, the resource control unit is used to initialize the free list and the N function lists and dynamically allocate interrupt vectors according to the working status of the Function and the actual use needs of the interrupt resources.

9. A PCIe device, characterized in that: The PCIe device uses the dynamic allocation method described in any one of claims 1 to 7 to dynamically allocate interrupt resources to the Function on it.

10. The interrupt request method of the PCIe device according to claim 9, characterized in that: The interrupt request method comprises: First, find the head of the function list corresponding to Function; After that, find the corresponding node of the function linked list according to the Function global interrupt vector number, and use the MSI-X Table ID marked by the node to address the required MSI-X Table; Then, get the Message Address information from the MSI-X Table; Finally, write the Message Data structure to the Message Address to interrupt the write packet.