Method for optimizing GPU operation efficiency based on processing capacity of PCI device of computer

Abstract

The invention belongs to the technical field of hardware resource management, and particularly relates to a method for optimizing GPU (Graphics Processing Unit) operation efficiency based on processing capacity of PCI (Peripheral Component Interconnect) equipment of a computer. Comprising a resource management module and a PCI channel virtualization module, the resource management module comprises a process isolation module, a queue optimization module, a resource allocation module, a dynamic adjustment module and a process recording module, and the PCI channel virtualization module comprises a TVIO interface module, a GPAD-HPAD conversion module, a tvio iommu, a tvio pci module and a hardware drive calling module. According to the method, the processing capacity of the PCI channel equipment is subjected to time dimension slicing and virtualization, so that the parallel processing capacity of equipment such as a GPU (Graphics Processing Unit) is improved, and the utilization rate of resources is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of hardware resource management, and in particular relates to a method for optimizing GPU computing efficiency based on computer PCI device processing capability. Background Art

[0002] Traditional PCI devices implement functional division by encapsulating hardware physical addresses and specific logic circuits through dedicated driver files. The driver files guide the operating system to complete data exchange, processing and distribution through a unified external interface. In a typical computer architecture, each PCI channel corresponds to a set of register addresses. When processing multiple task requests of PCI devices at the same time, it is done in a linear queue sequence. When processing tasks with high bandwidth requirements, multiple PCI channels will also process the same task in parallel to ensure speed. Due to the limitations of this physical structure, this queued processing flow causes a waste of efficiency from the dimension of computing power. When processing non-full-load tasks, the idle computing power is reduced; when processing overloaded tasks, no more computing power can be provided, resulting in a longer processing time and a waste of time. Summary of the invention

[0003] 1. Purpose of the invention In order to overcome the above shortcomings, the purpose of the present invention is to provide a method for optimizing GPU computing efficiency based on computer PCI device processing capabilities to solve the above technical problems.

[0004] (II) Technical solution To achieve the above objectives, the technical solutions provided by this application are as follows: A method for optimizing GPU computing efficiency based on computer PCI device processing capability, comprising a resource management module and a PCI channel virtualization module. The resource management module includes a process isolation module, a queue optimization module, a resource allocation module, a dynamic adjustment module and a process recording module. The process isolation module of the resource management module is connected to the queue optimization module, the queue optimization module is connected to the resource allocation module, and the resource allocation module is connected to the dynamic adjustment module. Through different functional modules of the resource management module, the process group is isolated, the process group priority and authority are set, and resources are allocated. The resource consumption status is monitored in the process and the parameters are dynamically adjusted to optimize the combination of processes in these time slices to the maximum extent, and the corresponding scheduling logic is used to fully utilize the device capacity resources. The PCI channel virtualization module virtualizes the process group that occupies the PCI channel resources in each slice, and virtualizes a fixed PCI channel on the physical structure into multiple PCI virtual channels to implement multi-threaded processing of single-threaded GPU tasks. The PCI channel virtualization module includes a TVIO interface module, a GPAD-HPAD conversion module, a tvio _iommu, a tvio _pci module and a hardware driver calling module, wherein the TVIO interface module is connected to the GPAD-HPAD conversion module, the GPAD-HPAD conversion module is connected to the tvio _iommu and tvio _pci modules, and the tvio _iommu and tvio_pci modules are connected to the hardware driver calling module.

[0005] Preferably, the user uses the resource management module to control the system resources in a more refined manner in units of control groups. The resource management module is a hierarchical process group and has a mechanism for managing processes by group. The resource management module organizes all processes in the system in a tree structure. Each tree contains all processes in the system. Each node is a process group, and each tree is associated with one or more subsystems. The role of the tree is to group processes, and the role of the subsystem is to operate on these groups and allocate resources to them. Resource allocation includes CPU time, system memory, network bandwidth, or a combination of these resources. A control group is a group of processes with unified restriction rules, and resource restrictions are also defined on the control group. If a process is added to a control group, it will automatically be affected by the restriction rules defined on the control group. The control groups under a subsystem can be nested to form a tree structure. The child node control group will inherit the resource restriction rules of the parent node control group. If the child node control group redefines the rules of the same resource as the parent node control group, the child node will overwrite the parent node. In the resource management module management state, the system can slice the processing power of the device. By default, the system will automatically create a hierarchy of slices, scopes, and application component units to provide a unified structure for the resource management module tree. Users can also create custom slices through system commands. Each resource management module is a slice, and each slice can have its own sub-slices. By default, all resources will be divided into three resource management modules: system, user, and peripherals.

[0006] Process group isolation: different process groups can use different namespaces, and all processes can be isolated into process groups of different slices. Different process groups have their own processes, networks, and file system mounting spaces, so there will be no resource competition or mutual interference between them, thus achieving the purpose of isolation. Resource allocation: You can set a resource usage limit for the process group, including CPU and memory resource usage. Once the CPU or memory used by the process group reaches the limit, a request overflow will be triggered if it is applied again. Queue optimization: manage process group queues, adjust the order of requests in the queue, merge certain process group queues, or allocate specific resource capabilities and time resources to a process group; Process record: Process record is mainly used to record and observe the time used by a process group as a basis for dynamic adjustment; Dynamic adjustment: Dynamically adjust the process group, and suspend and resume a process group; The above modules are composed of multiple process groups. The modules are interconnected. The process control is set in the module user interface according to user needs and executed by the control group.

[0007] Preferably, the PCI channel virtualization module includes: TVIO interface module: provides users with an interface to access the virtual interface. Through this interface, users can access the GPAD-HPAD conversion module, tvio _iommu, tvio _pci module and hardware driver call module; access different functional modules of the resource management module to set IO rules and call various capabilities of TVIO; GPAD-HPAD conversion module: converts the virtual address GPADn into the host physical address HPADn and provides a detailed corresponding chart; tvio_iommu and tvio _pci modules: The tvio_iommu module is a unified software and hardware encapsulation of the iommu layer by TVIO. It is mainly used to implement the Remapping function, that is, to achieve the ability to manage the IOMMU page table; The tvio_pci module is a unified software and hardware encapsulation of TVIO for PCI device drivers. It works with user-mode processes to complete direct access to devices, including PCI configuration space simulation, PCI Bar space redirection, and reverse remapping. Hardware driver calling module: including iommu driver module and pci universal driver module. iommu driver module is a driver that matches the hardware platform. pci universal driver module is in the virtualized state of PCI channel. The virtual IO interface uses tvio_pci package to call the pci universal driver on the host to realize device registration and deregistration.

[0008] Preferably, resource allocation specifically includes the following contents: First, determine the system CPU resource usage and monitor the system load status based on the process records.

[0009] CPU usage of a slice i i =SCU i +UCUi +MCU i Where: SCU i is the CPU occupancy rate of the system running in slice i (i is the number of slices i=1, 2, 3...); UCU i is the CPU occupancy rate of user operations in slice i; MCU i is the CPU usage of the virtual machine in slice i. (i is the number of slices i=1, 2, 3...) UCU i =∑u k CU i (k is the number of users k=1, 2, 3...) MCU i =∑m j CU i (j is the number of virtual machine devices j=1, 2, 3...).

[0010] When the virtual machine device only has GPU calls, j=1, MCU i =∑m j CU i .

[0011] The CPU usage of the entire system: CUW = ∑SCU i +UCU i +MCU i =∑CUw i (i=1, 2…m) Next, determine the virtualization capability of PCI.

[0012] Set the PCI virtual channel capacity coefficient Y to indicate the PCI load: Y=CUW+GU*GW+PI*PW; Among them, GU represents the utilization rate of the virtual channel calling the GPU capability, PI represents the PCI channel utilization rate, MW and PW represent the GPU capability and PCI load capacity of the system respectively; Resource management allocation includes the following rules and steps: (1) Determination of storage capacity threshold, assuming that the storage capacity threshold is L m , since the number of slices m of the system should be less than or equal to the number of process groups g, and g should be less than or equal to KaC, that is: m≤g≤L t≤KaC, where KaC is the CPU capability value, K is the coefficient, and the empirical value is between 15 and 20; a is the number of CPU cores in the system, and C is the CPU main frequency coefficient, and the empirical value is between 1 and 2. Otherwise, the system may overflow; based on the m≤g≤KaC rule, the initial value of the number of slices is generally smaller than the initial value of the process group, and a certain amount of redundancy is left, where: L m is the storage capacity threshold; (2) Determination of computing power threshold: Assume that the computing power threshold is Lc, and the CPU occupancy rate of the entire system is CUW≤L c ≤100%, otherwise, the system may overflow. Based on the CUW≤100% rule, for the g process groups in the m slices, the CPU resource occupancy is initially set to be allocated in proportion to the duration of the process group or evenly distributed among the m slices, where: L c is the computing capability threshold.

[0013] Preferably, the specific process of the queue optimization module of the resource management module is as follows: (1) Determine the priority. The system will sort the requests in the request queue according to their priority; Let F be the weight of the queue request: F = α*P + β×R + γ×T; Among them, P represents the priority of the request, R represents the resource requirement of the request, T represents the time limit of the request, and α, β and γ represent weight coefficients; The system determines their order in the queue based on the weight value of F; (2) Merge requests: Merge requests of the same type into i Merge into one request Q; Q = ∑q i (where i=1, 2, 3…) (3) Update request: Update the attributes and quantity of the merged request, update the resource requirements and priority of the request, and update the number of requests; Request queue priority sorting: And select the maximum value q m The resource requirements for the merge request q m =max(q i ) (where i=1, 2, 3…).

[0014] Preferably, the dynamic adjustment of resource allocation in the dynamic adjustment module comprises the following steps: (1) If the number of process groups of the current PCI virtual channel i is lower than the storage resource utilization threshold L allocated to it s, the redundant storage resources can be allocated to the virtual channels that need more resources, that is, slices, by reducing the storage weight value W of the slice i. i , to reduce its storage resource allocation; increase the storage weight value W of other slices to increase their resource allocation; dynamically increase the number of virtual channels in the system; the adjustment method is to increase the storage weight value W of the virtual channel i Adjust to W i ', then recalculate the storage weight values ​​of all virtual channels, and finally allocate resources according to the new weight values; W i 'Calculated as: W i =W i ×(1-r); where r is the scaling factor, r∈0~1, which is dynamically adjusted according to the current load of the system; (2) If the number of process groups of the current PCI virtual channel i exceeds its allocated storage resource utilization threshold L s , then it is necessary to increase the storage resources of virtual channel i. The allocation method is to increase the storage weight value W of the virtual channel i , to increase its storage resource allocation; at the same time, reduce the storage weight value W of other virtual channels to reduce their resource allocation; the adjustment method is to increase the storage weight value W of the virtual channel i Adjust to W i ', then recalculate the weight values ​​of all virtual channels, and finally allocate resources according to the new stored weight values; W i =W i ×(1+r); where r is the scaling factor, r∈0~1, and the value of r is dynamically selected according to the current load of the system; (3) If the CPU utilization of the current PCI virtual channel i is lower than the computing capacity threshold L c When the computing power weight value V of the virtual channel is i Adjust to V i ', then recalculate the computing power weight values ​​of all virtual channels, and finally allocate resources according to the new computing power weight values: V i =V i ×(1-δ); where δ is a scaling factor, δ∈0~1, and the value of δ is dynamically selected according to the current load of the system; (4) If the CPU demand of the current PCI virtual channel i exceeds the computing power threshold L c When the computing power weight value V of the virtual channel is i Adjust to V i', then recalculate the computing power weight values ​​of all virtual channels, and finally allocate resources according to the new computing power weight values: V i =V i ×(1+δ); where δ is a scaling factor, δ∈0~1, and the value of δ is dynamically selected according to the current load of the system.

[0015] Preferably, the PCI channel virtualization processing is performed based on the processing capability of the PCI channel in the slice, so that the PCI channel forms a multi-threaded processing virtualization capability. This multi-threaded calling capability actually virtualizes an actual physical GPU address into n virtual GPU addresses D n , and then use these virtual addresses GPAD n Convert to host physical address HPAD n , the system generates n host physical addresses, and then the n basic parallel-running logic programs provided by the tvio_iommu and tvio_pci modules, and the corresponding driver provided by the hardware driver call module, can realize the parallel call of these n virtual GPU computing power at the same time; The size of n determines the number of PCI virtual channel resource allocations, and also directly affects the optimization efficiency of the system's ability to call GPU computing power.

[0016] Generally, m≤n≤g≤L t ≤KaC Where: m is the number of device resource slices; g is the number of process groups; n is the virtual address GPAD n Convert to host physical address HPAD n The number of t is the threshold of the process group; KaCKaC is the CPU capability value, K is the coefficient, and the empirical value is between 15 and 20; a is the number of CPU cores in the system, and C is the CPU main frequency coefficient, and the empirical value is between 1 and 2.

[0017] Beneficial effects: 1. Improve resource utilization: By slicing and virtualizing the processing capabilities of PCI channel devices in the time dimension, the parallel processing capabilities of devices such as GPUs are improved, thereby improving resource utilization.

[0018] 2. Enhance system flexibility and scalability: The combined use of the resource management module and the PCI channel virtualization module allows the system to dynamically adjust resource allocation according to process requirements, enhancing the flexibility and scalability of the system.

[0019] 3. Optimize process management: Modules such as process group isolation, resource allocation, and queue optimization work together to achieve refined management of processes, optimize process execution efficiency, and the overall performance of the system.

[0020] 4. Realize multi-threaded processing: Through virtualization technology, single-threaded GPU calls are transformed into multi-threaded processing, which significantly improves the processing speed of computing tasks and system responsiveness.

[0021] 5. Dynamic resource adjustment: The dynamic adjustment module can adjust resource allocation in real time according to the system load to ensure that resources are fully utilized while avoiding resource waste. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a technical architecture diagram of the present invention; Figure 2 is a schematic diagram of resource management module slicing according to an embodiment of the present invention; Figure 3 is a diagram of a GPAD-HPAD address conversion process according to an embodiment of the present invention; Figure 4 is a technical flow chart of a PCI channel virtualization module according to an embodiment of the present invention; Figure 5 It is a technical flow chart of a resource management module according to an implementation mode of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following is a detailed description of the present invention in conjunction with the specific implementation methods and with reference to the attached drawings. Figure 1-5 , the present invention is further described in detail. It should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concept of the present invention.

[0024] The present invention provides a method for optimizing GPU computing efficiency based on the processing power of computer PCI devices, which is composed of two modules: a resource management module and a PCI channel virtualization module. The resource management module is responsible for slicing the processing power of the PCI channel device in the dimension of time. The function of the PCI channel virtualization module is to virtualize the PCI channel processing power in each slice to form the parallel processing power of the GPU, and then isolate the process group, set the process group priority, authority, and resource allocation through different functional modules of the resource management module. Monitor the resource consumption status in the process and adjust the parameters dynamically based on this, so as to optimize the combination of processes in these slices to the maximum extent, and at the same time, use the corresponding scheduling logic to fully utilize the device capacity resources.

[0025] See attached Figure 1 ,The technical architecture consists of two modules: the resource management module and the PCI channel virtualization module.

[0026] The resource management module includes a process isolation module, a queue optimization module, a resource allocation module, a dynamic adjustment module and a process recording module. The process isolation module of the resource management module is connected to the queue optimization module, the queue optimization module is connected to the resource allocation module, and the resource allocation module is connected to the dynamic adjustment module. Through the different functional modules of the resource management module, the process group is isolated, the process group priority and authority are set, and resources are allocated. The resource consumption status is monitored in the process and the parameters are dynamically adjusted to maximize the optimization and combination of processes in these time slices. At the same time, the corresponding scheduling logic is used to fully utilize the device capacity resources.

[0027] Process group isolation: different process groups can use different namespaces, and all processes can be isolated into process groups of different slices. Different process groups have their own processes, networks, and file system mounting spaces, so there will be no resource competition or mutual interference between them, thus achieving the purpose of isolation. Resource allocation: You can set a resource usage limit for the process group, including CPU and memory resource usage. Once the CPU or memory used by the process group reaches the limit, a request overflow will be triggered if it is applied again. Queue optimization: manage process group queues, adjust the order of requests in the queue, merge certain process group queues, or allocate specific resource capabilities and time resources to a process group; Process record: Process record is mainly used to record and observe the time used by a process group as a basis for dynamic adjustment; Dynamic adjustment: Dynamically adjust the process group, and suspend and resume a process group; The above modules are composed of multiple process groups. The modules are interconnected. The process control is set in the module user interface according to user needs and executed by the control group.

[0028] The PCI channel virtualization module virtualizes the process group that occupies the PCI channel resources in each slice, and virtualizes a fixed PCI channel on the physical structure into multiple PCI virtual channels to implement multi-threaded processing of single-threaded GPU tasks. See attached Figure 4The PCI channel virtualization module includes a TVIO interface module, a GPAD-HPAD conversion module, a tvio _iommu, a tvio _pci module and a hardware driver calling module, wherein the TVIO interface module is connected to the GPAD-HPAD conversion module, the GPAD-HPAD conversion module is connected to the tvio _iommu and tvio _pci modules, and the tvio _iommu and tvio _pci modules are connected to the hardware driver calling module.

[0029] The overall technical process of the present invention is that, based on the management mode of the resource management module, the system first slices the processing capacity (resources) of the device. Then, the PCI channel in each slice calls the device's capacity for address virtualization, and converts the virtual address GPADn into the host physical address HPADn, so that the PCI channel forms a virtualization capability for multi-threaded processing. At the same time, an interface for accessing the virtual interface (TVIO) is provided to the user. Through the different functional module interfaces of the resource management module built into the TVIO interface module of the PCI channel virtualization module, the user isolates the process group, sets the process group priority, authority, and resource allocation, monitors the resource consumption status in the process, and dynamically adjusts the parameters based on this, so as to optimize the combination of the processes in these slices to the maximum extent, and at the same time, the corresponding scheduling logic is used to fully utilize the device capacity resources.

[0030] Through the resource management module, users can use control groups as units to make more precise control over the system resources they use. According to the user's needs, the system first slices the PCI channel and related device capability resources, and then sets some restriction rules for certain resources used by these slices, such as setting the proportion of CPU resources used. Then, some processes with similar needs are added to the slices or process groups bound by this rule, and the corresponding restrictions will take effect on these process groups.

[0031] The specific process of virtualizing the processing power of PCI devices is to first use the user interface of the resource management module built into the TVIO interface. Users can set and call the function modules of the resource management module, set the IO rules and call various capabilities of TVIO. According to the user's setting requirements and based on the management mode of the resource management module, the system first slices the processing power of the device, that is, the resources. Then, the PCI channel in each slice virtualizes the address of the device calling capability. Through the remapping technology function provided in the GPAD-HPAD conversion module, the system first virtualizes the actual physical GPU address of the device to be called by the user into one or more virtual GPU addresses Dn according to the capability, and then converts these virtual addresses GPADn into host physical addresses HPADn, and provides detailed corresponding charts. After obtaining these host physical addresses, the basic parallel running logic programs provided by the tvio_iommu and tvio_pci modules, and the corresponding drivers provided by the hardware driver call module, can realize the parallel call of the GPU computing power under these virtual addresses at the same time. In this way, the single-threaded GPU call under physical conditions can realize the calling capability of multi-threaded GPU computing power.

[0032] Reference Figure 5 ,The technical process of resource management module is: the resource management module includes process group isolation, resource allocation, queue optimization, process recording, and dynamic adjustment modules. The resource management module is responsible for allocating, scheduling, monitoring, and dynamically adjusting hardware resources between virtual channels. Among them, the process group isolation module is connected to the resource allocation module, the resource allocation module is connected to the queue optimization module, the queue optimization module is connected to the process recording module, and the process recording module is connected to the dynamic adjustment module; The resource management module is a hierarchical process group with a mechanism to manage processes by group. The resource management module organizes all processes in the system in a tree structure. Each tree contains all processes in the system. Each node is a process group, and each tree is associated with one or more subsystems. The role of the tree is to group processes, and the role of the subsystem is to operate on these process groups and allocate resources to them, such as CPU time, system memory, network bandwidth, or a combination of these resources.

[0033] A control group is a group of processes with uniform restriction rules. Resource restrictions are also defined on the control group. If a process is added to a control group, it will automatically be affected by the restriction rules defined on the control group.

[0034] Functions of the resource management module: When the resource management module is managed, the system can slice the processing power of the device, that is, the resources. By default, the system will automatically create a hierarchy of slices, scopes, and application component units to provide a unified structure for the resource management module tree. Users can also create custom slices through system commands. Each resource management module is a slice, and each slice can have its own sub-slices. By default, all resources will be divided into three resource management modules: system, user, and peripherals.

[0035] Through the process group isolation, resource allocation, queue optimization, process recording, and setting management control group of the resource management module, the processes contained in these slices are called, set, and managed, and fine-grained control is exercised in the allocation, sorting, rejection, management, and monitoring of system resources, so that device resources can be intelligently allocated between applications and users, thereby increasing overall efficiency.

[0036] Therefore, first determine the system CPU resource usage. Monitor the system load status based on process records: CPU usage of a slice i i =SCU i +UCU i +MCU i Where: SCU i is the CPU occupancy rate of the system running in slice i (i is the number of slices i=1, 2, 3...); UCU i is the CPU occupancy rate of user operations in slice i; MCU i is the CPU usage of the virtual machine in slice i. (i is the number of slices i=1, 2, 3...) UCU i =∑u k CU i (k is the number of users k=1, 2, 3...) MCU i =∑m j CU i (j is the number of virtual machine devices j=1, 2, 3...).

[0037] When the virtual machine device only has GPU calls, j=1, MCU i =∑m j CU i .

[0038] The CPU usage of the entire system: CUW = ∑SCU i +UCU i +MCU i =∑CUw i(i=1, 2…m) Next, determine the virtualization capability of PCI.

[0039] Set the PCI virtual channel capacity coefficient Y to indicate the PCI load: Y=CUW+GU*GW+PI*PW; Among them, GU represents the utilization rate of the virtual channel calling the GPU capability, PI represents the PCI channel utilization rate, MW and PW represent the GPU capability and PCI load capacity of the system respectively; Resource management allocation includes the following rules and steps: (1) Determination of storage capacity threshold, assuming that the storage capacity threshold is L m , since the number of slices m of the system should be less than or equal to the number of process groups g, and g should be less than or equal to KaC, that is: m≤g≤L t ≤KaC, where KaC is the CPU capability value, K is the coefficient, and the empirical value is between 15 and 20; a is the number of CPU cores in the system, and C is the CPU main frequency coefficient, and the empirical value is between 1 and 2. Otherwise, the system may overflow; based on the m≤g≤KaC rule, the initial value of the number of slices is generally smaller than the initial value of the process group, and a certain amount of redundancy is left, where: L m is the storage capacity threshold; (2) Determination of computing power threshold: Assume that the computing power threshold is Lc, and the CPU occupancy rate of the entire system is CUW≤L c ≤100%, otherwise, the system may overflow. Based on the CUW≤100% rule, for the g process groups in the m slices, the CPU resource occupancy is initially set to be allocated in proportion to the duration of the process group or evenly distributed among the m slices, where: L c is the computing capability threshold.

[0040] The queue optimization module in the resource management module is responsible for managing the process group queues. It can adjust the order of requests in the queue, merge certain process group queues, or allocate specific resource capabilities and time resources to a process group.

[0041] The specific process is as follows: (1) Determine the priority. The system will sort the requests in the request queue according to their priority; Let F be the weight of the queue request: F = α*P + β×R + γ×T; Among them, P represents the priority of the request, R represents the resource requirement of the request, T represents the time limit of the request, and α, β and γ represent weight coefficients; The system determines their order in the queue based on the weight value of F; (2) Merge requests: Merge requests of the same type intoi Merge into one request Q; Q = ∑q i (where i=1, 2, 3…) (3) Update request: Update the attributes and quantity of the merged request, update the resource requirements and priority of the request, and update the number of requests; Request queue priority sorting: And select the maximum value q m The resource requirements for the merge request q m =max(q i ) (where i=1, 2, 3…).

[0042] Furthermore, when the system process record monitoring detects that the PCI virtual channel is overloaded or redundant, the dynamic adjustment module will dynamically adjust the resource allocation between the PCI virtual channels, ie, slices i.

[0043] Dynamically adjusting resource allocation includes the following steps: (1) If the number of process groups of the current PCI virtual channel i is lower than the storage resource utilization threshold L allocated to it s , the redundant storage resources can be allocated to the virtual channels that need more resources, that is, slices, by reducing the storage weight value W of the slice i. i , to reduce its storage resource allocation; increase the storage weight value W of other slices to increase their resource allocation; dynamically increase the number of virtual channels in the system; the adjustment method is to increase the storage weight value W of the virtual channel i Adjust to W i ', then recalculate the storage weight values ​​of all virtual channels, and finally allocate resources according to the new weight values; W i 'Calculated as: W i =W i ×(1-r); where r is the scaling factor, r∈0~1, which is dynamically adjusted according to the current load of the system; (2) If the number of process groups of the current PCI virtual channel i exceeds its allocated storage resource utilization threshold L s , then it is necessary to increase the storage resources of virtual channel i. The allocation method is to increase the storage weight value W of the virtual channel i , to increase its storage resource allocation; at the same time, reduce the storage weight value W of other virtual channels to reduce their resource allocation; the adjustment method is to increase the storage weight value W of the virtual channel i Adjust to W i', then recalculate the weight values ​​of all virtual channels, and finally allocate resources according to the new stored weight values; W i =W i ×(1+r); where r is the scaling factor, r∈0~1, and the value of r is dynamically selected according to the current load of the system; (3) If the CPU utilization of the current PCI virtual channel i is lower than the computing capacity threshold L c When the computing power weight value V of the virtual channel is i Adjust to V i ', then recalculate the computing power weight values ​​of all virtual channels, and finally allocate resources according to the new computing power weight values: V i =V i ×(1-δ); where δ is a scaling factor, δ∈0~1, and the value of δ is dynamically selected according to the current load of the system; (4) If the CPU demand of the current PCI virtual channel i exceeds the computing power threshold L c When the computing power weight value V of the virtual channel is i Adjust to V i ', then recalculate the computing power weight values ​​of all virtual channels, and finally allocate resources according to the new computing power weight values: V i =V i ×(1+δ); where δ is a scaling factor, δ∈0~1, and the value of δ is dynamically selected according to the current load of the system.

[0044] Virtualization of PCI device processing capabilities is the basis for implementing GPU multi-threaded calls and improving computing efficiency. PCI channel virtualization processing is performed based on the processing capabilities of the PCI channel in the slice. The system virtualizes the address through the device capabilities called by the PCI channel in each slice, and then converts the virtual address GPAD into the host physical address HPAD, and provides a detailed corresponding chart. This enables the PCI channel to form a multi-threaded processing virtualization capability. This multi-threaded calling capability actually virtualizes an actual physical GPU address into n virtual GPU addresses D n , and then use these virtual addresses GPAD n Convert to host physical address HPAD n , the system generates n host physical addresses, and then the n basic parallel-running logic programs provided by the tvio_iommu and tvio_pci modules, and the corresponding driver provided by the hardware driver call module, can realize the parallel call of these n virtual GPU computing power at the same time; The size of n determines the number of PCI virtual channel resource allocations, and also directly affects the optimization efficiency of the system's ability to call GPU computing power.

[0045] Generally, m≤n≤g≤L t ≤KaC Where: m is the number of device resource slices; g is the number of process groups; n is the virtual address GPAD n Convert to host physical address HPAD n The number of t is the threshold of the process group; KaC is the CPU capability value, K is the coefficient, and the empirical value is between 15 and 20; a is the number of CPU cores in the system, and C is the CPU main frequency coefficient, and the empirical value is between 1 and 2.

[0046] The improved effects of the present invention are as follows: 1. Improve resource utilization: By slicing and virtualizing the processing capabilities of PCI channel devices in the time dimension, the parallel processing capabilities of devices such as GPUs are improved, thereby improving resource utilization.

[0047] 2. Enhance system flexibility and scalability: The combined use of the resource management module and the PCI channel virtualization module allows the system to dynamically adjust resource allocation according to process requirements, enhancing the flexibility and scalability of the system.

[0048] 3. Optimize process management: Modules such as process group isolation, resource allocation, and queue optimization work together to achieve refined management of processes, optimize process execution efficiency, and the overall performance of the system.

[0049] 4. Realize multi-threaded processing: Through virtualization technology, single-threaded GPU calls are transformed into multi-threaded processing, which significantly improves the processing speed of computing tasks and system responsiveness.

[0050] 5. Dynamic resource adjustment: The dynamic adjustment module can adjust resource allocation in real time according to the system load to ensure that resources are fully utilized while avoiding resource waste.

[0051] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0052] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing GPU computing efficiency based on computer PCI device processing capability, characterized in that: Including resource management module and PCI channel virtualization module, The resource management module includes a process isolation module, a queue optimization module, a resource allocation module, a dynamic adjustment module and a process recording module. The process isolation module of the resource management module is connected to the queue optimization module, the queue optimization module is connected to the resource allocation module, and the resource allocation module is connected to the dynamic adjustment module. Through different functional modules of the resource management module, the process group is isolated, the process group priority and authority are set, and resources are allocated. The resource consumption status is monitored in the process and the parameters are dynamically adjusted to optimize the combination of processes in these time slices to the maximum extent, and the corresponding scheduling logic is used to fully utilize the device capacity resources. The PCI channel virtualization module virtualizes the process group that occupies the PCI channel resources in each slice, and virtualizes a fixed PCI channel on the physical structure into multiple PCI virtual channels to implement multi-threaded processing of single-threaded GPU tasks. The PCI channel virtualization module includes a TVIO interface module, a GPAD-HPAD conversion module, a tvio _iommu, a tvio _pci module and a hardware driver calling module, wherein the TVIO interface module is connected to the GPAD-HPAD conversion module, the GPAD-HPAD conversion module is connected to the tvio _iommu and tvio _pci modules, and the tvio _iommu and tvio _pci modules are connected to the hardware driver calling module.

2. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 1, characterized in that: The user can use the resource management module to control the system resources in a more detailed manner, with the control group as the unit. The resource management module is a hierarchical process group, and has a mechanism to manage processes by group. The resource management module organizes all processes in the system in a tree structure. Each tree contains all processes in the system. Each node is a process group, and each tree is associated with one or more subsystems. The role of the tree is to group processes, and the role of the subsystem is to operate on these groups and allocate resources to them. Resource allocation includes CPU time, system memory, network bandwidth, or a combination of these resources. A control group is a group of processes with unified restriction rules, and resource restrictions are also defined on the control group. If a process is added to a control group, it will automatically be affected by the restriction rules defined on the control group. The control groups under a subsystem can be nested to form a tree structure. The child node control group will inherit the resource restriction rules of the parent node control group. If the child node control group redefines the rules of the same resource as the parent node control group, the child node will overwrite the parent node. In the resource management module management state, the system can slice the processing power of the device. By default, the system will automatically create a hierarchy of slices, scopes, and application component units to provide a unified structure for the resource management module tree. Users can also create custom slices through system commands. Each resource management module is a slice, and each slice can have its own sub-slices. By default, all resources will be divided into three resource management modules: system, user, and peripherals.

3. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 1, characterized in that: Process group isolation: different process groups can use different namespaces, and all processes can be isolated into process groups of different slices. Different process groups have their own processes, networks, and file system mounting spaces, so there will be no resource competition or mutual interference between them, thus achieving the purpose of isolation. Resource allocation: You can set a resource usage limit for the process group, including CPU and memory resource usage. Once the CPU or memory used by the process group reaches the limit, a request overflow will be triggered if it is applied again. Queue optimization: manage process group queues, adjust the order of requests in the queue, merge certain process group queues, or allocate specific resource capabilities and time resources to a process group; Process record: Process record is mainly used to record and observe the time used by a process group as a basis for dynamic adjustment; Dynamic adjustment: Dynamically adjust the process group, and suspend and resume a process group; The above modules are composed of multiple process groups. The modules are interconnected. The process control is set in the module user interface according to user needs and executed by the control group.

4. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 1, characterized in that: PCI channel virtualization modules include: TVIO interface module: provides users with an interface to access the virtual interface. Through this interface, users can access the GPAD-HPAD conversion module, tvio _iommu, tvio _pci module and hardware driver call module; access different functional modules of the resource management module to set IO rules and call various capabilities of TVIO; GPAD-HPAD conversion module: converts the virtual address GPADn into the host physical address HPADn and provides a detailed corresponding chart; tvio_iommu and tvio _pci modules: The tvio_iommu module is a unified software and hardware encapsulation of the iommu layer by TVIO. It is mainly used to implement the Remapping function, that is, to achieve the ability to manage the IOMMU page table; The tvio_pci module is a unified software and hardware encapsulation of TVIO for PCI device drivers. It works with user-mode processes to complete direct access to devices, including PCI configuration space simulation, PCI Bar space redirection, and reverse remapping. Hardware driver calling module: including iommu driver module and pci universal driver module. iommu driver module is a driver that matches the hardware platform. pci universal driver module is in the virtualized state of PCI channel. The virtual IO interface uses tvio_pci package to call the pci universal driver on the host to realize device registration and deregistration.

5. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 3, characterized in that: Resource allocation specifically includes the following: First, determine the system CPU resource usage and monitor the system load status based on the process records: CPU usage of a slice i i =SCU i +UCU i +MCU i Where: SCU i is the CPU occupancy rate of the system running in slice i (i is the number of slices i=1, 2, 3...); UCU i is the CPU occupancy rate of user operations in slice i; MCU i is the CPU usage of the virtual machine in slice i. (i is the number of slices i=1, 2, 3...) UCU i =∑u k CU i (k is the number of users k=1, 2, 3...) MCU i =∑m j CU i (j is the number of virtual machine devices j=1, 2, 3...). When the virtual machine device only has GPU calls, j=1, MCU i =∑m j CU i . The CPU usage of the entire system: CUW = ∑SCU i +UCU i +MCU i =∑CUw i (i=1, 2…m) Next, determine the virtualization capability of PCI. Set the PCI virtual channel capacity coefficient Y to indicate the PCI load: Y=CUW+GU*GW+PI*PW; Among them, GU represents the utilization rate of the virtual channel calling the GPU capability, PI represents the PCI channel utilization rate, MW and PW represent the GPU capability and PCI load capacity of the system respectively; Resource management allocation includes the following rules and steps: (1) Determination of storage capacity threshold, assuming that the storage capacity threshold is L m , since the number of slices m of the system should be less than or equal to the number of process groups g, and g should be less than or equal to KaC, that is: m≤g≤L t ≤KaC, where KaC is the CPU capability value, K is the coefficient, and the empirical value is between 15 and 20; a is the number of CPU cores in the system, and C is the CPU main frequency coefficient, and the empirical value is between 1 and 2. Otherwise, the system may overflow; based on the m≤g≤KaC rule, the initial value of the number of slices is generally smaller than the initial value of the process group, and a certain amount of redundancy is left, where: L m is the storage capacity threshold; (2) Determination of computing power threshold: Assume that the computing power threshold is Lc, and the CPU occupancy rate of the entire system is CUW≤L c ≤100%, otherwise, the system may overflow. Based on the CUW≤100% rule, for the g process groups in the m slices, the CPU resource occupancy is initially set to be allocated in proportion to the duration of the process group or evenly distributed among the m slices, where: L c is the computing capability threshold.

6. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 3, characterized in that: The specific process of the queue optimization module of the resource management module is as follows: (1) Determine the priority. The system will sort the requests in the request queue according to their priority; Let F be the weight of the queue request: F = α*P + β×R + γ×T; Among them, P represents the priority of the request, R represents the resource requirement of the request, T represents the time limit of the request, and α, β and γ represent weight coefficients; The system determines their order in the queue based on the weight value of F; (2) Merge requests: Merge requests of the same type into i Merge into one request Q; Q = ∑q i (where i=1, 2, 3…) (3) Update request: Update the attributes and quantity of the merged request, update the resource requirements and priority of the request, and update the number of requests; Request queue priority sorting: And select the maximum value q m The resource requirements for the merge request q m =max(q i ) (where i=1, 2, 3…).

7. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 3, characterized in that: The dynamic adjustment of resource allocation in the dynamic adjustment module comprises the following steps: (1) If the number of process groups of the current PCI virtual channel i is lower than the storage resource utilization threshold L allocated to it s , the redundant storage resources can be allocated to the virtual channels that need more resources, that is, slices, by reducing the storage weight value W of the slice i. i , to reduce its storage resource allocation; increase the storage weight value W of other slices to increase their resource allocation; dynamically increase the number of virtual channels in the system; the adjustment method is to increase the storage weight value W of the virtual channel i Adjust to W i ', then recalculate the storage weight values ​​of all virtual channels, and finally allocate resources according to the new weight values; W i 'Calculated as: W i =W i ×(1-r); where r is the scaling factor, r∈0~1, which is dynamically adjusted according to the current load of the system; (2) If the number of process groups of the current PCI virtual channel i exceeds its allocated storage resource utilization threshold L s , then it is necessary to increase the storage resources of virtual channel i. The allocation method is to increase the storage weight value W of the virtual channel i , to increase its storage resource allocation; at the same time, reduce the storage weight value W of other virtual channels to reduce their resource allocation; the adjustment method is to increase the storage weight value W of the virtual channel i Adjust to W i ', then recalculate the weight values ​​of all virtual channels, and finally allocate resources according to the new stored weight values; W i =W i ×(1+r); where r is the scaling factor, r∈0~1, and the value of r is dynamically selected according to the current load of the system; (3) If the CPU utilization of the current PCI virtual channel i is lower than the computing capacity threshold L c When the computing power weight value V of the virtual channel is i Adjust to V i ', then recalculate the computing power weight values ​​of all virtual channels, and finally allocate resources according to the new computing power weight values: V i =V i ×(1-δ); where δ is a scaling factor, δ∈0~1, and the value of δ is dynamically selected according to the current load of the system; (4) If the CPU demand of the current PCI virtual channel i exceeds the computing power threshold L c When the computing power weight value V of the virtual channel is i Adjust to V i ', then recalculate the computing power weight values ​​of all virtual channels, and finally allocate resources according to the new computing power weight values: V i =V i ×(1+δ); where δ is a scaling factor, δ∈0~1, and the value of δ is dynamically selected according to the current load of the system.

8. The method for optimizing GPU computing efficiency based on computer PCI device processing capability according to claim 3, characterized in that: PCI channel virtualization processing is performed based on the processing capacity of the PCI channel in the slice, so that the PCI channel can form a multi-threaded processing virtualization capability. This multi-threaded calling capability actually virtualizes an actual physical GPU address into n virtual GPU addresses D n , and then use these virtual addresses GPAD n Convert to host physical address HPAD n , the system generates n host physical addresses, and then the n basic parallel-running logic programs provided by the tvio_iommu and tvio_pci modules, and the corresponding driver provided by the hardware driver call module, can realize the parallel call of these n virtual GPU computing power at the same time; The size of n determines the number of PCI virtual channel resource allocations, and also directly affects the optimization efficiency of the system's ability to call GPU computing power. Generally, m≤n≤g≤L t ≤KaC Where: m is the number of device resource slices; g is the number of process groups; n is the virtual address GPAD n Convert to host physical address HPAD n The number of t is the threshold of the process group; KaC is the CPU capability value, K is the coefficient, and the empirical value is between 15 and 20; a is the number of CPU cores in the system, and C is the CPU main frequency coefficient, and the empirical value is between 1 and 2.

Images