GPU segmentation configuration method and server

Abstract

The invention relates to a GPU segmentation configuration method and a server. The method comprises the following steps: generating a GPU configuration interface; wherein the GPU configuration interface at least comprises a GPU segmentation strategy configuration area, the GPU segmentation strategy configuration area comprises a GPU selection control and at least one configuration group, and the configuration group comprises an instance number control and a resource piece number configuration control; receiving a target operation of a user in the GPU segmentation strategy configuration area, wherein the target operation comprises the steps of determining a target GPU needing to be segmented through a GPU selection control, determining the number of segmented instances through an instance number control, and determining the number of resource pieces occupied by each instance through a resource piece number configuration control; and in response to the target operation, executing the segmentation strategy on the target GPU. In this way, a user can carry out GPU MIG configuration through the GPU configuration interface, the usability of configuration can be improved, manual operation errors easily generated when the user carries out GPU MIG configuration through a command line are avoided, the segmentation configuration efficiency can be improved to a great extent, and user operation is facilitated.

Description

Technical Field

[0001] This application relates to the field of GPU (graphics processing unit) technology, and in particular to a GPU partitioning and configuration method and server. Background Technology

[0002] With the continuous development of AI (artificial intelligence) large-scale model technology, many AI companies require a large number of servers for AI large-scale model computing. Since GPUs in servers are relatively expensive, how to improve GPU resource utilization has become a real concern for users.

[0003] The GPU MIG (multi-instance GPU) technology provided in related technologies can improve GPU resource utilization. However, due to the high operational threshold of GPU MIG technology, which requires professional personnel to enter command lines for GPU MIG configuration, configuration failures often occur due to operator errors. Therefore, this technology suffers from low efficiency in GPU MIG partitioning and configuration. Summary of the Invention

[0004] This application provides a GPU partitioning configuration method and server, which can improve the partitioning configuration efficiency of GPU MIG and facilitate user operation.

[0005] According to a first aspect of the embodiments of this application, a GPU partitioning configuration method is provided, the method comprising:

[0006] Generate a GPU configuration interface; wherein the GPU configuration interface includes at least one GPU partitioning strategy configuration area, the GPU partitioning strategy configuration area includes GPU selection controls, and at least one configuration group, the configuration group includes: instance number controls and resource slice number configuration controls;

[0007] Receive the user's target operation in the GPU partitioning strategy configuration area. The target operation includes determining the target GPU to be partitioned through the GPU selection control, determining the number of instances to be partitioned through the instance number control, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control.

[0008] In response to the target operation, a partitioning strategy is executed on the target GPU; wherein the partitioning strategy includes obtaining the number of instances to partition the target GPU and the number of resource slices occupied by each instance.

[0009] In this solution, users can perform split configuration through the GPU configuration interface, which improves the ease of configuration and avoids human error such as spelling mistakes that are easy to occur when configuring GPU MIG through the command line. This greatly improves the efficiency of GPU MIG split configuration and makes it easier for users to operate.

[0010] In one possible implementation provided in this application, the GPU configuration interface further includes a configuration add control and a configuration remove control. The configuration add control is used to add at least one configuration group, and the configuration remove control is used to remove at least one configuration group.

[0011] In this solution, users can add one or more configuration groups through the configuration add control, and users can remove one or more existing configuration groups through the configuration remove control, which enables flexible configuration.

[0012] In one possible implementation provided in this application, the GPU partitioning strategy configuration area includes a GPU selection control, a first configuration group, and a second configuration group; the target operation further includes determining the target GPU to be partitioned through the GPU selection control, determining the number of first instances to be partitioned through the instance number control in the first configuration group, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control in the first configuration group.

[0013] Additionally, the number of second instances to be split is determined by the instance quantity control in the second configuration group, and the number of resource shards occupied by each instance is determined by the resource shard number configuration control in the second configuration group.

[0014] This solution allows users to determine the number of instances to be split and the number of resource slices required for each configuration group, as needed.

[0015] In one possible implementation provided in this application, the GPU configuration interface further includes a partitioning strategy addition control and a partitioning strategy removal control; the partitioning strategy addition control is used to add at least one set of partitioning strategies, and the partitioning strategy removal control is used to remove at least one set of partitioning strategies.

[0016] In this solution, users can remove one or more existing segmentation strategies using the segmentation strategy removal control. After removal, the segmentation strategy numbers are automatically adjusted to reflect the order of the segmentation strategies on the interface. This allows users to flexibly configure segmentation strategies by adding or removing controls based on them.

[0017] In one possible implementation provided in this application, the GPU configuration interface includes a first GPU partitioning strategy configuration area and a second GPU partitioning strategy configuration area. The first and second GPU partitioning strategy configuration areas each include a GPU selection control and at least one configuration group. The first GPU partitioning strategy configuration area is used to generate a first partitioning strategy, and the second GPU partitioning strategy configuration area is used to generate a second partitioning strategy. The target operation further includes determining the first GPU to be partitioned, the number of instances, and the number of resource fragments occupied by each instance through the GPU selection control, instance quantity control, and resource fragment number configuration control in the first GPU partitioning strategy configuration area, and determining the second GPU to be partitioned, the number of instances, and the number of resource fragments occupied by each instance through the GPU selection control, instance quantity control, and resource fragment number configuration control in the second GPU partitioning strategy configuration area, respectively. The target GPU includes both the first and second GPUs.

[0018] In this solution, users can add at least one set of partitioning strategies based on the partitioning strategy. Each set of partitioning strategies includes a GPU partitioning strategy configuration area, which includes the corresponding GPU selection control, instance quantity control, and resource fragment quantity configuration control.

[0019] In one possible implementation provided in this application, the method may further include:

[0020] In response to the target operation, verify whether the segmentation strategy meets the verification conditions;

[0021] When the partitioning strategy meets the verification conditions, the target GPU is partitioned into multiple instances based on the partitioning strategy.

[0022] This solution verifies the segmentation strategy and executes it only when the verification is successful, thus ensuring successful GPU MIG segmentation. This avoids the situation where GPU MIG configuration strategy execution fails due to human error when related technologies directly execute the GPU MIG configuration strategy sent by the user, thereby greatly improving the efficiency of GPU MIG segmentation and configuration.

[0023] In one possible implementation provided in this application, the method further includes:

[0024] Verify whether the partitioning strategy meets the partitioning specifications of the target GPU;

[0025] When the partitioning strategy meets the partitioning specifications of the target GPU, it is determined that the partitioning strategy meets the verification conditions.

[0026] In this scheme, since the target GPU has a predefined corresponding partitioning specification, by matching the number of instances to be partitioned and the number of resource fragments occupied by each instance with the partitioning template, it can be determined in time whether the partitioning strategy meets the partitioning specification of the target GPU, thus avoiding GPU MIG partitioning failure due to non-compliance with the partitioning specification.

[0027] In one possible implementation provided in this application, verifying whether the partitioning strategy meets the partitioning specifications of the target GPU includes:

[0028] The number of instances and resource fragments are matched with the partitioning template of the target GPU; the partitioning template includes a variety of predefined partitioning specifications;

[0029] When a partitioning specification exists in the partitioning template that matches the number of instances and resource fragments, the partitioning strategy is determined to satisfy the partitioning specification of the target GPU.

[0030] In this scheme, since the target GPU has a predefined corresponding partitioning specification, by matching the number of instances to be partitioned and the number of resource fragments occupied by each instance with the partitioning template, it can be determined in time whether the partitioning strategy meets the partitioning specification of the target GPU, thus avoiding GPU MIG partitioning failure due to non-compliance with the partitioning specification.

[0031] In one possible implementation provided in this application, the method further includes:

[0032] When the partitioning strategy meets the partitioning specifications of the target GPU, obtain the remaining resources of the target GPU and the first resource occupied when partitioning the target GPU based on the partitioning strategy;

[0033] Determine if the remaining resources are less than the first resource;

[0034] If the remaining resources are not less than the first resource, the partitioning strategy is determined to meet the verification conditions.

[0035] In this solution, whether the target GPU has a partitioned instance or not, the remaining resources are determined based on the number of resource fragments and the amount of video memory used, respectively. This can effectively determine whether the remaining resources of the target GPU can meet the requirements of the instance to be partitioned, and avoid the failure of GPU MIG partitioning configuration due to insufficient remaining resources of the target GPU.

[0036] According to a second aspect of the embodiments of this application, a GPU partitioning configuration apparatus is provided, the apparatus comprising:

[0037] The interface generation module is used to generate a GPU configuration interface; wherein the GPU configuration interface includes at least one GPU partitioning strategy configuration area, the GPU partitioning strategy configuration area includes a GPU selection control, and at least one configuration group, the configuration group includes: instance number control and resource slice number configuration control;

[0038] The operation receiving module is used to receive the user's target operation in the GPU partitioning strategy configuration area. The target operation includes determining the target GPU to be partitioned through the GPU selection control, determining the number of instances to be partitioned through the instance number control, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control.

[0039] The strategy execution module is used to execute a partitioning strategy on the target GPU in response to the target operation; wherein, the partitioning strategy includes obtaining the number of instances to partition the target GPU and the number of resource slices occupied by each instance.

[0040] According to a third aspect of the embodiments of this application, a server is provided. The server includes: at least one GPU, a memory, and a processor. The memory stores a computer program, and the at least one processor is configured to execute instructions to implement the GPU partitioning configuration method described above, and to perform multi-instance partitioning on the at least one GPU.

[0041] According to a fourth aspect of the embodiments of this application, a GPU partitioning configuration system is provided, the system including a terminal for displaying a GPU configuration interface and a server for the third aspect.

[0042] According to a fifth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described above in this application.

[0043] According to a sixth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above in this application. Attached Figure Description

[0044] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0045] Figure 1 A schematic diagram of the system architecture provided for an exemplary embodiment of this application;

[0046] Figure 2 A schematic diagram of a GPU configuration interface provided for an exemplary embodiment of this application;

[0047] Figure 3 A schematic diagram of a GPU configuration interface provided for another exemplary embodiment of this application;

[0048] Figure 4 A schematic diagram of a GPU configuration interface provided as yet another exemplary embodiment of this application;

[0049] Figure 5 A flowchart illustrating a segmentation strategy verification method provided for an exemplary embodiment of this application;

[0050] Figure 6 This is an intentional representation of a combination of split configurations supported by a certain type of GPU, provided as an exemplary embodiment of this application.

[0051] Figure 7 A flowchart illustrating a GPU partitioning configuration method provided in an exemplary embodiment of this application;

[0052] Figure 8 A schematic block diagram of the functional modules of a GPU partitioning configuration apparatus provided in an exemplary embodiment of this application;

[0053] Figure 9 A structural block diagram of a server provided for an exemplary embodiment of this application. Detailed Implementation

[0054] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0055] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0056] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0057] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0058] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0059] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0060] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this application's technical solution, based on the prompt message.

[0061] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application; other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0062] To improve the efficiency of GPU partitioning and configuration, in the embodiments provided in this application, such as Figure 1 As shown, Figure 1 This is a schematic diagram of a system architecture provided for an embodiment of this application. The system may include a server 10 and a terminal 20.

[0063] Server 10 is used to generate the GPU configuration interface.

[0064] In this embodiment, server 10 may include one or more GPU cards, each GPU card having a specific model and identifier. When a user needs to partition one or more GPU cards in server 10 into instances, server 10 can obtain its own GPU resource information, which may include the number of GPUs, the model of the GPUs, and the partitioned instances, etc.

[0065] In this embodiment, GPU information can be obtained through a GPU information retrieval command in server 10. This GPU information may include the GPU serial number and identifier, the MIG partitioning specifications supported by each GPU, and the currently partitioned instances. The currently partitioned instances refer to GPUs that have already undergone MIG partitioning, allowing for further MIG partitioning of the corresponding GPUs based on the existing partitioning.

[0066] For example, a command can be used to retrieve the GPU information of a server, such as `nvidia-smi-L`. This GPU information could be: GPU 0: NVIDIA A100-SXM4-40GB (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx). Where:

[0067] GPU 0: Indicates the identified GPU index number, i.e., the sequence number.

[0068] NVIDIA A100-SXM4-40GB: This indicates the specific model of the GPU. It confirms that it is a model that supports MIG splitting, such as A100 or H100.

[0069] UUID: Represents the unique identifier of this GPU.

[0070] In this embodiment, the sharding specifications supported by a specified GPU can be obtained through relevant commands. For example, this can be done using `sudo nvidia-smi mig -i`.<gpu_id> The `--list-gpu-instance-profiles` command retrieves the GPU's supported partitioning profiles, including...<gpu_id> This indicates the GPU's ID; for example, 0 indicates the first GPU in the server.

[0071] Taking the A100 40GB GPU as an example, referring to Table 1, the following expected results can be obtained:

[0072] Table 1:

[0073]

[0074] In Table 1, Name represents the name of the configuration file (Xg.Ygb). X represents the number of compute resource fragments, and Y represents the size of the video memory.

[0075] #GPU slices: indicates the number of slices of computing resources used.

[0076] Mem (MB): The size of the allocated video memory.

[0077] sm(%): Percentage of SM (Streaming Multiprocessor) in use.

[0078] Profile ID: This is the identifier of the configuration file and is required when creating an instance.

[0079] Profile tag: Usually represents an alias for Profile ID.

[0080] In this embodiment, the corresponding GPU-allocated instances on the server can be retrieved using relevant commands, such as `nvidia-smi-L`, to obtain the expected results. For example:

[0081] GPU 0: NVIDIAA100-SXM4-40GB (UUID: GPU...);

[0082] MIG 1g.5gb Device 0: (UUID:MIG-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx);

[0083] MIG 1g.5gb Device 1: (UUID: MIG-yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy).

[0084] Among them, GPU 0 contains the GPU of the partitioned instance, Device 0 represents the first MIG device created through the partitioned instance, Device 1 represents the second MIG device created through the partitioned instance, Device 0 and Device 1 are the MIG device indexes allocated by the system, and 1g.5gb means that the partitioned instance occupies 1 resource fragment and 5gb of video memory.

[0085] In this embodiment, the server 10 can generate a GPU configuration interface based on the GPU resource information obtained above, and can display the GPU configuration interface on the terminal 20.

[0086] Terminal 20 is used to display the GPU configuration interface and receive user access operations on the GPU configuration interface, and send a GPU multi-instance partitioning request strategy to server 10. The GPU multi-instance partitioning request strategy includes a partitioning strategy.

[0087] In this embodiment, a user can access server 10 through terminal 20. For example, a user can enter the access address of server 10 in the browser on terminal 20 and send an access request to server 10 based on the access address. Server 10 can send the data of the GPU configuration interface to terminal 20 based on the access request. After receiving the data of the GPU configuration interface, terminal 20 can parse and display the GPU configuration interface through the browser.

[0088] Users can operate through the GPU configuration interface via terminal 20, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of a GPU configuration interface provided as an example in an embodiment of this application. The configuration interface may include controls for adding and removing partitioning strategies. Users can add or remove partitioning strategies by clicking the add or remove control.

[0089] The configuration interface may also include a GPU selection control for selecting the GPU to be partitioned, for example... Figure 2 The A100-GPU1 will be selected as the target GPU for partitioning. This configuration interface may also include MIG configuration controls and instance quantity controls. Users can use the MIG configuration controls to select the number of resource slices each instance needs, and the instance quantity controls to determine the number of instances to be partitioned. Additionally, users can add or remove controls corresponding to the MIG configuration controls and instance quantity controls under this partitioning strategy.

[0090] like Figure 3 As shown, Figure 3 This is a schematic diagram of a GPU configuration interface provided as yet another exemplary embodiment of this application. The user configures the interface by clicking... Figure 2 When adding controls to the segmentation strategy section, you can add a new segmentation strategy. To distinguish between different segmentation strategies, the first segmentation strategy to appear can be designated as Segmentation Strategy 1, and subsequent segmentation strategies added in sequence can be designated as Segmentation Strategy 2. See [link to documentation] for details. Figure 3 As shown.

[0091] like Figure 3 As shown, the GPU configuration interface can retrieve the GPU information obtained above via an HTTP request, and then render the drop-down list of the GPU selection box on the GPU configuration interface based on the GPU information. When a user selects a different GPU, they can query the MIG splitting template information supported by the corresponding GPU based on the GPU serial number, and the MIG configuration drop-down options will be rendered. The user can then configure the MIG splitting according to their needs.

[0092] As can be seen from the above embodiments, GPU information can be obtained through the GPU information acquisition command in server 10. This GPU information may include the GPU serial number and identifier, the MIG partitioning specifications supported by each GPU, and the currently partitioned instances. The currently partitioned instances refer to GPUs that have already undergone MIG partitioning, allowing for further MIG partitioning of the corresponding GPUs based on the existing partitioning.

[0093] For example, as described in the above embodiments, a command can be used to retrieve the GPU information of the server, such as `nvidia-smi-L`, to retrieve the GPU information in the server. For instance, the GPU information could be: GPU 0: NVIDIA A100-SXM4-40GB (UUID: GPU-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx). Wherein:

[0094] GPU 0: Indicates the identified GPU index number, i.e., the sequence number.

[0095] NVIDIA A100-SXM4-40GB: This indicates the specific model of the GPU. It confirms that it is a model that supports MIG splitting, such as A100 or H100.

[0096] UUID: Represents the unique identifier of this GPU.

[0097] In this embodiment, the sharding specifications supported by a specified GPU can be obtained through relevant commands. For example, this can be done using `sudo nvidia-smi mig -i`.<gpu_id> The `--list-gpu-instance-profiles` command retrieves the GPU's supported partitioning profiles, including...<gpu_id> This indicates the GPU's ID; for example, 0 indicates the first GPU in the server.

[0098] Taking the A100 40GB GPU as an example, see Table 1, which will not be repeated here.

[0099] Combination Figure 3 As shown, users can click the "﹀" button under the MIG configuration control to view a dropdown menu of MIG configuration information available for selection. Figure 3 In the diagram, 1g indicates that each instance occupies one resource slice. A single GPU can include multiple resource slices. It should be noted that the MIG configuration information displayed via the MIG configuration control in Figure 3 allows users to select options such as "1g," "2g," and "3g" from the drop-down menu, representing the requirement of 1, 2, or 3 resource slices, respectively.

[0100] For example, such as Figure 4 As shown, Figure 4 The MIG configuration control displays user-selectable MIG configuration information. Users can click drop-down menus to see options like "1g.5gb," "2g.10gb," and "3g.20gb," representing the required memory allocation of 1 resource slice (5gb), 2 resource slices (10gb), and 3 resource slices (20gb), respectively. This guides users on selecting the appropriate number of resource slices and memory size for each instance. The implementation can display corresponding information on the GPU configuration interface as needed; however, this implementation is not limited to this.

[0101] It should be noted that, based on the above Figures 2-4 In the GPU configuration interface described above, users can configure multi-instance GPU partitioning for one or more GPUs on a server. Users can add new partitioning strategies using the add control corresponding to the partitioning strategy, or remove existing partitioning strategies using the remove control corresponding to the partitioning strategy. Specifically, a shared partitioning strategy applies MIG partitioning to the same GPU.

[0102] In this embodiment, within the same partitioning strategy, the user can select the GPU to be partitioned using MIG, such as A100-GPU1. The user can select the number of instances to be partitioned and the number of resource slices occupied by each instance through the corresponding controls in the MIG configuration. Furthermore, under the same partitioning strategy, the user can configure multiple different MIG partitioning schemes for the same GPU.

[0103] In conjunction with the above embodiments, after the user completes the MIG partitioning configuration through the GPU configuration interface, the user can click the "OK" control on the GPU configuration interface via terminal 20. At this time, server 10 will receive the GPU multi-instance partitioning request strategy, which includes a partitioning strategy containing the MIG partitioning configuration information completed by the user through the GPU configuration interface.

[0104] Upon receiving the partitioning strategy, server 10 will validate it. If the validation passes, server 10 will perform MIG partitioning on the corresponding GPUs according to the strategy. If the validation fails, an error message will be generated, prompting the user to adjust the strategy.

[0105] In this embodiment, different GPUs may support different MIG partitioning specifications; for example, the MIG partitioning specification may represent a table of partitioning configuration combinations supported by the GPU. Therefore, the partitioning specification of the partitioning strategy needs to be verified. Additionally, since the resources occupied by the instances partitioned in the partitioning strategy may exceed the total resources of the corresponding GPU, the GPU may not be able to meet the resource requirements of this MIG partitioning. Therefore, the resource consumption of the partitioning strategy also needs to be verified.

[0106] In this embodiment, when a user configures GPU MIG partitioning through the aforementioned GPU configuration interface, the generated partitioning strategy may include one or more partitioning strategies. Therefore, this embodiment can verify each partitioning strategy contained within the partitioning strategy separately.

[0107] like Figure 5 As shown, Figure 5 This is a flowchart illustrating the segmentation strategy verification method provided in an embodiment of this application. The verification method may include the following steps:

[0108] In step S510, a segmentation strategy is obtained. This segmentation strategy can be one of the segmentation strategies available.

[0109] In the embodiment, regarding one of the segmentation strategies, for example, when a user... Figures 2 to 4 Enter the target operation on the interface shown, and the server can generate information including the following based on the target operation:

[0110]

[0111] The `GPU:0` field stores the GPU number information for the current partitioning strategy. The `Instance` field stores the MIG partitioning configuration information for the current partitioning strategy. This information is generated by converting the MIG configuration and instance count information received from the user through the GPU configuration interface into an object stored under the `Instance` field. A corresponding number of objects are written to the `Instance` field based on the instance count. The `mode` field in each object contains the user-selected MIG configuration information, and the `status` field stores the current instance's status. For example, 'web' indicates the instance is still in the interface planning stage and not yet in use; 'used' indicates the GPU has been partitioned into instances that are currently in use.

[0112] In this embodiment, server 10 performs partitioning specification verification, resource fragment count verification, and video memory usage verification on the partitioning strategy. Each partitioning strategy is verified once. If all partitioning strategies pass, the user configuration verification passes; otherwise, the verification fails. During the verification process, if any one of the partitioning specification verification, resource fragment count verification, or video memory usage verification fails, the verification stops, and a verification failure message is output.

[0113] In step S520, it is determined whether the segmentation strategy meets the segmentation specifications.

[0114] In this embodiment, the server 10 will perform a segmentation specification verification on the segmentation strategy, that is, verify whether the segmentation strategy in the segmentation strategy meets the segmentation combination information supported by the corresponding GPU.

[0115] like Figure 6 As shown, Figure 6 This diagram illustrates the splitting configuration combinations supported by a specific GPU model, as provided in this embodiment. Since each GPU supporting MIG splitting typically has its own supported splitting combinations listed by the manufacturer, this diagram is combined with... Figure 6 As shown, each row represents a possible combination of partitions. For example, the second row represents partitioning the GPU into two instances: one with 4 compute resource fragments and the other with 3 compute resource fragments. The partitioning order must follow the left-to-right order in the table. In this embodiment, this information can be pre-converted into a string array and stored in server 10.

[0116] For example, it can be Figure 6 Convert the information to the following format: [

[0118] '7',

[0119] '4,3',

[0120] '4,2,1',

[0121] '4,1,1,1',

[0122] '2,2,1,1',

[0123] … ]

[0125] The code of the string elements in the array Figure 6 One type of partitioning combination, taking the element '4,3' as an example, represents Figure 6 The second row of the text is a combination of segments.

[0126] In this embodiment, the information of the GPU selected by the current partitioning strategy, including the already partitioned instances and the partitioning combination currently configured by the user, can be integrated. For example, the partitioning strategy currently configured by the user is as follows:

[0127]

[0128] The example will take the already segmented instance information, such as '2g.10gb,2g.10gb', and then convert it into objects {mode:2g.10g,status:'used'} and {mode:2g.10g,status:'used'} by separating them with commas. Finally, this information will be inserted into the header of the Instance field in the segmentation configuration object. The final result is as follows, which can be obtained as an integrated information of the GPU's already segmented instance information and the instance information that the user is preparing to segment.

[0129]

[0130] Here, 'used' refers to instances that have already been split, and 'web' refers to instances that are currently to be split.

[0131] In the embodiment, the first character of the original mode field in the Instance array can be extracted and separated by commas to form a string, such as '2,2,2,1', which represents the combination information of the split instances and the instances to be split.

[0132] If the segmentation strategy meets the segmentation specifications, proceed to step S530; otherwise, proceed to step S540.

[0133] In step S540, if the splitting strategy does not meet the splitting specifications, an error is reported and information indicating that the splitting specifications are not supported is output.

[0134] In this embodiment, the string corresponding to the segmentation combination information, such as '2,2,2,1', can be matched with the segmentation combinations (i.e., segmentation specifications) supported by the corresponding GPU model. If any element in the segmentation combination array for the corresponding GPU model is exactly equal to the segmentation combination string to be verified, the verification passes, and subsequent verifications are performed. Otherwise, the verification fails, and the user is prompted that the current segmentation strategy's segmentation combination does not meet the requirements of the current GPU.

[0135] For example, by combining the string '2,2,2,1' with Figure 6 To perform a match, you can find... Figure 6 The segmentation mode corresponding to line 12 is then determined to be valid.

[0136] In step S530, it is determined whether the number of resource fragments occupied by the partitioning strategy meets the requirements, that is, whether the number of resource fragments occupied by the partitioning strategy is not greater than the total number of resource fragments of the corresponding GPU.

[0137] Specifically, after the splitting and combining verification passes, the embodiment will verify the number of computing resource slices. The embodiment can split the integrated string obtained above into individual numbers according to commas, and then add them together to obtain the total number of integrated computing resource slices. When this value is less than the current maximum number of computing resource slices of the GPU, the verification passes and the next verification step is performed; otherwise, the verification fails and the user is prompted that the number of resource slices occupied by the current splitting strategy exceeds the total number of computing resource slices of the GPU.

[0138] For example, adding the numbers in the string '2,2,2,1' gives a total of 7 resource shards after consolidation. This means the sum of the resource shards occupied by the instance under the current partitioning strategy and the resource shards occupied by the partitioned instance is 7. Figure 6 As shown, the total number of resource fragments contained in the GPU is 7, which does not exceed the total number of resource fragments contained in the GPU, so the verification passes.

[0139] In this embodiment, when the number of resource fragments does not meet the requirements, step S550 is executed, an error is reported, and a message is displayed indicating that the number of resource fragments used has exceeded the limit.

[0140] In this embodiment, when the number of resource fragments meets the requirements, step S560 is executed to determine whether the current strategy's use of video memory meets the requirements.

[0141] Specifically, after the resource slice count verification passes, the program will verify the video memory size. In this example, the program can extract the second half of the `mode` field of each element in the `Instance` array of the aforementioned integration information, representing the video memory size, using a regular expression, and then sum them to obtain the total integrated video memory size. If this value is greater than the total video memory capacity of the graphics card, the verification fails, and the user is prompted that the current partitioning strategy's video memory capacity exceeds the GPU's video memory size. If this value is greater than the total video memory capacity of the graphics card, the verification passes, and the entire partitioning strategy passes verification.

[0142] For example, by adding the video memory occupied by the current instance to be split and the already split instance, i.e., 10gb + 10gb + 10gb + 5gb, we get 35gb. We can then determine whether this 35gb is greater than the total video memory of the corresponding GPU, and thus determine whether the video memory occupied by the current splitting strategy meets the requirements.

[0143] If the current partitioning strategy meets the memory requirements, proceed to step S570, and the verification passes.

[0144] When the current partitioning strategy does not meet the requirements for video memory usage, step S580 is executed, and an error is reported, indicating that the video memory limit has been exceeded.

[0145] In the embodiments provided in this application, the server obtains GPU resource information, such as GPU information, to determine the currently partitioned MIG instances, and can return this information to the aforementioned GPU configuration interface via an HTTP interface. This allows the server to display a MIG partitioning configuration form based on GPU information, such as a GPU list. Users can configure partitioning for a single GPU or multiple GPUs using the MIG information in the GPU configuration interface, generating partitioning strategies as needed. The server validates each partitioning strategy; if any partitioning strategy fails validation, the corresponding error message is displayed on the GPU configuration interface. When all partitioning strategies pass validation, the server generates partitioning instructions based on the partitioning strategies to partition the corresponding GPU instances.

[0146] The embodiment, through a visual GPU configuration interface and a function to verify the rationality of the partitioning strategy, can overcome the problems that users are prone to making spelling errors and cannot verify the partitioning configuration when configuring GPU MIG partitioning using the command line. The embodiment improves the ease of use of GPU MIG partitioning through the GPU configuration interface, thereby greatly improving the user's configuration efficiency.

[0147] Based on the above embodiments, this application also provides a GPU multi-instance configuration method, which can be applied to servers, such as... Figure 7 As shown, the method may include the following steps:

[0148] In step S710, a GPU configuration interface is generated.

[0149] The GPU configuration interface includes at least one GPU partitioning strategy configuration area, which includes a GPU selection control and at least one configuration group, which includes an instance number control and a resource slice number configuration control.

[0150] In step S720, the user's target operation in the GPU partitioning strategy configuration area is received.

[0151] The target operation includes determining the target GPU to be partitioned using the GPU selection control, determining the number of instances to be partitioned using the instance number control, and determining the number of resource fragments occupied by each instance using the resource fragment number configuration control.

[0152] In step S730, in response to the target operation, a partitioning strategy is executed on the target GPU; wherein, the partitioning strategy includes obtaining the number of instances to partition the target GPU and the number of resource slices occupied by each instance.

[0153] In this embodiment, the GPU configuration interface is used to generate a partitioning strategy with multiple GPU instances. The GPU configuration interface includes at least one partitioning strategy configuration area. Each partitioning strategy configuration area includes a GPU selection control and at least one configuration group. The configuration group includes an instance number determination control and a resource fragment number configuration control. The GPU selection control is used to determine the target GPU to be partitioned, the instance number determination control is used to determine the number of instances of the target GPU, and the resource fragment number configuration control is used to determine the number of resource fragments occupied by each instance.

[0154] In the embodiments, the above can be combined with Figure 3 or Figure 4 As shown, the configuration area for this segmentation strategy can be... Figure 3 or Figure 4 The configuration area includes either partitioning strategy 1 or partitioning strategy 2. Taking partitioning strategy 1 as an example, this partitioning strategy configuration area includes a GPU selection control, which allows you to select the target GPU to be partitioned, such as A100-GPU1. This configuration group includes an instance number determination control and a resource fragment number configuration control, which allows you to select the number of instances to be partitioned, for example... Figure 3 or Figure 4 The number of instances is 2. The resource fragment number configuration control determines the number of resource fragments occupied by each instance. For example, 1g5gb means that 1 resource fragment corresponds to 5gb of video memory.

[0155] In this embodiment, the server generates a GPU configuration interface and sends the data of the GPU configuration interface to the terminal, so that the terminal can display the GPU configuration interface. In this way, the user can configure the GPU MIG through the GPU configuration interface, which can improve the ease of configuration and avoid human operation errors that are easy to occur when configuring the GPU MIG through the command line, such as spelling errors. This can greatly improve the efficiency of GPU MIG splitting and configuration.

[0156] In this embodiment, the GPU configuration interface also includes the ability to configure adding and removing controls, such as... Figures 2-4 As shown, the configuration add control is used to add at least one configuration group, and the configuration remove control is used to remove at least one configuration group.

[0157] Users can add one or more configuration groups through the configuration add control, and users can remove one or more existing configuration groups through the configuration remove control, which enables flexible configuration.

[0158] In this embodiment, a user can add a configuration group through the configuration add control. For example, if the GPU configuration interface includes a first configuration group, when adding another configuration group, the GPU configuration interface can include the first configuration group and the second configuration group.

[0159] Specifically, the GPU partitioning strategy configuration area includes a GPU selection control, a first configuration group, and a second configuration group. The target operation also includes determining the target GPU to be partitioned using the GPU selection control, determining the number of first instances to be partitioned using the instance quantity control in the first configuration group, and determining the number of resource fragments occupied by each instance using the resource fragment number configuration control in the first configuration group. Furthermore, it also determines the number of second instances to be partitioned using the instance quantity control in the second configuration group, and determines the number of resource fragments occupied by each instance using the resource fragment number configuration control in the second configuration group.

[0160] For example, see Figure 3 The partitioning strategy 2 includes a GPU selection control, as well as an instance number control and a resource fragment number configuration control in the first configuration group and a second configuration group. The instance number control allows the user to select the number of instances to be partitioned based on the user's operation, and the resource fragment number configuration control is used to determine the number of resource fragments required for each instance based on the user's selection operation.

[0161] This allows users to determine the number of instances to be split and the number of resource shards required for each configuration group, as needed.

[0162] In this embodiment, the GPU configuration interface also includes a partitioning strategy addition control and a partitioning strategy removal control. The partitioning strategy addition control is used to add at least one set of partitioning strategies, and the partitioning strategy removal control is used to remove at least one set of partitioning strategies.

[0163] Specifically, in combination Figure 4 As shown, the GPU configuration interface includes a first GPU partitioning strategy configuration area and a second GPU partitioning strategy configuration area. Each area includes a GPU selection control and at least one configuration group. The first GPU partitioning strategy configuration area is used to generate a first partitioning strategy, and the second GPU partitioning strategy configuration area is used to generate a second partitioning strategy. The target operation also includes determining the first GPU to be partitioned, the number of instances, and the number of resource fragments per instance using the GPU selection control, instance quantity control, and resource fragment number configuration control in the first GPU partitioning strategy configuration area, and determining the second GPU to be partitioned, the number of instances, and the number of resource fragments per instance using the same controls in the second GPU partitioning strategy configuration area. The target GPU includes both the first and second GPUs.

[0164] In this embodiment, the user can add at least one set of partitioning strategies according to the partitioning strategy addition control. Each set of partitioning strategies includes a GPU partitioning strategy configuration area, which includes a corresponding GPU selection control, instance quantity control, and resource fragment quantity configuration control.

[0165] Combination Figures 2-4 As shown, users can add one or more segmentation strategies using the segmentation strategy addition control. In this example, the segmentation strategies can be named sequentially, such as Segmentation Strategy 1, Segmentation Strategy 2, etc. Additionally, users can remove one or more existing segmentation strategies using the segmentation strategy removal control. After removal, the segmentation strategy numbers are automatically adjusted to reflect the order of the segmentation strategies on the interface. This allows users to flexibly configure segmentation strategies through the segmentation strategy addition and removal controls.

[0166] Based on the above embodiments, in another embodiment provided in this application, the method may include the following steps:

[0167] In step S740, in response to the above target operation, it is verified whether the segmentation strategy meets the verification conditions.

[0168] In this embodiment, the partitioning strategy includes the identifier of the target GPU, the number of instances to be partitioned, and the number of resource slices occupied by each instance. This partitioning strategy is used to instruct the server to partition the target GPU into multiple instances.

[0169] In this embodiment, users can perform GPU MIG configuration operations through the GPU configuration interface provided in the above embodiment. After the configuration is completed, the splitting strategy can be generated by clicking the control on the GPU configuration interface, and the strategy can be verified.

[0170] This verification condition is used to verify the GPU partitioning specifications and resource usage of the number of instances and resource fragments.

[0171] In this embodiment, the verification segmentation strategy can be one of the segmentation strategies in the above embodiments, such as segmentation strategy 1 or segmentation strategy 2. The segmentation strategy includes the corresponding number of instances and the number of resource fragments occupied by each instance. For example, there are 2 instances, and each instance occupies 1 resource fragment. It is necessary to verify whether the segmentation strategy meets the verification conditions, such as whether the segmentation specifications and resource usage meet the requirements.

[0172] In step S750, when the segmentation strategy meets the verification conditions, the target GPU is segmented into multiple instances based on the segmentation strategy.

[0173] This embodiment validates the partitioning strategy. If the strategy meets the validation criteria, the server can partition the target GPU into multiple instances based on the strategy. If the strategy fails to meet the validation criteria, an error message is generated, allowing the user to readjust the partitioning strategy accordingly. By pre-validating the partitioning strategy and executing it only after successful validation, successful partitioning of the GPU MIG is guaranteed. This avoids the failure of GPU MIG configuration strategies due to human error when directly executing user-sent configurations, thus significantly improving the efficiency of GPU MIG partitioning and configuration.

[0174] Based on the above embodiments, in another embodiment provided in this application, the method may further include the following steps:

[0175] In step S760, it is verified whether the splitting strategy meets the splitting specifications of the target GPU.

[0176] In step S770, if the partitioning strategy meets the partitioning specifications of the target GPU, it is determined that the partitioning strategy meets the verification conditions.

[0177] In this embodiment, the number of instances to be partitioned and the number of resource fragments occupied by each instance can be matched with the partitioning template of the target GPU. When a partitioning specification that matches the number of instances and the number of resource fragments exists in the partitioning template, it is determined that the partitioning strategy meets the partitioning specification of the target GPU. This partitioning template includes multiple predefined partitioning specifications.

[0178] For example, see Figure 6 As shown, Figure 6 For a specific GPU model, the supported partitioning combinations can be used as the partitioning template for that GPU. Figure 6 Each row in the code represents a segmentation specification, for example... Figure 6 The third line indicates that the split strength occupies 3 resource fragments, 2 resource fragments, and 1 resource fragment respectively. Among them, Figure 6 The serial number on the left indicates the row number and also corresponds to a segmentation specification or segmentation mode.

[0179] In this embodiment, since the target GPU has a predefined corresponding partitioning specification, by matching the number of instances to be partitioned and the number of resource fragments occupied by each instance with the partitioning template, it can be determined in a timely manner whether the partitioning strategy meets the partitioning specification of the target GPU, thus avoiding GPU MIG partitioning failure due to non-compliance with the partitioning specification.

[0180] Based on the above embodiments, for example, the string in the above embodiments, such as '2,2,2,1', can be matched with the segmentation combination (i.e., segmentation specification) supported by the corresponding GPU model. For example, by matching the string '2,2,2,1' with... Figure 6 To perform a match, you can find... Figure 6 The segmentation mode corresponding to line 12 is then determined to be valid.

[0181] Based on the above embodiments, in another embodiment provided in this application, the method may further include the following steps:

[0182] In step S781, when the partitioning strategy meets the partitioning specifications of the target GPU, the remaining resources of the target GPU and the first resources occupied when partitioning the target GPU based on the partitioning strategy are obtained.

[0183] In this embodiment, since the target GPU may include partitioned instances, the resources occupied by the partitioned instances can be obtained, and the total resources of the target GPU can be obtained, thus obtaining the remaining resources of the target GPU.

[0184] Specifically, in this embodiment, when obtaining the remaining resources of the target GPU, it can be determined whether the target GPU has any already partitioned instances. If the target GPU has partitioned instances, the second resource occupied by the partitioned instances is obtained, and the remaining resources of the target GPU are determined based on the total resources of the target GPU and the second resource. Alternatively, if the target GPU does not have any already partitioned instances, the total resources of the target GPU are used as the remaining resources of the target GPU.

[0185] In this embodiment, the partitioning combination in the partitioning strategy can also be obtained, and the partitioning combination in the remaining resources of the target GPU can be matched according to the partitioning combination. If the partitioning combination in the remaining resources of the target GPU contains the partitioning combination in the partitioning strategy, it can be determined that the partitioning strategy meets the partitioning specification of the target GPU.

[0186] In addition, since some GPUs already have partitioned instances, when partitioning instances on the same GPU, the partitioning can be done on the basis of the original partitioning. This requires considering the resources occupied by the partitioned instances and the total resources of the GPU to obtain the remaining resources of the GPU. This helps determine whether the remaining resources can meet the requirements of the instances to be partitioned, and avoids GPU MIG partitioning configuration failure due to insufficient remaining resources of the GPU.

[0187] In step S782, it is determined whether the remaining resources are less than the first resource.

[0188] If the remaining resources are not less than the first resource, proceed to step 783; if the remaining resources are less than the first resource, proceed to step S784.

[0189] In step S783, it is determined that the segmentation strategy meets the verification conditions.

[0190] In step S784, it is determined that the segmentation strategy does not meet the verification conditions.

[0191] In the embodiment, when determining whether the remaining resource is less than the first resource, there are two possible scenarios.

[0192] In the first scenario, when the target GPU has already been partitioned, the number of second occupied resource fragments and the number of second occupied video memory are obtained. If the sum of the first and second occupied resource fragments is not greater than the total number of resource fragments in the target GPU, and the sum of the first and second occupied video memory is not greater than the total video memory in the target GPU, then the remaining resources are determined to be no less than the first resource.

[0193] Alternatively, if the sum of the first number of occupied resource fragments and the second number of occupied resource fragments is greater than the total number of resource fragments contained in the target GPU, or if the sum of the first number of occupied video memory and the second number of occupied video memory is greater than the total video memory contained in the target GPU, then the remaining resources are determined to be less than the first resource.

[0194] In the second scenario, if there are no partitioned instances on the target GPU, and the number of resource fragments occupied by the first instance is not greater than the total number of resource fragments contained in the target GPU, and the number of video memory occupied by the first instance is not greater than the total video memory contained in the target GPU, then the remaining resources are determined to be not less than the first resource. Alternatively, if there are no partitioned instances on the target GPU, and the number of resource fragments occupied by the first instance is greater than the total number of resource fragments contained in the target GPU, or the number of video memory occupied by the first instance is greater than the total video memory contained in the target GPU, then the remaining resources are determined to be less than the first resource.

[0195] In the embodiment, in both cases where there are already partitioned instances on the target GPU or when there are no already partitioned instances on the target GPU, the remaining resources are determined based on the number of resource fragments occupied and the amount of video memory occupied, respectively. This can effectively determine whether the remaining resources of the target GPU can meet the requirements of the instance to be partitioned, and avoid the failure of GPU MIG partitioning configuration due to insufficient remaining resources of the target GPU.

[0196] In the embodiment, since the instance to be split in the splitting strategy will occupy a certain amount of GPU resources, the remaining resources in the target GPU are obtained and it is determined whether the remaining resources are less than the resources occupied by the instance to be split, thereby realizing the verification of the resources occupied when splitting the target GPU.

[0197] The embodiment can effectively determine whether the partitioning strategy meets the verification conditions by verifying the partitioning specifications and resource usage of the target GPU, thus avoiding GPU MIG partitioning failures when executing the configuration strategy and improving the execution efficiency of the strategy.

[0198] In the case of dividing each functional module according to its corresponding functions, this application provides a GPU partitioning configuration device, which can be a server, a terminal, or a chip applied to a server. Figure 8 A schematic block diagram of the functional modules of a GPU partitioning configuration apparatus provided for an exemplary embodiment of this application. Figure 8 As shown, the GPU partitioning configuration device includes:

[0199] The interface generation module 81 is used to generate a GPU configuration interface; wherein the GPU configuration interface includes at least one GPU partitioning strategy configuration area, the GPU partitioning strategy configuration area includes a GPU selection control, and at least one configuration group, the configuration group includes: instance number control and resource fragment number configuration control;

[0200] The operation receiving module 82 is used to receive the target operation of the user in the GPU partitioning strategy configuration area. The target operation includes determining the target GPU to be partitioned through the GPU selection control, determining the number of instances to be partitioned through the instance number control, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control.

[0201] The strategy execution module 83 is used to execute a partitioning strategy on the target GPU in response to the target operation; wherein the partitioning strategy includes obtaining the number of instances to partition the target GPU and the number of resource slices occupied by each instance.

[0202] In this solution, users can perform split configuration through the GPU configuration interface, which improves the ease of configuration and avoids human error such as spelling mistakes that are easy to occur when configuring GPU MIG through the command line. This can greatly improve the efficiency of GPU MIG split configuration.

[0203] In one possible implementation provided in this application, the GPU configuration interface further includes a configuration add control and a configuration remove control. The configuration add control is used to add at least one configuration group, and the configuration remove control is used to remove at least one configuration group.

[0204] In this solution, users can add one or more configuration groups through the configuration add control, and users can remove one or more existing configuration groups through the configuration remove control, which enables flexible configuration.

[0205] In one possible implementation provided in this application, the GPU partitioning strategy configuration area includes a GPU selection control, a first configuration group, and a second configuration group; the target operation further includes determining the target GPU to be partitioned through the GPU selection control, determining the number of first instances to be partitioned through the instance number control in the first configuration group, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control in the first configuration group.

[0206] Additionally, the number of second instances to be split is determined by the instance quantity control in the second configuration group, and the number of resource shards occupied by each instance is determined by the resource shard number configuration control in the second configuration group.

[0207] This solution allows users to determine the number of instances to be split and the number of resource slices required for each configuration group, as needed.

[0208] In one possible implementation provided in this application, the GPU configuration interface further includes a partitioning strategy addition control and a partitioning strategy removal control; the partitioning strategy addition control is used to add at least one set of partitioning strategies, and the partitioning strategy removal control is used to remove at least one set of partitioning strategies.

[0209] In this solution, users can remove one or more existing segmentation strategies using the segmentation strategy removal control. After removal, the segmentation strategy numbers are automatically adjusted to reflect the order of the segmentation strategies on the interface. This allows users to flexibly configure segmentation strategies by adding or removing controls based on them.

[0210] In one possible implementation provided in this application, the GPU configuration interface includes a first GPU partitioning strategy configuration area and a second GPU partitioning strategy configuration area. The first and second GPU partitioning strategy configuration areas each include a GPU selection control and at least one configuration group. The first GPU partitioning strategy configuration area is used to generate a first partitioning strategy, and the second GPU partitioning strategy configuration area is used to generate a second partitioning strategy. The target operation further includes determining the first GPU to be partitioned, the number of instances, and the number of resource fragments occupied by each instance through the GPU selection control, instance quantity control, and resource fragment number configuration control in the first GPU partitioning strategy configuration area, and determining the second GPU to be partitioned, the number of instances, and the number of resource fragments occupied by each instance through the GPU selection control, instance quantity control, and resource fragment number configuration control in the second GPU partitioning strategy configuration area, respectively. The target GPU includes both the first and second GPUs.

[0211] In this solution, users can add at least one set of partitioning strategies based on the partitioning strategy. Each set of partitioning strategies includes a GPU partitioning strategy configuration area, which includes the corresponding GPU selection control, instance quantity control, and resource fragment quantity configuration control.

[0212] In one possible implementation provided in this application, the device may further include a verification module, specifically used for:

[0213] In response to the target operation, verify whether the segmentation strategy meets the verification conditions;

[0214] When the partitioning strategy meets the verification conditions, the target GPU is partitioned into multiple instances based on the partitioning strategy.

[0215] This solution verifies the segmentation strategy and executes it only when the verification is successful, thus ensuring successful GPU MIG segmentation. This avoids the situation where GPU MIG configuration strategy execution fails due to human error when related technologies directly execute the GPU MIG configuration strategy sent by the user, thereby greatly improving the efficiency of GPU MIG segmentation and configuration.

[0216] In one possible implementation provided in this application, the verification module is further used for:

[0217] Verify whether the partitioning strategy meets the partitioning specifications of the target GPU;

[0218] When the partitioning strategy meets the partitioning specifications of the target GPU, it is determined that the partitioning strategy meets the verification conditions.

[0219] In this scheme, since the target GPU has a predefined corresponding partitioning specification, by matching the number of instances to be partitioned and the number of resource fragments occupied by each instance with the partitioning template, it can be determined in time whether the partitioning strategy meets the partitioning specification of the target GPU, thus avoiding GPU MIG partitioning failure due to non-compliance with the partitioning specification.

[0220] In one possible implementation of the embodiments provided in this application, the verification module is further configured to:

[0221] The number of instances and resource fragments are matched with the partitioning template of the target GPU; the partitioning template includes a variety of predefined partitioning specifications;

[0222] When a partitioning specification exists in the partitioning template that matches the number of instances and resource fragments, the partitioning strategy is determined to satisfy the partitioning specification of the target GPU.

[0223] In this scheme, since the target GPU has a predefined corresponding partitioning specification, by matching the number of instances to be partitioned and the number of resource fragments occupied by each instance with the partitioning template, it can be determined in time whether the partitioning strategy meets the partitioning specification of the target GPU, thus avoiding GPU MIG partitioning failure due to non-compliance with the partitioning specification.

[0224] In one possible implementation provided in this application, the verification module is further used for:

[0225] When the partitioning strategy meets the partitioning specifications of the target GPU, obtain the remaining resources of the target GPU and the first resource occupied when partitioning the target GPU based on the partitioning strategy;

[0226] Determine if the remaining resources are less than the first resource;

[0227] If the remaining resources are not less than the first resource, the partitioning strategy is determined to meet the verification conditions.

[0228] In this solution, whether the target GPU has a partitioned instance or not, the remaining resources are determined based on the number of resource fragments and the amount of video memory used, respectively. This can effectively determine whether the remaining resources of the target GPU can meet the requirements of the instance to be partitioned, and avoid the failure of GPU MIG partitioning configuration due to insufficient remaining resources of the target GPU.

[0229] Based on the above embodiments, in another embodiment provided in this application, a GPU partitioning configuration system is also provided. This system includes a terminal for displaying a GPU configuration interface and the aforementioned server. See details... Figure 1 The corresponding implementation examples are not described here.

[0230] This application also provides a server, including: at least one processor and at least one GPU; the processor is used to store at least one processor-executable instruction in a memory; wherein the at least one processor is configured to execute instructions to implement the methods disclosed in the embodiments of this application.

[0231] The aforementioned processor can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this application can be implemented by integrated logic circuits in the processor's hardware or by software instructions. The aforementioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in memory, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0232] Furthermore, various operations / processes according to embodiments of this application, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a server with a dedicated hardware architecture, such as... Figure 9 The server 1900 shown is equipped with the programs that constitute the software. When various programs are installed on the server, it is able to perform various functions, including those mentioned above. Figure 9 A structural block diagram of a server provided for an exemplary embodiment of this application.

[0233] Server 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments described and / or claimed herein.

[0234] like Figure 9As shown, server 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 1902 or loaded into random access memory (RAM) 1903 from storage unit 1908. RAM 1903 may also store various programs and data required for the operation of server 1900. Server 1900 also includes a GPU 1910. Computing unit 1901, ROM 1902, GPU 1910, and RAM 1903 are interconnected via bus 1904. Input / output (I / O) interface 1905 is also connected to bus 1904. The number of GPUs 1910 may include multiple GPUs.

[0235] Multiple components in server 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information to server 1900. Input unit 1906 can receive input numeric or character information and generate key signal inputs related to user settings and / or function control of the server. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, disks and optical discs. Communication unit 1909 allows server 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0236] The computing unit 1901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in the embodiments of this application can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on a server via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0237] This application also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by the processor of a server, the server is able to perform the methods disclosed in the embodiments of this application.

[0238] The computer-readable storage medium in this application embodiment may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0239] The aforementioned computer-readable medium may be included in the aforementioned server; or it may exist independently and not assembled into the server.

[0240] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this application.

[0241] In embodiments of this application, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0242] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0243] The modules, components, or units described in the embodiments of this application can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0244] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0245] The above description is merely an embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0246] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A GPU partitioning configuration method, characterized in that, Applied to a server, the method includes: Generate a GPU configuration interface; wherein the GPU configuration interface includes at least one GPU partitioning strategy configuration area, the GPU partitioning strategy configuration area includes a GPU selection control, and at least one configuration group, the configuration group including: instance number control and resource fragment number configuration control; The system receives a user's target operation in the GPU partitioning strategy configuration area. The target operation includes determining the target GPU to be partitioned through the GPU selection control, determining the number of instances to be partitioned through the instance quantity control, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control. In response to the target operation, a partitioning strategy is executed on the target GPU; wherein the partitioning strategy includes obtaining the number of instances to partition the target GPU and the number of resource slices occupied by each instance.

2. The method according to claim 1, characterized in that, The GPU configuration interface also includes a configuration add control and a configuration remove control. The configuration add control is used to add at least one of the configuration groups, and the configuration remove control is used to remove at least one of the configuration groups.

3. The method according to claim 2, characterized in that, The GPU partitioning strategy configuration area includes a GPU selection control, a first configuration group, and a second configuration group; the target operation also includes determining the target GPU to be partitioned through the GPU selection control, determining the number of first instances to be partitioned through the instance number control in the first configuration group, and determining the number of resource fragments occupied by each instance through the resource fragment number configuration control in the first configuration group. In addition, the number of second instances to be split is determined by the instance quantity control in the second configuration group, and the number of resource fragments occupied by each instance is determined by the resource fragment number configuration control in the second configuration group.

4. The method according to claim 1, characterized in that, The GPU configuration interface also includes a partitioning strategy addition control and a partitioning strategy removal control; the partitioning strategy addition control is used to add at least one set of partitioning strategies, and the partitioning strategy removal control is used to remove at least one set of partitioning strategies.

5. The method according to claim 4, characterized in that, The GPU configuration interface includes a first GPU partitioning strategy configuration area and a second GPU partitioning strategy configuration area. The first and second GPU partitioning strategy configuration areas each include a GPU selection control and at least one configuration group. The first GPU partitioning strategy configuration area is used to generate a first partitioning strategy, and the second GPU partitioning strategy configuration area is used to generate a second partitioning strategy. The target operation further includes determining the first GPU to be partitioned, the number of instances, and the number of resource fragments occupied by each instance using the GPU selection control, instance quantity control, and resource fragment number configuration control in the first GPU partitioning strategy configuration area; and determining the second GPU to be partitioned, the number of instances, and the number of resource fragments occupied by each instance using the GPU selection control, instance quantity control, and resource fragment number configuration control in the second GPU partitioning strategy configuration area. The target GPU includes the first GPU and the second GPU.

6. The method according to claim 1, characterized in that, The method includes: In response to the target operation, verify whether the segmentation strategy meets the verification conditions; When the segmentation strategy meets the verification condition, the target GPU is segmented into multiple instances based on the segmentation strategy.

7. The method according to claim 6, characterized in that, The method further includes: Verify whether the segmentation strategy meets the segmentation specifications of the target GPU; When the partitioning strategy meets the partitioning specifications of the target GPU, it is determined that the partitioning strategy meets the verification conditions.

8. The method according to claim 7, characterized in that, The verification of whether the partitioning strategy meets the partitioning specifications of the target GPU includes: The number of instances and the number of resource fragments are matched with the partitioning template of the target GPU; the partitioning template includes a variety of predefined partitioning specifications; When a partitioning specification exists in the partitioning template that matches the number of instances and the number of resource fragments, it is determined that the partitioning strategy satisfies the partitioning specification of the target GPU.

9. The method according to claim 7, characterized in that, The method further includes: When the partitioning strategy meets the partitioning specifications of the target GPU, the remaining resources of the target GPU and the first resources occupied when partitioning the target GPU based on the partitioning strategy are obtained. Determine whether the remaining resources are less than the first resource; When the remaining resources are not less than the first resources, the segmentation strategy is determined to meet the verification condition.

10. A server, characterized in that, include: At least one processor; At least one GPU; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to perform multi-instance partitioning of the at least one GPU when implementing the method as described in any one of claims 1-9.

Images