Method, apparatus, device, medium and product for determining AI computing environment

Abstract

This invention discloses a method, apparatus, device, medium, and product for determining an AI computing environment. The method includes: responding to an event of deploying a target application, retrieving computing environment scheduling parameters associated with an application identifier of the target application, wherein the computing environment scheduling parameters are used to characterize the demand data of the AI ​​accelerator cards adapted to the target application; determining at least one target AI accelerator card adapted to the target application based on the computing environment scheduling parameters and the accelerator card data of at least one AI accelerator card received; distributing the target application to the target hardware device corresponding to the at least one target AI accelerator card to deploy the target application on the target hardware device, and scheduling the target AI accelerator card based on the target application. The technical solution provided by this invention achieves the effect of automatically determining the target computing environment adapted to the AI ​​accelerator card.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a method, apparatus, device, medium and product for determining an AI computing environment. Background Technology

[0002] With the development of artificial intelligence, various tasks can be processed based on neural network models. To improve the processing efficiency of neural network models, AI accelerator cards are typically added to the hardware devices that execute these models. AI accelerator cards generally consist of a hardware layer, a device driver layer, a parallel computing library, a model engine layer, and a model application layer.

[0003] When the inventors implemented the above technical solution in the above manner, they discovered the following problems:

[0004] The model engine layer and model application layer have specific requirements for the version of the parallel computing library. Similarly, the parallel computing library also has specific requirements for the hardware layer and device driver layer. Deploying the model application layer and model engine layer to the corresponding hardware devices requires manual analysis and processing of data and application data from the AI ​​accelerator card, which is costly and inefficient. Summary of the Invention

[0005] This invention provides a method, apparatus, device, medium, and product for determining an AI computing environment, so as to automatically determine the target computing environment, i.e. the target application, that is adapted to the AI ​​accelerator card, thereby improving the efficiency of target application deployment and the compatibility with the AI ​​accelerator card.

[0006] In a first aspect, embodiments of the present invention provide a method for determining an AI computing environment, the method comprising:

[0007] In response to an event that a target application is deployed, computing environment scheduling parameters associated with the application identifier of the target application are retrieved, wherein the computing environment scheduling parameters are used to characterize the AI ​​accelerator card requirements of the target application.

[0008] Based on the computing environment scheduling parameters and the received acceleration card data of at least one AI acceleration card, determine at least one target AI acceleration card that is compatible with the target application.

[0009] The target application is distributed to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

[0010] Furthermore, the method also includes:

[0011] Based on the scanning nodes deployed in the hardware device, read the acceleration card data of at least one AI acceleration card plugged into the hardware device;

[0012] The scanning node reports the accelerator card identifier of the at least one AI accelerator card, the device identifier of the hardware device to which it belongs, and the accelerator card data.

[0013] Furthermore, prior to the event of deploying the target application, the method further includes:

[0014] In response to an event that creates an image corresponding to the target application, the definition specifications of the target application are obtained;

[0015] Based on the defined specifications, the target metadata corresponding to the target application is added to the image, and the image is uploaded to the image repository. In response to the event of deploying the target application, the target metadata of the target application is retrieved from the image repository and used as the scheduling parameters of the computing environment.

[0016] Furthermore, retrieving the computing environment scheduling parameters associated with the application identifier of the target application includes:

[0017] Based on the application identifier of the target application, determine the storage address of the image corresponding to the target application in the image repository;

[0018] Based on the storage address, the target metadata corresponding to the target application is retrieved from the image repository, and the target metadata is used as the computing environment scheduling parameter.

[0019] Furthermore, the computing environment scheduling parameters include at least hardware architecture version requirements and parallel computing library version requirements, and the accelerator card data includes at least the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card.

[0020] Furthermore, determining the at least one target AI accelerator card suitable for the target application based on the computing environment scheduling parameters and the received accelerator card data of at least one AI accelerator card includes:

[0021] Based on the hardware architecture version requirement parameters and parallel computing library version requirement parameters in the computing environment scheduling parameters, as well as the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card in the accelerator card data, at least one target AI accelerator card that meets the computing environment scheduling parameters is determined.

[0022] Furthermore, deploying the target application to the hardware device corresponding to the at least one target AI accelerator card includes:

[0023] Based on the device identifier of the hardware device to which the at least one target AI accelerator card belongs, the target hardware device to which the target application is to be deployed is determined;

[0024] The target application and the accelerator card identifier of the target AI accelerator card associated with the target application are sent to the target hardware device so as to determine the application scope of the target application based on the accelerator card identifier in the target hardware device.

[0025] Secondly, embodiments of the present invention also provide an apparatus for determining an AI computing environment, the apparatus comprising:

[0026] The computing environment scheduling parameter retrieval module is used to retrieve computing environment scheduling parameters associated with the application identifier of the target application in response to an event of deploying the target application. The computing environment scheduling parameters are used to characterize the demand data of the AI ​​acceleration card adapted to the target application.

[0027] The target AI accelerator card determination module is used to determine at least one target AI accelerator card that is compatible with the target application based on the computing environment scheduling parameters and the accelerator card data of at least one AI accelerator card received.

[0028] The target application deployment module is used to distribute the target application to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

[0029] Thirdly, embodiments of the present invention provide a server, the server comprising:

[0030] One or more processors;

[0031] Memory, used to store one or more programs;

[0032] When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the AI ​​computing environment as provided in any embodiment of the present invention.

[0033] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining an AI computing environment as provided in any embodiment of the present invention.

[0034] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the method for determining the AI ​​computing environment as described in any embodiment of the present invention.

[0035] The technical solution provided by this invention, in response to an event of deploying a target application, can retrieve computing environment scheduling parameters associated with the target application. Since these computing environment scheduling parameters mainly characterize the demand data of AI accelerator cards adapted to the target application, based on the computing environment scheduling data and the received accelerator card data of the AI ​​accelerator cards, it can be determined which target AI accelerator cards can use the target application, that is, which are adapted to the target application. Furthermore, the target application can be distributed to at least one target hardware device corresponding to the target AI accelerator card to deploy the target application on the target hardware device, and then the target AI accelerator cards can be scheduled based on the target application. This solves the problem of low efficiency and poor automation in the prior art, which requires manual determination of which accelerator cards use which applications based on the accelerator card data and application data. It realizes that regardless of whether an AI accelerator card is added to the hardware device or the accelerator card parameters of the AI ​​accelerator card change, the AI ​​accelerator card adapted to the target application can be automatically determined, achieving the effect of self-matching and self-adaptation of the AI ​​accelerator card computing scheduling environment. Attached Figure Description

[0036] To more clearly illustrate the technical solutions of exemplary embodiments of the present invention, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the drawings of the embodiments to be described in this invention, and not all of the drawings. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0037] Figure 1 A flowchart illustrating a method for determining an AI computing environment as provided in an embodiment of the present invention;

[0038] Figure 2 A flowchart illustrating a method for determining an AI computing environment as provided in an embodiment of the present invention;

[0039] Figure 3 A flowchart illustrating a method for determining an AI computing environment as provided in an embodiment of the present invention;

[0040] Figure 4 A schematic diagram illustrating a method for determining an AI computing environment according to an embodiment of the present invention;

[0041] Figure 5 A schematic diagram of the structure of an AI computing environment determination device provided in an embodiment of the present invention;

[0042] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0043] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0045] Before introducing the technical solutions provided in the embodiments of the present invention, the application scenarios can be illustrated first.

[0046] To improve the processing efficiency of various types of network models, corresponding AI accelerator cards can be configured for hardware devices. An AI accelerator card can be understood as an acceleration card specifically designed for artificial intelligence applications. It is primarily an acceleration card plugged into a computer or server, containing multiple high-performance processors or chips to improve the speed and efficiency of the computer in processing artificial intelligence tasks. AI accelerator cards can accelerate deep learning applications during training and inference, providing higher computational speed and performance, thereby achieving more accurate predictions and results. For example, in deep learning networks, tasks that AI accelerator cards can perform include: training neural networks, inference based on neural network results, image processing, etc. In short, AI accelerator cards are one of the key components for accelerating deep learning applications, and can accelerate a large number of computationally intensive tasks.

[0047] AI accelerator cards are typically divided into a hardware layer, a device driver layer, a parallel computing library, a model engine layer, and a model application layer. In terms of the calling relationships, from top to bottom, the layers are: the model application layer, which calls the model engine layer; the model engine layer calls the parallel computing library; the parallel computing library calls the device driver layer; and the device driver layer calls the hardware layer. Based on these calling relationships, the AI ​​accelerator card can perform corresponding tasks.

[0048] For AI accelerator cards, the hardware mainly includes a hardware layer, a device driver layer, and a parallel computing library. The model engine layer and model application layer are the software that calls the AI ​​accelerator card to run. The solution provided in this embodiment mainly determines the computing environment compatibility between the AI ​​accelerator card and the software. Furthermore, in order for the AI ​​accelerator card to perform corresponding tasks, the software needs to be compatible with the AI ​​accelerator card, which can be achieved based on the technical solution provided in this embodiment.

[0049] Based on the above description, it can be understood that the solution provided by the embodiments of the present invention is applicable to automatically adapting to target applications corresponding to AI accelerator cards, thereby achieving the effect of automatic adaptation of AI accelerator card computing environment.

[0050] Figure 1 This is a flowchart of a method for determining an AI computing environment provided by an embodiment of the present invention. This embodiment can be applied to automatically determining the target application corresponding to the AI ​​accelerator card, and then installing it in the target hardware device to which the AI ​​accelerator card belongs, so that the target application and the AI ​​accelerator card cooperate to perform target business in the target hardware device. The method for determining the AI ​​computing environment provided in this embodiment can be executed by an AI computing environment determining device, which can be integrated into an electronic device. Optionally, the electronic device can be a mobile terminal or a server terminal.

[0051] like Figure 1 As shown, the method specifically includes the following steps:

[0052] S110. In response to an event that deploys the target application, retrieve the computing environment scheduling parameters associated with the application identifier of the target application.

[0053] The target application can be any application that calls the AI ​​accelerator card. Optionally, the target application can be either a model application layer or a model engine layer. The model application layer can be understood as the task that the model can perform. When processing specific business based on the model application layer, the model engine layer can be called to execute specific content, such as generating text.

[0054] In this embodiment, the deployment of the target application can be triggered based on a scheduled task. Alternatively, the operations and maintenance personnel can determine whether to trigger the deployment of the target application based on actual needs. Optionally, when the operations and maintenance personnel determine that the target application needs to be deployed, they can click the control for deploying the target application in the display interface, and determine the event to trigger the deployment of the target application based on the triggering operation of the control.

[0055] It should be noted that different manufacturers produce AI accelerator cards, resulting in variations in their parameters. For example, the driver version, hardware version, and parallel computing library version may differ within the AI ​​accelerator card. Furthermore, AI accelerator cards with different parameters are suited to different target applications. Therefore, the appropriate AI accelerator card can be determined based on the scheduling parameters of the computing environment corresponding to the target application.

[0056] In this embodiment, the computing environment scheduling parameters are used to characterize the demand data of the AI ​​accelerator card adapted to the target application. That is, the demand data is the accelerator card parameters of the AI ​​accelerator card adapted to the target application. It should be noted that the computing environment scheduling parameters are pre-set based on the accelerator card parameters of the AI ​​accelerator card adapted to the target application. For different target applications, the corresponding computing environment scheduling parameters differ to some extent. To accurately retrieve the computing environment scheduling parameters for a specific target application, the computing environment scheduling parameters can be retrieved based on the application identifier of the target application. This is because the application identifier is unique for each target application, and therefore, it can be used as a credential to determine the computing environment scheduling parameters.

[0057] Specifically, when an event triggering the deployment of a target application is detected, the application identifier of the target application can be obtained first, and then the pre-stored computing environment scheduling parameters corresponding to the target application can be retrieved based on the application identifier of the target application.

[0058] S120. Based on the computing environment scheduling parameters and the received acceleration card data of at least one AI acceleration card, determine at least one target AI acceleration that is compatible with the target application.

[0059] It should be noted that the solution provided in this embodiment of the invention can correspond to a system for defining an AI computing environment. This system can communicate with at least one hardware device, optionally a PC. The target user can insert an AI accelerator card into at least one hardware device according to actual needs.

[0060] The number of AI accelerator cards can be one or more, depending on the number of AI accelerator cards inserted into at least one hardware device. For each AI accelerator card, its accelerator card data can be obtained after insertion into the hardware device. Typically, the accelerator card data includes hardware version, driver version, and parallel computing library version. At least one target AI accelerator card refers to at least one AI accelerator card that can be used with the target application.

[0061] It should be noted that the AI ​​accelerator card data can be reported periodically or in real time to ensure that whether it is a newly inserted AI accelerator card or an AI accelerator card with an upgraded version of the program, it can be dynamically adapted to the target application, thereby achieving the effect of timely adaptation between the target application and the AI ​​accelerator card.

[0062] Specifically, after obtaining the computing environment scheduling parameters corresponding to the target application, it can be determined whether the accelerator card data of at least one AI accelerator card meets the corresponding conditions based on the computing environment scheduling parameters. If so, the AI ​​accelerator card is selected as the target AI accelerator card; otherwise, if the corresponding conditions are not met, the AI ​​accelerator card is not selected as the target AI accelerator card. In this embodiment, determining whether the accelerator card data of the AI ​​accelerator card meets the corresponding preset conditions is mainly based on the demand data in the computing environment scheduling parameters.

[0063] S130. Distribute the target application to the target hardware device corresponding to at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

[0064] Specifically, the hardware device to which the target AI accelerator card belongs is designated as the target hardware device. After identifying the target hardware device—that is, determining which hardware devices can deploy the target application—the target application can be distributed to and installed on the target hardware device. Once the target application is installed, the AI ​​accelerator card can be invoked based on the target application when executing corresponding tasks, thereby improving the efficiency of executing the target task.

[0065] Specifically, after the system identifies the target hardware device, the target application can be distributed to at least one target hardware device associated with a target AI accelerator card. The target application can then be deployed on the target hardware device. When executing the target task, the corresponding AI accelerator card can be invoked based on the target application.

[0066] Based on the above technical solutions, it should also be noted that for each hardware device, the number of AI accelerator cards inserted can be one or more. In the case of inserting only one AI accelerator card, the AI ​​accelerator card can be directly invoked based on the target application. For hardware devices with multiple AI accelerator cards inserted, the hardware architecture parameters of the inserted AI accelerator cards can be the same or different. After determining the target AI accelerator card compatible with the target application and distributing the target application to the target hardware device, the target application can be installed on the target hardware device, and the target application can be associated with the target AI accelerator card. This ensures that when performing tasks based on the target application, the bound target AI accelerator card can be invoked, guaranteeing the compatibility between the target application and the target AI accelerator card.

[0067] Based on the above technical solutions, it should also be noted that when there is a version upgrade or a new AI accelerator card is inserted into the hardware device, since the scanning node can report data in real time or periodically, it is possible to determine whether the previously inserted AI accelerator card and the upgraded AI accelerator card are compatible with the target application. That is, it is possible to quickly determine the application corresponding to the newly inserted AI accelerator card and the upgraded AI accelerator card, and further achieve the effect of automatic matching between the application and the AI ​​accelerator card without special maintenance.

[0068] The technical solution provided by this invention, in response to an event of deploying a target application, can retrieve computing environment scheduling parameters associated with the target application. Since these computing environment scheduling parameters mainly characterize the demand data of the AI ​​accelerator cards adapted to the target application, based on the computing environment scheduling data and the received accelerator card data of the AI ​​accelerator cards, it can be determined which target AI accelerator cards can use the target application. Furthermore, the target application can be distributed to at least one target hardware device corresponding to the target AI accelerator card to deploy the target application on the target hardware device, and then the target AI accelerator cards can be scheduled based on the target application. This solves the problem of low efficiency and poor automation in the prior art, which requires manual determination of which accelerator cards use which applications based on the accelerator card data and application data. It realizes that regardless of whether an AI accelerator card is added to the hardware device or the accelerator card parameters of the AI ​​accelerator card change, the AI ​​accelerator card adapted to the target application can be automatically determined, achieving the effect of adaptive AI accelerator card computing scheduling environment.

[0069] Figure 2 This is a flowchart illustrating a method for determining an AI computing environment according to an embodiment of the present invention. Based on the aforementioned embodiments, the accelerator card data of the AI ​​accelerator card plugged into the hardware device can be acquired and uploaded based on a scheduled task or in real time. This allows for timely and efficient updates of the target application corresponding to the AI ​​accelerator card when the target application is deployed, thereby achieving the effect of compatibility between the AI ​​accelerator card and the target application. For specific implementation details, please refer to the detailed description of this technical solution. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated in this embodiment.

[0070] S210. Based on the scanning node deployed in the hardware device, read the acceleration card data of at least one AI acceleration card plugged into the hardware device.

[0071] It should be noted that the system implementing the technical solution provided in the embodiments of the present invention includes a scanning node. The scanning node can be an XPUAgent (scanning module), which can be deployed in a hardware device. Its main function is to scan the accelerator card data of at least one AI accelerator card plugged into the hardware device.

[0072] It should also be noted that scanning nodes can be installed on any hardware device that can accept AI accelerator cards. When an AI accelerator card is inserted into the hardware device, the accelerator card data can be scanned and read. Even when no AI accelerator card is inserted into the hardware device, the accelerator card data can still be scanned, but the data read will be empty.

[0073] In this embodiment, the scanning node in the system can be pre-deployed into the hardware device. The scanning node can be a scanning software, and its specific content is not limited in this embodiment, as long as it can read the acceleration card data of the AI ​​acceleration card plugged into the hardware device.

[0074] S220. Based on the scanning node, the accelerator card identifier of the at least one AI accelerator card, the device identifier of the hardware device to which it belongs, and the accelerator card data are reported.

[0075] Each AI accelerator card has a unique identifier, which can be used as the accelerator card's identifier. To facilitate subsequent identification of the hardware device to which the AI ​​accelerator card is plugged in, the device identifier of the hardware device to which the AI ​​accelerator card belongs can also be obtained. Similarly, the device identifier is unique for each hardware device; therefore, the hardware device to which the AI ​​accelerator card is plugged in can be identified based on the device identifier.

[0076] In this embodiment, the reason and benefit of determining the accelerator card identifier and device identifier are that, for each AI accelerator card, when determining whether the AI ​​accelerator card is compatible with the target application, the computer can accurately identify which target AI accelerator card is compatible. Furthermore, after determining the target AI accelerator card, in order to accurately deploy it to the hardware device, it is necessary to combine the device identifier associated with the AI ​​accelerator card to determine its corresponding target hardware device, thereby further ensuring the correct installation of the target application. Finally, after the target application is installed on the target hardware device, the target accelerator card compatible with the target application can be determined based on the accelerator card identifier, further ensuring the compatibility effect between the AI ​​accelerator card and the target application.

[0077] The technical solution provided by this invention can read accelerator card data of at least one AI accelerator card plugged into a hardware device based on a scanning node deployed in the hardware device. Furthermore, based on the accelerator card identifier, the device identifier of the associated hardware device, and the accelerator card data reported by the scanning node, when responding to an event of deploying a target application, the system can determine the target AI accelerator card compatible with the target application based on the received accelerator card data of at least one accelerator card and the computing environment scheduling parameters of the target application, achieving the technical effect of automatically matching the AI ​​accelerator card's computing environment.

[0078] Figure 3 This is a flowchart illustrating a method for determining an AI computing environment according to an embodiment of the present invention. Based on the aforementioned embodiments, the backend can determine the target application to be deployed according to actual needs and create an image corresponding to the target application. In response to an event of deploying the target application, the computing environment scheduling parameters can be obtained from the image. For specific implementation details, please refer to the detailed description of this technical solution. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated in this embodiment.

[0079] S310. In response to the event of creating an image corresponding to the target application, obtain the definition specification of the target application.

[0080] It should be noted that operations and maintenance personnel can create images according to actual needs, and schedule the computing environment corresponding to the target application based on the image.

[0081] During the development phase, after the target application is completed, application specifications can be set for it. These specifications can include the required parameters for the target application, or, in other words, the application parameters related to the version data of the accelerator card to which the target application is compatible. The application specifications can be used as the definition guidelines for the target application.

[0082] This can be understood as follows: after determining the target application, the pre-determined application specifications corresponding to the target application can be retrieved and used as the definition specifications.

[0083] S320. Based on the defined specifications, add the target metadata corresponding to the target application to the image and upload the image to the image repository. In response to the event of deploying the target application, retrieve the target metadata of the target application from the image repository and use it as the computing environment scheduling parameter.

[0084] The definition specification can include requirements for accelerator card manufacturers, accelerator card components, driver versions, parallel computing library versions, and so on. Corresponding target metadata can be added based on these requirements; that is, the target metadata is primarily added based on the requirements in the definition specification. After adding the target definition specification to the image, the image can be uploaded to an image repository.

[0085] In this embodiment, the target metadata corresponding to the target application can be determined based on the definition specifications of the target application. The target metadata is then added to the image. Simultaneously, to facilitate the subsequent determination of the AI ​​accelerator card compatible with the target application, the image corresponding to the target application can be uploaded to an image repository. This allows the image repository to retrieve the target metadata corresponding to the target application from the image when responding to an event that triggers the deployment of the target application, and to use the target metadata as a computing environment scheduling parameter.

[0086] Based on the above technical solution, to facilitate the subsequent retrieval of target metadata corresponding to the target application's image from the image repository, a storage address can be determined after the image is uploaded to the image repository. This storage address is then associated with the target application, so that in response to events from the target application, the target metadata in the image can be retrieved from the image repository based on the storage address, thereby obtaining the computing environment scheduling parameters.

[0087] S330. In response to the event of deploying the target application, determine the storage address of the image corresponding to the target application in the image repository based on the application identifier of the target application.

[0088] The application identifier is a unique identifier for the target application. For a system, there are many applications to be used. To distinguish each application, the application identifier for each application can be recorded separately, and the application to be deployed can be used as the target application. Furthermore, this application identifier primarily serves as the identifier for the target application.

[0089] Specifically, in response to an event indicating that a target application has been deployed, the application identifier of the target application can be obtained. Based on the application identifier, the storage address associated with the target application can be determined. This storage address primarily represents the storage address of the image corresponding to the target application within the image repository.

[0090] S340. Retrieve the target metadata corresponding to the target application from the image repository based on the storage address, and use the target metadata as the computing environment scheduling parameter.

[0091] Specifically, after obtaining the storage address, the target metadata corresponding to the target application can be retrieved from the image repository based on that storage address. The retrieved target metadata can then be used as scheduling parameters for the computing environment.

[0092] S350. Based on the hardware architecture version requirement parameters and parallel computing library version requirement parameters in the computing environment scheduling parameters, as well as the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card in the accelerator card data, determine at least one target AI accelerator card that meets the computing environment scheduling parameters.

[0093] The hardware architecture version requirement parameters can include hardware layer version requirement data and driver version requirement data. The parallel computing library version requirement parameters mainly refer to which versions of parallel computing libraries the AI ​​accelerator card can call. Version requirement data mainly refers to which software versions the target application is compatible with. For example, if the driver version requirement data specifies version number 2.0 or higher, then any driver version 2.0 or higher is compatible with the target application. Accelerator card data includes the accelerator card model, driver version, and parallel computing library version information.

[0094] It should be noted that the accelerator card data mainly refers to the actual version data of the AI ​​accelerator card currently plugged into the hardware device. Therefore, based on the accelerator card data and computing environment scheduling parameters, it is possible to clearly determine which target applications can call which AI accelerator cards.

[0095] Specifically, the target AI accelerator card that meets the computing environment scheduling parameters can be determined based on the driver version requirement parameters, hardware layer version parameters, and parallel computing library version requirement parameters in the computing environment scheduling parameters, as well as the accelerator card model, driver version, and parallel computing library version information corresponding to each received AI accelerator card. In this embodiment, when comparing the above parameters, data for the same indicator can be compared, and if the data for all indicators meets the conditions, the AI ​​accelerator card is selected as the target AI accelerator card.

[0096] S360: Determine the target hardware device to which the target application will be deployed, based on the device identifier of the hardware device to which at least one target AI accelerator card belongs.

[0097] It should be noted that after identifying the target AI accelerator card, that is, after identifying the AI ​​accelerator card that the target application can call, the target application can be deployed on the hardware device to which the target AI accelerator card belongs.

[0098] In this embodiment, the advantage of using the above method to determine the target AI accelerator card corresponding to the target application is that, regardless of whether a new AI accelerator card is added or a hardware upgrade is performed, the accelerator card data can be reported by the scanning node. Furthermore, based on the target application's computing environment scheduling parameters and accelerator card data, the target AI accelerator card that the target application can schedule can be determined, achieving an automatic matching effect. That is, it reduces the problem of manual matching by users in data analysis, which leads to incompatibility between the target application and the AI ​​accelerator card, preventing the effective use of the AI ​​accelerator card.

[0099] S370: Send the target application and the target accelerator card identifier of the target AI accelerator card associated with the target application to the target hardware device, so as to determine that the target application can call the target AI accelerator card based on the target accelerator card identifier.

[0100] Among them, the accelerator card identifier corresponding to the target AI accelerator card is recorded as the target accelerator card identifier.

[0101] Specifically, to facilitate the target application in the target hardware device in identifying the target AI accelerator card it can call, the target application and the target accelerator card identifier associated with the target application can be sent to the target hardware device. After receiving the target application, the target hardware device can deploy the target application. Simultaneously, when calling an AI accelerator card based on the target application, the target accelerator card identifier can be used to determine and invoke the appropriate AI accelerator card.

[0102] The advantage of this approach is that when executing tasks based on a target application, it can accurately call the target AI accelerator card associated with the target application, thereby improving the accuracy and efficiency of task execution. Simultaneously, this approach effectively avoids the problem of not being able to accurately determine the compatible AI accelerator card for each application when multiple AI accelerator cards are plugged into a single hardware device. That is, even when the number of AI accelerator cards plugged into the hardware device is multiple, and the parameters of these AI accelerator cards differ, it can automatically determine the target software corresponding to each AI accelerator card, achieving automatic matching and updating.

[0103] The technical solution provided by this invention, in response to an event of deploying a target application, can retrieve computing environment scheduling parameters associated with the target application. Since these computing environment scheduling parameters mainly characterize the demand data of the AI ​​accelerator cards adapted to the target application, based on the computing environment scheduling data and the received accelerator card data of the AI ​​accelerator cards, it can be determined which target AI accelerator cards can use the target application. Furthermore, the target application can be distributed to at least one target hardware device corresponding to the target AI accelerator card to deploy the target application on the target hardware device, and then the target AI accelerator cards can be scheduled based on the target application. This solves the problem of low efficiency and poor automation in the prior art, which requires manual determination of which accelerator cards use which applications based on the accelerator card data and application data. It realizes that regardless of whether an AI accelerator card is added to the hardware device or the accelerator card parameters of the AI ​​accelerator card change, the AI ​​accelerator card adapted to the target application can be automatically determined, achieving the effect of adaptive AI accelerator card computing scheduling environment.

[0104] As an optional embodiment of the above embodiments, Figure 4 This is a schematic diagram of a method for determining an AI computing environment provided by an embodiment of the present invention. Based on the foregoing embodiments, the technical solution can be further described. In this embodiment, the same or corresponding technical terms are not repeated.

[0105] It should be noted that the solution provided in the embodiments of the present invention can be integrated into a system, which may include XPUAgent (scanning module), XPU controller, and XPU controller. XPUAgent is deployed in various hardware devices, while XPU controller and XPU controller are deployed in the control devices corresponding to the maintenance personnel.

[0106] See Figure 4 Based on the XPUAgent (scanning module) deployed in the hardware device (hardware node), it can scan the accelerator card data of at least one AI accelerator card inserted in the hardware device, corresponding to... Figure 4 The scanning nodes contain information about the AI ​​accelerator card model, driver version, and parallel computing library version. This accelerator card data can include the manufacturer, model, driver version, parallel computing library version, and interconnection topology information for each accelerator card. Accelerator card data from at least one AI accelerator card obtained through XPUAgen scanning can be reported to the central controller (XPU controller), corresponding to... Figure 4 The system reports information on the accelerator cards, their number, driver versions, and parallel computing library versions for each node.

[0107] When creating a container image, target metadata can be added according to the definition specifications of the application to be deployed. Target metadata can include hardware architecture version and version requirements for parallel computing libraries. The hardware architecture version can include the hardware layer version and driver version. After the image is created, it can be stored in an image repository, and the corresponding storage address can be provided. This storage address is associated with the application to be deployed. Once the target application is determined, the storage address of the image corresponding to the target application can be determined based on the application identifier of the target application. Then, based on the storage address, the target metadata in the image can be retrieved, thereby obtaining the computing environment scheduling data of the target application.

[0108] When an event satisfying the deployment target application is detected, i.e., during container scheduling, the scheduler can invoke the container orchestration service to determine the container to be scheduled, i.e., the image to be scheduled. After determining the image, the storage address corresponding to the target application (image) can be obtained, and then the target metadata in the image can be retrieved from the image repository based on the storage address. The obtained target metadata is used as the computing environment scheduling parameters. After obtaining the computing environment scheduling parameters, they can be sent to the XPU controller based on the scheduler. The XPU controller can then match the computing resource scheduling parameters with the accelerator card data of at least one AI accelerator card. Since the computing resource scheduling parameters include the required driver version and parallel computing library version data, it can be determined whether the accelerator card data of at least one AI accelerator card meets the computing resource scheduling parameters. This mainly involves matching data for the same evaluation metric, and selecting the successfully matched AI accelerator card as the target AI accelerator card.

[0109] The data reported by XPUAgent can also include the accelerator card identifier and the device identifier of the hardware device to which the AI ​​accelerator card belongs. Therefore, after identifying the target AI accelerator card, the target hardware device to which the target application should be deployed can be determined and deployed. After the target application is deployed to the target hardware device, it can be deployed on the target hardware device so that when the target application performs related tasks, the corresponding AI accelerator card can be retrieved based on the accelerator card identifier associated with the target application.

[0110] It should be noted that the above description is based solely on the order of presentation and does not limit the order of execution.

[0111] The scheduling method that automatically matches images with AI computing environments eliminates the need to specify node groups or other information during the scheduling phase. The scheduling service simply needs to automatically match and deploy suitable target hardware devices based on the computing environment information collected from each hardware device. Furthermore, by combining the accelerator card identifier associated with the target application, the AI ​​accelerator card used by the target application can be determined, achieving highly efficient task execution.

[0112] The technical solution provided by this invention, in response to an event of deploying a target application, can retrieve computing environment scheduling parameters associated with the target application. Since these computing environment scheduling parameters mainly characterize the demand data of the AI ​​accelerator cards adapted to the target application, based on the computing environment scheduling data and the received accelerator card data of the AI ​​accelerator cards, it can be determined which target AI accelerator cards can use the target application. Furthermore, the target application can be distributed to at least one target hardware device corresponding to the target AI accelerator card to deploy the target application on the target hardware device, and then the target AI accelerator cards can be scheduled based on the target application. This solves the problem of low efficiency and poor automation in the prior art, which requires manual determination of which accelerator cards use which applications based on the accelerator card data and application data. It realizes that regardless of whether an AI accelerator card is added to the hardware device or the accelerator card parameters of the AI ​​accelerator card change, the AI ​​accelerator card adapted to the target application can be automatically determined, achieving the effect of adaptive AI accelerator card computing scheduling environment.

[0113] The following are embodiments of the AI ​​computing environment determination device provided in this invention. This device and the AI ​​computing environment determination method in the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the AI ​​computing environment determination device can be found in the embodiments of the AI ​​computing environment determination method described above.

[0114] Figure 5 This is a schematic diagram of the structure of an AI computing environment determination device provided in an embodiment of the present invention. The device specifically includes: a computing environment scheduling parameter retrieval module 410, a target AI accelerator card determination module 420, and a target application deployment module 430.

[0115] The computing environment scheduling parameter retrieval module is used to retrieve computing environment scheduling parameters associated with the application identifier of the target application in response to an event of deploying the target application. The computing environment scheduling parameters are used to characterize the demand data of the AI ​​acceleration card adapted to the target application.

[0116] The target AI accelerator card determination module is used to determine at least one target AI accelerator card that is compatible with the target application based on the computing environment scheduling parameters and the accelerator card data of at least one AI accelerator card received.

[0117] The target application deployment module is used to distribute the target application to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

[0118] The technical solution provided by this invention, in response to an event of deploying a target application, can retrieve computing environment scheduling parameters associated with the target application. Since these computing environment scheduling parameters mainly characterize the demand data of the AI ​​accelerator cards adapted to the target application, based on the computing environment scheduling data and the received accelerator card data of the AI ​​accelerator cards, it can be determined which target AI accelerator cards can use the target application. Furthermore, the target application can be distributed to at least one target hardware device corresponding to the target AI accelerator card to deploy the target application on the target hardware device, and then the target AI accelerator cards can be scheduled based on the target application. This solves the problem of low efficiency and poor automation in the prior art, which requires manual determination of which accelerator cards use which applications based on the accelerator card data and application data. It realizes that regardless of whether an AI accelerator card is added to the hardware device or the accelerator card parameters of the AI ​​accelerator card change, the AI ​​accelerator card adapted to the target application can be automatically determined, achieving the effect of adaptive AI accelerator card computing scheduling environment.

[0119] Based on the above technical solutions, the device further includes:

[0120] An accelerator card data reading module is used to read accelerator card data of at least one AI accelerator card plugged into the hardware device based on a scanning node deployed in the hardware device.

[0121] The accelerator card data reporting module is used to report the accelerator card identifier, the device identifier of the hardware device to which the at least one AI accelerator card belongs, and the accelerator card data based on the scanning node.

[0122] Based on the above technical solutions, the device further includes:

[0123] The specification acquisition module is used to acquire the definition specification of the target application in response to an event that creates an image corresponding to the target application;

[0124] The image data upload module is used to add the target metadata corresponding to the target application to the image based on the defined specifications, and upload the image to the image repository. In response to the event of deploying the target application, the module retrieves the target metadata of the target application from the image repository and uses it as the scheduling parameters of the computing environment.

[0125] Based on the above technical solutions, the computing environment scheduling parameter retrieval module in the device includes:

[0126] The storage address determination unit is used to determine the storage address of the image corresponding to the target application in the image repository based on the application identifier of the target application;

[0127] The computing environment scheduling parameter determination unit is used to retrieve the target metadata corresponding to the target application from the image repository based on the storage address, and use the target metadata as the computing environment scheduling parameter.

[0128] Based on the above technical solutions, the computing environment scheduling parameters include at least hardware architecture version requirements and parallel computing library version requirements, and the accelerator card data includes at least the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card.

[0129] Based on the above technical solutions, the target AI acceleration card determination module is also used for:

[0130] Based on the hardware architecture version requirement parameters and parallel computing library version requirement parameters in the computing environment scheduling parameters, as well as the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card in the accelerator card data, at least one target AI accelerator card that meets the computing environment scheduling parameters is determined.

[0131] Based on the above technical solutions, the target application deployment module includes:

[0132] The application delivery unit is used to determine the target hardware device to which the target application is to be deployed, based on the device identifier of the hardware device to which the at least one target AI accelerator card belongs.

[0133] The application deployment unit is used to send the target application and the target accelerator card identifier of the target AI accelerator card associated with the target application to the target hardware device, so as to determine the target AI accelerator card that the target application can call based on the target accelerator card identifier.

[0134] The AI ​​computing environment determination device provided in this embodiment of the invention can execute the AI ​​computing environment determination method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the AI ​​computing environment determination method.

[0135] It is worth noting that in the embodiments of the AI ​​computing environment determination device described above, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0136] Figure 6 This is a schematic diagram of a server structure provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary server 12 suitable for implementing embodiments of the present invention is shown. Figure 6The server 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0137] like Figure 6 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0138] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0139] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0140] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0141] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0142] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0143] The processing unit 16 executes various functional applications and determines the AI ​​computing environment by running programs stored in the system memory 28. For example, it implements the steps of the method for determining the AI ​​computing environment provided in Embodiment 1 of this invention, which includes:

[0144] In response to an event that a target application is deployed, computing environment scheduling parameters associated with the application identifier of the target application are retrieved, wherein the computing environment scheduling parameters are used to characterize the AI ​​accelerator card requirements of the target application.

[0145] Based on the computing environment scheduling parameters and the received acceleration card data of at least one AI acceleration card, determine at least one target AI acceleration card that is compatible with the target application.

[0146] The target application is distributed to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

[0147] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the method for determining the AI ​​computing environment provided in any embodiment of the present invention.

[0148] This embodiment provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the program implements the steps of the method for determining an AI computing environment as provided in the foregoing embodiments of the present invention. The method includes:

[0149] In response to an event that a target application is deployed, computing environment scheduling parameters associated with the application identifier of the target application are retrieved, wherein the computing environment scheduling parameters are used to characterize the AI ​​accelerator card requirements of the target application.

[0150] Based on the computing environment scheduling parameters and the received acceleration card data of at least one AI acceleration card, determine at least one target AI acceleration card that is compatible with the target application.

[0151] The target application is distributed to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

[0152] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0153] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0154] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0155] Computer program code for performing the operations of embodiments of the present invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0156] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for determining an AI computing environment, characterized in that, include: In response to an event that a target application is deployed, computing environment scheduling parameters associated with the application identifier of the target application are retrieved, wherein the computing environment scheduling parameters are used to characterize the AI ​​accelerator card requirements of the target application. Based on the computing environment scheduling parameters and the received acceleration card data of at least one AI acceleration card, determine at least one target AI acceleration card that is compatible with the target application. The target application is distributed to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

2. The method according to claim 1, characterized in that, The method further includes: Based on the scanning nodes deployed in the hardware device, read the acceleration card data of at least one AI acceleration card plugged into the hardware device; The scanning node reports the accelerator card identifier, the device identifier of the hardware device to which the at least one AI accelerator card belongs, and the accelerator card data.

3. The method according to claim 1, characterized in that, Prior to the event of deploying the target application, the method further includes: In response to an event that creates an image corresponding to the target application, the definition specifications of the target application are obtained; Based on the defined specifications, the target metadata of the target application is determined, and the target metadata is added to the image; The image is uploaded to the image repository in response to the event of deploying the target application. The target metadata of the target application is retrieved from the image repository and used as the scheduling parameters of the computing environment.

4. The method according to claim 3, characterized in that, The retrieval of computing environment scheduling parameters associated with the application identifier of the target application includes: Based on the application identifier of the target application, obtain the storage address of the image corresponding to the target application in the image repository; Based on the storage address, the target metadata corresponding to the target application is retrieved from the image repository, and the target metadata is used as the computing environment scheduling parameter.

5. The method according to claim 1, characterized in that, The computing environment scheduling parameters include at least hardware architecture version requirements and parallel computing library version requirements, and the accelerator card data includes at least the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card.

6. The method according to claim 5, characterized in that, The step of determining at least one target AI accelerator card suitable for the target application based on the computing environment scheduling parameters and the received accelerator card data of at least one AI accelerator card includes: Based on the hardware architecture version requirement parameters and parallel computing library version requirement parameters in the computing environment scheduling parameters, as well as the accelerator card model, driver version, and parallel computing library version information of the AI ​​accelerator card in the accelerator card data, at least one target AI accelerator card that meets the computing environment scheduling parameters is determined.

7. The method according to claim 1, characterized in that, The step of distributing the target application to the target hardware device corresponding to the at least one target AI accelerator card includes: Based on the device identifier of the hardware device to which the at least one target AI accelerator card belongs, the target hardware device to which the target application is to be deployed is determined; The target application and the target AI accelerator card identifier associated with the target application are sent to the target hardware device to determine the target AI accelerator card that the target application can call based on the target accelerator card identifier.

8. A device for determining an AI computing environment, characterized in that, include: The computing environment scheduling parameter retrieval module is used to retrieve computing environment scheduling parameters associated with the application identifier of the target application in response to an event of deploying the target application. The computing environment scheduling parameters are used to characterize the demand data of the AI ​​acceleration card adapted to the target application. The target AI accelerator card determination module is used to determine at least one target AI accelerator card that is compatible with the target application based on the computing environment scheduling parameters and the accelerator card data of at least one AI accelerator card received. The target application deployment module is used to distribute the target application to the target hardware device corresponding to the at least one target AI accelerator card, so as to deploy the target application on the target hardware device and schedule the target AI accelerator card based on the target application.

9. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining the AI ​​computing environment as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for determining the AI ​​computing environment as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for determining the AI ​​computing environment as described in any one of claims 1-7.

Images