A zero-invasive computing power acceleration card information self-defining system and method

Abstract

The application provides a zero-invasion computing power acceleration card information customization system and method, and belongs to the technical field of container cloud platforms. Information customization components are containerized and deployed on each node of the container cloud platform, and the information customization components actively listen to the creation or startup event of a business container. After listening to the event, a custom interception dynamic library, a dynamic library sequence control file, and a custom hardware information database are injected into the business container using an NRI mechanism. The technical solution combines the NRI mechanism with the dynamic link library loading mechanism, the custom hardware information database, and other technologies, so that different computing power acceleration card information acquisition technology paths with different sources and levels can obtain preset hidden information as a return result. Moreover, the solution can access the container cloud platform computing power acceleration card hidden or custom capabilities without modifying user business code or modifying the business container image, and has a wide range of application scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of container cloud platform, and particularly relates to a zero-intrusion computing power acceleration card information customization system and method. BACKGROUND

[0002] Computing power acceleration card refers to a hardware expansion card specially used for accelerating computing tasks, which is usually inserted into the mainboard of a server or workstation to improve processing speed, computing efficiency or energy consumption ratio. Typical computing power acceleration cards include graphics processing units (GPUs), neural network processors (NPUs) and tensor processing units (TPUs), which are used in large-scale neural network training and inference businesses with the development of AI technology.

[0003] Computing power GPU cloudization refers to providing computing power of physical GPU acceleration cards to users through cloud services, so that users can flexibly rent and use GPU resources on demand like using cloud hosts (such as ECS), without having to own and manage physical GPU hardware. In the computing power GPU cloudization scenario, the computing power provider does not want the end users of the computing power cloud to be able to view the specific computing power acceleration card information through programs or commands, so it is very practical to “hide” (i.e., customize) the GPU acceleration card information.

[0004] In some prior art, GPU virtualization technology (such as NVIDIA MIG, vGPU) is used to realize virtualization and hiding of GPU computing power usage and allocation by redirecting the CUDA dynamic link library. However, on the one hand, the above technology relies on virtualization technology to realize, and in practice, not all computing power acceleration cards run in a virtualization environment. On the other hand, for container cloud platform (such as Kubernetes) vendors, only providing virtualization and hiding of computing power cannot meet the current complex and changing business needs. For example, in some special business scenarios, it is necessary to customize and hide detailed hardware information of computing power acceleration cards such as specific models, architectures and driver versions. In addition, the implementation of the above technology requires a large amount of configuration information to be manually set in the workloads of the container cloud platform, which forms a large amount of intrusion to user business code and business containers, and is not conducive to the promotion and use of the container cloud platform. Therefore, a technical solution for customizing computing power acceleration card information with wide application range and zero intrusion to user business code or business containers is needed. SUMMARY

[0005] The present application aims to provide a zero-intrusion computing power acceleration card information customization system and method to solve or alleviate the problems in the prior art.

[0006] To achieve the above object, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides an information customization component, which is containerized and deployed on each node of a container cloud platform. The information customization component, in response to listening to a creation or startup event of a service container, uses a container runtime extension interface (NRI) mechanism to inject a customized interception dynamic library, a dynamic library sequence control file and a customized hardware information database into the service container, so that the service container can return preset anonymous information when receiving a query instruction of computing power acceleration card information.

[0008] The dynamic library sequence control file is used to use a dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the service container loads an original computing power acceleration card interface library.

[0009] The customized interception dynamic library is used to intercept a query instruction sent to the original computing power acceleration card interface library and return preset anonymous information as a query result.

[0010] The customized hardware information database is used to store a preset hardware information mapping rule, which is used to map original computing power acceleration card information into preset anonymous information.

[0011] In combination with the first aspect, in some possible implementation manners, after listening to the creation or startup event of the service container, the information customization component is further used to check whether a configuration of the service container meets a preset filtering condition. If yes, the operation of injecting the customized interception dynamic library, the dynamic library sequence control file and the customized hardware information database into the service container is performed. If the preset filtering condition is not met, the above injection operation is not performed.

[0012] In a second aspect, the present application provides a service container. When the service container is created or started, an information customization component according to any one of claims 1 to 2 uses a container runtime extension interface (NRI) plug-in to inject a customized interception dynamic library, a customized hardware information database and a dynamic library sequence control file into the service container. The dynamic library sequence control file is used to use a dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the service container loads an original computing power acceleration card interface library.

[0013] In response to the service container receiving a query instruction for the computing power acceleration card information, the service container returns preset obfuscated information according to the type of the query instruction, or reads preset obfuscated information in the custom hardware information database and returns the preset obfuscated information as a query result.

[0014] In a third aspect, the embodiment provides a zero-invasive computing power acceleration card information customization method, which is applied to a container cloud platform. The container cloud platform includes the information customization component in any of the above embodiments and the service container provided in the second aspect. The zero-invasive computing power acceleration card information customization method includes the following steps:

[0015] In response to listening to a creation or startup event of the service container, the information customization component injects a custom interception dynamic library, a dynamic library sequence control file and a custom hardware information database into the service container by using a container runtime extension interface (NRI) mechanism.

[0016] In response to receiving a query instruction for the computing power acceleration card information, the service container returns preset obfuscated information.

[0017] The dynamic library sequence control file is used to load the custom interception dynamic library in priority to an original computing power acceleration card interface library by using a dynamic link library loading mechanism before the service container loads the original computing power acceleration card interface library.

[0018] The custom interception dynamic library is used to intercept a query instruction in the original computing power acceleration card interface library and return preset obfuscated information as a query result.

[0019] The custom hardware information database is used to store a mapping relationship between the computing power acceleration card and the preset obfuscated information to cover an original hardware information database file.

[0020] In a fourth aspect, the embodiment provides a zero-invasive computing power acceleration card information customization system. The system runs on a container cloud platform. The information customization component in any of the above embodiments and the service container provided in the second aspect are configured to, in response to listening to a creation or startup event of the service container, inject a custom interception dynamic library, a dynamic library sequence control file and a custom hardware information database into the service container by using a container runtime extension interface (NRI) mechanism.

[0021] The service container is configured to, in response to receiving a query instruction for the computing power acceleration card information, return preset obfuscated information.

[0022] The dynamic library sequence control file is used to load the custom interception dynamic library in priority to the original computing power acceleration card interface library by using a dynamic link library loading mechanism before the business container loads the original computing power acceleration card interface library.

[0023] The custom interception dynamic library is used to intercept a query instruction in the original computing power acceleration card interface library and return preset obfuscated information as a query result.

[0024] The custom hardware information database is used to store a mapping relationship between the computing power acceleration card and the preset obfuscated information to cover an original hardware information database file.

[0025] Beneficial effects:

[0026] The technical scheme of the embodiment of the application containerizes deployment of an information customization component on each node of a container cloud platform, the information customization component actively listens to a creation or start event of a business container, and after listening to the event, uses a container runtime extension interface NRI mechanism to inject a custom interception dynamic library, a dynamic library sequence control file, and a custom hardware information database into the business container. The technical scheme uses the Container Runtime NRI mechanism and the organic combination of a dynamic link library loading mechanism (such as an LD_PRELOAD environment variable) and a custom hardware information database to enable different computing power acceleration card information acquisition technical paths with different sources and levels to obtain preset obfuscated information as a return result. Moreover, the scheme can access the obfuscation or customization capability of the computing power acceleration card of the container cloud platform without modifying user business code, modifying a user image, or changing a standard running environment, and the user has zero perception, which is conducive to popularization and use and can be applied to more application scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 FIG. 1 is a schematic diagram of a hardware information query instruction calling flow of an NVIDIA computing power acceleration card, in which (a) is a calling flow of a query instruction in an original computing power acceleration card interface library, and (b) is a calling flow of the query instruction of the original computing power acceleration card interface library intercepted by using a custom interception dynamic library.

[0028] Figure 2 FIG. 2 is a schematic diagram of a flow of returning custom GPU hardware information for an lspci command.

[0029] Figure 3 FIG. 3 is a schematic diagram of a working mechanism of an information customization component.

[0030] Figure 4 FIG. 4 is a logic block diagram of implementation of a file injection of an information customization component. DETAILED DESCRIPTION

[0031] The application scenario of the present embodiment is briefly described as follows.

[0032] In the present embodiment, the container cloud platform refers to a cloud computing platform constructed based on container technology, which provides unified scheduling, deployment, running, scaling, monitoring and management capabilities for containerized applications. It is a combination of "container technology + cloud platform architecture", and the core goal is to achieve resource elasticity, high availability and automated operation. Typical container cloud platforms include platforms built by Kubernetes (k8s for short) systems.

[0033] Components refer to software modules responsible for completing different functions in the container cloud platform. Components in k8s include native components and custom components. Native components refer to components provided by the k8s system itself for maintaining basic functions of container orchestration. Native components include Control Plane Components and Node Components. Control Plane Components run on the control node (Master Node) and are used for global scheduling, state management and configuration control of the cluster. Node Components run on each worker node (worker node) and are responsible for local running and resource management of containers on the node.

[0034] In the computing power GPU cloud scene, there are many cases that require the anonymity and customization of computing power acceleration card (such as GPU) information. For example, before the formal release of the front-end chip, it needs to be tested and put into production, or in a specific business scenario, due to the need for privacy protection and data desensitization, the end user does not want to be able to view the specific model information of the underlying computing power acceleration card. For example, in the teaching scenario, different computing power acceleration cards need to be displayed as general hardware information (such as model); when developing architecture-independent or cross-architecture GPU applications, compatibility development and hardware-independent testing are required, and at this time, the hardware information of the computing power acceleration card needs to be customized to simulate the response of different models of GPU to verify the compatibility of the code on unknown hardware devices. In some heterogeneous GPU pooling scheduling and unified abstraction scenarios, especially in the model inference scenario, due to the technical wave brought by the inference cost reduction of DeepSeek, more types of computing power acceleration cards can be more fully utilized. The end user deploys a large model and selects the appropriate model and specification from a large number of computing power acceleration cards, which is a great challenge. Through the information anonymity / customization of computing power acceleration cards, computing power acceleration cards can be abstracted by grade, significantly reducing the difficulty of use for users: for example, NvidiaH20 can run most 32BFP16 precision models based on the Transformers architecture, so NvidiaH20 acceleration cards can be abstracted as LLM-32B models, making it easier for users to choose according to the model size, and also more convenient for cloud platforms to dynamically allocate computing power resources.

[0035] Specifically, the main scenarios of computing power acceleration cards (GPUs) that can be perceived by end users include the following categories:

[0036] 1. Users query the hardware information of computing power acceleration cards through Python code: functions such as torch.cuda.get_device_name() and torch.cuda.get_device_properties();

[0037] 2. Users query the hardware information of computing power acceleration cards through CUDA C++ programming, such as the cudaGetDeviceProperties() function;

[0038] 3. Users query the hardware information of computing power acceleration cards through the pynvml.nvmlDeviceGetName() function of the NVML library;

[0039] 4. Users view card models through command lines such as nvdia-smi (call NVML library path);

[0040] 5. The user views the model of the PCI device through the Linux command lspci -nn.

[0041] In the above application scenarios, the hardware information of the computing power acceleration card perceived by the end user needs to be rewritten. Among them, the pooling scheduling of heterogeneous GPUs and the unified abstraction scenario are closely related to virtualization technology, and most of the other scenarios are not bound to virtualization technology. Therefore, the existing custom processing of computing power usage and allocation through GPU virtualization technology is not sufficient to meet the needs of diversified and complex business scenarios. There is currently no open-source or open technology that can achieve custom processing of information and models of computing power cards in addition to computing power allocation. This is two different technical implementation paths.

[0042] Therefore, the present embodiment provides a zero-intrusion computing power acceleration card information customization system and method. The technical solution containerizes the deployment of information customization components on each node of the container cloud platform, listens to specific events of the business container, and combines the NRI mechanism to inject the self-developed interception dynamic library and the corresponding control loading logic and related data into the specified location of the business container. The interception dynamic library can intercept the query instructions sent to the original computing power acceleration card interface library by rewriting the query interface in the original computing power acceleration card interface library, and return the preset concealed information as the query result. The customized hardware information database can return the preset concealed information when receiving the query instruction of the operating system to the hardware. Therefore, when the injected content is executed in the business container, the business container can return the preset concealed information as the result to the query instruction of different technical paths, different sources, and different levels (such as python code, C++ code, nvidia-smi or lspci command). Moreover, the process is completely transparent to the development of the business container and the packaging of the business container image, and truly realizes zero intrusion.

[0043] The terms "first", "second", "third", and "fourth" and the like in the specification and claims of the present application and the drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or device.

[0044] Reference to“an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment. The appearances of the phrase“in one embodiment” in various places in the specification are not necessarily referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combined with other embodiments.

[0045] “Multiple” refers to two or more. “And / or”, describing the association relationship of the associated objects, means that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist together, and B exists alone. The character“ / ” generally represents that the associated objects before and after are in an“or” relationship.

[0046] The embodiments of the present application are described below with reference to the accompanying drawings.

[0047] The embodiment provides an information customization component, the information customization component is containerized and deployed on each node of a container cloud platform, and the information customization component injects a customized interception dynamic library, a dynamic library sequence control file and a customized hardware information database into a business container using a container runtime extension interface NRI (Node Resource Interface) plug-in in response to listening to a creation or startup event of the business container, so that the business container can return preset anonymous information when receiving a query instruction of computing power acceleration card information; wherein the dynamic library sequence control file is used to use a dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the business container loads an original computing power acceleration card interface library; the customized interception dynamic library is used to intercept a query instruction in the original computing power acceleration card interface library and return preset anonymous information as a query result; and the customized hardware information database is used to store a preset hardware information mapping rule, and the hardware information mapping rule is used to map the original computing power acceleration card information to the preset anonymous information.

[0048] In the embodiment, the information customization component is a new component developed on the basis of a native component provided by a container cloud platform (such as Kubernetes, referred to as k8s), and the component is used to inject related technical logic and files and data into a business container in a user zero-aware manner, so as to realize the anonymization and customized return of hardware information of a GPU / NPU computing power acceleration card.

[0049] In Kubernetes, a Pod is the smallest unit to deploy an application, which encapsulates one or more tightly related containers (usually 1, but also multiple), and is accompanied by shared storage, network, and running configuration, so in some technical literature, Pod is also called container group. In the standard Pod definition, a Pod includes containers, network stacks, volumes, and configuration metadata (i.e. Pod Spec, YAML format file).

[0050] Business containers refer to containers that actually run the core application logic in the Pod. The corresponding concept is system / auxiliary containers, such as sidecar containers, initialization containers, monitoring or agent containers, etc. That is, business containers are the actual containers that bear user function services in the container cloud, such as running web services, AI reasoning, databases, model training, etc.

[0051] Since only the final user needs to be hidden in the application scenario of the information customization of the computing power acceleration card, and the container cloud platform system itself does not need to be disguised and hidden, considering that system / auxiliary containers are usually used to run core services of the cloud platform (such as kubelet, etcd of Kubernetes, etc.), their functions are irrelevant to the hardware information of the computing power acceleration card, therefore, the present embodiment distinguishes the types of containers, and the target object of the information customization component is limited to business containers, and system / auxiliary containers are not monitored and processed. This is conducive to reducing unnecessary resource consumption, while enhancing the controllability of dynamic library injection.

[0052] In this embodiment, the information customization component is used to listen to the creation or start event of the service container. Specifically, the information customization component is a self-developed and customized NRI plug-in. The plug-in implements the NRI plug-in interface corresponding to the above-mentioned CreateContainer and StartContainer events by writing corresponding codes, subscribes to the CreateContainer and StartContainer events in the life cycle events of the service container, and then uses the NRI client (such as NRI adaptation) to interface with the container runtime (such as containerd or CRI-O). Before the service container is started, the customized interception dynamic library, dynamic library sequence control file, and customized hardware information database are injected into the service container. The advantage of this is that the information customization component dynamically injects the interception dynamic library and configuration file through the NRI plug-in during the container startup phase, without the need for business system developers to modify the image of the service container or adjust the application code, ensuring that the algorithm acceleration card information concealment / customization can be directly applied to the existing service container, without the need for the release personnel of the business system to restructure or repack the image, which is conducive to the rapid deployment and popularization of the technology. At the same time, the service container developer does not need to be aware of the existence of the underlying interception mechanism, and the application logic remains unchanged. In other words, the developer and end user of the service container do not need to be aware that the hardware information has been disguised, and the algorithm acceleration function (such as GPU computing and AI inference) of the application program can operate normally, but the returned hardware information is replaced with the preset concealed information, realizing the transparency of the algorithm acceleration card hardware information disguise. In addition, by interfacing with the container runtime through the NRI standardized interface, the existing cloud native tool chain (such as Kubernetes and Docker) does not need to be bypassed or damaged, ensuring the seamless compatibility of the technology with the Kubernetes native system.

[0053] The dynamic library sequence control file is used to use the dynamic link library loading mechanism to load the customized interception dynamic library (such as libfakegpu.so) before the service container loads the original algorithm acceleration card interface library.

[0054] Here, the original computing power acceleration card interface library refers to the dynamic link library (so file) provided by the computing power acceleration card manufacturer for accessing and controlling the computing power acceleration card device. These dynamic link libraries provide functions such as querying device information, managing resources, and performing calculations. Taking NVIDIA GPU as an example, the original computing power acceleration card interface library can be CUDA API such as libcuda.so, libcudart.so, or NVML (NVIDIA Management Library) libnvidia-ml.so. Among them, libcuda.so belongs to CUDA Driver API, which is responsible for communication with the kernel module (nvidia.ko); libcudart.so belongs to CUDA Runtime API, which is used to encapsulate Driver API for users to directly program; libnvidia-ml.so is used for GPU state / temperature / power management.

[0055] The dynamic library sequence control file is a configuration file in the Linux system for controlling the loading sequence of dynamic link libraries (so files). Its core function is to force the dynamic linker (ld.so) to load the specified dynamic library first before loading other dynamic libraries. In this technical solution, this file is used to ensure that the custom interception dynamic library is loaded before the original computing power acceleration card interface library, thereby achieving interception of the hardware information query function.

[0056] For example, the dynamic library sequence control file can be an ld.so.preload file, which is usually located in the operating system directory of the business container (such as / etc / ld.so.preload) or in a readable and writable path in the business container. In terms of content and format, the file contains the paths of dynamic libraries that need to be loaded first, one path per line, for example: / path / to / libfakegpu.so. The dynamic library loading mechanism it depends on is that the Linux dynamic linker (ld.so) will first read the libraries listed in the ld.so.preload file when loading any dynamic library, and will load these libraries first. Therefore, by writing the path of the custom interception dynamic library (libfakegpu.so) into this file, it can be ensured that the custom interception dynamic library can be loaded before the original computing power acceleration card interface library is loaded.

[0057] In this embodiment, the custom interception dynamic library (libfakegpu.so) is used to intercept at least one hardware information query function in the original computing power acceleration card interface library and return preset obfuscated information.

[0058] Here, the query instruction refers to a function or instruction provided by an operating system, a computing power acceleration card manufacturer, or a programming language for querying hardware device properties, states, or resource information.

[0059] That is, there are two types of query instructions, one is the call of the query function of the original computing power acceleration card interface library, and the other is the query instruction at the operating system level in the service container. These functions / instructions allow programs / command line windows to obtain hardware-level information such as GPU model, hardware ID, etc. For example, the query function of the original computing power acceleration card can be a query function called by the end user through Python code, such as torch.cuda.get_device_name() and torch.cuda.get_device_properties(), or a function called by the end user through CUDA C++ programming, such as cudaGetDeviceProperties(), or a call to the interface of the NVML library by the manufacturer's monitoring tool command line such as nvdia-smi. The query instruction can be a Linux operating system command: lspci -nn, which is used to view the model information of PCI devices, etc.

[0060] When the end user initiates a call to the hardware information query function or instruction, the custom interception dynamic library can intercept the above call and return the preset obfuscated information (or custom information).

[0061] Specifically, for the call of the query function of the original computing power acceleration card interface library, the custom interception dynamic library can implement the corresponding interface in the code, and hijack the function call of the original computing power acceleration card interface library through function redirection technology, and then return the disguised custom information (i.e., the preset obfuscated information).

[0062] As a specific example, taking the interception of cudaGetDeviceProperties as an example, the implementation code in libfakegpu.so is exemplarily as follows:

[0063]

[0064] It should be noted that in this embodiment, the obfuscated information refers to not returning the real computing power acceleration card information as the query result of the end user, but disguising the real computing power acceleration card information as needed, also known as computing power acceleration card hardware disguised information, custom information, wherein the specific setting of the obfuscated information can be set according to different application scenarios, such as setting the custom computing power acceleration card hardware information according to the needs of privacy protection and data desensitization, the needs of simulating different model GPU test scenarios, and the needs of heterogeneous GPU pooling and unified scheduling.

[0065] For the operating system level query instruction in the business container, such as calling the business container operating system command line lspci -nn to query the hardware information of the computing power acceleration card, the preset concealed information is returned through the query of the self-defined hardware information database.

[0066] It should be noted that the lspci instruction is an instruction in the Linux operating system for reading the PCI device information provided by the kernel, and these PCI devices include graphics cards (such as NVIDIA GPU), network cards, USB controllers, etc. Among them, the "human-readable format" of the device manufacturer ID and device model is mapped through the local database file pci.ids, that is, when the user executes the lspci command, the device information will be returned, and the instruction is essentially read from the local database file pci.ids to return the corresponding information, such as "10de:1eb8, NVIDIA A100", where 10de is the manufacturer ID, 1eb8 is the device model, and the whole record means that "10de:1eb8" is mapped to the human-readable format of hardware information "NVIDIA A100", and these mapping rules can be queried in the local database file pci.ids, therefore, "10de:1eb8, NVIDIA A100" is the original computing power acceleration card information. The path of the pci.ids file may be, for example: / usr / share / misc / pci.ids or / usr / share / hwdata / pci.ids.

[0067] In this embodiment, the self-defined hardware information database is a replacement version of the local database file pci.ids, that is, the original / usr / share / misc / pci.ids or / usr / share / hwdata / pci.ids is replaced with the self-defined hardware information database, and the self-defined hardware information database can store new, preset hardware information mapping rules, through which the original computing power acceleration card information can be mapped to preset concealed information. For example, the following record is stored in the self-defined hardware information database: "10de:1eb8 FakeCorporationFakeCard-001", through which the original real information "NVIDIA A100" of the manufacturer ID and device model "10de:1eb8" can be disguised as "FakeCorporation FakeCard-001", that is, the computing power acceleration card information of the operating system query instruction is concealed / defined.

[0068] In order to avoid intrusion on the business container or business image, in the embodiment, the customized hardware information database is injected into the business container by the information customization component using the NRI mechanism during the creation or startup of the business container.

[0069] Specifically, the information customization component replaces the content of the customized hardware information database (i.e., the new pci.ids file) with the preset hardware information mapping rule during the docker compilation process of itself, for example, replaces the real computing power acceleration card hardware information with the preset concealed information, and stores the new pci.ids file in the preset position of the container where the information customization component is located; then, the information customization component listens to the creation or startup event of the business container, and after listening to the creation or startup of the business container, copies and mounts the customized hardware information database (i.e., the new pci.ids file) from the container where the information customization component is located to the corresponding directory of the business container, such as the / usr / share / misc / pci.ids or / usr / share / hwdata / pci.ids directory, through the NRI mechanism. In this way, through the cooperation of the information customization component and the NRI mechanism, the non-perception replacement of the customized hardware information database is realized, and the concealment / customization of the computing power acceleration card hardware information at the operating system level is realized.

[0070] The following takes Nvidia's computing power acceleration card (GPU) as an example to illustrate the scheme of the embodiment. It should be understood that the following examples should not be understood as a limitation of the technical scheme of the present application. In fact, the scheme provided by the embodiment can be applied to the information concealment and customization of any manufacturer and any model of computing power acceleration card, and is not limited to Nvidia's computing power acceleration card, that is, similar architecture computing power cards are generally applicable.

[0071] The embodiment is dedicated to providing a lightweight and zero-intrusion way for user business code or business container to realize the customization of computing power acceleration card hardware information, and various technical paths (such as through python code, nvidia-smi and lspci command) can view the customized hardware information.

[0072] For Nvidia's computing power acceleration card, Nvidia's CUDA provides query interfaces for end users to obtain GPU hardware information through dynamic library linking libnvidia-ml.so, libcuda.so and other files. The custom interception dynamic library can return custom hardware information by rewriting the relevant interfaces in libnvidia-ml.so, libcuda.so and other files. The ld.so.preload configuration provided by Linux can change the loading order of dynamic libraries, so that the custom interception dynamic library can be loaded and executed in priority to libnvidia-ml.so, libcuda.so and other files, providing technical conditions for realizing user code zero intrusion. The technology of Containerd NRI is the technical condition for realizing user environment zero intrusion. Through the cooperation of the above technologies, the hardware information of the computing power acceleration card under different technical paths can be hidden / customized, and such improvement is completely transparent to the business container, ensuring easy promotion and use.

[0073] It should be noted that certain special computing power acceleration cards cannot use the scheme provided by the present embodiment to realize the hiding / customization of the hardware information of the computing power acceleration card if the hardware information is queried by a static link library.

[0074] For the scenario of zero intrusion to user code (i.e., the problem of zero intrusion to user code needs to be solved) and the scenario of computing power acceleration card (GPU) that can be perceived by the end user under multiple different technical paths, such as nvidia-smi command execution or Python code, CUDA C++ code running, the custom return of the computing power card model is realized, including the following aspects:

[0075] 1. nvidia-smi command returns custom GPU hardware information (such as card model)

[0076] nvidia-smi (NVIDIA System Management Interface) is a command-line tool provided by NVIDIA for monitoring and managing the status of NVIDIA GPUs. nvidia-smi is usually installed with the NVIDIA driver and is widely used in Linux and Windows systems to view the resource usage of the graphics card.

[0077] The user will call the API of CUDA in the process of executing nvidia-smi to obtain the card model information of the Nvidia device, such as nvmlDeviceGetName, cuDeviceGetName, cudaGetDeviceProperties, etc. The three methods respectively involve three dynamic libraries developed by Nvidia: libnvidia-ml.so, libcuda.so, and libcudart.so.

[0078] In this embodiment, by intercepting the return value of the above-mentioned methods through a self-defined intercepting dynamic library, the hardware information (such as the model) of the Nvidia card queried by the end user is modified.

[0079] Specifically, as shown in Figure 1 Part (a) of the figure is the calling flow of the query instruction in the original computing power acceleration card interface library, and part (b) is the calling flow of the query instruction in the original computing power acceleration card interface library using the self-defined intercepting dynamic library. As shown in part (a) of the figure, in a large model application (such as Deepseek and a large language model LLM), the application uses a dynamic linking method when loading the CUDA library, rather than a static linking method, that is, the LLM large model application uses a dynamic linking method when loading the CUDA library, and obtains the real hardware information of the GPU by calling the CUDA API. In order to conceal / customize the real hardware information of the GPU, the technical idea shown in part (b) of the figure is adopted in this embodiment, that is, a self-defined intercepting dynamic library (.so file) is developed to intercept the calling of the CUDA API, so as to return the customized GPU hardware information (such as the card model).

[0080] 2. The lspci command returns the customized GPU hardware information (such as the card model)

[0081] The user will return device information when executing the lspci command, and the lspci command reads from the local database file, and obtains from the / usr / share / misc / pci.ids and / usr / share / hwdata / pci.ids files, respectively.

[0082] In this embodiment, as shown in Figure 2As shown, by independently developing an information customization component, when building the service image corresponding to the information customization component, that is, replacing the content of the pci.ids file in the base image during the docker compilation process to make it become a new pci.ids file (a customized hardware information database) which stores preset hardware information mapping rules (that is, stores the customized card model) for mapping the original computing power acceleration card information to the preset anonymous information, and containerizing and deploying the information customization component to each node of the container cloud platform. After that, when the business container is created or started, the new pci.ids file is mounted to the business container through the NRI mechanism, so that the lspci command returns the disguised GPU hardware information.

[0083] 3. Hijack / etc / ld.so.preload to implement user code zero intrusion

[0084] LLM large model application uses dynamic linking when loading CUDA library, / etc / ld.so.preload is a preloaded configuration file of Linux / Unix system, which can affect the dynamic linker during program running, allowing the required dynamic link library to be loaded in priority before the program runs (i.e. before the business container starts).

[0085] In this embodiment, by modifying the / etc / ld.so.preload file, the custom interception dynamic library can be loaded in priority before the user loads the original computing power acceleration card interface library (dynamic library) such as libnvidia-ml.so, to intercept the system function library (i.e. the original computing power acceleration card interface library), achieving zero intrusion to the user code.

[0086] 4. Use Containerd NRI mechanism to achieve user environment zero intrusion

[0087] When LLM, Deepseek and other large model applications are deployed in a cloud-native environment, zero intrusion and zero perception of the user environment are required.

[0088] In the example of this embodiment, the Containerd NRI working mechanism is used to achieve this, by developing a custom NRI plug-in (i.e. information customization component), to mount (mount) custom files to the container before the business container starts, these custom files include: custom interception dynamic library file (independently developed and compiled so file), dynamic library sequence control file (modified preload file), custom hardware information database (replaced pci.ids file).

[0089] The working mechanism of the information customization component will be described below. Figure 3 .

[0090] In this embodiment, the information customization component is containerized and deployed in each node as an NRI plugin. NRI is an extension mechanism in containerd located in the CRI plugin. NRI can provide an interface for different life cycle events of a container, and users can add custom logic without modifying the source code of the container runtime.

[0091] As shown in Figure 3 When the business container is created, the configuration information (Pod Spec) of the business container is received by the Kubelet component (not shown in the figure) on each node of the container cloud platform, and then the Kubelet component converts the Pod Spec into a CRI request and sends it to the container runtime (such as containerd, containerd, CRI-O, etc.). The containerd receives the CRI request and further converts the CRI request into an OCI Spec (OCI specification), where OCI stands for Open Container Initiative, and the Chinese name is: Open Container Standard Organization. The goal of OCI is to develop open and standardized runtime and image specifications for container technology to avoid "fragmentation" and "incompatibility" between different container platforms, for example, Docker, containerd, CRI-O, etc. all follow the OCI standard. The OCI Spec is a JSON format configuration file that will be read when the container starts. The information customization component as an NRI plugin can obtain a subset of the OCI Spec (Init Spec) through the NRI adapter (NRI adaptation) in the form of an NRI request, and modify the information in the subset, such as writing a custom interception dynamic library (.so file), a dynamic library order control file (preload file), a custom hardware information database (replaced, new pci.ids file), etc. (as shown in Figure 4 The modified information is returned to the NRI adapter as an NRI response, and the containerd obtains the NRI response through the NRI adapter and converts it into an OCI Spec (Adjusted Spec). The corresponding low-level container runtime (such as kata, runc, etc.) is started through the Adjusted Spec, and then the low-level container runtime starts the process of the business container.

[0092] It needs to be emphasized again that in this application, the self-defined interception dynamic library (.so file) is a dynamic library developed independently for intercepting the compilation of CUDA API code, which is used by nvidia-smi and LLM large model application. The self-defined hardware information database is a new pci database file obtained by replacing the original pci.ids file during Docker compilation, which is used by the lspci command. The content of the dynamic library sequence control file ( / etc / ld.so.preload file) is the file address of the self-defined interception dynamic library that needs to be injected. By writing the file address to the preload file, the self-defined interception dynamic library is loaded in priority to the original computing power acceleration card interface library, thereby realizing code zero invasion of the business container.

[0093] In the above process, the developer of the business container only needs to complete the application development according to the original business logic and configure the Pod Spec of the business application, without making any additional settings to the GPU acceleration card. The hardware information hiding / customization logic of GPU acceleration cards with different technical paths is all developed by the container cloud platform vendor and is transparently and imperceptibly injected into the business container through the cooperation of the information customization component and the NRI mechanism. In this way, the complex and variable business scenario requirements can be met, and the business container can imperceptibly obtain the GPU hardware information hiding / customization capability without relying on the implementation of virtualization technology, thereby widening the applicable scenario range of the technology.

[0094] In some embodiments, after listening to the creation or startup event of the business container, the information customization component is further configured to: check whether the configuration of the business container meets a preset filtering condition, and if so, perform the operation of injecting the self-defined interception dynamic library, the dynamic library sequence control file, and the self-defined hardware information database into the business container; and if not, do not perform the above injection operation.

[0095] In this embodiment, it is checked whether the configuration of the service container meets the preset filtering condition, and the purpose is to determine whether the container is a service container running a GPU acceleration card. The configuration of the service container is read from the configuration information (PodSpec) of the service container. For example, the setting of the environment variable (env) in the configuration information can be obtained, and whether the environment variable contains items such as CUDA_VISIBLE_DEVICES, NVIDIA_VISIBLE_DEVICES, etc. is used as a filtering condition to determine whether the container needs to run a GPU acceleration card. The mounting path set in the configuration information of the service container can also be checked to determine whether the container needs to run a GPU acceleration card. The Label / annotation field in the configuration information of the service container can also be read to check whether the field contains a label using a GPU acceleration card, and then determine whether the container needs to run a GPU acceleration card.

[0096] If it is determined that the service application running in the service container needs a GPU acceleration card, the customized interception dynamic library, dynamic library sequence control file, and customized hardware information database are injected into the service container before starting or creating, otherwise, the above injection operation is not performed. The advantage of this is that, on the one hand, through the determination of the filtering condition, only the service container that actually uses the GPU is injected with the interception logic, avoiding irrelevant interference and achieving precise control of the injection range; on the other hand, before injecting the related files into the service container, it is first determined whether to enable the obfuscation / customized information scheme for the service container according to the label, env, image name, etc. of the container, so that the obfuscation / customization of the GPU acceleration card hardware information is dynamically controllable. In addition, the introduction of this scheme also enables the administrator of the container cloud platform to customize more flexible operation and maintenance strategies, such as setting up a blacklist / whitelist mechanism for GPU acceleration card obfuscation / customization, allowing service containers in the whitelist to use real GPU hardware information, while service containers in the blacklist are hidden from the end users.

[0097] In summary, the technical solution provided in this embodiment not only enables the obfuscation / customization of GPU hardware information in different technical paths in a completely transparent manner, but also enables the service application provider to deploy large models more intuitively and easily, reducing the problem that the deployed large model application cannot be started due to the selection of incorrect GPU hardware.

[0098] Based on the same inventive concept, the embodiment provides a service container, at the creation or startup of the service container, the information customization component provided by any of the above embodiments uses the container runtime extension interface NRI plug-in to inject the customized interception dynamic library, the customized hardware information database and the dynamic library sequence control file into the service container, wherein the dynamic library sequence control file is used to use the dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the service container loads the original computing power acceleration card interface library; in response to the service container receiving a query instruction of computing power acceleration card information; the service container calls the interface provided by the customized interception dynamic library to return the preset anonymous information according to the type of the query instruction, or reads the preset anonymous information in the customized hardware information database and returns the preset anonymous information as the query result.

[0099] In the embodiment, the development of the service application operated by the service container and the packaging of the container image are completed by the service application developer, and the container cloud platform needs to realize the anonymization / customization of the computing power acceleration card hardware information without the awareness of the service application developer and the service container image packaging personnel. In order to achieve the above technical purpose, the technical scheme adopted by the embodiment is to provide an information customization component, which can inject corresponding files at the creation or startup of the service container. When the files are executed in the service container, the GPU hardware information query instructions of different technical paths can all obtain the customized information, so that the real GPU hardware information is hidden.

[0100] The embodiment also provides a zero-intrusion computing power acceleration card information customization method, which is applied to a container cloud platform, the container cloud platform is deployed with the information customization component provided by any of the above embodiments and a service container, and the zero-intrusion computing power acceleration card information customization method comprises the following steps:

[0101] Step S1, the information customization component uses the container runtime extension interface NRI mechanism to inject a customized interception dynamic library, a dynamic library sequence control file and a customized hardware information database into the service container in response to listening to a creation or startup event of the service container;

[0102] Step S2, the service container returns preset anonymous information in response to receiving a query instruction of computing power acceleration card information;

[0103] The dynamic library sequence control file is used to use the dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the service container loads the original computing power acceleration card interface library.

[0104] The customized interception dynamic library is used to intercept the query instruction in the original computing power acceleration card interface library and return the preset anonymous information as the query result.

[0105] The self-defined hardware information database is configured to store a mapping relationship between the algorithm acceleration card and the preset anonymous information, so as to cover the original hardware information database file.

[0106] The technical effect obtained by the self-defined algorithm acceleration card information method provided in the embodiment can be referred to the description of the foregoing embodiments, which will not be repeated here.

[0107] The embodiment further provides a self-defined algorithm acceleration card information system, which runs on a container cloud platform. The system comprises the information self-definition component provided in any of the foregoing embodiments, and the service container provided in the foregoing embodiments. The information self-definition component is configured to, in response to listening to a creation or start event of the service container, inject the self-defined interception dynamic library, the dynamic library sequence control file, and the self-defined hardware information database into the service container by using a container runtime extension interface (NRI) mechanism.

[0108] The service container is configured to, in response to receiving a query instruction of the algorithm acceleration card information, return the preset anonymous information.

[0109] The dynamic library sequence control file is configured to, by using a dynamic link library loading mechanism, load the self-defined interception dynamic library preferentially before the service container loads the original algorithm acceleration card interface library.

[0110] The self-defined interception dynamic library is configured to intercept the query instruction in the original algorithm acceleration card interface library, and return the preset anonymous information as a query result.

[0111] The self-defined hardware information database is configured to store a mapping relationship between the algorithm acceleration card and the preset anonymous information, so as to cover the original hardware information database file.

[0112] The technical effect obtained by the self-defined algorithm acceleration card information system provided in the embodiment can be referred to the description of the foregoing embodiments, which will not be repeated here.

[0113] The preferred embodiments of the present application are described above, and the present application is not limited to the above. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An information customization component for a device, characterized in that, the information customization component is containerized and deployed on each node of a container cloud platform, and in response to listening to a creation or startup event of a service container, the information customization component uses a container runtime extension interface (NRI) mechanism to inject a customized interception dynamic library, a dynamic library sequence control file, and a customized hardware information database into the service container, so that the service container can return preset obfuscated information when receiving a query instruction for computing power acceleration card information; wherein the dynamic library sequence control file is used to use a dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the service container loads an original computing power acceleration card interface library; the customized interception dynamic library is used to intercept the query instruction sent to the original computing power acceleration card interface library and return the preset obfuscated information as a query result; the customized hardware information database is used to store a preset hardware information mapping rule, which is used to map the original computing power acceleration card information to the preset obfuscated information.

2. The information self-definition component according to claim 1, characterized in that, After listening to the creation or startup event of the service container, the information customization component is further used to check whether the configuration of the service container meets a preset filtering condition, and if so, perform the operation of injecting the customized interception dynamic library, the dynamic library sequence control file, and the customized hardware information database into the service container; if the preset filtering condition is not met, the above injection operation is not performed.

3. A service container, characterized in that, when the service container is created or started, the information customization component of any one of claims 1-2 uses a container runtime extension interface (NRI) mechanism to inject a customized interception dynamic library, a customized hardware information database, and a dynamic library sequence control file into the service container, and the dynamic library sequence control file is used to use a dynamic link library loading mechanism to preferentially load the customized interception dynamic library before the service container loads an original computing power acceleration card interface library; in response to the service container receiving a query instruction for computing power acceleration card information; the service container calls an interface provided by the customized interception dynamic library to return preset obfuscated information according to the type of the query instruction, or reads preset obfuscated information in the customized hardware information database and returns the preset obfuscated information as a query result.

4. A zero-invasive computing power acceleration card information self-defining method, characterized in that, The method is applied to a container cloud platform, which includes the information customization component of any one of claims 1-2 and the service container of claim 3, and the zero-intrusion computing power acceleration card information customization method includes the following steps: the information customization component uses a container runtime extension interface (NRI) mechanism to inject a customized interception dynamic library, a dynamic library sequence control file, and a customized hardware information database into the service container in response to listening to a creation or startup event of the service container; the service container returns preset obfuscated information in response to receiving a query instruction for computing power acceleration card information; The dynamic library sequence control file is configured to use a dynamic link library loading mechanism to load the custom interception dynamic library in priority to an original computing power acceleration card interface library before the business container loads the original computing power acceleration card interface library. The custom interception dynamic library is configured to intercept a query instruction in the original computing power acceleration card interface library and return preset obfuscated information as a query result. The custom hardware information database is configured to store a mapping relationship between a computing power acceleration card and the preset obfuscated information to cover an original hardware information database file.

5. A zero-invasive computing power acceleration card information self-defining system, characterized in that, The system runs on a container cloud platform and includes the information customization component of any one of claims 1-2 and the business container of claim 3. The information customization component is configured to, in response to listening to a creation or startup event of the business container, use a container runtime extension interface (NRI) mechanism to inject the custom interception dynamic library, the dynamic library sequence control file and the custom hardware information database into the business container. The business container is configured to, in response to receiving a query instruction for computing power acceleration card information, return the preset obfuscated information. The dynamic library sequence control file is configured to use a dynamic link library loading mechanism to load the custom interception dynamic library in priority to an original computing power acceleration card interface library before the business container loads the original computing power acceleration card interface library. The custom interception dynamic library is configured to intercept a query instruction in the original computing power acceleration card interface library and return preset obfuscated information as a query result. The custom hardware information database is configured to store a mapping relationship between a computing power acceleration card and the preset obfuscated information to cover an original hardware information database file.

Images