Self-adaptive software stack optimization method and system based on GPU (Graphics Processing Unit) server configuration

Abstract

The invention relates to an adaptive software stack optimization method and system based on GPU server configuration, and the method comprises the steps: automatically collecting hardware parameters and scene constraint parameters of a GPU server through hardware recognition, and converting the hardware parameters and the scene constraint parameters into structured parameters; calling a resource library to perform semantic matching retrieval based on preset keywords and the structured parameters, and eliminating outdated knowledge in combination with a time decay factor algorithm to form a dynamic knowledge base; based on the structured parameters and the dynamic knowledge base, assembling large language model reasonable structured cue words according to environmental constraints, optimal practices and to-be-solved problems; performing multi-stage reasoning based on the structured cue word to generate a configuration reasoning result; analyzing the configuration reasoning result into a modular configuration scheme; and obtaining an optimization case based on the operation index after the deployment of the modular configuration scheme and user feedback, and updating the optimization case to the resource library.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of large models, in particular to a self-adaptive software stack optimization method and system based on GPU server configuration. BACKGROUND

[0002] With the rapid development of large model technology and the continuous expansion of application scenarios, the software stack complexity of large models is growing exponentially. Currently, the software ecosystem of large models faces significant technical challenges: on the one hand, software components exhibit "multiple varieties and multiple versions" characteristics, covering operating systems, programming language runtimes, compilation tool chains, inference acceleration libraries, and other basic software, and compatibility requirements between versions of each component are strict; on the other hand, hardware (such as GPU) updates and iterations are short, and the computing power, memory configuration, and interface specifications of different generations of hardware differ significantly. Under this background, ordinary users have difficulty accurately grasping software and hardware parameter matching rules, and even more so in efficiently filtering out the best configuration scheme that adapts to specific application scenarios.

[0003] In practical applications, large model application runtimes frequently experience functional abnormalities or substandard performance, which is mainly attributed to insufficient software environment adaptability: some applications rely on incompatible operating system versions and hardware drivers, or programming language runtime versions and model training frameworks have version conflicts, or even compilation tool chain configurations and hardware instruction sets do not match; more critically, some core software components (such as CUDA, TensorRT, etc.) have strong dependencies on hardware features (such as GPU memory capacity, MIG slicing capability), and when hardware parameters do not meet the requirements, the application cannot be started or crashes.

[0004] Based on this, the application provides a self-adaptive software stack optimization method and system based on GPU server configuration. SUMMARY

[0005] To improve the problem of frequent functional abnormalities or substandard performance of large model application runtimes, the application provides a self-adaptive software stack optimization method and system based on GPU server configuration.

[0006] In a first aspect, the application provides a self-adaptive software stack optimization method based on GPU server configuration, which adopts the following technical solution: comprising:

[0007] The hardware parameters and scenario constraint parameters of the GPU server are automatically collected through hardware recognition, and the hardware parameters and scenario constraint parameters are converted into structured parameters through the multi-modal perception capability of the large model hub;

[0008] Based on the preset keywords and the structured parameters, a resource library is called to perform semantic matching retrieval, outdated knowledge is eliminated by combining a time decay factor algorithm, and a dynamic knowledge base adapted to the structured parameters is formed;

[0009] Based on the structured parameters and the dynamic knowledge base, structured prompt words that can be inferred by a large language model are assembled according to environmental constraints, best practices and problems to be solved;

[0010] A large language model cluster is called to perform multi-stage inference based on the structured prompt words, and a configuration inference result is generated;

[0011] The configuration inference result is parsed into a modular configuration scheme, which includes a basic required module, an optional enhancement module and an interpretable accessory;

[0012] Based on the running indicators and user feedback after deployment of the modular configuration scheme, an optimization case is obtained, and the optimization case is updated to the resource library.

[0013] Preferably, the hardware parameters and scene constraint parameters of the GPU server are automatically collected through hardware recognition, and the hardware parameters and scene constraint parameters are converted into structured parameters through the multi-modal perception ability of the large model hub, including:

[0014] The hardware parameters include GPU model, video memory capacity, MIG support capability, PCIe version and hardware topology structure;

[0015] The scene constraint parameters include inference delay requirement, system throughput demand, model type and accuracy requirement;

[0016] The multi-modal perception ability of the large model hub converts the hardware parameters and the scene constraint parameters into structured parameters in JSON format by fusing real-time hardware state data collected through a hardware management interface, user input text requirements and similar scene parameters in a historical configuration database, and using a multi-modal fusion model. The structured parameters include fields in three dimensions of hardware characteristics, scene requirements and constraint conditions.

[0017] Preferably, based on the preset keywords and the structured parameters, a resource library is called to perform semantic matching retrieval, outdated knowledge is eliminated by combining a time decay factor algorithm, and a dynamic knowledge base adapted to the structured parameters is formed, including:

[0018] The internal component set of the data layer calls an enterprise platform to obtain chip original factory recommended documents, and pulls large model optimization container metadata;

[0019] The user open source configuration case is collected from a large model community warehouse by an external resource acquisition unit, and an industry benchmark test report is obtained from a professional agency website;

[0020] The structured parameters are used to perform accurate keyword retrieval on a text database of a knowledge layer to obtain MIG fragmentation strategies and software version compatibility matrix knowledge in a chip original factory recommended document;

[0021] The vector database is used to perform semantic level matching retrieval of "large model low latency inference optimization" to obtain performance benchmark data in the industry benchmark test report;

[0022] The time decay factor algorithm is used to assign dynamic weights to knowledge items, and the weights of knowledge items exceeding a preset time limit are automatically reduced or removed to ensure that the knowledge base only retains the latest configuration scheme adapted to the current hardware;

[0023] The MIG fragmentation strategies, the software version compatibility matrix knowledge, the performance benchmark data, and the latest configuration scheme are fused to form the dynamic knowledge base adapted to the structured parameters.

[0024] Preferably, based on the structured parameters and the dynamic knowledge base, a structured prompt word that can be inferred by a large language model is assembled according to environmental constraints, best practices, and problems to be solved, including:

[0025] The environmental constraints include room temperature limits, power supply power limits, and network bandwidths;

[0026] The best practices include historical optimization cases, chip original factory recommended strategies, and industry standards stored in the dynamic knowledge base;

[0027] The problems to be solved include conflict points in current configuration requirements, unmet functional requirements, and potential risks;

[0028] The hardware characteristics, scene requirements, environmental constraints, best practices, and problems to be solved of the structured parameters are hierarchically assembled to form an inferable prompt word containing hardware context, requirement description, knowledge reference, and conflict to be solved, which is used for multi-stage inference by a large language model cluster.

[0029] Preferably, the large language model cluster is called to perform multi-stage inference based on the structured prompt word to generate a configuration inference result, including:

[0030] The current GPU hardware state is obtained in real time by the large language model based on the structured prompt word calling NVMLAPI, the matching of MIG fragmentation and video memory requirements is verified, and a hardware constraint verification conclusion is output;

[0031] The software version compatibility matrix in the dynamic knowledge base is called by a large language model, combined with software constraints in the structured prompt words, to verify the matching of candidate software versions and scene requirements, and a compatibility verification report is output;

[0032] Based on the scene constraints in the structured prompt words and the industry benchmark test reports and historical deployment cases in the dynamic knowledge base, a performance prediction model is used to calculate the throughput, delay and resource utilization of the candidate configuration, and a performance simulation result is output;

[0033] The hardware constraint verification conclusion, the software compatibility verification report and the performance simulation result are fused to generate the configuration reasoning result including hardware configuration suggestions, software version lists and performance expectation values.

[0034] Preferably, the configuration reasoning result is parsed into a modular configuration scheme, and the modular configuration scheme includes a basic required module, an optional enhancement module and an interpretable accessory, which includes:

[0035] Based on the hardware constraint verification conclusion and the compatibility verification report in the configuration reasoning result, components that meet the minimum hardware operation requirements and scene basic requirements are extracted to form the basic required module;

[0036] Based on the performance simulation result and scene expansion requirements in the configuration reasoning result, components that can improve performance or expand functions are selected based on the dynamic knowledge base to form the optional enhancement module;

[0037] Based on the hardware constraint verification conclusion, the software version compatibility matrix and the performance simulation data in the configuration reasoning result, an interpretable accessory is generated, which includes parameter basis, performance expectation and alternative options.

[0038] Preferably, based on the running indicators and user feedback after deployment of the modular configuration scheme, an optimization case is obtained, and the optimization case is updated to the resource library, which includes:

[0039] The running indicators of the modular configuration scheme are collected in real time by a monitoring tool in the deployment environment, and the running indicators include but are not limited to single-card inference delay, system throughput, GPU utilization, video memory occupancy, hardware temperature and abnormal logs;

[0040] User feedback is collected through a user interaction interface, a technical support ticket system and a community forum, and the feedback content includes performance satisfaction, functional requirements and fault description;

[0041] The collected operation indicators and the user feedback are associated and analyzed to extract key optimization points, and a structured optimization case including a problem scenario, a configuration parameter, an optimization measure and an effect verification is generated based on the actual effect of the configuration scheme;

[0042] The optimization case is imported into the resource library through an external resource acquisition unit of the resource library, and after being verified based on the timeliness rule of the dynamic knowledge base, is stored as a new knowledge item in the text database and the vector database;

[0043] Meanwhile, time decay factor algorithm is triggered for old knowledge items in the resource library that conflict with the optimization case, and the weight of the old knowledge items is reduced or the old knowledge items are marked as historical references, so as to complete dynamic updating of the resource library.

[0044] In a second aspect, an adaptive software stack optimization device based on a GPU server configuration is provided, which adopts the following technical scheme, comprising:

[0045] A collection module is configured to automatically collect hardware parameters and scene constraint parameters of a GPU server through hardware recognition, and convert the hardware parameters and the scene constraint parameters into structured parameters through multi-modal perception capability of a large model hub;

[0046] A retrieval module is configured to perform semantic matching retrieval by calling a resource library based on preset keywords and the structured parameters, eliminate outdated knowledge by combining a time decay factor algorithm, and form a dynamic knowledge base adapted to the structured parameters;

[0047] A prompt module is configured to assemble structured prompt words that can be inferred by a large language model according to environmental constraints, best practices and problems to be solved based on the structured parameters and the dynamic knowledge base;

[0048] An inference module is configured to call a large language model cluster, perform multi-stage inference based on the structured prompt words, and generate a configuration inference result;

[0049] A configuration module is configured to parse the configuration inference result into a modular configuration scheme, wherein the modular configuration scheme includes a basic required module, an optional enhancement module and an interpretable accessory;

[0050] An optimization module is configured to obtain an optimization case based on operation indicators and user feedback after deployment of the modular configuration scheme, and update the optimization case to the resource library.

[0051] In a third aspect, the application further provides a control device, which comprises:

[0052] The device includes a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to perform the adaptive software stack optimization method based on the GPU server configuration as described above.

[0053] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program capable of being loaded and executed by a processor to perform the adaptive software stack optimization method based on GPU server configuration as described above.

[0054] In summary, in the present application, GPU hardware parameters and scene constraint parameters are automatically collected and structured, a reasoning prompt word is generated in combination with a resource library dynamic knowledge base, a large model cluster is called to perform multi-stage reasoning and output a configuration result, and the result is parsed into a modular solution and continuously optimized through running indicators and user feedback. The resource library solves the problem that users are difficult to match the best configuration under the condition of complex large model software stack, numerous versions, and rapid iteration of hardware. Through dynamic knowledge updating, multi-modal perception, and large model reasoning, intelligent adaptation of software and hardware parameters is achieved, the configuration difficulty is reduced, application running errors caused by environmental incompatibility are reduced, and the stability and usability of large model applications are improved. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a schematic diagram of the hierarchical architecture and core components of the system.

[0056] Figure 2 is a flowchart of an adaptive software stack optimization method based on GPU server configuration.

[0057] Figure 3 is a flowchart of software stack optimization.

[0058] Figure 4 is a structural block diagram of an adaptive software stack optimization device based on GPU server configuration. DETAILED DESCRIPTION

[0059] The following will be described in detail in combination with Figure 1 - Figure 4 The present application will be further described in detail.

[0060] Reference is made to Figure 1The present application aims at the pain points of "version complexity, hardware adaptation difficulty, and configuration black box" of large model software stack, and proposes a technical architecture of "bottom-up layered decoupling + inter-layer collaborative closed loop". Through full-link collaboration of infrastructure layer (hardware-system base) → data layer (resource aggregation) → knowledge layer (storage + generative enhancement) → intelligent recommendation layer (large model central inference) → recommended content layer (modular output), with the help of deep integration of large language model (LLM) cluster and retrieval augmented generation (RAG) technology, the hardware characteristic perception → knowledge dynamic activation → intelligent inference decision → configuration interpretable output of software and hardware collaborative optimization is realized, and the full-link technical solution from hardware base to AI application recommendation is realized, providing a high-performance and high-reliability software stack configuration solution for large model scenarios (taking H100 GPU deployment of Evo2 inference service as an example).

[0061] The system adopts a "five-layer decoupling + inter-layer collaboration" architecture, and the functions and core components of each layer are as follows:

[0062] 1. Infrastructure layer (bottom running support unit)

[0063] Provide "physical-software base" for the system, including:

[0064] Hardware specification configuration module: define CPU, memory, storage, etc. Basic computing power (such as CPU multi-core scheduling of H100 server, memory bandwidth optimization);

[0065] Firmware version module: load firmware program (such as NvidiaBOOTROM) adapted to H100, realize software call of hardware function;

[0066] Operating system module: deploy Ubuntu22.04 OS, etc. Complete software-level scheduling of H100 resources (such as NVMe storage IO optimization).

[0067] 2. Data layer (resource aggregation and input unit)

[0068] Take on the function of "internal and external resource aggregation-supply", and provide AI ecological tools and external resources for the upper layer:

[0069] Internal component set: integrate Nvidia-ai-enterprise (Nvidia AI enterprise platform), ngc (Nvidia resource platform), nim (Nvidia inference microservice), etc.

[0070] External resource acquisition unit: through two channels of official website / github (official technical community) and professional institutions / self-maintenance (custom or third-party resource library), realize collection and access of external technical resources.

[0071] 3. Knowledge layer (data storage and augmented processing unit)

[0072] Build the core knowledge base of "storage + generative enhancement", and realize the dynamic management of knowledge through "storage + generative enhancement" technology:

[0073] Database (DB): Use relational / non-relational databases (such as MySQL, MongoDB, etc.) to realize the persistent storage and retrieval of structured data;

[0074] Retrieval and enhancement generation module (RAG): Through the technical path of "data retrieval → generative AI processing → knowledge enhancement output", the semantic expansion and intelligent enhancement of DB stored data are realized, and dynamic knowledge support is provided for the intelligent recommendation layer.

[0075] 4. Intelligent recommendation layer (core reasoning and interaction unit)

[0076] System "intelligent center", based on knowledge layer enhanced knowledge to realize AI recommendation logic and external interaction:

[0077] Interaction portal (Portal): As the unified entrance of the system for external interaction, it supports user request receiving, recommendation result output and other interactive operations, such as receiving user "4 card H100 deployment Evo2 reasoning" and other requirements;

[0078] Large language model cluster (LLM): Deploy multiple large language models (such as GPT, open source LLM, etc.), and the models interact with each other to form collaborative reasoning capability; At the same time, the LLM cluster and the DB and RAG modules of the knowledge layer are linked bidirectionally —— both obtaining knowledge input from DB / RAG to drive reasoning, and feeding back the reasoning generation results to the knowledge layer (or generating recommendation instructions based on knowledge layer data).

[0079] 5. Recommended content layer (output carrier and expansion unit)

[0080] The final output carrier of the intelligent recommendation layer, adopts modular design of "basic required + optional enhancement" to adapt to different scenarios:

[0081] Basic recommendation module (framework-level required component): Includes PyTorch (AI computing framework), hardware driver (hardware adaptation program), cuda software stack (GPU computing acceleration tool chain), and key category api / sdk / runtime / library (general tool set), which constitutes the core foundation of recommended content;

[0082] Optional expansion module (flexible enhancement component): Set Colossal-AI (distributed training framework) and "other" reserved expansion bit, support loading additional enhancement components (such as domain-specific tools, self-developed modules, etc.) according to scene requirements.

[0083] Referring to Figure 2 and Figure 3 The embodiments of the present application at least include steps S10 to S60.

[0084] S10, automatically collecting hardware parameters and scene constraint parameters of the GPU server through hardware recognition, and converting the hardware parameters and scene constraint parameters into structured parameters through the multi-modal perception ability of the large model hub.

[0085] S20, based on the preset keywords and structured parameters, calling the resource library for semantic matching retrieval, combining the time decay factor algorithm to eliminate outdated knowledge, and forming a dynamic knowledge base adapted to the structured parameters.

[0086] S30, based on the structured parameters and the dynamic knowledge base, assembling the environment constraints, best practices and the to-be-solved problems into structured prompt words that can be reasoned by the large language model.

[0087] S40, calling a large language model cluster, performing multi-stage reasoning based on the structured prompt words, and generating a configuration reasoning result.

[0088] S50, parsing the configuration reasoning result into a modular configuration scheme, which includes a basic required module, an optional enhancement module and an interpretable accessory.

[0089] S60, based on the running indicators and user feedback after the deployment of the modular configuration scheme, obtaining an optimization case, and updating the optimization case to the resource library.

[0090] Specifically, by automatically collecting and structuring the hardware parameters and scene constraint parameters of the GPU server through hardware recognition, combining the resource library semantic retrieval and the time decay algorithm to dynamically update the knowledge base, assembling the large model reasoning prompt words based on the environment constraints, best practices and to-be-solved problems, calling the large model cluster to perform multi-stage reasoning to generate a configuration result, parsing it into a modular scheme containing required, optional modules and interpretable accessories, and continuously optimizing the resource library through the running indicators and user feedback after deployment. Its role is to solve the problem that users are difficult to match the best configuration under the condition of complex large model software stack, numerous versions and rapid iteration of hardware. Through dynamic knowledge update, multi-modal perception and large model intelligent reasoning, accurate adaptation of software and hardware parameters is realized, the configuration threshold is reduced, application running errors caused by environment incompatibility are reduced, and the stability and usability of large model applications are improved.

[0091] In some embodiments, step S10 specifically comprises the following steps: the hardware parameters include GPU model, memory capacity, MIG support capability, PCIe version and hardware topology; the scene constraint parameters include inference delay requirement, system throughput demand, model type and accuracy requirement; the multi-modal perception capability of the large model hub converts the real-time hardware state data collected through the fusion hardware management interface, the text requirements input by the user and the similar scene parameters in the historical configuration database into structured parameters in JSON format by using a multi-modal fusion model, and the structured parameters include fields in three dimensions of hardware characteristics, scene demand and constraint conditions.

[0092] In fact, when the user initiates the requirement of “deploying an Evo2 model inference service on a 4-card H100-SXM5-80GB GPU server, with a requirement of single-card inference delay < 50ms and throughput ≥ 200 tokens / second”:

[0093] Hardware dimension: the intelligent recommendation layer automatically collects H100 core parameters such as GPU model (H100-SXM5-80GB), driver version, CUDA computing power level, NVLink topology and hardware characteristics through NVML / CUDA API; scene dimension: collect Evo2 model constraints (model parameter volume ≈ 100B, input sequence length 512, output sequence length 256, service QPS ≥ 100).

[0094] Specifically: through the multi-modal perception capability of the large model hub, “hardware physical characteristics (such as H100 FP8 acceleration)” and “scene business constraints (such as Evo2 inference delay)” are converted into structured parameters understandable by LLM, providing accurate input for subsequent inference.

[0095] In some embodiments, step S20 specifically comprises the following steps: the enterprise platform is called through the internal component set of the data layer to obtain the chip original factory recommended document, and the large model optimization container metadata is pulled; the user open source configuration case is collected from the large model community warehouse through the external resource acquisition unit, and the industry benchmark test report is obtained through the professional agency website; based on the structured parameters, the text database of the knowledge layer is used for accurate keyword retrieval to obtain the MIG sharding strategy and software version compatibility matrix knowledge in the chip original factory recommended document; the semantic level matching retrieval of “large model low delay inference optimization” is performed through the vector database to obtain the performance benchmark data in the industry benchmark test report; the dynamic weight is assigned to the knowledge item through the time decay factor algorithm, and the weight or removal of the knowledge item exceeding the preset time limit is automatically reduced, so that the knowledge base only retains the latest configuration scheme adapted to the current hardware; the MIG sharding strategy, software version compatibility matrix knowledge, performance benchmark data and latest configuration scheme are fused to form a dynamic knowledge base adapted to the structured parameters.

[0096] In fact, the system triggers the data layer + knowledge layer collaborative retrieval:

[0097] Data layer resource call:

[0098] Internal components: call Nvidia-ai-enterprise to get "H100 inference optimization guide" (including TransformerEngine deployment manual), ngc pull Evo2 official optimization container (built-in TensorRT-LLM inference plug-in);

[0099] External resources: collect user open source inference configuration cases from GitHub (Evo2 community repository), and get "H100 inference performance benchmark report (ID:20250315)" from MLPerf official website.

[0100] Knowledge layer dual-mode retrieval:

[0101] Text database (Elasticsearch): Based on the keyword "Evo2H100 inference configuration", recall "Multi-Instance GPU (MIG) fragmentation strategy (4 cards each cut 2x20GB instance, adapt to Evo2 memory demand)" "TensorRT-LLM and CUDA12.3 compatibility matrix" in Nvidia official document;

[0102] Vector database (FAISS, Sentence-BERT semantic embedding): Based on semantic matching "large model low latency inference optimization", recall "H100+TensorRT-LLM in batch_size=32 optimal solution for latency" in MLPerf report.

[0103] Dynamic update mechanism takes effect: The knowledge layer automatically filters the configuration schemes below CUDA12.2 that are more than 3 months old (because H100 supports CUDA12.3+ better, and the performance of old versions decays by ≥20%), ensuring the timeliness of recommended knowledge.

[0104] Specifically: Through "data layer multi-source aggregation + knowledge layer RAG dual-mode retrieval + dynamic knowledge elimination", solve the pain points of "fast version iteration, knowledge fragmentation" of large model software stack, and provide "authoritative, real-time, scene adaptation" knowledge base for LLM inference.

[0105] In some embodiments, step S30 specifically comprises the following steps: the environmental constraints include machine room temperature limits, power supply power limits, and network bandwidth; the best practices include historical optimization cases stored in the dynamic knowledge base, chip manufacturer recommended strategies, and industry standards; the problems to be solved include conflict points in current configuration requirements, unmet functional requirements, and potential risks; by hierarchically assembling the hardware characteristics of structured parameters, scene requirements, environmental constraints, best practices, and problems to be solved, a reasonabable prompt word is formed, which includes hardware context, requirement description, knowledge reference, and conflict to be solved, for the multi-stage reasoning of the large language model cluster.

[0106] In fact, the intelligent recommendation layer automatically assembles "hardware parameters, scene constraints, knowledge base search results" into structured prompts that can be reasoned by LLM according to the "environmental constraints + best practices + problems to be solved" template.

[0107] For example, known hardware: 4xH100-SXM5-80GB(CUDA12.3, driver545.23.08, NVLink bandwidth 800GB / s, MIG support 20GB slice);

[0108] Scenario requirements: Evo2 model inference service(model parameter ≈100B, input seq_len=512, output seq_len=256, single-card delay <50ms, QPS≥100);

[0109] Existing knowledge: ①Nvidia official: H100 inference recommends enabling MIG(4 cards each divided into 2x20GB instances to reduce memory fragmentation), TensorRT-LLM needs to be compatible with CUDA12.3(recommended TensorRT-LLM0.9.0); ②MLPerf report: H100+TensorRT-LLM at batch_size=32, delay 45ms, throughput 220tokens / second(meet QPS requirements); ③Community practice: Evo2 inference needs to enable PyTorch2.2(including 2.0+ compilation optimization)+FasterTransformer plugin.

[0110] Please generate a configuration scheme that meets the following constraints: hardware resource utilization rate ≥90%(achieved through MIG slicing and NVLink topology optimization); software version compatibility(e.g.TensorRT-LLM and CUDA, driver version matching); performance indicators meet the standards(single-card delay <50ms, throughput ≥200tokens / second); the scheme should include "parameter basis, performance expectations, alternative options"(enhance explainability).

[0111] Specifically: through the "environmental parameter → prompt word automatic conversion rule", the fragmented knowledge is converted into the "decision context" of the LLM, which not only restricts the reasoning direction (such as the MIG strategy of H100), but also opens up the optimization space (such as the fine-tuning of batch_size), achieving a balance between "rule constraints + intelligent reasoning".

[0112] In some embodiments, step S40 specifically comprises the following steps: through the large language model, the current GPU hardware state is obtained in real time based on the structured prompt word calling NVMLAPI, the matching of MIG slices and video memory requirements is verified, and the hardware constraint verification conclusion is output; through the large language model, the software version compatibility matrix in the dynamic knowledge base is called, combined with the software constraints in the structured prompt word, the matching of the candidate software version and the scene requirement is verified, and the compatibility verification report is output; based on the large language model, according to the scene constraints in the structured prompt word and the industry benchmark test report and historical deployment cases in the dynamic knowledge base, the throughput, delay and resource utilization of the candidate configuration are calculated through the performance prediction model, and the performance simulation result is output; the hardware constraint verification conclusion, the software compatibility verification report and the performance simulation result are fused to generate a configuration reasoning result containing hardware configuration suggestions, software version list and performance expectation value.

[0113] In fact, the intelligent recommendation layer calls the LLM cluster (such as industry customized model + open source model cooperation) to perform chain reasoning based on the above prompt words:

[0114] First stage: verify hardware constraints (such as whether the video memory after MIG slicing meets the Evo2 reasoning requirements: 2x20GB MIG instance → single instance video memory 20GB, Evo2 reasoning video memory peak 18GB → meet);

[0115] Second stage: match software version (TensorRT-LLM0.9.0 and CUDA12.3, driver 545.23.08 compatibility verification → Nvidia official document confirms compatibility);

[0116] Third stage: performance simulation (combined with MLPerf data, when batch_size = 32, 4-card cluster throughput = 4x220 = 880 tokens / second → meet QPS≥100 requirements).

[0117] Specifically: through multi-LLM collaborative reasoning, the "hardware understanding ability" (such as Nvidia customized LLM) and "scene optimization experience" (such as industry vertical LLM) of different models are used to solve the problem of "insufficient knowledge coverage and reasoning deviation" of a single model, and ensure the technical feasibility of the solution.

[0118] In some embodiments, step S50 specifically comprises the following steps: based on the hardware constraint verification conclusion and the compatibility verification report in the configuration reasoning result, extracting components that meet the hardware minimum operation requirements and the scene basic needs to form a basic mandatory module; based on the performance simulation result in the configuration reasoning result and the scene expansion demand, filtering components that can improve performance or expand functions based on the dynamic knowledge base to form an optional enhancement module; based on the hardware constraint verification conclusion, the software version compatibility matrix and the performance simulation data in the configuration reasoning result, generating an interpretable attachment, which includes: parameter basis, performance expectation, alternative options.

[0119] In fact, after the LLM reasoning generates a configuration scheme, the recommended content layer is parsed into a modular output of “basic mandatory + optional enhancement + interpretable attachment”:

[0120] Basic mandatory module: CUDA12.3 (H100 hardware acceleration basis), PyTorch2.2 (including FasterTransformer plug-in, supporting Evo2 inference optimization), Nvidia driver 545.23.08 (adapting to H100 MIG characteristics), TensorRT-LLM0.9.0 (inference acceleration core tool);

[0121] Optional enhancement module: Colossal-AI (enable InferenceServing module, optimize multi-card load balancing), TritonInferenceServer (implement model dynamic scheduling, respond to QPS fluctuations);

[0122] Interpretable attachment: parameter basis (NvidiaMIG document section 6.2, MLPerf report ID#20250315, Evo2 community configuration case commit#abc123); performance expectation (single-card delay 42ms, throughput 225tokens / second, 4-card cluster total throughput 900tokens / second, resource utilization rate 92%); alternative options (if the delay is over-standard, downgrade MIG to 1x40GB instance, enable PyTorch2.2 INT8 quantization, expected delay 48ms, throughput 190tokens / second). Deep logic: through modular design + interpretability enhancement, it not only meets the needs of “basic function mandatory stable” and “scene enhancement optional flexible”, but also provides users with “technical decision basis” (such as why choose TensorRT-LLM0.9.0) and “risk plan” (such as the downgrade scheme when the delay is over-standard), solving the traditional configuration recommendation “black box, untrusted” pain points.

[0123] In some embodiments, step S60 specifically comprises the following steps: collecting running indicators of the modular configuration scheme in real time through a monitoring tool in the deployment environment, the running indicators including but not limited to single-card inference delay, system throughput, GPU utilization, video memory occupancy, hardware temperature and abnormal logs; collecting user feedback through a user interaction interface, a technical support ticket system and a community forum, the feedback content including performance satisfaction, functional requirements and fault description; performing correlation analysis on the collected running indicators and user feedback, extracting key optimization points, combining the actual effect of the configuration scheme, and generating a structured optimization case including problem scenarios, configuration parameters, optimization measures and effect verification; importing the optimization case into the resource library through the external resource acquisition unit of the resource library, and after verification based on the timeliness rules of the dynamic knowledge base, storing it as a new knowledge item in the text database and the vector database; at the same time, triggering the time decay factor algorithm for the old knowledge items in the resource library that conflict with the optimization case, reducing their weights or marking them as historical references, and completing the dynamic update of the resource library.

[0124] In fact, after the user deploys the Evo2 inference service according to the recommended scheme, the system automatically collects running metrics (such as actual single-card delay 40ms, throughput 230 tokens / second, resource utilization 95%), combines user feedback ("MIG sharding improves inference stability, no video memory overflow error"), and triggers the case submission process of the data layer and the artificial coordination layer:

[0125] After the user submits the "4xH100+Evo2 inference optimization case", the knowledge layer triggers a hot update after verification, and adds the case to the text / vector database; the knowledge weight algorithm gives a high time decay factor weight (preferential recommendation) to the "recent effective case", and automatically reduces the weight of the old scheme of "non-H100 architecture" one year ago, ensuring that the knowledge base continuously adapts to hardware iteration. Deep logic: through the closed loop of "user practice feedback→knowledge base evolution", the system "learns from the industry's best practices and evolves towards user individual needs", solving the industry problem of "fast technical iteration of large model software stack and non-adaptation of general scheme to business".

[0126] The implementation principle of the adaptive software stack optimization method based on GPU server configuration in the embodiment of the present application is as follows: First, the GPU hardware parameters and scenario constraint parameters are automatically collected and structured, and then the inferable prompt words are generated in combination with the dynamic knowledge base of the resource library. The multi-stage inference output configuration result of the large model cluster is called, and after parsing into a modular solution, the resource library is continuously optimized through operation indicators and user feedback. This method solves the problem that users find it difficult to match the optimal configuration under the complex large model software stack, numerous versions and rapid hardware iteration. Through dynamic knowledge updating, multimodal perception and large model reasoning, intelligent adaptation of software and hardware parameters is achieved, configuration difficulty is reduced, application operation errors caused by environmental incompatibility are reduced, and the stability and availability of large model applications are improved.

[0127] Figure 2 FIG. 1 is a flow chart of an adaptive software stack optimization method based on GPU server configuration in one embodiment. It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be executed in other orders; and Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0128] Based on the same technical concept, Figure 4 The embodiment of the present application further provides an adaptive software stack optimization device based on GPU server configuration, which adopts the following technical solution and includes:

[0129] The acquisition module is used to automatically collect the hardware parameters and scenario constraint parameters of the GPU server through hardware identification, and convert the hardware parameters and scenario constraint parameters into structured parameters through the multimodal perception capabilities of the large model center;

[0130] The retrieval module is used to call the resource library for semantic matching retrieval based on preset keywords and structured parameters, and to eliminate outdated knowledge by combining the time decay factor algorithm to form a dynamic knowledge base adapted to the structured parameters;

[0131] The prompt module is used to assemble structured prompt words that can be reasoned by the large language model based on structured parameters and a dynamic knowledge base according to environmental constraints, best practices, and problems to be solved;

[0132] The reasoning module is configured to call a large language model cluster, perform multi-stage reasoning based on structured prompt words, and generate a configuration reasoning result.

[0133] The configuration module is configured to parse the configuration reasoning result into a modular configuration scheme, which includes a basic required module, an optional enhancement module, and an interpretable accessory.

[0134] The optimization module is configured to obtain an optimization case based on running indicators and user feedback after deployment of the modular configuration scheme, and update the optimization case to the resource library.

[0135] In some embodiments, the collection module is specifically configured to collect hardware parameters including GPU model, video memory capacity, MIG support capability, PCIe version, and hardware topology structure.

[0136] The scene constraint parameters include inference delay requirements, system throughput requirements, model types, and accuracy requirements.

[0137] The multi-modal perception capability of the large model hub converts hardware parameters and scene constraint parameters into structured parameters in JSON format by using a multi-modal fusion model, which fuses real-time hardware state data collected through a hardware management interface, user input text requirements, and similar scene parameters in a historical configuration database. The structured parameters include fields in three dimensions of hardware characteristics, scene requirements, and constraint conditions.

[0138] In some embodiments, the retrieval module is specifically configured to call an enterprise platform through an internal component set of the material layer to obtain chip original factory recommended documents, and to pull large model optimization container metadata.

[0139] The external resource acquisition unit collects user open source configuration cases from a large model community repository, and obtains industry benchmark test reports through professional agency websites.

[0140] Based on the structured parameters, the knowledge layer performs accurate keyword retrieval on a text database to obtain MIG fragmentation strategies and software version compatibility matrix knowledge in the chip original factory recommended documents.

[0141] The vector database performs semantic level matching retrieval of "large model low delay reasoning optimization" to obtain performance benchmark data in the industry benchmark test report.

[0142] The time decay factor algorithm assigns dynamic weights to knowledge items, automatically reduces the weights or removes knowledge items that exceed the preset time limit, and ensures that the knowledge base only retains the latest configuration scheme adapted to the current hardware.

[0143] The MIG fragmentation strategy, software version compatibility matrix knowledge, performance benchmark data, and the latest configuration scheme are fused to form a dynamic knowledge base adapted to the structured parameters.

[0144] In some embodiments, the prompt module is specifically configured to include environment constraints, including machine room temperature limits, power supply power limits, and network bandwidths.

[0145] Best practices include historical optimization cases stored in the dynamic knowledge base, chip manufacturer recommended strategies, and industry standards.

[0146] Problems to be solved include conflict points in current configuration requirements, unmet functional requirements, and potential risks.

[0147] By hierarchically assembling the hardware characteristics of structured parameters, scene requirements, environment constraints, best practices, and problems to be solved, a reasoning prompt word is formed, which includes hardware context, requirement description, knowledge reference, and conflicts to be solved. Four parts are used for multi-stage reasoning by large language model clusters.

[0148] In some embodiments, the inference module is specifically configured to use the large language model to call NVML API in real time to obtain the current GPU hardware state based on the structured prompt word, verify the matching of MIG slices and video memory requirements, and output hardware constraint verification conclusions.

[0149] By calling the software version compatibility matrix in the dynamic knowledge base through the large language model, combined with the software constraints in the structured prompt word, the matching of candidate software versions and scene requirements is verified, and a compatibility verification report is output.

[0150] Based on the scene constraints in the structured prompt word and the industry benchmark test reports and historical deployment cases in the dynamic knowledge base, the performance prediction model is used to calculate the throughput, delay, and resource utilization of the candidate configuration, and the performance simulation result is output.

[0151] The hardware constraint verification conclusion, software compatibility verification report, and performance simulation result are fused to generate a configuration reasoning result including hardware configuration suggestions, software version lists, and performance expectation values.

[0152] In some embodiments, the configuration module is specifically configured to extract components that meet the minimum running requirements of hardware and the basic needs of scenes based on the hardware constraint verification conclusions and compatibility verification reports in the configuration reasoning result, and form a basic required module.

[0153] Based on the performance simulation result in the configuration reasoning result and the scene expansion requirement, based on the dynamic knowledge base, components that can improve performance or expand functions are screened to form an optional enhancement module.

[0154] Based on the hardware constraint verification conclusion, software version compatibility matrix, and performance simulation data in the configuration reasoning result, an interpretable attachment is generated, which includes parameter basis, performance expectation, and alternative options.

[0155] In some embodiments, the optimization module is specifically configured to collect running indicators of the modular configuration scheme in real time through a monitoring tool in the deployment environment, the running indicators including but not limited to single-card inference delay, system throughput, GPU utilization, GPU memory occupancy, hardware temperature, and exception logs.

[0156] User feedback is collected through a user interaction interface, a technical support ticket system, and a community forum, and the feedback content includes performance satisfaction, functional requirements, and fault descriptions.

[0157] The collected running indicators and user feedback are associated and analyzed to extract key optimization points, and combined with the actual effect of the configuration scheme, a structured optimization case including problem scenarios, configuration parameters, optimization measures, and effect verification is generated.

[0158] The optimization case is imported into the resource library through an external resource acquisition unit of the resource library, and after being verified based on the timeliness rule of the dynamic knowledge base, it is stored as a new knowledge item in the text database and the vector database.

[0159] At the same time, the old knowledge items in the resource library that conflict with the optimization case trigger a time decay factor algorithm, which reduces their weight or marks them as historical references, completing the dynamic update of the resource library.

[0160] The embodiment of the application also discloses a control device.

[0161] Specifically, the control device includes a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to perform the adaptive software stack optimization method based on the GPU server configuration.

[0162] The embodiment of the application also discloses a computer readable storage medium.

[0163] Specifically, the computer readable storage medium stores a computer program capable of being loaded and executed by the processor to perform the adaptive software stack optimization method based on the GPU server configuration, and the computer readable storage medium includes various program code storage media such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0164] The above are preferred embodiments of the application, and are not intended to limit the protection scope of the application, therefore: any equivalent changes made on the structure, shape, principle of the application shall be covered within the protection scope of the application.

Claims

1. An adaptive software stack optimization method based on GPU server configuration, characterized in that: include: Automatically collect the hardware parameters and scenario constraint parameters of the GPU server through hardware identification, and convert the hardware parameters and scenario constraint parameters into structured parameters through the multimodal perception capability of the large model center; Based on the preset keywords and the structured parameters, the resource library is called to perform semantic matching retrieval, and outdated knowledge is eliminated by combining the time decay factor algorithm to form a dynamic knowledge base adapted to the structured parameters; Based on the structural parameters and the dynamic knowledge base, assembling structured prompt words that can be reasoned by a large language model according to environmental constraints, best practices, and problems to be solved; Calling a large language model cluster to perform multi-stage reasoning based on the structured prompt words to generate a configuration reasoning result; Parsing the configuration reasoning result into a modular configuration scheme, wherein the modular configuration scheme includes basic mandatory modules, optional enhancement modules, and interpretable attachments; Based on the operational indicators and user feedback after the modular configuration solution is deployed, an optimization case is obtained, and the optimization case is updated to the resource library.

2. The method for adaptive software stack optimization based on GPU server configuration according to claim 1, characterized in that: The automatic collection of GPU server hardware parameters and scenario constraint parameters through hardware identification, and the conversion of the hardware parameters and scenario constraint parameters into structured parameters through the multimodal perception capability of the large model hub, include: The hardware parameters include GPU model, video memory capacity, MIG support capability, PCIe version and hardware topology; The scenario constraint parameters include inference latency requirements, system throughput requirements, model type and accuracy requirements; The multimodal perception capability of the large model hub integrates the real-time hardware status data collected by the hardware management interface, the text requirements input by the user, and similar scenario parameters in the historical configuration database, and uses a multimodal fusion model to convert the hardware parameters and the scenario constraint parameters into structured parameters in JSON format. The structured parameters include fields in three dimensions: hardware characteristics, scenario requirements, and constraints.

3. The method for adaptive software stack optimization based on GPU server configuration according to claim 2, characterized in that: The method of calling a resource library to perform semantic matching retrieval based on preset keywords and the structured parameters, eliminating obsolete knowledge in combination with a time decay factor algorithm, and forming a dynamic knowledge base adapted to the structured parameters includes: The enterprise platform is called through the internal component set of the data layer to obtain the chip manufacturer's recommended documents and pull the large model optimization container metadata; Collect user open source configuration cases from large model community repositories through external resource acquisition units, and obtain industry benchmark test reports through professional organization websites; Based on the structured parameters, accurate keyword retrieval is performed through the text database of the knowledge layer to obtain the MIG sharding strategy and software version compatibility matrix knowledge in the chip manufacturer's recommended documents; Perform semantic-level matching searches for "Large Model Low-Latency Inference Optimization" in the vector database to obtain performance benchmark data from the industry benchmark test report. The time decay factor algorithm is used to assign dynamic weights to knowledge items, and the weights of knowledge items that exceed the preset timeliness are automatically reduced or removed, ensuring that the knowledge base only retains the latest configuration solutions that are compatible with the current hardware; The MIG sharding strategy, the software version compatibility matrix knowledge, the performance benchmark data, and the latest configuration solution are integrated to form the dynamic knowledge base adapted to the structured parameters.

4. The method for adaptive software stack optimization based on GPU server configuration according to claim 3, characterized in that: The structured parameters and the dynamic knowledge base are based on environmental constraints, best practices, and problems to be solved to assemble structured prompt words that can be reasoned by a large language model, including: The environmental constraints include room temperature limits, power limits, and network bandwidth; The best practices include historical optimization cases, chip manufacturer recommended strategies, and industry standards stored in the dynamic knowledge base; The problems to be solved include conflicts in current configuration requirements, uncovered functional requirements and potential risks; By hierarchically assembling the hardware characteristics, scenario requirements, environmental constraints, best practices and unresolved problems of the structured parameters, a reasonable prompt word consisting of four parts: hardware context, requirement description, knowledge reference, and unresolved conflicts is formed for use by large language model clusters for multi-stage reasoning.

5. The method for adaptive software stack optimization based on GPU server configuration according to claim 4, characterized in that: The calling of the large language model cluster, performing multi-stage reasoning based on the structured prompt words, and generating a configuration reasoning result includes: Using the large language model and the structured prompt words, the NVML API is called to obtain the current GPU hardware status in real time, verify the matching of MIG sharding with video memory requirements, and output the hardware constraint verification conclusion; The software version compatibility matrix in the dynamic knowledge base is called by a large language model, and the matching of candidate software versions and scenario requirements is verified in combination with the software constraints in the structured prompt words, and a compatibility verification report is output; Based on the large language model, according to the scenario constraints in the structured prompt words and the industry benchmark test reports and historical deployment cases in the dynamic knowledge base, the throughput, latency and resource utilization of the candidate configuration are calculated through the performance prediction model, and the performance simulation results are output; The hardware constraint verification conclusion, the software compatibility verification report and the performance simulation result are integrated to generate the configuration reasoning result including hardware configuration suggestions, software version list and performance expected value.

6. The method for adaptive software stack optimization based on GPU server configuration according to claim 5, characterized in that: The configuration reasoning result is parsed into a modular configuration scheme, wherein the modular configuration scheme includes basic mandatory modules, optional enhancement modules, and interpretable attachments, including: Based on the hardware constraint verification conclusion and the compatibility verification report in the configuration reasoning result, extract components that meet the minimum hardware operating requirements and basic scenario requirements to form the basic mandatory modules; Based on the performance simulation results and scenario expansion requirements in the configuration reasoning results, components that can improve performance or expand functions are screened based on the dynamic knowledge base to form the optional enhancement module; Based on the hardware constraint verification conclusion, the software version compatibility matrix and the performance simulation data in the configuration reasoning result, an explainable attachment is generated, and the explainable attachment includes: parameter basis, performance expectation, and alternative options.

7. The method for adaptive software stack optimization based on GPU server configuration according to claim 6, characterized in that: Obtaining an optimization case based on the operational indicators and user feedback after the modular configuration solution is deployed, and updating the optimization case to the resource library, includes: The monitoring tools in the deployment environment are used to collect the operational indicators of the modular configuration solution in real time. The operational indicators include, but are not limited to, single-card inference latency, system throughput, GPU utilization, video memory occupancy, hardware temperature, and exception logs. Collect user feedback through the user interface, technical support ticket system, and community forums, including performance satisfaction, functional requirements, and fault descriptions; Perform correlation analysis on the collected operating indicators and user feedback, extract key optimization points, and generate a structured optimization case including problem scenarios, configuration parameters, optimization measures, and effect verification based on the actual effect of the configuration solution; Importing the optimization case into the resource library through the external resource acquisition unit of the resource library, and after verification based on the timeliness rules of the dynamic knowledge base, storing it as a new knowledge item in the text database and the vector database; At the same time, the time decay factor algorithm is triggered for old knowledge items in the resource library that conflict with the optimization case, and their weights are reduced or marked as historical references to complete the dynamic update of the resource library.

8. An adaptive software stack optimization device based on GPU server configuration, characterized in that: The device comprises: The acquisition module is used to automatically acquire the hardware parameters and scenario constraint parameters of the GPU server through hardware identification, and convert the hardware parameters and scenario constraint parameters into structured parameters through the multimodal perception capability of the large model center; A retrieval module is used to call a resource library to perform semantic matching retrieval based on preset keywords and the structured parameters, and to eliminate outdated knowledge in combination with a time decay factor algorithm to form a dynamic knowledge base adapted to the structured parameters; A prompt module, configured to assemble structured prompt words that can be reasoned by a large language model based on the structured parameters and the dynamic knowledge base according to environmental constraints, best practices, and problems to be solved; An inference module, configured to call a large language model cluster, perform multi-stage inference based on the structured prompt words, and generate a configuration inference result; A configuration module, configured to parse the configuration reasoning result into a modular configuration scheme, wherein the modular configuration scheme includes basic mandatory modules, optional enhancement modules, and interpretable attachments; The optimization module is used to obtain optimization cases based on the operational indicators and user feedback after the modular configuration solution is deployed, and update the optimization cases to the resource library.

9. A control device, characterized in that: The device comprises: The method comprises a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 7.

Images