Adaptive software stack optimization method and system based on GPU server configuration

Abstract

The application relates to an adaptive software stack optimization method and system based on a GPU server configuration, which comprises automatically collecting hardware parameters and scene constraint parameters of a GPU server through hardware recognition, converting the hardware parameters and the scene constraint parameters into structured parameters; based on preset keywords and the structured parameters, calling a resource library for semantic matching retrieval, eliminating obsolete knowledge by combining a time decay factor algorithm to form a dynamic knowledge base; based on the structured parameters and the dynamic knowledge base, assembling an environment constraint, an optimal practice and a problem to be solved into a structured prompt word that can be inferred by a large language model; performing multi-stage inference based on the structured prompt word to generate a configuration inference result; parsing the configuration inference result into a modular configuration scheme; based on running indexes and user feedback after deployment of the modular configuration scheme, obtaining an optimization case, and updating the optimization case to the resource library.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of large models, in particular to a self-adaptive software stack optimization method and system based on GPU server configuration. BACKGROUND

[0002] With the rapid development of large model technology and the continuous expansion of application scenarios, the software stack complexity of large models is growing exponentially. Currently, the software ecosystem of large models faces significant technical challenges: on the one hand, software components exhibit "multiple varieties and multiple versions" characteristics, covering operating systems, programming language runtimes, compilation tool chains, inference acceleration libraries, and other basic software, and compatibility requirements between versions of each component are strict; on the other hand, hardware (such as GPU) updates and iterations are short, and there are significant differences in computing power, memory configuration, and interface specifications between different generations of hardware. Under this background, ordinary users have difficulty accurately grasping software and hardware parameter matching rules, and even more so in efficiently filtering out the best configuration scheme that adapts to specific application scenarios.

[0003] In practical applications, large model application runtimes frequently experience functional abnormalities or substandard performance, which is mainly attributed to insufficient software environment adaptability: some applications rely on incompatible operating system versions and hardware drivers, or there are version conflicts between programming language runtime versions and model training frameworks, or even compilation tool chain configurations and hardware instruction sets do not match; more critically, some core software components (such as CUDA, TensorRT, etc.) have strong dependencies on hardware features (such as GPU memory capacity and MIG slicing capabilities), and when hardware parameters do not meet the requirements, the application cannot be started or crashes.

[0004] Based on this, the application provides a self-adaptive software stack optimization method and system based on GPU server configuration. SUMMARY

[0005] To improve the problem of frequent functional abnormalities or substandard performance of large model application runtimes, the application provides a self-adaptive software stack optimization method and system based on GPU server configuration.

[0006] In a first aspect, the application provides a self-adaptive software stack optimization method based on GPU server configuration, which adopts the following technical solution: comprising:

[0007] The hardware parameters and scenario constraint parameters of the GPU server are automatically collected through hardware recognition, and the hardware parameters and scenario constraint parameters are converted into structured parameters through the multi-modal perception capability of the large model hub;

[0008] Based on the preset keywords and the structured parameters, a resource library is called to perform semantic matching retrieval, outdated knowledge is eliminated by combining a time decay factor algorithm, and a dynamic knowledge base adapted to the structured parameters is formed;

[0009] Based on the structured parameters and the dynamic knowledge base, structured prompt words that can be inferred by a large language model are assembled according to environmental constraints, best practices and problems to be solved;

[0010] A large language model cluster is called to perform multi-stage inference based on the structured prompt words, and a configuration inference result is generated;

[0011] The configuration inference result is parsed into a modular configuration scheme, which includes a basic required module, an optional enhancement module and an interpretable accessory;

[0012] Based on the running indicators and user feedback after deployment of the modular configuration scheme, an optimization case is obtained, and the optimization case is updated to the resource library.

[0013] Preferably, the hardware parameters and scene constraint parameters of the GPU server are automatically collected through hardware recognition, and the hardware parameters and scene constraint parameters are converted into structured parameters through the multi-modal perception ability of the large model hub, including:

[0014] The hardware parameters include GPU model, video memory capacity, MIG support capability, PCIe version and hardware topology structure;

[0015] The scene constraint parameters include inference delay requirement, system throughput demand, model type and accuracy requirement;

[0016] The multi-modal perception ability of the large model hub converts the hardware parameters and the scene constraint parameters into structured parameters in JSON format by fusing real-time hardware state data collected through a hardware management interface, user input text requirements and similar scene parameters in a historical configuration database, and using a multi-modal fusion model. The structured parameters include fields in three dimensions of hardware characteristics, scene requirements and constraint conditions.

[0017] Preferably, based on the preset keywords and the structured parameters, a resource library is called to perform semantic matching retrieval, outdated knowledge is eliminated by combining a time decay factor algorithm, and a dynamic knowledge base adapted to the structured parameters is formed, including:

[0018] The internal component set of the data layer calls an enterprise platform to obtain chip original factory recommended documents, and pulls large model optimization container metadata;

[0019] The user open source configuration case is collected from a large model community warehouse by an external resource acquisition unit, and an industry benchmark test report is obtained from a professional agency website;

[0020] The structured parameters are used to perform accurate keyword retrieval on a text database of a knowledge layer to obtain MIG fragmentation strategies and software version compatibility matrix knowledge in a chip original factory recommended document;

[0021] The vector database is used to perform semantic level matching retrieval of "large model low latency inference optimization" to obtain performance benchmark data in the industry benchmark test report;

[0022] The time decay factor algorithm is used to assign dynamic weights to knowledge items, and the weights of knowledge items exceeding a preset time limit are automatically reduced or removed to ensure that the knowledge base only retains the latest configuration scheme adapted to the current hardware;

[0023] The MIG fragmentation strategies, the software version compatibility matrix knowledge, the performance benchmark data, and the latest configuration scheme are fused to form the dynamic knowledge base adapted to the structured parameters.

[0024] Preferably, based on the structured parameters and the dynamic knowledge base, a structured prompt word that can be inferred by a large language model is assembled according to environmental constraints, best practices, and problems to be solved, including:

[0025] The environmental constraints include room temperature limits, power supply power limits, and network bandwidths;

[0026] The best practices include historical optimization cases, chip original factory recommended strategies, and industry standards stored in the dynamic knowledge base;

[0027] The problems to be solved include conflict points in current configuration requirements, unmet functional requirements, and potential risks;

[0028] The hardware characteristics, scene requirements, environmental constraints, best practices, and problems to be solved of the structured parameters are hierarchically assembled to form an inferable prompt word containing four parts of hardware context, requirement description, knowledge reference, and conflict to be solved, for use by a large language model cluster for multi-stage inference.

[0029] Preferably, the large language model cluster is called to perform multi-stage inference based on the structured prompt word to generate a configuration inference result, including:

[0030] The current GPU hardware state is obtained in real time by the large language model based on the structured prompt word calling the NVMLAPI, the matching of MIG fragmentation and video memory requirements is verified, and a hardware constraint verification conclusion is output;

[0031] The software version compatibility matrix in the dynamic knowledge base is called by a large language model, combined with software constraints in the structured prompt words, to verify the matching of candidate software versions and scene requirements, and a compatibility verification report is output;

[0032] Based on the scene constraints in the structured prompt words and the industry benchmark test reports and historical deployment cases in the dynamic knowledge base, a performance prediction model is used to calculate the throughput, delay and resource utilization of the candidate configuration, and a performance simulation result is output;

[0033] The hardware constraint verification conclusion, the compatibility verification report and the performance simulation result are fused to generate the configuration reasoning result including hardware configuration suggestions, software version lists and performance expectation values.

[0034] Preferably, the configuration reasoning result is parsed into a modular configuration scheme, and the modular configuration scheme includes a basic required module, an optional enhancement module and an interpretable accessory, which includes:

[0035] Based on the hardware constraint verification conclusion and the compatibility verification report in the configuration reasoning result, components that meet the minimum hardware operation requirements and scene basic requirements are extracted to form the basic required module;

[0036] Based on the performance simulation result and scene expansion requirements in the configuration reasoning result, components that can improve performance or expand functions are selected based on the dynamic knowledge base to form the optional enhancement module;

[0037] Based on the hardware constraint verification conclusion, the software version compatibility matrix and the performance simulation data in the configuration reasoning result, an interpretable accessory is generated, which includes parameter basis, performance expectation and alternative options.

[0038] Preferably, based on the running indicators and user feedback after deployment of the modular configuration scheme, an optimization case is obtained, and the optimization case is updated to the resource library, which includes:

[0039] The running indicators of the modular configuration scheme are collected in real time by a monitoring tool in the deployment environment, and the running indicators include but are not limited to single-card inference delay, system throughput, GPU utilization, video memory occupancy, hardware temperature and abnormal logs;

[0040] The feedback content of the user is collected through a user interaction interface, a technical support work order system and a community forum, and the feedback content includes performance satisfaction, function demand and fault description;

[0041] The collected operation indicators and the user feedback are associated and analyzed to extract key optimization points, and a structured optimization case including a problem scenario, a configuration parameter, an optimization measure and an effect verification is generated based on the actual effect of the configuration scheme;

[0042] The optimization case is imported into the resource library through an external resource acquisition unit of the resource library, and after being verified based on the timeliness rule of the dynamic knowledge base, is stored as a new knowledge item in the text database and the vector database;

[0043] Meanwhile, time decay factor algorithm is triggered for old knowledge items in the resource library that conflict with the optimization case, and the weight of the old knowledge items is reduced or the old knowledge items are marked as historical references, so as to complete dynamic updating of the resource library.

[0044] In a second aspect, an adaptive software stack optimization device based on a GPU server configuration is provided, which adopts the following technical scheme, comprising:

[0045] A collection module is configured to automatically collect hardware parameters and scene constraint parameters of a GPU server through hardware recognition, and convert the hardware parameters and the scene constraint parameters into structured parameters through multi-modal perception capability of a large model hub;

[0046] A retrieval module is configured to perform semantic matching retrieval by calling a resource library based on preset keywords and the structured parameters, eliminate outdated knowledge by combining a time decay factor algorithm, and form a dynamic knowledge base adapted to the structured parameters;

[0047] A prompt module is configured to assemble structured prompt words that can be inferred by a large language model according to environmental constraints, best practices and problems to be solved based on the structured parameters and the dynamic knowledge base;

[0048] An inference module is configured to call a large language model cluster, perform multi-stage inference based on the structured prompt words, and generate a configuration inference result;

[0049] A configuration module is configured to parse the configuration inference result into a modular configuration scheme, wherein the modular configuration scheme includes a basic required module, an optional enhancement module and an interpretable accessory;

[0050] An optimization module is configured to obtain an optimization case based on operation indicators and user feedback after deployment of the modular configuration scheme, and update the optimization case to the resource library.

[0051] In a third aspect, the application further provides a control device, which comprises:

[0052] The device includes a memory and a processor, and the memory stores a computer program capable of being loaded and executed by the processor to perform the adaptive software stack optimization method based on the GPU server configuration as described above.

[0053] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program capable of being loaded and executed by a processor to perform the adaptive software stack optimization method based on GPU server configuration as described above.

[0054] In summary, in the present application, GPU hardware parameters and scene constraint parameters are automatically collected and structured, a reasoning prompt word is generated in combination with a resource library dynamic knowledge base, a large model cluster is called to perform multi-stage reasoning and output a configuration result, and the result is parsed into a modular solution and continuously optimized through running indicators and user feedback. The resource library solves the problem that users are difficult to match the best configuration under the condition of complex large model software stack, numerous versions, and rapid iteration of hardware. Through dynamic knowledge updating, multi-modal perception, and large model reasoning, intelligent adaptation of software and hardware parameters is achieved, the configuration difficulty is reduced, application running errors caused by environmental incompatibility are reduced, and the stability and usability of large model applications are improved. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a schematic diagram of the hierarchical architecture and core components of the system.

[0056] Figure 2 is a flowchart of an adaptive software stack optimization method based on GPU server configuration.

[0057] Figure 3 is a flowchart of software stack optimization.

[0058] Figure 4 is a structural block diagram of an adaptive software stack optimization device based on GPU server configuration. DETAILED DESCRIPTION

[0059] The following will be described in detail in combination with Figure 1 - Figure 4 The present application will be further described in detail.

[0060] Reference is made to Figure 1The present application aims at the pain points of "version complexity, hardware adaptation difficulty, and configuration black box" of large model software stack, and proposes a technical architecture of "bottom-up layered decoupling + inter-layer collaborative closed loop". Through full-link collaboration of infrastructure layer (hardware-system base) → data layer (resource aggregation) → knowledge layer (storage + generative enhancement) → intelligent recommendation layer (large model central inference) → recommended content layer (modular output), with the help of deep integration of large language model (LLM) cluster and retrieval enhancement generation (RAG) technology, the hardware characteristic perception → knowledge dynamic activation → intelligent inference decision → configuration interpretable output of software and hardware collaborative optimization is realized, and the full-link technical solution from hardware base to AI application recommendation is realized, providing a high-performance and high-reliability software stack configuration solution for large model scenarios (taking H100 GPU deployment Evo2 inference service as an example).

[0061] The system adopts a "five-layer decoupling + inter-layer collaboration" architecture, and the functions and core components of each layer are as follows:

[0062] 1. Infrastructure layer (bottom running support unit)

[0063] Provide "physical-software base" for the system, including:

[0064] Hardware specification configuration module: define CPU, memory, storage, etc. Basic computing power (such as CPU multi-core scheduling of H100 server, memory bandwidth optimization);

[0065] Firmware version module: load firmware program (such as NvidiaBOOTROM) adapted to H100, realize software call of hardware function;

[0066] Operating system module: deploy Ubuntu22.04 OS, etc. Complete software-level scheduling of H100 resources (such as NVMe storage IO optimization).

[0067] 2. Data layer (resource aggregation and input unit)

[0068] Responsible for "internal and external resource aggregation-supply" function, providing AI ecological tools and external resources for the upper layer:

[0069] Internal component set: integrate Nvidia-ai-enterprise (Nvidia AI enterprise platform), ngc (Nvidia resource platform), nim (Nvidia inference microservice), etc.

[0070] External resource acquisition unit: through two channels of official website / github (official technical community) and professional institutions / self-maintenance (custom or third-party resource library), realize collection and access of external technical resources.

[0071] 3. Knowledge layer (data storage and enhanced processing unit)

[0072] Build the core knowledge base of "storage + generative enhancement", and realize the dynamic management of knowledge through "storage + generative enhancement" technology:

[0073] Database (DB): Use relational / non-relational databases (such as MySQL, MongoDB, etc.) to realize the persistent storage and retrieval of structured data;

[0074] Retrieval and enhancement generation module (RAG): Through the technical path of "data retrieval → generative AI processing → knowledge enhancement output", the semantic expansion and intelligent enhancement of DB stored data are realized, and dynamic knowledge support is provided for the intelligent recommendation layer.

[0075] 4. Intelligent recommendation layer (core reasoning and interaction unit)

[0076] System "intelligent center", based on knowledge layer enhanced knowledge to realize AI recommendation logic and external interaction:

[0077] Interaction portal (Portal): As the unified entrance of the system's external interaction, it supports user request receiving, recommendation result output and other interactive operations, such as receiving user "4 card H100 deployment Evo2 reasoning" and other requirements;

[0078] Large language model cluster (LLM): Deploy multiple large language models (such as GPT, open source LLM, etc.), and the models interact with each other to form collaborative reasoning capabilities; At the same time, the LLM cluster and the DB and RAG modules of the knowledge layer are bidirectionally linked - both obtaining knowledge input from DB / RAG to drive reasoning, and feeding back the reasoning generation results to the knowledge layer (or generating recommendation instructions based on knowledge layer data).

[0079] 5. Recommended content layer (output carrier and expansion unit)

[0080] The final output carrier of the intelligent recommendation layer, using "basic required + optional enhancement" modular design to adapt to different scenarios:

[0081] Basic recommendation module (framework-level required component): Includes PyTorch (AI computing framework), hardware driver (hardware adaptation program), cuda software stack (GPU computing acceleration tool chain), and key category api / sdk / runtime / library (general tool set), which constitutes the core foundation of recommended content;

[0082] Optional expansion module (flexible enhancement component): Set Colossal-AI (distributed training framework) and "other" reserved expansion bit, support loading additional enhancement components (such as domain-specific tools, self-developed modules, etc.) according to scene requirements.

[0083] Reference Figure 2 and Figure 3 The embodiments of this application include at least steps S10 to S60.

[0084] S10 automatically collects hardware parameters and scene constraint parameters from the GPU server through hardware recognition, and transforms these parameters into structured parameters through the multimodal perception capability of the large model hub.

[0085] S20, based on preset keywords and structured parameters, calls the resource library for semantic matching retrieval, and combines the time decay factor algorithm to eliminate outdated knowledge, forming a dynamic knowledge base adapted to the structured parameters.

[0086] S30, based on structured parameters and a dynamic knowledge base, is assembled into structured prompts for reasoning within a large language model according to environmental constraints, best practices, and unsolved problems.

[0087] S40 invokes a large language model cluster to perform multi-stage reasoning based on structured prompts and generates configuration reasoning results.

[0088] S50 parses the configuration inference results into modular configuration schemes, which include basic mandatory modules, optional enhancement modules, and interpretable appendices.

[0089] Based on the operational metrics and user feedback after deployment of S60 using a modular configuration scheme, optimized cases are obtained and updated to the resource library.

[0090] Specifically, the system automatically collects and structures GPU server hardware parameters and scene constraint parameters through hardware recognition. It then dynamically updates the knowledge base using semantic retrieval from the resource library and a time decay algorithm. Based on environmental constraints, best practices, and unsolved problems, it assembles reasoning prompts for the large model. The system then calls upon the large model cluster for multi-stage inference to generate configuration results, which are parsed into a modular solution containing mandatory and optional modules and interpretable attachments. The resource library is continuously optimized based on post-deployment performance metrics and user feedback. Its purpose is to address the challenges of matching optimal configurations to large model software stacks, numerous versions, and rapid hardware iterations. Through dynamic knowledge updates, multimodal perception, and intelligent reasoning from the large model, it achieves precise adaptation of software and hardware parameters, lowers the configuration threshold, reduces application errors caused by environmental incompatibility, and improves the stability and usability of large model applications.

[0091] In some embodiments, step S10 specifically includes the following steps: hardware parameters include GPU model, video memory capacity, MIG support capability, PCIe version, and hardware topology; scene constraint parameters include inference latency requirements, system throughput requirements, model type, and accuracy requirements; the multimodal perception capability of the large model hub transforms the hardware parameters and scene constraint parameters into structured parameters in JSON format by fusing real-time hardware status data collected from the hardware management interface, user-inputted text requirements, and similar scene parameters in the historical configuration database, using a multimodal fusion model. The structured parameters contain fields in three dimensions: hardware characteristics, scene requirements, and constraints.

[0092] In practice, when a user initiates a request to "deploy Evo2 model inference service on a 4-card H100-SXM5-80GB GPU server, requiring single-card inference latency <50ms and throughput ≥200 tokens / second":

[0093] Hardware dimension: The intelligent recommendation layer automatically collects H100 core parameters through NVML / CUDAAPI: GPU model (H100-SXM5-80GB), driver version, CUDA computing power level, NVLink topology, and hardware characteristics; Scenario dimension: Collects Evo2 model constraints (model parameter count ≈ 100B, input sequence length 512, output sequence length 256, service QPS ≥ 100).

[0094] Specifically, by leveraging the multimodal perception capabilities of the large model hub, "hardware physical features (such as FP8 acceleration of H100)" and "scenario business constraints (such as Evo2 inference latency)" are transformed into structured parameters that LLM can understand, providing accurate input for subsequent inference.

[0095] In some embodiments, step S20 specifically includes the following steps: obtaining chip manufacturer recommendation documents by calling the enterprise platform through the internal component set of the data layer, and pulling metadata of the large model optimization container; collecting user open-source configuration cases from the large model community repository through the external resource acquisition unit, and obtaining industry benchmark test reports through professional institution websites; performing precise keyword retrieval through the text database of the knowledge layer based on structured parameters to obtain MIG sharding strategies and software version compatibility matrix knowledge from the chip manufacturer recommendation documents; performing semantic-level matching retrieval of "large model low-latency inference optimization" through the vector database to obtain performance benchmark data from the industry benchmark test report; assigning dynamic weights to knowledge items through the time decay factor algorithm, automatically reducing the weight or removing knowledge items that exceed the preset time limit, ensuring that the knowledge base only retains the latest configuration scheme adapted to the current hardware; and integrating the MIG sharding strategy, software version compatibility matrix knowledge, performance benchmark data, and the latest configuration scheme to form a dynamic knowledge base adapted to the structured parameters.

[0096] In practice, the system triggers a collaborative retrieval process involving both the data layer and the knowledge layer:

[0097] Data layer resource access:

[0098] Internal components: Call Nvidia-ai-enterprise to obtain the "H100 Inference Optimization Guide" (including the TransformerEngine deployment manual), and pull the official Evo2 optimization container (with built-in TensorRT-LLM inference plugin) from ngc.

[0099] External resources: Collected open-source inference configuration examples from GitHub (Evo2 community repository) and obtained the "H100 Inference Performance Benchmark Report (ID:20250315)" from the MLPerf official website.

[0100] Knowledge-layer dual-mode retrieval:

[0101] Text database (Elasticsearch): Based on the keyword "Evo2H100 inference configuration", retrieve "Multi-Instance GPU (MIG) Sharding Strategy (4 cards each split into 2×20GB instances to adapt to Evo2 memory requirements)" and "TensorRT-LLM and CUDA 12.3 Compatibility Matrix" from Nvidia's official documentation;

[0102] Vector Database (FAISS, Sentence-BERT Semantic Embedding): Based on semantic matching, “Large Model Low-Latency Inference Optimization”, recalling the “H100+TensorRT-LLM with Optimal Latency at batch_size=32” in the MLPerf report.

[0103] Dynamic update mechanism in effect: The knowledge layer automatically filters out CUDA 12.2 and earlier configurations from 3 months ago using a time decay factor algorithm (because H100 has better support for CUDA 12.3+, the performance of older versions is attenuated by ≥20%), ensuring the timeliness of recommended knowledge.

[0104] Specifically, by combining "multi-source aggregation at the data layer + RAG dual-mode retrieval at the knowledge layer + dynamic knowledge elimination," the pain points of "rapid version iteration and fragmented knowledge" in large model software stacks are addressed, providing an "authoritative, real-time, and scenario-adaptive" knowledge foundation for LLM inference.

[0105] In some embodiments, step S30 specifically includes the following steps: environmental constraints include data center temperature limits, power limits, and network bandwidth; best practices include historical optimization cases stored in the dynamic knowledge base, chip manufacturer recommended strategies, and industry standards; unresolved issues include conflict points in the current configuration requirements, uncovered functional requirements, and potential risks; by assembling the hardware characteristics, scenario requirements, environmental constraints, best practices, and unresolved issues of the structured parameters in a hierarchical manner, a reasonable prompt word containing four parts—hardware context, requirement description, knowledge reference, and unresolved conflicts—is formed for use by the large language model cluster for multi-stage reasoning.

[0106] In practice, the intelligent recommendation layer automatically assembles "hardware parameters, scene constraints, and knowledge base retrieval results" into structured prompts that can be reasoned by the LLM, based on the template of "environmental constraints + best practices + unsolved problems".

[0107] Example: Known hardware: 4×H100-SXM5-80GB (CUDA 12.3, driver 545.23.08, NVLink bandwidth 800GB / s, MIG supports 20GB sharding; hardware features: TransformerEngine enabled, FP8 precision supported).

[0108] Scenario Requirements: Evo2 model inference service (model parameter count ≈ 100B, input seq_len = 512, output seq_len = 256, single-card latency < 50ms, QPS ≥ 100).

[0109] Existing knowledge: ① Nvidia official recommendation: H100 inference recommends enabling MIG (split 4 cards into 2×20GB instances each to reduce VRAM fragmentation), TensorRT-LLM needs to be compatible with CUDA 12.3 (TensorRT-LLM 0.9.0 is recommended); ② MLPerf report: H100 + TensorRT-LLM with batch_size=32 has a latency of 45ms and a throughput of 220 tokens / second (meeting QPS requirements); ③ Community practice: Evo2 inference requires enabling PyTorch 2.2 (including 2.0+ compilation optimization) + FasterTransformer plugin.

[0110] Please generate a configuration scheme that meets the following constraints: hardware resource utilization ≥ 90% (achieved through MIG sharding and NVLink topology optimization); software version compatibility (e.g., TensorRT-LLM and CUDA, driver version matching); performance indicators meet the requirements (single card latency < 50ms, throughput ≥ 200 tokens / second); the scheme must include "parameter basis, performance expectations, and alternative options" (to enhance interpretability).

[0111] Specifically, by using the "environmental parameters → prompt word automatic conversion rule", fragmented knowledge is transformed into the "decision context" of LLM, which both constrains the reasoning direction (such as the MIG strategy of H100) and opens up optimization space (such as the fine-tuning of batch_size), thus achieving a balance between "rule constraints + intelligent reasoning".

[0112] In some embodiments, step S40 specifically includes the following steps: using a large language model to call the NVML API based on structured prompts to obtain the current GPU hardware status in real time, verifying the matching between MIG sharding and memory requirements, and outputting hardware constraint verification conclusions; using a large language model to call the software version compatibility matrix in the dynamic knowledge base, combined with the software constraints in the structured prompts, verifying the matching between candidate software versions and scenario requirements, and outputting a compatibility verification report; based on the large language model and the scenario constraints in the structured prompts, as well as industry benchmark test reports and historical deployment cases in the dynamic knowledge base, calculating the throughput, latency, and resource utilization of candidate configurations through a performance prediction model, and outputting performance simulation results; and integrating the hardware constraint verification conclusions, compatibility verification report, and performance simulation results to generate configuration inference results containing hardware configuration recommendations, software version lists, and expected performance values.

[0113] In practice, the intelligent recommendation layer calls upon an LLM cluster (such as industry-customized models + open-source models working together) to perform chained reasoning based on the aforementioned prompts:

[0114] Phase 1: Verify hardware constraints (e.g., whether the video memory after MIG fragmentation meets the Evo2 inference requirements: 2×20GB MIG instance → 20GB video memory per instance, Evo2 inference video memory peak of 18GB → meets the requirements).

[0115] Phase 2: Matching software versions (TensorRT-LLM0.9.0 compatibility verification with CUDA 12.3 and driver 545.23.08 → Nvidia official documentation confirms compatibility);

[0116] Phase 3: Performance simulation (Based on MLPerf data, it is estimated that when batch_size=32, the throughput of a 4-card cluster = 4 × 220 = 880 tokens / second → meets the requirement of QPS≥100).

[0117] Specifically, by using multi-LLM collaborative reasoning, leveraging the "hardware understanding capabilities (such as Nvidia's custom LLM)" and "scenario optimization experience (such as industry-specific LLM)" of different models, the problem of "insufficient knowledge coverage and reasoning bias" of a single model can be solved, ensuring the technical feasibility of the solution.

[0118] In some embodiments, step S50 specifically includes the following steps: based on the hardware constraint verification conclusions and compatibility verification reports in the configuration inference results, extract components that meet the minimum hardware operating requirements and basic scenario needs to form basic mandatory modules; based on the performance simulation results and scenario expansion requirements in the configuration inference results, select components that can improve performance or expand functions based on the dynamic knowledge base to form optional enhancement modules; based on the hardware constraint verification conclusions, software version compatibility matrix and performance simulation data in the configuration inference results, generate interpretable appendices, which include: parameter basis, performance expectations, and alternative options.

[0119] In practice, after LLM inference generates the configuration scheme, it is recommended that the content layer be parsed into a modular output of "basic mandatory + optional enhancement + interpretable attachments":

[0120] Essential modules: CUDA 12.3 (H100 hardware acceleration foundation), PyTorch 2.2 (including FasterTransformer plugin, supporting Evo2 inference optimization), Nvidia driver 545.23.08 (adapted to H100 MIG features), TensorRT-LLM 0.9.0 (core inference acceleration tool).

[0121] Optional enhancement modules: Colossal-AI (enables the InferenceServing module to optimize multi-card load balancing), TritonInferenceServer (enables dynamic model scheduling to cope with QPS fluctuations);

[0122] Explainable attachments: Parameter basis (Nvidia MIG documentation section 6.2, MLPerf report ID#20250315, Evo2 community configuration case commit#abc123); Performance expectations (single-card latency 42ms, throughput 225 tokens / second, total throughput of 4-card cluster 900 tokens / second, resource utilization 92%); Alternative options (if latency exceeds the limit, downgrade MIG to a 1×40GB instance, enable PyTorch 2.2 INT8 quantization, expected latency 48ms, throughput 190 tokens / second). Deep logic: Through modular design and enhanced explainability, it satisfies both the requirements of "essential and stable basic functions" and "flexible optional enhancements for specific scenarios," while providing users with "technical decision-making basis" (e.g., why choose TensorRT-LLM0.9.0) and "risk contingency plans" (e.g., downgrade solutions when latency exceeds the limit), addressing the pain points of traditional configuration recommendations being "black box and unreliable."

[0123] In some embodiments, step S60 specifically includes the following steps: Real-time collection of operational metrics of the modular configuration scheme using monitoring tools in the deployment environment. These metrics include, but are not limited to, single-card inference latency, system throughput, GPU utilization, video memory usage, hardware temperature, and anomaly logs. User feedback is collected through user interaction interfaces, technical support ticket systems, and community forums. This feedback includes performance satisfaction, functional requirements, and fault descriptions. Correlation analysis is performed on the collected operational metrics and user feedback to extract key optimization points. Combined with the actual effects of the configuration scheme, a structured optimization case containing problem scenarios, configuration parameters, optimization measures, and effect verification is generated. The optimization case is imported into the resource library through the external resource acquisition unit. After verification based on the timeliness rules of the dynamic knowledge base, it is stored as a new knowledge item in the text database and vector database. Simultaneously, a time decay factor algorithm is triggered for old knowledge items in the resource library that conflict with the optimization case, reducing their weight or marking them as historical references, thus completing the dynamic update of the resource library.

[0124] In practice, after users deploy the Evo2 inference service according to the recommended plan, the system automatically collects runtime metrics (such as actual single-card latency of 40ms, throughput of 230 tokens / second, and resource utilization of 95%), and combines this with user feedback ("Inference stability improved after MIG sharding, with no memory overflow errors"), triggering the case submission process in the data layer and manual collaboration layer.

[0125] A user submits a "4×H100+Evo2 inference optimization case." After review, the knowledge layer triggers a hot update, adding the case to the text / vector database. The knowledge weight algorithm assigns a high time decay factor weight (preferred recommendation) to this "recently effective case," while automatically reducing the weight of older solutions from one year ago ("non-H100 architecture"), ensuring the knowledge base continuously adapts to hardware iterations. Deeper logic: Through a closed loop of "user practice feedback → knowledge base evolution," the system "learns from industry best practices and evolves towards personalized user needs," solving the industry challenge of "rapid technology iteration and general solutions not adapting to business needs" in large model software stacks.

[0126] The implementation principle of an adaptive software stack optimization method based on GPU server configuration in this application is as follows: First, GPU hardware parameters and scene constraint parameters are automatically collected and structured. Combined with a dynamic knowledge base of the resource library, inference prompts are generated. The configuration results are then output through multi-stage inference of a large model cluster. After parsing into a modular solution, the resource library is continuously optimized based on operational metrics and user feedback. This solves the problems of complex large model software stacks, numerous versions, and rapid hardware iteration making it difficult for users to match optimal configurations. Through dynamic knowledge updates, multimodal perception, and large model inference, intelligent adaptation of software and hardware parameters is achieved, reducing configuration difficulty, minimizing application runtime errors caused by environmental incompatibility, and improving the stability and usability of large model applications.

[0127] Figure 2 This is a flowchart illustrating an adaptive software stack optimization method based on GPU server configuration in one embodiment. It should be understood that, although... Figure 2 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows; unless explicitly stated herein, there is no strict order requirement for the execution of these steps, and they can be executed in other orders; and Figure 2 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0128] Based on the same technical concept, referring to Figure 4 This application also provides an adaptive software stack optimization device based on GPU server configuration, which adopts the following technical solution: The device includes:

[0129] The acquisition module is used to automatically acquire hardware parameters and scene constraint parameters of the GPU server through hardware recognition, and to convert the hardware parameters and scene constraint parameters into structured parameters through the multimodal perception capability of the large model hub.

[0130] The retrieval module is used to call the resource library for semantic matching retrieval based on preset keywords and structured parameters, and combine the time decay factor algorithm to eliminate outdated knowledge, forming a dynamic knowledge base that adapts to the structured parameters.

[0131] The prompt module is used to assemble structured prompt words into reasonable prompts for a large language model based on structured parameters and a dynamic knowledge base, according to environmental constraints, best practices, and unsolved problems.

[0132] The reasoning module is used to call a large language model cluster, perform multi-stage reasoning based on structured prompts, and generate the configured reasoning results.

[0133] The configuration module is used to parse the configuration inference results into modular configuration schemes, which include basic mandatory modules, optional enhancement modules, and interpretable attachments.

[0134] The optimization module is used to obtain optimization cases based on the operational metrics and user feedback after deployment based on the modular configuration scheme, and then update the optimization cases to the resource library.

[0135] In some embodiments, the acquisition module is specifically used to collect hardware parameters including GPU model, video memory capacity, MIG support capability, PCIe version, and hardware topology.

[0136] Scenario constraints include inference latency requirements, system throughput requirements, model type, and accuracy requirements;

[0137] The multimodal perception capability of the large model hub integrates real-time hardware status data collected from the hardware management interface, textual requirements input by users, and similar scenario parameters from the historical configuration database. It uses a multimodal fusion model to transform hardware parameters and scenario constraint parameters into structured parameters in JSON format. The structured parameters contain fields in three dimensions: hardware characteristics, scenario requirements, and constraints.

[0138] In some embodiments, the retrieval module is specifically used to call the enterprise platform through the internal component set of the data layer to obtain the chip manufacturer's recommended documents and pull the metadata of the large model optimization container;

[0139] We collect open-source configuration examples from large model community repositories through external resource acquisition units, and obtain industry benchmark test reports from professional organization websites.

[0140] Based on structured parameters, precise keyword retrieval is performed through a text database of the knowledge layer to obtain knowledge of MIG sharding strategies and software version compatibility matrices from the chip manufacturer's recommended documents.

[0141] Semantic-level matching retrieval for "large model low-latency inference optimization" is performed using a vector database to obtain performance benchmark data from industry benchmark test reports.

[0142] Dynamic weights are assigned to knowledge items using a time decay factor algorithm. Knowledge items that exceed the preset time limit are automatically reduced in weight or removed, ensuring that the knowledge base retains only the latest configuration scheme that is compatible with the current hardware.

[0143] By integrating MIG sharding strategies, software version compatibility matrix knowledge, performance benchmark data, and the latest configuration schemes, a dynamic knowledge base adapted to structured parameters is formed.

[0144] In some embodiments, the prompting module is specifically used for environmental constraints including computer room temperature limits, power limits, and network bandwidth;

[0145] Best practices include historical optimization cases, chip manufacturer recommendations, and industry standards stored in the dynamic knowledge base;

[0146] Unresolved issues include conflicts in current configuration requirements, uncovered functional requirements, and potential risks;

[0147] By assembling the hardware characteristics, scenario requirements, environmental constraints, best practices, and unresolved problems of structured parameters in a hierarchical manner, inference prompts containing four parts—hardware context, requirement description, knowledge reference, and unresolved conflicts—are generated for use by large language model clusters in multi-stage inference.

[0148] In some embodiments, the inference module is specifically used to call the NVMLAPI based on structured prompts through a large language model to obtain the current GPU hardware status in real time, verify the matching between MIG sharding and memory requirements, and output hardware constraint verification conclusions.

[0149] By calling the software version compatibility matrix in the dynamic knowledge base through the large language model and combining it with the software constraints in the structured prompts, the matching of candidate software versions and scenario requirements is verified, and a compatibility verification report is output.

[0150] Based on the large language model, the throughput, latency, and resource utilization of candidate configurations are estimated through the performance prediction model, taking into account the scenario constraints in the structured prompts and the industry benchmark test reports and historical deployment cases in the dynamic knowledge base, and the performance simulation results are output.

[0151] By integrating hardware constraint verification conclusions, compatibility verification reports, and performance simulation results, a configuration inference result is generated that includes hardware configuration recommendations, a software version list, and expected performance values.

[0152] In some embodiments, the configuration module is specifically used to extract components that meet the minimum hardware operating requirements and basic scenario requirements based on the hardware constraint verification conclusions and compatibility verification reports in the configuration inference results, forming basic mandatory modules;

[0153] Based on the performance simulation results and scenario expansion requirements in the configuration inference results, components that can improve performance or expand functions are selected based on the dynamic knowledge base to form optional enhancement modules;

[0154] Based on the hardware constraint verification conclusions, software version compatibility matrix, and performance simulation data in the configuration inference results, an interpretable appendix is ​​generated. The interpretable appendix includes: parameter basis, performance expectation, and alternative options.

[0155] In some embodiments, the optimization module is specifically used to collect the operating metrics of the modular configuration scheme in real time through monitoring tools in the deployment environment. The operating metrics include, but are not limited to: single-card inference latency, system throughput, GPU utilization, video memory usage, hardware temperature, and abnormal logs.

[0156] User feedback is collected through user interaction interfaces, technical support ticket systems, and community forums. The feedback includes performance satisfaction, functional requirements, and fault descriptions.

[0157] Correlation analysis is performed on the collected operational metrics and user feedback to extract key optimization points. Combined with the actual effects of the configuration scheme, a structured optimization case is generated, which includes problem scenarios, configuration parameters, optimization measures, and effect verification.

[0158] The optimized cases are imported into the resource library through the external resource acquisition unit. After being verified based on the timeliness rules of the dynamic knowledge base, they are stored as new knowledge items in the text database and vector database.

[0159] Simultaneously, the time decay factor algorithm is triggered for old knowledge items in the resource library that conflict with the optimization cases, reducing their weight or marking them as historical references to complete the dynamic update of the resource library.

[0160] This application also discloses a control device.

[0161] Specifically, the control device includes a memory and a processor, the memory storing a computer program that can be loaded by the processor and executed using the aforementioned adaptive software stack optimization method based on GPU server configuration.

[0162] This application also discloses a computer-readable storage medium.

[0163] Specifically, the computer-readable storage medium stores a computer program that can be loaded by a processor and executed as described above in the adaptive software stack optimization method based on GPU server configuration. The computer-readable storage medium includes, for example, various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0164] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. An adaptive software stack optimization method based on GPU server configuration, characterized in that, include: Hardware parameters and scene constraint parameters of the GPU server are automatically collected through hardware identification, and the hardware parameters and scene constraint parameters are transformed into structured parameters through the multimodal perception capability of the large model hub. Based on preset keywords and the structured parameters, the resource library is invoked for semantic matching retrieval. The time decay factor algorithm is used to eliminate outdated knowledge, forming a dynamic knowledge base that is adapted to the structured parameters. Based on the structured parameters and the dynamic knowledge base, structured prompt words for reasoning in a large language model are assembled according to environmental constraints, best practices, and unsolved problems. The large language model cluster is invoked to perform multi-stage reasoning based on the structured prompt words, and the configured reasoning results are generated. The configuration reasoning result is parsed into a modular configuration scheme, which includes basic mandatory modules, optional enhancement modules, and interpretable attachments. Based on the operational metrics and user feedback after the deployment of the modular configuration scheme, optimization cases are obtained, and the optimization cases are updated to the resource library.

2. The adaptive software stack optimization method based on GPU server configuration according to claim 1, characterized in that, The process of automatically collecting hardware parameters and scene constraint parameters from the GPU server through hardware identification, and converting these parameters into structured parameters using the multimodal perception capabilities of the large model hub, includes: The hardware parameters include GPU model, video memory capacity, MIG support capability, PCIe version and hardware topology. The scenario constraint parameters include inference latency requirements, system throughput requirements, model type, and accuracy requirements; The multimodal perception capability of the large model hub integrates real-time hardware status data collected by the hardware management interface, textual requirements input by the user, and similar scenario parameters in the historical configuration database. It uses a multimodal fusion model to transform the hardware parameters and scenario constraint parameters into structured parameters in JSON format. The structured parameters contain fields in three dimensions: hardware characteristics, scenario requirements, and constraints.

3. The adaptive software stack optimization method based on GPU server configuration according to claim 2, characterized in that, The process involves calling a resource library for semantic matching based on preset keywords and structured parameters, combining this with a time decay factor algorithm to eliminate outdated knowledge, and forming a dynamic knowledge base adapted to the structured parameters, including: The data layer uses a collection of internal components to call the enterprise platform to obtain chip manufacturer recommendation documents and pull metadata from the large model optimization container. We collect open-source configuration examples from large model community repositories through external resource acquisition units, and obtain industry benchmark test reports from professional organization websites. Based on the structured parameters, precise keyword retrieval is performed through the text database of the knowledge layer to obtain the MIG sharding strategy and software version compatibility matrix knowledge in the chip manufacturer's recommended document; The performance benchmark data in the industry benchmark test report is obtained by performing semantic-level matching retrieval for "large model low-latency inference optimization" through vector database. The time decay factor algorithm is used to assign dynamic weights to knowledge items, and the weights of knowledge items that exceed the preset time limit are automatically reduced or removed, ensuring that the knowledge base only retains the latest configuration scheme that is compatible with the current hardware. The MIG sharding strategy, the software version compatibility matrix knowledge, the performance benchmark data, and the latest configuration scheme are integrated to form the dynamic knowledge base that is adapted to the structured parameters.

4. The adaptive software stack optimization method based on GPU server configuration according to claim 3, characterized in that, The structured prompts, assembled from the structured parameters and the dynamic knowledge base according to environmental constraints, best practices, and unsolved problems, into a large language model capable of reasoning, include: The environmental constraints include room temperature limits, power consumption limits, and network bandwidth. The best practices include historical optimization cases, chip manufacturer recommendation strategies, and industry standards stored in the dynamic knowledge base; The unresolved issues include conflicts in the current configuration requirements, uncovered functional requirements, and potential risks. By assembling the hardware characteristics, scenario requirements, environmental constraints, best practices, and unresolved problems of the structured parameters in a hierarchical manner, inference prompts containing four parts—hardware context, requirement description, knowledge reference, and unresolved conflicts—are formed for use by large language model clusters in multi-stage inference.

5. The adaptive software stack optimization method based on GPU server configuration according to claim 4, characterized in that, The invocation of the large language model cluster, based on the structured prompt words, performs multi-stage reasoning to generate a configured reasoning result, including: The current GPU hardware status is obtained in real time by calling NVMLAPI based on the structured prompt words using a large language model, the matching between MIG fragmentation and memory requirements is verified, and the hardware constraint verification conclusion is output. By calling the software version compatibility matrix in the dynamic knowledge base through the large language model, and combining it with the software constraints in the structured prompt words, the matching of candidate software versions and scenario requirements is verified, and a compatibility verification report is output. Based on the large language model, according to the scenario constraints in the structured prompts and the industry benchmark test reports and historical deployment cases in the dynamic knowledge base, the throughput, latency and resource utilization of the candidate configuration are estimated through the performance prediction model, and the performance simulation results are output. The hardware constraint verification conclusion, the compatibility verification report, and the performance simulation results are combined to generate the configuration inference result, which includes hardware configuration recommendations, a software version list, and expected performance values.

6. The adaptive software stack optimization method based on GPU server configuration according to claim 5, characterized in that, The configuration reasoning result is parsed into a modular configuration scheme, which includes basic mandatory modules, optional enhancement modules, and interpretable attachments, including: Based on the hardware constraint verification conclusion and the compatibility verification report in the configuration reasoning results, components that meet the minimum hardware operating requirements and basic scenario requirements are extracted to form the basic mandatory modules; Based on the performance simulation results and scenario expansion requirements in the configuration inference results, components that can improve performance or expand functions are selected based on the dynamic knowledge base to form the optional enhancement modules; Based on the hardware constraint verification conclusions, the software version compatibility matrix, and the performance simulation data in the configuration inference results, an interpretable appendix is ​​generated. The interpretable appendix includes: parameter basis, performance expectation, and alternative options.

7. The adaptive software stack optimization method based on GPU server configuration according to claim 6, characterized in that, The process of obtaining optimized cases based on operational metrics and user feedback after deployment using the modular configuration scheme, and updating these optimized cases to the resource library, includes: The operational metrics of the modular configuration scheme are collected in real time by monitoring tools in the deployment environment. These operational metrics include, but are not limited to: single-card inference latency, system throughput, GPU utilization, video memory usage, hardware temperature, and abnormal logs. User feedback is collected through user interaction interfaces, technical support ticket systems, and community forums. The feedback includes performance satisfaction, functional requirements, and fault descriptions. The collected operational metrics and user feedback are correlated and analyzed to extract key optimization points. Combined with the actual effect of the configuration scheme, a structured optimization case is generated, which includes problem scenarios, configuration parameters, optimization measures and effect verification. The optimized cases are imported into the resource library through the external resource acquisition unit of the resource library. After being verified based on the timeliness rules of the dynamic knowledge base, they are stored as new knowledge items in the text database and the vector database. Simultaneously, the time decay factor algorithm is triggered for old knowledge items in the resource library that conflict with the optimization case, reducing their weight or marking them as historical references, thereby completing the dynamic update of the resource library.

8. An adaptive software stack optimization device based on GPU server configuration, characterized in that, The device includes: The acquisition module is used to automatically acquire the hardware parameters and scene constraint parameters of the GPU server through hardware recognition, and to convert the hardware parameters and scene constraint parameters into structured parameters through the multimodal perception capability of the large model hub. The retrieval module is used to call the resource library for semantic matching retrieval based on preset keywords and the structured parameters, and combine the time decay factor algorithm to eliminate outdated knowledge to form a dynamic knowledge base that is adapted to the structured parameters. The prompting module is used to assemble structured prompt words that can be reasoned by the large language model based on the structured parameters and the dynamic knowledge base, according to environmental constraints, best practices and unsolved problems; The reasoning module is used to call the large language model cluster, perform multi-stage reasoning based on the structured prompt words, and generate the configured reasoning results; A configuration module is used to parse the configuration reasoning result into a modular configuration scheme, which includes a basic mandatory module, an optional enhancement module, and an interpretable appendix. The optimization module is used to obtain optimization cases based on the operational metrics and user feedback after the deployment of the modular configuration scheme, and to update the optimization cases to the resource library.

9. A control device, characterized in that, The device includes: It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1 to 7.

Images