Large model management software and hardware integrated device and management method

Abstract

The invention provides a large model management software and hardware integrated device and a management method. The large model management software and hardware integrated device comprises a heterogeneous computing unit, a computing power processor integrating multiple architectures and a storage device. The framework scheduling unit is deployed on the heterogeneous computing unit, dynamically allocates computing power based on physical resources of the heterogeneous computing unit and adapts to an AI framework; the model full-period management unit is used for directly calling the execution capability of the framework scheduling unit and completing model training, parameter storage and reasoning service packaging; the application construction unit is connected with the reasoning service of the model full-period management unit through a standardized interface and provides knowledge enhancement, agent cooperation and application arrangement capabilities; and the optimization decision-making unit is used for collecting operation data of the heterogeneous calculation unit and the model full-period management unit in real time, dynamically generating a resource configuration strategy and feeding back the resource configuration strategy to the framework scheduling unit to form closed-loop control.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and is applicable to the financial field or other fields, particularly to a large-scale model management integrated hardware and software device and management method. Background Technology

[0002] With the rapid development of Large Language Model (LLM) technology, its application in fields such as finance, healthcare, and manufacturing is becoming increasingly widespread. Current mainstream implementation methods include: public cloud services (such as OpenAI API and Azure AI): users call cloud models via APIs, but this carries the risk of data leakage and cannot meet the needs of private deployment; local deployment solutions: enterprises build their own computing clusters to deploy open-source models (such as Llama and ChatGLM), requiring independent configuration of hardware, frameworks, and inference tools.

[0003] In localized deployment scenarios, large-scale model applications face three major technical obstacles: 1. Hardware compatibility challenges: The computing hardware ecosystem is severely fragmented (NVIDIA GPUs / CUDA, Huawei Ascend NPUs / CANN, etc.), and different architectures require dedicated AI frameworks. Existing solutions cannot uniformly schedule heterogeneous hardware, resulting in insufficient resource utilization; 2. Fragmented toolchain: Training and inference are separated: The training end relies on frameworks such as PyTorch / TensorFlow, while the inference end requires independent configuration of tools such as ollama / vllm / TGI; toolchain switching leads to extended deployment cycles; 3. High barriers to application development: Vertical scenario applications (such as financial knowledge bases and intelligent investment advisors) require additional development: domain knowledge injection modules and multi-agent collaboration logic; existing platforms lack standardized application building capabilities, and development costs account for a high proportion of total investment.

[0004] While some optimization solutions exist in existing technologies to address the aforementioned issues, none have fundamentally resolved them. For example, the integrated online inference and training method and apparatus for large models provided by patent CN117668153A focuses on optimizing local computing power but fails to address multi-framework adaptation and application construction problems. The large model service management solution provided by patent CN117914971A optimizes API protocol conversion but relies on cloud services and ignores hardware compatibility. Therefore, the industry urgently needs a large model management solution to address these issues. Summary of the Invention

[0005] The purpose of this application is to provide an integrated hardware and software device and management method for large model management. By integrating computing hardware, AI computing platform, AI computing framework and basic large model, it supports private deployment and training and inference of various large models. It can reduce the high cost, technical threshold and data security threshold in the application of large models, and has high compatibility and flexibility to meet the needs of large-scale AI training and inference.

[0006] To achieve the above objectives, the large-scale model management hardware and software integrated device provided in this application includes: a heterogeneous computing unit, integrating computing processors and storage devices with multiple architectures; a framework scheduling unit, deployed on the heterogeneous computing unit, dynamically allocating computing power based on its physical resources and adapting to the AI ​​framework; a model lifecycle management unit, directly calling the execution capabilities of the framework scheduling unit to complete model training, parameter storage, and inference service encapsulation; an application building unit, connecting to the inference service of the model lifecycle management unit through a standardized interface, providing knowledge enhancement, agent collaboration, and application orchestration capabilities; and an optimization decision-making unit, collecting real-time operating data from the heterogeneous computing unit and the model lifecycle management unit, dynamically generating resource allocation strategies, and feeding them back to the framework scheduling unit to form a closed-loop control.

[0007] In the above-mentioned device, optionally, the framework scheduling unit includes: a hardware abstraction subunit, which parses the topology of heterogeneous computing units and converts instruction sets; a framework runtime subunit, which loads dependent libraries in response to the call requests of the model full-cycle management unit; and a resource monitoring subunit, which synchronizes computing power utilization data to the optimization decision unit.

[0008] Optionally, in the above apparatus, the model lifecycle management unit includes: a distributed training controller that submits training tasks to the framework scheduling unit; a versioned model repository that stores snapshots of parameters output by the training controller; and an inference service gateway that loads model repository data and encapsulates API services.

[0009] Optionally, in the above apparatus, the versioned model repository further includes receiving version write requests from the training controller; the inference service gateway loads the model version in response to the call instruction from the application building unit.

[0010] Optionally, in the above-mentioned device, the application building unit includes: a knowledge enhancement engine that intercepts input requests from the inference service gateway and injects domain knowledge; an agent collaboration engine that orchestrates task flows and invokes the inference service gateway; and a service orchestration engine that records component dependencies and feeds back logs to the optimization decision unit.

[0011] Optionally, in the above-described apparatus, the knowledge enhancement engine may further include an input stream that sends the retrieval results back to the inference service gateway; the agent collaboration engine may trigger inference service calls via a message bus.

[0012] In the above-mentioned device, optionally, the optimization decision unit includes a training cost data generation performance prediction model based on the model lifecycle management unit; adjusting the hardware combination scheme according to the service latency log of the application building unit; and dynamically injecting resource configuration instructions into the framework scheduling unit.

[0013] This application also provides a large model management method applicable to the aforementioned large model management hardware and software integrated device. The method includes: integrating multi-architecture computing processors and storage devices to construct a heterogeneous computing resource pool; dynamically allocating computing power based on the heterogeneous computing resource pool to adapt to the operating environment of different AI computing frameworks; calling the adapted AI framework to perform distributed training, storing versioned model parameters and encapsulating multi-mode inference services; calling the inference service to construct vertical domain applications through knowledge enhancement and agent collaboration; and collecting hardware resource utilization and service response data in real time, dynamically feeding back resource configuration instructions to the framework scheduling stage to form a control closed loop.

[0014] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.

[0015] This application also provides a computer-readable storage medium storing a computer program that performs the above-described methods.

[0016] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0017] The beneficial technical effects of this application are as follows: It employs a hardware abstraction layer to dynamically convert heterogeneous processor instruction sets (such as GPU / NPU), eliminating the cost of multi-framework adaptation and supporting seamless switching between AI frameworks such as CUDA / CANN; a three-level linkage mechanism of model training-inference-application reduces intermediate data transfer overhead; a knowledge enhancement engine injects domain knowledge in real time, improving the accuracy of financial risk control models; and an intelligent agent collaboration engine improves the timeliness of complex task processing through parallel scheduling of DAG task flows. The optimized decision-making unit constructs a "monitoring-prediction-control" closed loop, dynamically adjusting computing power allocation strategies; under fluctuating load scenarios, the idle rate of hardware resources is effectively reduced, the energy efficiency ratio (TOPS / W) is improved, and hardware investment is reduced for the same computing power requirements; localized private deployment eliminates the risk of cloud service data leakage, meets financial-grade security and compliance requirements, and one-click model version management reduces operational complexity and saves labor costs. Attached Figure Description

[0018] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. In the drawings:

[0019] Figure 1 A schematic diagram of the integrated hardware and software device for large model management provided in an embodiment of this application;

[0020] Figure 2This is a schematic diagram of the structure of a framework scheduling unit provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the structure of a model lifecycle management unit provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an application building unit provided in an embodiment of this application;

[0023] Figure 5 This is a schematic diagram of the structure of an optimization decision unit provided in an embodiment of this application;

[0024] Figure 6 A flowchart illustrating a large model management method provided in an embodiment of this application;

[0025] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] The following will describe in detail the implementation methods of this application with reference to the accompanying drawings and embodiments, so as to fully understand how this application uses technical means to solve technical problems and achieve technical effects, and to implement it accordingly. It should be noted that, as long as there is no conflict, the various embodiments and features in each embodiment of this application can be combined with each other, and the resulting technical solutions are all within the protection scope of this application.

[0027] Furthermore, the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0028] Please refer to Figure 1 As shown, the large model management hardware and software integrated device provided in this application includes: a heterogeneous computing unit, integrating computing processors and storage devices of various architectures; a framework scheduling unit, deployed on the heterogeneous computing unit, dynamically allocating computing power based on its physical resources and adapting to the AI ​​framework; a model lifecycle management unit, directly calling the execution capabilities of the framework scheduling unit to complete model training, parameter storage, and inference service encapsulation; an application building unit, connecting to the inference service of the model lifecycle management unit through a standardized interface, providing knowledge enhancement, agent collaboration, and application orchestration capabilities; and an optimization decision-making unit, collecting real-time operating data from the heterogeneous computing unit and the model lifecycle management unit, dynamically generating resource configuration strategies, and feeding them back to the framework scheduling unit to form a closed-loop control.

[0029] In this embodiment, during the model training phase, the framework scheduling unit automatically identifies the risk control training task requirements, then allocates the Ascend NPU cluster to load the MindSpore framework, and finally outputs the trained model with hundreds of billions of parameters to the version repository. During the service deployment phase, the inference service gateway loads the quantized model, encapsulates it as a gRPC interface, and exposes it to the application building unit. The version repository automatically synchronizes the deployment configuration template. In the risk control decision application, the knowledge enhancement engine can access the regulatory policy database in real time. When a loan application is received, the basic model API is called to generate an initial risk assessment, and the latest credit policies are retrieved to enhance the judgment basis. Finally, a risk control report with policy clause references is output. For the dynamic optimization triggering logic, in actual work, if the optimization decision unit detects an increase in the peak delay of midday approvals, it automatically executes the following strategies: expanding the NPU computing nodes, switching to INT4 quantization mode to improve throughput, and reducing the P99 latency to a set value within a preset time. To facilitate a clearer understanding of the integrated hardware and software device for large model management provided in this application, the following will further explain the specific implementation methods of each module in the above embodiments. Those skilled in the art will understand that this explanation does not limit this application in any way.

[0030] Please refer to Figure 2 As shown, in one embodiment of this application, the framework scheduling unit includes: a hardware abstraction subunit, which parses the topology of heterogeneous computing units and converts instruction sets; a framework runtime subunit, which loads dependent libraries in response to the call requests of the model lifecycle management unit; and a resource monitoring subunit, which synchronizes computing power utilization data to the optimization decision unit.

[0031] Specifically, in practice, when the training controller submits standard training instructions, the hardware abstraction subunit automatically executes, scans connected computing devices (identifying GPU / NPU hybrid architectures), and converts general matrix operation instructions into the device's native instruction set. This allows the same training task to run seamlessly on accelerators from different brands. Regarding the runtime dynamic response mechanism, after receiving a loading request, the framework runtime subunit automatically selects a framework version matching the device architecture (e.g., loading a customized PyTorch for the NPU) and builds an isolated dependency environment to avoid conflicts. The returned runtime handle is then directly used for training task scheduling. For resource monitoring-driven optimization decisions, when GPU utilization is detected to be consistently above a threshold, the resource monitoring subunit marks the device as overloaded, triggering the optimization decision unit to generate a scheduling strategy; it also automatically supplements NPU nodes to share the computational load.

[0032] Please refer to Figure 3As shown in one embodiment of this application, the model lifecycle management unit includes: a distributed training controller that submits training tasks to the framework scheduling unit; a versioned model repository that stores parameter snapshots output by the training controller; and an inference service gateway that loads model repository data and encapsulates API services. The versioned model repository further includes receiving version write requests from the training controller; and the inference service gateway loads model versions in response to invocation instructions from the application building unit.

[0033] Specifically, banks need to build a loan risk assessment system and implement the following processes through a model-based full-cycle management unit:

[0034] 1. Training Phase:

[0035] The distributed training controller receives risk control model training tasks and submits standardized training instructions (including dataset path and hyperparameter configuration) to the framework scheduling unit.

[0036] The framework scheduling unit dynamically allocates heterogeneous computing resources (such as NPU clusters) and loads the adapted AI framework to perform distributed training.

[0037] After training is completed, the training controller writes the generated model with hundreds of billions of parameters and its metadata (such as accuracy and training time) into the versioned model repository as version v2.1.

[0038] 2. Deployment Phase:

[0039] The inference service gateway responds to the call instructions of the application building blocks, loads the v2.1 model from the version repository and encapsulates it as a RESTful API service;

[0040] Gateway auto-injection model optimization strategy: Enable INT4 quantization inference for high-concurrency scenarios, and retain FP16 mode for accuracy-sensitive scenarios.

[0041] 3. Application stage:

[0042] When the risk control system receives a loan application, the application building blocks call the model service via API;

[0043] If a model upgrade is required, the training controller submits the new version v2.2 to the repository, and the inference gateway seamlessly migrates to the new version through a hot-swapping mechanism, ensuring zero downtime for the business system.

[0044] As a result, versioned repositories enable traceability of model assets, reduce intermediate transfer steps in the training-inference closed loop, and shorten the model deployment cycle.

[0045] Please refer to Figure 4As shown in one embodiment of this application, the application building unit includes: a knowledge enhancement engine that intercepts input requests from the inference service gateway and injects domain knowledge; an agent collaboration engine that orchestrates task flows and invokes the inference service gateway; and a service orchestration engine that records component dependencies and feeds back logs to the optimization decision unit. The knowledge enhancement engine further includes an input stream that sends retrieval results back to the inference service gateway; the agent collaboration engine triggers inference service calls via a message bus.

[0046] Specifically, in practical work, taking the construction of a real-time transaction anti-fraud application by a bank as an example, it requires dynamically supplementing the latest fraud case knowledge, coordinating multiple stages such as risk identification, user verification, and manual review, and supporting the canary release of new strategies. Based on this requirement, the process implemented in the application construction unit is as follows: Knowledge enhancement engine operation: intercepts the input request of the transaction risk control model, retrieves the internal fraud knowledge base in real time, and injects relevant information. The enhanced complete data stream is then sent back to the inference service gateway. Intelligent agent collaboration engine scheduling: orchestrates the DAG task flow, and triggers service calls through the message bus according to the determined task flow: when the model output is initially judged to be of medium risk, the SMS verification service API is automatically called; if the intelligent agent fails to respond within the timeout period during manual review, the process is escalated to the risk control specialist. Service orchestration engine monitoring: records the component dependency chain and feeds back key logs to the optimization decision unit. Thus, compared with pure model inference, knowledge enhancement improves the accuracy of fraud identification, parallel scheduling of intelligent agents shortens the processing time of complex cases, and dependency tracing enables accurate canary release.

[0047] Please refer to Figure 5 As shown, in one embodiment of this application, the optimization decision unit includes a training cost data generation performance prediction model based on the model lifecycle management unit; adjusting the hardware combination scheme according to the service latency log of the application building unit; and dynamically injecting resource configuration instructions into the framework scheduling unit.

[0048] Specifically, in practical work, the performance prediction model construction process can be based on the training cost data submitted by the model full-cycle management unit, including historical task GPU hour consumption and accuracy decay curves of different quantization strategies; then, a prediction model is generated: input (expected trading volume + model accuracy requirements), output optimal hardware combination scheme (e.g., NPU cluster + INT8 quantization); then, in the real-time hardware combination adjustment stage, the service latency logs of the application building unit can be monitored (APIP99 latency rises to the threshold limit during the opening period and knowledge enhancement engine retrieval latency increases significantly), and then the strategy is automatically executed, i.e., monitoring data, and decision analysis is performed based on the monitored data to determine whether to perform adjustments, expand NPU nodes, or switch to FP16 inference mode; among them, in the resource configuration instruction injection stage, instructions can be dynamically issued to the framework scheduling unit, and / or non-critical services (such as offline report generation) can be closed simultaneously to release memory.

[0049] Please refer to Figure 6 As shown, in one embodiment of this application, this application also provides a large model management method applicable to the aforementioned large model management hardware and software integrated device, the method comprising:

[0050] The S601 integrates multi-architecture computing processors and storage devices to build a heterogeneous computing resource pool.

[0051] S602 dynamically allocates computing power based on the heterogeneous computing resource pool to adapt to the operating environment of different AI computing frameworks;

[0052] S603 calls the adapted AI framework to perform distributed training, stores versioned model parameters, and encapsulates multi-mode inference services.

[0053] S604 invokes the inference service to build vertical domain applications through knowledge enhancement and collaboration with intelligent agents;

[0054] The S605 collects hardware resource utilization and service response data in real time, and dynamically feeds back resource configuration instructions to the framework scheduling stage to form a control closed loop.

[0055] Since the implementation logic of each step has been described one by one in the foregoing embodiments, the implementation process of each step can be performed with reference to the foregoing embodiments, and will not be described in detail here.

[0056] The beneficial technical effects of this application are as follows: It employs a hardware abstraction layer to dynamically convert heterogeneous processor instruction sets (such as GPU / NPU), eliminating the cost of multi-framework adaptation and supporting seamless switching between AI frameworks such as CUDA / CANN; a three-level linkage mechanism of model training-inference-application reduces intermediate data transfer overhead; a knowledge enhancement engine injects domain knowledge in real time, improving the accuracy of financial risk control models; and an intelligent agent collaboration engine improves the timeliness of complex task processing through parallel scheduling of DAG task flows. The optimized decision-making unit constructs a "monitoring-prediction-control" closed loop, dynamically adjusting computing power allocation strategies; under fluctuating load scenarios, the idle rate of hardware resources is effectively reduced, the energy efficiency ratio (TOPS / W) is improved, and hardware investment is reduced for the same computing power requirements; localized private deployment eliminates the risk of cloud service data leakage, meets financial-grade security and compliance requirements, and one-click model version management reduces operational complexity and saves labor costs.

[0057] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.

[0058] This application also provides a computer-readable storage medium storing a computer program that performs the above-described methods.

[0059] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0060] like Figure 7 As shown, the electronic device 600 may also include: a communication module 110, an input unit 120, an audio processor 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily need to include these components. Figure 7 All components shown; in addition, the electronic device 600 may also include Figure 7 For components not shown, please refer to existing technologies.

[0061] like Figure 7 As shown, the central processing unit 100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device. The central processing unit 100 receives inputs and controls the operation of various components of the electronic device 600.

[0062] The memory 140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 100 may execute the program stored in the memory 140 to perform information storage or processing, etc.

[0063] Input unit 120 provides input to central processing unit 100. Input unit 120 may be, for example, a keypad or touch input device. Power supply 170 provides power to electronic device 600. Display 160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0064] The memory 140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 140 can also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142 for storing application programs and function programs or processes for executing the operation of the electronic device 600 via the central processing unit 100.

[0065] The memory 140 may also include a data storage unit (data 143) for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit (driver 144) of the memory 140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0066] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processing unit 100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0067] Based on different communication technologies, multiple communication modules 110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132, thereby enabling typical telecommunications functions. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 130 is coupled to a central processing unit 100, enabling on-device recording via the microphone 132 and on-device playback of stored audio via the speaker 131.

[0068] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0069] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A large model management hardware and software integrated device, characterized by, include: Heterogeneous computing units integrate computing processors and storage devices with multiple architectures; The framework scheduling unit is deployed on the heterogeneous computing unit and dynamically allocates computing power based on its physical resources and adapts to the AI ​​framework. The model lifecycle management unit directly calls the execution capabilities of the framework scheduling unit to complete model training, parameter storage, and inference service encapsulation. Application building units connect to the inference services of the model lifecycle management unit through standardized interfaces, providing knowledge enhancement, agent collaboration, and application orchestration capabilities; The optimization decision-making unit collects real-time operational data from heterogeneous computing units and model lifecycle management units, dynamically generates resource allocation strategies, and feeds them back to the framework scheduling unit to form a closed-loop control.

2. The apparatus of claim 1, wherein, The framework scheduling unit includes: Hardware abstraction subunit: parses the topology of heterogeneous computing units and converts instruction sets; The framework runtime subunit loads dependent libraries in response to the call requests from the model lifecycle management unit. The resource monitoring subunit synchronizes computing power utilization data with the optimization decision-making unit.

3. The apparatus of claim 1, wherein, The model lifecycle management unit includes: The distributed training controller submits training tasks to the framework scheduling unit. Versioned model repository, storing snapshots of parameters output by the training controller; The inference service gateway loads model repository data and encapsulates API services.

4. The apparatus of claim 3, wherein, The versioned model repository also includes receiving version write requests from the training controller; the inference service gateway loads the model version in response to the call instructions from the application building unit.

5. The apparatus of claim 1, wherein, The application building unit includes: The knowledge enhancement engine intercepts input requests from the inference service gateway and injects domain knowledge. The agent collaboration engine orchestrates task flows and invokes the inference service gateway. The service orchestration engine records component dependencies and feeds back logs to the optimization decision-making unit.

6. The apparatus of claim 5, wherein, The knowledge enhancement engine also includes an input stream that sends retrieval results back to the inference service gateway; the agent collaboration engine triggers inference service calls via a message bus.

7. The apparatus of claim 1, wherein, The optimization decision-making unit includes generating an efficiency prediction model based on training cost data from the model lifecycle management unit; adjusting the hardware combination scheme according to the service latency logs of the application building unit; and dynamically injecting resource configuration instructions into the framework scheduling unit. 8.A large model management method suitable for the large model management software and hardware integrated device according to any one of claims 1 to 7, characterized in that, The method includes: Integrate multi-architecture computing processors and storage devices to build a heterogeneous computing resource pool; Based on the aforementioned heterogeneous computing resource pool, computing power is dynamically allocated to adapt to the operating environment of different AI computing frameworks; Call the adapted AI framework to perform distributed training, store versioned model parameters and encapsulate multi-mode inference services; The inference service is invoked to build vertical domain applications through knowledge enhancement and collaboration with intelligent agents. Real-time collection of hardware resource utilization and service response data, and dynamic feedback of resource configuration instructions to the framework scheduling stage to form a closed-loop control system.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of claim 8.

11. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 8.

Images