Deployment and reasoning system and deployment method of multi-modal large model
By microservice-izing the multimodal large model, independent deployment and parallel operation are achieved, solving the problem of low deployment efficiency, improving iteration and inference speed, and optimizing resource utilization.
Patent Information
- Application Number
- CN202511736510.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-25
- Publication Date
- 2026-02-17
AI Technical Summary
Existing technologies suffer from low deployment efficiency, slow updates and iterations, insufficient resource utilization, and failure to effectively utilize the parallel processing capabilities of modern computing architectures, especially GPU accelerators.
We adopt a microservice architecture for a multimodal large model, dividing the model into multiple functional modules that run in parallel and are deployed independently. We use namespaces to manage dependencies and execution order between modules and optimize the model inference process through microservice orchestration.
It improves model iteration speed and deployment efficiency, reduces development and maintenance costs, significantly enhances inference speed, and optimizes resource utilization efficiency, especially the parallel processing capabilities of GPUs.
Smart Images
Figure CN121541890A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent manufacturing technology, and in particular to a multimodal large-scale model deployment and inference system and deployment method. Background Technology
[0002] In the field of intelligent manufacturing, the deployment of multimodal large models is crucial for the practical application of machine learning and deep learning algorithms. Currently, the industry's common deployment strategy is to encapsulate the entire model into a single container image and, with the help of containerization technologies such as Docker or Kubernetes, combined with CI / CD tools, to achieve deployment on servers or server clusters. While this approach simplifies the deployment process and allows models to run across different environments and platforms, it also brings a series of technical challenges and limitations.
[0003] First, monolithic deployment models require replacing the entire image during updates and iterations, often involving a large amount of unchanged content, which reduces the efficiency of model update and iteration. Second, monolithic deployment limits the reusability of common components or modules in the model, increasing the difficulty of sharing and reusing functionality across applications, thereby driving up development and maintenance costs. Furthermore, the deployment of a single container image fails to fully utilize the parallel processing capabilities of modern computing architectures, such as GPU accelerators, and fails to fully leverage their acceleration potential, resulting in limited current inference speeds.
[0004] While existing technologies employ techniques such as model pruning and quantization to reduce model size, or utilize cloud services for elastic deployment to adapt to different computing needs, these methods do not fundamentally change the model's deployment and operation mechanism. They often fail to adequately consider the specific needs of multimodal large models in intelligent manufacturing when processing various types of data, and they also fail to effectively leverage the parallel processing advantages of modern computing architectures to specifically improve the inference speed of multimodal large models. Summary of the Invention
[0005] The technical problem to be solved by the present invention is to provide a deployment and inference system and method for multimodal large models, which can improve the efficiency, reusability and scalability of model deployment and accelerate the inference speed of the model.
[0006] The technical solution adopted by this invention to solve its technical problem is: to provide a deployment and inference system for a multimodal large model, comprising:
[0007] The first namespace is used to obtain the corresponding microservice orchestration code and model runtime data according to the received external request. Based on the microservice orchestration code, multiple functional modules are called to form a multimodal large model, and then the model output results are obtained by inference based on the model runtime data.
[0008] The second namespace is used to store the microservice orchestration code, which includes the execution order and dependencies of each functional module.
[0009] The third namespace is used to store the functional modules;
[0010] The fourth namespace is used to store the model's runtime data and the parameters of each of the functional modules.
[0011] Furthermore, the functional modules that do not have dependencies run in parallel.
[0012] Furthermore, the functional modules include multiple feature extraction modules for different modalities, as well as a multimodal feature fusion module.
[0013] Furthermore, the multiple feature extraction modules for different modalities operate in parallel.
[0014] Furthermore, the first namespace includes:
[0015] The model request processing module is used to obtain microservice orchestration code based on received external requests and send data call requests;
[0016] The database module is used to retrieve the model running data from the fourth namespace according to the data call request and send it to the model request processing module.
[0017] Furthermore, the first namespace also includes:
[0018] The platform service processing module is used to send data call requests based on received external requests;
[0019] An external request processing module is used to receive external requests and forward them to the model request processing module and / or the platform service processing module according to the request content.
[0020] Furthermore, each of the microservice orchestration codes uniquely corresponds to one of the external requests.
[0021] Furthermore, when the functional module is invoked, the functional module is configured based on parameters obtained from the fourth namespace.
[0022] This invention also provides a method for deploying a multimodal large model, applied to the system described above, comprising the following steps:
[0023] Obtain the large multimodal model to be deployed and its parameters;
[0024] Obtain the model's runtime data and store it in the fourth namespace;
[0025] The large multimodal model to be deployed is divided into multiple functional modules to be deployed, and compared with the existing functional modules in the third namespace to determine the new functional modules that need to be added.
[0026] Store the newly added functional modules in the third namespace;
[0027] Store the parameters corresponding to each of the functional modules to be deployed into the fourth namespace;
[0028] Generate microservice orchestration code to set the dependencies and execution order of each of the functional modules to be deployed, and store the microservice orchestration code in the second namespace;
[0029] Update the microservice orchestration code, model runtime data, and corresponding external requests in the first namespace.
[0030] Beneficial effects
[0031] By adopting the above-mentioned technical solutions, this invention has the following advantages and positive effects compared with the prior art: This invention, through a microservice architecture for a multimodal large model, allows for independent deployment and updates of each module of the multimodal large model, greatly improving the model's iteration speed and deployment efficiency, and effectively solving the inefficiency problem caused by updating the entire image in monolithic deployments; the modular design of the system enhances the reusability of model components, simplifies the process of sharing and reusing functions across applications, and reduces development and maintenance costs; furthermore, considering the characteristics of multimodal large models, this invention fully utilizes the advantages of modern computing architectures, especially the parallel processing capabilities of GPU accelerators, and introduces orchestration services to achieve parallel operation of dependency-free artificial intelligence modules, significantly improving the model's inference speed and optimizing the overall resource utilization efficiency. Attached Figure Description
[0032] Figure 1 This is a schematic diagram of the structure of the first embodiment of the present invention;
[0033] Figure 2 This is a flowchart of the deployment method of the first and second embodiments of the present invention;
[0034] Figure 3 These are flowcharts of the model update process according to the first and second embodiments of the present invention;
[0035] Figure 4 This is a schematic diagram of the structure of the first embodiment of the present invention applied to the field of intelligent manufacturing. Detailed Implementation
[0036] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.
[0037] The first embodiment of the present invention relates to a microservice-based deployment and inference system for multimodal large models, architectured on a server cluster, such as... Figure 1 As shown, it includes:
[0038] The comprehensive interface namespace A1 is used to receive and process external access requests, store models and datasets in the data storage namespace A4, and call the module orchestration module A21 in the module orchestration namespace A2 that uniquely corresponds to the external request; the comprehensive interface namespace A1 specifically includes:
[0039] External request processing module A11 is used to receive external access requests and forward the requests to model request processing module A12 and / or platform service processing module A14 simultaneously or separately, depending on the request content.
[0040] Model request processing module A12 is used to receive and process requests forwarded by external request processing module A11. After receiving the request forwarded by external request processing module A11, model request processing module A12 stores and retrieves data in database module A13 according to the request content, and calls module orchestration module A21 in module orchestration namespace A2 that uniquely corresponds to the request forwarded by external request processing module A11.
[0041] Database module A13 is used to respond to data storage or call requests from model request processing module A12 and platform service processing module A14;
[0042] Platform service processing module A14 is used to receive and process requests forwarded by external request processing module A11. After receiving the request forwarded by external request processing module A11, platform service processing module A14 stores and retrieves data in database module A13 according to the content of the request, and stores the model and dataset in data storage namespace A4 according to the content of the request.
[0043] The module orchestration namespace A2 is used to receive call requests from the model request processing module A12, retrieve datasets from the data storage namespace A4, and sequentially call the corresponding AI model modules A31 in the AI model module namespace A3 according to the module orchestration order recorded in the module orchestration module A21 called by the call request from the model processing request module A12. The module orchestration namespace A2 includes several module orchestration modules A21, each module orchestration module A21 corresponding to a unique multimodal large model, and is used to record the orchestration order of all AI model modules A31 of the unique multimodal large model. While the module orchestration module A21 calls all the AI model modules A31 recorded internally, it will also retrieve the datasets (if any) required by the AI model module A31 from the data storage namespace A4.
[0044] The AI model module namespace A3 is used to receive call requests from the module orchestration module A21 and call models from the data storage namespace A4. AI model module A33 includes several AI model modules A31, which are derived from the division of a large multimodal model. Specifically, the large multimodal model is divided into multiple modules based on functional division and reusability. The divided modules can still perform the functions of the large multimodal model. AI model modules A31 will be called by the module orchestration module A21, which will retrieve the required model parameters (if any) from the data storage namespace A4.
[0045] Data storage namespace A4 contains several data storage modules A41, which are deployed with a storage system (such as Ceph). Each data storage module A41 stores model parameters and datasets according to the management of the storage system, and provides the corresponding model parameters and datasets when invoked.
[0046] The following section provides further explanation of how to use the system described above.
[0047] like Figure 2 As shown, the above system usage process includes the following steps:
[0048] Step B1 involves dividing an existing large multimodal model into multiple distinct modules based on functional division and reusability. For example, a common trimodal sentiment analysis model (with language, sound, and image as the three modalities) can be functionally divided into a feature extraction module and a feature fusion module. The feature extraction module can be further divided into a language feature extraction module, a sound feature extraction module, and an image feature extraction module based on functional division and reusability. Specifically, it should be ensured that, for a given input, each of the divided modules can produce the same output as the undivided large multimodal model using at least one execution order.
[0049] Step B2 involves writing microservice orchestration code, defining the execution order and dependencies between the different modules split in the previous step. The microservice orchestration code determines the inference efficiency of the multimodal large model after splitting. Using the example from Step B1, first, the dependencies between different modules need to be defined. The feature fusion module requires the feature extraction results from the three modalities as input; therefore, the feature fusion module depends on the language feature extraction module, the sound feature extraction module, and the image feature extraction module, and has no other dependencies. Next, the execution order is defined. Since the three feature extraction modules have no mutual dependencies, they can run in parallel. After all three feature extraction modules have run, the dependencies of the feature fusion module are satisfied, and it can begin running. Furthermore, microservice orchestration can be aware of the load on all GPUs and allocate modules without dependencies to idle GPUs.
[0050] Step B3 involves uploading the module code and microservice orchestration code to the multimodal large-scale model microservice deployment and inference system. If there are datasets or other files needed during the operation of the multimodal large-scale model before partitioning, they should also be uploaded to the system. Using the example from Step B1, after uploading all the partitioned modules and microservice orchestration code, the text, image, and audio datasets needed for training and prediction should be uploaded simultaneously.
[0051] Step B4: Send a request to the system to obtain the model output results.
[0052] like Figure 3 As shown, the model update process of the above system includes the following steps:
[0053] Step C1: Modify the module code that needs to be updated (if necessary). Step C1 has three scenarios: if you want to update an existing module, you need to modify that module's code; if you want to add a new module to an existing module, you need to write new module code; if you want to delete a module, you do not need to modify any code.
[0054] Step C2: Modify the microservice orchestration code (if necessary). Step C2 has three scenarios: If you want to update an existing module without affecting the execution order or dependencies, you do not need to modify the microservice orchestration code; if you want to add a new module to the existing module, you need to modify the microservice orchestration code to add the new module according to the execution order and dependencies; if you want to delete a module, you need to modify the microservice orchestration code to delete the module and ensure that other modules can run normally and produce the expected results.
[0055] Step C3 involves uploading the modified module code, microservice orchestration code, and any newly required files or datasets to the multimodal large-scale model microservice deployment and inference system.
[0056] Step C4: Send a request to the system to obtain the model output results.
[0057] like Figure 4 As shown, when the above system is applied to multimodal equipment fault detection and prevention in the field of intelligent manufacturing, it includes:
[0058] The module orchestration namespace D2 (i.e., module orchestration namespace A2 in the aforementioned general system) includes a module orchestration module D21, which is the module orchestration module for the large model of multimodal equipment fault detection and prevention in the field of intelligent manufacturing. It records the execution order of different modules in the artificial intelligence model module namespace D3. While the module orchestration module D21 calls the artificial intelligence model modules recorded internally, it also calls the datasets required by the artificial intelligence model modules from the data storage namespace D4. Specifically, when the module orchestration module D21 calls the image processing module D31, it simultaneously calls the image data storage module D44 and the image processing model storage module D41; when the module orchestration module D21 calls the sound processing module D32, it simultaneously calls the sound data storage module D45 and the sound processing model storage module D42; when the module orchestration module D21 calls the modal fusion module D33, it calls the outputs of the image processing module D31 and the sound processing module D32, as well as the modal fusion model storage module D33.
[0059] The AI model module namespace D3 (i.e., the AI model module namespace A3 in the aforementioned general system) includes the graphics processing module D31, the sound processing module D32, and the modal fusion module D33. These modules are derived from a large model for multimodal equipment fault detection and prevention in the field of intelligent manufacturing, segmented according to functional division and reusability. The graphics processing module D31, sound processing module D32, and modal fusion module D33 will be called by the module orchestration module D21. When the model orchestration module D21 calls the graphics processing module D31, sound processing module D32, and modal fusion module D33, the corresponding data will be called synchronously, as discussed in the previous paragraph.
[0060] Data storage namespace D4 (i.e., data storage namespace A4 in the aforementioned general system) includes image processing model storage module D41, sound processing model storage module D42, modality fusion model storage module D43, image data storage module D44, and sound data storage module D45. A Ceph distributed file storage system is deployed on it. Image processing model storage module D41 stores parameters of image processing module D31, sound processing model storage module D42 stores parameters of sound processing module D32, modality fusion model storage module D43 stores parameters of modality fusion module D33, image data storage module D44 stores data needed by image processing module D31 during operation, and sound data storage module D45 stores data needed by sound processing module D32 during operation.
[0061] The second embodiment of the present invention relates to a microservice deployment method for a multimodal large model, applied to the system in the first embodiment, and includes the following steps:
[0062] Step B1 involves dividing an existing large multimodal model into multiple distinct modules based on functional division and reusability. For example, a common trimodal sentiment analysis model (with language, sound, and image as the three modalities) can be functionally divided into a feature extraction module and a feature fusion module. The feature extraction module can be further divided into a language feature extraction module, a sound feature extraction module, and an image feature extraction module based on functional division and reusability. Specifically, it should be ensured that, for a given input, each of the divided modules can produce the same output as the undivided large multimodal model using at least one execution order.
[0063] Step B2 involves writing microservice orchestration code, defining the execution order and dependencies between the different modules split in the previous step. The microservice orchestration code determines the inference efficiency of the multimodal large model after splitting. Using the example from Step B1, first, the dependencies between different modules need to be defined. The feature fusion module requires the feature extraction results from the three modalities as input; therefore, the feature fusion module depends on the language feature extraction module, the sound feature extraction module, and the image feature extraction module, and there are no other dependencies. Next, the execution order is defined. Since there are no dependencies between the three feature extraction modules, they can run in parallel. After all three feature extraction modules have run, the dependencies of the feature fusion module are satisfied, and it can begin running.
[0064] Step B3 involves uploading the module code and microservice orchestration code to the multimodal large-scale model microservice deployment and inference system. If there are datasets or other files needed during the operation of the multimodal large-scale model before partitioning, they should also be uploaded to the system. Using the example from Step B1, after uploading all the partitioned modules and microservice orchestration code, the text, image, and audio datasets needed for training and prediction should be uploaded simultaneously.
[0065] Step B4: Send a request to the system to obtain the model output results.
[0066] like Figure 3 As shown, the update process includes the following steps:
[0067] Step C1: Modify the module code that needs to be updated (if necessary). Step C1 has three scenarios: if you want to update an existing module, you need to modify that module's code; if you want to add a new module to an existing module, you need to write new module code; if you want to delete a module, you do not need to modify any code.
[0068] Step C2: Modify the microservice orchestration code (if necessary). Step C2 has three scenarios: If you want to update an existing module without affecting the execution order or dependencies, you do not need to modify the microservice orchestration code; if you want to add a new module to the existing module, you need to modify the microservice orchestration code to add the new module according to the execution order and dependencies; if you want to delete a module, you need to modify the microservice orchestration code to delete the module and ensure that other modules can run normally and produce the expected results.
[0069] Step C3 involves uploading the modified module code, microservice orchestration code, and any newly required files or datasets to the multimodal large-scale model microservice deployment and inference system.
[0070] Step C4: Send a request to the system to obtain the model output results.
[0071] In summary, this invention implements a microservice-based deployment and inference system for multimodal large models. Through innovative architectural design, this system effectively solves the problems of low efficiency in model update iteration, poor reusability of general components, insufficient resource utilization, and underutilization of GPU parallel processing capabilities in existing technologies. First, the microservice architecture of this system allows for independent deployment and updates of each module of the multimodal large model, greatly improving the model iteration speed and deployment efficiency, effectively solving the inefficiency problem caused by updating the entire image in monolithic deployments. Second, the modular design of the system enhances the reusability of model components, simplifies the process of sharing and reusing functions across applications, and reduces development and maintenance costs. Furthermore, considering the characteristics of multimodal large models, this invention fully utilizes the advantages of modern computing architectures, especially the parallel processing capabilities of GPU accelerators. By introducing orchestration services to achieve parallel operation of dependency-free artificial intelligence modules, it significantly improves the model inference speed and optimizes the overall resource utilization efficiency.
Claims
1. A system for deployment and inference of a multi-modal large model, the system comprising: The system comprises: a first namespace for obtaining corresponding microservice orchestration code and model running data based on received external requests, calling multiple functional modules to combine into a multi-modal large model based on the microservice orchestration code, and then performing inference based on the model running data to obtain model output results; a second namespace for storing the microservice orchestration code, which comprises the running order and dependency relationship of each functional module; a third namespace for storing the functional modules; a fourth namespace for storing the model running data and the parameters of each functional module.
2. The system of claim 1, wherein, The functional modules without dependency relationships are run in parallel.
3. The system of claim 2, wherein, The functional modules comprise multiple feature extraction modules for different modalities and a multi-modal feature fusion module.
4. The system of claim 3, wherein, The multiple feature extraction modules for different modalities are run in parallel.
5. The system of claim 1, wherein, The first namespace comprises: a model request processing module for obtaining microservice orchestration code based on received external requests and sending data call requests; a database module for obtaining the model running data from the fourth namespace based on the data call requests and sending them to the model request processing module.
6. The system of claim 5, wherein, The first namespace further comprises: a platform service processing module for sending data call requests based on received external requests; an external request processing module for receiving external requests and forwarding the external requests to the model request processing module and / or the platform service processing module based on the request content.
7. The system of claim 1, wherein, Each microservice orchestration code corresponds to one external request.
8. The system of claim 1, wherein, When the functional modules are called, the functional modules are set based on the parameters obtained from the fourth namespace.
9. A method for deploying a multi-modal large model, characterized in that, The system of claims 1-8 comprises the following steps: obtaining a multi-modal large model to be deployed and its parameters; obtaining model running data and storing them in the fourth namespace; cutting the multi-modal large model to be deployed into multiple functional modules to be deployed, comparing them with the existing functional modules in the third namespace, and determining new functional modules to be added; storing the new functional modules in the third namespace; storing the parameters of each functional module to be deployed in the fourth namespace; generating microservice orchestration code to set the dependency relationship and running order of each functional module to be deployed, and storing the microservice orchestration code in the second namespace; updating the microservice orchestration code, the model running data, and their corresponding external requests in the first namespace.