Medical large model reasoning and deploying method and device, equipment and medium
By constructing a structured data table for the model and matching the target inference framework, generating and validating the model inference image, the problems of increasing model parameters and inflexible multi-model architecture in traditional methods are solved, and efficient model deployment and service provision are achieved.
Patent Information
- Application Number
- CN202511623704.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-07
- Publication Date
- 2026-01-23
AI Technical Summary
Traditional methods for medical large-scale model inference and deployment face problems such as decreased inference speed due to increased model parameters, inflexible use of multi-model architectures, and waste of computing resources due to repeated inference.
By constructing a structured data table for the model, determining the model type, matching the target inference framework, generating the target model inference image, verifying and encapsulating it in a container environment, and finally deploying it to the medical cluster nodes to provide services.
It improves the flexibility and efficiency of model deployment, and can provide model capabilities through interface services, facilitating integration with application systems and platforms.
Smart Images
Figure CN121390307A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of artificial intelligence, in particular to a medical large model reasoning and deployment method and device, equipment and medium. BACKGROUND
[0002] With the rapid development of artificial intelligence technology, deep learning models have been widely applied in various fields, covering image recognition, natural language processing, autonomous driving and many other scenarios, especially in the medical field. However, with the continuous improvement of model complexity and the diversification of application scenarios, traditional model reasoning and deployment methods face many challenges. First, the increasing number of model parameters leads to a decrease in reasoning speed; second, the increasing number of scenarios makes the multi-modal architecture expand, and how to flexibly use the multi-modal architecture becomes a problem to be solved. In addition, repeated scene re-reasoning wastes computing resources, and how to efficiently cache reasoning results is also a problem to be solved.
[0003] Therefore, a new method is needed to improve the flexibility and efficiency of model deployment, and to provide the capabilities of the model to users in the form of services through an interface, providing convenience for the use of application systems or the integration of platforms. SUMMARY
[0004] Therefore, the purpose of the present application is to provide a medical large model reasoning and deployment method, device, equipment and medium, which can provide the capabilities of the medical large model to users in the form of services through an interface, thereby improving the flexibility and efficiency of model deployment. The specific scheme is as follows:
[0005] In a first aspect, the present application discloses a medical large model reasoning and deployment method, applied to a reasoning node, comprising:
[0006] Constructing a model structured data table based on the input modality, output modality and model metadata of a to-be-deployed medical large model, and determining the model type of the to-be-deployed medical large model according to the input modality and the output modality;
[0007] Determining a target reasoning framework matched with the model type from a preset reasoning framework library, to generate a target model reasoning image corresponding to the to-be-deployed medical large model according to the target reasoning framework and the model structured data table;
[0008] Performing reasoning verification on the target model reasoning image in a preset container environment, and encapsulating the reasoning verification process of the target model reasoning image in the preset container environment after verification to obtain a target reasoning service;
[0009] Deploy the target inference service to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference requirement of the preset medical cluster node through the target inference service.
[0010] Optionally, the input modality, the output modality, and the model metadata of the to-be-deployed medical large model are used to construct a model structured data table, and the model type of the to-be-deployed medical large model is determined according to the input modality and the output modality, including:
[0011] The input modality, the output modality, and the model metadata of the to-be-deployed medical large model are determined, and a model file corresponding to the to-be-deployed medical large model is read to determine the model metadata of the to-be-deployed medical large model;
[0012] The input modality, the output modality, and the model metadata are used to construct a model structured data table corresponding to the to-be-deployed medical large model;
[0013] The types of the input modality and the output modality are determined, if the input modality is an image and / or a video and the output modality is text, it is determined that the model type of the to-be-deployed medical large model is an autoregressive type;
[0014] If the input modality is text and the output modality is an image and / or a video, it is determined that the model type of the to-be-deployed medical large model is a diffusion type.
[0015] Optionally, before the target inference framework matching the model type is determined from the preset inference framework library to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table, the method further includes:
[0016] A plurality of first inference frameworks of autoregressive models and a plurality of second inference frameworks of diffusion models are collected, and a preset inference framework library is constructed based on the plurality of first inference frameworks and the plurality of second inference frameworks.
[0017] Optionally, the target inference framework matching the model type is determined from the preset inference framework library to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table, including:
[0018] If the model type is the autoregressive type, a target inference framework corresponding to the to-be-deployed medical large model is matched from the plurality of first inference frameworks in the preset inference framework library;
[0019] If the model type is the diffusion type, a target inference framework corresponding to the to-be-deployed medical large model is matched from the plurality of second inference frameworks in the preset inference framework library.
[0020] searching for a target image component corresponding to the target inference framework, and generating a target model inference image corresponding to the medical large model to be deployed according to the target image component, the target inference framework, and the model metadata in the model structured data table.
[0021] Optionally, the inference verification of the target model inference image in the preset container environment comprises:
[0022] loading the target model inference image into the preset container environment, and performing inference verification on the target model inference image through a preset test script;
[0023] Correspondingly, the inference verification of the target model inference image through the preset test script comprises:
[0024] inputting a preset test input to the target model inference image to obtain a to-be-converted inference result of the target model inference image on the preset test input;
[0025] converting the to-be-converted inference result into a target inference result corresponding to a preset output format, and verifying the target inference result to determine whether the target inference result is abnormal;
[0026] If there is no abnormality in the target inference result, it is determined that the target model inference image passes the inference verification;
[0027] If there is an abnormality in the target inference result, repairing the target model inference image based on abnormal information in the target inference result, and re-performing inference on the obtained repaired inference image.
[0028] Optionally, the inference verification process of the target model inference image in the preset container environment is encapsulated to obtain a target inference service, comprising:
[0029] establishing a mapping relationship between a preset mapping table and each image parameter in the target model inference image to obtain a target mapping table;
[0030] creating a model inference service script based on a target interface, the target mapping table, and the preset test script, and encapsulating the model inference service script into a target inference service; the target interface is an interface for receiving an inference request corresponding to the target model inference image.
[0031] Optionally, the target inference service is deployed to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference requirement of the preset medical cluster node through the target inference service, comprising:
[0032] generate a deployment file of the target inference service through a preset containerization tool, and deploy the target inference service to a preset medical cluster node through the deployment file, so that a target node in the preset medical cluster node feeds back a medical inference demand to the inference node after receiving the medical inference demand;
[0033] perform model inference corresponding to the medical inference demand based on the target inference service, and feed back a medical inference result obtained to the target node.
[0034] In a second aspect, the present application discloses an inference and deployment device of a medical large model, applied to an inference node, comprising:
[0035] a model information determination module, configured to construct a model structured data table based on an input modality, an output modality and model metadata of a to-be-deployed medical large model, and determine a model type of the to-be-deployed medical large model according to the input modality and the output modality;
[0036] an image generation module, configured to determine a target inference framework matched with the model type from a preset inference framework library, and generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table;
[0037] an image packaging module, configured to perform inference verification on the target model inference image in a preset container environment, and package an inference verification process of the target model inference image in the preset container environment after verification, to obtain a target inference service;
[0038] a service deployment module, configured to deploy the target inference service to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference demand of the preset medical cluster node through the target inference service.
[0039] In a third aspect, the present application discloses an electronic device, comprising:
[0040] a memory, configured to save a computer program;
[0041] a processor, configured to execute the computer program to implement the inference and deployment method of the medical large model as described above.
[0042] In a fourth aspect, the present application discloses a computer readable storage medium, configured to save a computer program, wherein the computer program is executed by a processor to implement the inference and deployment method of the medical large model as described above.
[0043] In the present application, the model structured data table can be constructed based on the input modality, output modality and model metadata of the to-be-deployed medical large model, and the model type of the to-be-deployed medical large model is determined according to the input modality and the output modality; a target inference framework matched with the model type is determined from a preset inference framework library, so as to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table; the target model inference image is subjected to inference verification in a preset container environment, and the inference verification process of the target model inference image in the preset container environment is encapsulated after the verification is passed, so as to obtain a target inference service; and the target inference service is deployed to a preset medical cluster node, so as to feed back a medical inference result corresponding to the medical inference requirement of the preset medical cluster node through the target inference service. As can be seen, through the method of the present application, the model structured data table can be constructed based on the input modality, output modality and model metadata of the to-be-deployed medical large model, and a target inference framework matched with the model type of the large model is determined from a preset inference framework library, so as to generate a corresponding target model inference image according to the target inference framework and the model structured data table; the target model inference image is subjected to inference verification, and the inference verification process is encapsulated after the verification is passed, so as to obtain a target inference service; and the target inference service is deployed to a preset medical cluster node, so as to feed back a medical inference result corresponding to the medical inference requirement of the preset medical cluster node through the target inference service. In this way, the deployment flexibility and efficiency of the model can be improved, and the capability of the medical large model can be provided to the user in the form of a service through an interface, thereby providing convenience for the use of the application system or the integration of the platform. BRIEF DESCRIPTION OF DRAWINGS
[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of the provided drawings.
[0045] Figure 1 A medical large model inference and deployment method flow chart disclosed by the present application;
[0046] Figure 2 A medical large model inference and deployment method processing flowchart disclosed by the present application;
[0047] Figure 3 A medical large model inference and deployment device structure diagram disclosed by the present application;
[0048] Figure 4 A structure diagram of an electronic device is disclosed in the present application. DETAILED DESCRIPTION
[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.
[0050] In the prior art, with the continuous improvement of model complexity and the diversification of application scenarios, the traditional model inference and deployment method faces many challenges. First, the increasing number of model parameters leads to a decrease in inference speed; second, the increase in scenarios causes the continuous expansion of multi-module architecture, and how to flexibly use the multi-module architecture becomes a problem to be solved. In addition, repeated scene re-inference wastes computing resources, and how to efficiently cache inference results is also a problem to be solved.
[0051] In order to overcome the above technical problems, the present application discloses a medical large model inference and deployment method, device, equipment and medium, which can provide the capabilities of the medical large model to the user in the form of services through the interface, thereby improving the deployment flexibility and efficiency of the model.
[0052] Referring to Figure 1 The embodiments of the present application disclose a medical large model inference and deployment method, applied to an inference node, comprising:
[0053] Step S11, constructing a model structured data table based on the input modality, output modality and model metadata of the medical large model to be deployed, and determining the model type of the medical large model to be deployed according to the input modality and the output modality.
[0054] In the present embodiment, as Figure 2As shown, in order to make the inference image, first, the model type needs to be determined, specifically, the input modality and output modality of the to-be-deployed medical large model need to be determined, and the model file corresponding to the to-be-deployed medical large model is read to determine the model metadata of the to-be-deployed medical large model. It needs to be explained that since the to-be-deployed medical large model is a multi-modal large model, it needs to be distinguished based on two different types of multi-modal large models, and then combined with the combination of model input and output modalities for division. The input of the autoregressive multi-modal large model is image or video, and the output is text. The input of the diffusion multi-modal large model contains text, and the output is image or video. Therefore, the model type can be determined according to the input modality and output modality of the model, and the model file corresponding to the to-be-deployed medical large model needs to be read to determine the model metadata of the to-be-deployed medical large model, wherein the model metadata at least includes: input modality set (text / image / video) and output modality (text / image / video); model architecture identifier, resolution, precision (fp16 / bf16); model file size and GPU (Graphics Processing Unit, graphics processor) memory requirement threshold.
[0055] Further, the model structured data table corresponding to the to-be-deployed medical large model needs to be constructed based on the input modality, output modality and model metadata. It needs to be explained that the model structured data table is a dynamic model registry (Huggingface) ), which needs to maintain the mapping relationship between the model architecture identifier and the Diffusers pipeline class and the dispatcher, as well as various model information. It needs to be explained that maintaining the information of the to-be-deployed medical large model and determining the model type is a key decision point in the entire deployment process, and a structured data table needs to be established to record the model deployment information metadata in this process. Taking the diffusion-based text-to-image generation model stable-diffusion-xl-base-1.0 as an example, the model information is as follows: this model can complete image generation according to text prompts, and can also complete image modification output based on text prompts and input images. Therefore, the input modality of this model is recorded as text and image, and the output modality is image, and the precision, model size and required memory space information of this model are maintained, which prepares for subsequent deployment.
[0056] ;
[0057] Further, the type of input modality and output modality needs to be determined. If the input modality is an image and / or a video and the output modality is text, it is determined that the model type of the medical large model to be deployed is autoregressive. If the input modality is text and the output modality is an image and / or a video, it is determined that the model type of the medical large model to be deployed is diffusion. Therefore, the model type determination logic can be further determined as follows: if the output modality is text, it is classified as an autoregressive model, and if the output modality is an image / video, it is classified as a diffusion model.
[0058] In step S12, a target inference framework matching the model type is determined from a preset inference framework library to generate a target model inference image corresponding to the medical large model to be deployed according to the target inference framework and the model structured data table.
[0059] In this embodiment, a target inference framework matching the model type needs to be determined from a preset inference framework library. It needs to be noted that before determining the target inference framework, the preset inference framework library needs to be constructed first. Specifically, a plurality of first inference frameworks of autoregressive models and a plurality of second inference frameworks of diffusion models are collected, and the preset inference framework library is constructed based on the plurality of first inference frameworks and the plurality of second inference frameworks. It needs to be noted that in a multi-modal understanding model mainly using autoregressive models, a visual-language model with an image or video as input modality and text as output modality is the most common type, such as LLaVA (Large Language and Vision Assistant), Flamingo, etc. The inference image thereof needs to contain a specific decoding library to support efficient token-by-token sequence generation. VLLM (Very Large Language Model) itself as a text generation inference engine can support a multi-modal large model with a "visual encoder + text generator" architecture. Only the output (image feature vector) of the visual encoder needs to be spliced into the text input as the "visual prefix" (Visual Prefix) of the LLM (Large Language Mode), and the subsequent text generation can be efficiently performed by the VLLM. For multi-modal generation models mainly using diffusion models, such as text-to-image models like Stable Diffusion, text-to-video models, etc., the inference image thereof needs to contain a special library supporting the diffusion process, such as the Diffusers library, the ComfyUI backend engine, etc. The image / video decoder and the scheduler need to be optimized in terms of execution efficiency, and a GPU-accelerated vision processing library needs to be integrated.
[0060] Further, as Figure 2As shown, according to the model type, if the model type is autoregressive type, a target inference framework corresponding to the to-be-deployed medical large model is matched from a plurality of first inference frameworks in a preset inference framework library; if the model type is diffusion type, a target inference framework corresponding to the to-be-deployed medical large model is matched from a plurality of second inference frameworks in the preset inference framework library; a target image component corresponding to the target inference framework is searched, and a target model inference image corresponding to the to-be-deployed medical large model is generated according to the target image component, the target inference framework, and model metadata in a model structured data table. Taking the Diffusers inference image as an example, first, the Diffusers library provides a predefined model pipeline, different types of schedulers, and a standard interface for model component loading, then the transformers library for processing text input, and the accelerate library for helping to implement distributed inference and hardware acceleration. In addition, in order to maximize the utilization of GPU and inference speed, the image needs to be deeply integrated with CUDA (Compute Unified Device Architecture), cuDNN (CUDA Deep Neural Network library), and other underlying computing libraries, and usually contains the xformers library to optimize the computational efficiency and memory occupancy of the attention mechanism. Finally, in order to adapt to service deployment, a lightweight and efficient HTTP (Hypertext Transfer Protocol, Hypertext Transfer Protocol) / REST (Representational State Transfer, Representational State Transfer) or gRPC (google Remote Procedure Call, Remote Procedure Call) service interface (usually based on FastAPI or similar asynchronous framework) needs to be encapsulated in the inference image to receive requests containing text prompts (Prompt), negative prompts (Negative Prompt), generation parameters such as step number, guidance intensity, seed, size, etc., drive the Diffusers pipeline to perform a complete diffusion generation process, including text encoding, latent space diffusion, image decoding, etc., and finally return the generated image or video data to the caller after encoding. Taking a Dockerfile example for a Diffusers multi-modal generation model and an image making process as an example:
[0061] ;
[0062] In this way, for multi-modal generation models, dependencies such as diffusers, transformers, Fastapi, uvicorn, etc. are imported on the pytorch base image to build a custom Diffusers inference image, realizing flexible production of multi-modal medical large models.
[0063] Step S13: Perform inference verification on the target model inference image in a preset container environment, and after the verification is passed, encapsulate the inference verification process of the target model inference image in the preset container environment to obtain the target inference service.
[0064] In this embodiment, as Figure 2 As shown, the image needs to be started and verified. Specifically, the target model inference image needs to be loaded into a preset container environment, and the target model inference image needs to be verified through a preset test script. A preset test input needs to be input into the target model inference image to obtain the inference result to be converted from the preset test input. Then, the inference result to be converted is converted into the target inference result corresponding to the preset output format, and the target inference result is verified to determine whether the target inference result is abnormal. It should be noted that the preset output format is Base64 encoded. Further, considering that the inference results of diffusion models such as text-based graphs are in the form of images or videos, and the inference service called through the interface has poor support for such modalities, the test returns the image in a text-encoded format in this step to facilitate better result transmission and processing later. Therefore, in the test script, the model's output image needs to be converted to Base64 encoded format and verified through log output or return value to ensure the accuracy and usability of the model inference.
[0065] If anomalies are found in the target inference result, the target model inference image is repaired based on the anomaly information, and the repaired inference image is re-inferred until no anomalies are found in the target inference result. If no anomalies are found in the target inference result, the target model inference image is deemed to have passed inference verification. Then, a mapping relationship between a preset mapping table and each image parameter in the target model inference image needs to be established to obtain the target mapping table. Based on the target interface, the target mapping table, and the preset test script, a model inference service script is created and encapsulated as a target inference service. The target interface is the interface that receives inference requests corresponding to the target model inference image. It should be noted that the model inference service script is a service-oriented inference script written in the working directory of the preset container environment, allowing external applications to call the model inference service via HTTP requests. Furthermore, the target interface is designed according to RESTful architecture principles, supporting common HTTP methods such as GET and POST, facilitating users to make inference requests and obtain results from the model. It should be noted that in the model inference service script for the diffusion model, a mapping relationship between a preset mapping table and each image parameter in the target model inference image needs to be established, for example, maintaining a... The mapping relationship to the pipeline and the scheduler enables the Diffusers tool to load the model and complete inference using the correct pipeline and scheduler, for example:
[0066] .
[0067] In this way, by encapsulating inference functions as services, clients can access these services through network requests to obtain inference results. This approach overcomes the inflexibility of deployment and inference applications in traditional methods and meets the needs of different types of intelligent models and application scenarios.
[0068] Step S14, deploying the target inference service to the preset medical cluster node, so as to feed back the medical inference result corresponding to the medical inference demand of the preset medical cluster node through the target inference service.
[0069] In the embodiment, as shown in Figure 3 , it is necessary to deploy the target inference service to the preset medical cluster node. Specifically, a deployment file of the target inference service needs to be generated through a preset containerization tool, and the target inference service is deployed to the preset medical cluster node through the deployment file. It should be noted that the preset containerization tool is Kubernetes, which automatically manages the life cycle, resource allocation, load balancing, and fault recovery of containers through its container orchestration capabilities. Then, by writing the deployment file (Deployment), service file (Service), and horizontal automatic expansion file (Horizontal Pod Autoscaler) of Kubernetes, the container of the inference service is deployed to the preset medical cluster node. The deployment file defines the image, number of replicas, resource limits, and other key parameters of the container, and the service file is used to expose the port of the inference service, so that external requests can access the inference service. Further, after the target node in the preset medical cluster node receives the medical inference demand, it feeds back the medical inference demand to the inference node, then performs model inference corresponding to the medical inference demand through the target inference service, and feeds back the obtained medical inference result to the target node. In this way, by providing the user with the capabilities of the model through the interface in the form of a service, it provides convenience for the use of application systems or the integration of platforms, and improves the application value of multi-modal large models.
[0070] In this embodiment, the model structured data table can be constructed based on the input modality, output modality and model metadata of the to-be-deployed medical large model, and the model type of the to-be-deployed medical large model is determined according to the input modality and the output modality; a target inference framework matched with the model type is determined from a preset inference framework library, so as to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table; inference verification is performed on the target model inference image in a preset container environment, and the inference verification process of the target model inference image in the preset container environment is encapsulated after the verification is passed, so as to obtain a target inference service; and the target inference service is deployed to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference requirement of the preset medical cluster node through the target inference service. As can be seen, through the method of the present application, the model structured data table can be constructed based on the input modality, output modality and model metadata of the to-be-deployed medical large model, and the target inference framework matched with the model type of the large model is determined from the preset inference framework library, so as to generate the corresponding target model inference image according to the target inference framework and the model structured data table; the target model inference image is subjected to inference verification, and the inference verification process is encapsulated after the verification is passed, so as to obtain the target inference service; and the target inference service is deployed to the preset medical cluster node, so as to feed back the medical inference result corresponding to the medical inference requirement of the preset medical cluster node through the target inference service. In this way, the deployment flexibility and efficiency of the model can be improved, and the capability of the medical large model can be provided to the user in the form of a service through an interface, thereby providing convenience for the use of the application system or the integration of the platform.
[0071] Referring to Figure 4 As shown in the figure, the embodiment of the present application discloses an inference and deployment device of a medical large model, applied to an inference node, comprising:
[0072] A model information determination module 11 is configured to construct a model structured data table based on the input modality, output modality and model metadata of a to-be-deployed medical large model, and determine the model type of the to-be-deployed medical large model according to the input modality and the output modality;
[0073] An image generation module 12 is configured to determine a target inference framework matched with the model type from a preset inference framework library, so as to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table;
[0074] The mirror packaging module 13 is configured to perform inference verification on the target model inference mirror in a preset container environment, and package the inference verification process of the target model inference mirror in the preset container environment after the verification is passed, to obtain a target inference service.
[0075] The service deployment module 14 is configured to deploy the target inference service to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference demand of the preset medical cluster node through the target inference service.
[0076] In the embodiment, the model structured data table can be constructed based on the input modality, the output modality and the model metadata of the to-be-deployed medical large model, and the model type of the to-be-deployed medical large model can be determined according to the input modality and the output modality; the target inference framework matched with the model type is determined from a preset inference framework library, so as to generate the target model inference mirror corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table; the inference verification is performed on the target model inference mirror in a preset container environment, and the inference verification process of the target model inference mirror in the preset container environment is packaged after the verification is passed, to obtain a target inference service; and the target inference service is deployed to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference demand of the preset medical cluster node through the target inference service. Therefore, by the method of the present application, the model structured data table can be constructed based on the input modality, the output modality and the model metadata of the to-be-deployed medical large model, and the target inference framework matched with the model type of the large model is determined from a preset inference framework library, so as to generate the corresponding target model inference mirror according to the target inference framework and the model structured data table; the inference verification is performed on the target model inference mirror, and the inference verification process is packaged after the verification is passed, to obtain a target inference service; and the target inference service is deployed to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference demand of the preset medical cluster node through the target inference service. In this way, the deployment flexibility and efficiency of the model can be improved, and the capability of the medical large model can be provided to the user in the form of a service through an interface, thereby providing convenience for the use of the application system or the integration of the platform.
[0077] In some embodiments, the model information determination module 11 can specifically include:
[0078] The data determination unit is configured to determine the input modality and the output modality of the to-be-deployed medical large model, and read a model file corresponding to the to-be-deployed medical large model, to determine model metadata of the to-be-deployed medical large model;
[0079] a data table construction unit configured to construct a model structured data table corresponding to the medical large model to be deployed based on the input modal, the output modal, and the model metadata;
[0080] a first model type determination unit configured to determine the types of the input modal and the output modal, and determine the model type of the medical large model to be deployed as an autoregressive type if the input modal is an image and / or a video and the output modal is text;
[0081] a second model type determination unit configured to determine the model type of the medical large model to be deployed as a diffusion type if the input modal is text and the output modal is an image and / or a video.
[0082] In some embodiments, the inference and deployment apparatus of the medical large model can further include:
[0083] a reasoning framework library construction unit configured to collect a plurality of first reasoning frameworks of a plurality of autoregressive models and collect a plurality of second reasoning frameworks of a plurality of diffusion models, and construct a preset reasoning framework library based on the plurality of first reasoning frameworks and the plurality of second reasoning frameworks.
[0084] In some embodiments, the image generation module 12 can specifically include:
[0085] a first framework determination unit configured to match a target reasoning framework corresponding to the medical large model to be deployed from the plurality of first reasoning frameworks in the preset reasoning framework library if the model type is the autoregressive type;
[0086] a second framework determination unit configured to match a target reasoning framework corresponding to the medical large model to be deployed from the plurality of second reasoning frameworks in the preset reasoning framework library if the model type is the diffusion type;
[0087] an image generation unit configured to find a target image component corresponding to the target reasoning framework, and generate a target model inference image corresponding to the medical large model to be deployed according to the target image component, the target reasoning framework, and the model metadata in the model structured data table.
[0088] In some embodiments, the image packaging module 13 can specifically include:
[0089] an image inference submodule configured to load the target model inference image into a preset container environment, and perform inference verification on the target model inference image through a preset test script;
[0090] In some embodiments, the image inference submodule can specifically include:
[0091] The mirror inference unit is configured to infer a mirror input preset test input to the target model to obtain a to-be-converted inference result of the target model in response to the preset test input;
[0092] The inference verification unit is configured to convert the to-be-converted inference result into a target inference result corresponding to a preset output format, and verify the target inference result to determine whether the target inference result is abnormal;
[0093] The verification result determination unit is configured to determine that the target model inference mirror passes the inference verification if there is no abnormality in the target inference result.
[0094] The re-inference unit is configured to repair the target model inference mirror based on abnormal information in the target inference result if there is abnormality in the target inference result, and re-infer the repaired inference mirror.
[0095] In some embodiments, the mirror packaging module 13 can specifically include:
[0096] The mapping table establishment unit is configured to establish a mapping relationship between a preset mapping table and each mirror parameter in the target model inference mirror to obtain a target mapping table.
[0097] The mirror packaging unit is configured to create a model inference service script based on a target interface, the target mapping table, and the preset test script, and package the model inference service script into a target inference service; the target interface is an interface for receiving an inference request corresponding to the target model inference mirror.
[0098] In some embodiments, the service deployment module 14 can specifically include:
[0099] The service deployment unit is configured to generate a deployment file of the target inference service through a preset containerization tool, and deploy the target inference service to a preset medical cluster node through the deployment file, so that a target node in the preset medical cluster node feeds back a medical inference demand to the inference node after receiving the medical inference demand.
[0100] The inference result feedback unit is configured to perform model inference corresponding to the medical inference demand based on the target inference service, and feed back a medical inference result obtained to the target node.
[0101] Further, the embodiments of the present application also disclose an electronic device, Figure 4 is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be considered as any limitation on the use range of the present application.
[0102] Figure 4 A structural schematic diagram of an electronic device 20 is provided in the embodiments of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is configured to store a computer program, and the processor 21 is configured to load and execute the computer program to implement the related steps in the inference and deployment method of the medical large model disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the embodiments of the present application can be specifically an electronic computer.
[0103] In the embodiments of the present application, the power supply 23 is configured to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solutions of the present application, which is not limited specifically herein; the input / output interface 25 is configured to obtain external input data or output data to the outside, and the specific interface type can be selected according to the specific application needs, which is not limited specifically herein.
[0104] In addition, the memory 22 as a carrier for resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage mode can be temporary storage or permanent storage.
[0105] The operating system 221 is configured to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the inference and deployment method of the medical large model executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.
[0106] Further, the present application further discloses a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the inference and deployment method of the medical large model disclosed in the foregoing embodiments. For the specific steps of the method, refer to the corresponding contents disclosed in the foregoing embodiments, which will not be repeated here.
[0107] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.
[0108] Those skilled in the art will further appreciate that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, various components have been described above generally in terms of their functionality, without referring to the details of their implementation. Whether such functionality is implemented in hardware or software depends on the particular application and design constraints imposed on the overall system. Skilled persons can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
[0109] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in random access memory (RAM), flash memory, read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0110] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but are used to distinguish one element from another, and cannot imply or infer any such relationship or order between the elements. Moreover, the terms "comprise", "comprises", or any other variant thereof are intended to cover non-exclusive inclusions, such that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed, or other elements inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.
[0111] The above provides a detailed description of the technical solutions of the present application. The principles and implementation modes of the present application are described by specific examples. The above description of the examples is only to help understand the method and core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as limiting the present application.
Claims
1. A medical large model inference and deployment method, characterized in that, The application is applied to an inference node, and comprises the following steps of: constructing a model structured data table based on input modalities, output modalities and model metadata of a to-be-deployed medical large model, and determining a model type of the to-be-deployed medical large model according to the input modalities and the output modalities; determining a target inference framework matched with the model type from a preset inference framework library, so as to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table; performing inference verification on the target model inference image in a preset container environment, and encapsulating an inference verification process of the target model inference image in the preset container environment after verification, so as to obtain a target inference service; deploying the target inference service to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference requirement of the preset medical cluster node through the target inference service. 2.The medical large model inference and deployment method of claim 1, wherein, The method comprises the following steps of: determining input modalities, output modalities and reading a model file corresponding to the to-be-deployed medical large model, so as to determine model metadata of the to-be-deployed medical large model; constructing a model structured data table corresponding to the to-be-deployed medical large model based on the input modalities, the output modalities and the model metadata; judging types of the input modalities and the output modalities, and determining that the model type of the to-be-deployed medical large model is a self-recurrence type if the input modalities are images and / or videos and the output modalities are texts; determining that the model type of the to-be-deployed medical large model is a diffusion type if the input modalities are texts and the output modalities are images and / or videos. 3.The medical large model inference and deployment method of claim 2, wherein, Before the step of determining a target inference framework matched with the model type from a preset inference framework library, so as to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table, the method further comprises the following steps of: collecting a plurality of first inference frameworks of a plurality of self-recurrence models and a plurality of second inference frameworks of a plurality of diffusion models, and constructing a preset inference framework library based on the plurality of first inference frameworks and the plurality of second inference frameworks. 4.The medical large model inference and deployment method of claim 3, The step of determining a target inference framework matched with the model type from a preset inference framework library, so as to generate a target model inference image corresponding to the to-be-deployed medical large model according to the target inference framework and the model structured data table, comprises the following steps of: if the model type is the self-recurrence type, matching a target inference framework corresponding to the to-be-deployed medical large model from the plurality of first inference frameworks in the preset inference framework library; if the model type is the diffusion type, matching a target inference framework corresponding to the to-be-deployed medical large model from the plurality of second inference frameworks in the preset inference framework library; search for a target image component corresponding to the target inference framework, and generate a target model inference image corresponding to the medical large model to be deployed according to the target image component, the target inference framework, and the model metadata in the model structured data table. 5.The medical large model inference and deployment method of claim 1, wherein, The inference verification of the target model inference image in the preset container environment comprises: loading the target model inference image into the preset container environment, and performing inference verification on the target model inference image through a preset test script; Correspondingly, the inference verification of the target model inference image through the preset test script comprises: inputting a preset test input into the target model inference image to obtain a to-be-converted inference result of the target model inference image on the preset test input; convert the to-be-converted inference result into a target inference result corresponding to a preset output format, and verify the target inference result to determine whether the target inference result is abnormal; if there is no abnormality in the target inference result, it is determined that the target model inference image passes the inference verification; if there is an abnormality in the target inference result, repair the target model inference image based on the abnormal information in the target inference result, and re-perform inference on the obtained repaired inference image. 6.The medical large model reasoning and deployment method of claim 5, wherein, The inference verification process of the target model inference image in the preset container environment is encapsulated to obtain a target inference service, which comprises: establishing a mapping relationship between a preset mapping table and each image parameter in the target model inference image to obtain a target mapping table; creating a model inference service script based on a target interface, the target mapping table, and the preset test script, and encapsulating the model inference service script into a target inference service; the target interface is an interface for receiving an inference request corresponding to the target model inference image.
7. The medical large model reasoning and deployment method according to any one of claims 1 to 6, characterized in that, The target inference service is deployed to a preset medical cluster node, so that the target inference service feeds back a medical inference result corresponding to a medical inference requirement of the preset medical cluster node, which comprises: generating a deployment file of the target inference service through a preset containerization tool, and deploying the target inference service to a preset medical cluster node through the deployment file, so that a target node in the preset medical cluster node feeds back a medical inference requirement to the inference node after receiving the medical inference requirement; based on the target inference service, performing model inference corresponding to the medical inference requirement, and feeding back the obtained medical inference result to the target node.
8. An apparatus for inference and deployment of a medical large model, characterized in that, Applied to an inference node, comprising: a model information determination module configured to construct a model structured data table based on input modalities, output modalities, and model metadata of a medical large model to be deployed, and determine a model type of the medical large model to be deployed according to the input modalities and the output modalities; an image generation module configured to determine a target inference framework matching the model type from a preset inference framework library, and generate a target model inference image corresponding to the medical large model to be deployed according to the target inference framework and the model structured data table; The mirror packaging module is used for performing inference verification on the target model inference mirror in a preset container environment, and packaging the inference verification process of the target model inference mirror in the preset container environment after the verification is passed, to obtain a target inference service; The service deployment module is used for deploying the target inference service to a preset medical cluster node, so as to feed back a medical inference result corresponding to a medical inference demand of the preset medical cluster node through the target inference service.
9. An electronic device, comprising: Comprise: A memory for saving a computer program; A processor for executing the computer program to implement the inference and deployment method of the medical large model according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, A memory for saving a computer program, wherein the computer program is executed by a processor to implement the inference and deployment method of the medical large model according to any one of claims 1 to 7.