Service update method, electronic device, storage medium and program product

Abstract

The present application discloses a service update method, electronic device, storage medium and program product, which relate to the field of cloud computing technology, including: when receiving a service update request sent by a client, the first storage path of the original reasoning framework code can be determined based on the service indication information and location indication information included in the service update request, and then the original version information is read based on the first storage path. When it is determined to perform an update operation based on the updated version information and the original version information, the update operation is performed on the original reasoning framework code stored under the first storage path according to a preset pull method. When it is detected that the update operation is completed, the updated reasoning framework code is read from the first storage path. Based on the updated reasoning framework code, the reasoning framework container is recreated and started. In this way, only the original reasoning framework code needs to be updated, and after the update, only the reasoning framework container needs to be restarted, so that the reasoning framework service can be timely provided.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a service update method, electronic device, storage medium, and program product. Background Art

[0002] In the field of cloud computing, container groups are often used to deploy various services, such as question-answering services and inference services, to improve resource utilization. Typically, each compute node in a computer cluster can be provided with multiple accelerator cards, and each accelerator card can be deployed with at least one container group.

[0003] In the inference service, as user needs continue to change, the inference framework code will change, and thus the container group needs to be updated. When performing updates, in order to avoid affecting the inference service, a rolling update mechanism is generally adopted. That is, the image file of the updated inference service is first obtained, and a new container group is created and started based on the updated inference service image file. When the new container group is detected to have started successfully, the business flow of the original container group is first switched to the new container group, and then the old container group is deleted. However, in order to fully utilize the accelerator card resources, users usually occupy all accelerator cards when deploying services on the computing nodes. As a result, when updating the container group, there are no additional accelerator cards available for deploying the new container group, resulting in the inference framework service being unable to complete the update in a timely manner. Summary of the Invention

[0004] This application provides a service update method, device, electronic device, storage medium and program product to solve the problem of untimely update of reasoning framework services.

[0005] This application provides a service update method, including:

[0006] receiving a service update request sent by a client, wherein the service update request includes service indication information, location indication information, and update version information;

[0007] determining original version information of the original reasoning framework code according to the service indication information and the location indication information;

[0008] When it is determined to perform an update operation based on the updated version information and the original version information, the update operation is performed on the original reasoning framework code according to the preset update method and the updated version information;

[0009] When the update operation is detected to be complete, the reasoning framework container is restarted based on the updated reasoning framework code to complete the update operation of the reasoning framework service.

[0010] This application also provides a service update device, including:

[0011] A receiving module, configured to receive a service update request sent by a client, wherein the service update request includes service indication information, location indication information, and update version information;

[0012] A determination module, configured to determine original version information of the original reasoning framework code according to the service indication information and the location indication information;

[0013] An update module, configured to, when determining to perform an update operation based on the updated version information and the original version information, perform an update operation on the original reasoning framework code according to a preset update method and the updated version information;

[0014] The restart module is used to restart the reasoning framework container based on the updated reasoning framework code when detecting that the update operation is completed, so as to complete the update operation of the reasoning framework service.

[0015] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned service update methods when executing the computer program.

[0016] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned service update methods are implemented.

[0017] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned service update methods when executed by a processor.

[0018] Through this application, when receiving a service update request from a client, the first storage path of the original reasoning framework code can be determined based on the service indication information and location indication information included in the service update request. Then, based on the first storage path, the original version information of the original reasoning framework code can be read. When it is determined to perform an update operation based on the updated version information and the original version information, the update operation is performed on the original reasoning framework code stored in the first storage path according to a preset pull method. When the update operation is detected to be complete, the updated reasoning framework code is read from the storage path. Based on the updated reasoning framework code, the reasoning framework container is restarted. In this way, only the original reasoning framework code needs to be updated, and after the update, only the reasoning framework container needs to be restarted. Even without an additional accelerator card, the updated reasoning framework code can be applied promptly, that is, the update of the reasoning framework service can be completed promptly. In addition, the related art requires restarting the container group. Because the container group includes the inference engine, the startup time of the inference engine is long, so restarting the container group is slow. However, this solution only requires restarting the inference framework container, which can improve the update speed of the entire service and apply the new reasoning framework code more promptly. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A schematic diagram of the architecture of a container group orchestration platform provided in an embodiment of the present application;

[0021] Figure 2 A flowchart of a service update method provided in an embodiment of the present application;

[0022] Figure 3 A flow chart of a service deployment method provided in an embodiment of the present application;

[0023] Figure 4 A flowchart of a data recording and storage method provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the architecture of a container group deployment provided in an embodiment of the present application;

[0025] Figure 6 A flowchart of another service deployment method provided in an embodiment of the present application;

[0026] Figure 7 A flowchart of another service update method provided in an embodiment of the present application;

[0027] Figure 8 A schematic diagram of a process flow of a service updating device provided in an embodiment of the present application;

[0028] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0031] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0032] The professional terms involved in this application are explained in detail below.

[0033] Container: A lightweight, portable, self-contained software packaging and runtime environment that packages an application and all its dependencies into a standardized unit.

[0034] Pod: The smallest scheduling and management unit in the container group orchestration platform. It is a collection of one or more closely related containers that can share storage and network resources.

[0035] Inference service: This refers to the task process of performing predictions (inference) on deep learning models in actual application scenarios. It is used to provide practical services, such as face recognition, speech translation, and recommendation systems. For example, a deep learning model can be a large language model (LLM), which will be referred to as a model in the following.

[0036] Inference framework: refers to software that provides full-process support for model inference. It is used to provide an Application Programming Interface (API) for the inference engine, interact with the inference engine using preconfigured chat templates (for example, chat templates using Jinja2 syntax), manage the inference engine lifecycle, and monitor the inference engine, such as model loading and unloading, version control, resource allocation, and health checks.

[0037] Inference Engine: refers to the runtime core responsible for hardware underlying acceleration and efficient calculation execution. It is used to deploy model files to the accelerator card and use the accelerator card to perform model inference and complete the inference business.

[0038] A persistent volume (PVR) is a storage resource in a computer cluster that can be pre-configured by an administrator or dynamically configured using a storage class. The storage class can be a Network File System (NFS) class.

[0039] Persistent Volume Claim: A declaration of storage resource requirements (e.g., resource size, access mode, etc.). Container orchestration platforms typically bind or create corresponding PVs based on PVCs.

[0040] The service update method provided in this application can be applied to the container group orchestration platform. Figure 1 As shown, the container group orchestration platform may include at least one computing node. The computing node may be a server, for example, an NFS server. When the computer cluster includes only one computing node, the computing node may serve as both a management node and a storage node. Alternatively, when the computer cluster includes multiple computing nodes, one computing node may be set as a management node among the multiple computing nodes, and each computing node may serve as a storage node. The above-mentioned computing nodes may only perform computing services (for example, deploying container groups), and accordingly, storage nodes may also be provided in the computer cluster. Each computing node may include a central processing unit (CPU), memory, multiple acceleration cards, etc. For example, the acceleration card may be a graphics processing unit (GPU).

[0041] The container group may be a business container group, a code management container group, or the like.

[0042] A business container group can provide actual services for external clients or webpage calls. This business container group can be an inference business container group, which provides actual inference services for external clients or webpage calls. For example, it can be an LLM inference business container group. The inference business container group can include an initial full-data pull container, an inference engine container, and an inference framework container. Both the inference engine container and the inference framework container are container instances. Initializing the full-data pull container is executed only once when the inference business container group starts. It pulls the full code and starts the inference engine and framework containers. The inference engine container deploys model files to the accelerator card and uses the accelerator card to perform model inference and complete the inference service. The inference framework container provides an application programming interface (API) for interacting with the inference engine container. Furthermore, the inference framework container manages the inference engine container's lifecycle and monitors its performance metrics. The business container group can be mounted on at least one PV to store relevant code files, such as the inference framework code. The inference framework code can be written in Python.

[0043] The code management container group can be used to manage the code files stored in the PVs mounted by different business container groups, and to independently update and centrally manage the code files stored in the PVs. The code management container group can include a code management container.

[0044] When the inference business container group is performing inference business, if the inference framework code is updated, the inference framework code needs to be updated in a timely manner, and the inference business is executed based on the updated inference framework code.

[0045] The following is a detailed explanation using the container group orchestration platform including a computing node as an example.

[0046] The embodiment of the present application provides a service update method that can be executed by the above-mentioned computing node to complete the update and application of the reasoning framework code in a timely manner. Figure 2 As shown, the specific processing steps of the service update method may include:

[0047] Step S201: receiving a service update request sent by a client.

[0048] Among them, the service update request may include service indication information, location indication information, and update version information. The service indication information may include one or more of scenario identification information, service identification information, etc. The scenario identification information can be used to indicate the specific environment on which the service depends, and may be a scenario name, for example, "inference", "llm", "chat". The service identification information can be used to distinguish different services, and the service names may be, for example, "qwen3-instruce", "deepseek-r1", "inf-svc", "ocr", etc. The location indication information can be used to indicate the storage location of the relevant code involved in the service, and may be a PVC name, for example, "qwen3infcode", "dsr1", etc. The update version information may be a version number.

[0049] Specifically, a user can perform an update operation on the client (for example, by entering the update version information in any service interface and clicking the Update button). After detecting the user's update operation, the client generates a service update request (carrying the update version information) and sends it to the corresponding computing node. Upon receiving the service update request from the client, the computing node can parse the service update request to obtain service indication information, location indication information, and update version information.

[0050] Step S202: determining the original version information of the original reasoning framework code according to the service indication information and the location indication information.

[0051] Specifically, since the version specified by the user may be consistent with the version of the original reasoning framework code, if the service update operation is directly triggered upon receiving the service update request, it will result in resource waste (resources will be wasted by restarting the container). Therefore, to avoid this problem, the computing node can first determine the original version information of the original reasoning framework code based on the service indication information and location indication information for subsequent version comparison operations. Accordingly, the computing node can find the original version information according to the following specific steps:

[0052] Step 1: Determine a first storage path of the original reasoning framework code according to the service indication information and the location indication information.

[0053] Step 1: Obtain the second storage path where the pre-built code management container is mounted.

[0054] The lower-level path of the second storage path may include at least one sub-path. The second storage path may be the storage path corresponding to the PV mounted by the code management container group to which the code management container belongs. The sub-path may be the storage path corresponding to the PV mounted by the business container group, for example, the storage path corresponding to the PV mounted by the LLM reasoning business container group. The sub-path format may be "scenario identification information-location indication information-unique identifier", where the unique identifier may be a universally unique identifier (UUID). Since PVs can be created through storage classes, a host path must be specified during creation, i.e., the path corresponding to the PV mounted by each business container group is a sub-path under the storage class path under the host path. Therefore, the code management container group only needs to mount the path corresponding to the storage class to manage the code files stored in all PVs. For example, the host path can be / mnt / inaisfs / , and the subpath can be / mnt / inaisfs / llm-codeV1-21b8e9ec-187c-11f0-b184-0bfa87fb8069. "llm" can be the scenario identifier, "codeV1" can be the name of the PV storing the inference framework code, and "codeV1-21b8e9ec-187c-11f0-b184-0bfa87fb8069" can be the UUID. The code management container group can be mounted to / mnt / inaisfs / (the secondary storage path) to manage the code of all business container groups.

[0055] Step 2: According to the service indication information and the location indication information, determine a sub-path that matches both the service indication information and the location indication information in at least one sub-path as a first storage path.

[0056] Specifically, because the second storage path mounted by the code management container group to which the code management container belongs is located above the storage path mounted by the business container group (i.e., the aforementioned sub-path), the computing node can first obtain the second storage path mounted by the code management container group, and then, in the lower-level path of the second storage path, determine a sub-path that matches both the service indication information and the location indication information as the first storage path. In an LLM scenario, the inference framework used by different services can be the same. When matching the first storage path, the matching operation can be performed solely based on the scenario identification information in the service indication information in combination with the location indication information. Alternatively, if different services in an LLM scenario use different inference frameworks, the matching operation can be performed based on a combination of the scenario identification information, service identification information, and location indication information in the service indication information.

[0057] For example, if the same inference framework applies to all services in the LLM scenario, the storage path structure can be:

[0058] " / data / code-root

[0059] ├──chat-qwen7b-chat-123abc

[0060] ├──llm-qwen3infcode-456def

[0061] └──…”

[0062] " / data / code-root" can be the second storage path mentioned above, and "chat-qwen7b-chat -123abc" and "llm-qwen3infcode-456def" are both sub-paths. When the service indicator is "llm" and the location indicator is "qwen3infcode", the determined first storage path is "llm-qwen3infcode-456def". All LLM inference service container groups can be mounted to the same storage path, for example, all mounted to "llm-qwen3infcode-456def".

[0063] By establishing a dedicated code management container group, mounting the code management container group to a specified path, and mounting each business container group to a lower-level path of the code management container group mounted to the specified path, centralized management of code files can be achieved. In addition, during the code update process, the service indication information and location indication information can be simply matched to determine the corresponding first storage path, accurately determining the location of the inference framework code that needs to be updated.

[0064] Under the above storage path structure, users can access any sub-path under the second storage path mounted by the code management container through the client, view, modify, and delete the code files under any sub-path, and realize centralized management of the code.

[0065] Step 2: Read the original version information based on the first storage path.

[0066] Step 1: Based on the first storage path, read the original reasoning framework code from the storage location corresponding to the first storage path.

[0067] Step 2: According to the preset field identification information, determine the target field corresponding to the preset field identification information in the original reasoning framework code.

[0068] Step 3: Extract the original version information from the target field.

[0069] Specifically, the computing node can read the original reasoning framework code from the storage location corresponding to the first storage path based on the first storage path, and then, based on the preset field identification information, determine the target field corresponding to the preset field identification information in the original reasoning framework code, and extract the original version information from the target field.

[0070] Alternatively, the computing node can directly read the original version information from the storage location corresponding to the first storage path based on the first storage path. In this way, the original version information of the currently used inference framework code can be promptly determined through the first storage path, facilitating the subsequent accurate determination of whether to perform an update operation and avoiding resource waste.

[0071] In some optional implementations, the computing node may also determine the first storage path in the following manner:

[0072] Based on the service indication information and the location indication information, a target unique identifier that matches both the service indication information and the location indication information is determined in a pre-built correspondence table; then, based on the second storage path, the service indication information, the location indication information, and the second storage path, a first storage path is generated.

[0073] In this way, the first storage path can be directly generated, and the original reasoning framework code can be read without performing a storage path search operation, which is more convenient.

[0074] Step S203 : When it is determined to perform an update operation based on the updated version information and the original version information, the update operation is performed on the original reasoning framework code based on the preset update method and the updated version information.

[0075] The preset update mode may be an incremental update mode or a full update mode.

[0076] Specifically, the computing node can determine whether the updated version information is consistent with the original version information. If so, it determines to perform the update operation; if not, it determines not to perform the update operation, which can save resources. When determining to perform the update operation, the computing node can perform the update operation on the original reasoning framework code according to the preset update method and the updated version information. Accordingly, the computing node can perform the following specific steps:

[0077] Step 1: According to the preset update method and update version information, the update code of the inference framework is obtained from the preset storage location.

[0078] Specifically, to improve update flexibility, this solution provides multiple methods. Computing nodes can perform corresponding update operations based on preset update methods. Specifically, they first obtain storage location information corresponding to the preset update method based on the preset update method. Then, based on the preset update method and update version information, they obtain the updated code for the inference framework from the storage location corresponding to the storage location information. Specifically, these two methods can be used:

[0079] Method 1: When the preset update method is incremental update, the storage location corresponding to the storage location information is the target code repository provided by the target code platform, and the updated code is incremental update code. Accordingly, the compute node can use the pre-built code pull tool and update version information to incrementally pull the incremental update code from the target code repository.

[0080] The code pull tool records the first version of the inference framework code that the compute node recently pulled. The compute node can input the updated version information into the code pull tool. The code pull tool then automatically pulls the incremental update code from the target code repository based on the first and updated version information. In other words, it pulls the difference between the first and updated versions. For example, the code pull tool can be Go-git, and the target code repository can be the GitLab repository.

[0081] Since method 1 is suitable for services that are updated frequently and the update cycle of the inference framework code is short, this method can update the inference framework code more efficiently.

[0082] Method 2: When the preset update method is the full update method, the storage location corresponding to the storage location information is the target storage node, and the update code is the full update code corresponding to the update version information. Accordingly, the computing node can obtain the full update code by following the steps below:

[0083] Step 1: Obtain a second storage path corresponding to the target storage node.

[0084] Step 2: Generate a code update instruction according to the preset instruction template, the second storage path, and the update version information.

[0085] The preset instruction template may be an "scp instruction", and the "scp instruction" is a Linux instruction.

[0086] Step 3: Send the code update instruction to the target storage node to notify the target storage node to obtain the full update code corresponding to the updated version information.

[0087] Step 4: Receive the full update code corresponding to the updated version information sent by the target storage node.

[0088] Specifically, the target storage node can be a node in the container group orchestration platform, which is specifically used to store various versions of the reasoning framework code. Therefore, when updating, the computing node can first obtain the second storage path corresponding to the target storage node. Add the second storage path to the first specified position of the preset instruction template, and add the updated version information to the second specified position of the preset instruction target to generate a code update instruction. The computing node can send the generated code update instruction to the target storage node. In this way, when the target storage node receives the code update instruction, it can parse the updated version information therefrom, and based on the updated version information, determine the target reasoning framework code corresponding to the updated version information as the full update code among the various versions of the reasoning framework code stored in the target storage node. The target storage node can send the full update code to the computing node, and the computing node can receive the full update code, that is, obtain the update code of the reasoning framework.

[0089] In either way, by updating the version information, version control can be achieved, and any version can be rolled back or updated, rather than only being able to passively update to the latest version.

[0090] Step 2: Based on the updated code, perform an update operation on the original reasoning framework code in the first storage path.

[0091] Method 1: When the update code is an incremental update code, the computing node may directly add the incremental update code to the original reasoning framework code under the first storage path to complete the update operation of the original reasoning framework code.

[0092] In the second method, when the updated code is a full update code, the computing node may replace the original reasoning framework code stored in the first storage path with the full update code.

[0093] Step S204 : When it is detected that the update operation is completed, the reasoning framework container is restarted based on the updated reasoning framework code to complete the update operation of the reasoning framework service.

[0094] Specifically, when the computing node detects that the update operation is completed, it can first close the original reasoning framework container, write and read the updated reasoning framework code from the first storage path, and create and start a new reasoning framework container based on the updated reasoning framework code to complete the update operation of the reasoning framework service.

[0095] In some optional implementations, after step S204, the computing node may use the restarted inference framework container to manage the inference engine container. Accordingly, the computing node may further perform the following steps:

[0096] Step 1: Obtain the port information corresponding to at least one pre-built inference engine container.

[0097] Step 2: Based on the port information, re-take over the inference engine container corresponding to the port information.

[0098] Specifically, the inference engine container periodically sends registration requests to the inference framework container in its inference business container group. After the inference framework container is restarted, the registration request can be used to obtain the port information corresponding to at least one inference engine container in real time. This information can then be used to continue monitoring the inference engine container. This allows for timely monitoring of the inference engine container and timely resolution of any inference engine container failures.

[0099] In the service update method of an embodiment of the present application, upon receiving a service update request from a client, a computing node can first determine a first storage path for the original reasoning framework code based on the service indication information and location indication information included in the service update request. Then, based on the first storage path, the computing node can retrieve the original version information of the original reasoning framework code. When it is determined to perform an update operation based on the updated version information and the original version information, the computing node performs an update operation on the original reasoning framework code stored in the first storage path according to a preset pull method. When the update operation is detected to be complete, the computing node retrieves the updated reasoning framework code from the storage path. Based on the updated reasoning framework code, the computing node restarts the reasoning framework container. This allows only the original reasoning framework code to be updated, and after the update, only the reasoning framework container needs to be restarted. Even without an additional accelerator card, the updated reasoning framework code can be applied promptly, thus completing the update of the reasoning framework service in a timely manner. Furthermore, in related art, restarting the container group is required. However, since the container group includes an inference engine, the startup time of the inference engine is relatively long, and therefore restarting the container group is slow. However, this solution only requires restarting the inference framework container, which can improve the update speed of the entire service and apply the new reasoning framework code more promptly.

[0100] Before executing the above service update method, the computing node may first deploy the service. Accordingly, the embodiment of the present application provides a service deployment method that can be executed by any of the above computing nodes. Figure 3 As shown, the specific processing steps of the service update method may include:

[0101] Step S301: receiving a service deployment request sent by a client.

[0102] The service deployment request includes annotation information.

[0103] Specifically, the user can create a configuration file on the client. For example, the configuration file can be in YAML format and named "service.yaml." Accordingly, the user can enter a first command and the name of the configuration file on the client (for example, "kubectl apply –f your-service.yaml"). The client can then generate a service deployment request based on the first command and the name of the configuration file and send it to the client. For example, the first command can be a "kubectl command." Alternatively, the user can click a target button on the relevant service interface on the client. The client can then automatically invoke a first interface (for example, the Kubernetes API server) to generate a service deployment request and send it to the compute node. After receiving the service deployment request from the client, the compute node can parse the request, obtain annotation information, and then perform different service deployment operations based on the annotation information.

[0104] Step S302: Determine whether to perform a deployment operation of the reasoning framework service based on the annotation information.

[0105] Specifically, the computing node may determine whether to perform a deployment operation of the reasoning framework service based on the content included in the annotation information. Accordingly, the computing node may perform the following specific steps:

[0106] Step 1: Determine whether the annotation information includes a preset field.

[0107] Among them, the preset field can be the "LLM-service field".

[0108] Step 2: When it is determined whether the annotation information includes the preset field, it is determined to perform the reasoning framework service deployment operation.

[0109] or,

[0110] Step three: when it is determined that the annotation information does not include the preset field, it is determined not to perform the inference framework service deployment operation.

[0111] Specifically, the computing node can first determine whether the annotation information includes the "LLM-service field". If so, it is determined that the LLM reasoning service is currently being performed and the reasoning framework service needs to be deployed first. If not, it is determined that other services are currently being performed and can be processed according to the deployment method of other services without the need for special processing of this solution.

[0112] Step S303: When it is determined to perform a deployment operation of the reasoning framework service, the latest version of the reasoning framework code is obtained from a preset storage location according to a preset update method.

[0113] Method 1: Use a pre-built code pull tool to pull the latest version of the inference framework code from the target code repository.

[0114] Method 2: Obtain a second storage path corresponding to the preset storage location. Generate a code read instruction based on the preset instruction template and the second storage path. Send the code read instruction to the target storage node to instruct the target storage node to search for and send the latest version of the inference framework code. Receive the latest version of the inference framework code from the target storage node.

[0115] Method 3: Extract the latest version of the inference framework code from the pre-built initialized full pull container.

[0116] Specifically, in method 1, the computing node only needs to input the service deployment instruction into the code pulling tool, which will trigger the code pulling tool to pull the latest version of the inference framework code from the target code repository. This is more convenient.

[0117] In method 2, the computing node can add the second storage path to the first designated position of the preset instruction template, generate a code acquisition instruction, and send it to the target storage node. In this way, when the target storage node receives the code acquisition instruction, it can determine the latest version of the inference framework code from the various versions of the inference framework code stored in itself and send it to the computing node. The computing node can then obtain the latest version of the inference framework code sent by the target storage node. In this way, obtaining code from the local target storage node is more stable and has a fast deployment speed, and multiple computing nodes can obtain code from the same storage node at the same time, which is suitable for large-scale distributed training and inference scenarios.

[0118] In method three, initializing a full pull container pre-configures the latest version of the inference framework code. Accordingly, when deploying the LLM inference service, the latest version of the inference framework code can be extracted from the code location corresponding to the preset location. Because the code is pre-packaged in the container, it can be used by simply extracting it from the preset location, with virtually no network latency, making deployment of the inference framework service highly efficient.

[0119] Step S304: store the latest version of the inference framework code in the first storage path.

[0120] Among them, the latest version of the reasoning framework code is the original reasoning framework code.

[0121] Specifically, the computing node may directly store the latest version of the reasoning framework code in the first storage path.

[0122] Step S305 : creating and starting an inference framework container based on the latest version of the inference framework code read from the first storage path.

[0123] Specifically, the computing node may read the latest version of the reasoning framework code from the first storage path, create and start a reasoning framework container, and the reasoning framework container started here may be the original reasoning framework container mentioned in step S204 above.

[0124] The service deployment method of the embodiment of the present application controls the deployment logic through simple annotations without the need to modify the deployment script or manual intervention, which is relatively simple. In addition, the latest version of the inference framework code is obtained from the preset storage location to ensure that the deployment always uses the latest and verified code version, avoiding functional defects or security issues caused by local caching of old version code. In addition, the latest version of the code is saved to the first storage path as the original inference framework code used at runtime, so that all inference business container groups mounted on the first storage path can use the same version of the original inference framework code, ensuring the consistency of the inference service deployment.

[0125] In order to facilitate users to view the service deployment information of the reasoning framework at any time, when executing the above service update method, the computing node can also record the relevant information of the reasoning framework to the database. Accordingly, the embodiment of the present application provides a data recording and storage method that can be executed by any of the above computing nodes. Figure 4 As shown, the specific processing steps of the data record storage method may include:

[0126] Step S401: Acquire storage resource configuration information corresponding to a first storage path.

[0127] The storage resource configuration information may be the aforementioned PVC.

[0128] Specifically, the computing node may determine, based on the first storage path, a PVC from the at least one pre-purchased PVC that matches the first storage path as the above-mentioned storage resource configuration information.

[0129] Step S402: Generate a data record corresponding to the inference framework code according to the storage resource configuration information, the service indication information, the first storage path, and the original version information.

[0130] Specifically, the computing node may combine the storage resource configuration information, the service indication information, the first storage path, and the original version information to obtain a data record corresponding to the inference framework code.

[0131] Step S403: storing the data record in a pre-built target database for querying the service deployment information of the reasoning framework.

[0132] Specifically, the computing node may store the data records in a pre-built target database, so that a user may subsequently view the data records in the target database.

[0133] The data record storage method of the embodiments of the present application generates and stores data records in a target database during the service deployment process, allowing users to easily understand the status of the currently deployed inference framework code. Furthermore, when users learn the original code version information through the data records, they can determine whether to perform an update operation, thereby avoiding the waste of computing node resources caused by entering updated version information consistent with the original version information without prior knowledge.

[0134] The following describes the above-mentioned service deployment method, data record storage method, and service update method in detail using a specific example.

[0135] like Figure 5 As shown, LLM reasoning business container group 1 and LLM reasoning business container group 2 can be deployed on the computing node. Both LLM reasoning business container groups can include an initialization full-scale pull container, an inference engine container, and an inference framework container. The inference framework code used by the inference framework container can be managed by the code management container in the code management container group, that is, the user can manage the inference framework code stored in the storage locations corresponding to the lower-level paths of the storage path mounted by the code management container group through the user interface (UI). The following takes any LLM reasoning business container group as an example.

[0136] In such Figure 5 Under the architecture shown, the above Figures 2 to 4 The various operations shown may be performed by a computing node using different program components (eg, the intermediate components mentioned below) and containers.

[0137] Steps S201, S301, and S302 can be performed by the compute node using an intermediary component (e.g., a webhook). Steps S202 and S203 are performed by the compute node using the code management container, and step S204 is performed by the compute node using the inference framework container. Steps S303 through S305 can be performed by the compute node using the initialization full pull container. Steps S401 through S403 can also be performed by the code management container.

[0138] In the service deployment phase, the process of the service deployment method can be as follows Figure 6 shown.

[0139] Users can create LLM inference services through "kubectl instructions" or "Kubernetes API". Accordingly, the client sends service deployment requests to the computing nodes based on the user's operations.

[0140] The intermediate component performs the above steps S301 and S302. When the intermediate component of the computing node obtains the service deployment request, it can determine whether the "annotations" (that is, the above-mentioned annotation information) included in the service deployment request includes the "LLM-service field". When the intermediate component determines that the "annotations" include the "LLM-service field", it determines that the LLM reasoning service needs to be deployed currently. Alternatively, when the intermediate component determines that the "annotations" do not include the "LLM-service field", it determines that other services need to be deployed currently (this solution does not interfere in this case). When it is determined that the LLM reasoning service needs to be deployed currently, the intermediate component can extract the service scenario name, service name, PVC name, etc. from the service deployment request and send it to the code management container. In addition, the intermediate component can send an inference service deployment request to the initialized full-scale pull container in the LLM reasoning business container group.

[0141] Initialize the full pull container and execute steps S303 to S305 above. When the initialized full pull container receives the inference service deployment request, it can pull the inference framework code and store the pulled inference framework code in the storage path where it is mounted (that is, the storage path where the inference business container group to which it belongs is mounted, for example, the first storage path mentioned above). Then, it reads the inference framework code from the storage path and creates and starts the inference framework container based on the inference framework code.

[0142] The code management container executes steps S401 to S403 described above. Based on the received service scenario name and PVC name, the code management container can determine the storage path corresponding to the inference framework code and, based on this storage path, determine the version information of the retrieved inference framework code. Furthermore, the code management container can also obtain the PVC corresponding to the LLM inference business container group. Finally, the code management container can generate data records based on the service scenario name, service name, PVC name, the retrieved inference framework code version information, and PVC, and store them in the database.

[0143] In the service update phase, the process of the service update method can be as follows Figure 7 shown.

[0144] The intermediate component executes step S201. After receiving the service update request sent by the client, the intermediate component may forward the service update request to the code management container.

[0145] The code management container executes step S202 and step S203. When the code management container receives the service update request forwarded by the intermediate component, it can update the reasoning framework code stored in the storage path mounted by the LLM reasoning business container group through a preset update method.

[0146] The inference framework container executes step S204. Unicorn WSGI (Web Server Gateway Interface) code can be set in the inference framework container. Unicorn WSGI code is a type of web server. The "--reload" parameter can be set in this code. When this parameter is enabled, Unicorn monitors whether the inference framework code has been updated. If an update is detected, Unicorn WSGI is reloaded, which in turn triggers a restart of the inference framework container.

[0147] After the inference framework container is restarted, it can receive registration requests periodically sent by the inference engine container. The inference framework container can extract port information from the registration request and then monitor relevant information of the inference engine container based on the port information. For example, it can obtain the name and version of the loaded model of the inference engine container, as well as statistical information and health check information of related performance indicators.

[0148] A proxy model can be used between the inference framework container and the inference engine container. This abstraction creates an inference engine service object within the inference framework service to implement the aforementioned interaction functionality. This abstracted inference engine service object can be a code class that includes methods corresponding to operations implemented by the inference engine code. This allows the inference framework container to operate the inference engine container as if it were its own internal code. This code class can be written using the Remote Procedure Call (gRPC) protocol or the Hypertext Transfer Protocol (HTTP).

[0149] When the code class is written using the gRPC protocol, the code class can be as follows:

[0150] syntax = "proto3";

[0151] package vllm_proxy;

[0152] service VLLMService {

[0153] / / Method to create an inference engine container

[0154] rpc CreateEngine(EngineArgs) returns (EngineResponse);

[0155] / / The method for generating text is a streaming RPC, that is, the computing node can return multiple responses (streaming output)

[0156] rpc Generate(GenerateRequest) returns (stream GenerateResponse);

[0157] / / Method for checking the health of the inference engine container

[0158] rpc CheckHealth(HealthCheckRequest) returns (HealthCheckResponse);

[0159] / / Method to close the inference engine container

[0160] rpc ShutdownEngine(ShutdownRequest) returns (ShutdownResponse);

[0161] }

[0162] / / Corresponding to the configuration parameters of AsyncEngineArgs, the parameters that need to be configured when creating the inference engine container

[0163] message EngineArgs {

[0164] string model = 1;

[0165] bool enable_lora = 2;

[0166] int32 max_loras = 3;

[0167] string tokenizer_mode = 4;

[0168] bool trust_remote_code = 5;

[0169] int32 tensor_parallel_size = 6;

[0170] int32 block_size = 7;

[0171] int32 swap_space = 8;

[0172] float gpu_memory_utilization = 9;

[0173] int32 max_num_seqs = 10;

[0174] string quantization = 11;

[0175] int32 max_model_len = 12;

[0176] map<string, int32> limit_mm_per_prompt = 13;

[0177] string guided_decoding_backend = 14;

[0178] string scheduling_policy = 15;

[0179] bool enable_prefix_caching = 16;

[0180] }

[0181] / / Response method after creating the inference engine container

[0182] message EngineResponse {

[0183] string engine_id = 1;

[0184] bool success = 2;

[0185] string error_message = 3;

[0186] }

[0187] / / Low-Rank Adaptation (LoRA) request, used to specify LoRA in the generated request

[0188] message LoRARequest {

[0189] string lora_name = 1;

[0190] int32 lora_int_id = 2;

[0191] string lora_local_path = 3;

[0192] }

[0193] / / Method for sampling parameters

[0194] message SamplingParams {

[0195] string lora_name = 1;

[0196] int32 n = 2;

[0197] int32 best_of = 3;

[0198] float presence_penalty = 4;

[0199] float frequency_penalty = 5;

[0200] float temperature = 6;

[0201] float top_p = 7;

[0202] int32 top_k = 8;

[0203] int32 max_tokens = 9;

[0204] repeated string stop = 10;

[0205] repeated int32 stop_token_ids = 11;

[0206] bool skip_special_tokens = 12;

[0207] / / Guide decoding to generate specific parameters

[0208] GuidedDecodingParams guided_decoding = 13;

[0209] }

[0210] message GuidedDecodingParams {

[0211] string json = 1;

[0212] string regex = 2;

[0213] repeated string choice = 3;

[0214] string grammar = 4;

[0215] bool json_object = 5;

[0216] string backend = 6;

[0217] string whitespace_pattern = 7;

[0218] }

[0219] / / Multimodal data

[0220] message MultiModalData {

[0221] repeated string image = 1;

[0222] }

[0223] / / Generation request

[0224] message GenerateRequest {

[0225] string engine_id = 1;

[0226] oneof input {

[0227] string text_prompt = 2;

[0228] ComplexPrompt complex_prompt = 3;

[0229] }

[0230] SamplingParams sampling_params = 4;

[0231] string request_id = 5;

[0232] LoRARequest lora_request = 6;

[0233] }

[0234] message ComplexPrompt {

[0235] string prompt = 1;

[0236] MultiModalData multi_modal_data = 2;

[0237] }

[0238] / / Generated token output

[0239] message OutputToken {

[0240] int32 index = 1;

[0241] string text = 2;

[0242] repeated float logprobs = 3;

[0243] string finish_reason = 4;

[0244] repeated int32 token_ids = 5;

[0245] }

[0246] / / Generate response

[0247] message GenerateResponse {

[0248] string request_id = 1;

[0249] repeated OutputToken outputs = 2;

[0250] repeated int32 prompt_token_ids = 3;

[0251] bool is_final = 4;

[0252] string error_message = 5;

[0253] }

[0254] / / Health check request

[0255] message HealthCheckRequest {

[0256] string engine_id = 1;

[0257] }

[0258] message HealthCheckResponse {

[0259] bool is_healthy = 1;

[0260] string status_message = 2;

[0261] }

[0262] / / Request to close the inference engine container

[0263] message ShutdownRequest {

[0264] string engine_id = 1;

[0265] }

[0266] message ShutdownResponse {

[0267] bool success = 1;

[0268] }

[0269] Compute nodes can set up a monitoring component (for example, a liveness probe), set the monitoring cycle of the monitoring component to a value greater than 15 seconds, and set the failure threshold of the monitoring component to a value greater than 2. In this way, the monitoring component can monitor the normal operation of the LLM reasoning service container group according to the monitoring cycle length. If the LLM reasoning service container group is detected to be operating abnormally for multiple consecutive cycles (greater than 2), the LLM reasoning service container group will be restarted. Because it takes a certain amount of time to restart the inference framework container, during this period, the LLM reasoning service container group may be operating abnormally. If the LLM reasoning service container group is restarted directly, the restart time will be longer, introducing unnecessary time loss. At the same time, in order to complete the necessary health checks, periodic checks can be performed according to the monitoring cycle length set above.

[0270] Multiple inference engine containers can be deployed in an inference service container group. Therefore, to prevent conflicts, a random port can be assigned to each inference engine container. Furthermore, to facilitate management, the inference framework container can specify a port using an environment variable. The port in this environment variable can be consistent with the target port parameter during service deployment. This allows each inference engine container to determine the port of the inference framework container using the target port parameter and, based on this port, send a registration request to the inference framework container. The inference framework container can then extract the port information from the registration request and, based on this port information, retake control of each inference engine container in its inference service container group.

[0271] In related technologies, in order to facilitate the control of resources occupied by the inference engine, the inference framework is generally used as the parent process and the inference engine as the child process. Therefore, when updating the inference framework code, it is necessary to end both the process where the inference framework is located and the process where the inference engine is located. However, restarting the inference engine takes a long time, resulting in a long interruption of the inference service. By isolating the inference framework and the inference engine, that is, dividing the inference framework container and the inference engine container into two completely independent containers, and using the proxy mode to allow the inference framework container to operate the inference engine container, when updating, there is no need to restart or shut down the inference engine container, the restart speed is faster, and the new inference framework code can be applied in time for inference service. The code management container is specifically used to perform related functions of code management, which can realize centralized management of the code of all business container groups, facilitate user management, and realize flexible control of code versions.

[0272] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0273] The embodiment of the present application also provides a service update device, such as Figure 8 Shown, including:

[0274] The receiving module 810 is configured to receive a service update request sent by a client, wherein the service update request includes service indication information, location indication information, and update version information;

[0275] A determination module 820 is configured to determine original version information of the original reasoning framework code based on the service indication information and the location indication information;

[0276] An update module 830 is configured to, when determining to perform an update operation based on the updated version information and the original version information, perform an update operation on the original reasoning framework code according to a preset update method and the updated version information;

[0277] The restart module 840 is configured to restart the reasoning framework container based on the updated reasoning framework code when it is detected that the update operation is completed, so as to complete the update operation of the reasoning framework service.

[0278] In some optional implementations, the determination module 820 is specifically configured to:

[0279] Determining a first storage path of the original reasoning framework code according to the service indication information and the location indication information;

[0280] Based on the first storage path, original version information is read.

[0281] In some optional implementations, the determination module 820 is specifically configured to:

[0282] Obtain a second storage path on which the pre-built code management container is mounted, wherein a lower-layer path of the second storage path includes at least one sub-path;

[0283] According to the service indication information and the location indication information, a sub-path that matches both the service indication information and the location indication information is determined in at least one sub-path as a first storage path.

[0284] In some optional implementations, the determination module 820 is specifically configured to:

[0285] Based on the first storage path, reading the original reasoning framework code from a storage location corresponding to the first storage path;

[0286] According to the preset field identification information, a target field corresponding to the preset field identification information is determined in the original reasoning framework code;

[0287] Extracts the original version information from the target field.

[0288] In some optional implementations, the update module 830 is specifically configured to:

[0289] According to the preset update method and update version information, the update code of the inference framework is obtained from the preset storage location;

[0290] Based on the updated code, an update operation is performed on the original reasoning framework code in the first storage path.

[0291] In some optional implementations, when the preset update mode is an incremental update mode, the preset storage location is a target code repository provided by the target code platform, and the update code is an incremental update code; the update module 830 is specifically configured to:

[0292] Use pre-built code pulling tools and update version information to incrementally pull incremental update codes from the target code repository.

[0293] In some optional implementations, when the preset update mode is a full update mode, the preset storage location is the target storage node, and the update code is the full update code corresponding to the updated version information; the update module 830 is specifically configured to:

[0294] Obtaining a second storage path corresponding to the target storage node;

[0295] Generate a code update instruction according to a preset instruction template, the second storage path, and the update version information;

[0296] Sending a code update instruction to the target storage node to notify the target storage node to obtain the full update code corresponding to the updated version information;

[0297] Receive the full update code corresponding to the updated version information sent by the target storage node.

[0298] In some optional implementations, the update module 830 is specifically configured to:

[0299] Replace the original inference framework code stored in the first storage path with the fully updated code.

[0300] In some optional embodiments, the apparatus further includes a deployment module 850 configured to:

[0301] receiving a service deployment request sent by a client, wherein the service deployment request includes annotation information;

[0302] Determine whether to deploy the inference framework service based on the annotation information;

[0303] When it is determined to deploy the inference framework service, the latest version of the inference framework code is obtained from the preset storage location according to the preset update method;

[0304] Storing the latest version of the reasoning framework code in the first storage path, wherein the latest version of the reasoning framework is the original reasoning framework code;

[0305] Based on the latest version of the inference framework code read from the first storage path, create and start the inference framework container.

[0306] In some optional implementations, the deployment module 850 is specifically configured to:

[0307] Determining whether the annotation information includes a preset field;

[0308] When determining whether the annotation information includes the preset field, determining to perform the reasoning framework service deployment operation;

[0309] or,

[0310] When it is determined that the annotation information does not include the preset field, it is determined not to perform the inference framework service deployment operation.

[0311] In some optional embodiments, the device further includes a storage module 860, configured to:

[0312] Acquire storage resource configuration information corresponding to the first storage path;

[0313] Generate a data record corresponding to the reasoning framework code according to the storage resource configuration information, the service indication information, the first storage path, and the original version information;

[0314] The data records are stored in a pre-built target database to query the service deployment information of the inference framework.

[0315] In some optional embodiments, the device further includes a monitoring module 870 for:

[0316] Obtain port information corresponding to at least one pre-built inference engine container;

[0317] Based on the port information, the inference engine container corresponding to the port information is taken over again.

[0318] For the description of the features in the embodiment corresponding to the service updating device, reference can be made to the relevant description of the embodiment corresponding to the service updating method, which will not be repeated here.

[0319] The embodiment of the present application also provides an electronic device, such as Figure 9 As shown, it includes a memory 10 and a processor 20. The memory 10 stores a computer program, and the processor 20 is configured to run the computer program to execute the steps in any of the above service update method embodiments. The electronic device may be the above computing node.

[0320] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned service updating method embodiments when running.

[0321] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0322] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above service update method embodiments are implemented.

[0323] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned service update method embodiments are implemented.

[0324] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0325] The above is a detailed introduction to a service update method, device, electronic device, storage medium and program product provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core ideas of the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A service update method, characterized in that: include: Receiving a service update request sent by a client, wherein the service update request includes service indication information, location indication information, and update version information; Determining original version information of the original reasoning framework code according to the service indication information and the location indication information, wherein the service indication information includes one or more of scenario identification information, service identification information, etc.; When it is determined to perform an update operation based on the updated version information and the original version information, performing an update operation on the original reasoning framework code according to a preset update method and the updated version information; When it is detected that the update operation is completed, the reasoning framework container is restarted based on the updated reasoning framework code to complete the update operation of the reasoning framework service.

2. The service update method according to claim 1, characterized in that: The determining, according to the service indication information and the location indication information, original version information of the original reasoning framework code includes: Determining a first storage path of the original reasoning framework code according to the service indication information and the location indication information; Based on the first storage path, the original version information is read.

3. The service update method according to claim 2, characterized in that: The determining, according to the service indication information and the location indication information, a first storage path of the original reasoning framework code includes: Obtain a second storage path where the pre-built code management container is mounted, wherein a lower-layer path of the second storage path includes at least one sub-path; According to the service indication information and the location indication information, a sub-path that matches both the service indication information and the location indication information is determined in at least one of the sub-paths as the first storage path.

4. The service update method according to claim 2, wherein: The reading the original version information based on the first storage path includes: Based on the first storage path, reading the original reasoning framework code from a storage location corresponding to the first storage path; According to the preset field identification information, determining a target field corresponding to the preset field identification information in the original reasoning framework code; The original version information is extracted from the target field.

5. The service update method according to any one of claims 2 to 4, characterized in that: When determining to perform an update operation based on the updated version information and the original version information, performing an update operation on the original reasoning framework code based on a preset update method and the updated version information includes: According to the preset update method and the update version information, obtaining the update code of the reasoning framework from a preset storage location; Based on the updated code, an update operation is performed on the original reasoning framework code in the first storage path.

6. The service update method according to claim 5, characterized in that: When the preset update mode is an incremental update mode, the preset storage location is a target code repository provided by a target code platform, and the update code is an incremental update code; The acquiring the update code of the reasoning framework from the preset storage location according to the preset update mode and the update version information includes: The incremental update code is incrementally pulled from the target code repository using a pre-built code pulling tool and the update version information.

7. The service update method according to claim 5, characterized in that: When the preset update mode is a full update mode, the preset storage location is a target storage node, and the update code is a full update code corresponding to the update version information; The acquiring the update code of the reasoning framework from the preset storage location according to the preset update mode and the update version information includes: Acquire a second storage path corresponding to the target storage node; Generate a code update instruction according to a preset instruction template, the second storage path, and the update version information; Sending the code update instruction to the target storage node to notify the target storage node to obtain the full update code corresponding to the update version information; Receive the full update code corresponding to the update version information sent by the target storage node.

8. The service update method according to claim 7, characterized in that: The updating operation on the original reasoning framework code in the first storage path based on the updated code includes: The original reasoning framework code stored in the first storage path is replaced with the full update code.

9. The service update method according to claim 5, characterized in that: Before receiving the service update request sent by the client, the method further includes: receiving a service deployment request sent by the client, wherein the service deployment request includes annotation information; Determining whether to perform a deployment operation of the reasoning framework service based on the annotation information; When it is determined to perform the deployment operation of the reasoning framework service, obtaining the latest version of the reasoning framework code from the preset storage location according to the preset update method; Storing the latest version of the reasoning framework code in the first storage path, wherein the latest version of the reasoning framework is the original reasoning framework code; Based on the latest version of the reasoning framework code read from the first storage path, the reasoning framework container is created and started.

10. The service update method according to claim 9, characterized in that: The determining, based on the annotation information, whether to perform a deployment operation of the reasoning framework service includes: Determining whether the annotation information includes a preset field; When determining whether the annotation information includes a preset field, determining to perform the reasoning framework service deployment operation; or, When it is determined that the annotation information does not include a preset field, it is determined not to perform the inference framework service deployment operation.

11. The service update method according to claim 9, wherein: The method further comprises: Acquire storage resource configuration information corresponding to the first storage path; generating a data record corresponding to the inference framework code according to the storage resource configuration information, the service indication information, the first storage path, and the original version information; The data records are stored in a pre-built target database for querying service deployment information of the reasoning framework.

12. The service update method according to any one of claims 1 to 4, characterized in that: When the update operation is detected to be complete, restarting the reasoning framework container based on the updated reasoning framework code to complete the update operation of the reasoning framework service, the method further includes: Obtain port information corresponding to at least one pre-built inference engine container; Based on the port information, the inference engine container corresponding to the port information is taken over again.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the service updating method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the service updating method according to any one of claims 1 to 12 are implemented.

15. A computer program product, characterized in that The computer program product includes a computer program, wherein when the computer program is executed by a processor, the steps of the service update method according to any one of claims 1 to 12 are implemented.

Images