Cloud computing service method, device, electronic device and computer storage medium

Abstract

The embodiments of the present invention provide a cloud computing service method, device, electronic device and computer storage medium. The cloud computing service method includes: receiving a model service request initiated by a user; allocating an execution node for the model service request; processing the model service request on the execution node using a container management application instance, wherein the container management application instance includes at least the following containers: an initialization container and a model container; and returning the processing result to the user. In the scheme of the embodiments of the present invention, since the initialization container and the model container maintain the independence of the containers from each other, they can be updated or maintained independently, thereby improving the flexibility of computing platform updates.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computer technology, and in particular to a cloud computing service method, device, electronic device, and computer storage medium. Background Art

[0002] Due to the reasoning ability of artificial intelligence algorithm models, more and more users hope to use the ability of algorithm models to improve their service value. Usually, a complete machine learning system involves multiple links, including data capture, data cleaning, model training, model tuning, model service, etc. Therefore, by providing a platform for algorithm models through network technology, deploying application services and algorithm model services on a server as a computing platform to provide services to the outside world, the deployment cost can be greatly reduced. However, when updating the computing platform, this solution requires the application code and algorithm code to be released at the same time. Summary of the invention

[0003] In view of this, embodiments of the present invention provide a cloud computing service method, device, electronic device, and computer storage medium to improve the flexibility of computing platform updates.

[0004] According to a first aspect of an embodiment of the present invention, a cloud computing service method is provided, comprising: receiving a model service request initiated by a user; allocating an execution node to the model service request; processing the model service request on the execution node using a container management application instance, the container management application instance comprising at least the following containers: an initialization container and a model container; and returning the processing result to the user.

[0005] According to a second aspect of an embodiment of the present invention, a cloud computing service method is provided, comprising: receiving a model service request sent to a container management application instance, the container management application instance comprising a service container and a model container, the service container being used to provide application services, and the model container being used to run a prediction model; responding to the model service request, calling the prediction model in the model container through the service container to perform prediction; and returning a prediction result of the prediction model.

[0006] According to a third aspect of an embodiment of the present invention, a cloud computing service method is provided, comprising: receiving a prediction model deployment request sent to the container management application instance, the container management application instance includes a service container, and the service container is used to provide application services; in response to the prediction model deployment request, creating a second container and an initialization container in the container management application instance, the model container is used to run the prediction model, and the initialization container is used to obtain the prediction model; obtaining the prediction model into the model container through the initialization container, so as to call the prediction model in the model container through the service container for prediction.

[0007] According to a fourth aspect of an embodiment of the present invention, a cloud computing service method is provided, comprising: receiving a neural network model service request sent to a container management application instance, the model service request including semantic data to be predicted, the container management application instance including a service container and a model container, the service container being used to provide application services, and the model container being used to run a neural network prediction model; responding to the model service request, calling the prediction model in the model container through the service container, performing semantic processing on the semantic data to be predicted, and obtaining a semantic processing result; and returning a prediction result including the semantic processing result.

[0008] According to a fifth aspect of an embodiment of the present invention, a cloud computing service device is provided, comprising: a request receiving module, receiving a model service request initiated by a user; a node allocation module, allocating an execution node to the model service request; a request processing module, processing the model service request on the execution node using a container management application instance, the container management application instance comprising at least the following containers: an initialization container and a model container; and a processing result returning module, returning the processing result to the user.

[0009] According to a sixth aspect of an embodiment of the present invention, a cloud computing service device is provided, comprising: a model service request receiving module, receiving a model service request sent to a container management application instance, wherein the container management application instance includes a service container and a model container, wherein the service container is used to provide application services, and the model container is used to run a prediction model; a container calling module, responding to the model service request, calling the prediction model in the model container through the service container to perform prediction; and a prediction result returning module, returning the prediction result of the prediction model. .

[0010] According to a seventh aspect of an embodiment of the present invention, a cloud computing service device is provided, including: a deployment request receiving module, receiving a prediction model deployment request sent to the container management application instance, the container management application instance includes a service container, and the service container is used to provide application services; a container creation module, the container responds to the prediction model deployment request, creates a model container and an initialization container in the container management application instance, the model container is used to run the prediction model, and the initialization container is used to obtain the prediction model; a model acquisition module, acquiring the prediction model into the model container through the initialization container, so as to call the prediction model in the model container through the service container for prediction.

[0011] According to an eighth aspect of an embodiment of the present invention, there is provided a cloud computing service device, comprising: a model service request receiving module, receiving a neural network model service request sent to a container management application instance, the model service request including semantic data to be predicted, the container management application instance including a service container and a model container, the service container being used to provide application services, and the model container being used to run a neural network prediction model; a semantic processing module, responding to the model service request, calling the prediction model in the model container through the service container, performing semantic processing on the semantic data to be predicted, and obtaining a semantic processing result; and a prediction result returning module, returning a prediction result including the semantic processing result.

[0012] According to a ninth aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; and a computer-readable medium configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the first to fourth aspects.

[0013] According to a tenth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the first to fourth aspects is implemented.

[0014] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0016] Figure 1A A schematic diagram of a network architecture applicable to a cloud computing service method according to an embodiment of the present invention;

[0017] Figure 1B A schematic flow chart of a cloud computing service method according to another embodiment of the present invention;

[0018] Figure 1C A schematic diagram of a deployment service phase of a cloud computing service method according to another embodiment of the present invention;

[0019] Figure 1D A schematic diagram of a prediction service phase of a cloud computing service method according to another embodiment of the present invention;

[0020] Figure 1E A schematic flow chart of a cloud computing service method according to another embodiment of the present invention;

[0021] Figure 2A A schematic flow chart of a cloud computing service method according to another embodiment of the present invention;

[0022] Figure 2B A schematic diagram of a prediction service phase of a cloud computing service method according to another embodiment of the present invention;

[0023] Figure 3 A schematic flow chart of a cloud computing service method according to another embodiment of the present invention;

[0024] Figure 4 A schematic flow chart of a cloud computing service method according to another embodiment of the present invention;

[0025] Figure 5 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention;

[0026] Figure 6 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention;

[0027] Figure 7 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention;

[0028] Figure 8 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention;

[0029] Fig. 9 is a schematic structural diagram of an electronic device according to another embodiment of the present invention;

[0030] Fig.10 The present invention is another embodiment of the hardware structure of an electronic device. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0032] Figure 1A The schematic diagram of the network architecture applicable to the cloud computing service method of one embodiment of the present invention is shown in the figure. As shown in the figure, in the network architecture, the cloud computing service method can be implemented by the client 30 and the cloud computing server 10 and the server 20. In this example, the number of servers and the number of clients shown in the figure are only exemplary, and it should be understood that the number of servers and clients can be arbitrary. In this example, the server 10 can be implemented as one or more data processing devices or other computing devices, and its system architecture includes server hardware 11, host system 12, cloud computing service container 131 and cloud computing service container 132. It should be understood that the host system 12 can call the hardware interface of the server hardware 11, such as the hardware virtual layer interface. The cloud computing service container 131 and the cloud computing service container 132 are installed in the host system 12, and the containers can access the storage resources of the server 10, and can also be configured with their own storage resources in the container. For example, either of the container 131 and the container 132 can access the storage resources of the server 10, and the other can access the storage resources configured by itself. Similarly, the server 20 can be implemented as one or more data processing devices or other computing devices. The host system 22 can call the hardware interface of the server hardware 21 and schedule or control the computing resources of the hardware 21. The cloud computing service container 23 is installed in the host system 22. The container 23 can access the storage resources of the server 20, and can also be configured with its own storage resources. The server hardware 11 and 12 include but are not limited to multi-processor CPU architecture, storage system, etc. The above-mentioned multi-processor CPU architecture includes processors capable of vector calculation, scalar calculation, and tensor calculation.

[0033] In addition, the host system 12 and the host system 22 may be the same operating system or different operating systems. The host system 12 and the host system 22 may be server operating systems such as LINUX. Although managed as a server function, the above operating system may also be a desktop operating system or an embedded operating system. In addition, the host system 12 and the host system 22 may be different operating systems.

[0034] It should also be understood that each of the server 10 and the server 20 can be implemented as at least one physically separated multiple servers. It can also be implemented as multiple virtual servers installed in the same physical server. For example, the server 10 and the server 20 can be installed in the same physical server as different virtual servers. In addition, the server 10 and the server 20 can be in different physical locations and can have different capabilities and computing architectures.

[0035] It should also be understood that the client 30 may be a desktop computer, a mobile terminal device, etc. The operating system 32 may be a desktop operating system or an embedded operating system, etc. The client application may send a request to the server 10 or the server 20, and receive a result returned by the server based on the request.

[0036] In this example, the client 30 can communicate with the server 10 and the server 20 via the link 50 via the network 40. It should be understood that in other examples, the client 30 can also bypass the network 40 to communicate with the server 10 and the server 20. In addition, the server 10 and the server 20 can communicate through a network such as the network 40. For example, the server 10 and the server 20 can communicate through an internal network. The above-mentioned communication can be based on the same network, and can be heterogeneous network communication, for example, communicating with each other or with other devices through one or more gateways. In addition, the network 40 includes but is not limited to a wired network such as Ethernet or a wireless network such as WIFI (Wireless High Fidelity).

[0037] In this example, the cloud computing service containers 131 and 132 of the server 10 and the cloud computing service container 23 of the server 20 may include application services and algorithm models. For example, the algorithm model may be a prediction model such as natural language processing (NLP) and machine translation (MT), or a prediction model for computer vision services. The algorithm model may include a processing module for semantic processing or computer vision processing. The application service may be responsible for functions such as data download, authentication, current limiting, statistics, and intervention. In addition, the above-mentioned containers may also include engines for tensor acceleration or vector acceleration, and may also include maintenance tools and development tools such as project management, compilation and debugging, process orchestration, and log management.

[0038] In this example, the user can customize the model through the client application 33, and can directly call the API (application programming interface) of the general online service on the server 10 or 20 to obtain capabilities such as natural language processing and machine translation, in other words, to obtain prediction services based on the capabilities. In addition, the user can also provide data to train the prediction model, and then make the model a service to achieve further service customization.

[0039] The specific implementation of the embodiment of the present invention is further described below in conjunction with the accompanying drawings of the embodiment of the present invention.

[0040] Figure 1B Schematic flow chart of a cloud computing service method of another embodiment of the present invention. The cloud computing service method of the embodiment of the present invention can be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing service network, a resource object execution node can communicate with a client to provide a specific service requested by the client (e.g., a service provided based on a prediction model). In addition, the service end of the cloud computing service network may include an execution node and a management node (e.g., a resource object management node). The resource object management node can manage at least one resource object execution node. For example, a resource object execution node can obtain a service request (e.g., a model service request) from a client, and generate a sub-request to a resource object execution node based on the service emotion. The resource object execution node can respond to the sub-request and return a service result (e.g., a prediction result) or an intermediate result to the resource object management node. The resource object execution node can forward the service result to the client; or process based on the intermediate result to obtain a service result, and return it to the client. Figure 1B The methods include:

[0041] 110: receiving a model service request initiated by a user.

[0042] 120: Allocate execution nodes for model service requests.

[0043] 130: On the execution node, the model service request is processed using a container management application instance, where the container management application instance includes at least the following containers: an initialization container and a model container.

[0044] 140: Return the processing result to the user.

[0045] It should be understood that the cloud computing service method in the text can be implemented based on a cloud service platform, for example, based on a cloud native service platform. The overall architecture of the cloud service platform can be divided into two layers: application service and model service. The application service is responsible for user-related service logic, such as data management, project management, and user management. The cloud computing service method of an embodiment of the present invention can be applied to cloud native computing services. The computing service is responsible for resource scheduling, model training, model deployment, and model online services. The computing service is deployed as a whole in a resource object management cluster network, especially to provide resource scheduling, service discovery, load balancing, health checks, and the like. For example, a resource object management cluster network may include a resource object execution node and a resource object management node (for example, a cloud native object management host). The service container may be referred to as an application service container or a platform service container. The model container may be referred to as a prediction model container or a model service container.

[0046] The container management application instance in the text may be an object for creating containers, deleting containers, orchestrating containers, managing containers, and the like. The container management application instance may be implemented as a specific abstraction layer in an execution node. The container in the container management application instance may also manage at least one container runtime (runtime, as an instance of a container) through a container engine. The container engine may be used to run at least one container, for example, the container engine may run a container by creating a container runtime or deleting a container runtime. In addition, the container management application instance may be installed in the container engine, thereby achieving unified management of the container runtime and the container, and achieving compatibility with existing container technologies (e.g., containers or container clouds). Optionally, the container engine may be installed in the container management application instance, thereby achieving orchestration and management of the container while ensuring the independence of the container engine.

[0047] It should be understood that for receiving a model service request sent for a container management application instance, the model service request can be obtained through a local agent of the management node. The request can be sent to the container management application instance of the execution node through the management node. In addition, the execution node can also directly receive the model service request. In addition, the model service request includes a first network address, which can indicate the network address of the execution node or the network address of the container management application instance in the execution node. For example, if there are multiple container management application instances in the execution node, the multiple container management application instances all have the first network address. In addition, different container management application instances in the same execution node can have multiple different virtual ports. The virtual port and the above-mentioned first network address are combined to indicate the unique address of the container management application instance. In addition, different execution nodes (logical nodes) can have different network addresses. When a specific execution node is implemented by multiple physical servers, the multiple physical servers can have the same network address. The client can send a model service request based on the first network address. In other words, the client directly calls the service provided by the container management application instance in the execution node. For example, the client may also send a model service request to the management node (or to the management node via the application server in the server), and the management node may determine the address of the container-managed application instance in response to the model service request, for example, determine the first network address of the container-managed application instance. The management node may send a sub-request of the model service request to the execution node of the container-managed application instance based on the first network address, or forward the model service request. The execution node may determine the container-managed application instance based on the virtual port of the container-managed application instance, and forward the sub-request or model service request to it. Alternatively, the management node may also determine the virtual port of the container-managed application instance, and send a sub-request of the model service request to the container-managed application instance, or forward the model service request.

[0048] For receiving a model service request sent by a management node to a container management application instance having a first network address, the model service request may be received through a network interface of the container management application instance based on the first network address. The service container and the model container may be installed in the container management application instance. For example, the container management application instance may be installed in an operating system (e.g., a server operating system, etc.) of a resource object execution node, and the operating system may serve as a host system of the container management application instance.

[0049] It should be understood that the model service request may be a request for a model prediction service such as a neural network model prediction service. The model service request may include data as input in the prediction model, for example, the data may be such as voice data, audio data, semantic data, image data, text data, video data waiting for prediction data, so that the model container uses the corresponding prediction model to perform prediction processing, and returns text data such as the recognition result of the input audio data or video data, the text data of the second language corresponding to the text data of the input first language, and the text data corresponding to the input image data. For responding to the model service request, the service container and the model container can access each other by calling commands to call the prediction model in the model container. Alternatively, the service container and the model container can access each other through inter-container communication. In addition, the model service request may include data to be predicted. The model service request is a request sent or forwarded by a local agent of the management node. In addition, the prediction model can run on the image (IMAGE) configured by the model container. The image can be a read-only program, and one or more images can be pulled from the image warehouse in the creation of the container to configure the container.

[0050] The beneficial effect of the solution of the embodiment of the present invention is as follows: since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of computing platform updates.

[0051] In another implementation of the present invention, using a container to manage an application instance to process a model service request includes: initializing the container to copy the user's data to the local execution node; and the model container using the user's data to provide the user with a model service.

[0052] In one example, the initialization container acquires the prediction model to the storage space of the container-managed application instance. In one example, the initialization container acquires the prediction model to the model container. For example, the initialization container can share the storage space of the container-managed application instance with the model container, and the initialization container can acquire the prediction model from the storage space of the container-managed application instance. For example, any method can be used for the initialization container to acquire the prediction model from the storage space of the container-managed application instance, and the embodiment of the present invention does not limit this.

[0053] In another implementation of the present invention, the container management application instance further includes: a service container; and using the container management application instance to process the model service request further includes: the service container providing the user with corresponding performance services of the model service.

[0054] The service container (i.e., application service container) of the embodiment of the present invention is responsible for the application service logic of the cloud computing service framework. The service container has functions such as caching, authentication, intervention, and batch processing arrangement. The model container (prediction model container) is responsible for the online service of the model (i.e., algorithm model). It should also be understood that the user can define the required model development framework and corresponding language according to the needs. For example, the model development framework can be tensorflow, pytorch, etc. For example, the language developed in the development framework can be such as python, c++, java, etc.

[0055] In other words, in the solution of the embodiment of the present invention, since the prediction model and the application service can be configured in different containers, for the prediction model container, multiple containers supporting different algorithm environments can be configured; in addition, for the application service model, multiple containers can also be built for multiple versions of application service codes, thereby avoiding external dependency conflicts between different algorithm models and different application service codes.

[0056] In another implementation of the present invention, the model container uses the user's data to provide model services to the user, including: the model container defines a corresponding model framework and language according to the user's needs, and provides model services to the user through the model framework and language.

[0057] In another implementation of the present invention, the initialization container, the model container, and the service container share a network and a disk.

[0058] In another implementation of the present invention, allocating an execution node to a model service request includes: responding to the model service request and allocating a route of an execution node to the model service request based on load status of a plurality of candidate execution nodes.

[0059] In another implementation of the present invention, in response to a model service request, a route of an execution node is assigned to the model service request based on the load status of multiple candidate execution nodes, including: in response to the model service request, a load status query request is sent to multiple candidate execution nodes respectively; and a route of an execution node is assigned to the model service request by receiving current load status information sent by multiple candidate execution nodes.

[0060] Figure 1E A schematic flowchart of a cloud computing service method according to another embodiment of the present invention. Figure 1E The methods include:

[0061] 170: Receive a model service request sent to a container management application instance, where the container management application instance includes a service container and a model container, where the service container is used to provide application services and the model container is used to run a prediction model.

[0062] 180: Respond to the model service request and call the prediction model in the model container through the service container to perform prediction.

[0063] 190: Returns the prediction results of the prediction model.

[0064] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0065] It should be understood that, for returning the prediction result of the prediction model to the management node based on the first network address, the prediction result of the prediction model can be returned to the management node through the port of the first network address. For example, the prediction result of the prediction model is returned through the local agent of the management node. For example, the prediction result of the prediction model is returned through the local agent of the management node based on the port of the first network address. It should be understood that the prediction result is the output of the prediction model, corresponding to the input data provided in response to the model service request.

[0066] It should be understood that the services of the embodiments of the present invention include, but are not limited to, various cloud technology services such as natural language processing and / or machine translation. Resource object management networks include, but are not limited to, container management networks such as Kubernetes (a container cluster management system). Container management application instances may include, but are not limited to, resource objects such as POD (in Kubernetes, POD is a container deployment unit, i.e., a container deployment object).

[0067] It should also be understood that the network address in the embodiment of the present invention can be a network address of any network. For example, a network address based on the Internet, or a network address of a mobile network, etc. The embodiment of the present invention is not limited to this. The container in the embodiment of the present invention may also be any virtualized object. For example, the container may be a portable container such as DOCKER (a container engine). The container in the embodiment of the present invention may be a container in container cloud technology. The container management application instance in the embodiment of the present invention may be in the upper layer of the host system. In one example, the container management application instance in the embodiment of the present invention may be in the upper layer of the container runtime. In another example, the container management application instance may be in the lower layer of the container runtime. The embodiment of the present invention is not limited to this. Preferably, the container management application instance is located in the upper layer of the container runtime to achieve tighter management and scheduling of the container.

[0068] It should also be understood that in the application scenario of the embodiment of the present invention, when a user initiates a request to the resource object management cluster network (e.g., a self-learning platform) through a client, the resource object management node will find the current configuration of the container management application instance in each execution node and initiate the correct route. For example, the request may include the prediction model to be requested and the data to be predicted, and the resource object management node requests the prediction service from the prediction model based on the address of the container running the prediction model. In addition, since there may be multiple requests initiated by multiple clients requesting the same or different prediction services, and the number and address of the container will change due to the object management node creating and deleting the container, before the request reaches the container where the prediction model is located, it can be processed by the load balancing of the application service container and the prediction model container in the container management application instance. Then, when the request arrives at the application service container, the application service container can call the container running the prediction model (i.e., the request arrives at the prediction model service). Then, the prediction model service processes the request (e.g., performs natural language processing, translation processing, or computer vision processing, etc.), and returns the result (e.g., recognition result or translation result) to the application service. For example, the model service request is a command of the scheduling center of the management node (for example, the scheduler of the kubernetes host). For example, the scheduling center configures two containers into the same container management application instance (for example, POD) by creating container instructions.

[0069] In addition, in the solution of the embodiment of the present invention, since the prediction model container and the application service model are independent, whether the platform operation and maintenance personnel want to update the platform application code in the application service model, or the algorithm personnel want to update the algorithm code in the prediction model, they can be published independently, that is, they do not need to be coupled together for simultaneous publication.

[0070] As an example, receiving a model service request sent for a container management application instance includes: receiving a model service request sent by a management node for a container management application instance of a first network address, and returning a prediction result of a prediction model, including: returning the prediction result of the prediction model to the management node based on the first network address.

[0071] In this example, the service container and the model container use a network stack based on the same execution node, so that when calling the prediction model, there is no additional delay compared to the calling process in the same container. That is, the above prediction model and application service are decoupled under the premise of ensuring low latency.

[0072] In another implementation of the present invention, after the initialization container copies the user's data to the local execution node, the method further includes: destroying the initialization container.

[0073] As an example, the service container has a first virtual port based on a first network address, and the method further includes: responding to a service container update instruction sent by the management node, creating an updated service container in the container management application instance, and allocating a second virtual port based on the first network address to the updated service container, the second virtual port being different from the first virtual port; and deleting the service container from the container management application instance.

[0074] As an example, the model container has a third virtual port based on the first network address, and the method also includes: responding to the model container update instruction sent by the management node, creating an updated model container in the container management application instance, and allocating a fourth virtual port based on the first network address to the updated model container, the fourth virtual port being different from the third virtual port; and deleting the model container from the container management application instance.

[0075] As an example, before creating an initialization container in a container management application instance in response to a model deployment request, the method further includes: responding to a container management application instance creation instruction sent by a management node, creating a container management application instance, and allocating a first network address.

[0076] It should be understood that the cloud computing service method of the embodiment of the present invention can be applicable to cloud service types such as public cloud, proprietary cloud or private cloud. The above-mentioned stage of using the prediction model to make predictions can be applicable to various types of cloud services. The cloud computing service method of this example may also include a model deployment stage and an application service deployment stage. In the model deployment stage, the method may also include: receiving a model deployment request sent by the management node for the container management application instance; in response to the model deployment request, creating a model container and / or initializing a container in the container management application instance. In the application service deployment stage, the method may also include: receiving an application service deployment request sent by the management node for the container management application instance; in response to the application service deployment request, creating a service container in the container management application instance.

[0077] The cloud computing service method of the embodiment of the present invention may also include a model training phase. In the model training phase, the user may provide training samples in a specific field, and use the initial model provided by the model service framework provided by the platform for training, thereby obtaining a prediction model in the field. In one example, in the model training phase, a model training container may be created using a container management application instance, and the trained prediction model may be stored in one or more storage media.

[0078] In another implementation of the present invention, before receiving a model service request sent to a container management application instance, the method further includes: responding to a prediction model deployment request sent to the container management application instance, creating an initialization container in the container management application instance for obtaining a prediction model; and obtaining the prediction model into the model container through the initialization container.

[0079] As an example, before receiving a model service request sent by a management node for a container management application instance having a first network address, the method further includes: receiving a model deployment request sent by a management node for a container management application instance having a first network address; creating an initialization container in the container management application instance in response to the model deployment request; and obtaining a prediction model into the model container through the initialization container. For example, the initialization container, the service container, and the model container can share a network (network address) and storage space (e.g., a disk). Since the initialization container also has the network address of the execution node where the container management application instance is located in the container management application instance, it is possible to ensure that the three containers communicate or schedule with each other based on the same network address, and that the three containers are managed by the container management application instance.

[0080] Figure 1C FIG. 1 is a schematic diagram of a deployment service phase of a cloud computing service method according to another embodiment of the present invention. Figure 1C As shown, the initialization container obtains the pre-trained prediction model from at least one of the first storage medium, the second storage medium and the third storage medium. For example, the first storage medium, the second storage medium and the third storage medium can be different storage service platforms or different storage modes. For example, the storage medium includes but is not limited to storage media provided by storage services such as network attached storage (NAS), object storage service (OSS), CEPH (a distributed storage system that supports block storage, file storage and object storage), and MONGO database (a distributed non-storage-based database).

[0081] For a proprietary cloud, one or more containers may be used for model training. Multiple prediction models may be trained using one container, or multiple prediction models may be trained in multiple containers. In another example, the model may be trained in other ways, and the trained model may be stored in one or more of the above-mentioned storage media so as to obtain the trained model from the one or more storage media. The use of different storage media such as OSS, MONGO, CEPH or local disks can improve the flexibility of storage, making it easy to provide a model training service based on a proprietary cloud for different service objects. In addition, the initialization container of an embodiment of the present invention can obtain models from different storage media or storage platforms in order to make predictions based on the prediction model, thereby achieving decoupling between the storage medium or storage platform and the prediction model update. Therefore, for a proprietary cloud, the above configuration can provide a large number of proprietary cloud users with flexible updates and optimal resource configuration.

[0082] In addition, for private cloud services, model deployment, model training, and application services are usually configured in the same platform, which may include a hardware platform and a software platform. In this case, the solution configuration storage medium (or storage platform), prediction model service, or application service of the embodiment of the present invention may be arranged in the private cloud platform. By implementing the above-mentioned various services on a separate platform, security can be improved, and targeted optimization can be performed based on the characteristics of the prediction model. For example, based on the statistical results of the number, amount of calculation, and computing power of various prediction model services, the hardware platform, software platform, the number of containers, the number of containers in each execution node, the type of storage medium, the number of storage media, and the like are configured. The prediction model service and the application service are configured in different containers, respectively, so that the application service or the prediction model service can be updated or upgraded respectively. In addition, obtaining the model from different types of storage media or storage platforms to the container running the prediction model enables the underlying medium to be shielded, and the abstraction of the storage medium is realized, thereby facilitating the secure and flexible management of the proprietary cloud platform.

[0083] Therefore, as described above, compared with the solution of downloading data and models from the storage medium to the local, whether in the public cloud, proprietary cloud or private cloud scenario, the solution of the embodiment of the present invention eliminates the need to use different storage medium clients to adapt to different storage media, thereby achieving decoupling of the storage medium.

[0084] In another implementation of the present invention, the initialization container and the model container share the storage space of the container management application instance, wherein obtaining the prediction model into the model container through the initialization container includes: obtaining the prediction model from the storage medium into the storage space through the initialization container, so that the model container obtains the prediction model from the storage space.

[0085] In another implementation of the present invention, a container management application instance has a first network address, and a storage management object with a second network address is used to manage a storage medium, wherein obtaining a prediction model from the storage medium to a storage space by initializing a container includes: initiating an access request for the storage medium to the second network address based on the first network address by initializing the container, so that the storage management object responds to the access request and returns the prediction model to the storage space. The storage management object is configured with a unified storage medium access interface for multiple storage media. For example, the above-mentioned storage management object includes but is not limited to a resource object for storage including a persistent volume (PV, persistent-volumes). For access requests to storage media, a Persistent Volume Claim can be used to bind to a PV, and then access various storage media managed by the PV. Since the storage management object is configured with a unified storage medium access interface for multiple storage media, it is possible to conveniently obtain the model stored in the storage medium at different storage media. In other words, the storage management object is used to shield the underlying storage medium.

[0086] In addition, in the solution of the embodiment of the present invention, while decoupling the model drawing and the platform service, it is possible to ensure that the two can be scheduled as a unit and share physical resources.

[0087] It should be understood that the storage management object can be in the same management node as the container management application instance. The second network address is different from the first network address. The initialization container (or initialization container) realizes the acquisition of the prediction model with the storage medium in the storage management object through network communication and calling. At the same time, the independence between different storage services or different storage media is guaranteed.

[0088] In another implementation of the present invention, after obtaining the prediction model from the storage medium to the storage space through the initialization container, the process includes: deleting the initialization container from the container management application instance. For example, Figure 1D FIG. 1 is a schematic diagram of a prediction service stage of a cloud computing service method according to another embodiment of the present invention. Figure 1DAs shown, after the initialization container in the container management application instance obtains the prediction model from outside the container management application instance, the initialization container can be deleted, thereby increasing the storage space of the container management application instance. It should be understood that after the initialization container obtains the prediction model from outside the container management application instance, it can mark the completion of the model deployment phase. After the model deployment phase, the model prediction phase of using the model for prediction can be entered. Thereafter, on the one hand, in response to the user's request, instructions can be sent to the execution node through the management node, so that the application service container calls the prediction model in the prediction model container. On the other hand, in the model prediction phase, the maintenance personnel or model algorithm personnel can update the application service code and algorithm code at any time.

[0089] In another implementation of the present invention, the service container has a first virtual port based on a first network address, and the method further includes: responding to a service container update instruction sent by the management node, creating an updated service container in the container management application instance, and allocating a second virtual port based on the first network address to the updated service container, the second virtual port being different from the first virtual port; deleting the service container from the container management application instance. In other words, the initialization container is first executed, mounted to the storage medium through the storage management object, and is responsible for copying the data on the storage medium to the local. After the data is copied, the initialization container no longer occupies resources.

[0090] It should be understood that by using different virtual ports based on the same network address, dynamic allocation of ports is achieved. After the communication task is completed or the service container is updated, the port can be deregistered or deleted, thereby saving communication resources while ensuring computing efficiency. In addition, the network address corresponds to a container management application instance, and the container management application instance can be created and deleted. The container in the container management application instance can be created and deleted. As an example, in response to the model deployment request, before creating an initialization container in the container management application instance, the method may also include: responding to the container management application instance creation instruction sent by the resource object management node, creating a container management application instance, and assigning a first network address. For example, when the container management application instance is updated, the first network address can be deleted so that the first network address is available again or assigned to other container management application instances. Thus, the creation and deletion of network addresses and virtual ports better matches the container management application instance and the virtualization of the container, so that the entire platform architecture does not rely on the physical environment to manage, schedule and allocate resources to a certain extent.

[0091] As an example, the model container has a third virtual port based on the first network address, and the method also includes: responding to the model container update instruction sent by the management node, creating an updated model container in the container management application instance, and allocating a fourth virtual port based on the first network address to the updated model container, the fourth virtual port being different from the third virtual port; and deleting the model container from the container management application instance.

[0092] It should be understood that by using different virtual ports based on the same network address, dynamic allocation of ports is achieved. After the communication task is completed or the model container is updated, the port can be deregistered or deleted, thereby saving communication resources while ensuring computing efficiency. In addition, the network address corresponds to a container management application instance, and the container management application instance can be created and deleted. Containers in the container management application instance can be created and deleted. Thus, the creation and deletion of network addresses and virtual ports better matches the container management application instance and the virtualization of the container, so that the entire platform architecture does not rely on the physical environment to manage, schedule and allocate resources to a certain extent.

[0093] In another implementation of the present invention, the method further includes: creating a service container and a model container in the container management application instance in response to a container creation instruction sent by the resource object management host to the first network address.

[0094] Figure 2A A schematic block diagram of a cloud computing service method according to another embodiment of the present invention; Figure 2A The cloud computing service method can be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing service network, a resource object execution node can communicate with a client to provide the client with a specific service requested (e.g., a service provided based on a prediction model). In addition, the service end of the cloud computing service network may include an execution node and a management node (e.g., a resource object management node). The resource object management node can manage at least one resource object execution node. The resource object execution node can obtain a service request (e.g., a model service request) from a client, and generate a sub-request to the resource object execution node based on the service emotion. The resource object execution node can respond to the sub-request and return a service result (e.g., a prediction result) or an intermediate result to the resource object management node. The resource object execution node can forward the service result to the client; or process based on the intermediate result to obtain a service result, and return it to the client. Figure 2A The methods include:

[0095] 210: Receive a model service request sent to a service container management application instance, where the service container management application instance includes a service container, and the service container is used to provide application services.

[0096] It should be understood that the container management application instance in the text can be an object for creating containers, deleting containers, arranging containers, managing containers, etc. The container management application instance can be implemented as a specific abstraction layer in the execution node. The container in the container management application instance can also manage at least one container runtime (runtime, as an instance of a container) through a container engine. The container engine can be used to run at least one container. For example, the container engine can run the container by creating a container runtime or deleting a container runtime. In addition, the container management application instance can be installed in the container engine, thereby achieving unified management of the container runtime and the container, and achieving compatibility with existing container technologies (e.g., containers or container clouds). Optionally, the container engine can be installed in the container management application instance, thereby achieving orchestration and management of the container while ensuring the independence of the container engine. For receiving a model service request sent by a management node for a container management application instance having a third network address, the model service request can be obtained through a local agent of the management node. The model service request can include a third network address. The container management application instance can be determined based on the model service request. The service container can be called and accessed by calling commands. For receiving a model service request sent by a management node to a container management application instance having a third network address, the model service request may be received through a network interface of the container management application instance based on the first network address. The service container may be installed in the container management application instance. The container management application instance may be installed in an operating system of a server as a host system. The container management application instance may be installed on a container engine. In one example, the container engine may be installed on the container management application instance. In another example, the container management application instance is installed under a container runtime and manages the container runtime instance.

[0097] 220: Respond to the model service request and determine a model container management application instance, wherein the model container management application instance includes a model container, and the model container is used to run the prediction model.

[0098] It should be understood that in response to the model service request, the model container management application instance with the fourth network address is determined, and the model container management application instance is determined by communicating with the third network address and the fourth network address. The model container management application instance can be determined by the local agent of the management node. The model container can be called and accessed by calling commands. Among them, the model container can be installed in the container management application instance.

[0099] 230: Call the prediction model in the model container through the service container to perform prediction.

[0100] It should be understood that, for responding to a model service request, the service container calls the prediction model in the model container for prediction, and the service container and the model container can access each other by calling commands. Alternatively, the service container and the model container can access each other through inter-container communication. In addition, the model service request may include data to be predicted. The model service request may be a request sent or forwarded by a local agent of the management node. In addition, the prediction model may run on an image (IMAGE) configured by the model container. The image may be a read-only program, and one or more images may be pulled from the image warehouse in creating the container to configure the container. For responding to a model service request, the service container calls the prediction model in the model container for prediction, and the model service request may include data to be predicted. The model service request may be a request sent or forwarded by a local agent of the management node. The prediction model may run on the image of the model container. In addition, the service container calls the prediction model in the model container through the communication between the third network address and the fourth network address. The service container may call the prediction model in the model container through the bridge between the service container and the model container. The service container may also communicate with the second agent of the management node through the first agent of the management node to call the prediction model in the model container. The first agent and the second agent may be managed by the management node. The service container may call the prediction model in the model container through the layer 2 network between the service container and the model container. For example, the service container may have a port of the third network address, and the model container may have a port of the fourth network address.

[0101] 240: Returns the prediction results of the prediction model.

[0102] It should be understood that, for returning the prediction result of the prediction model to the management node based on the first network address, the prediction result of the prediction model can be returned to the management node through the port of the first network address. The prediction result of the prediction model can be returned through the local agent of the management node. For example, the prediction result of the prediction model can be returned through the local agent of the management node based on the port of the first network address. In addition, the prediction result can be the output of the prediction model, corresponding to the input data provided in response to the model service request.

[0103] In other words, the solution of this embodiment puts the service container and the model container into an independent container management application instance, for example, the entire cluster shares the same set of service containers. Figure 2B FIG. 1 is a schematic diagram of a prediction service stage of a cloud computing service method according to another embodiment of the present invention. Figure 2BAs shown, the service container management application instance and the model container management application instance have different network addresses. Different container management application instances, or containers in different container management application instances, can communicate using network addresses. On the one hand, since containers in different container management application instances can still be updated or replaced independently, the flexibility of platform or algorithm updates is improved. On the other hand, since the service container and the model container are in different container management application instances, the two containers have higher independence compared to the scheme in the above embodiment, and further decoupling is achieved. For example, a service container management application instance can be used to create more service containers based on different services. Similarly, a model container management application instance can be used to create more model containers based on different services. Therefore, at the cluster level, the disorder of scheduling service containers based on different services and model containers based on different services is avoided, thereby achieving effective utilization of resources as a whole, and saving communication resources and scheduling resources as a whole.

[0104] In one example, after the initialization container obtains the prediction model from outside the container management application instance, the initialization container can be deleted, thereby increasing the storage space of the container management application instance while shielding the difference in storage media. It should be understood that after the initialization container obtains the prediction model from outside the container management application instance, it can mark the completion of the model deployment phase. After that, on the one hand, the platform can respond to the user's request and send instructions through the management node to call the prediction model in the model container. On the other hand, the platform maintainer or model algorithm personnel can update the platform code at any time.

[0105] Figure 3 A schematic flow chart of a cloud computing service method according to another embodiment of the present invention; Figure 3 The cloud computing service method can be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing service network, a resource object execution node can communicate with a client to provide the client with a specific service requested (e.g., a service provided based on a prediction model). In addition, the service end of the cloud computing service network may include an execution node and a management node (e.g., a resource object management node). The resource object management node can manage at least one resource object execution node. For example, a resource object execution node can obtain a service request (e.g., a model service request) from a client, and generate a sub-request to the resource object execution node based on the service emotion. The resource object execution node can respond to the sub-request and return a service result (e.g., a prediction result) or an intermediate result to the resource object management node. The resource object execution node can forward the service result to the client; or process based on the intermediate result to obtain a service result, and return it to the client. Figure 3 The methods include:

[0106] 310: Receive a prediction model deployment request sent for a container management application instance, where the container management application instance includes a service container, and the service container is used to provide application services.

[0107] The container management application instance in the text may be an object for creating containers, deleting containers, orchestrating containers, managing containers, and the like. The container management application instance may be implemented as a specific abstraction layer in an execution node. The container in the container management application instance may also manage at least one container runtime (runtime, as an instance of a container) through a container engine. The container engine may be used to run at least one container, for example, the container engine may run a container by creating a container runtime or deleting a container runtime. In addition, the container management application instance may be installed in the container engine, thereby achieving unified management of the container runtime and the container, and achieving compatibility with existing container technologies (e.g., containers or container clouds). Optionally, the container engine may be installed in the container management application instance, thereby achieving orchestration and management of the container while ensuring the independence of the container engine.

[0108] 320: In response to the prediction model deployment request, a second container and an initialization container are created in the container management application instance. The model container is used to run the prediction model, and the initialization container is used to obtain the prediction model.

[0109] It should be understood that the initialization container can share the storage space of the container management application instance with the model container, and the initialization container can obtain the prediction model from the storage space of the container management application instance. In addition, any method can be used for the initialization container to obtain the prediction model from the storage space of the container management application instance, and the embodiment of the present invention does not limit this.

[0110] 330: Obtain the prediction model into the model container through the initialization container, so as to call the prediction model in the model container through the service container for prediction.

[0111] It should be understood that in response to the model service request, the service container calls the prediction model in the model container for prediction, and the service container and the model container can access each other through calling commands. Alternatively, the service container and the model container can access each other through inter-container communication. In addition, the model service request may include data to be predicted. For example, the model service request is a request sent or forwarded by a local agent of the management node. In addition, the prediction model can run on the image (IMAGE) configured by the model container. The image can be a read-only program, and one or more images can be pulled from the image repository in creating the container to configure the container.

[0112] It should be understood that the cloud computing service method of the embodiment of the present invention may also include an application service deployment phase. In the model deployment phase, for example, the method may also include: receiving an application service deployment request sent by a management node for a container management application instance; responding to the application service deployment request, and creating a service container in the container management application instance.

[0113] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0114] Figure 4 A schematic flow chart of a cloud computing service method according to another embodiment of the present invention; Figure 4 The cloud computing service method can be applied to an execution node (e.g., a resource object execution node) in a cloud computing service network. In a cloud computing service network, a resource object execution node can communicate with a client to provide the client with a specific service requested (e.g., a service provided based on a prediction model). In addition, the service end of the cloud computing service network may include an execution node and a management node (e.g., a resource object management node). The resource object management node can manage at least one resource object execution node. For example, a resource object execution node can obtain a service request (e.g., a model service request) from a client, and generate a sub-request to the resource object execution node based on the service emotion. The resource object execution node can respond to the sub-request and return a service result (e.g., a prediction result) or an intermediate result to the resource object management node. The resource object execution node can forward the service result to the client; or process based on the intermediate result to obtain a service result, and return it to the client.

[0115] It should also be understood that in this example, the cloud computing service method is for neural network computing services such as natural language processing or machine translation. The prediction model is a natural language processing model or a language recognition model or a machine translation module, and the prediction model is obtained by training a recurrent neural network (RNN). In addition, the client requesting the prediction service from the execution node or the management node can be a machine translation engine, a machine translation application, a speech recognition application, etc. Figure 4 The methods include:

[0116] 410: Receive a neural network model service request sent to a container management application instance, where the model service request includes semantic data to be predicted. The container management application instance includes a service container and a model container. The service container is used to provide application services, and the model container is used to run a neural network prediction model.

[0117] The container management application instance in the text may be an object for creating containers, deleting containers, orchestrating containers, managing containers, and the like. The container management application instance may be implemented as a specific abstraction layer in an execution node. The container in the container management application instance may also manage at least one container runtime (runtime, as an instance of a container) through a container engine. The container engine may be used to run at least one container, for example, the container engine may run a container by creating a container runtime or deleting a container runtime. In addition, the container management application instance may be installed in the container engine, thereby achieving unified management of the container runtime and the container, and achieving compatibility with existing container technologies (e.g., containers or container clouds). Optionally, the container engine may be installed in the container management application instance, thereby achieving orchestration and management of the container while ensuring the independence of the container engine.

[0118] 420: Respond to the model service request, call the prediction model in the model container through the service container, perform semantic processing on the semantic data to be predicted, and obtain the semantic processing result.

[0119] It should be understood that for responding to a model service request, the service container calls the prediction model in the model container for prediction, and the service container and the model container can access each other by calling commands. Alternatively, the service container and the model container can access each other through inter-container communication. In addition, the model service request may include data to be predicted. The model service request may be a request sent or forwarded by a local agent of the management node. The prediction model can run on the image (IMAGE) configured by the model container. The image can be read-only and pulled from the image repository when creating the container.

[0120] 430: Return the prediction result including the semantic processing result.

[0121] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0122] Figure 5 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention; Figure 5 The cloud computing service devices include:

[0123] The model service request receiving module 510 receives a model service request sent to a container management application instance, wherein the container management application instance includes a service container and a model container, wherein the service container is used to provide application services and the model container is used to run a prediction model;

[0124] The container calling module 520 responds to the model service request and calls the prediction model in the model container through the service container to perform prediction;

[0125] The prediction result returning module 530 returns the prediction result of the prediction model.

[0126] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0127] In other words, since the platform service is independent in the service container, it no longer shares the environment with the model service. Therefore, only one copy of the platform code and operating environment needs to be maintained. At the same time, it is not restricted by the algorithm environment, and more customized performance and expansion are achieved. Accordingly, the model service is independent in the model container and does not share the environment with the platform service. Algorithm personnel can use model frameworks, languages, and dependencies more flexibly.

[0128] It should be understood that in the solution of the embodiment of the present invention, even if the model container and the platform service model are in the same container management application instance (e.g., POD) to provide external services, the container management framework (e.g., the Kubernetes release mechanism (e.g., rolling update mechanism)) can only update one container in the container management application instance, so the service container and the model container can be decoupled.

[0129] In another implementation of the present invention, the device also includes: a container creation module, which responds to a prediction model deployment request sent to a container management application instance, creates an initialization container in the container management application instance for obtaining the prediction model; and a model acquisition module, which acquires the prediction model into the model container through the initialization container.

[0130] In another implementation of the present invention, the initialization container and the model container share the storage space of the container management application instance, wherein the model acquisition module is specifically used to: acquire the prediction model from the storage medium to the storage space through the initialization container, so that the model container acquires the prediction model from the storage space.

[0131] In another implementation of the present invention, the container management application instance has a first network address, and the storage management object with a second network address is used to manage the storage medium, wherein the model acquisition module is specifically used to: initiate an access request for the storage medium to the second network address based on the first network address by initializing the container, so that the storage management object responds to the access request and returns the prediction model to the storage space.

[0132] In another implementation of the present invention, after acquiring the prediction model from the storage medium to the storage space by initializing the container, the device further includes: a container deletion module, which deletes the initialization container from the container management application instance.

[0133] In another implementation of the present invention, the model service request receiving module is specifically used to: receive a model service request sent by the management node for the container management application instance of the first network address,

[0134] The prediction result returning module is specifically used to return the prediction result of the prediction model to the management node based on the first network address.

[0135] In another implementation of the present invention, the service container has a first virtual port based on a first network address, and the device also includes: a service container update module, which responds to a service container update instruction sent by a management node, creates an updated service container in a container management application instance, and allocates a second virtual port based on the first network address to the updated service container, where the second virtual port is different from the first virtual port; and deletes the service container from the container management application instance.

[0136] In another implementation of the present invention, the model container has a third virtual port based on the first network address, and the device also includes: a model container update module, which responds to the model container update instruction sent by the management node, creates an updated model container in the container management application instance, and allocates a fourth virtual port based on the first network address to the updated model container, and the fourth virtual port is different from the third virtual port; deletes the model container from the container management application instance.

[0137] In another implementation of the present invention, before responding to the model deployment request and creating an initialization container in the container management application instance, the device also includes: a container creation module, which responds to the container management application instance creation instruction sent by the management node, creates the container management application instance, and allocates a first network address.

[0138] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0139] Figure 6 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention; Figure 6 The cloud computing service devices include:

[0140] The request receiving module 610 receives a model service request initiated by a user;

[0141] A node allocation module 620 allocates execution nodes to model service requests;

[0142] The request processing module 630 processes the model service request by using the container management application instance on the execution node, and the container management application instance includes at least the following containers: an initialization container and a model container;

[0143] The processing result returning module 640 returns the processing result to the user.

[0144] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0145] In another implementation of the present invention, the request processing module is specifically used to: initialize the container to copy the user's data to the local execution node; and the model container uses the user's data to provide model services for the user.

[0146] In another implementation of the present invention, the container management application instance further includes: a service container; the request processing module is further used for: the service container to provide the user with corresponding performance services of the model service.

[0147] In another implementation of the present invention, after the initialization container copies the user's data to the local execution node, the device further includes: a deletion module that destroys the initialization container.

[0148] In another implementation of the present invention, the request processing module is specifically used for: the model container defines the corresponding model framework and language according to the user's needs, and provides model services to the user through the model framework and language.

[0149] In another implementation of the present invention, the initialization container, the model container, and the service container share a network and a disk.

[0150] In another implementation of the present invention, the node allocation module is specifically used to: respond to a model service request and allocate a route of an execution node to the model service request based on the load status of multiple candidate execution nodes.

[0151] In another implementation of the present invention, the node allocation module is specifically used to: respond to a model service request, send load status query requests to multiple candidate execution nodes respectively; and allocate execution node routes to the model service request by receiving current load status information sent by multiple candidate execution nodes.

[0152] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0153] The device of this embodiment is used to implement the corresponding methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module in the device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here.

[0154] Figure 7 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention; Figure 7 The cloud computing service device includes:

[0155] A deployment request receiving module 710 receives a prediction model deployment request sent to the container management application instance, wherein the container management application instance includes a service container, and the service container is used to provide application services;

[0156] The container creation module 720, in response to the prediction model deployment request, creates a model container and an initialization container in the container management application instance, wherein the model container is used to run the prediction model, and the initialization container is used to obtain the prediction model;

[0157] The model acquisition module 730 acquires the prediction model into the model container through the initialization container, so as to call the prediction model in the model container through the service container to perform prediction.

[0158] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0159] Figure 8 is a schematic block diagram of a cloud computing service device according to another embodiment of the present invention; Figure 8 The cloud computing service device includes:

[0160] The model service request receiving module 810 receives a neural network model service request sent to a container management application instance, wherein the model service request includes semantic data to be predicted, and the container management application instance includes a service container and a model container, wherein the service container is used to provide application services, and the model container is used to run a neural network prediction model;

[0161] The semantic processing module 820 responds to the model service request, calls the prediction model in the model container through the service container, performs semantic processing on the semantic data to be predicted, and obtains a semantic processing result;

[0162] The prediction result returning module 830 returns the prediction result including the semantic processing result.

[0163] In the solution of the embodiment of the present invention, since the initialization container and the model container maintain the independence of the containers, they can be updated or maintained independently, thus improving the flexibility of the computing platform update.

[0164] Fig. 9 This is a schematic diagram of the structure of an electronic device according to another embodiment of the present invention; the electronic device may include:

[0165] One or more processors 901;

[0166] The computer readable medium 902 may be configured to store one or more programs,

[0167] When the one or more programs are executed by the one or more processors, the one or more processors implement the cloud computing service method as described in the above embodiments.

[0168] Fig.10 The hardware structure of the electronic device of another embodiment of the present invention is as follows; Fig.10 As shown, the hardware structure of the electronic device may include: a processor 1001, a communication interface 1002, a computer readable medium 1003 and a communication bus 1004;

[0169] The processor 1001, the communication interface 1002, and the computer-readable medium 1003 communicate with each other via a communication bus 1004;

[0170] Optionally, the communication interface 1002 may be an interface of a communication module;

[0171] Among them, the processor 1001 can be specifically configured to: receive a model service request sent to a container management application instance, the container management application instance includes a service container and a model container, the service container is used to provide application services, and the model container is used to run the prediction model; respond to the model service request, call the prediction model in the model container through the service container to perform prediction; return the prediction result of the prediction model.

[0172] Alternatively, the processor 1001 can be specifically configured to: receive a model service request sent to a service container management application instance, the service container management application instance includes a service container, and the service container is used to provide application services; respond to the model service request, determine the model container management application instance, wherein the model container management application instance includes a model container, and the model container is used to run the prediction model; call the prediction model in the model container through the service container to perform prediction; and return the prediction result of the prediction model.

[0173] Alternatively, the processor 1001 can be specifically configured to: receive a prediction model deployment request sent to a container management application instance, the container management application instance includes a service container, and the service container is used to provide application services; respond to the prediction model deployment request, create a second container and an initialization container in the container management application instance, the model container is used to run the prediction model, and the initialization container is used to obtain the prediction model; obtain the prediction model into the model container through the initialization container, so as to call the prediction model in the model container through the service container for prediction.

[0174] Alternatively, processor 1001 can be specifically configured to: receive a neural network model service request sent to a container management application instance, the model service request includes semantic data to be predicted, the container management application instance includes a service container and a model container, the service container is used to provide application services, and the model container is used to run a neural network prediction model; respond to the model service request, call the prediction model in the model container through the service container, perform semantic processing on the semantic data to be predicted, and obtain a semantic processing result; return a prediction result including the semantic processing result.

[0175] The processor 1001 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0176] The computer readable medium 1003 can be, but is not limited to, a random access memory medium (RAM), a read-only memory medium (ROM), a programmable read-only memory medium (PROM), an erasable read-only memory medium (EPROM), an electrically erasable read-only memory medium (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0177] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through a communication part, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), the above functions defined in the method of the present application are executed. It should be noted that the computer-readable medium described in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access storage medium (RAM), a read-only storage medium (ROM), an erasable programmable read-only storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only storage medium (CD-ROM), an optical storage medium, a magnetic storage medium, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, an apparatus or a device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program configured to be used by or in combination with an instruction execution system, an apparatus or a device. The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0178] Computer program code configured to perform the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0179] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. There are specific sequential relationships in the above-mentioned specific embodiments, but these sequential relationships are only exemplary. When the specific implementation is implemented, these steps may be less, more, or the execution order may be adjusted. That is, in some implementations as replacements, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The modules involved in the embodiments described in this application may be implemented by software or hardware. The names of these modules do not constitute limitations on the modules themselves in some cases.

[0181] As another aspect, the present application further provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the cloud computing service method as described in the above embodiments.

[0182] As another aspect, the present application also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by the device, the device: receives a model service request sent by a management node to a container management application instance with a first network address, wherein the container management application instance includes a service container and a model container; responds to the model service request, calls the prediction model in the model container through the service container for prediction; based on the first network address, returns the prediction result of the prediction model to the management node, or,

[0183] Receive a model service request sent by a management node for a service container having a third network address, wherein the service container management application instance includes a service container; respond to the model service request, determine a model container management application instance having a fourth network address, wherein the model container management application instance includes a model container; based on the third network address, call the prediction model in the model container from the fourth network address through the service container to perform prediction; and based on the third network address, return the prediction result of the prediction model to the management node.

[0184] The expressions "first", "second", "the first" or "the second" used in various embodiments of the present disclosure may modify various components regardless of order and / or importance, but these expressions do not limit the corresponding components. The above expressions are only configured for the purpose of distinguishing an element from other elements. For example, a first user device and a second user device represent different user devices, although both are user devices. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the present disclosure.

[0185] When one element (e.g., a first element) is referred to as being "(operably or communicatively) coupled" or "(operably or communicatively) coupled to" or "connected to" another element (e.g., a second element), it is understood that the one element is directly connected to the other element or the one element is indirectly connected to the other element via yet another element (e.g., a third element). Conversely, it is understood that when an element (e.g., a first element) is referred to as being "directly connected" or "directly coupled" to another element (the second element), no element (e.g., a third element) is interposed between the two.

[0186] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A cloud computing service method, comprising: Receive model service requests initiated by users; Allocating an execution node for the model service request; On the execution node, the model service request is processed by using a container management application instance, wherein the container management application instance includes at least the following containers: an initialization container, a model container, and a service container, the initialization container copies the user's data to the local execution node, the model container uses the user's data to provide the user with a model service, the initialization container and the model container are independent of each other, and the service container provides the user with a corresponding performance service of the model service; The processing result is returned to the user.

2. The method according to claim 1, wherein: After the initialization container copies the user's data to the local execution node, the method further includes: Destroy the initialization container.

3. The method according to claim 1, wherein: The model container uses the user's data to provide model services for the user, including: The model container defines a corresponding model framework and language according to the needs of the user, and provides model services to the user through the model framework and language.

4. The method according to claim 1, wherein: The initialization container, the model container, and the service container share a network and a disk.

5. The method according to any one of claims 1 to 4, wherein: The allocating an execution node for the model service request includes: In response to the model service request, a route of the execution node is assigned to the model service request based on the load status of multiple candidate execution nodes.

6. The method according to claim 5, wherein: The responding to the model service request and allocating a route of the execution node to the model service request based on load states of a plurality of candidate execution nodes includes: In response to the model service request, a load status query request is sent to each of the plurality of candidate execution nodes; By receiving the current load status information sent by the multiple candidate execution nodes, the routing of the execution nodes is allocated to the model service request.

7. A cloud computing service method, comprising: A model service request sent in response to a container management application instance, wherein the container management application instance includes a service container and a model container, wherein the service container is used to provide application services and the model container is used to run a prediction model; In response to the model service request, the prediction model in the model container is called by the service container to perform prediction; Returns the prediction result of the prediction model.

8. A cloud computing service method, comprising: receiving a prediction model deployment request sent for a container management application instance, wherein the container management application instance includes a service container, and the service container is used to provide an application service; In response to the prediction model deployment request, a model container and an initialization container are created in the container management application instance, wherein the model container is used to run the prediction model, the initialization container is used to obtain the prediction model, and the model container and the initialization container are independent of each other; The prediction model is acquired into the model container through the initialization container, so that the prediction model in the model container is called by the service container for prediction.

9. A cloud computing service method, comprising: Receiving a neural network model service request sent to a container management application instance, wherein the model service request includes semantic data to be predicted, and the container management application instance includes a service container and a model container, wherein the service container is used to provide application services, and the model container is used to run a neural network prediction model; In response to the model service request, the prediction model in the model container is called through the service container to perform semantic processing on the semantic data to be predicted to obtain a semantic processing result; A prediction result including the semantic processing result is returned.

10. A cloud computing service device, comprising: A model service request receiving module receives a model service request sent to a container management application instance, wherein the container management application instance includes a service container and a model container, wherein the service container is used to provide application services, and the model container is used to run a prediction model; A container calling module, responding to the model service request, calling the prediction model in the model container through the service container to perform prediction; The prediction result returning module returns the prediction result of the prediction model.

11. A cloud computing service device, comprising: Request receiving module, receiving model service requests initiated by users; A node allocation module allocates execution nodes to the model service request; a request processing module, on the execution node, using a container management application instance to process the model service request, wherein the container management application instance includes at least the following containers: an initialization container, a model container, and a service container, wherein the initialization container copies the user's data to the local execution node, the model container uses the user's data to provide the user with a model service, the initialization container and the model container are independent of each other, and the service container provides the user with a corresponding performance service of the model service; The processing result returning module returns the processing result to the user.

12. A cloud computing service device, comprising: A deployment request receiving module receives a prediction model deployment request sent to a container management application instance, wherein the container management application instance includes a service container, and the service container is used to provide application services; A container creation module, wherein the container responds to the prediction model deployment request and creates a model container and an initialization container in the container management application instance, wherein the model container is used to run the prediction model, the initialization container is used to obtain the prediction model, and the model container and the initialization container are independent of each other; The model acquisition module acquires the prediction model into the model container through the initialization container, so as to call the prediction model in the model container through the service container to perform prediction.

13. A cloud computing service device, comprising: A model service request receiving module receives a neural network model service request sent to a container management application instance, wherein the model service request includes semantic data to be predicted, and the container management application instance includes a service container and a model container, wherein the service container is used to provide application services, and the model container is used to run a neural network prediction model; A semantic processing module, in response to the model service request, calls the prediction model in the model container through the service container, performs semantic processing on the semantic data to be predicted, and obtains a semantic processing result; The prediction result returning module returns the prediction result including the semantic processing result.

14. An electronic device, comprising: one or more processors; A computer readable medium configured to store one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 9.

15. A computer readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images