Service deployment method, apparatus and electronic device

Abstract

The application discloses a service deployment method and device and electronic equipment, wherein the method comprises the following steps: receiving an access request of a user, and dispatching the access request to a to-be-deployed service via routing; determining a video memory requirement of the to-be-deployed service based on the access request, and generating a configuration file based on the video memory requirement and a graphics card located on a server; allocating video memory for the to-be-deployed service according to the configuration file, deploying an instance in the video memory through the to-be-deployed service, wherein the video memory is located in a target graphics card of a target server, the target server is a server used for deploying the to-be-deployed service, and the target graphics card is a graphics card in the target server which provides video memory resources for the to-be-deployed service; and allocating the access request to the deployed instance and running the instance, so as to deploy the to-be-deployed service to the target server. The application automatically allocates the video memory in the target graphics card of the target server for the to-be-deployed service through the configuration file, realizes the automatic deployment of the service, and does not need to deploy the service to the target server by manpower.

Description

Technical Field

[0001] This invention relates to the field of service operation and maintenance technology, and specifically to a service deployment method, apparatus, and electronic device. Background Technology

[0002] When the number of services to be deployed is small, when a user requests access to a service, the corresponding amount of video memory can be manually allocated to the service based on the specific access request, and manual maintenance can be performed. Since graphics cards have various different video memory capacities, and the video memory usage of each service varies, and services occupy video memory during the execution of instances without releasing it, careful manual management of video memory allocation is necessary when running multiple instances with limited video memory resources.

[0003] However, as business volume increased, the number of services to be deployed also increased, requiring more servers to be deployed, making it impossible to deploy services manually. Summary of the Invention

[0004] The main objective of this invention is to provide a service deployment method, apparatus, and electronic device to solve the problem that the number of services to be deployed has increased to the point that it is no longer possible to deploy services manually.

[0005] To achieve the above objectives, a first aspect of the present invention provides a service deployment method, comprising:

[0006] Receive user access requests and route them to the service to be deployed.

[0007] The memory requirements of the service to be deployed are determined based on the access request, and a configuration file is generated based on the memory requirements and the graphics card located on the server.

[0008] According to the configuration file, video memory is allocated to the service to be deployed. An instance of the service is then deployed within this video memory. The video memory resides in the target graphics card on the target server, which is the server used to deploy the service, and the target graphics card is the graphics card on the target server used to provide the video memory resources for the service.

[0009] The access request is assigned to the deployed instance and the instance is run to deploy the service to the target server.

[0010] Optionally, receiving a user's access request includes:

[0011] Access requests are received through Kubernetes, where routes are configured and Kubernetes receives access requests through these routes.

[0012] This includes routing access requests to the service to be deployed, which includes:

[0013] Obtain the type of the access request, and determine the corresponding service to be deployed based on the access request type; and

[0014] Obtain the IP address and port of the service to be deployed, and then route access requests to the service to be deployed using the IP address and port of the service to be deployed.

[0015] Optionally, the video memory requirements of the service to be deployed are determined based on the access request, and a configuration file is generated based on the video memory requirements and the graphics card located on the server, including:

[0016] Based on the video memory resources required by each access request in the service to be deployed, determine the video memory requirements of the service to be deployed.

[0017] Based on the video memory requirements and the available video memory capacity of each graphics card in each server, the target server and the target graphics card in the target server are determined for the service to be deployed, wherein each server includes one or more graphics cards.

[0018] Determine the percentage of video memory required by the target graphics card and generate a configuration file.

[0019] Optionally, allocating video memory for the service to be deployed according to the configuration file includes:

[0020] Based on the configuration file, determine the target server assigned to the service to be deployed and the target graphics card located on the target server;

[0021] Based on the proportion of video memory demand to the video memory capacity of the target graphics card, GPU container sharing technology is used to allocate video memory in the target graphics card.

[0022] Allocate a portion of the video memory to the services to be deployed, so that the services to be deployed can use the video memory.

[0023] Optionally, before receiving a user's access request, the method further includes:

[0024] The server is used as a platform to run containers, and multiple containers are created on each server using Kubernetes;

[0025] Containerize the service to be deployed and start the service in a container manner.

[0026] Furthermore, the running instances include:

[0027] Run instances within containers, where each container contains one instance;

[0028] After running the instance, the method also includes:

[0029] Obtain the instance's operational status, including whether it is running normally or abnormally; and

[0030] If the instance is running abnormally, adjust the number of instances to be deployed through the service to be deployed.

[0031] Optionally, the method further includes:

[0032] The scripts of the services to be tested are used to periodically perform health checks on the services themselves. The services to be tested include services to be deployed and services that have already been deployed.

[0033] Determine whether the service to be tested has passed the health check;

[0034] If the service to be tested passes the health check, then the service to be tested is normal, and the tested service is obtained;

[0035] If the service to be tested fails the health check, the service to be tested is considered abnormal. The service to be tested will be taken offline and a new service to be tested will be started to maintain the number of requests per second for the entire service to be tested.

[0036] A second aspect of the present invention provides a service deployment apparatus, comprising:

[0037] The receiving unit is used to receive user access requests and route the access requests to the service to be deployed via routing.

[0038] The generation unit is used to determine the video memory requirements of the service to be deployed based on the access request, and to generate a configuration file based on the video memory requirements and the graphics card located on the server.

[0039] An allocation unit is used to allocate video memory to the service to be deployed according to a configuration file, and deploy instances of the service in the video memory. The video memory is located in the target graphics card of the target server, which is the server used to deploy the service, and the target graphics card is the graphics card on the target server used to provide video memory resources for the service.

[0040] The deployment unit is used to distribute access requests to deployed instances and run the instances to deploy the service to the target server.

[0041] A third aspect of the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to perform the service deployment method provided in any of the first aspects.

[0042] A fourth aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform a service deployment method provided in any of the first aspects.

[0043] In the service deployment method provided in this embodiment of the invention, the video memory requirements of the service to be deployed are determined based on the access request, and a configuration file is generated based on the video memory requirements and the graphics card located on the server; video memory is allocated to the service to be deployed according to the configuration file, and an instance of the service to be deployed is deployed in the video memory, wherein the video memory is located in the target graphics card of the target server, the target server is the server used to deploy the service to be deployed, and the target graphics card is the graphics card in the target server used to provide video memory resources for the service to be deployed; by automatically allocating video memory in the target graphics card of the target server to the service to be deployed through the configuration file containing the video memory requirements, manual operation is reduced, and there is no need to manually consider whether the video memory resources in the target server to which the service is deployed are sufficient;

[0044] Access requests are distributed to deployed instances and the instances are run to deploy the services to the target server, thus achieving automated service deployment without the need for manual deployment to the target server. This solves the problem that the number of services to be deployed increases and it becomes impossible to deploy them manually. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 A flowchart illustrating the service deployment method provided in an embodiment of the present invention;

[0047] Figure 2 A flowchart illustrating the service deployment method provided in this embodiment of the invention;

[0048] Figure 3 A block diagram of a service deployment device provided in an embodiment of the present invention;

[0049] Figure 4 This is a block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0053] When the number of services to be deployed is small, each user request can be manually allocated a corresponding amount of video memory based on the specific access request, allowing for manual maintenance. However, since graphics cards come in various memory capacities, and each service's memory usage varies, and services occupy and do not release video memory during runtime, careful manual management of memory allocation is necessary when running multiple instances within limited video memory resources. Furthermore, as business volume increases, the number of services to be deployed grows, requiring deployment on more servers, making manual deployment no longer feasible.

[0054] To address the above problems, embodiments of the present invention provide a service deployment method, such as... Figure 1 As shown, the method includes the following steps S101 to S104:

[0055] Step S101: Receive the user's access request and route the access request to the service to be deployed via routing;

[0056] Optionally, the access request can be a request to access the service to be deployed. For example, a user's access request can be an operation such as accessing the homepage, logging in, querying data, acquiring data, starting the fundus camera, or taking fundus images.

[0057] In an optional embodiment of the present invention, before receiving the user's access request in step S101, the method further includes:

[0058] The server is used as the carrier to run containers, and multiple containers are created on each server through Kubernetes. Kubernetes, abbreviated as k8s, is an open source application for managing containerized applications on multiple servers in a cloud platform. It is an open source container orchestration platform for running distributed applications and services at scale, and its goal is to make the deployment of containerized applications simple and efficient. This embodiment of the invention introduces k8s for service management and transforms services into a way that is suitable for k8s execution.

[0059] Containerize the service to be deployed and start it as a container. In Kubernetes, the service to be deployed is started as a container, and a configuration file corresponding to the service is written.

[0060] The step S101 of receiving the user's access request includes:

[0061] Access requests are received through Kubernetes, which has routing configured to receive access requests via these routes.

[0062] Specifically, step S101, which routes the access request to the service to be deployed, includes:

[0063] Obtain the type of access request, and determine the service to be deployed corresponding to the access request based on the type of access request;

[0064] Optionally, each service to be deployed has a mapping relationship with a corresponding type of access request. Access requests are divided into different types according to their usage frequency. Different types of access requests are mapped to different services to be deployed in Kubernetes. Access requests and services to be deployed correspond to each other through the mapping relationship. Furthermore, different services to be deployed have different numbers of concurrent instances. The more concurrent instances, the more similar access requests can be processed at the same time, resulting in higher memory resource utilization and shorter processing time.

[0065] For example, the types of access requests can include accessing the homepage, logging in, data querying, data acquisition, starting the fundus camera, and taking fundus images. Different types of access requests are mapped to different services to be deployed. When the frequency of users accessing the homepage is greater than that of data querying, then accessing the homepage is considered an important type compared to data querying. The number of concurrent instances of the service to be deployed corresponding to accessing the homepage is greater than the number of concurrent instances of the service to be deployed corresponding to data querying, so as to ensure the allocation of GPU memory resources for important types of access requests in the services to be deployed and improve the utilization of GPU memory resources.

[0066] Obtain the IP address and port of the service to be deployed. Using the IP address and port of the service to be deployed, route the access request to the service to be deployed. When the access request enters Kubernetes, the Kubernetes router forwards the access request to the service to be deployed using the IP address and unique port number of the service to be deployed.

[0067] When a user's access request enters Kubernetes, it is first forwarded to the router. The router forwards the access request to the corresponding service to be deployed based on the type of access request. The service to be deployed has multiple instances deployed concurrently in multiple containers to handle access requests. Based on the frequency of different types of access requests, access requests of a certain type are scheduled to the corresponding service to be deployed. The service to be deployed with multiple instances deployed concurrently can handle multiple access requests of the same type at the same time, saving processing time.

[0068] The flowchart of the service deployment method provided in this embodiment of the invention is as follows: Figure 2 As shown, in this diagram, Service represents a service, with Service1, Service2, and Service3 representing Service 1, Service 2, and Service 3 respectively; Node represents a node, i.e., a server, with Node1, Node2, and Node3 representing Server 1, Server 2, and Server 3 respectively; Container represents a container, with each container containing one instance; Service 1 can be for accessing the homepage, Service 2 can be for logging in, and Service 3 can be for data querying. When the number of requests to display the homepage exceeds the number of requests for data querying, Service 1 can be configured to concurrently deploy three instances of instance 1, and Service 3 can be configured to deploy one instance of instance 3. When there are three requests to access the homepage at the same time, Service 1 can concurrently process these three requests, reducing processing time. Furthermore, multiple different services can be deployed on one or more graphics cards of a server, improving the utilization of video memory resources; for example, in Figure 2 In the middle, server 2 deploys service 1 and service 2, and server 3 deploys service 2 and service 3.

[0069] In one optional embodiment of the present invention, when determining the service to be deployed corresponding to the access request, in order to ensure load balancing, the loads of all currently deployed services can be sorted in ascending order of load. It is then determined whether the load of the lightest deployed service exceeds a load threshold. If it does, a new deployed service is started; otherwise, the user's access request is forwarded to the lightest deployed service. Alternatively, the number of currently running deployed services can be determined, and the maximum number of deployed services supported by the graphics card's memory capacity can be calculated. If the maximum number of deployed services supported by the graphics card's memory capacity is greater than the current number of deployed services, a new deployed service is started; otherwise, the user's access request is forwarded to the lightest deployed service.

[0070] By calculating the relationship between the load and load threshold of all services to be deployed, or by calculating the relationship between the number of services to be deployed and the maximum number of services supported by the video memory, the system determines whether to start a new service to be deployed to respond to user access requests or to forward user access requests to the least loaded service. This can speed up the response to user access requests and achieve load balancing among the services to be deployed.

[0071] In one optional embodiment of the present invention, when determining the service to be deployed corresponding to the access request, dynamic routing can be performed using a fixed load balancing strategy to meet the requirements for routing control. When multiple user access requests enter Kubernetes, the access requests are scheduled using various load balancing strategies to ensure that the load on each service to be deployed is balanced. The load balancing strategy can be a round-robin algorithm, an IP address hash algorithm, a random algorithm, a least-connections algorithm, etc.

[0072] Step S102: Determine the video memory requirements of the service to be deployed based on the access request, and generate a configuration file based on the video memory requirements and the graphics card located on the server;

[0073] Optionally, the video memory requirement can be the video memory requirement of the service to be deployed on the graphics card. The video memory requirement of the service to be deployed on the graphics card is determined based on the resource usage of the access requests. A target graphics card with a video memory capacity greater than the video memory requirement of the service to be deployed is selected from the server's graphics cards. After generating a configuration file based on the video memory requirement and the graphics card located on the server, the video memory resources in the target graphics card on the target server that are greater than the current video memory requirement are allocated to the service to be deployed according to the configuration file. Since it is necessary to avoid the service to be deployed from running out of resources due to responding to new access requests, video memory resources greater than the current video memory requirement are pre-allocated to the service to be deployed.

[0074] Specifically, step S102 includes:

[0075] Based on the video memory resources required by each access request in the service to be deployed, determine the video memory requirements of the service to be deployed.

[0076] Based on the video memory requirements and the available video memory capacity of each graphics card in each server, the target server and the target graphics card in the target server are determined for the service to be deployed. Each server includes one or more graphics cards, and the target graphics card is the graphics card in the target server used to provide video memory resources for the service to be deployed. The services to be deployed corresponding to two different access requests can be assigned to the same target graphics card to avoid wasting video memory resources in the graphics card.

[0077] Determine the percentage of video memory required relative to the target graphics card's video memory capacity, and generate a configuration file. The video memory capacity includes both available and used video memory. The portion of the video memory already in use on the target graphics card is designated as the used video memory.

[0078] Different servers may have some differences. Some servers have a single graphics card, while others have multiple graphics cards. Due to differences in graphics card models, the amount of video memory will also vary, generally ranging from 12GB to 32GB. Each service to be deployed allocates video memory proportionally upon startup. For example, assuming a service to be deployed requires 3GB of video memory on a 12GB graphics card, the video memory requirement accounts for 0.25% of the target graphics card's video memory capacity. If the service to be deployed is on a 32GB graphics card, then the proportion of the target graphics card's video memory capacity is 0.1%. To avoid insufficient resources for the service to be deployed in responding to new access requests, it is necessary to pre-allocate video memory resources greater than the current video memory requirement for the service to be deployed.

[0079] Step S103: Allocate video memory for the service to be deployed according to the configuration file, and deploy the instance of the service to be deployed in the video memory. The video memory is located in the target graphics card of the target server. The target server is the server used to deploy the service to be deployed, and the target graphics card is the graphics card in the target server used to provide video memory resources for the service to be deployed.

[0080] By automatically allocating video memory from the target graphics card of the target server to the service to be deployed through the configuration file, manual operation is reduced and there is no need to manually consider whether the video memory resources of the target server are sufficient.

[0081] Specifically, step S103, which involves allocating video memory to the service to be deployed based on the configuration file, includes:

[0082] Based on the configuration file, determine the target server assigned to the service to be deployed and the target graphics card located on the target server;

[0083] Based on the proportion of GPU memory required to the target GPU's total GPU memory capacity, GPU container sharing technology is used to allocate GPU memory within the target GPU. Since GPU memory is required, this technology is used to partition the memory, ensuring each instance sees only the memory it needs. For example, using GPU container sharing technology, only 5GB of GPU memory can be seen in the containers where the service is deployed. When the memory requirement is 3GB, the proportion is 0.6. When the service is deployed to a server, Kubernetes automatically finds a server with more than 5GB of GPU memory for deployment. All instances can only see this 5GB of GPU memory and use 60% of it, i.e., 3GB of GPU memory resources. The GPU container sharing technology can be cGPU provided by Alibaba Cloud or qGPU provided by Tencent Cloud.

[0084] Allocate a portion of the video memory to the services to be deployed, so that the services to be deployed can use the video memory.

[0085] Step S104: Distribute the access request to the deployed instance and run the instance to deploy the service to the target server.

[0086] After the service to be deployed is deployed to the target server, the service to be deployed becomes a deployed service, realizing the automated deployment of services. It eliminates the need for manual deployment to the target server and solves the problem that the number of services to be deployed increases and it becomes impossible to deploy services manually.

[0087] In one optional embodiment of the present invention, the running example in step S104 includes:

[0088] Run instances within containers, where each container contains one instance;

[0089] Following the running instance in step S104, the service deployment method further includes:

[0090] Get the instance's running status, including whether it is running normally or abnormally;

[0091] If an instance is found to be malfunctioning, the number of instances deployed will be adjusted based on the services to be deployed. Each service to be deployed can have multiple instances deployed or started, and corresponding load balancing strategies can be established for automatic load scheduling.

[0092] When the number of services to be deployed is large, in order to facilitate deployment, the services to be deployed can be divided into groups, and multiple services to be deployed can be deployed to the server at one time. Alternatively, Kubernetes can be used to break down the services to be deployed into smaller granularities, deploy instances through the services to be deployed, and deploy different numbers of instances for different services to be deployed, thereby improving the utilization of memory resources, flexibly configuring the number of service instances, and controlling service throughput.

[0093] When instances are running abnormally and resource utilization is low, and the execution time of instances is long, the number of instances to be deployed can be increased for services corresponding to important types of access requests, thereby speeding up service deployment and improving the utilization of GPU memory resources. On the other hand, for services corresponding to unimportant types of access requests, the number of instances to be deployed can be reduced to avoid wasting GPU memory resources.

[0094] In a preferred embodiment of the present invention, the service deployment method further includes:

[0095] The scripts of the services to be tested are used to periodically perform health checks on the services themselves. The services to be tested include services to be deployed and services that have already been deployed. For both services to be deployed and services that have already been deployed, health checks can be added to them periodically. For example, a health check can be run every 5 minutes to determine whether the service to be tested is normal.

[0096] Determine whether the service to be tested has passed the health check;

[0097] If the service to be tested passes the health check, then the service to be tested is normal, and the tested service is obtained;

[0098] If the service to be tested fails the health check, the service to be tested is considered abnormal. The service to be tested will be taken offline and a new service to be tested will be started to maintain the number of requests per second for the entire service to be tested.

[0099] When a service fails a health check, it is automatically taken offline. A new service is then automatically restarted via Kubernetes to maintain the total number of requests per second for the entire service, ensuring the server can process requests normally and control the service throughput.

[0100] As can be seen from the above description, the present invention achieves the following technical effects:

[0101] This invention automatically allocates video memory from the target graphics card of the target server to the service to be deployed through a configuration file, reducing manual operation and eliminating the need for manual consideration of whether the video memory resources in the target server are sufficient.

[0102] It enables automated service deployment, eliminating the need for manual deployment to the target server and solving the problem of service deployment becoming impossible due to an increasing number of services to be deployed.

[0103] By flexibly deploying the number of service instances, service throughput can be controlled, and resource utilization can be improved.

[0104] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0105] This invention also provides a service deployment apparatus for implementing the above-described service deployment method, such as... Figure 3 As shown, the device includes:

[0106] Receiving unit 31 is used to receive user access requests and route the access requests to the service to be deployed via routing.

[0107] The generation unit 32 is used to determine the video memory requirements of the service to be deployed based on the access request, and to generate a configuration file based on the video memory requirements and the graphics card located on the server.

[0108] Allocation unit 33 is used to allocate video memory to the service to be deployed according to the configuration file, and deploy instances of the service to be deployed in the video memory. The video memory is located in the target graphics card of the target server, the target server is the server used to deploy the service to be deployed, and the target graphics card is the graphics card in the target server used to provide video memory resources for the service to be deployed.

[0109] Deployment unit 34 is used to allocate access requests to deployed instances and run the instances to deploy the service to the target server.

[0110] This invention also provides an electronic device, such as... Figure 4 As shown, the electronic device includes one or more processors 41 and a memory 42. Figure 4 Take a processor 41 as an example.

[0111] The controller may also include an input device 43 and an output device 44.

[0112] The processor 41, memory 42, input device 43, and output device 44 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0113] Processor 41 can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips. The general-purpose processor can be a microprocessor or any conventional processor.

[0114] The memory 42, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the control method in this embodiment of the invention. The processor 41 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 42, thereby implementing the service deployment method of the above method embodiment.

[0115] The memory 42 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the processing device operated by the server. Furthermore, the memory 42 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 42 may optionally include memory remotely located relative to the processor 41, and these remote memories can be connected to a network connection device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0116] Input device 43 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the server's processing device. Output device 44 may include display devices such as a display screen.

[0117] One or more modules are stored in memory 42, and when executed by one or more processors 41, they perform actions such as... Figure 1 The method shown.

[0118] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the motor control methods described above. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory (FM), hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0119] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A service deployment method, characterized in that, include: Receive user access requests and route the access requests to the service to be deployed via routing; The video memory requirements of the service to be deployed are determined based on the access request, and a configuration file is generated based on the video memory requirements and the graphics card located on the server. According to the configuration file, video memory is allocated to the service to be deployed, and an instance of the service to be deployed is deployed in the video memory. The video memory is located in the target graphics card of the target server, the target server is the server used to deploy the service to be deployed, and the target graphics card is the graphics card on the target server used to provide video memory resources for the service to be deployed. The access request is assigned to a deployed instance and the instance is run to deploy the service to the target server. Receiving user access requests includes: The access request is received through Kubernetes, wherein the Kubernetes system has the route configured, and the Kubernetes system receives the access request through the route. Specifically, routing the access request to the service to be deployed includes: The type of the access request is obtained, and the corresponding service to be deployed is determined based on the type of the access request. Access requests are categorized into different types according to their frequency of use, including accessing the homepage, logging in, data querying, data acquisition, launching a fundus camera, and capturing fundus images. Different types of access requests are mapped to different services to be deployed in Kubernetes. Access requests and services are associated through these mappings, and different services to be deployed are executed with different numbers of concurrent instances. A higher number of concurrent instances allows for the simultaneous processing of more access requests of the same type. Obtain the IP address and port of the service to be deployed. Using the IP address and port of the service to be deployed, the access request is scheduled to the service to be deployed via the routing. The access request is scheduled to the corresponding service to be deployed according to the frequency of different types of access requests. Multiple instances of the service to be deployed are deployed concurrently to handle multiple access requests of the same type at the same time.

2. The method according to claim 1, characterized in that, The step of determining the video memory requirements of the service to be deployed based on the access request, and generating a configuration file based on the video memory requirements and the graphics card located on the server, includes: Based on the video memory resources required by each access request in the service to be deployed, determine the video memory requirements of the service to be deployed. Based on the video memory requirements and the available video memory capacity of each graphics card in each server, the target server and the target graphics card in the target server are determined for the service to be deployed, wherein each server includes one or more graphics cards. Determine the proportion of the video memory requirement to the video memory capacity of the target graphics card, and generate a configuration file.

3. The method according to claim 1, characterized in that, The step of allocating video memory to the service to be deployed according to the configuration file includes: Based on the configuration file, the target server to be assigned to the service to be deployed and the target graphics card located on the target server are determined; According to the proportion of the video memory requirement to the video memory capacity of the target graphics card, the video memory in the target graphics card is divided using GPU container sharing technology; Allocate the video memory that is to be deployed to the service to be deployed, so that the service to be deployed can use the video memory.

4. The method according to claim 1, characterized in that, Before receiving a user's access request, the method further includes: The server is used as a platform to run containers, and multiple containers are created on each server using Kubernetes; Containerize the service to be deployed and start the service in a containerized manner.

5. The method according to claim 4, Its features are, in, Running the instance includes: Run instances within the containers, wherein each container contains one instance; After running the instance, the method further includes: Obtain the running status of the instance, which includes whether it is running normally or abnormally; as well as If the instance is running abnormally, the number of instances to be deployed is adjusted through the service to be deployed.

6. The method according to claim 1, characterized in that, The method further includes: A health check is periodically performed on the service to be tested using a script, wherein the service to be tested includes the service to be deployed and the service that has already been deployed; Determine whether the service to be tested passes the health check; If the service to be tested passes the health check, then the service to be tested is normal, and the tested service is obtained; If the service to be tested fails the health check, the service to be tested is considered abnormal. The service to be tested will be taken offline and a new service to be tested will be started to maintain the number of requests per second for the entire service to be tested.

7. A service deployment device, characterized in that, include: A receiving unit is used to receive user access requests and route the access requests to the service to be deployed via routing. The generation unit is used to determine the video memory requirements of the service to be deployed based on the access request, and to generate a configuration file based on the video memory requirements and the graphics card located on the server. An allocation unit is configured to allocate video memory to the service to be deployed according to the configuration file, and deploy an instance of the service to be deployed in the video memory, wherein the video memory is located in the target graphics card of the target server, the target server is a server used to deploy the service to be deployed, and the target graphics card is the graphics card in the target server used to provide video memory resources for the service to be deployed; and A deployment unit is used to allocate the access request to a deployed instance and run the instance to deploy the service to be deployed to the target server; Receiving user access requests includes: The access request is received through Kubernetes, wherein the Kubernetes system has the route configured, and the Kubernetes system receives the access request through the route. Specifically, routing the access request to the service to be deployed includes: The type of the access request is obtained, and the corresponding service to be deployed is determined based on the type of the access request. Access requests are categorized into different types according to their frequency of use, including accessing the homepage, logging in, data querying, data acquisition, launching a fundus camera, and capturing fundus images. Different types of access requests are mapped to different services to be deployed in Kubernetes. Access requests and services are associated through these mappings, and different services to be deployed are executed with different numbers of concurrent instances. A higher number of concurrent instances allows for the simultaneous processing of more access requests of the same type. Obtain the IP address and port of the service to be deployed. Using the IP address and port of the service to be deployed, the access request is scheduled to the service to be deployed via the routing. The access request is scheduled to the corresponding service to be deployed according to the frequency of different types of access requests. Multiple instances of the service to be deployed are deployed concurrently to handle multiple access requests of the same type at the same time.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the service deployment method according to any one of claims 1-6.

9. An electronic device, characterized in that, The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the at least one processor to perform the service deployment method according to any one of claims 1-6.

Images