Micro-service system, operation method, electronic device, storage medium, and program product

Abstract

The application relates to the technical field of computers, in particular to a micro-service system, a running method, an electronic device, a storage medium and a program product, wherein the system comprises: a Redis task queue configured to receive and store a task request to be processed; at least one micro-service module configured to schedule a corresponding AI model based on the task request; and at least one AI model corresponding to the at least one micro-service module and configured to execute the task, wherein the AI model is constructed based on a Pytorch or ONNX framework. Thus, the problems that high concurrency requirements cannot be met and the expansibility and flexibility are limited in the related art are solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a microservice system, operating method, electronic device, storage medium, and program product. Background Technology

[0002] With the development of cloud computing and internet technologies, microservice architecture has become a popular approach for building large-scale distributed systems. Microservice architecture breaks down a complex application into a set of small, independent services, each responsible for performing a specific business function. These services can be developed, tested, deployed, and scaled independently, greatly improving the system's flexibility and maintainability.

[0003] However, traditional microservice architectures also face some challenges, especially in scenarios involving high concurrency, large data volume processing, and complex task scheduling. They cannot meet high concurrency requirements and have limited scalability and flexibility. Summary of the Invention

[0004] This application provides a microservice system, operating method, electronic device, storage medium, and program product to solve the problems in related technologies that cannot meet high concurrency requirements and have limited scalability and flexibility.

[0005] The first aspect of this application provides a microservice system deployed in a Kubernetes cluster environment. The system includes: a Redis task queue for receiving and storing task requests to be processed; at least one microservice module for scheduling corresponding AI models based on task requests; and at least one AI model corresponding to the at least one microservice module for executing tasks. The AI ​​model is built based on a PyTorch or ONNX framework.

[0006] Optionally, in one embodiment of this application, a cluster model is configured on the Redis task queue to support concurrent processing of multiple tasks.

[0007] Optionally, in one embodiment of this application, the microservice system further includes a task management module, used to retrieve task requests from the Redis task queue and assign them to the corresponding microservice module.

[0008] Optionally, in one embodiment of this application, the microservice system further includes a monitoring system for monitoring the operating status and performance metrics of the microservice system.

[0009] Optionally, in one embodiment of this application, the microservice system supports asynchronous processing of multiple task requests.

[0010] Optionally, in one embodiment of this application, the microservice module, task management module, and monitoring system are all built based on the Flask application framework.

[0011] A second aspect of this application provides a method for operating a microservice system, comprising: receiving a task request submitted by a user and storing it in a Redis task queue; retrieving the task request from the Redis task queue and assigning it to a corresponding microservice module, wherein the microservice module schedules a corresponding AI model, the AI ​​model executes the task and outputs the execution result; and feeding back the execution result to the user.

[0012] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to perform a method for operating a microservice architecture as described in the above embodiments.

[0013] A fourth aspect of this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which are executed by a processor to perform a method for operating a microservice system as described in the above embodiments.

[0014] The fifth aspect of this application provides a computer program product, including a computer program or instructions, which, when executed, implement a method for operating a microservice system as described in the above embodiments.

[0015] Therefore, this application has at least the following beneficial effects:

[0016] This application's embodiments can build a microservice system based on Kubernetes and Redis. The microservice system executes user task requests, leveraging Kubernetes' powerful features such as auto-scaling, high availability, and load balancing to ensure stability and reliability. Using Redis as a task queue improves the system's high-concurrency performance and ensures no task loss. Each microservice module handles a specific type of task and schedules the corresponding AI model, enabling the system to flexibly handle multiple task types. Furthermore, AI models built on PyTorch or ONNX frameworks can perform various inference tasks. Therefore, this solves the technical problems of related technologies, such as the inability to meet high-concurrency requirements and limited scalability and flexibility.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0019] Figure 1 This is a schematic diagram of a microservice system provided according to an embodiment of this application;

[0020] Figure 2 This is a schematic diagram of the structure of a microservice system provided according to an embodiment of this application;

[0021] Figure 3 This is a flowchart of the operation method of the microservice system provided according to the embodiments of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0024] The following description, with reference to the accompanying drawings, outlines a microservice system, operating method, electronic device, storage medium, and program product according to embodiments of this application. Addressing the challenges mentioned in the background section, traditional microservice architectures face, particularly in scenarios involving high concurrency, large-scale data processing, and complex task scheduling. They often fail to meet high concurrency demands and suffer from limited scalability and flexibility. This application provides a microservice system built upon Kubernetes and Redis. This system executes user task requests, leveraging the powerful capabilities of Kubernetes to ensure stability and reliability. Using Redis as a task queue improves high-concurrency performance and ensures no task loss. Each microservice module handles specific task types and schedules corresponding AI models, enabling the system to flexibly handle various task types. Furthermore, AI models built on PyTorch or ONNX frameworks can perform various inference tasks. This solves the problems of insufficient high concurrency, limited scalability, and limited flexibility in related technologies.

[0025] Specifically, Figure 1 This is a schematic diagram of a microservice system provided in an embodiment of this application.

[0026] like Figure 1 As shown, the microservice system 10 is deployed in a Kubernetes cluster environment and includes: a Redis task queue 11, a microservice module 12, and an AI model 13.

[0027] The Redis task queue 11 is used to receive and store task requests to be processed; there is at least one microservice module 12, which is used to schedule the corresponding AI model 13 based on the task request; the AI ​​model 13 corresponds one-to-one with the microservice model 12 and is used to execute tasks, wherein the AI ​​model is built based on the PyTorch or ONNX framework.

[0028] It is understood that the embodiments of this application can build a microservice system based on Kubernetes and Redis, and execute user task requests based on the microservice system. Specifically, the microservice module schedules the corresponding AI model based on the task request in the Redis task queue, and the AI ​​model executes the corresponding task. By deploying in a Kubernetes cluster environment, the microservice system 10 can take advantage of the powerful functions of Kubernetes, such as automatic scaling, high availability, and load balancing, to ensure stability and reliability. Using Redis as a task queue can improve the high concurrency performance of the system and ensure that tasks are not lost. Each microservice module is responsible for handling specific types of tasks and scheduling the corresponding AI model. This allows the system to flexibly handle multiple task types, and the AI ​​model built based on the PyTorch or ONNX framework can execute various inference tasks.

[0029] In this embodiment of the application, a cluster model is configured on the Redis task queue 12 to support concurrent processing of multiple tasks.

[0030] It is understood that the Redis task queue 12 in this embodiment of the application is configured with a cluster model, which can support the simultaneous processing of multiple tasks, thereby improving the system throughput and response speed. Furthermore, the cluster model can provide better high availability, ensuring that the task queue can still operate normally even if some nodes fail.

[0031] In this embodiment of the application, the microservice system 10 further includes a task management module.

[0032] The task management module is used to retrieve task requests from the Redis task queue and assign them to the corresponding microservice modules.

[0033] It is understood that the task management module in this embodiment of the application is responsible for retrieving task requests from the Redis task queue and assigning them to appropriate microservice modules for processing, so as to ensure that tasks can be efficiently scheduled and executed.

[0034] In this embodiment of the application, the microservice system 10 further includes a monitoring system.

[0035] The monitoring system is used to monitor the running status and performance indicators of the microservice system.

[0036] It is understood that the monitoring system in this application embodiment can monitor the running status and performance indicators of the entire microservice system in real time, helping operation and maintenance personnel to discover and solve problems in a timely manner, and improve the stability and availability of the system.

[0037] In this embodiment, the microservice system 10 supports asynchronous processing of multiple task requests.

[0038] It is understood that the embodiments of this application support asynchronous processing of multiple task requests, which can improve the system's concurrent processing capability and enable the system to better cope with high load conditions.

[0039] In this embodiment, the microservice module, task management module, and monitoring system are all built on the Flask application framework.

[0040] It is understood that the embodiments of this application can use the Flask framework to build microservice modules, task management modules and monitoring systems, which can simplify the development process and improve the development efficiency and maintenance convenience of the system by leveraging the flexibility and scalability of the Flask framework.

[0041] The implementation process of the microservice system of this application is illustrated below through a specific embodiment. The technical conditions of the system include Kubernetes, Redis, Flask, PyTorch or ONNX, and it requires a GPU server cluster device to run the Kubernetes cluster, Redis service and Flask application.

[0042] like Figure 2 As shown, the specific components include: database-1, heterogeneous resources-2, Kubernetes cluster environment-3, Flask application-4, and AI model architecture-5. The implementation steps are as follows:

[0043] 1. Deploy a microservice architecture based on Kubernetes.

[0044] (1) Deploy infrastructure such as Pods and Services in a Kubernetes cluster to support the operation and communication of microservices;

[0045] (2) Deploy containerized versions of Flask applications to the Kubernetes cluster to ensure high availability and automated management of the system.

[0046] 2. Deploy the task queue service (Redis).

[0047] (1) Deploy Redis service in Kubernetes cluster as a task queue service;

[0048] (2) Configure and optimize the Redis cluster to support concurrent processing of large-scale tasks.

[0049] 3. Develop a task scheduling and management module (based on Flask).

[0050] (1) Develop a task scheduling and management module using the Flask framework;

[0051] (2) Create a task queue interface to receive and process task requests;

[0052] (3) Write task scheduling logic, including retrieving task requests from the Redis task queue and allocating them to available microservice instances;

[0053] 4. Create microservice modules (based on Flask).

[0054] (1) Develop multiple microservice modules using the Flask framework, with each module responsible for handling different types of tasks;

[0055] (2) Implement task processing logic in each microservice module based on AI model frameworks such as PyTorch / ONNX;

[0056] (3) Provide a RESTful API interface to receive task requests and return processing results.

[0057] 5. Write the task processing logic (based on Flask).

[0058] (1) Write task processing logic in each microservice module and call the corresponding processing algorithm based on AI model frameworks such as PyTorch / ONNX according to the task type.

[0059] (2) Use Flask’s asynchronous processing method to ensure that the system can handle multiple task requests at the same time.

[0060] 6. Implement a task concurrency control mechanism (based on Flask).

[0061] (1) Design a concurrency control mechanism to limit the concurrent processing capacity of each microservice module in order to avoid system overload;

[0062] (2) Use Flask’s concurrency control features or custom concurrency management logic to control the number of concurrent tasks.

[0063] 7. Monitoring and optimization (based on Flask).

[0064] (1) Integrate the monitoring and logging system into the Flask application to monitor the system's operating status and performance in real time;

[0065] (2) Optimize and adjust the system based on monitoring data to improve the system's stability and performance.

[0066] In summary, compared to related technologies, this application uses Redis as a task queue and the Flask framework as a microservice development tool, providing a more efficient solution for high concurrency performance, task management, and data processing. Using Redis as a task queue enables more efficient task scheduling and management, ensuring the system can handle large-scale concurrent tasks. The use of the Flask framework simplifies and speeds up microservice development and deployment, reducing system maintenance costs and development complexity. This application includes various model quality inspection functions and automatic annotation / labeling functions, further enriching the system's functionality and application scenarios, enabling it to better meet user needs. The use of the Kubernetes, Redis, and Flask frameworks provides greater scalability and flexibility, allowing for customization and expansion according to different needs and scenarios, thus better adapting to constantly changing business environments.

[0067] This embodiment can provide a high-concurrency AI model inference microservice architecture, support multiple model quality inspection functions, and implement automatic annotation and labeling functions to improve data processing efficiency, enrich the data labeling system, enhance system stability and scalability, and reduce maintenance costs.

[0068] The microservice system proposed in the embodiments of this application can be built on Kubernetes and Redis. The microservice system executes user task requests and can leverage the powerful features of Kubernetes, such as automatic scaling, high availability, and load balancing, to ensure stability and reliability. Using Redis as a task queue can improve the system's high concurrency performance and ensure that tasks are not lost. Each microservice module is responsible for handling specific types of tasks and scheduling corresponding AI models, which enables the system to flexibly handle multiple task types. Furthermore, AI models built on PyTorch or ONNX frameworks can perform various inference tasks.

[0069] Next, the operation method of the microservice system proposed according to the embodiments of this application is described with reference to the accompanying drawings.

[0070] Figure 3 This is a flowchart of the operation method of the microservice system according to an embodiment of this application.

[0071] like Figure 3 As shown, the operation method of this microservice system includes the following steps:

[0072] In step S101, the task request submitted by the user is received and stored in the Redis task queue.

[0073] It is understood that the embodiments of this application can receive task requests submitted by users and store them in a Redis queue for subsequent processing of task requests in the Redis task queue.

[0074] In step S102, a task request is retrieved from the Redis task queue and assigned to the corresponding microservice module. The microservice module schedules the corresponding AI model, and the AI ​​model executes the task and outputs the execution result.

[0075] In this embodiment, the corresponding microservice module can be determined based on the task type contained in the task request.

[0076] It is understood that, in this embodiment of the application, task requests can be retrieved from the Redis task queue and assigned to the corresponding microservice module. The microservice module is used to schedule the corresponding AI model based on the task type. The AI ​​model executes the task and outputs the execution result.

[0077] In step S103, the execution result is fed back to the user.

[0078] In summary, the specific steps of the microservice system operation method according to the embodiments of this application are as follows:

[0079] 1. Set up a Kubernetes cluster environment, deploy the Redis service, and load the PyTorch model and Flask application;

[0080] 2. Receive task requests or query requests submitted by users and add them to the task queue;

[0081] 3. Retrieve task requests from the queue, call the corresponding model for inference, execute QA or auto-label / auto-tag functions, and generate results;

[0082] 4. Feedback the results to the user and continue listening on the specified port.

[0083] It should be noted that the foregoing explanation of the microservice system embodiment also applies to the operation method of the microservice system in this embodiment, and will not be repeated here.

[0084] According to the microservice system operation method proposed in the embodiments of this application, after receiving a user's task request, the task request can be stored in a Redis task queue, and the task request can be retrieved from the Redis task queue to determine its corresponding microservice module. The task request can be processed based on the microservice module, and multiple modules can process tasks simultaneously, which improves the efficiency of task processing.

[0085] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0086] The memory 401, the processor 402, and the computer program stored on the memory 401 and capable of running on the processor 402.

[0087] When processor 402 executes the program, it implements the microservice system operation method provided in the above embodiments.

[0088] Furthermore, electronic devices also include:

[0089] Communication interface 403 is used for communication between memory 401 and processor 402.

[0090] The memory 401 is used to store computer programs that can run on the processor 402.

[0091] The memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0092] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0093] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0094] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0095] This application also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the above-described microservice system operation method.

[0096] This application also provides a computer program product, including a computer program or instructions, which, when executed, implement the above-described method for operating a microservice system.

[0097] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0098] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0099] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0100] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0101] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A microservice system, characterized in that, The micro-service system is deployed in a Kubernetes cluster environment, wherein the system comprises: a Redis task queue configured to receive and store task requests to be processed; at least one micro-service module configured to schedule a corresponding AI model based on the task requests; at least one AI model corresponding to the at least one micro-service module, configured to execute the task, wherein the AI model is built based on a Pytorch or ONNX framework.

2. The microservice system of claim 1, wherein, The Redis task queue is configured with a cluster model to support concurrent processing of multiple tasks.

3. The microservice system of claim 1, wherein, The micro-service system further comprises a task management module configured to take out task requests from the Redis task queue and assign them to corresponding micro-service modules.

4. The microservice system of claim 3, wherein, The micro-service system further comprises a monitoring system configured to monitor the running state and performance indicators of the micro-service system.

5. The microservice system of claim 3, wherein, The micro-service system supports asynchronous processing of multiple task requests.

6. The microservice system of claim 4, wherein, The micro-service module, the task management module, and the monitoring system are all built based on a Flask application framework.

7. A method of operating a microservice system, characterized in that, The method comprises the following steps: receiving task requests submitted by users and storing them in a Redis task queue; taking out task requests from the Redis task queue and assigning them to corresponding micro-service modules, wherein the micro-service modules schedule corresponding AI models, the AI models execute the tasks and output execution results; feeding back the execution results to the users.

8. An electronic device, comprising: The computer program or instructions are executed by a processor to implement the running method of the micro-service system as claimed in claim 7. The computer program or instructions are executed by a processor to implement the running method of the micro-service system as claimed in claim 7.

9. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed to implement the running method of the micro-service system as claimed in claim 7.

10. A computer program product comprising computer programs or instructions, characterized in that, ​

Images