Micro-service system, operation method, electronic device, storage medium and program product

By deploying microservice systems in Kubernetes clusters, using Redis task queues and Flask frameworks, combined with AI models of Pytorch or ONNX frameworks, the scalability and flexibility of traditional microservice architectures in high concurrency scenarios are solved, and efficient Task processing and inference task execution.

CN119065785AActive Publication Date: 2024-12-03CHERY AUTOMOBILE CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411051709.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-12-03
Estimated Expiration
2044-08-01

AI Technical Summary

Technical Problem

Traditional microservice architectures cannot meet the high concurrency needs in high concurrency, large data processing and complex task scheduling scenarios, and their scalability and flexibility are limited.

Method used

Deploy microservice systems in Kubernetes cluster environments, use Redis task queues and AI models built based on Pytorch or ONNX frameworks, and develop task management and monitoring systems in combination with the Flask framework to achieve high concurrent task processing and flexible task scheduling.

Benefits of technology

Through Kubernetes' automatic scaling, high availability and load balancing functions, ensuring system stability and reliability, Redis task queues improve high concurrency performance, and the Flask framework simplifies development and maintenance, achieving flexible processing and efficient inference task execution of multiple task types.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065785A_ABST
    Figure CN119065785A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computers, in particular to a micro-service system, an operation method, electronic equipment, a storage medium and a program product, and the system comprises a Redis task queue which is used for receiving and storing a to-be-processed task request; the at least one micro-service module is used for scheduling a corresponding AI model based on the task request; and the at least one AI model corresponding to the at least one micro-service module is used for executing a task, and the AI model is constructed based on a Pytorch or ONNX framework. Therefore, the problems that the high concurrency requirement cannot be met, and the expansibility and the flexibility are limited in the prior art are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a microservice system, an operation method, an electronic device, a storage medium, and a program product. Background Art

[0002] With the development of cloud computing and Internet technology, microservice architecture has become a popular approach to building large distributed systems. Microservice architecture decomposes a complex application into a set of small, independent services, each of which is responsible for performing a specific business function. These services can be independently developed, tested, deployed, and expanded, greatly improving the flexibility and maintainability of the system.

[0003] However, traditional microservice architecture also faces some challenges, especially in scenarios of high concurrency, large data volume processing and complex task scheduling. It cannot meet high concurrency requirements and has limited scalability and flexibility. Summary of the invention

[0004] The present application provides a microservice system, an operation method, an electronic device, a storage medium, and a program product to solve the problems in related technologies that high concurrency requirements cannot be met and that scalability and flexibility are limited.

[0005] The first aspect of the present application provides a microservice system, which is deployed in a Kubernetes cluster environment, wherein the system includes: a Redis task queue for receiving and storing task requests to be processed; at least one microservice module for scheduling corresponding AI models based on task requests; and at least one AI model corresponding to at least one microservice module for executing tasks, wherein the AI ​​model is built based on a Pytorch or ONNX framework.

[0006] Optionally, in one embodiment of the present application, a cluster model is configured on the Redis task queue to support concurrent processing of multiple tasks.

[0007] Optionally, in one embodiment of the present application, the microservice system further includes: a task management module, configured to take out task requests from the Redis task queue and assign them to corresponding microservice modules.

[0008] Optionally, in one embodiment of the present application, the microservice system further includes: a monitoring system for monitoring the operating status and performance indicators of the microservice system.

[0009] Optionally, in one embodiment of the present application, the microservice system supports asynchronous processing of multiple task requests.

[0010] Optionally, in one embodiment of the present application, the microservice module, the task management module and the monitoring system are all built based on the Flask application framework.

[0011] The second aspect of the present application provides an operation method of a microservice system, including: receiving a task request submitted by a user and storing it in a Redis task queue; taking out the task request from the Redis task queue and assigning it to a corresponding microservice module, wherein the microservice module schedules the corresponding AI model, the AI ​​model executes the task and outputs the execution result; and feeding back the execution result to the user.

[0012] The third aspect of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to execute an operating method of a microservice architecture as in the above-mentioned embodiment.

[0013] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. The computer program or instruction is executed by a processor to execute the operating method of the microservice system as described in the above embodiment.

[0014] The fifth aspect of the present application provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed, a method for operating a microservice system as in the above embodiment is implemented.

[0015] Therefore, this application has at least the following beneficial effects:

[0016] The embodiment of the present application can build a microservice system based on Kubernetes and Redis, and execute user task requests based on the microservice system. It can use the powerful functions of Kubernetes, such as automatic expansion, high availability, load balancing, etc., to ensure stability and reliability. Redis is used as a task queue to improve the high concurrency performance of the system and ensure that tasks will not be lost. Each microservice module is responsible for processing a specific type of task and scheduling the corresponding AI model, which can enable the system to flexibly handle multiple task types, and the AI ​​model built based on the Pytorch or ONNX framework can perform various reasoning tasks. As a result, technical problems such as the inability to meet high concurrency requirements and limited scalability and flexibility in related technologies are solved.

[0017] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A schematic diagram of a microservice system provided according to an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the structure of a microservice system provided according to an embodiment of the present application;

[0021] Figure 3 A flowchart of a method for operating a microservice system according to an embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0024] The following describes the microservice system, operation method, electronic device, storage medium and program product of the embodiment of the present application with reference to the accompanying drawings. In view of the above-mentioned background technology center, the traditional microservice architecture also faces some challenges, especially in the scenarios of high concurrency, large data volume processing and complex task scheduling, it cannot meet the high concurrency requirements, and the scalability and flexibility are limited. The present application provides a microservice system, in which a microservice system can be built based on Kubernetes and Redis, and the user's task request is executed based on the microservice system. The powerful functions of Kubernetes can be used to ensure stability and reliability. Redis is used as a task queue to improve the high concurrency performance of the system and ensure that tasks will not be lost. Each microservice module is responsible for processing a specific type of task and scheduling the corresponding AI model, which can enable the system to flexibly handle multiple task types, and the AI ​​model built based on the Pytorch or ONNX framework can perform various reasoning tasks. As a result, the problems of being unable to meet high concurrency requirements and limited scalability and flexibility in the related technology are solved.

[0025] Specifically, Figure 1 A schematic diagram of a microservice system provided in an embodiment of the present application.

[0026] like Figure 1 As shown, the microservice system 10 is deployed in a Kubernetes cluster environment, including: a Redis task queue 11, a microservice module 12 and an AI model 13.

[0027] Among them, the Redis task queue 11 is used to receive and store task requests to be processed; there is at least one microservice module 12, which is used to schedule the corresponding AI model 13 based on the task request; the AI ​​model 13 corresponds one-to-one with the microservice model 12, and is used to execute tasks, wherein the AI ​​model is built based on the Pytorch or ONNX framework.

[0028] It can be understood that the embodiments of the present application can build a microservice system based on Kubernetes and Redis, and execute the user's task request based on the microservice system. Specifically, the microservice module schedules the corresponding AI model based on the task request in the Redis task queue, and the AI ​​model executes the corresponding task. By being deployed in a Kubernetes cluster environment, the microservice system 10 can utilize the powerful functions of Kubernetes, such as automatic expansion, high availability, load balancing, etc., to ensure stability and reliability. Using Redis as a task queue can improve the high concurrency performance of the system and ensure that tasks will not be lost. Each microservice module is responsible for processing a specific type of task and scheduling the corresponding AI model, which enables the system to flexibly handle multiple task types, and the AI ​​model built based on the Pytorch or ONNX framework can perform various reasoning tasks.

[0029] In an embodiment of the present application, a cluster model is configured on the Redis task queue 12 to support concurrent processing of multiple tasks.

[0030] It can be understood that the Redis task queue 12 of the embodiment of the present application is configured with a cluster model, which can support the simultaneous processing of multiple tasks, thereby improving the throughput and response speed of the system, and the cluster model can provide better high availability, ensuring that the task queue can still operate normally even if some nodes fail.

[0031] In the embodiment of the present application, the microservice system 10 further includes: a task management module.

[0032] Among them, the task management module is used to take out the task request from the Redis task queue and assign it to the corresponding microservice module.

[0033] It can be understood that the task management module of the embodiment of the present application is responsible for taking out the task request from the Redis task queue and assigning it to the appropriate microservice module for processing, ensuring that the task can be efficiently scheduled and executed.

[0034] In the embodiment of the present application, the microservice system 10 also includes: a monitoring system.

[0035] Among them, the monitoring system is used to monitor the operating status and performance indicators of the microservice system.

[0036] It can be understood that the monitoring system of the embodiment of the present application can monitor the operating status and performance indicators of the entire microservice system in real time, help operation and maintenance personnel to promptly discover and solve problems, and improve the stability and availability of the system.

[0037] In an embodiment of the present application, the microservice system 10 supports asynchronous processing of multiple task requests.

[0038] It can be understood that the embodiments of the present application support asynchronous processing of multiple task requests, which can improve the concurrent processing capability of the system and enable the system to better cope with high load conditions.

[0039] In the embodiment of the present application, the microservice module, the task management module and the monitoring system are all built based on the Flask application framework.

[0040] It can be understood that the embodiments of the present application can use the Flask framework to build microservice modules, task management modules and monitoring systems, which can simplify the development process and utilize the flexibility and scalability of the Flask framework to improve the development efficiency and maintenance convenience of the system.

[0041] The following is an example of a specific implementation of the microservice system of the present application. The technical requirements of the system include Kubernetes, Redis, Flask, Pytorch or ONNX, etc., and a GPU server cluster device is required to run the Kubernetes cluster, Redis service and Flask application.

[0042] like Figure 2 As shown, it specifically includes: database-1, heterogeneous resources-2, Kubernetes cluster environment-3, Flask application-4 and AI model architecture-5. The implementation steps are as follows:

[0043] 1. Deploy a microservice architecture based on Kubernetes.

[0044] (1) Deploy infrastructure such as Pods and Services in the Kubernetes cluster to support the operation and communication of microservices;

[0045] (2) Deploy the containerized version of the Flask application to the Kubernetes cluster to ensure high availability and automated management of the system.

[0046] 2. Deploy the task queue service (Redis).

[0047] (1) Deploy the Redis service in the Kubernetes cluster as a task queue service;

[0048] (2) Configure and optimize the Redis cluster to support concurrent processing of large-scale tasks.

[0049] 3. Develop task scheduling and management modules (based on Flask).

[0050] (1) Use the Flask framework to develop task scheduling and management modules;

[0051] (2) Create a task queue interface to receive and process task requests;

[0052] (3) Write task scheduling logic, including taking task requests from the Redis task queue and assigning them to available microservice instances;

[0053] 4. Create a microservice module (based on Flask).

[0054] (1) Use the Flask framework to develop multiple microservice modules, each of which is responsible for different types of task processing;

[0055] (2) Implement task processing logic in each microservice module based on AI model frameworks such as pytorch / ONNX;

[0056] (3) Provide a RESTful API interface for receiving task requests and returning processing results.

[0057] 5. Write task processing logic (based on Flask).

[0058] (1) Write task processing logic in each microservice module and call the corresponding processing algorithm based on AI model frameworks such as pytorch / ONNX according to the task type.

[0059] (2) Use Flask's asynchronous processing method to ensure that the system can handle multiple task requests simultaneously.

[0060] 6. Implement task concurrency control mechanism (based on Flask).

[0061] (1) Design a concurrency control mechanism to limit the concurrent processing capacity of each microservice module to avoid system overload;

[0062] (2) Use Flask's concurrency control function or custom concurrency management logic to control the number of concurrent processing tasks.

[0063] 7. Monitoring and optimization (based on Flask).

[0064] (1) Integrate monitoring and logging systems into Flask applications to monitor system operation status and performance indicators in real time;

[0065] (2) Optimize and adjust the system based on monitoring data to improve system stability and performance.

[0066] In summary, compared with related technologies, this application uses Redis as a task queue and Flask framework as a microservice development tool, which provides a more efficient solution for the system's high concurrency performance, task management and data processing; this application uses Redis as a task queue, which can achieve more efficient task scheduling and management, ensuring that the system can handle large-scale concurrent tasks. The use of the Flask framework makes the development and deployment of microservices easier and faster, reducing the system's maintenance costs and development difficulty; this application involves a variety of model quality inspection functions and automatic annotation and labeling functions, further enriching the system's functions and application scenarios, so that the system can better meet user needs; this application uses k8s, Redis, and Flask frameworks, which have stronger scalability and flexibility, and can be customized and expanded according to different needs and scenarios, so as to better adapt to the ever-changing business environment.

[0067] This embodiment can provide a high-concurrency AI model reasoning microservice architecture, support multiple model quality inspection functions; and realize automatic annotation and labeling functions, improve data processing efficiency, enrich the data labeling system; improve system stability and scalability, and reduce maintenance costs.

[0068] According to the microservice system proposed in the embodiment of the present application, a microservice system can be built based on Kubernetes and Redis. The user's task requests can be executed based on the microservice system. The powerful functions of Kubernetes, such as automatic expansion, high availability, load balancing, etc., can be utilized to ensure stability and reliability. Using Redis as a task queue can improve the high concurrency performance of the system and ensure that tasks will not be lost. Each microservice module is responsible for processing a specific type of task and scheduling the corresponding AI model, which enables the system to flexibly handle multiple task types, and the AI ​​model built based on the Pytorch or ONNX framework can perform various reasoning tasks.

[0069] Next, the operation method of the microservice system proposed in accordance with the embodiment of the present application is described with reference to the accompanying drawings.

[0070] Figure 3 It is a flowchart of the operation method of the microservice system of an embodiment of the present application.

[0071] like Figure 3 As shown, the operation method of the microservice system includes the following steps:

[0072] In step S101, a task request submitted by a user is received and stored in a Redis task queue.

[0073] It is understandable that the embodiment of the present application can receive task requests submitted by users and store them in a Redis queue so as to subsequently process the task requests in the Redis task queue.

[0074] In step S102, the task request is taken out from the Redis task queue and assigned to the corresponding microservice module, wherein the microservice module schedules the corresponding AI model, and the AI ​​model executes the task and outputs the execution result.

[0075] Among them, the embodiment of the present application can determine the corresponding microservice module based on the task type included in the task request.

[0076] It can be understood that the embodiment of the present application can take out the task request from the Redis task queue and assign it to the corresponding microservice module. The microservice module is used to schedule the corresponding AI model based on the task type. The AI ​​model executes the task and outputs the execution result.

[0077] In step S103, the execution result is fed back to the user.

[0078] In summary, the specific steps of the operation method of the microservice system in the embodiment of the present application are as follows:

[0079] 1. Build a Kubernetes cluster environment, deploy Redis service, load pytorch model and Flask application;

[0080] 2. Receive task requests or query requests submitted by users and add them to the task queue;

[0081] 3. Take the task request from the queue, call the corresponding model for reasoning, perform QA or auto-label / auto-tag functions, and generate results;

[0082] 4. Feedback the results to the user and continue to monitor the specified port.

[0083] It should be noted that the aforementioned explanation of the microservice system embodiment is also applicable to the operation method of the microservice system of this embodiment, which will not be repeated here.

[0084] According to the operation method of the microservice system proposed in the embodiment of the present application, after receiving the user's task request, the task request can be stored in the Redis task queue, and the task request can be taken out from the Redis task queue to determine its corresponding microservice module. The task request is processed based on the microservice module, and multiple modules can process tasks simultaneously, thereby improving the efficiency of task processing.

[0085] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0086] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0087] When the processor 402 executes the program, the operating method of the microservice system provided in the above embodiment is implemented.

[0088] Furthermore, the electronic device further comprises:

[0089] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0090] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0091] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0092] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0093] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0094] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0095] An embodiment of the present application also provides a computer-readable storage medium having a computer program or instructions stored thereon, and when the computer program or instructions are executed by a processor, the operating method of the microservice system as described above is implemented.

[0096] An embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed, implements the above-mentioned method for operating the microservice system.

[0097] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0098] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0099] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0100] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0101] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

Claims

1. A microservice system, characterized in that: The microservice architecture is deployed in a Kubernetes cluster environment, wherein the system includes: Redis task queue, used to receive and store pending task requests; At least one microservice module, configured to schedule a corresponding AI model based on the task request; At least one AI model corresponding to the at least one microservice module is used to perform the task, wherein the AI ​​model is built based on a Pytorch or ONNX framework.

2. The microservice system according to claim 1, characterized in that: The Redis task queue is configured with a cluster model to support concurrent processing of multiple tasks.

3. The microservice system according to claim 1, characterized in that: The microservice system also includes: a task management module, which is used to take out task requests from the Redis task queue and allocate them to corresponding microservice modules.

4. The microservice system according to claim 1, characterized in that: The microservice system also includes: a monitoring system for monitoring the operating status and performance indicators of the microservice system.

5. The microservice system according to claim 1, characterized in that: The microservice system supports asynchronous processing of multiple task requests.

6. The microservice system according to claim 4, characterized in that: The microservice module, the task management module and the monitoring system are all built based on the Flask application framework.

7. A method for operating a microservice system, characterized in that: The following steps are involved: Receive task requests submitted by users and store them in the Redis task queue; Taking out the task request from the Redis task queue and assigning it to the corresponding microservice module, wherein the microservice module schedules the corresponding AI model, and the AI ​​model executes the task and outputs the execution result; The execution result is fed back to the user.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to execute the operating method of the microservice system as claimed in claim 7.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: The computer program or instruction is executed by a processor to implement the operating method of the microservice system as claimed in claim 7.

10. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed, the operating method of the microservice system as claimed in claim 7 is implemented.

Citation Information

Patent Citations

  • Task dispatching system with distributed calculating frame

    CN105760221A

  • AI reasoning method and system, electronic equipment, readable storage medium and product

    CN113835828A

  • Generation of cloud service inventory

    US11579937B1

  • Machine generated automation code for software development and infrastructure operations

    US20200387357A1

  • System and method for benchmarking a container orchestration platform

    US20210303365A1