AI service request processing methods, devices, electronic equipment, and storage media

Abstract

This disclosure relates to the field of artificial intelligence technology, particularly to the field of container technology, specifically to a method, apparatus, electronic device, and storage medium for processing AI service requests. The specific implementation involves: acquiring an AI service request; acquiring the status of the AI ​​service process related to the AI ​​service request; the status of the AI ​​service process includes: a working state or a recycled state; in response to the AI ​​service process being in a working state, processing the AI ​​service request through the AI ​​service process; in response to the AI ​​service process being in a recycled state, adjusting the status of the AI ​​service process from the recycled state to the working state, and then processing the AI ​​service request through the AI ​​service process. This disclosure can recycle AI service processes that are not frequently called, and AI service processes can be quickly awakened when they need to process corresponding AI service requests, avoiding the AI ​​service processes occupying a large amount of resources, thereby improving platform resource utilization and service availability at startup.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of container technology, specifically to a method, apparatus, electronic device, and storage medium for processing AI (Artificial Intelligence) service requests. Background Technology

[0002] With the development of artificial intelligence (AI) technology, AI has greatly improved the convenience of people's lives and promoted rapid social development by empowering various industries. The deployment of AI services, as the final and most crucial link in the entire AI workflow, presents significant challenges to AI inference development platforms due to the massive deployment of AI models and online inference services. On the one hand, AI models, especially deep learning models, often have complex network structures and numerous parameters. A single service deployment instance requires a large amount of memory, typically in the gigabyte range. AI development platforms need to support tens of thousands of model services, thus requiring massive memory resources. However, many model services remain inactive for extended periods, resulting in significant resource consumption, low resource utilization, and high machine costs. On the other hand, since AI services require loading models to start, startup times are long, leading to data loss and impacting service availability. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, and storage medium for processing AI service requests.

[0004] According to a first aspect of this disclosure, a method for processing AI service requests is provided, comprising:

[0005] Request AI services;

[0006] Obtain the status of the AI ​​service process related to the AI ​​service request; the status of the AI ​​service process includes: working status or recycling status.

[0007] In response to the AI ​​service process being in a working state, the AI ​​service request is processed through the AI ​​service process;

[0008] In response to the AI ​​service process being in a recycled state, the state of the AI ​​service process is adjusted from the recycled state to the working state, and then the AI ​​service request is processed through the AI ​​service process.

[0009] According to a second aspect of this disclosure, an apparatus for processing AI service requests is provided, comprising:

[0010] The AI ​​service proxy module is configured to obtain AI service requests;

[0011] The monitoring module is configured to acquire the status of the AI ​​service process related to the AI ​​service request; the status of the AI ​​service process includes: working status or recycling status;

[0012] The AI ​​service module is configured to be in a working state in response to the status of the AI ​​service process, and to process the AI ​​service request through the AI ​​service process;

[0013] The AI ​​service proxy module is configured to respond to the AI ​​service process being in a recycled state. After adjusting the state of the AI ​​service process from the recycled state to the working state, the AI ​​service module processes the AI ​​service request through the AI ​​service process.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the AI ​​service request processing method described in any of the above technical solutions.

[0015] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform a processing method for an AI service request according to any one of the above-described technical solutions.

[0016] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements a method for processing AI service requests according to any one of the above-described technical solutions.

[0017] This disclosure addresses the unique characteristics of AI services by adaptively adjusting AI service processes within a container. This allows for the reclamation of infrequently called AI service processes, while ensuring that AI service processes can be quickly activated when needed to handle related AI service requests. This approach avoids AI service processes consuming excessive resources, thereby improving platform resource utilization and enhancing service availability upon startup.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0020] Figure 1This is a schematic diagram illustrating the steps of the AI ​​service request processing method in the embodiments of this disclosure;

[0021] Figure 2 This is a schematic diagram illustrating the steps of the AI ​​service recycling method in this embodiment of the disclosure;

[0022] Figure 3 This is a schematic block diagram of the first type of AI service request processing device in the embodiments of this disclosure;

[0023] Figure 4 This is a schematic block diagram of a processing device for the second type of AI service request in this embodiment of the present disclosure;

[0024] Figure 5 This is a flowchart of the process of the AI ​​service request processing device in the embodiments of this disclosure;

[0025] Figure 6 This is a schematic block diagram of an example electronic device in an embodiment of this disclosure. Detailed Implementation

[0026] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0027] Existing solutions to the problems caused by massive online AI (Artificial Intelligence) services mainly include the following approaches: The first approach is based on serverless solutions to address low machine resource utilization. This involves adjusting the number of service instances based on traffic and custom strategies. However, this method suffers from long initialization latency for AI services, reaching minutes for complex, large models. Serverless solutions cannot solve the problem of request timeouts and traffic loss caused by long initialization times. Furthermore, serverless solutions require the introduction of service mesh governance modules, leading to high platform migration and transformation costs. The second approach involves model compression through pruning, quantization, and distillation to reduce model loading time and improve AI service startup speed, addressing the issues of long startup times and traffic loss. While model compression reduces loading latency, it still has the following drawbacks: each model requires corresponding adaptation development, significantly increasing the workload for developers; it only reduces memory usage to a certain extent and cannot completely release memory resources when there are no service calls; and while the initialization speed of model services is improved, inference accuracy is reduced.

[0028] To address the aforementioned technical issues, this disclosure provides a method for processing AI service requests, such as... Figure 1 As shown, it includes:

[0029] Step S101: Obtain AI service request. The AI ​​service request can be initiated by the client. First, accept the AI ​​service request traffic sent by the client.

[0030] Step S102: Obtain the status of the AI ​​service process related to the AI ​​service request. The status of the AI ​​service process includes: working state or recycling state. The working state can mean that the AI ​​service process is running normally, and the recycling state means that the AI ​​service process is frozen and suspended. The AI ​​service process in this embodiment can include four states: initialization state (Init), waking up state (Creating), working state (Running), and recycling state (Closing). Among them, the Init state is the initialization state when the entire proxy service starts. It is only in the Init state when the container starts or is transformed from the Closing state; the Creating state is the state of the proxy service during the process of waking up the AI ​​service. This state is transformed from the Init state; the Running state is the state of the proxy service when the AI ​​service is working normally. This state is transformed from the Creating state; the Closing state is the state of the proxy service when the AI ​​service is being recycled. This state can only be transformed from the Running state.

[0031] Step S103: In response to the AI ​​service process being in a working state, the AI ​​service request is processed through the AI ​​service process. If the monitoring shows that the current AI service process is in a Running state and can process request traffic, then the AI ​​service request is processed directly through that AI service process.

[0032] Step S104: In response to the AI ​​service process being in a recycled state, the AI ​​service process's state is adjusted from recycled to working state, and then the AI ​​service process processes AI service requests. If the AI ​​service process is detected to be in any state other than Running (i.e., Init, Creating, or Closing), it can be considered to be in a recycled state, meaning the AI ​​service process is not running. In this case, the AI ​​service is woken up, causing it to enter the Running state before processing AI service requests.

[0033] The above technical solution can reclaim AI service processes that are not frequently called, and the AI ​​service processes can be quickly woken up when they need to handle corresponding AI service requests, avoiding the AI ​​service processes from occupying a lot of resources, thereby improving the utilization rate of platform resources and the availability of services when they are started.

[0034] As an optional implementation method, such as Figure 2 As shown, the method further includes: step S201, when the AI ​​service process is in the working state, monitoring whether the AI ​​service process has reached the recycling condition; step S202, in response to the AI ​​service process reaching the recycling condition, adjusting the state of the AI ​​service process from the working state to the recycling state. In this embodiment, the process of recycling the AI ​​service process is monitored. By recording the number of service requests being processed and the response time of the last request, these two indicators are used to calculate whether the AI ​​service process needs to be recycled according to the configured strategy; alternatively, the response time of the last request can be used alone to determine whether recycling is necessary. If recycling is required, a recycling signal is directly sent to the AI ​​service process for recycling, and its state is set to the Closing state. A corresponding coroutine is then started to monitor the entire recycling process of the AI ​​service process. If recycling fails, an alarm is issued and the container is restarted. If recycling is detected as successful, its state is set to the Init state.

[0035] As an optional implementation, monitoring whether an AI service process meets the recycling conditions includes: obtaining the number of AI service requests being processed by the AI ​​service process, and the time when the number of AI service requests being processed by the AI ​​service process is zero, and the time when the AI ​​service process last responded to an AI service request; in response to the number of AI service requests being zero, and the time when the AI ​​service process last responded to an AI service request reaching a preset threshold, determining that the AI ​​service process meets the recycling conditions. For example, the preset threshold can be determined according to the actual application scenario, such as being set to 10 seconds. When it is monitored that the number of AI service requests currently being processed by an AI service process is 0, i.e., there is no request traffic being processed, and the time since the last response to an AI service request reaches 10 seconds, the AI ​​service process is recycled to prevent the AI ​​service process from occupying platform resources for a long time and to improve the platform's resource utilization. Alternatively, recycling can be determined solely by the response time of the last request, for example, a preset threshold of 30 seconds, where the AI ​​service is recycled when the response time of the last request reaches 30 seconds.

[0036] As an optional implementation, the method further includes: when the AI ​​service process is in a working state, periodically sending heartbeat messages to the AI ​​service process to detect whether the AI ​​service process is abnormal; in response to an abnormal AI service process, sending an alarm message and restarting the AI ​​service process. In this embodiment, heartbeat messages are sent to detect activity; if the AI ​​service process is normal, it can respond to the heartbeat message; otherwise, it cannot respond. This method is mainly used to perform health checks on the AI ​​service process in the Running state. The check methods support, but are not limited to, HTTP (Hypertext Transfer Protocol), TCP (Transmission Control Protocol), and Shell (shell script). Through health checks, it is ensured that the AI ​​service process is available in the Running state.

[0037] As an optional implementation, step S104, adjusting the state of the AI ​​service process from the recycled state to the working state, includes: obtaining the execution status file of the AI ​​service process from the disk; and controlling the AI ​​service process to resume execution at the point where it was last frozen based on the execution status file. When the AI ​​service process meets the recycling conditions, adjusting the state of the AI ​​service process from the working state to the recycled state includes: freezing the AI ​​service process; and storing the execution status file of the AI ​​service process on the disk. By freezing the recycled AI service process on the disk, it can be quickly loaded from the disk when the service is woken up, without consuming a lot of time and resources to load the AI ​​model, thus achieving rapid wake-up of the AI ​​service process. In this embodiment, a multi-process + multi-threaded approach is used to handle the inference work of the AI ​​service, and CRIU (Checkpoint RestoreIn UserNamespace) technology is used to achieve rapid startup of the AI ​​service process.

[0038] This disclosure also provides an apparatus for processing AI service requests, such as... Figure 3 As shown, it includes:

[0039] The AI ​​service proxy module 301 is configured to obtain AI service requests. The AI ​​service requests can be initiated by the client. The AI ​​service proxy module 301 first accepts the AI ​​service request traffic sent by the client.

[0040] The monitoring module 302 is configured to acquire the status of the AI ​​service process related to the AI ​​service request. The status of the AI ​​service process includes: working status or recycling status. The working status indicates that the AI ​​service process is running normally, while the recycling status indicates that the AI ​​service process is frozen and suspended. The AI ​​service process in this embodiment can include four states: Init state, Creating state, Running state, and Closing state. Among them, the Init state is the initialization state when the entire proxy service starts, and it is only in this state when the container starts or is transformed from the Closing state; the Creating state is the state of the proxy service during the process of waking up the AI ​​service, and this state is transformed from the Init state; the Running state is the state of the proxy service when the AI ​​service is working normally, and this state is transformed from the Creating state; the Closing state is the state of the proxy service when the AI ​​service is being recycled, and this state can only be transformed from the Running state.

[0041] AI service module 303 is configured to be in a working state in response to the status of the AI ​​service process, and processes AI service requests through the AI ​​service process. When monitoring module 302 detects that the current AI service process is in a Running state and can process request traffic, AI service module 303 directly processes the AI ​​service request through the AI ​​service process.

[0042] The AI ​​service proxy module 301 is configured to respond to the AI ​​service process's state being in a recycled state. After adjusting the AI ​​service process's state from recycled to working state, it processes AI service requests through the AI ​​service process. If the monitoring module 302 detects that the current AI service process is in a state other than Running, i.e., in Init, Creating, or Closing state, it can be considered that the AI ​​service process is in a recycled state, meaning the AI ​​service process is not running. In this case, the AI ​​service is woken up, and the state of the AI ​​service proxy module 301 is switched back to Running state before processing AI service requests.

[0043] The AI ​​service request processing device described above can reclaim AI service processes that are not frequently called, and the AI ​​service processes can be quickly woken up when they need to handle the corresponding AI service requests, thus avoiding the AI ​​service processes occupying a lot of resources and improving the utilization rate of platform resources and the availability of services when they are started.

[0044] As an optional implementation, the monitoring module 302 further includes: monitoring whether the AI ​​service process has reached the recycling condition when its state is working; and adjusting the state of the AI ​​service process from working to recycling state in response to the AI ​​service process reaching the recycling condition. This embodiment monitors the process of recycling the AI ​​service process. By recording the number of service requests being processed and the response time of the last request, the system calculates whether the AI ​​service process needs to be recycled according to the configured strategy; alternatively, it can be determined solely by the response time of the last request. If recycling is required, a recycling signal is sent directly to the AI ​​service process, its state is set to Closing, and a corresponding coroutine is started to monitor the entire recycling process. If recycling fails, an alarm is issued and the container is restarted; if recycling is successful, its state is set to Init. This technical solution avoids the AI ​​service process from continuously running and consuming server resources.

[0045] As an optional implementation, the monitoring module 302 monitors whether the AI ​​service process has reached the recycling condition by: obtaining the number of AI service requests being processed by the AI ​​service process, and the time when the number of AI service requests being processed by the AI ​​service process is zero, and the time when the AI ​​service process last responded to an AI service request; in response to the number of AI service requests being zero, and the time when the AI ​​service process last responded to an AI service request reaching a preset threshold, determining that the AI ​​service process has reached the recycling condition. For example, the preset threshold can be determined according to the actual application scenario, such as being set to 10 seconds. When it is monitored that the number of AI service requests currently being processed by an AI service process is 0, i.e., there is no request traffic being processed, and the time since the last response to an AI service request reaches 10 seconds, the AI ​​service process is recycled to prevent the AI ​​service process from occupying platform resources for a long time and to improve the platform's resource utilization. Alternatively, recycling can be determined solely by the response time of the last request, for example, a preset threshold of 30 seconds, where the AI ​​service is recycled when the response time of the last request reaches 30 seconds.

[0046] As an optional implementation method, such as Figure 4 As shown, the device also includes an AI service activation module 304, configured to periodically send heartbeat messages to the AI ​​service process when the AI ​​service process is in a working state to detect whether the AI ​​service process is abnormal; in response to an AI service process abnormality, it sends an alarm message to the AI ​​service module 303 and restarts the AI ​​service process. In this embodiment, the AI ​​service activation module 304 periodically sends heartbeat messages to activate the AI ​​service process. If the AI ​​service process is normal, it can respond to the heartbeat message; otherwise, it cannot respond. This method is mainly used to perform health checks on the AI ​​service process in the Running state. The check methods support, but are not limited to, HTTP, TCP, Shell, etc. Through health checks, it is ensured that the AI ​​service process is available in the Running state.

[0047] For example, such as Figure 5As shown, when traffic arrives at the AI ​​service proxy module 301, if the AI ​​service proxy module 301 is in the Init state, it will receive the traffic, wake up the AI ​​service process, and start the monitoring module 302 to monitor the service wake-up process. It will then poll the status until it reaches the Running state, forwarding the request traffic to the AI ​​service process of the AI ​​service module 303 to perform model inference and return the AI ​​service result to the client. If the AI ​​service activation module 304 detects that the AI ​​service has not started successfully within the configured time, it will restart the container and report the exception. If the AI ​​service proxy module 301 is in the Running state when traffic arrives, it will directly forward the request to the backend service and return the processing result directly. If the AI ​​service proxy module 301 is in the Creating or Closing state when traffic arrives, it will receive the traffic, and the monitoring module 302 will continuously poll the status of the AI ​​service proxy module 301 until it reaches the Running state, at which point it will forward the AI ​​service request.

[0048] As an optional implementation, the AI ​​service agent module 301 adjusts the state of the AI ​​service process from the recycled state to the working state by: obtaining the execution status file of the AI ​​service process from the disk; and controlling the AI ​​service process to resume execution at the point where it was last frozen based on the execution status file. When the AI ​​service process meets the recycling conditions, the monitoring module 302 adjusts the state of the AI ​​service process from the working state to the recycled state by: freezing the AI ​​service process; and storing the execution status file of the AI ​​service process on the disk. By freezing the recycled AI service process on the disk, it can be quickly loaded from the disk when the service is woken up, without consuming a lot of time and resources to load the AI ​​model, thus achieving rapid wake-up of the AI ​​service process. In this embodiment, a multi-process + multi-threaded approach is used to handle the inference work of the AI ​​service, while CRIU technology is used to achieve rapid startup of the AI ​​service process.

[0049] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0050] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0051] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0052] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0053] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0054] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning objective function algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the AI ​​service request processing method. For example, in some embodiments, the AI ​​service request processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the AI ​​service request processing method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the AI ​​service request processing method by any other suitable means (e.g., by means of firmware).

[0055] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0056] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0057] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0058] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0059] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0060] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0061] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0062] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for processing an AI service request, comprising: obtaining an AI service request; obtaining a state of an AI service process related to the AI service request; the state of the AI service process comprises a working state or a recycling state; in response to the state of the AI service process being the working state, processing the AI service request by the AI service process; in response to the state of the AI service process being the recycling state, obtaining an execution state file of the AI service process from a disk, and controlling the AI service process to resume execution at a point where the AI service process was last frozen, and then processing the AI service request by the AI service process based on the execution state file. 2.The method of claim 1, further comprising: monitoring whether the AI service process meets a recycling condition when the state of the AI service process is the working state; in response to the AI service process meeting the recycling condition, adjusting the state of the AI service process from the working state to the recycling state. 3.The method of claim 1, further comprising: periodically sending a heartbeat message to the AI service process when the state of the AI service process is the working state, and detecting whether the AI service process is abnormal; in response to the AI service process being abnormal, sending an alarm information, and restarting the AI service process.

4. The method of claim 2, wherein, the monitoring whether the AI service process meets the recycling condition comprises: obtaining a number of AI service requests being processed by the AI service process, and a time when the AI service process last responded to the AI service request when the number of AI service requests being processed by the AI service process is zero; in response to the number of AI service requests being zero, and the time when the AI service process last responded to the AI service request reaching a preset threshold, determining that the AI service process meets the recycling condition.

5. The method of claim 2, wherein, the adjusting the state of the AI service process from the working state to the recycling state comprises: freezing the AI service process; storing an execution state file of the AI service process in a disk. 6.An apparatus for processing an AI service request, comprising: an AI service agent module configured to obtain an AI service request; a monitoring module configured to obtain a state of an AI service process related to the AI service request; the state of the AI service process comprises a working state or a recycling state; an AI service module configured to process the AI service request by the AI service process in response to the state of the AI service process being the working state; the AI service agent module is configured to obtain an execution state file of the AI service process from a disk in response to the state of the AI service process being the recycling state, and control the AI service process to resume execution at a point where the AI service process was last frozen, and then the AI service module processes the AI service request by the AI service process based on the execution state file.

7. The apparatus of claim 6, wherein, the monitoring module further comprises: monitoring whether the AI service process meets a recycling condition when the state of the AI service process is the working state; In response to the AI service process reaching the recycling condition, adjusting the state of the AI service process from the working state to the recycling state.

8. The apparatus of claim 6, further comprising: an AI service alive detection module configured to periodically send a heartbeat message to the AI service process when the state of the AI service process is the working state, and detect whether the AI service process is abnormal; in response to the AI service process being abnormal, sending an alarm information to the AI service module, and restarting the AI service process.

9. The apparatus of claim 7, wherein, The monitoring module monitors whether the AI service process reaches the recycling condition comprises: obtaining the number of AI service requests being processed by the AI service process, and the time when the AI service process last responds to the AI service request when the number of AI service requests being processed by the AI service process is zero; in response to the number of AI service requests being zero and the time when the AI service process last responds to the AI service request reaching a preset threshold, determining that the AI service process reaches the recycling condition.

10. The apparatus of claim 7, wherein, The monitoring module adjusts the state of the AI service process from the working state to the recycling state comprises: freezing the AI service process; storing the execution state file of the AI service process in the disk.

11. An electronic device comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-5.

13. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1-5.

Images