Request scheduling method and device

Abstract

The invention provides a request scheduling method and device, and relates to the technical field of artificial intelligence and cloud computing, in particular to the technical field of cloud services. A specific embodiment of the method comprises the following steps: in response to a received server sending event SSE request, creating SSE long connection; generating a session identifier, and returning the session identifier through the SSE long connection; in response to a received message request carrying the session identifier, sending the message request to a server corresponding to the session identifier for processing to obtain response data; and returning response data through the SSE long connection.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence and cloud computing technology, and in particular to the field of cloud service technology. Background Art

[0002] With the increasing popularity of Large Language Model (LLM), Model Context Protocol (MCP) has become a core open standard for connecting LLM with business systems due to its ability to standardize contextual interactions within AI (Artificial Intelligence) applications.

[0003] MCP defines how contextual information is exchanged between applications and AI models, enabling developers to connect various data sources, tools, and functions to AI models in a consistent manner, much like USB-C allows different devices to connect through the same interface. MCP's goal is to create a universal standard that makes the development and integration of AI applications simpler and more uniform. With the rise of MCP, convenient and efficient hosting of MCP servers has become a key research area for major cloud vendors. Summary of the Invention

[0004] The embodiments of the present disclosure provide a request scheduling method, apparatus, device, storage medium, and program product.

[0005] In the first aspect, an embodiment of the present disclosure proposes a request scheduling method, including: in response to receiving an SSE request for a server to send an event, creating an SSE long connection; generating a session identifier, and returning the session identifier through the SSE long connection; in response to receiving a message request carrying the session identifier, sending the message request to the server corresponding to the session identifier for processing to obtain response data; and returning the response data through the SSE long connection.

[0006] In the second aspect, an embodiment of the present disclosure proposes a request scheduling device, including: a creation module, configured to create an SSE long connection in response to receiving an SSE event request sent by a server; a generation module, configured to generate a session identifier and return the session identifier through the SSE long connection; a processing module, configured to send the message request to the server corresponding to the session identifier for processing in response to receiving a message request carrying the session identifier, and obtain response data; a return module, configured to return the response data through the SSE long connection.

[0007] In a third aspect, an embodiment of the present disclosure proposes an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0009] In a fifth aspect, an embodiment of the present disclosure proposes a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0010] The key or important features of the embodiments of the present disclosure are not intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Other features, objects, and advantages of the present disclosure will become more apparent upon reading the detailed description of the non-limiting embodiments made with reference to the following drawings. The drawings are provided for a better understanding of the present disclosure and do not constitute a limitation of the present disclosure. Among them: Figure 1 is a flowchart of a first embodiment of a request scheduling method according to the present disclosure; Figure 2 This is a flowchart of the first plan for CFC to host MCP; Figure 3 This is a flow chart of the second scenario of CFC hosting MCP; Figure 4 This is a flow chart of the third option for CFC to host MCP; Figure 5 is a flowchart of a second embodiment of a request scheduling method according to the present disclosure; Figure 6 This is the flow chart of the weather tool MCP server; Figure 7 This is the overall architecture diagram of CFC hosting MCP; Figure 8 This is the system flow chart of CFC hosting MCP; Figure 9 is a structural diagram of an embodiment of a request scheduling device according to the present disclosure; Figure 10 It is a block diagram of an electronic device used to implement the request scheduling method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0013] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0014] Figure 1 A process 100 of an embodiment of a request scheduling method according to the present disclosure is shown. The request scheduling method includes the following steps: Step 101: In response to receiving a server-sent event SSE request, create an SSE persistent connection.

[0015] In this embodiment, when an SSE (Server-Sent Events) request sent by a client is received, an SSE persistent connection may be established between the server and the client.

[0016] The client can use HTTP (Hypertext Transfer Protocol) to send SSE requests to the server. SSE persistent connections can maintain a long connection between the client and the server, mainly used by the server to push messages to the client.

[0017] SSE response headers can include Content-Type, Cache-Control, and Connection. The Content-Type can be text / event-stream, indicating that data is transmitted using text streams. Cache-Control can be no-cache, indicating that data is not cached to maintain real-time availability. The Connection can be keep-alive, indicating that the connection is maintained, allowing the server to continuously push messages to the client.

[0018] SSE response data can be messages. The server can send multiple messages to the client at a time. Messages can be separated by a first special character (such as \n\n). Each message can consist of multiple lines, separated by a second special character (such as \n). Messages can include, but are not limited to, at least one of the following attributes: content (data), event type (event), message identifier (id), and retry time (retry).

[0019] Step 102: Generate a session identifier and return the session identifier via an SSE persistent connection.

[0020] In this embodiment, the server may generate a session identifier and return the session identifier to the user through an SSE persistent connection.

[0021] The first time a session identifier is returned to the user through an SSE persistent connection, the user's subsequent message requests must carry this session identifier. This is a sticky session.

[0022] Step 103: In response to receiving the message request carrying the session identifier, the message request is sent to a server corresponding to the session identifier for processing to obtain response data.

[0023] In this embodiment, upon receiving a message request carrying a session identifier from a client, the message request may be sent to a server corresponding to the session identifier for processing to obtain response data. The server corresponding to the session identifier may be the server that generated the session identifier. The message request must be perceived by the server corresponding to the session identifier.

[0024] Because service-oriented services are required, servers must be highly available and cannot be a single point of presence. Therefore, solutions for horizontally scalable, highly available sticky sessions can mainly include: 1. Based on the server load, the receiving server is determined; the receiving server writes the message request into the message queue. The server corresponding to the session ID can detect the message request from the message queue and process it.

[0025] Figure 2 A flow chart showing a first scenario in which a CFC hosts an MCP.

[0026] Step 201: The MCP client sends an SSE request to the load balancing server.

[0027] Step 202: The load balancing server sends an SSE request to MCP server 1.

[0028] Step 203: The MCP server 1 establishes a persistent connection with the MCP client, generates a session identifier, and returns it to the MCP client.

[0029] Step 204: The MCP client sends a message request to the load balancing server.

[0030] Step 205 : The load balancing server sends the message request to the MCP server 2 .

[0031] Step 206: The MCP server 2 writes the message request into the message queue.

[0032] Step 207: MCP server 1 subscribes to the message queue to sense the message request.

[0033] Step 208: The MCP server 1 returns response data to the MCP client.

[0034] Through load balancing, each server can process SSE requests and generate session identifiers. However, the message request carrying the session identifier needs to be perceived by the server that generates the session identifier.

[0035] 2. After generating a session identifier, a mapping relationship can be created between the session identifier and the server corresponding to the session identifier. Subsequently, upon receiving a message request, the receiving server can be determined based on server load. The message request is sent to the receiving server, which then queries the mapping relationship based on the session identifier and routes the message request to the server corresponding to the session identifier for processing.

[0036] Figure 3 A flow chart showing a second scenario in which a CFC hosts an MCP.

[0037] Step 301: The MCP client sends an SSE request to the load balancing server.

[0038] Step 302: The load balancing server sends an SSE request to MCP server 1.

[0039] Step 303: The MCP server 1 establishes a persistent connection with the MCP client, generates a session identifier, and returns it to the MCP client.

[0040] Step 304: MCP server 1 writes the mapping relationship between the session identifier and MCP server 1 into the database.

[0041] Step 305: The MCP client sends a message request to the load balancing server.

[0042] Step 306 : The load balancing server sends the message request to the MCP server 2 .

[0043] Step 307: The MCP server 2 queries the database for a mapping relationship.

[0044] Step 308: Routing the message request to MCP server 1 for processing.

[0045] Step 309: The MCP server 1 returns response data to the MCP client.

[0046] Use shared storage to persist the mapping of session identifiers to servers, and route message requests carrying the same session identifier to the same server for processing.

[0047] 3. After generating the session identifier, a mapping relationship between the session identifier and the server corresponding to the session identifier can be created. Subsequently, upon receiving a message request, the mapping relationship can be queried based on the session identifier to determine the server corresponding to the session identifier; the message request can be sent to the server corresponding to the session identifier for processing.

[0048] Figure 4 A flowchart showing a third solution of CFC hosting MCP.

[0049] Step 401: The MCP client sends an SSE request to the scheduling request.

[0050] Step 402 : The scheduling request sends an SSE request to the MCP server 1 .

[0051] In step 403, the MCP server 1 establishes a persistent connection with the MCP client, generates a session identifier, and returns it to the MCP client.

[0052] Step 404: The scheduling request writes the mapping relationship between the session identifier and the MCP server 1 into the database.

[0053] Step 405: The MCP client sends a message request to the scheduling request.

[0054] Step 406: The scheduling request queries the mapping relationship in the database.

[0055] Step 407: The scheduling request sends the message request to the MCP server 1 for processing.

[0056] Step 408: The MCP server 1 returns response data to the MCP client.

[0057] By perceiving the traffic scheduling of the session identifier, the load balancing server records the mapping relationship between the session identifier and the server, and can route message requests carrying the same session identifier to the corresponding server for processing.

[0058] Step 104: Return response data via the SSE persistent connection.

[0059] In this embodiment, the server can return response data of the message request to the user through the SSE long connection.

[0060] This disclosure provides a request scheduling method that implements complete session lifecycle management for the MCP protocol. By establishing a dedicated data channel through SSE, the consistency of requests and responses within the same session is guaranteed, resolving the conflict between the traditional Serverless stateless architecture and the MCP stateful protocol.

[0061] Continue to refer Figure 5, which shows a process 500 of another embodiment of a request scheduling method according to the present disclosure. The request scheduling method includes the following steps: Step 501 : In response to receiving a cloud computing function CFC, a Hypertext Transfer Protocol (HTTP) trigger is created for the CFC and a domain name is allocated as a service endpoint of a Model Context Protocol (MCP) server.

[0062] In this embodiment, when a CFC (Cloud Function Compute) created by a user is received, an HTTP trigger (httptrigger) may be created for the CFC and an independent domain name may be allocated as a service endpoint of the MCP server.

[0063] By creating HTTP triggers and assigning domain names to CFCs, MCP services can be deployed out-of-the-box. This automatically creates HTTP triggers, while domain isolation ensures multi-user security. This eliminates manual configuration costs and prevents service endpoint conflicts.

[0064] CFC provides event-based, elastic, highly available, scalable, and responsive cloud-based serverless computing. Focusing solely on the business logic code, CFC eliminates the need to manage and configure server resources. It supports multiple function triggers to meet diverse event-triggered scenarios.

[0065] MCP is an open protocol that standardizes how applications provide context to large language models, similar to universal application interfaces (such as USB-C). MCP generally adopts a client / server (C / S) architecture, with transport protocols including HTTP and SSE. Clients can use HTTP to send requests to servers, and servers can use SSE to send responses to clients. MCP provides at least one of the following capabilities: data sources (Resources), function tools (Tools), root configuration (Roots), sampling (Sampling), and prompt templates (Prompts).

[0066] Taking the weather tool MCP server as an example, the relationship between the user, MCP client, server and large language model in the entire process can be as follows: Figure 6 As shown: Step 601, start the server, such as tools, resources and prompt templates.

[0067] Step 602: Configure the MCP server.

[0068] Step 603, query what the weather is like in a certain city today.

[0069] Step 604: Obtain tools, resources, and prompt templates.

[0070] Step 605: Call the interface of the large model based on the integrated model, tool functions, and queries.

[0071] Step 606: Select the most appropriate tool, analyze the intent, and determine whether the weather function can be called.

[0072] Step 607: No matching function, conventional model answer.

[0073] Step 608: Return the matching function.

[0074] Step 609, calling the function.

[0075] Step 610: Call an external weather API.

[0076] Step 611, return weather data {"temp":18C, "humidity": 60%}.

[0077] Step 612, return weather data.

[0078] Step 613, query and polish weather data.

[0079] Step 614: data format conversion.

[0080] Step 615: The temperature in a certain city is 18℃ today, and the humidity is 60%.

[0081] Step 616: The temperature in a certain city is 18℃ today, and the humidity is 60%.

[0082] Step 502: In response to receiving an MCP server access request sent based on a service endpoint, allocate a container group.

[0083] In this embodiment, upon receiving an MCP server access request from a user using a service endpoint, a container group (pod) may be allocated. Container group resources are allocated on demand, thus achieving dynamic resource provisioning.

[0084] A pod is the smallest deployable unit in Kubernetes. It contains an application container (or, in some cases, multiple containers), storage resources, a unique network IP address, and options that determine how the container runs. A pod represents an independent running instance of an application in Kubernetes, which may consist of a single container or several tightly coupled containers.

[0085] In some embodiments, an event hub (eventhub) can send allocation requests to a pool manager, which can then allocate container groups. The pool manager initializes the container group based on CFC information and pulls the code for the MCP server associated with the CFC to start the MCP server service. The unified access provided by the event hub and the centralized resource allocation provided by the pool manager enable distributed scheduling optimization, avoid single-point bottlenecks, and support horizontal scalability.

[0086] Step 503: Establish an SSE persistent connection based on the container group.

[0087] In this embodiment, an SSE persistent connection can be established based on the container group.

[0088] In some embodiments, an SSE persistent connection may be established by invoking the service of an MCP server on a container group through an event center.

[0089] Step 504: Generate a session identifier and return the session identifier via the SSE persistent connection.

[0090] In this embodiment, the specific operation of step 504 has been Figure 1 In the illustrated embodiment, step 102 is described in detail and will not be repeated here.

[0091] Step 505: In response to receiving the message request carrying the session identifier, a scheduling request is sent to the pool management through the event center.

[0092] In this embodiment, when a message request carrying a session identifier is received from the client, a scheduling request may be sent to the pool management through the event center.

[0093] Step 506: Affinity scheduling is performed through the pool manager to determine the container group.

[0094] In this embodiment, the pool manager performs affinity scheduling to determine the container group.

[0095] Step 507: dispatch the message request to the container group for processing through the event center to obtain response data.

[0096] In this embodiment, the message request is dispatched to the container group for processing through the event center to obtain response data.

[0097] Step 508: Return response data via the SSE persistent connection.

[0098] In this embodiment, the specific operation of step 508 has been Figure 1 In the embodiment shown, step 104 is described in detail and will not be repeated here.

[0099] This disclosure provides a request scheduling method that implements complete session lifecycle management for the MCP protocol. By establishing a dedicated data channel through SSE, it ensures request and response consistency within the same session, resolving the conflict between traditional Serverless stateless architectures and the MCP stateful protocol. It supports SSE protocol and HTTP triggers, session affinity scheduling, and multi-instance concurrent tasks.

[0100] Figure 7 The figure shows the overall architecture of CFC hosting MCP. Figure 7 As shown, the architecture of CFC-hosted MCP can include clients, applications, and CFC. Clients can include cursors, AB nodes, and Claude. Applications can include chatGPT, Bochaai, and Github. CFC can include MCP hosting, MCP calls, scheduling, and cluster resource management. MCP hosting can include metadata management, configuration management, code management, function scheduling, network management, storage management, monitoring and logging, and version management. MCP calls can forward function events. Scheduling can include elastic scaling and SSE session affinity scheduling. Cluster resource management can manage function instances during the function runtime.

[0101] Figure 8 Shows the system flow chart of CFC hosting MCP.

[0102] When a user creates a CFC function, CFC will create an httptrigger by default and assign an independent domain name to the function as the service endpoint of the MCP server.

[0103] Users initiate access to the MCP server using the service endpoint.

[0104] Eventhub requests PoolManager to allocate pod resources. PoolManager initializes the resource pods based on the user's function information (at runtime), pulls the MCPserver code associated with the function, and starts the service.

[0105] Eventhub calls the MCP server service on the pod to complete the establishment of the SSE long connection throughout the entire process.

[0106] Eventhub obtains the session_id returned by the MCP server service, writes the mapping between pod and session_id to Redis, and increases the number of connections by 1 (to implement concurrency control later).

[0107] Returns user sse created successfully.

[0108] The user initiates a message request with the session_id.

[0109] Eventhub carries the session_id to request the poolmanager to perform session affinity scheduling.

[0110] PoolManager obtains the corresponding pod based on session_id.

[0111] Eventhub sends the message request to the scheduled pod to implement a sticky session.

[0112] Further references Figure 9 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a request scheduling device. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0113] like Figure 9 As shown, the request scheduling device 600 of this embodiment may include: a creation module 901, a generation module 902, a processing module 903, and a return module 904. The creation module 901 is configured to create an SSE persistent connection in response to receiving an SSE request for sending an event from a server; the generation module 902 is configured to generate a session identifier and return the session identifier through the SSE persistent connection; the processing module 903 is configured to send the message request to the server corresponding to the session identifier for processing in response to receiving a message request carrying the session identifier, and obtain response data; the return module 904 is configured to return the response data through the SSE persistent connection.

[0114] In this embodiment, in the request scheduling device 900, the specific processing of the creation module 901, the generation module 902, the processing module 903 and the return module 904 and the technical effects thereof can be referred to in the respective Figure 1 The relevant descriptions of steps 101-104 in the corresponding embodiment are not repeated here.

[0115] In some optional implementations of this embodiment, the request scheduling device 900 also includes: a determination module, configured to: in response to receiving a cloud computing function CFC, create a hypertext transfer protocol HTTP trigger and assign a domain name for the CFC as a service endpoint of the model context protocol MCP server.

[0116] In some optional implementations of this embodiment, the creation module 901 is further configured to: allocate a container group in response to receiving an MCP server access request sent based on a service endpoint; and establish an SSE persistent connection based on the container group.

[0117] In some optional implementations of this embodiment, the creation module 901 is further configured to: send an allocation request to the pool manager through the event center; and allocate the container group through the pool manager.

[0118] In some optional implementations of this embodiment, the creation module 901 is further configured to: initialize the container group according to the CFC information; pull the code of the MCP server associated with the CFC, and start the service of the MCP server.

[0119] In some optional implementations of this embodiment, the creation module 901 is further configured to: call the service of the MCP server on the container group through the event center to establish an SSE persistent connection.

[0120] In some optional implementations of this embodiment, the processing module 903 is further configured to: send a scheduling request to the pool management through the event center; perform affinity scheduling through the pool manager to determine the container group; and schedule the message request to the container group for processing through the event center to obtain response data.

[0121] In some optional implementations of this embodiment, the request scheduling apparatus 900 further includes: a mapping module configured to create a mapping relationship between a session identifier and a server corresponding to the session identifier.

[0122] In some optional implementations of this embodiment, the processing module 903 is further configured to: query the mapping relationship based on the session identifier to determine the server corresponding to the session identifier; and send the message request to the server corresponding to the session identifier for processing.

[0123] In some optional implementations of this embodiment, the processing module 903 is further configured to: determine the receiving server based on the server load situation; send the message request to the receiving server, so that the receiving server queries the mapping relationship based on the session identifier, and routes the message request to the server corresponding to the session identifier for processing.

[0124] In some optional implementations of this embodiment, the processing module 903 is further configured to: determine the receiving server based on the server load situation; write the message request into the message queue through the receiving server, wherein the server corresponding to the session identifier perceives the message request from the message queue and processes it.

[0125] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0126] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0127] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0128] like Figure 10 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. RAM 1003 may also store various programs and data required for the operation of device 1000. Computing unit 1001, ROM 1002, and RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to bus 1004.

[0129] Various components in device 1000 are connected to I / O interface 1005, including an input unit 1006, such as a keyboard, mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, optical disk, etc.; and a communication unit 1009, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1009 allows device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0130] Computing unit 1001 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 1001 performs the various methods and processes described above, such as the request scheduling method. For example, in some embodiments, the request scheduling method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by computing unit 1001, one or more steps of the request scheduling method described above can be performed. Alternatively, in other embodiments, computing unit 1001 can be configured to perform the request scheduling method in any other suitable manner (e.g., via firmware).

[0131] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0135] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0136] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0137] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions provided by this disclosure can be achieved. This is not limited herein.

[0138] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A request scheduling method, comprising: In response to receiving the server-sent event SSE request, creating an SSE persistent connection; Generate a session identifier and return the session identifier through the SSE persistent connection; In response to receiving the message request carrying the session identifier, sending the message request to the server corresponding to the session identifier for processing to obtain response data; The response data is returned through the SSE long connection.

2. The method according to claim 1, wherein The method further comprises: In response to receiving the cloud computing function CFC, a Hypertext Transfer Protocol (HTTP) trigger is created for the CFC and a domain name is allocated as a service endpoint of a Model Context Protocol (MCP) server.

3. The method according to claim 2, wherein: The step of creating an SSE persistent connection in response to receiving a server-sent event (SSE) request includes: In response to receiving an MCP server access request sent based on the service endpoint, allocating a container group; The SSE persistent connection is established based on the container group.

4. The method according to claim 3, wherein: The distribution container group includes: Sending an allocation request to the pool manager via the event center; The container group is allocated by the pool manager.

5. The method according to claim 4, wherein Allocating the container group by the pool manager includes: Initialize the container group according to the CFC information; Pull the code of the MCP server associated with the CFC and start the service of the MCP server.

6. The method according to claim 5, wherein: The establishing the SSE persistent connection based on the container group includes: The service of the MCP server on the container group is called by the event center to establish the SSE long connection.

7. The method according to claim 6, wherein: The sending of the message request to the server corresponding to the session identifier for processing to obtain response data includes: Sending a scheduling request to the pool management via the event center; Performing affinity scheduling by the pool manager to determine the container group; The message request is dispatched to the container group for processing through the event center to obtain the response data.

8. The method according to any one of claims 1 to 7, wherein The method further comprises: A mapping relationship between the session identifier and the server corresponding to the session identifier is created.

9. The method according to claim 8, wherein The sending the message request to the server corresponding to the session identifier for processing includes: Querying the mapping relationship based on the session identifier to determine the server corresponding to the session identifier; The message request is sent to the server corresponding to the session identifier for processing.

10. The method according to claim 8, wherein The sending the message request to the server corresponding to the session identifier for processing includes: Determine the receiving server based on the server load; The message request is sent to the receiving server, so that the receiving server queries the mapping relationship based on the session identifier and routes the message request to the server corresponding to the session identifier for processing.

11. The method according to any one of claims 1 to 7, wherein The sending of the message request to the server corresponding to the session identifier for processing to obtain response data includes: Determine the receiving server based on the server load; The message request is written into a message queue through the receiving server, wherein the server corresponding to the session identifier perceives the message request from the message queue and processes it.

12. A request scheduling device, comprising: A creation module, configured to create an SSE persistent connection in response to receiving a server-sent event (SSE) request; A generating module is configured to generate a session identifier and return the session identifier through the SSE persistent connection; a processing module configured to, in response to receiving a message request carrying the session identifier, send the message request to a server corresponding to the session identifier for processing, and obtain response data; The return module is configured to return the response data through the SSE long connection.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to execute the method according to any one of claims 1 to 11.

15. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images