Hybrid orchestration method of traditional service and LLM service
By semantically parsing user requests and dynamically generating node execution plans, the problem of low scheduling efficiency of LLM services in existing technologies is solved, and efficient parallel scheduling of traditional services and LLM services is achieved to meet dynamic user needs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-03-10
AI Technical Summary
The scheduling process of existing LLM services is inefficient and cannot meet the dynamic needs of users. Furthermore, the parsing and scheduling time of external LLM services for business processes is too long.
By semantically parsing user requests, node execution plans for traditional services and LLM services are dynamically generated, and then scheduled in parallel to ultimately organize the complete output results.
It achieves efficient collaborative scheduling of traditional services and LLM services, can dynamically adapt to user needs, and improves scheduling efficiency and response speed.
Smart Images
Figure CN121636104A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of large model scheduling, in particular to a hybrid arrangement method of traditional services and LLM services. BACKGROUND
[0002] With the rapid development of large language model (LLM) technology, such as the emergence of platforms like Baidu Qianfan AppBuilder and OpenAI GPT series, enterprises are eager to integrate the powerful LLM capabilities with existing stable running traditional services (such as database query, business logic processing, API calling, etc.) to build an intelligent hybrid arrangement system. Currently, the closest existing technical solutions mainly have two types:
[0003] (1) Static parallel service aggregation mode: This mode receives requests through a central controller (such as a backend API server) and pre-encodes the list of traditional services to be called and their parameter mapping logic in the code. The controller calls these traditional services in parallel, aggregates the results, and may directly return or pass them to the LLM service for subsequent processing. Although this mode can achieve parallelism between services, its flow, parameters, and calling targets are static and fixed, and cannot adapt to dynamic needs based on natural language input.
[0004] (2) LLM as central dispatcher mode: This mode uses the understanding and planning capabilities of LLM to analyze user requests directly. LLM generates an execution plan (such as using ReAct, Tool Use, etc.) to determine which tools (i.e. traditional services) need to be called, and then the executor parses the LLM output and calls the corresponding tools. Finally, the execution results of the tools are summarized and may be handed over to the LLM again for summary, and the final result is returned.
[0005] However, in actual implementation, the inventors found that this type of technical solution arranges through LLM services or fixed processes, which can lead to poor planning results and cannot meet user dynamic needs, or the external LLM service takes too long to analyze and schedule business processes. SUMMARY
[0006] To solve the above problems in the prior art, a hybrid arrangement method of traditional services and LLM services is provided.
[0007] The specific technical solution is as follows:
[0008] A hybrid arrangement method of traditional services and LLM services, comprising:
[0009] Step S1: After obtaining a user request input by a user, a node execution plan is generated according to the semantics of the user request.
[0010] Step S2: calling the traditional service and the LLM service respectively based on the node execution plan to obtain service return results;
[0011] Step S3: aggregating according to the service return results to obtain an output result.
[0012] In another aspect, the step S1 comprises:
[0013] Step S11: parsing the user request to obtain a request semantic, extracting a key parameter and context information associated with the key parameter from the request semantic;
[0014] Step S12: sequentially generating corresponding processing nodes according to the key parameter;
[0015] Step S13: organizing the processing nodes according to the context information to obtain the node execution plan.
[0016] In another aspect, the step S11 comprises:
[0017] Step S111: forwarding the user request to the LLM service and extracting the request semantic based on the LLM service;
[0018] Step S112: matching the key parameter from the request semantic;
[0019] Step S113: returning the key parameter to the LLM service to obtain the context information.
[0020] In another aspect, the step S12 comprises:
[0021] Step S121: respectively matching the key parameter to determine whether the key parameter can be obtained in the traditional service, and dividing into a traditional service processing parameter and an LLM service processing parameter;
[0022] Step S122: generating an API calling instruction and establishing the processing node for the traditional service processing parameter, and generating a prompt word and establishing the processing node for the LLM service processing parameter.
[0023] In another aspect, the step S13 comprises:
[0024] Step S131: judging the dependency relationship between the processing nodes according to the context information;
[0025] Step S132: sorting the processing nodes according to the dependency relationship, thereby obtaining the node execution plan.
[0026] In another aspect, the step S2 comprises:
[0027] Step S21: obtaining the next processing node to be executed according to the node execution plan;
[0028] Step S22: executing the processing node and determining whether the execution is completed;
[0029] Step S23: returning to the step S21 to obtain the next processing node until the node execution plan is executed completely.
[0030] In another aspect, in the step S21, the processing nodes without dependency relationship are extracted and executed in parallel.
[0031] In another aspect, the step S3 comprises:
[0032] Step S31: generating a text organization template according to the context information and the semantics;
[0033] Step S32: obtaining the service return result and adding the service return result to the text organization template to output.
[0034] The above technical solution has the following advantages or beneficial effects:
[0035] In view of the problem that the scheduling flow of the LLM service in the prior art is poor in efficiency, in the present solution, the semantics of the user request is analyzed, the node execution plan for scheduling the traditional service and the LLM service respectively is dynamically generated, and then parallel scheduling of the traditional service and the LLM service is respectively realized, and finally a complete output result is organized. BRIEF DESCRIPTION OF DRAWINGS
[0036] Reference will now be made to the drawings to describe the embodiments of the present application in more detail. However, the accompanying drawings are only used for illustration and explanation, and do not constitute a limitation on the scope of the present application.
[0037] Figure 1 is a whole schematic diagram of an embodiment of the present application;
[0038] Figure 2 is a schematic diagram of step S1 in an embodiment of the present application;
[0039] Figure 3 is a schematic diagram of step S11 in an embodiment of the present application;
[0040] Figure 4 is a schematic diagram of step S12 in an embodiment of the present application;
[0041] Figure 5 is a schematic diagram of step S13 in an embodiment of the present application;
[0042] Figure 6 This is a schematic diagram of step S2 in an embodiment of the present invention;
[0043] Figure 7 This is a schematic diagram of step S3 in an embodiment of the present invention. Detailed Implementation
[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0045] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.
[0046] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.
[0047] This invention includes:
[0048] A hybrid orchestration approach combining traditional services and LLM services, such as Figure 1 As shown, it includes:
[0049] Step S1: After obtaining the user's input request, generate a node execution plan according to the semantics of the user request;
[0050] Step S2: Based on the node execution plan, call the traditional service and the LLM service respectively to obtain the service return results;
[0051] Step S3: Aggregate the results returned by the service to obtain the output results.
[0052] Specifically, to address the issue of poor efficiency in the scheduling process of LLM services in existing technologies, this solution parses the semantics of user requests and dynamically generates node execution plans for scheduling traditional services and LLM services respectively. This enables parallel scheduling of traditional services and LLM services, ultimately resulting in a complete output.
[0053] Specifically, the aforementioned hybrid orchestration method is mainly configured as a software implementation in computer devices, such as a backend server, to receive user requests and respond to and return corresponding information in practical applications.
[0054] The backend server connects to multiple traditional services, which refer to various database query services, various local or external APIs, etc. These backend services return query results according to the corresponding message format by inputting query content or conditions through specific structured instructions.
[0055] Furthermore, the system also integrates an LLM service. The LLM service is a backend service built on a pre-tuned, large-scale language model, either externally or locally. It can understand unspecified user input and return corresponding responses. Depending on the model configuration, it can also automatically invoke relevant APIs to query external sources. The LLM service is invoked through a local LLM gateway.
[0056] Based on the above settings, the system will obtain user instructions through the front end. For example, the user can input unspecified text content as a user request through an AI assistant or other equivalent means. For instance, the user can input "I want to go to a warm place for vacation tomorrow" into a travel app.
[0057] For this part of the content, the backend will extract its semantics in advance and perform word segmentation to obtain key parameters. For example, the user request includes at least information such as date, number of people associated, weather, travel plan and destination.
[0058] Regarding the above information, the backend service contains some information that can be retrieved through traditional services, such as available travel options and booking information for the corresponding dates in the ticketing database, as well as information that can be extracted from fixed API interfaces, such as weather information for various regions.
[0059] As for the vacation destination, since it is a vague semantic term, it is retrieved through LLM services to obtain suitable vacation locations, and then returned to traditional services to assist in finding information such as weather and tickets.
[0060] The aforementioned process of searching and retrieving information is divided into several processing nodes. Each processing node records the services that need to be retrieved and the corresponding search instructions. Furthermore, considering the relationships between some processed information—for example, the possible destinations should first be determined through the LLM service before searching for information such as weather and ticketing—the execution nodes are arranged according to these relationships to form a node execution plan.
[0061] Based on the node execution plan, the LLM service and traditional service are called sequentially to obtain the required information. This information is then aggregated to form the output content and returned to the user, thereby realizing the collaborative work of traditional services and LLM services.
[0062] In one embodiment, such asFigure 2 As shown, step S1 includes:
[0063] Step S11: Parse the user request to obtain the request semantics, and extract the key parameters and contextual information associated with the key parameters from the request semantics;
[0064] Step S12: Generate the corresponding processing nodes in sequence according to the key parameters;
[0065] Step S13: Organize the processing nodes according to the context information to obtain the node execution plan.
[0066] Specifically, in order to achieve a better orchestration effect for processing nodes, in this embodiment, the user request is first parsed to obtain the request semantics, and the keywords mentioned in the request semantics are matched to obtain key parameters.
[0067] For each key parameter, the request semantics are segmented, and then forward and backward searches are performed to obtain context information of a certain length corresponding to the key parameter.
[0068] Based on this, the semantics corresponding to the key parameters determine whether to perform the search using traditional services or obtain feedback through LLM services, and corresponding query instructions or prompts are added according to the service type to assemble the processing nodes.
[0069] Finally, the dependencies between processing nodes are determined based on context information, and then the logical order of the processing nodes is rearranged to obtain the actual node execution plan.
[0070] In one embodiment, such as Figure 3 As shown, step S11 includes:
[0071] Step S111: Forward the user request to the LLM service and extract the request semantics based on the LLM service;
[0072] Step S112: Obtain key parameters by matching from the request semantics;
[0073] Step S113: Return the key parameters to the LLM service to obtain context information.
[0074] Specifically, in order to achieve better extraction of key parameters, in this embodiment, a semantic parsing instruction is first sent to the execution engine and the original user request is sent to the context manager. After receiving the instruction, the execution engine calls the LLM gateway, which compiles prompt words for the user request and forwards them to the LLM service. This allows the LLM service to perform semantic understanding and logical deduction on the questions contained in the user request, forming request semantics corresponding to the user's actual needs.
[0075] After obtaining the request semantics, the LLM gateway validates the returned format and sends it to the context manager. Then, it uses dictionary matching or semantic lookup to find relevant content within the request semantics as key parameters, such as date, entity name, and action. Finally, the key parameters are returned to the LLM service to match the original user request and obtain the context information associated with each key parameter.
[0076] In one embodiment, such as Figure 4 As shown, step S12 includes:
[0077] Step S121: Match the key parameters separately to determine whether the key parameters can be obtained in the traditional service, and divide them into traditional service processing parameters and LLM service processing parameters;
[0078] Step S122: For traditional service processing parameters, generate API call instructions and establish processing nodes; for LLM service processing parameters, generate prompt words and establish processing nodes.
[0079] Specifically, to achieve better task construction for processing nodes, in this embodiment, after extracting key parameters, dictionary matching is performed on the key parameters. This dictionary matching process follows pre-configured local traditional services. For example, if there is a corresponding ticketing database and pre-connected weather and map APIs, corresponding descriptive dictionaries such as "weather" and "climate" are compiled for each traditional service. After obtaining the key parameters, the dictionaries compiled according to each traditional service are matched one by one to determine whether there are key parameters that can be processed by the traditional services.
[0080] If a corresponding hit dictionary does not exist, the critical parameter is considered to need to be handled by an external LLM service. Based on this process, critical parameters are divided into traditional service processing parameters and LLM service processing parameters.
[0081] Subsequently, for the traditional service processing parameters, the corresponding traditional service to be called is determined according to the dictionary it hits. Each traditional service is pre-configured with a corresponding query template and semantic extraction model. Based on the semantic extraction model, descriptive variables such as specific dates and locations are extracted from key parameters and then assembled into the query template to form corresponding query instructions, API call statements, etc., as processing nodes, and the corresponding text return format is determined.
[0082] Similarly, for the LLM service processing parameters, their semantics are extracted, and then the corresponding prompt word templates are obtained and assembled to obtain the query statement as the processing node. The corresponding text return format is agreed upon for the LLM service.
[0083] In one embodiment, such as Figure 5 As shown, step S13 includes:
[0084] Step S131: Determine the dependencies between processing nodes based on the context information;
[0085] Step S132: Sort the processing nodes according to their dependencies to obtain the node execution plan.
[0086] Specifically, in order to achieve a better sorting effect for nodes, in this embodiment, after constructing the processing nodes, the pre-extracted context information is first extracted for each processing node. Then, the context information is parsed for part-of-speech tags, mainly extracting modifiers and adverbs corresponding to key parameters as the modification information of the nodes. At the same time, the original input user request is segmented according to the context information, and then the location of the key parameters is located.
[0087] Based on location and modification information, the position of each key parameter in the user request and the related text content can be determined. Based on this information, the dependencies between processing nodes can be determined. Then, the processing nodes are sorted according to the dependencies to obtain the node execution plan.
[0088] In one embodiment, such as Figure 6 As shown, step S2 includes:
[0089] Step S21: Obtain the next processing node to be executed according to the node execution plan;
[0090] Step S22: Execute the processing node and determine whether the execution is complete;
[0091] Step S23: Return to step S21 to obtain the next processing node until the node execution plan is completed.
[0092] Specifically, in order to effectively obtain relevant information, in this embodiment, after establishing the node execution plan, the node execution plan is executed sequentially through the node engine to obtain the processing results returned by each processing node, including the results returned by the LLM service and the results returned by the traditional service, and then the next node is executed until all nodes are completed.
[0093] In one embodiment, in step S21, processing nodes that do not have dependencies are extracted and executed in parallel.
[0094] Specifically, to improve processing efficiency, in this embodiment, during the configuration of node execution plans, nodes that do not have dependencies are extracted. When the node execution plan is started, nodes that do not have dependencies are added to the parallel processing queue for processing, thereby improving processing efficiency.
[0095] In one embodiment, such as Figure 7 As shown, step S3 includes:
[0096] Step S31: Generate a text organization template based on contextual information and request semantics;
[0097] Step S32: Obtain the service return results and add them to the text organization template before outputting.
[0098] Specifically, in order to achieve a better output effect for the results returned by multiple services, this embodiment determines the type of content to be returned to the user based on the pre-extracted request semantics. At the same time, it further determines the framework of the specific content to be returned to the user based on the context information, including the content to be returned, the relationship between the content, etc. On this basis, a text organization template is generated according to the framework content, and the service return results are entered into the template to form a complete output content to be returned to the user.
[0099] Those skilled in the art will understand that various aspects, or possible implementations of various aspects, of the present invention can be embodied as systems, methods, or computer program products. Therefore, various aspects, or possible implementations of various aspects, of the present invention can take the form of entirely hardware embodiments, entirely software embodiments (including firmware, resident software, etc.), or embodiments combining software and hardware aspects, all collectively referred to herein as "circuit," "module," or "system." Furthermore, various aspects, or possible implementations of various aspects, of the present invention can take the form of computer program products, which are computer instructions stored in memory.
[0100] The memory can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof, such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable read-only memory (CD-ROM).
[0101] A processor in a computer reads computer instructions stored in memory, enabling the processor to execute the functional actions specified in each step or combination of steps in a flowchart; and to generate means for implementing the functional actions specified in each block or combination of blocks in a flowchart.
[0102] It should be understood that a processor in a computer can be understood as one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components used to execute the aforementioned computer instructions.
[0103] Computer instructions may be executed entirely on the user's local computer, partially on the user's local computer, as a separate software package, partially on the user's local computer and partially on a remote computer, or entirely on a remote computer or server. It should also be noted that in some alternative implementations, the functions indicated by the steps in the flowchart or the blocks in the block diagram may not occur in the order shown in the diagram. For example, depending on the functions involved, two consecutive steps or blocks may actually be executed approximately simultaneously, or these blocks may sometimes be executed in reverse order.
[0104] Of course, in practical applications, the various components of a computer system are coupled together through a bus system. The bus system is used to enable communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus.
[0105] The above are merely preferred embodiments of the present invention and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should recognize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.
Claims
1. A hybrid orchestration method of traditional services and LLM services, characterized in that, The method comprises the following steps: Step S1: generating a node execution plan according to the semantics of a user request after obtaining the user request input by a user; Step S2: calling a traditional service and an LLM service respectively based on the node execution plan to obtain service return results; Step S3: performing aggregation according to the service return results to obtain an output result.
2. The hybrid orchestration method of claim 1, wherein, The step S1 comprises: Step S11: parsing the user request to obtain request semantics, extracting a key parameter and context information associated with the key parameter from the request semantics; Step S12: generating corresponding processing nodes in sequence according to the key parameter; Step S13: organizing the processing nodes according to the context information to obtain the node execution plan.
3. The hybrid orchestration method of claim 2, wherein, The step S11 comprises: Step S111: forwarding the user request to the LLM service and extracting the request semantics based on the LLM service; Step S112: matching the key parameter from the request semantics; Step S113: returning the key parameter to the LLM service to obtain the context information.
4. The hybrid orchestration method of claim 2, wherein, The step S12 comprises: Step S121: matching the key parameter respectively to determine whether the key parameter can be obtained in the traditional service, and dividing the key parameter into a traditional service processing parameter and an LLM service processing parameter; Step S122: generating API calling instructions and establishing the processing node for the traditional service processing parameter, and generating a prompt word and establishing the processing node for the LLM service processing parameter.
5. The hybrid orchestration method of claim 2, wherein, The step S13 comprises: Step S131: judging the dependency relationship between the processing nodes according to the context information; Step S132: sorting the processing nodes according to the dependency relationship to obtain the node execution plan.
6. The hybrid orchestration method of claim 1, wherein, The step S2 comprises: Step S21: obtaining the next processing node to be executed according to the node execution plan; Step S22: executing the processing node and judging whether the execution is completed; Step S23: returning to step S21 to obtain the next processing node until the node execution plan is executed completely.
7. The hybrid orchestration method of claim 6, wherein, In the step S21, the processing nodes without dependency relationship are extracted and executed in parallel.
8. The hybrid orchestration method of claim 2, wherein, The step S3 comprises: Step S31: generating a text organization template according to the context information and the request semantics; and Step S32: obtaining the service return results and adding the service return results to the text organization template to output.