Robot tour guide task planning method and system based on multi-modal large model

Abstract

The application discloses a robot guide interpretation task planning method and system based on a multimodal large model, which comprises the following steps: registering dynamic environment resources and tool components relied on by guide interpretation task data to a unified MCP server, and allocating unique identifiers and defining parameter formats; acquiring robot positioning information and guide task instructions, and analyzing and generating task execution logic containing points and manuals; collecting multimodal sensing data such as images and voices in real time, encapsulating the multimodal sensing data and the task execution logic into a planning task through an MCP protocol, and initiating a request to an LLM server; determining a context type according to the logic by the LLM, calling resources or tool components through the MCP to generate inference results, converting the results into a PDDL format file, searching for an output global ordered action sequence by using a PDDL planner, adapting the PDDL symbolic task to an MCP physical world task through a rule mapping library by an MCP parser, and encapsulating the MCP physical world task into an instruction, which is executed by an underlying execution tool in an MCP client and monitored.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a robot tour guide and explanation task planning method and system based on a multimodal large model. Background Technology

[0002] With the deepening of the construction of "smart cultural tourism" and "smart museums", major museums and exhibition halls have put forward higher requirements for improving the visitor experience and management efficiency. Guide robots have become key equipment to alleviate the shortage of guide resources during holidays and provide intelligent services.

[0003] Current research on tour guide robots mainly focuses on two dimensions: underlying path planning and basic human-computer interaction. In the field of path planning, existing technologies, such as patent applications CN118936500A and CN110044359A, propose solutions based on improved RRT*, ant colony algorithm, DWA algorithm, and a fusion of artificial potential field method and PRM algorithm, respectively. When faced with complex tour guide tasks, these methods fail to understand the semantic logic behind the task and struggle to dynamically adjust the tour order and dwell time at points based on the depth of the content and real-time feedback from visitors, resulting in a logical disconnect between action and explanation. Regarding human-computer interaction, patent CN120422235A proposes an interaction system utilizing the ROS framework, a finite state machine multi-turn dialogue engine, and the iFlytek recognition engine. Although this system improves the stability of the dialogue through state machines, the state machine architecture based on preset scripts is too rigid when dealing with open-ended questions and personalized explanation needs.

[0004] Regarding the interconnection between AI models and robot control, patent application CN120791784A explores a semantic mapping method based on the Model Context Protocol (MCP), which improves the security and real-time performance of instruction transmission through security tokens and adaptive protocol channels. However, this solution mainly focuses on the heterogeneous conversion and security verification of underlying instructions, and does not fully leverage the potential of the MCP protocol in tool invocation and context sharing. Patent CN 120932849A proposes an edge-cloud collaborative service system. This system deploys a lightweight edge model locally on the robot and a heavyweight model in the cloud, and obtains user health data, uploads cloud prompts, and controls the robot's functional nodes to execute actions through the MCP service process. However, this solution mainly relies on confidence thresholds for task allocation, and the generated tasks are usually unstructured instructions directly output by the large model. In complex navigation scenarios, this black-box generation method is prone to logical illusions and lacks a rigorous symbolic logic planning layer to ensure the global orderliness and physical feasibility of long-term task action sequences. Patent application CN 120785713 A discloses a heterogeneous device intelligent management system and method based on artificial intelligence technology. The technical solution includes establishing a hierarchical management framework, converting the communication protocols and data formats of different devices into Model Context Protocols (MCPs) through a device adaptation layer, and abstracting device capabilities using a unified description language for heterogeneous devices. Simultaneously, the system uses a large language model to decompose natural language instructions into atomic tasks and combines knowledge graphs and multi-agent reinforcement learning algorithms to generate scheduling paths. However, this solution is primarily positioned as a general-purpose heterogeneous hardware management platform. Its core logic focuses on the dynamic allocation of computing resources (such as CPU and GPU computing power) and protocol compatibility conversion, lacking a motion logic planning mechanism for specific explanation tasks of the guide robot. Since its task scheduling mainly relies on device matching matrices and reinforcement learning, it is difficult to ensure the rigor and global orderliness of the robot's action sequences in scenarios with complex long-range logical constraints (such as the deep coupling between navigation point order optimization and explanation content). Furthermore, this approach cannot effectively correct for the illusions or uncertainties that may arise in the decomposition of large models into complex tasks, resulting in insufficient reliability and robustness of the system when handling tour guide tasks with high logical determinism requirements. Summary of the Invention

[0005] Purpose of the invention: In order to overcome the shortcomings of the existing technology, the present invention provides a robot tour guide and explanation task planning method and system based on a multimodal large model, which can integrate multimodal perception data, schedule dynamic resources using a unified protocol, and ensure the global orderly execution of complex tasks through rigorous symbolic logic.

[0006] Technical Solution: To achieve the above objectives, the present invention provides a robot guided tour and explanation task planning method based on a multimodal large model, the method comprising:

[0007] The dynamic environment resources and tool components on which the guided tour task data depends are registered in a unified MCP server, and a unique identifier is assigned to each resource. At the same time, the input and output parameter formats of the tool components are defined. The tool components are registered in the logical layer of the MCP server, and the tool components correspond to the preset underlying execution tools in the execution layer.

[0008] The system acquires the robot's positioning information and tour guide instructions, parses the instructions to generate task execution logic, specifically, based on the instructions and tour guide data, including exhibition hall locations and brochures; it also collects multimodal perception data during tours in real time, including image and audio data; and encapsulates the task execution logic and the multimodal perception data into a planning task using the MCP protocol, and sends an inference request to the Large Language Model (LLM) server.

[0009] The LLM server determines the required context type (location information, exhibit location, explanation content, interactive skills, etc.) based on the task execution logic, queries and calls the tool components or resources in the MCP server through the MCP protocol, and performs inference in combination with the multimodal perception data to generate inference results that include task objectives, environmental constraints, and robot states; specifically, the interactive content includes robot full-body motion command information, robot navigation and obstacle avoidance task command information, robot real-time explanation content generation, and robot interactive content generation.

[0010] The reasoning results are converted into PDDL format files, and a heuristic search is performed using a PDDL planner to output a globally ordered sequence of actions.

[0011] The MCP parser adapts the PPL symbolic tasks in the action sequence into MCP physical world tasks through a rule mapping library, instantiates and encapsulates them into MCP instructions, and executes the MCP instructions and monitors the status through the underlying execution tools in the MCP client.

[0012] Furthermore, the dynamic environmental resources and tools include a vector map of the exhibition hall, a JSON knowledge base of exhibits, and real-time sensor data streams; the tool components include one or more of the following tools: path planning tools, knowledge query tools, speech synthesis tools, body movement control tools, and human face tracking tools.

[0013] Furthermore, if no guidance task instruction is received, the robot will intermittently introduce itself and wait to be woken up.

[0014] Furthermore, the process of converting the inference results into a PDDL format file, using a PDDL planner for heuristic search, and outputting a globally ordered sequence of actions includes:

[0015] The LLM module structures the inference results into a domain description file (.domain) and a problem description file (.problem) that conform to the PDDL standard. The domain file defines a general action model (such as move_to, describe) in the exhibition hall environment, while the problem file specifies the initial state and target state of the current task.

[0016] The domain description file and problem description file are input into a PDDL planner (such as Fast-Downward), which performs symbol-based automatic reasoning and heuristic search based on the defined action logic and state transition rules.

[0017] When the PDDL planner solves the problem successfully, it outputs a sequence of actions consisting of basic actions.

[0018] When the PDDL planner fails to solve the problem or times out, it feeds back the error type as a new context to the LLM module. The LLM module analyzes the reasons for the failure based on the feedback, adjusts the task decomposition method, and regenerates the PDDL problem file until a feasible sequence of actions is obtained.

[0019] Furthermore, the process of adapting the PPPL symbolic tasks in the action sequence into MCP physical world tasks through the MCP parser using a rule mapping library, and instantiating and encapsulating them into MCP instructions, specifically includes:

[0020] Extract the operators and logical parameters of each action in the action sequence, and identify whether the action is a compound action based on the action attributes. If so, decompose the compound action into a subsequence composed of multiple basic atomic operations according to preset rules.

[0021] For each of the aforementioned operations or basic atomic operations and their corresponding logical parameters, a search and matching process is performed in the registered MCP resources, and the logical parameters are mapped to specific physical feature parameters or hardware interface instruction IDs.

[0022] According to the JSON-RPC format corresponding to the MCP protocol, the extracted operators are encapsulated into tool invocation methods, and the matched physical feature parameters or hardware interface instruction IDs are encapsulated into corresponding tool invocation parameters to generate the MCP instruction.

[0023] Furthermore, the step of executing the MCP instructions and monitoring the status through the underlying execution tools in the MCP client specifically includes:

[0024] The MCP client sends the encapsulated JSON-RPC format request to the MCP server that provides exhibition hall services;

[0025] The MCP server calls the underlying execution tool to execute instructions. The underlying execution tool acts as an actuator to drive the robot's hardware to perform physical actions or operations, and returns the execution result, including success, failure, or progress, to the MCP client through the MCP protocol. The execution result includes success, failure, and specific progress.

[0026] Furthermore, the method also includes: the MCP client maintaining the robot state machine based on the received feedback information; if the execution result is successful, the state machine updates the robot's current world state and triggers the next action in the action sequence; if the execution result is blocked or failed, the action sequence is immediately paused, and the state machine uses the exception information containing the error type as a new context, and feeds it back to the upstream LLM server or replanning module through the MCP protocol. The LLM can then decide to retry or generate a new PDDL problem description based on this new context to trigger local replanning.

[0027] A robot guided tour and narration task planning system based on a multimodal large model is used to implement the aforementioned robot guided tour and narration task planning system based on a multimodal large model. The system includes:

[0028] The perception and input layer is used to acquire location information, multimodal perception data, and navigation task instructions;

[0029] The cognition and planning layer, which includes an LLM server, an MCP server, a PDDL planner, and a PDDL-MCP adapter, is used to realize the logical generation and mapping from natural language instructions to action sequences.

[0030] The execution and output layer, which includes the MCP client and robot actuator, is used to receive MCP requests and drive the robot to complete navigation, voice and interactive actions.

[0031] An exception handling loop is used to monitor the execution status in real time and feed back exception information to the cognition and planning layer in a closed loop.

[0032] Beneficial Effects: The robot guided tour and explanation task planning method and system based on a multimodal large model of the present invention has the following beneficial effects:

[0033] (1) The method of the present invention achieves dynamic planning and execution of the tour guide task through the collaboration of multimodal large model and MCP protocol, effectively improving the robot's adaptability to complex environment and task flexibility, enhancing the accuracy and interactivity of the explanation content, and ensuring global optimization of action sequence through PDDL planning, thereby improving task execution efficiency and intelligence level.

[0034] (2) Compared with existing technologies such as CN120791784A, the MCP protocol plays a central role in the entire method of this invention. On the one hand, it enables the LLM to call tools and data resources as needed during the inference process, thereby improving the accuracy of semantic understanding and task planning. On the other hand, the MCP protocol serves as a bridge between the LLM and the physical execution layer, mapping the abstract action sequence generated by PDDL planning into specific executable instructions through the MCP parser, and driving the robot's various actuators to complete composite tasks such as navigation and explanation through the MCP client. Throughout the process, MCP not only realizes dynamic registration, secure scheduling, and standardized communication of resources, but also solves the problems of difficult fusion of multi-source heterogeneous data and high module coupling through unified protocol encapsulation and context management, thereby enhancing the scalability, real-time performance, and robustness of the system. At the same time, with the help of the bidirectional communication capability of MCP, the system can provide real-time feedback on the execution status and support dynamic replanning, realizing a closed-loop intelligent navigation of "perception-decision-execution-feedback".

[0035] (3) By converting the LLM inference results into PDDL symbolic logic, heuristic search is used to ensure the global orderliness and logical rigor of the action sequence of long-term tasks. On the one hand, the determinism of symbolic programming compensates for the instability of the output of large models. On the other hand, the constructed closed-loop replanning mechanism enables the system to dynamically adjust the task decomposition for solution failures, thereby improving the robot's success rate in handling logical constraints and abnormal interference in complex tour scenarios.

[0036] (4) By identifying and disassembling, parameter mapping, protocol encapsulation and distribution execution, a rigorous conversion from high-level abstract logic to low-level physical instructions is achieved, ensuring the precision and accuracy of task execution; at the same time, the parameter binding mechanism eliminates the difference between the logical context and the low-level hardware identifier, making the system highly adaptable to heterogeneous hardware. Attached Figure Description

[0037] Figure 1 A flowchart illustrating the task planning method for robot-guided tours based on a multimodal large model;

[0038] Figure 2 This is a sample code block;

[0039] Figure 3This is a system architecture diagram for a robot-guided tour planning system based on a multimodal large model. Detailed Implementation

[0040] The invention will now be further described with reference to the accompanying drawings.

[0041] like Figure 1 The robot guided tour task planning method shown is based on a multimodal large model. The method includes:

[0042] Step S101: Register the dynamic environment resources and tool components on which the guided tour task data depends to a unified MCP server, assign a unique identifier to each resource, and define the input and output parameter formats of the tool components; wherein, the tool components are registered in the logical layer of the MCP server, and the tool components correspond to preset underlying execution tools in the execution layer;

[0043] Step S102: Obtain the robot's positioning information and tour guide task instructions; parse the tour guide task instructions to generate task execution logic. Specifically, the task execution logic is obtained based on the tour guide task instructions and tour guide explanation task data, which includes exhibition hall explanation points and explanation content manuals; collect multimodal perception data in real time during tour explanation, which includes image data and voice data; encapsulate the task execution logic and the multimodal perception data into a planning task through the MCP protocol, and initiate an inference request to the Large Language Model (LLM) server;

[0044] In step S103, the LLM server determines the required context type based on the task execution logic, queries and calls the tool components or resources in the MCP server through the MCP protocol, and performs inference in combination with the multimodal perception data to generate inference results containing task objectives, environmental constraints, and robot states; specifically, the interactive content includes robot full-body motion command information, robot navigation and obstacle avoidance task command information, robot real-time explanation content generation, and robot interactive content generation.

[0045] Step S104: Convert the reasoning result into a PDDL format file, use the PDDL planner to perform heuristic search, and output a globally ordered sequence of actions.

[0046] Step S105: The PPL symbolic tasks in the action sequence are adapted into MCP physical world tasks through the rule mapping library by the MCP parser, instantiated and encapsulated as MCP instructions, and the MCP instructions are executed and the status is monitored by the underlying execution tool in the MCP client.

[0047] The method of this invention achieves dynamic planning and execution of tour guiding tasks through the collaboration of a multimodal large model and the MCP protocol, effectively improving the robot's adaptability to complex environments and task flexibility, enhancing the accuracy and interactivity of the explanation content, and ensuring global optimization of action sequences through PDDL planning, thereby improving task execution efficiency and intelligence level.

[0048] The MCP protocol plays a central role in the entire methodology. On one hand, it enables the LLM to call upon tools and data resources as needed during inference, improving the accuracy of semantic understanding and task planning. On the other hand, the MCP protocol acts as a bridge between the LLM and the physical execution layer, mapping the abstract action sequences generated by PDDL planning into specific executable instructions via the MCP parser. These instructions then drive the robot's various actuators to complete complex tasks such as navigation and explanation through the MCP client. Throughout this process, MCP not only achieves dynamic resource registration, secure scheduling, and standardized communication, but also solves the problems of difficult multi-source heterogeneous data fusion and high module coupling through unified protocol encapsulation and context management, enhancing the system's scalability, real-time performance, and robustness. Simultaneously, leveraging MCP's bidirectional communication capabilities, the system can provide real-time feedback on execution status and support dynamic replanning, achieving a closed-loop intelligent navigation system of "perception-decision-execution-feedback."

[0049] Preferably, the dynamic environmental resources and tools mentioned in step S101 above include a vector map of the exhibition hall, a JSON knowledge base of exhibits, and a real-time sensor data stream; the tool components include one or more of the following tools: path planning tools, knowledge query tools, speech synthesis tools, body movement control tools, and human face tracking tools.

[0050] Preferably, if no tour guide task instruction is obtained in step S102 above, the robot intermittently introduces itself and waits to be woken up.

[0051] Preferably, the step S105 above, which involves converting the reasoning result into a PDDL format file, using a PDDL planner for heuristic search, and outputting a globally ordered sequence of actions, includes the following steps S201-S204:

[0052] Step S201: The LLM module converts the inference results into a structured domain description file (.domain) and a problem description file (.problem) that conform to the PDDL standard. The domain file defines a general action model in the exhibition hall environment, such as (move_to, describe), while the problem file specifically specifies the initial state and target state of the current task.

[0053] Step S202: Input the domain description file and problem description file into the PDDL planner. The PDDL planner performs symbol-based automatic reasoning and heuristic search based on the defined action logic and state transition rules.

[0054] Step S203: When the PDDL planner solves the problem successfully, it outputs an action sequence consisting of basic actions.

[0055] Step S204: When the PDDL planner fails to solve the problem or times out, the error type is fed back to the LLM module as a new context. The LLM module analyzes the cause of failure based on the feedback, adjusts the task decomposition method, and regenerates the PDDL problem file until a feasible action sequence is obtained.

[0056] Each action is formatted as (operator parameter object 1 parameter object 2 ...), and one possible output sequence is as follows:

[0057] 1. (move_to robot1 zone_a) # Robot 1 moves to zone a;

[0058] 2. (describe_exhibit robot1 exhibit_123) # Robot1 describes exhibit 123;

[0059] 3. (navigate_to robot1 zone) #Robot 1 navigates to the target area;

[0060] 4. (answer_question robot1 "production year") #The robot answers the question about "production year".

[0061] By transforming LLM inference results into PDDL symbolic logic, heuristic search is used to ensure the global orderliness and logical rigor of long-term task action sequences. On the one hand, the determinism of symbolic programming compensates for the instability of large model outputs. On the other hand, the constructed closed-loop replanning mechanism enables the system to dynamically adjust task decomposition in response to solution failures, improving the robot's task execution success rate in complex navigation scenarios where it faces logical constraints and abnormal interference.

[0062] Preferably, the step S105 above, which involves adapting the PPPL symbolic task in the action sequence to an MCP physical world task through a rule mapping library using an MCP parser, and instantiating and encapsulating it as an MCP instruction, specifically includes the following steps S301-S302:

[0063] Step S301: Extract the operators and logical parameters of each action in the action sequence, and identify whether the action is a compound action based on the action attributes. If so, decompose the compound action into a subsequence composed of multiple basic atomic operations according to preset rules.

[0064] In this step, for example, the action (guide_to robot1 visitor1 exhibit_123) needs to be broken down into: find_visitor (locate the visitor), calculate_path (calculate the path), and move_along_path (move along the path), and then called by the corresponding MCP tools in subsequent steps.

[0065] Step S302: For each of the operations or basic atomic operations and their corresponding logical parameters, search and match in the registered MCP resources, and map the logical parameters to specific physical feature parameters or hardware interface instruction IDs.

[0066] In this step, for example, for (move_to robot1 zone_a), the converter needs to query the "showroom layout resources" and bind the logical location (zone_a) to specific physical coordinates or navigation point IDs (such as "gps_coord_101" or "nav_point_entrance").

[0067] Step S303: According to the JSON-RPC format corresponding to the MCP protocol, the extracted operators are encapsulated into tool call methods, and the matched physical feature parameters or hardware interface instruction IDs are encapsulated into corresponding tool call parameters. A tools / call request is encapsulated to generate the MCP instruction. Figure 2 This is an example code block that requests that the robot with ID robot1 perform a navigation task to go to a preset entry navigation point.

[0068] By employing a progressive logic of identification, decomposition, parameter mapping, protocol encapsulation, distribution, and execution, a rigorous transformation from high-level abstract logic to low-level physical instructions is achieved, ensuring the precision and accuracy of task execution. At the same time, the parameter binding mechanism eliminates the differences between the logical context and the underlying hardware identifier, giving the system strong heterogeneous hardware adaptability.

[0069] Preferably, the step S105 above, which involves executing the MCP instruction and monitoring the status through the underlying execution tool in the MCP client, specifically includes the following steps S401-S403:

[0070] Step S401: The MCP client sends the encapsulated JSON-RPC format request to the MCP server providing the exhibition hall service;

[0071] In step S402, the MCP server calls the underlying execution tool (such as robot navigation SDK, speech synthesis engine, knowledge base query interface, and limb movement interface, etc.) to execute instructions. The underlying execution tool acts as an actuator to drive the robot's hardware to perform physical actions or operations, and returns the execution result, including success, failure, or progress, to the MCP client through the MCP protocol. The execution result includes success, failure, and specific progress.

[0072] Preferably, the method further includes: the MCP client maintaining the robot state machine based on the received feedback information: if the execution result is successful, the state machine updates the robot's current world state (e.g., "the robot is in area A") and triggers the next action in the action sequence; if the execution result is blocked or failed (e.g., the path is blocked, the exhibit sensor does not respond), the action sequence is immediately paused, and the state machine uses the error-type exception information (e.g., "navigation timeout") as a new context, and feeds it back to the upstream LLM server or replanning module through the MCP protocol. The LLM can then decide to retry or generate a new PDDL problem description based on this new context to trigger local replanning.

[0073] The above method utilizes the actuator's perception of the underlying state, combined with state machine management, to enable the system to transform the execution obstruction at the physical layer into a semantic context at the logical layer in real time. This breaks through the limitations of blind execution in traditional robots, giving the system the ability to self-diagnose and dynamically correct itself in complex exhibition hall environments, thus enhancing the continuity and robustness of the tour guiding task.

[0074] This invention also provides a robot guided tour task planning system based on a multimodal large model, used to implement the aforementioned robot guided tour task planning system based on a multimodal large model, such as... Figure 3 As shown, the system includes:

[0075] The perception and input layer is used to acquire location information, multimodal perception data, and navigation task instructions;

[0076] The cognition and planning layer, which includes an LLM server, an MCP server, a PDDL planner, and a PDDL-MCP adapter, is used to realize the logical generation and mapping from natural language instructions to action sequences.

[0077] The execution and output layer, which includes the MCP client and robot actuator, is used to receive MCP requests and drive the robot to complete navigation, voice and interactive actions.

[0078] An exception handling loop is used to monitor the execution status in real time and feed back exception information to the cognition and planning layer in a closed loop.

[0079] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A robot-guided tour planning method based on a multimodal large model, characterized in that, The methods include: Register the dynamic environment resources and tool components on which the guided tour task data depends to a unified MCP server, assign a unique identifier to each resource, and define the input and output parameter formats of the tool components. Obtain the robot's location information and navigation task instructions, and parse the navigation task instructions to generate task execution logic; Multimodal perception data is collected in real time during guided tours. The task execution logic and the multimodal perception data are encapsulated into a planning task through the MCP protocol, and an inference request is sent to the Large Language Model (LLM) server. The LLM server determines the required context type based on the task execution logic, queries and calls the tool components or resources in the MCP server through the MCP protocol, and performs inference in combination with the multimodal perception data to generate inference results containing task objectives, environmental constraints, and robot states. The reasoning results are converted into PDDL format files, and a heuristic search is performed using a PDDL planner to output a globally ordered sequence of actions. The MCP parser adapts the PPL symbolic tasks in the action sequence into MCP physical world tasks through a rule mapping library, instantiates and encapsulates them into MCP instructions, and executes the MCP instructions and monitors the status through the underlying execution tools in the MCP client.

2. The robot guided tour and explanation task planning method based on a multimodal large model according to claim 1, characterized in that, The dynamic environment resources and tools include a vector map of the exhibition hall, a JSON knowledge base of exhibits, and real-time sensor data streams; the tool components include one or more of the following: path planning tools, knowledge query tools, speech synthesis tools, body movement control tools, and human face tracking tools.

3. The robot guided tour and explanation task planning method based on a multimodal large model according to claim 1, characterized in that, If no tour guide instruction is received, the robot will intermittently introduce itself and wait to be woken up.

4. The robot guided tour and explanation task planning method based on a multimodal large model according to claim 1, characterized in that, The process of converting the inference results into a PDDL format file, using a PDDL planner for heuristic search, and outputting a globally ordered sequence of actions includes: The LLM module converts the inference results into a structured domain description file and a problem description file that conform to the PDDL standard. The domain file defines a general action model in the exhibition hall environment, while the problem file specifies the initial state and target state of the current task. The domain description file and problem description file are input into the PDDL planner, which performs symbol-based automatic reasoning and heuristic search based on the defined action logic and state transition rules. When the PDDL planner solves the problem successfully, it outputs a sequence of actions consisting of basic actions. When the PDDL planner fails to solve the problem or times out, it feeds back the error type as a new context to the LLM module. The LLM module analyzes the reasons for the failure based on the feedback, adjusts the task decomposition method, and regenerates the PDDL problem file until a feasible sequence of actions is obtained.

5. The robot guided tour and explanation task planning method based on a multimodal large model according to claim 1, characterized in that, The process of adapting the PPPL symbolic tasks in the action sequence into MCP physical world tasks through the MCP parser using a rule mapping library, instantiating and encapsulating them into MCP instructions, specifically includes: Extract the operators and logical parameters of each action in the action sequence, and identify whether the action is a compound action based on the action attributes. If so, decompose the compound action into a subsequence composed of multiple basic atomic operations according to preset rules. For each of the aforementioned operations or basic atomic operations and their corresponding logical parameters, a search and matching process is performed in the registered MCP resources, and the logical parameters are mapped to specific physical feature parameters or hardware interface instruction IDs. According to the JSON-RPC format corresponding to the MCP protocol, the extracted operators are encapsulated into tool invocation methods, and the matched physical feature parameters or hardware interface instruction IDs are encapsulated into corresponding tool invocation parameters to generate the MCP instruction.

6. The robot guided tour and explanation task planning method based on a multimodal large model according to claim 1, characterized in that, The execution of the MCP instructions and monitoring of the status through the underlying execution tools in the MCP client specifically includes: The MCP client sends the encapsulated JSON-RPC format request to the MCP server; The MCP server calls the underlying execution tool to execute instructions. The underlying execution tool acts as an actuator to drive the robot's hardware to perform physical actions or operations, and returns the execution result, including success, failure, or progress, to the MCP client through the MCP protocol.

7. The robot guided tour and explanation task planning method based on a multimodal large model according to claim 6, characterized in that, The method further includes: the MCP client maintaining the robot state machine based on the received feedback information; if the execution result is successful, the state machine updates the robot's current world state and triggers the next action in the action sequence; if the execution result is blocked or failed, the state machine uses the exception information containing the error type as a new context and feeds it back to the upstream LLM server or replanning module through the MCP protocol to trigger local replanning.

8. A robot guided tour and explanation task planning system based on a multimodal large model, used to implement the robot guided tour and explanation task planning system based on a multimodal large model as described in claim 1, characterized in that the system... include: The perception and input layer is used to acquire location information, multimodal perception data, and navigation task instructions; The cognition and planning layer, which includes an LLM server, an MCP server, a PDDL planner, and a PDDL-MCP adapter, is used to realize the logical generation and mapping from natural language instructions to action sequences. The execution and output layer, which includes the MCP client and robot actuator, is used to receive MCP requests and drive the robot to complete navigation, voice and interactive actions. An exception handling loop is used to monitor the execution status in real time and feed back exception information to the cognition and planning layer in a closed loop.

Images