Natural language based mobile robot navigation control method and system

Abstract

The application belongs to the technical field of mobile robot navigation control, and relates to a mobile robot navigation control method and system based on natural language, which comprises the following steps: collecting multi-modal perception data and encapsulating, generating a dynamic priority event stream, and obtaining a discretized intention segment set through analysis, combining historical context to construct a situational intention flow graph, and generating a predictive task instruction package; then, the modeling granularity of the map is dynamically adjusted according to the instruction package and environmental data, and a multi-resolution fusion map is generated; the segmented path dynamic optimization is carried out on the map, and the robot motion trajectory is generated; then, the trajectory is converted into a driving robot execution action, and an execution state log is generated; finally, the log is fed back to the intention flow engine, and the intention flow graph is updated and corrected in real time; the application solves the problem of insufficient logical coherence in intention understanding, multi-modal data processing and path planning of the existing mobile robot.

Description

Technical Field

[0001] This invention belongs to the technical field of mobile robot navigation control, and relates to a mobile robot navigation control method and system based on natural language. Background Technology

[0002] Autonomous navigation and control technology for mobile robots, as an important branch of artificial intelligence, is widely used in various scenarios such as unmanned delivery, home services, and security patrols. Its core task lies in how to accurately interpret unstructured human-computer interaction commands and adjust motion strategies in real time based on environmental feedback to achieve safe and efficient passage in complex environments.

[0003] In existing mobile robot control systems, instruction parsing and motion execution are typically operated as two separate modules. Traditional control methods often rely on fixed command word matching or single-path task planning based on finite state machines. While this approach is effective for simple point-to-point movements, it often exhibits insufficient logical coherence when dealing with complex natural language instructions that contain multiple semantics, ambiguous references, or require long-term temporal contextual associations. For example, when instructions involve traversing multiple physical spaces or require dynamic decision-making incorporating real-time perception information, existing systems struggle to establish a deep semantic mapping mechanism between instructions and environmental states.

[0004] Furthermore, at the data processing and path planning level, existing control logic typically employs static or single-resolution map models. While this approach is reasonably efficient for macro-level path planning, it often results in stiff robot trajectories, insufficient control precision, or delayed responses to critical safety controls due to a lack of dynamic adjustment capabilities for environmental modeling granularity when performing finer operations (such as obstacle avoidance in narrow spaces or approaching specific targets). This is because such methods suffer from uneven allocation of computational resources. Summary of the Invention

[0005] In a first aspect, the present invention provides a navigation control method for mobile robots based on natural language, employing the following technical solution:

[0006] A natural language-based navigation control method for mobile robots includes the following steps:

[0007] S1. Collect multimodal sensing data and encapsulate it into events to generate a dynamic priority event stream;

[0008] S2. Perform distributed service parsing on the dynamic priority event stream to generate a set of discretized intent fragments;

[0009] S3. Construct a contextualized intent flow graph by performing temporal association and semantic fusion based on the discretized intent fragment set and historical context;

[0010] S4. Analyze the structure and evolution trend of the contextualized intent flow graph to generate predictive task instruction packages;

[0011] S5. Based on the predictive task instruction package and real-time environmental data, dynamically adjust the granularity of environmental modeling to generate a multi-resolution fusion map;

[0012] S6. Perform segmented path dynamic optimization on the multi-resolution fused map to generate the robot's motion trajectory;

[0013] S7. Convert the robot's motion trajectory into a sequence of underlying motor control instructions to drive the robot to perform physical actions and generate a robot execution status log.

[0014] S8. Feed back the robot's execution status log to the contextualized intent flow engine to update and correct the contextualized intent flow graph in real time.

[0015] A further aspect of the present invention generates a dynamic priority event stream, comprising the following steps:

[0016] Simultaneously activate the microphone, LiDAR, and vision sensor to continuously capture voice data, 3D point cloud, and video data;

[0017] Speech data is decomposed into word segments or phrase fragments with confidence scores, and 3D point cloud and video data are parsed into environmental object features containing object category, location and motion vectors;

[0018] Priority values ​​are assigned to each voice data segment and environmental object feature according to a preset security rule base;

[0019] All data units carrying priority values ​​are encoded in timestamp order and merged to generate a dynamic priority event stream.

[0020] A further aspect of the present invention generates a set of discretized intent fragments, comprising the following steps:

[0021] Publish the dynamic priority event stream to the message bus and broadcast it to the parallel-running intent parsing service pool;

[0022] Each service captures events according to subscription rules and generates intent fragments containing semantic tags, timestamps, and associated event identifiers;

[0023] Aggregate and encapsulate all intent fragments generated by services within a fixed time window to generate a discretized set of intent fragments;

[0024] The intent parsing service pool includes at least location entity recognition service, action intent extraction service, and referential relationship resolution service.

[0025] A further aspect of this invention involves constructing a contextualized intent flow graph, comprising the following steps:

[0026] Receive a set of discretized intent fragments and retrieve historical intent nodes as context;

[0027] New intent node candidates are generated based on the timestamp of the intent fragment, and the semantic association between them and historical nodes is calculated.

[0028] Connect new nodes with semantic relevance exceeding a threshold to historical nodes; otherwise, create new intent branches.

[0029] For nodes containing pronouns, search for the nearest context entity nodes and establish referencing edges to generate a contextualized intent flow graph with a directed acyclic graph structure.

[0030] A further aspect of the present invention generates a predictive task instruction package, comprising the following steps:

[0031] Identify the final target node and critical path in the contextualized intent flow graph and transform them into the main task;

[0032] Extract the attribute requirements implied by nodes on the critical path;

[0033] Based on the attribute requirements, preloaded algorithm models or perception patterns are mapped to generate predictive task instruction packages that include task triggering conditions and resource configuration lists.

[0034] A further aspect of the present invention generates a multi-resolution fused map, comprising the following steps:

[0035] Based on the main task in the predictive task instruction package, plan the macroscopic path skeleton on the global semantic map;

[0036] During the travel phase in areas where the obstacle density is detected to be lower than a preset density threshold, coarse-grained environment modeling is performed.

[0037] When the task triggering conditions are met, the target area will be switched to centimeter-level resolution modeling;

[0038] By stitching together coarse-grained environmental maps with centimeter-level resolution maps, a multi-resolution fused map is generated.

[0039] A further aspect of the present invention generates a robot motion trajectory, comprising the following steps:

[0040] Transform the security constraints parsed in the contextualized intent flow graph into high-weight terms in the path search cost function;

[0041] A first path search algorithm is used to generate global traversable paths in coarse-grained areas of a multi-resolution fused map.

[0042] A second optimization algorithm is used in fine-grained areas on the multi-resolution fused map to generate local fine paths by combining real-time sensor data;

[0043] The connection between the global path and the local fine path is smoothed to generate the robot's motion trajectory.

[0044] A further aspect of the present invention generates a robot execution status log, comprising the following steps:

[0045] The trajectory is differentiated to calculate the target linear velocity and angular velocity at each time step;

[0046] Based on the robot's kinematics model, the target velocity is converted into the target rotational speeds of the left and right wheel motors;

[0047] A pulse width modulation signal is generated based on the difference between the target speed and the actual speed to drive the motor;

[0048] The robot continuously collects its actual position, speed, task progress, and sensor readings to generate a robot execution status log.

[0049] A further aspect of the present invention, step S8, includes the following steps:

[0050] The robot's execution status log is parsed into fact nodes, and the fact nodes are connected to the corresponding intent nodes in the contextualized intent flow graph and incorporated into the contextualized intent flow graph.

[0051] Compare the fact nodes with the expected paths in the intent flow graph; if a deviation is detected, trigger the evaluation process.

[0052] Based on the evaluation results, nodes are inserted or paths are replanned in the intent flow graph to achieve a closed loop of instruction-execution-feedback.

[0053] Secondly, the present invention provides a natural language-based mobile robot navigation control system, which adopts the following technical solution:

[0054] A natural language-based mobile robot navigation control system includes the following modules:

[0055] The event stream generation module is used to collect multimodal sensing data and encapsulate it into events to generate dynamic priority event streams.

[0056] The intent fragment generation module is used to perform distributed service parsing of dynamic priority event streams and generate a set of discretized intent fragments.

[0057] The intent flow graph construction module constructs a contextualized intent flow graph by performing temporal association and semantic fusion based on a set of discretized intent fragments and historical context.

[0058] The task instruction generation module is used to parse the structure and evolution trend of the contextualized intent flow graph and generate predictive task instruction packages;

[0059] The fusion map generation module dynamically adjusts the granularity of environmental modeling based on predictive task instruction packages and real-time environmental data to generate multi-resolution fusion maps.

[0060] The motion trajectory generation module is used to perform segmented path dynamic optimization on a multi-resolution fused map and generate the robot's motion trajectory.

[0061] The execution and log generation module is used to convert the robot's motion trajectory into a sequence of underlying motor control instructions, drive the robot to perform physical actions, and generate a robot execution status log.

[0062] The feedback and update module is used to feed back the robot's execution status log to the contextualized intent flow engine, and to update and correct the contextualized intent flow graph in real time.

[0063] In summary, the present invention has the following beneficial technical effects:

[0064] 1. By generating dynamic priority event streams and uniformly encapsulating and dynamically allocating priorities for multimodal data, high-security environmental features such as obstacle avoidance can be prioritized for processing. Information fusion and hierarchical scheduling are achieved at the source of data processing, reducing the possibility of delays in the response of critical security information due to resource competition between modules, thereby enhancing the system's real-time responsiveness and overall security in dynamic environments.

[0065] 2. By constructing a distributed intent parsing service pool and a contextualized intent flow graph, the continuous speech stream is decomposed and then temporally associated and semantically fused based on semantic relevance and historical context, forming a dynamically evolving directed acyclic graph structure. This structure can associate the current instruction fragment with past intent nodes and environmental entities, effectively resolving fuzzy referencing issues, thereby achieving logical coherence in understanding user instructions across sentences and long contexts, improving the accuracy of interaction and context awareness.

[0066] 3. By generating predictive task instruction packages based on contextualized intent flow maps and dynamically adjusting the granularity of environmental modeling accordingly to generate multi-resolution fused maps, the required resources can be predicted before task execution. Specifically, based on task attribute requirements, specific algorithm models can be preloaded or the system can switch to perception mode, and key areas can be modeled. This enables the dynamic allocation of computing and perception resources according to task requirements, ensuring global path planning efficiency while providing the necessary environmental detail support for local fine-grained operations.

[0067] 4. By feeding back the robot's execution status log to the contextualized intent flow engine and updating and correcting it in real time, a closed loop of instruction-execution-feedback is formed. This enables the intent flow graph to dynamically synchronize the robot's actual execution status and environmental changes in the physical world. When execution deviations are detected, the cause can be evaluated and the intent flow graph can be corrected or replanned, thereby enhancing the adaptability to unexpected situations during execution and providing a more accurate real-time context for understanding subsequent instructions. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 A flowchart illustrating an embodiment of this application is disclosed.

[0070] Figure 2 Structural schematic diagrams of embodiments of this application are disclosed. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] The following is in conjunction with the appendix Figure 1 - Figure 2 A preferred description of the present invention is provided below.

[0073] See attached document Figure 1 This invention proposes a natural language-based navigation control method for mobile robots, comprising the following steps:

[0074] S1. Collect multimodal sensing data and encapsulate it into events to generate a dynamic priority event stream;

[0075] S2. Perform distributed service parsing on the dynamic priority event stream to generate a set of discretized intent fragments;

[0076] S3. Construct a contextualized intent flow graph by performing temporal association and semantic fusion based on the discretized intent fragment set and historical context;

[0077] S4. Analyze the structure and evolution trend of the contextualized intent flow graph to generate predictive task instruction packages;

[0078] S5. Based on the predictive task instruction package and real-time environmental data, dynamically adjust the granularity of environmental modeling to generate a multi-resolution fusion map;

[0079] S6. Perform segmented path dynamic optimization on the multi-resolution fused map to generate the robot's motion trajectory;

[0080] S7. Convert the robot's motion trajectory into a sequence of underlying motor control instructions to drive the robot to perform physical actions and generate a robot execution status log.

[0081] S8. Feed back the robot's execution status log to the contextualized intent flow engine to update and correct the contextualized intent flow graph in real time.

[0082] In one embodiment of the present invention, step S1 includes the following steps:

[0083] The microphone, LiDAR, and vision sensor are activated simultaneously to continuously capture voice data, 3D point cloud data, and video data. The voice data is decomposed into word segments or phrase fragments with confidence scores, and the 3D point cloud and video data are parsed into environmental object features containing object categories, positions, and motion vectors. Priority values ​​are assigned to each voice data fragment and environmental object feature according to a preset security rule base. All data units carrying priority values ​​are encoded in timestamp order and merged to generate a dynamic priority event stream.

[0084] Specifically, during data acquisition, three hardware modules—microphone, LiDAR, and vision sensor—are simultaneously activated to collect data. For example, the microphone continuously captures the raw audio waveform of the user's speech at a sampling rate of 16 kHz. The LiDAR performs a 360-degree rotating scan at a frequency of 10 Hz, generating 3D point cloud data frames describing the contours and distance information of the surrounding environment. The vision sensor, i.e., a high-definition camera, captures a color video stream at a rate of 30 frames per second for identifying colors, textures, and specific objects. These three data sources ensure alignment of acquisition timing using hardware timestamps or software synchronization signals.

[0085] The acquired raw multimodal data is then fed into the event-driven encapsulation processor. This multimodal sensing data refers to a collection of raw data from different types of sensors that are synchronized or correlated in time. Event-driven encapsulation refers to the process of segmenting and packaging continuous raw data streams, such as audio streams, point cloud sequences, and video streams, into discrete data units with clear semantic boundaries, temporal information, and additional attributes. For example, the event-driven encapsulation processor runs the following three processing threads in parallel:

[0086] For speech data, the thread can invoke a streaming automatic speech recognition model to decompose a continuous audio stream into word or phrase segments carrying confidence scores in real time. For example, for the audio segment "Please take me there", the output word segments may be "{text: 'please', confidence score: 0.98, start time: t1, end time: t2}" and "{text: 'take me there', confidence score: 0.95, start time: t2, end time: t3}".

[0087] For LiDAR point cloud data, the thread invokes a real-time point cloud clustering and feature extraction algorithm. The algorithm's processing steps include: first, performing ground filtering and non-ground point segmentation on a single frame of point cloud; then, using a clustering algorithm based on Euclidean distance, such as DBSCAN, to segment the non-ground point cloud into independent point cloud clusters; finally, calculating the minimum bounding cube for each point cloud cluster, using the center of the cube as the object's position coordinates, calculating the motion vector based on the displacement of the cube's center between frames, and matching the geometric features of the point cloud cluster with predefined feature templates to initially determine its object category, thus parsing each frame of point cloud into a series of discrete environmental object features. Each feature includes at least the object category (e.g., pedestrian, wall, chair, etc.), position coordinates with the robot coordinate system as the origin, and motion vectors calculated based on consecutive frames.

[0088] For visual video data, the thread calls a lightweight object detection model to identify each frame of the image, supplement or verify the object category identified by the LiDAR, and can add attributes such as color.

[0089] Subsequently, the dynamic priority allocator dynamically assigns priority values ​​to each generated voice data segment and each environmental object feature based on a preset security rule base and the current interaction state. The security rule base contains rules in at least the following dimensions and assigns the baseline priority vertically according to the following exemplary logic:

[0090] Object threat level rules: Set a base threat score based on the object's category, motion state, and relative distance and speed to the robot; for example, if the category is "rapidly approaching moving object" and the relative distance is less than 1 meter, the base priority is set to 100 (highest); if the category is "static furniture" and the relative distance is greater than 0.5 meters, the base priority is set to 30.

[0091] Command urgency rules: The urgency of a command is determined based on the keywords in the voice command; for example, a text segment containing keywords such as "stop" or "danger" has a base priority of 90; a text segment containing general navigation keywords such as "please" or "go" has a base priority of 70.

[0092] Key task phase rules: During key task phases such as going up and down stairs or passing through narrow passages, the priority of environmental perception events is increased as a whole. For example, during this phase, the baseline priority of all environmental object features is increased by 20 units.

[0093] The allocator fine-tunes its priorities based on the real-time relative position and speed of the event and the robot. This dynamic adjustment manifests as follows: when a new environmental object feature event and a currently processed low-priority language event occur within a time window (e.g., 500 milliseconds), and the latter's priority value is higher than the currently processed event's priority by a preset difference (e.g., 30), the system interrupts processing the low-priority event and prioritizes the high-priority event. All data units carrying priority values, timestamps, data sources, and original content or feature descriptions are uniformly encoded into a standard data packet format by the stream synthesizer in ascending order of timestamp. These data packets are sequentially pushed to a first-in-first-out memory buffer, forming a continuous, dynamically prioritized event stream. This event stream is designed to be preemptive; that is, when a new event with a higher priority value than the currently processed event arrives, the system can interrupt processing the low-priority event and prioritize the high-priority event.

[0094] It should be noted that the dynamic priority event stream is a data sequence sorted by timestamps, which mixes speech intent fragments and environmental state features. Each data unit carries a dynamically calculated priority value to guide the resource scheduling and response order of subsequent processing modules. The confidence score refers to a value output by the automatic speech recognition model or visual inspection model that represents the credibility of its recognition results, usually between 0 and 1. Environmental object features are structured information extracted from raw sensor data that describes independent objects in the environment and are the atomic units of environmental understanding. The security rule base is a predefined knowledge base containing various event types and their corresponding threat levels or importance, which is the basis for dynamically allocating priorities.

[0095] For example, after the system starts, suppose the user says, "Please go around that person and then go to the bench." Simultaneously, the LiDAR detects an object moving at a low speed about 1.5 meters ahead, and the visual sensor classifies it as a "pedestrian." The event-driven encapsulation processor works in parallel: the voice thread outputs the fragment {text: "Please go around that person and then go to the bench", confidence: 0.92, start time: 169950, end time: 170050}; the LiDAR thread outputs the feature {category: "pedestrian", location: [1.5, 0.2], motion vector: [0.1, 0.0], timestamp: 170050}; the visual thread outputs the feature {category: "pedestrian", color: "gray", timestamp: 170050}. The dynamic priority allocator, based on a rule base, sets the priority of a moving pedestrian to 90, indicating a higher safety risk; the navigation instruction "Please go around that person and then go to the bench" is a compound instruction, setting its priority to 85. The stream synthesizer generates two event packets based on the timestamp:

[0096] Event1:{Type: Voice, Content: "Please go around that person in front and then go to the bench", Priority: 85, Timestamp: 170000, Original Event ID: EV_sp123};

[0097] Event2: {Type: Environment, Content: {Category: Pedestrian, Location: [1.5, 0.2], Motion Vector: [0.1, 0.0]}, Priority: 90, Timestamp: 170050, Original Event ID: EV_vis789}, and is merged into the event stream. The above content illustrates the process of synchronous acquisition of multi-source data, feature parsing, dynamic priority allocation based on security rules, and synthesis of mixed event streams.

[0098] In one embodiment of the present invention, step S2 includes the following steps:

[0099] The dynamic priority event stream is published to the message bus and broadcast to the parallel-running intent parsing service pool. Each service captures events according to the subscription rules and generates intent fragments containing semantic tags, timestamps, and associated event identifiers. All intent fragments generated by the services are aggregated and encapsulated in a fixed time window to generate a set of discretized intent fragments. The intent parsing service pool includes at least location entity recognition service, action intent extraction service, and referential relationship resolution service.

[0100] The dynamic priority event stream is taken as input and published to a software-defined message bus. This bus broadcasts the event stream in real time to a pre-deployed intent resolution service pool. The intent resolution service pool is usually started during system initialization and includes microservice instances running in independent processes or containers. These instances register their event types, i.e., subscription rules, with the message bus through a service registry. Each process or container is a small, independent program module that specializes in analyzing specific semantic content. This is implemented based on a service architecture to achieve high cohesion, low coupling, and independent scalability. Examples include a location entity recognition service that specializes in identifying location names or descriptions, an action intent extraction service that specializes in extracting action instructions, and a referential relationship resolution service that specializes in handling the specific objects referred to by certain pronouns.

[0101] Among them, the location entity recognition service, action intent extraction service, and referential relationship resolution service are all units in the service pool. The entity recognition service is used to identify specific locations or places mentioned in natural language, such as "supermarket entrance" or "third bench". The action intent extraction service is used to extract the core action instructions that users want the robot to perform from natural language. The referential relationship resolution service is used to parse the real environmental objects that pronouns or indicator words refer to in a specific context.

[0102] Each service subscribes to the message bus for the specific event types it needs to process upon startup. The message bus automatically forwards the event to all services that have subscribed to that event type based on the metadata carried in the event data; for example, if the data type field is a speech fragment and the text content contains preset key verbs. Upon receiving the event, each service independently executes its core parsing logic. For instance, the action intent extraction service maintains a verb library containing core action words such as "walk," "stop," "turn," "accelerate," and "decelerate." It extracts matching verbs and related adverbs or noun complements from the text content carried by the event through string matching and regular expression scanning.

[0103] After processing, a new structured data record, called an intent fragment, is generated. This intent fragment contains three core fields: semantic tag, timestamp, and associated event identifier. The semantic tag field records the parsing result. For example, for the voice fragment "Please turn left," the semantic tag generated by the action intent extraction service is: {Action: Turn, Direction: Left, Pattern: Request}. The timestamp field can be directly copied from the timestamp of the input event to ensure temporal correlation. The timestamp is a data field of the intent fragment, recording the precise moment when the original event occurred, usually in milliseconds since system startup, and is used to build the temporal relationship between events. The associated event identifier field records the unique identifier of the original event that generated this intent fragment in the dynamic priority event stream, used to trace the data source.

[0104] All services run continuously, generating intent fragments in parallel. An independent aggregation service collects all intent fragments generated by all services within a fixed time window from the message bus. Instead of sorting or deeply integrating these intent fragments, the aggregation service simply encapsulates them in a dataset, generating a discretized set of intent fragments. These fragments are unordered within the time window, but each fragment carries its own timestamp and associated event identifier, laying the foundation for subsequent steps.

[0105] For example, given a dynamic priority event stream segment containing a voice segment event with a timestamp of 170000 milliseconds, a priority of 85, and the content "Please go around that person in front and then go to the bench", and an environmental object feature event with a timestamp of 170050 milliseconds, a priority of 90, and the content "{Category: Pedestrian, Location: [x1,y1], Motion Vector: [vx,vy]}", the message bus broadcasts this voice segment event to the action intent extraction service and the location entity recognition service. The action intent extraction service parses out two action intents, "go around" and "go", and generates two intent segments: segment A (semantic label: {action: obstacle avoidance, target type: pedestrian}, timestamp: 170000, associated event ID: EV_sp123) and segment B (semantic label: {action: go to, target type: bench}, timestamp: 170000, associated event ID: EV_sp123).

[0106] The location entity recognition service identifies "bench" as a location entity and generates intent fragment C (semantic label: {entity type: bench}, timestamp: 170000, associated event ID: EV_sp123). Simultaneously, the referential relationship resolution service subscribes to voice events and environmental object events. When it receives a voice fragment event (containing "that person") and an environmental object feature event (category: "pedestrian"), it generates intent fragment D (semantic label: {referential binding: "that person" - object ID: OBJ_ped456}, timestamp: 170050, associated event ID: EV_vis789) based on spatiotemporal proximity rules (close timestamps and a rough match between the voice direction "ahead" and the pedestrian's location).

[0107] At the end of the time window from 170,000 to 170,500 milliseconds, the aggregation service collects fragments A, B, C, and D, packages them, and outputs a discretized set of intent fragments containing these four intent fragments. This example verifies the process of parallel service processing, subscription-based event fetching, and structured generation of intent fragments, with the resulting set conforming to the characteristics of "multiple, unordered but context-dependent (via timestamps and event IDs)".

[0108] In one embodiment of the present invention, step S3 includes the following steps:

[0109] Receive a set of discretized intent fragments and retrieve historical intent nodes as context; generate new intent node candidates based on the timestamps of the intent fragments and calculate their semantic relevance with historical nodes; connect new nodes with semantic relevance exceeding a threshold to historical nodes, otherwise create new intent branches; for nodes containing pronouns, search for the nearest environmental entity nodes and establish reference edges to generate a contextualized intent flow graph with a directed acyclic graph structure.

[0110] Specifically, the contextualized intent stream engine is invoked. This engine is an independent software service module responsible for maintaining and updating the temporal and semantic relationship graph of intents, which is used for node association, semantic computation, and graph structure management. After the engine starts, it first receives the set of discretized intent fragments from the input as incremental data for the current processing cycle. At the same time, the engine retrieves and loads all historical intent nodes and their connections generated and retained in the most recent interaction cycle from its internally maintained persistent storage area, forming the current historical context.

[0111] For each intent fragment in the input discretized intent fragment set, its timestamp and semantic label are first extracted. The contextualized intent stream engine creates new intent node candidates for each intent fragment according to the order of its timestamps. Then, the semantic correlation between each new candidate node and all existing historical intent nodes is calculated one by one.

[0112] The formula for calculating semantic relevance is:

[0113]

[0114] Where R is the semantic relevance, which represents the strength of the association between a new node and a certain historical node, and its value range is [0,1]. This indicates the number of matches for the same entity name, such as "bench" or "pedestrian," in the semantic tags of the old and new nodes. This indicates the total number of entity names contained in the two nodes. This indicates the number of matches for the same action command, such as "go" or "avoid obstacles". This indicates the total number of action instructions contained in the two nodes. This indicates the number of matches with the same attribute, such as "left side" or "empty". This indicates the total number of attributes contained in the two nodes. , , These are weighting coefficients, and the sum of the three is 1. They are used to balance the importance of different semantic elements. Their setting can be based on statistical analysis of one hundred sets of real human-computer dialogue data. The optimization goal is to make the association result best match the human subjective judgment of dialogue coherence. Its specific numerical range is set based on experimental optimization. The value range is 0.4-0.6. The value range is 0.25-0.35. The value range is 0.15-0.25, and the typical setting value is... 0.5 0.3 It is 0.2.

[0115] The preset semantic relevance threshold ranges from 0.5 to 0.7, with a typical value of 0.6. For each new candidate node, it is connected to a historical intent node whose semantic relevance R value exceeds the preset threshold (e.g., 0.6), forming a directed edge from the historical node to the new node. The weight of the edge is the R value. If the R values ​​of a new candidate node and all historical nodes are below the threshold, a new intent branch is created starting from that node. During connection processing, the engine specifically addresses referential relationships. When the semantic label of a new node contains vague referents such as "that" or "there," the engine searches the currently constructed intent flow graph for nodes with the most recent timestamp and whose node type is an environment object entity, such as nodes derived from environment object feature events. The referent is then explicitly linked to this specific entity node, forming a special referencing edge. In other words, for nodes containing referents, the engine searches the graph for environment entity nodes based on the principle of the most recent timestamp and the closest spatial location, and establishes referencing edges.

[0116] After processing all intent fragments in the current cycle using the above steps, the updated node and edge relationships are saved back to persistent storage, forming a dynamically evolving directed acyclic graph (DAG) representing the user's continuous instruction logic, i.e., a contextualized intent flow graph. A DAG is a graph data structure consisting of vertices and directed edges, and it contains no loop paths. It is used to represent the temporal or logical causal evolutionary relationships between intents. The contextualized intent flow graph is a dynamically updated DAG, where nodes represent specific intents or environmental entities, and edges represent temporal, semantic, or referential relationships between intents.

[0117] For example, continuing the above example, suppose a discretized intent fragment set containing four intent fragments is generated. Fragment A is an action to avoid obstacles, the target type is pedestrian; fragment B is an action to move towards a bench, the target type is bench; fragment C is an entity type bench; and fragment D refers to the object ID OBJ_ped456 that is bound to that person. Assume there is already a historical intent node H in the historical context, whose semantic label is action movement. The contextualized intent stream engine first creates a new node A1 for fragment A. The semantic association between node A1 and the historical node H is calculated. Node A1's entity set is "pedestrian," its action set is "obstacle avoidance," and its attribute set is empty. Node H's entity set is empty, its action set is "movement," and its attribute set is empty. According to the formula, =0, =1, =0 (“Obstacle avoidance” is different from “movement”). =2, =0, =0. Using a typical weight setting, R=0, which is below the threshold of 0.6. Therefore, node A1 is not connected to node H, but instead a new intent branch is created starting from A1.

[0118] Next, segment D is processed, creating node D1. Since the semantic label of node D1 refers to the object IDOBJ_ped456 bound to that person, a node representing the "pedestrian" entity is searched in the graph. This node may have been generated by an earlier environmental object feature event, and node D1 is linked to this entity node as a result of the referential resolution. Then, segments B and C are processed, creating nodes B1 and C1. The semantic correlation between node B1's action of moving to the target type bench and node A1's action of avoiding the obstacle target type pedestrian is calculated. The two actions are different, the entities are different, and the calculated R value is below the threshold. However, since they have the same timestamp and come from the same voice command, a weak temporal connection can be established according to business rules, such as a common source event identifier or a lower threshold, to represent concurrent intent. Node C1's entity type bench matches the entity "bench" of node B1, showing a high semantic correlation. Therefore, node C1 is linked to node B1 as an entity supplement node. Ultimately, the generated contextualized intent flow graph contains multiple nodes (A1, D1, B1, C1, H and environment entity nodes) and connecting edges (such as D1 pointing to the pedestrian entity node, C1 pointing to B1), clearly recording the two intent branches "avoiding pedestrians" and "going to the bench" and their associated entity information.

[0119] In one embodiment of the present invention, step S4 includes the following steps:

[0120] Identify the final target node and critical path in the contextualized intent flow graph and transform them into the main task; extract the attribute requirements contained in the nodes on the critical path; map the preloaded algorithm model or perception mode according to the attribute requirements, and generate a predictive task instruction package including task triggering conditions and resource configuration list.

[0121] The task parsing service is invoked, taking a contextualized intent flow graph as input. Specifically, it first executes a graph traversal algorithm, starting from the node representing the robot's current execution state or the latest user intent, searching for all leaf nodes without outgoing edges or whose outgoing edges represent task completion. These leaf nodes are identified as potential final target nodes. The task parsing service comprehensively evaluates and selects the primary final target node based on the clarity of the node's semantic label, the frequency with which the node is emphasized by the user in historical context, and the sum of path weights from the node to the current state node. For example, if one leaf node's semantic label is "action: arrived at the target entity, a bench, with precise docking attributes," while another leaf node's semantic label is "action: observed the target entity, a billboard," the former is usually selected as the primary final target node.

[0122] After determining the final target node, a shortest path algorithm, such as Dijkstra's algorithm, is used. This algorithm treats each intent node in the contextualized intent flow graph as a vertex and the directed edges between nodes as edges. The edge weights are set to the reciprocal of the semantic relevance or a fixed value. The optimal path from the current state node to the final target node is calculated, and this path is identified as the critical path. The task parsing service logically concatenates and fuses the semantic labels of all intent nodes on this critical path, transforming them into a description of the robot's main task. For example, after concatenating the critical path, a complete navigation command might be parsed to go to chair B next to point A.

[0123] Next, the task parsing service traverses each node on the critical path, extracting attribute requirements from the node's semantic tags or from environmental entity nodes directly connected to that node. For example, it extracts path requirements to avoid moving obstacles from the action obstacle avoidance node, and target point requirements to precisely dock from the action arrival attribute precise docking node. These attribute requirements are organized into a structured list. Based on this list of attribute requirements, the task parsing service, according to a predefined mapping rule table, predicts the special algorithm models or perception modes that may need to be invoked or activated at each stage of task execution. Special algorithm models are dedicated software modules designed to complete specific high-precision or complex sub-tasks, such as precise docking controllers or dense crowd crossing algorithms. Perception modes are the sensor's operating state configurations. Under this configuration, the sensors acquire data at higher frequencies, higher resolutions, or specific focused areas to meet the needs of fine-grained tasks. The mapping rule table is pre-defined based on engineering experience. It is a two-dimensional lookup table or rule set, with the attribute requirements extracted from the intent node as input and the corresponding resource preloading instructions as output. For example, a typical mapping rule is: the IF attribute requires "precise docking," and the THEN resource preloading instruction must include {calling the algorithm model: "precise docking controller"; switching sensor modes: LiDAR to "high-frequency high-resolution mode," and visual sensor to "local area interest point tracking mode"}. That is, the attribute requirement for precise docking maps to the need to call centimeter-level positioning and closed-loop motion control algorithm models; the attribute requirement for avoiding moving obstacles maps to the need to activate the high-frequency LiDAR scanning mode and dynamic object tracking model.

[0124] Finally, the task parsing service encapsulates these predictions into a structured data packet. This data packet contains two main parts. The first part is the task triggering condition, described in the form of a logical expression, such as when the robot enters an area with a radius of two meters centered on the target point. The second part is a resource configuration list, which details the names of the algorithm models that need to be preloaded into memory, the sensor operating modes that need to be switched, and their parameters. For example, the list might include: Preloaded model: Precision docking controller v1.2; Switching mode: LiDAR to high-frequency mode 100 Hz, Vision camera to area exposure mode. This complete data packet constitutes the predictive task instruction packet.

[0125] The resource configuration list is an enumerated list that details the computing and perception resources required in advance to meet the attribute requirements. The predictive task instruction package is a structured data object that contains future task requirements derived from intent flow graph analysis and corresponding resource preparation instructions.

[0126] For example, connecting the contextualized intent flow graph generated in the previous example, which includes node B1's action to move to a target type bench and node C1's entity type bench, the task parsing service parses this graph. Traversal reveals that node B1 is a leaf node with clear semantics, thus identifying it as the final target node. The path from the latest node representing the current state to node B1 is determined as the critical path. Parsing this path generates the main task description: Navigate to the bench. Subsequently, analysis reveals that the attribute associated with node B1 requires precise docking, derived from its semantic label or the attributes of the entity nodes connected to it. According to the mapping rule table, the attribute requirement for precise docking corresponds to calling the special algorithm model Precision Dock Controller v1.2 and activating the perception mode LiDAR high-frequency scanning mode. Finally, the generated predictive task instruction package contains the following: Task trigger condition: When the robot navigation subsystem reports that its geometric distance to the target bench is less than 1.5 meters. Resource configuration list: Preload the algorithm model Precision Dock Controller v1.2 to memory for backup; Configure the LiDAR sensor to switch to high-frequency scanning mode with parameters of 100 Hz scanning frequency and 0.25 degrees angular resolution.

[0127] In one embodiment of the present invention, step S5 includes the following steps:

[0128] Based on the main task in the predictive task instruction package, a macroscopic path skeleton is planned on the global semantic map; during the travel phase in areas where the obstacle density is detected to be lower than a preset density threshold, coarse-grained environment modeling is performed; when the task triggering conditions are met, the target area is switched to centimeter-level resolution modeling; the coarse-grained environment map and the centimeter-level resolution map are stitched together to generate a multi-resolution fused map.

[0129] Specifically, the robot navigation system receives predictive task instruction packets as input. First, the instruction packet is parsed to extract the main task description. Based on this description, the system accesses its internally maintained global semantic map, which is a topological or grid map containing known landmarks, road networks, building outlines, and their semantic labels. Between the start and end points of the main task, a path search algorithm, such as Dijkstra's algorithm based on topology networks or A-algorithm based on grids, is used to plan a macroscopic path skeleton connecting the two points. This macroscopic path skeleton is a coherent channel representation planned on the global semantic map, connecting the task's start and end points, and defines the approximate spatial range and direction of task execution. This path skeleton does not consider centimeter-level details, only guaranteeing connectivity and approximate direction on a known static map. The path width is typically set to the safe envelope width of the robot body.

[0130] After the macroscopic path skeleton is planned, the system begins to model the environment using real-time sensor data. The modeling process is managed by a dynamic modeling controller. This controller continuously monitors the robot's real-time position on the macroscopic path skeleton and dynamically adjusts the modeling granularity based on task stage prediction information contained in the predictive task instruction package. During movement, the dynamic modeling controller calculates the obstacle density within a certain area around the robot in real time and compares it with a preset density threshold to decide whether to use coarse-grained modeling. The obstacle density is defined as the ratio of the number of grid cells identified as "occupied" within a specified evaluation area, for example, a square area centered on the robot with sides of 2 meters, to the total number of grid cells in that area. The preset density threshold is set within a range based on empirical data, for example, 0.1 to 0.3, with a typical value of 0.15. When the calculated real-time obstacle density is lower than this threshold, the controller determines that the current environment is open or simple, instructing the environment building module to use low-resolution grids (e.g., sides of 20 to 30 centimeters) for coarse-grained modeling; when the obstacle density is higher than this threshold, it maintains or switches to a higher-resolution modeling mode to handle complex environments. The controller instructs the environment modeling module to use low-resolution rasters for coarse-grained modeling. Coarse-grained environment modeling refers to using low-resolution raster maps with grid cell side lengths of 10 cm to 30 cm for representation. Low-resolution rasters are a grid division method used in environment modeling, where each grid cell is relatively large, used to efficiently represent the approximate occupancy of a large area of ​​the environment. This means that each grid cell represents a large physical size, such as a side length of ten centimeters. The modeling module integrates real-time LiDAR and visual data, downsamples this data, and fills it into the low-resolution rasters. Each grid cell is only marked as idle, occupied, or unknown, thus quickly generating computationally lightweight, frequently updated coarse-grained local maps.

[0131] Simultaneously, the controller makes predictions based on the resource configuration list and task triggering conditions in the predictive task instruction package. For example, when the instruction package indicates that precise docking is required at a distance of 1.5 meters from the target, the controller continuously calculates the distance between the robot and the target area. Once the distance reaches a preset threshold, the controller immediately sends an instruction to the environment modeling module, requesting centimeter-level resolution modeling of a square area centered on the target point with a side length of three meters. Centimeter-level resolution modeling refers to using a high-resolution raster map with grid cell side lengths of 1 to 5 centimeters. The environment modeling module then switches modes and calls the pre-loaded high-precision perception model, such as denoising and interpolating the original LiDAR point cloud to generate an ultra-high resolution occupancy raster map with each grid cell having a side length of one centimeter, finely depicting details such as obstacle edges and ground texture changes within the target area.

[0132] Finally, the system requires a map fusion engine. This engine seamlessly stitches together the two types of map data with different granularities. A coarse-grained global or large-scale local map serves as the base, while an ultra-high-resolution, fine-scale regional map is overlaid on top as an independent layer with precise global coordinates. At the boundary between the two map data, the fusion engine uses a bilinear interpolation algorithm for a smooth transition. The specific steps include: identifying the grid coordinates of the boundary region; calculating the transitional occupancy probability of the low-resolution grid at the boundary using bilinear interpolation based on the occupancy probability value of the high-resolution grid; and generating a smooth occupancy probability gradient region. This ensures that when the robot navigates from a coarse-grained region to a fine-grained region, the path planning will not jump or oscillate due to abrupt changes in map resolution. In this way, a multi-resolution fused map with a clear global path is ultimately generated. The multi-resolution fused map integrates environmental information at different spatial scales, containing both coarse-grained regions for efficient global navigation and ultra-high-resolution regions for fine-scale local operations.

[0133] It's important to note that coarse-grained modeling refers to a simplified, non-refined representation of the environment using a low-resolution grid, focusing on quickly identifying passable areas and large obstacles. Centimeter-level resolution, on the other hand, refers to the level of detail in environment modeling, where each grid cell corresponds to a physical size of one centimeter or smaller, used to accurately depict object outlines and subtle terrain variations.

[0134] For example, the predictive task instruction package generated in the previous example has a main task of navigating to a bench and predicting that precise docking should be initiated when the robot is 1.5 meters away from the target. Parsing this instruction, a macroscopic path skeleton is planned from the known current position of the robot in the global semantic map and the location marked as a bench. It is assumed that this path mostly traverses open lawns. While moving along this skeleton, the dynamic modeling controller instructs the robot to perform coarse-grained modeling of a 20-meter radius using a low-resolution grid with 10-centimeter sides. When the robot determines, based on real-time positioning data, that its distance from the target bench has entered the 2-meter threshold, the controller immediately triggers a mode switch. The environment modeling module performs centimeter-level modeling of a 3-meter-sided area centered on the bench, generating an ultra-high-resolution map with 1.5-centimeter grid sides, accurately depicting the bench's legs, seat, and any pebbles or depressions that may be present. The map fusion engine uses this detailed map as a layer, precisely aligns the coordinates, and overlays it onto the current coarse-grained global map, performing smooth interpolation transitions in the boundary areas. In the final multi-resolution fused map, most of the path area from the robot to the bench is represented in coarse-grained terms, while the key stopping areas around the bench are presented as a fine map with high detail.

[0135] In one embodiment of the present invention, step S6 includes the following steps:

[0136] The safety constraints parsed in the contextualized intent flow graph are transformed into high-weight terms in the path search cost function; a first path search algorithm is used to generate a global traversable path in the coarse-grained region of the multi-resolution fusion map; a second optimization algorithm is used in the fine-grained region of the multi-resolution fusion map, combined with real-time sensor data, to generate a local fine path; and a smooth transition is performed at the connection between the global traversable path and the local fine path to generate the robot's motion trajectory.

[0137] Specifically, the path planner receives a multi-resolution fused map as the core environment representation and extracts parsed safety constraints, such as detouring to the left, from the contextualized intent flow graph. The path planner first converts these safety constraints into input parameters for the path search algorithm. For semantically explicit geographic location constraints, such as detouring to the left, the planner transforms them into a high-weight penalty term in the cost function. Specifically, when calculating the cost of moving from the current node to its right neighbor, an additional penalty value is added, for example, ten times the standard movement cost, causing the algorithm to favor choosing the left-hand path when making trade-offs.

[0138] The planner employs a segmented planning strategy. In the coarse-grained areas of the multi-resolution fused map, the planner calls computationally efficient path search algorithms, such as Algorithm A. Algorithm A runs on the low-resolution raster map, with each raster as a node, and its cost function is as follows: middle, This represents the actual cost of moving from the starting point to node n, typically calculated based on the Manhattan distance. This is a heuristic function that estimates the minimum cost from node n to the target point, typically using Euclidean distance. The algorithm, considering the aforementioned high-weight penalty term, quickly plans a global path from the starting point of the coarse-grained region to the boundary of that region or the entrance to the fine-grained region.

[0139] When the robot enters a fine-grained area of ​​the map based on its localization information, the planner immediately switches its planning algorithm. In this area, the planner employs a second optimization algorithm capable of handling complex geometric constraints and dynamic information, such as the dynamic window method. Its evaluation function includes a proximity score between the trajectory endpoint and a local target, a minimum distance score between the trajectory and obstacles, and a trajectory smoothness score. It also transforms the safety constraints parsed from the contextualized intent flow graph into penalty weights in the distance score term. The dynamic window method combines real-time LiDAR and visual sensor data on the position and velocity of dynamic obstacles to sample a series of linear and angular velocity pairs in the robot's velocity space. For each velocity pair, the algorithm simulates the robot's trajectory within a short future time window and scores these trajectories based on multiple evaluation criteria. These criteria include: proximity between the trajectory endpoint and a local target, distance between the trajectory and static and dynamic obstacles, trajectory smoothness, and whether it meets specific action requirements parsed from the contextualized intent flow graph, such as slow passage. The dynamic window method selects the velocity pair with the highest score as the output of the current control cycle. This process is repeated in each control cycle to achieve fine-grained local path planning and real-time obstacle avoidance.

[0140] After planning the coarse-grained global path and fine-grained local path sequences, the trajectory post-processing module is activated. The main task of this module is to achieve smooth transitions between paths. Between the endpoint of the global path and the starting point of the local path sequence, and between different local path segments, Bézier curves or spline interpolation algorithms are used to connect them, ensuring that the generated combined path is continuous in position and direction. Simultaneously, the module performs velocity planning on the trajectory based on the robot's bottom kinematic model, such as a differential drive model, ensuring that the required velocity and angular velocity at each point on the trajectory are within the robot's physical execution capabilities. Finally, the continuous path, integrating global guidance, local obstacle avoidance capabilities, user safety constraints, and kinematic feasibility, is output as the robot's motion trajectory. The robot's motion trajectory is a sequence of timestamped position, orientation, linear velocity, and angular velocity data points. This sequence is globally navigable, locally obstacle-avoiding, conforms to user constraints, and satisfies the robot's own motion limitations.

[0141] It should be noted that safety constraints are restrictive conditions on the robot's motion path or behavior extracted from the contextualized intent flow graph, usually derived from explicit requirements in user instructions or reasoning based on safety rules. Path search algorithms are algorithms that find paths from a starting point to an end point on a graph or mesh. The A algorithm is a heuristic search algorithm that combines the ideas of best-first search and Dijkstra's algorithm. Optimization algorithms are algorithms that find optimal or satisfactory solutions under complex constraints. The dynamic window method is an optimization method used for robot local motion planning, which samples and evaluates in velocity space to select the optimal immediate motion command. Smoothing transitions refer to using mathematical curve fitting methods to make the segmented planned path continuous and smooth at connection points, eliminating abrupt changes in path direction. Robot kinematics refers to the physical constraints determined by the robot chassis motion mechanism. In smooth transition processing, the alternative smooth paths generated by the Bézier curve or spline interpolation algorithm used by the trajectory post-processing module must be back-verified through the kinematic model to ensure that the radius of curvature of any point on the path is greater than the robot's minimum turning radius, and that the pose changes between adjacent points can be realized within a given control cycle through the differential drive model, thereby ensuring that the generated smooth trajectory is kinematically feasible.

[0142] For example, connecting the multi-resolution fusion map generated in the previous example, the area around the bench is a fine-grained region at the centimeter level, while the rest is a coarse-grained region. Simultaneously, safety constraints for bypassing pedestrians are parsed from the contextualized intent flow map. The path planner first uses Algorithm A to plan a global path in the coarse-grained region. When a path point is calculated to be near a known pedestrian's historical location, because the safety constraint has been converted into a high-cost weight, the algorithm-generated path automatically deviates from the preset straight path, choosing to detour from the left, generating a globally accessible path. When the robot enters the fine-grained region around the bench along this path, the planner switches to a dynamic window method.

[0143] Suppose a real-time LiDAR detects an unexpected small moving obstacle to the right front of a bench. The dynamic window method samples in velocity space, simulates multiple trajectories, and scores each trajectory. A trajectory that points directly at the bench but approaches the obstacle is penalized for being too close; another trajectory that curves slightly around the obstacle before pointing towards the bench receives a higher score. The planner outputs the velocity command corresponding to the latter trajectory. The trajectory post-processing module receives the detour path points output by Algorithm A and a series of local path points generated by the dynamic window method. At the junction of the coarse and fine paths, cubic spline interpolation is used to generate a smooth transition curve, ensuring the robot's orientation smoothly transitions from the detour direction to the direction towards the bench. Finally, a continuous motion trajectory is output, starting from the starting point, first detouring to the left, then smoothly turning, and finally approaching and precisely stopping beside the bench.

[0144] In one embodiment of the present invention, step S7 includes the following steps:

[0145] The motion trajectory is differentiated to calculate the target linear velocity and angular velocity at each time step; the target velocity is converted into the target rotational speed of the left and right wheel motors based on the robot's kinematic model; a pulse width modulation signal is generated based on the difference between the target rotational speed and the actual rotational speed to drive the motors; the robot's actual position, speed, task progress, and sensor readings are continuously collected to generate a robot execution status log.

[0146] Specifically, the motion control module receives the robot's motion trajectory as input. This trajectory is a data sequence consisting of a series of timestamped pose points. Each pose point contains the robot's planar coordinates XY and heading angle θ in the global coordinate system. The control module first performs time differentiation processing on the trajectory to calculate the instantaneous linear velocity and angular velocity required for each trajectory point. Here, the linear velocity is the scalar velocity of the robot's center of mass moving along the current forward direction; the angular velocity is the scalar velocity of the robot rotating about its vertical axis around its center of mass.

[0147] For two adjacent pose points in the trajectory sequence, calculate the average linear velocity in the robot's body coordinate system during the robot's movement from one pose point to the other. and mean angular velocity .

[0148] The calculation formula is:

[0149]

[0150]

[0151] in:

[0152] ;

[0153] ;

[0154] ;

[0155] Let be the average linear velocity of the i-th segment of the trajectory; Let be the average angular velocity of the i-th segment of the trajectory; For robots in time Location, For robots in time The heading angle; Displacement distance; This represents the change in heading angle.

[0156] By traversing the entire trajectory sequence, the distance between each pair of adjacent points is calculated. and This generates a sequence of target linear velocity and angular velocity corresponding to the trajectory points. Then, the velocity converter, based on the kinematic model of the robot's underside, converts the target linear velocity... and angular velocity Converted to the target speed of the left and right drive wheels.

[0157] For a differentially driven robot, its kinematic model formula is:

[0158]

[0159] in, and These are the target linear velocities of the left and right wheels, respectively, in meters per second. This refers to the wheelbase between the robot's left and right drive wheels, measured in meters. It is a fixed physical parameter set according to the robot's hardware design drawings. , The linear velocity and angular velocity of the robot as a whole.

[0160] Subsequently, the motor driver calculates the target rotational speed based on the target linear velocity of the left and right wheels, combined with the wheel radius R. , The angular velocity of the robot's center of mass is the speed at which the robot rotates around its center. Finally, the proportional-integral-derivative (PID) controller inside the motor driver calculates the corresponding pulse width modulation (PWM) duty cycle based on the difference between the target speed and the actual speed feedback from the encoder. The parameters of the PLD controller are obtained through system identification and experimental tuning. Its tuning objective is to ensure trajectory tracking accuracy while suppressing disturbances caused by uneven ground or load changes, and to ensure smooth motor response to avoid sudden starts and stops that may cause user discomfort. A PWM signal sequence is generated and sent to the left and right wheel motors to drive the robot to move along the planned trajectory. It should be noted that the PWM signal simulates control signals with different voltage or power levels by adjusting the duty cycle of the square wave signal, and is used to precisely control the speed of the DC motor. The PLD controller is a classic feedback controller that calculates the control quantity through proportional, integral, and derivative terms.

[0161] During execution, an independent data logging service runs continuously. This service collects and records multi-source data at a fixed frequency, such as 50 Hz. The recorded data includes: the actual position from the fusion positioning module, the actual linear and angular velocities from the motor encoder feedback, the current task completion percentage from the task manager, and instantaneous readings from key sensors such as LiDAR and the inertial measurement unit. All of this data is bound to a system timestamp, formatted into structured log lines, and written to a circular buffer or file in real time. This continuously updated collection of data, containing timestamps and specific values, constitutes the robot execution status log, a collection of data records arranged chronologically and containing the robot's own state and perception data.

[0162] For example, connecting the robot motion trajectory generated in the previous example, which includes a series of pose points, let's assume we take two adjacent points P1 and P2, with coordinates X1=1.0 m, Y1=2.0 m, θ1=0.3 radians, timestamp t1=1000 milliseconds; and X2=1.05 m, Y2=2.08 m, θ2=0.32 radians, timestamp t2=1050 milliseconds. First, we calculate... It is 0.094 meters. It is 0.02 radians. It is 0.05 seconds. Substituting into the formula, we get... =0.094 / 0.05=1.88 meters per second =0.02 / 0.05 = 0.4 radians per second. Assuming the robot's wheelbase L = 0.5 meters, substituting into the kinematic model, we get: =1.88 - (0.4 + 0.5 / 2) = 1.88 - 0.1 = 1.78 meters per second. =1.88 + (0.4 + 0.5 / 2) = 1.88 + 0.1 = 1.98 meters per second. Assuming the wheel radius R = 0.1 meters, the target rotational speed of the left wheel is... =1.78 / 0.1=17.8 radians per second, target rotational speed of the right wheel. =1.98 / 0.1 = 19.8 radians per second. The motor driver generates a corresponding pulse width modulation signal based on this target value. Simultaneously, at timestamp t=1050 milliseconds, the data logging service records a log entry: Timestamp 1050, actual position X=1.048, Y=2.079, θ=0.319, actual speed... =1.85, =0.38, mission progress = 65%, the nearest obstacle distance for the lidar is 0.8 meters.

[0163] In one embodiment of the present invention, step S8 includes the following steps:

[0164] The robot's execution status log is parsed into fact nodes, and the fact nodes are connected to the corresponding intent nodes in the contextualized intent flow graph and incorporated into the contextualized intent flow graph. The expected paths in the fact nodes and intent flow graphs are compared, and if a deviation is detected, an evaluation process is triggered. Based on the evaluation results, nodes are inserted or paths are replanned in the intent flow graph to achieve a closed loop of instruction-execution-feedback.

[0165] Specifically, the data feedback service continuously monitors and reads new records from the robot's execution status log. It parses each log record, extracting key factual data such as actual location coordinates, descriptions of detected temporary obstacles, and task progress status. For each extracted factual data point, the data feedback service creates a new fact node in the contextualized intent stream engine. For example, for the log entry "Reached intermediate point X," a node is created with the semantic label "{Event Type: Status Update, Content: Arrived at location X, Confidence: 1.0}". The timestamp field is taken from the log record's timestamp. The associated event identifier field is left blank or points to the original task instruction node that generated this log.

[0166] After a node is created, the data feedback service calls the graph update interface of the contextualized intent flow engine to add this new node as a fact node to the existing contextualized intent flow graph. Based on the fact node's timestamp and semantic content, it automatically finds and connects to the most relevant intent node in the graph. For example, the new fact node "Arrived at location X" will be connected to the node representing the intent "Go to location X," serving as confirmation of the intent's execution result. When the data feedback service parses logs such as "Temporary obstacle detected ahead," it not only creates a fact node but also triggers a deviation assessment process. The current fact, i.e., the existence of a temporary obstacle, is compared with the expected path from the current node to the next target node in the intent flow graph. The expected path is derived from the projection of the previously planned critical path onto the environment map. It is determined whether the temporary obstacle is on or very close to the expected path. If so, it is marked as a "path deviation."

[0167] Subsequently, a deviation cause assessment based on a simple rule base is initiated. This rule base defines common deviation causes and corresponding handling strategies. For example, the rule states: "If the obstacle type is 'dynamic' and its expected duration is short, the cause assessment is 'temporary obstacle avoidance'; the handling strategy is to locally replan subsequent path nodes in the graph, generate detour sub-paths, and update the expected arrival time of subsequent nodes." The appropriate rule is matched based on the obstacle's attributes (dynamic or static) and context. Based on the matched handling strategy, the contextualized intent flow graph is corrected. For example, for "temporary obstacle avoidance," a new "detour action" intent node might be inserted into the graph, and the expected time for subsequent "arrival at the target" nodes might be adjusted. If the deviation is large, such as the original target point being blocked, a higher-level replanning might be initiated, or even a new task branch might be created. This update and correction process is continuous. By continuously incorporating the execution state as new facts and dynamically correcting the intent flow graph, this mechanism ensures that the intent flow graph not only statically reflects the user's initial command logic but also dynamically and in real-time synchronizes the robot's actual execution in the physical world. This forms a closed-loop command-execution-feedback cycle. The updated intent flow graph provides the system with precise and timely context for understanding the next instruction the user may give. For example, when the user says "keep going", the system can know from the latest graph state that it is continuing to walk towards the original goal from the current position after detouring, rather than the original starting point.

[0168] For example, the robot execution status log generated in the aforementioned example is assumed to contain the record: "Timestamp 2050, actual position X=5.1, Y=3.2, temporary obstacle detected: category pedestrian, position 2 meters ahead". The data feedback service parses this log and creates two fact nodes. Node F1: semantic label is "{state: position update, coordinates: [5.1, 3.2]}", timestamp 2050. Node F2: semantic label is "{event: environment change, content: pedestrian obstacle detected, position: relative to 2 meters ahead}", timestamp 2050. Node F1 is connected to the corresponding waypoint node on the "navigate to bench" path in the graph. At the same time, a deviation assessment is triggered, comparing the content of node F2 with the expected path from the current location to the next waypoint, and finding that the pedestrian's position is on the path. According to the rule base, the assessment reason is "temporary dynamic obstacle avoidance". The graph is then corrected: a new intent node is inserted between the current position node and the next waypoint node, with the semantic label "{action: obstacle avoidance, object: pedestrian, strategy: detour to the right}". Meanwhile, the edges that originally pointed directly to the next waypoint were changed to first point to the new intent node, and then from the new intent node to the next waypoint. After the correction, when a user asks "Where are we?" at this moment, the system, based on the latest intent flow graph, can answer "We are avoiding a pedestrian and will be on the bench soon." This verifies the complete closed-loop process of converting state logs into fact nodes, performing deviation assessment, and dynamically correcting the intent flow graph.

[0169] See appendix Figure 2 The present invention also proposes a natural language-based mobile robot navigation control system, comprising the following modules:

[0170] The event stream generation module collects multimodal perception data and encapsulates it into events to generate dynamic priority event streams. The intent fragment generation module performs distributed service parsing on the dynamic priority event streams to generate a set of discrete intent fragments. The intent stream graph construction module performs temporal association and semantic fusion based on the discrete intent fragment set and historical context to construct a contextualized intent stream graph. The task instruction generation module parses the structure and evolution trend of the contextualized intent stream graph to generate predictive task instruction packages. The fusion map generation module dynamically adjusts the granularity of environmental modeling based on the predictive task instruction packages and real-time environmental data to generate a multi-resolution fusion map. The motion trajectory generation module performs segmented path dynamic optimization on the multi-resolution fusion map to generate the robot's motion trajectory. The execution and log generation module converts the robot's motion trajectory into a sequence of underlying motor control instructions to drive the robot to perform physical actions and generate a robot execution status log. The feedback and update module feeds the robot's execution status log back to the contextualized intent stream engine to update and correct the contextualized intent stream graph in real time.

[0171] It should be noted that the formulas described above, through the principle of dimensional consistency and mathematical standardization methods (such as normalization, dimensionless parameter conversion, or unit system unification), can translate physical quantities with different properties into unitless standard values ​​or superimposed parameters of the same dimension. This eliminates the interference of different dimensions on the computational logic, allowing the formulas to retain the original data distribution characteristics while possessing mathematical rationality and adaptability to objective laws. The descriptions are merely exemplary embodiments of the present invention and should not be construed as limiting the scope of the invention.

[0172] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0173] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A natural language-based navigation control method for mobile robots, characterized in that, Includes the following steps: S1. Collect multimodal sensing data and encapsulate it into events to generate a dynamic priority event stream; S2. Perform distributed service parsing on the dynamic priority event stream to generate a set of discretized intent fragments; S3. Construct a contextualized intent flow graph by performing temporal association and semantic fusion based on the discretized intent fragment set and historical context; S4. Analyze the structure and evolution trend of the contextualized intent flow graph, identify the final target node and key path in the contextualized intent flow graph, and transform them into the main task; Extract the attribute requirements implied by nodes on the critical path; Based on the attribute requirements, preloaded algorithm models or perception modes are mapped to generate predictive task instruction packages that include task triggering conditions and resource configuration lists. The attribute requirements include avoiding moving obstacles and precise docking. The resource configuration list includes the names of algorithm models that need to be preloaded into memory, the sensor working modes that need to be switched, and their parameters. S5. Based on the task triggering conditions and real-time environmental data in the predictive task instruction package, dynamically adjust the environmental modeling granularity to generate a multi-resolution fusion map. S6. Transform the safety constraints parsed in the contextualized intent flow graph into high-weight terms in the path search cost function, perform segmented path dynamic optimization on the multi-resolution fusion map, and generate the robot's motion trajectory. S7. Convert the robot's motion trajectory into a sequence of underlying motor control instructions to drive the robot to perform physical actions and generate a robot execution status log. S8. Feed back the robot's execution status log to the contextualized intent flow engine to update and correct the contextualized intent flow graph in real time. The robot's execution status log is parsed into fact nodes, and the fact nodes are connected to the corresponding intent nodes in the contextualized intent flow graph and incorporated into the contextualized intent flow graph. The system compares the fact nodes with the expected paths in the contextualized intent flow graph. If a deviation is detected, an evaluation process is triggered. Based on the evaluation results, nodes are inserted or paths are replanned in the contextualized intent flow graph.

2. The natural language-based mobile robot navigation control method according to claim 1, characterized in that, Generating a dynamic priority event stream includes the following steps: Simultaneously activate the microphone, LiDAR, and vision sensor to continuously capture voice data, 3D point cloud, and video data; Speech data is decomposed into word segments or phrase fragments with confidence scores, and 3D point cloud and video data are parsed into environmental object features containing object category, location and motion vectors; Priority values ​​are assigned to each voice data segment and environmental object feature according to a preset security rule base; All data units carrying priority values ​​are encoded in timestamp order and merged to generate a dynamic priority event stream.

3. The natural language-based mobile robot navigation control method according to claim 1, characterized in that, Generating a set of discretized intent fragments includes the following steps: Publish the dynamic priority event stream to the message bus and broadcast it to the parallel-running intent parsing service pool; Each service captures events according to subscription rules and generates intent fragments containing semantic tags, timestamps, and associated event identifiers; Aggregate and encapsulate all intent fragments generated by services within a fixed time window to generate a discretized set of intent fragments; The intent parsing service pool includes at least location entity recognition service, action intent extraction service, and referential relationship resolution service. The referential relationship resolution service is used to parse the real environmental object referred to by pronouns or indicator words in a specific context.

4. The natural language-based mobile robot navigation control method according to claim 1, characterized in that, Constructing a contextualized intent flow graph includes the following steps: Receive a set of discretized intent fragments and retrieve historical intent nodes as context; New intent node candidates are generated based on the timestamp of the intent fragment, and the semantic association between them and historical nodes is calculated. Connect new nodes with semantic relevance exceeding a threshold to historical nodes; otherwise, create new intent branches. For nodes containing pronouns, search for the nearest context entity nodes and establish referencing edges to generate a contextualized intent flow graph with a directed acyclic graph structure.

5. The natural language-based mobile robot navigation control method according to claim 1, characterized in that, Generating a multi-resolution fused map includes the following steps: Based on the main task in the predictive task instruction package, plan the macroscopic path skeleton on the global semantic map; During the travel phase in areas where the obstacle density is detected to be lower than a preset density threshold, coarse-grained environment modeling is performed. When the task triggering conditions are met, the target area will be switched to centimeter-level resolution modeling; By stitching together coarse-grained environmental maps with centimeter-level resolution maps, a multi-resolution fused map is generated.

6. The natural language-based mobile robot navigation control method according to claim 1, characterized in that, Generating the robot's motion trajectory includes the following steps: Transform the security constraints parsed in the contextualized intent flow graph into high-weight terms in the path search cost function; A first path search algorithm is used to generate global traversable paths in coarse-grained areas of a multi-resolution fused map. A second optimization algorithm is used in fine-grained areas on the multi-resolution fused map to generate local fine paths by combining real-time sensor data; The connection between the global path and the local fine path is smoothed to generate the robot's motion trajectory.

7. The natural language-based mobile robot navigation control method according to claim 1, characterized in that, Generate a robot execution status log, including the following steps: The trajectory is differentiated to calculate the target linear velocity and angular velocity at each time step; Based on the robot's kinematics model, the target velocity is converted into the target rotational speeds of the left and right wheel motors; A pulse width modulation signal is generated based on the difference between the target speed and the actual speed to drive the motor; The robot continuously collects its actual position, speed, task progress, and sensor readings to generate a robot execution status log.

8. A natural language-based mobile robot navigation control system, characterized in that, Includes the following modules: The event stream generation module is used to collect multimodal sensing data and encapsulate it into events to generate dynamic priority event streams. The intent fragment generation module is used to perform distributed service parsing of dynamic priority event streams and generate a set of discretized intent fragments. The intent flow graph construction module constructs a contextualized intent flow graph by performing temporal association and semantic fusion based on a set of discretized intent fragments and historical context. The task instruction generation module is used to parse the structure and evolution trend of the contextualized intent flow graph, identify the final target node and critical path in the contextualized intent flow graph, and transform them into the main task; Extract the attribute requirements implied by nodes on the critical path; Based on the attribute requirements, preloaded algorithm models or perception modes are mapped to generate predictive task instruction packages that include task triggering conditions and resource configuration lists. The attribute requirements include avoiding moving obstacles and precise docking. The resource configuration list includes the names of algorithm models that need to be preloaded into memory, the sensor working modes that need to be switched, and their parameters. The fusion map generation module dynamically adjusts the environmental modeling granularity based on the task triggering conditions in the predictive task instruction package and real-time environmental data to generate a multi-resolution fusion map. The motion trajectory generation module is used to transform the safety constraints parsed in the contextualized intent flow graph into high-weight terms in the path search cost function, perform segmented path dynamic optimization on the multi-resolution fused map, and generate the robot's motion trajectory. The execution and log generation module is used to convert the robot's motion trajectory into a sequence of underlying motor control instructions, drive the robot to perform physical actions, and generate a robot execution status log. The feedback and update module is used to feed back the robot's execution status log to the contextualized intent flow engine, and to update and correct the contextualized intent flow graph in real time. The robot's execution status log is parsed into fact nodes, and the fact nodes are connected to the corresponding intent nodes in the contextualized intent flow graph and incorporated into the contextualized intent flow graph. The system compares the fact nodes with the expected paths in the contextualized intent flow graph. If a deviation is detected, an evaluation process is triggered. Based on the evaluation results, nodes are inserted or paths are replanned in the contextualized intent flow graph.

Images