A body-composite robot semantic topology perception and re-planning system

Abstract

The application discloses a body-combined robot semantic topology perception and re-planning system and belongs to the field of robot environment perception and path planning. The system organizes objects, regions, stations and dynamic events in a work environment into a semantic node set, organizes passable relations or work connection relations between nodes into a topology connection relation set, and associates passable constraints, risk labels and time efficiency labels to nodes or topology edges; when environmental changes meet triggering conditions, locally incrementally update affected nodes and topology connection relations; then generate a candidate path set according to a target task, score in combination with semantic constraints and passable costs, generate a planning or re-planning path by using alternative paths, alternative passable relations and auxiliary action nodes, and share local update results through a map version number and a local revision number. The application is suitable for navigation, carrying, picking and placing, inspection and multi-robot collaborative work in a dynamic environment.

Description

Technical Field

[0001] This invention relates to the field of embodied robot environmental perception and path planning technology, and in particular to an embodied composite robot semantic topology perception and replanning system, and to corresponding methods and storage media, applicable to navigation, picking and placing, handling, collaborative operation and task adjustment of mobile operation robots, composite robots, warehouse robots, inspection robots and service robots in dynamic environments. Background Technology

[0002] Embossed robots typically need to perform continuous tasks in open or semi-open environments. Unlike traditional automated equipment that operates only at fixed workstations, embodied robots not only need to identify obstacles and traversable areas in the environment, but also need to understand the semantic attributes of the work object, the functional meaning of the environmental area, and the impact of dynamic events on task execution.

[0003] Most existing environmental perception and planning solutions are based on geometric maps, occupancy grid maps, or local obstacle avoidance strategies. These solutions can accomplish obstacle detection and path finding to a certain extent, but they usually focus primarily on spatial geometry and lack a unified expression of area functions, object attributes, human activity patterns, and task-related semantics.

[0004] In dynamic environments, relying solely on geometric information for planning can easily lead to the following problems: First, while robots can determine whether a certain area is passable, they struggle to determine whether that area is suitable as the preferred path for the current task. Second, when the environment changes, traditional systems often only detour around local obstacles, lacking the ability to select overall alternative paths based on semantic relationships. Third, in scenarios involving human-robot interaction or multi-robot collaboration, geometric maps struggle to express semantic information such as personnel activity areas, temporary work areas, and priority workstations, resulting in inconsistencies between path planning results and on-site task constraints.

[0005] Furthermore, in tasks such as grasping, handling, placing, or inspecting, embodied robots typically need to consider the movement path in conjunction with the operational actions. For example, whether reaching a certain position facilitates the robotic arm's approach to the target, whether the current passage allows the passage of a load, and whether a certain area is temporarily unsuitable for entry due to personnel passing through all affect the overall task execution efficiency. If the planning system cannot model these constraints at the semantic level, it is easy to encounter problems where the path is short but not conducive to subsequent operations.

[0006] In multi-robot collaborative scenarios, environmental changes typically exhibit a pattern of local detection followed by global propagation. If one robot detects congestion, workstation occupancy, or personnel entry within its local area, other robots, if unable to obtain this information in a timely manner, will still follow their original paths into the high-risk area, leading to repeated obstacle avoidance, repeated mapping, or increased task waiting time.

[0007] Prior art document CN121835740A discloses an embodied intelligent multimodal perception robustness enhancement method based on feature network topology. It improves perception robustness through multimodal feature nodeization, feature network construction, community detection, and fault node repair. It primarily addresses topological redundancy and information repair in the feature layer, but does not construct a semantic topology map for the robot's operating environment, nor does it disclose a scheme for path planning and dynamic replanning based on environmental semantic nodes, topological connectivity, and traffic constraints. Prior art document CN121048642A discloses a robot trajectory prediction method based on time-frequency wavelet transform and graph networks. Its graph structure is used for spatiotemporal coupling modeling of historical trajectory features, aiming at trajectory prediction rather than expressing environmental semantics such as work areas, workstations, personnel activity areas, and passageway states. Prior art document CN121245791A discloses a robot control method based on large model combination. It generates sub-tasks and action increments through multi-view images and a large visual language model, but does not disclose dynamic updates of the semantic topology map, alternative traffic relationships, or multi-robot local semantic sharing mechanisms. The authorized patent CN117140527B mainly involves training of the underlying control strategy of a robotic arm based on DDPG, but does not involve upper-level environmental semantic topology modeling and path replanning.

[0008] Therefore, there is a need for a technical solution that can structure objects, regions, workstations, and dynamic events in the robot's working environment into a planarable semantic topology map, and can incrementally update the affected local map when environmental changes meet the triggering conditions. This solution can then perform path planning and replanning based on semantic path scoring, alternative passage relationships, and auxiliary action nodes, and support multi-robot versioned local semantic sharing, in order to improve the path planning adaptability, task execution efficiency, and collaborative capabilities of embodied composite robots in dynamic scenarios. Summary of the Invention

[0009] The purpose of this invention is to provide a semantic topology perception and replanning system for embodied composite robots, and further to provide corresponding methods and storage media to solve the problems in the prior art, such as the separation of environmental semantic information and spatial connectivity, the difficulty in timely reflecting dynamic environmental changes in the planning model, the lack of task semantic constraints in path planning results, and the difficulty in sharing environmental understanding among multiple robots.

[0010] To address the aforementioned technical problems, this invention uses a set of semantic nodes as the foundation for environmental semantic expression, a set of topological connection relationships as the carrier for regional accessibility and task coordination, and a set of traffic constraints as constraints for path selection and task executability. It organizes environmental understanding, local incremental updates, candidate path scoring, alternative traffic relationship search, auxiliary action node insertion, and multi-robot local update sharing into a unified processing link. Therefore, the information utilized by the robot during planning is no longer limited to obstacle location and distance information, but also includes environmental semantics directly related to the task outcome, such as regional function, object status, personnel activity, workstation occupancy, risk level, passage timeliness, and map version.

[0011] Based on the above ideas, the system provided by this invention includes an environment perception module, a semantic topology map construction module, a dynamic semantic update module, a semantic-guided planning and replanning module, a multi-robot semantic sharing module, and a motion execution module. The environment perception module is responsible for acquiring object information, obstacle information, spatial access information, and dynamic change information in the working environment; the semantic topology map construction module is responsible for constructing a semantic topology map including a set of semantic nodes, a set of topological connections, and a set of access constraints, and associating node type, spatial range, status label, risk label, and timeliness information with corresponding nodes or topological edges; the dynamic semantic update module is responsible for locating affected nodes and topological connections when environmental changes meet triggering conditions, and performing local incremental updates; the semantic-guided planning and replanning module is responsible for generating a set of candidate paths around the semantic nodes corresponding to the target task, scoring paths based on semantic constraints and access costs, and generating replanned paths based on the set of alternative paths and alternative access relationships when a path fails; the multi-robot semantic sharing module is responsible for time alignment, confidence fusion, conflict resolution, and version sharing of local update results; and the motion execution module is responsible for outputting robot control quantities based on the planning results.

[0012] In this invention, the semantic topology map is not simply a set of scene descriptions, but rather a structured graph data comprising a set of semantic nodes, a set of topological connections, and a set of travel constraints. Semantic nodes represent objects, areas, workstations, or dynamic events in the environment that have task significance, and can record node type, node center location, node coverage area, current state, risk level, allowed action set, allowed entry conditions, update time, and effective time window. Topological connections represent the connectivity, reachability, adjacency, priority passage, or job coordination relationships between semantic nodes, and can record connecting nodes, passage direction, edge length, passage status, congestion level, load limit, safety risk, recommended passage time period, and most recent update time. Travel constraints limit the robot's entry conditions, passage conditions, and work conditions on corresponding nodes or topological edges.

[0013] In this invention, the semantic topology map construction module includes a semantic extraction unit, a topology relationship generation unit, and a semantic topology fusion unit. The semantic extraction unit is used to extract object categories, object attributes, location semantics, and behavioral semantics from multimodal perception data; the topology relationship generation unit is used to generate topology relationships between environmental regions based on environmental layout, connected regions, and reachable paths; and the semantic topology fusion unit is used to associate semantic information with topology relationships to obtain a semantic topology map that can be directly called by the planning module.

[0014] In this invention, the dynamic semantic update module does not reconstruct the entire map indiscriminately, but rather performs local incremental updates on semantic nodes, topological connections, and their adjacent topological connections affected by environmental changes. These environmental changes include at least changes in target object location, the appearance or disappearance of obstacles, personnel entering or leaving the work area, changes in area occupancy status, and changes in passageway capacity. These changes are triggered when the target object displacement exceeds a preset displacement threshold, the passage occupancy rate exceeds a preset occupancy threshold, personnel dwell time exceeds a preset duration, the status of critical workstations changes, or the path risk score exceeds a preset upper limit. After the update, the system adds status labels, risk labels, timeliness labels, or local revision numbers to the affected semantic nodes and topological edges, enabling subsequent path calculations to directly utilize the change results.

[0015] In this invention, the semantic-guided planning and replanning module first generates a set of candidate paths based on the semantic nodes corresponding to the target task during path planning. Then, it forms path semantic constraints based on at least two of the following: target semantic priority, area accessibility semantics, obstacle risk semantics, personnel interaction semantics, and task semantics. These constraints are then combined with at least two of the following: path length cost, congestion cost, risk cost, waiting cost, task matching cost, and operational accessibility cost. After normalization, candidate path scores are generated according to weight parameters corresponding to the task type. The mechanism of this processing method is to expand the path selection criteria from a single geometric distance to a comprehensive evaluation process involving semantic constraints, accessibility costs, and subsequent operational accessibility, ensuring that the path selection result is coordinated with subsequent grabbing, placement, inspection, or human-machine avoidance behaviors.

[0016] During path execution, when the target area's state, access conditions, or obstacle distribution changes, the semantic guidance planning and replanning module generates a replanned path based on semantic node associations, a set of alternative paths, and alternative access relationships. These alternative access relationships include alternative topological edges adjacent to failed topological edges, alternative topological edges with valid associations to the target semantic node, or alternative access links generated after dynamic semantic updates based on the current set of reachable nodes. If necessary, the system also inserts auxiliary action nodes. These auxiliary action nodes are associated with the semantic node or topological connection that triggered their insertion and include the action type, target semantic node, and execution conditions. Auxiliary action nodes include at least one of the following: adjusting position, bypassing temporary obstacles, waiting for personnel to pass, switching target approach direction, switching grasping pose, switching operation sequence, retracting the robotic arm to a safe posture, and verifying perception. By establishing this connection between the path layer and the action layer, the situation of "reaching the area but the action cannot continue" can be reduced.

[0017] The multi-robot semantic sharing module is used to share at least some semantic node information, topological connections, and environmental change information from a semantic topology map among multiple robots. The shared information does not directly overwrite the local map; instead, it undergoes time alignment, confidence fusion, and conflict resolution processing, and is then assigned a source identifier, timestamp, confidence value, spatial extent identifier, map version number, and local revision number as a local update result, which is then distributed to each robot. This approach reduces the impact of time differences, duplicate observations, state conflicts, and map version inconsistencies between uploaded information from multiple robots on the planning results, and ensures that each robot obtains a more consistent local environmental perception before entering the relevant area.

[0018] Based on the above system, this invention also provides a semantic topology perception and replanning method for embodied composite robots. This method includes at least: acquiring object information, obstacle information, spatial access information, and dynamic change information in the robot's operating environment; constructing a semantic topology map based on the acquired information, including a set of semantic nodes, a set of topological connections, and a set of access constraints; when environmental changes meet triggering conditions, performing local incremental updates on the affected semantic nodes, topological connections, and their adjacent topological connections; generating a candidate path set around the semantic nodes corresponding to the target task, scoring and ranking the candidate paths according to semantic constraints and access costs to obtain a planned path and a set of alternative paths; when the current path fails, generating a replanned path based on semantic node associations, the set of alternative paths, and alternative access relationships; sharing local update results with map version numbers and local revision numbers in multi-robot scenarios; and outputting control quantities for the mobile chassis, robotic arm, or end effector based on the planning results.

[0019] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects.

[0021] First, by setting up a semantic topology map construction module, environmental semantic information and spatial connectivity are uniformly organized into a set of semantic nodes, a set of topological connections, and a set of travel constraints. Since semantic nodes and topological connections are further associated with status labels, risk labels, timeliness labels, and travel costs, the planning module can directly understand the regional function, target status, travel conditions, and subsequent operational accessibility based on this association structure. Therefore, it can improve the consistency between path planning results and specific task requirements.

[0022] Second, by setting up a dynamic semantic update module, and only performing local incremental updates on the affected semantic nodes, the affected topological connections, and their adjacent topological connections when environmental changes meet the triggering conditions; since the planning basic data can reflect on-site changes such as object movement, area occupancy, passage obstruction, and personnel activities in a timely manner, while avoiding indiscriminate reconstruction of the entire map, the system's adaptability to dynamic environments can be improved and the overhead of repeated global mapping can be reduced.

[0023] Third, by setting up semantic-guided planning and replanning modules, and introducing target semantic priority, regional access semantic, obstacle risk semantic, personnel interaction semantic, task semantic, as well as cost terms such as path length, congestion, risk, waiting, task matching, and operational accessibility into path sequencing; since path selection no longer depends solely on geometric distance, but simultaneously satisfies task execution constraints and subsequent operational conditions, the executability and task adaptability of planned and replanned paths can be improved.

[0024] Fourth, by introducing auxiliary action nodes associated with semantic nodes or topological connections during the replanning process; since the auxiliary action nodes record action type, target semantic node and execution conditions, the movement path adjustment can establish a corresponding relationship with the robotic arm's approach direction, waiting strategy, grasping pose, work sequence switching, safety posture switching or verification perception, thus reducing the ineffective positioning phenomenon caused by the disconnect between the path layer and the action layer, and improving the overall operation continuity of the embodied composite robot.

[0025] Fifth, by setting up a multi-robot semantic sharing module and performing time alignment, confidence fusion, conflict resolution, map version number allocation, and local revision number management on the shared information, each robot can learn about the versioned local environmental changes in advance before entering the target area and avoid the old state from overwriting the new state. This reduces repeated perception and detours and improves the consistency of multi-robot collaborative operations. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall structure of the embodied composite robot semantic topology perception and replanning system of the present invention.

[0027] Figure 2 This is a schematic diagram of the semantic topology map construction process of the present invention.

[0028] Figure 3 This is a schematic diagram of the dynamic semantic update process of the present invention.

[0029] Figure 4 This is a schematic diagram of the semantic-guided path planning and replanning process of the present invention.

[0030] Figure 5 This is a schematic diagram of the multi-robot semantic sharing process of the present invention.

[0031] Figure 6 This is a schematic diagram of the deployment structure of the embodied composite robot of the present invention. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Where there is no conflict, the technical features of the various embodiments can be combined with each other.

[0033] In this embodiment, the embodied composite robot includes a mobile chassis, a robotic arm, an end effector, a multimodal sensing component, and a controller. The mobile chassis is used to perform autonomous movement and positioning adjustments in the working environment. The robotic arm is used to perform grasping, picking up, placing, moving, transporting, or other operations. The end effector can be a gripper, a suction cup, a special tool head, or a detection probe. The multimodal sensing component is used to acquire visual information, depth information, laser scanning information, positioning information, and contact information from the environment.

[0034] Figure 1 The diagram shows the overall structure of the system of this invention. The environmental perception module collects information on object categories, object locations, area boundaries, obstacle states, personnel activity information, and passage status in the working environment. The semantic topology map construction module transforms the perception results into semantic nodes and topological connections, and organizes relevant passage constraints, status labels, risk labels, map version numbers, and local revision numbers into a semantic topology map data layer for planning. The dynamic semantic update module updates node attributes, edge states, and passage constraints after environmental changes occur. The semantic-guided planning and replanning module selects the target path based on the updated semantic topology map. The multi-robot semantic sharing module is used to synchronize local update results among multiple robots. The motion execution module outputs corresponding control quantities for the chassis, robotic arm, or end effector based on the planning results. Figure 1 The semantic topology map data layer is used to illustrate the logical storage and retrieval relationships of semantic nodes, topological connection relationships, access constraints, status labels, and version management information. It is not an independent functional module that is distinct from the aforementioned six modules.

[0035] The semantic nodes can be set according to the scenario type. For example, in a warehousing scenario, semantic nodes may include shelving areas, material receiving areas, material discharging areas, temporary storage areas, charging areas, restricted areas, and personnel passage areas; in an assembly scenario, semantic nodes may include loading stations, assembly stations, quality inspection stations, tool buffer stations, personnel standing areas, and areas to be avoided; in a service scenario, semantic nodes may include service counters, high-frequency customer areas, waiting areas, and cleaning areas. Through node-based representation, robots can directly utilize regional semantics when selecting paths.

[0036] Each semantic node, in addition to recording the node category, also records the node's center location, coverage area, current status, risk level, status label, set of allowed actions related to the task, allowed entry conditions, update time information, and effective time window. For target object nodes, it can also record the object type, object size, workstation to which the object belongs, graspable direction, and whether it is currently occupied; for area nodes, it can also record information such as area priority, passage capacity, load limit, time window, whether personnel and robots are allowed to coexist, and whether it is in a reserved occupancy state. Through the above field configuration, semantic nodes in different scenarios have a unified data structure, which facilitates the path planning module's use.

[0037] Topological connections can be represented using directed or undirected graphs. Each topological edge, in addition to representing the connectivity between two semantic nodes, can also record at least one of the following information: node identifier, edge length, traversable direction, current congestion level, load limit, security risk, risk level, travel cost, recommended travel time, last update time, and local revision number. Therefore, when selecting a path, the system considers not only geometric distance but also task-related semantic constraints and travel time.

[0038] Topological edges can also include a passage cost term. This cost term can be determined by a combination of at least two of the following: edge length cost, congestion cost, risk cost, waiting cost, and task matching cost. For tasks that require priority avoidance of personnel activity areas, the risk cost weight can be appropriately increased; for replenishment tasks with high timeliness requirements, the waiting cost weight can be appropriately increased; for heavy-load transportation tasks, the constraint strength related to load limits can be increased. Through the organization of cost terms, the system compares paths for multiple types of tasks in a unified manner.

[0039] Figure 2The diagram illustrates the semantic topology map construction process. The environment perception module first acquires raw perception data from cameras, depth sensors, LiDAR, and positioning devices. The semantic extraction unit performs target recognition, scene segmentation, and region semantic annotation on the perception data to obtain object categories, region attributes, and status labels. The topology relationship generation unit generates candidate topology edges based on the environment layout, connected regions, and path reachability. The semantic topology fusion unit associates semantic nodes with topology edges to form a semantic topology map, which is stored in the semantic topology map storage area at the ontology or edge level for use by the planning and sharing modules.

[0040] The semantic extraction unit extracts not only static semantics but also dynamic semantics. For example, when it detects that a person is pushing a cart into a channel, temporary goods are occupying a transfer area, or a workstation is being manually operated, these can be labeled with semantic states such as "temporary occupation," "high interaction risk," or "waiting required," respectively. Through dynamic semantic labeling, subsequent replanning can better reflect actual operational needs.

[0041] Semantic topology fusion units can be organized hierarchically during the fusion process. The bottom layer maintains regional connectivity and geometric reachability; the middle layer records regional functional semantics, object relational semantics, and event semantics; and the top layer records priority information, process dependency information, and operational constraint information directly related to the task. The advantage of this hierarchical organization is that when only geometric changes occur, the bottom and middle layers can be locally updated; when task state changes, the high-level constraints are further updated without having to recalculate all relationships.

[0042] Figure 3 The diagram illustrates the dynamic semantic update process. When the environmental perception module detects changes in object location, the addition or removal of obstacles, blocked passages, personnel entering hazardous areas, or changes in workstation status, the dynamic semantic update module updates the attributes of the affected semantic nodes and simultaneously adjusts the reachability, risk level, timeliness marker, and local revision number of the relevant topological edges and their adjacent topological edges. If the change only affects a local area, only the relevant local graph is updated; if the change affects the main path or critical workstations, the update scope of the affected local graph is expanded while maintaining the incremental update method, and the semantic-guided planning and replanning module triggers local or global replanning based on the update results.

[0043] The dynamic semantic update module can employ an incremental update mechanism. Unaffected nodes and topological edges retain their original state, while only the affected parts are recalculated. This reduces real-time update overhead and improves the system's response speed in dynamic scenarios.

[0044] The dynamic semantic update module allows setting change trigger conditions. These conditions may include at least one of the following: object displacement exceeding a preset threshold, channel occupancy exceeding a preset ratio, personnel dwell time exceeding a preset duration, key workstation status switching, and path risk score exceeding a preset upper limit. The system only performs a formal update on the corresponding nodes and topology edges when the change trigger conditions are met, thereby reducing frequent replanning due to short-term noise or momentary false detections.

[0045] For time-sensitive status information, the dynamic semantic update module can assign a valid time window. For example, if information about a channel being temporarily occupied does not receive a new confirmation of occupancy after a preset validity period, it will automatically be downgraded to a pending confirmation status; the interaction risk level of a high-frequency activity area can be automatically reduced after entering a rest period. By introducing time-sensitive processing, the dynamic semantics in the map will not be retained indefinitely, which helps improve the consistency between the planning results and the actual on-site state.

[0046] Alternative travel relationships can be pre-recorded as a set of candidate topological edges between semantic nodes, or they can be regenerated based on the current set of reachable nodes after dynamic semantic updates. When the current path fails, the semantic-guided planning and replanning module prioritizes searching for alternative travel relationships among candidate topological edges that are adjacent to the failed topological edge, have a valid association with the target semantic node, or match the current robot load state. If no candidate topological edge that satisfies the travel constraints exists within a local area, the search scope is further expanded to perform global replanning. By structurally representing alternative travel relationships, the replanning process has clear search boundaries and switching criteria.

[0047] Figure 4 The diagram illustrates the semantic-guided path planning and replanning process. Upon receiving the target task, the system first determines the target semantic nodes based on the task objective, such as "reaching assembly station A and retrieving the part," "going to shelf B and grabbing the target box," or "detouring through a densely populated area to reach inspection point C." Subsequently, the semantic-guided planning and replanning module generates a candidate path set based on the semantic topology map. It then generates candidate path scores according to normalized path length, access semantics, task priority, obstacle risk, personnel interaction constraints, waiting costs, and operational accessibility costs, using weighted parameters corresponding to the task type. Finally, it selects the target path and retains the set of alternative paths.

[0048] When generating a candidate path set, the semantic-guided planning and replanning module can not only use single-path search, but also first filter candidate regions by target semantic nodes, and then generate alternative passageways between candidate regions. For tasks that require entering a specific workstation and performing robotic arm actions, the system prioritizes generating candidate paths that facilitate chassis docking and robotic arm deployment; for tasks that only require inspection passage, it prioritizes generating candidate paths with high coverage and stable passage. This avoids using the exact same path evaluation criteria for different task categories.

[0049] The path ranking result can be represented as an ordered set of the target path and several alternative paths. While executing the main path, the system can also retain one or more alternative paths and their applicable triggering conditions. These applicable triggering conditions may include changes in the target area's state, deterioration of traffic conditions, temporary obstacles entering the path, the target object being moved, personnel entering a high-interaction-risk area, or a change in the state of a critical workstation. Therefore, when the main path becomes invalid due to local semantic changes, the system prioritizes switching to existing alternative paths instead of re-searching the entire topology from scratch, thereby shortening the replanning response time.

[0050] During path execution, if a change in the current target area status, deterioration of passage conditions, the entry of a temporary obstacle into the path, or the movement of the target object is detected, the dynamic semantic update module sends an update result to the semantic guidance planning and replanning module. Based on this, the semantic guidance planning and replanning module searches for alternative paths that meet the task semantic constraints again, and inserts auxiliary action nodes when necessary, such as adjusting the position, waiting for personnel to pass, switching the grabbing side, detouring around the temporarily occupied area, or changing the work sequence.

[0051] Compared to replanning based solely on geometric distance, the replanning in this invention is constrained by semantic relationships. For example, when two traversable detour paths exist, the system can prioritize the path that is more suitable for the robotic arm to approach the target, or prioritize a safe path away from areas with high human activity. For tasks requiring coordinated grasping and placement, the system can also prioritize the path that facilitates maintaining the placement posture.

[0052] Auxiliary action nodes exist as extended attributes of path nodes and also as independent action sequences. Each auxiliary action node records the action type, target semantic node, associated path node, triggering reason, and execution condition. When a path requires the robot to complete a positioning adjustment before entering the grasping state, the system can insert a "position correction" auxiliary action node before the target node; when a detour path requires the robotic arm to retract to a safe posture first, the system can insert a "posture switching" auxiliary action node; when the confidence of the current local semantic state is insufficient, the system can insert a "verification and perception" auxiliary action node. Through this representation, the path adjustment result can be directly mapped to the motion execution module, eliminating the need for manual supplementation of action logic.

[0053] Figure 5 The diagram illustrates the multi-robot semantic sharing process. Multiple robots perform perception and tasks within their respective local areas, uploading changes in local semantic nodes, topological edge states, and environmental events to the multi-robot semantic sharing module. The module performs time alignment, confidence weighting, and conflict resolution on the information uploaded by different robots, generating a shared update result, which is then synchronized to the relevant robots.

[0054] When robot A detects that a passage is temporarily occupied by goods and robot B has not yet arrived at the area, robot B can learn about the passage's status in advance through the multi-robot semantic sharing module and adjust its path before entering the area. This reduces unnecessary detours caused by local information delays.

[0055] The multi-robot semantic sharing module can assign source identifiers, timestamps, confidence values, spatial range identifiers, map version numbers, and local revision numbers to shared information. If two robots give different conclusions about the state of the same node, the information that is more recent, has a higher confidence value, an updated local revision number, or a higher correlation with the target area will be prioritized. If a decision still cannot be made, the node will be marked as pending confirmation, and a verification process will be triggered when the relevant robot approaches the area. This conflict resolution method helps prevent erroneous information sharing or older version information from directly overwriting the local map.

[0056] The multi-robot semantic sharing module only synchronizes local map changes, not the complete map. For status updates affecting only the area surrounding a single workstation, a local update result with the map version number and local revision number can be sent only to robots that are about to pass through that workstation or are related to the task at that workstation. Using local synchronization reduces communication load and improves the applicability of the sharing mechanism under edge network conditions.

[0057] Figure 6The diagram illustrates the deployment structure of the embodied composite robot. It highlights the deployment relationship between the robot body and the edge server; in other implementations, the control system can be further deployed on a cloud platform. The environmental perception module can be deployed on the robot body, while the semantic topology map construction module, dynamic semantic update module, and semantic guidance planning and replanning module can be deployed on the robot body or the edge. The multi-robot semantic sharing module can be deployed on the edge server, or, when cross-regional collaboration is required, extended to the cloud platform to adapt to different communication conditions and real-time requirements in various scenarios.

[0058] The body controller is primarily responsible for real-time perception access, local node updates, and control command output; the edge server is mainly responsible for local sharing and fusion, global semantic topology maintenance, and multi-robot conflict coordination; the cloud platform, as an optional extended deployment location, can be used for historical event archiving, cross-time period statistical analysis, and parameter configuration distribution. The advantage of this distributed deployment approach is that, while ensuring the real-time performance of body control, it can utilize edge or cloud computing resources to handle multi-robot sharing and long-term optimization tasks.

[0059] The following describes the operation of this invention using a warehouse handling scenario as an example. The task objective is to "remove a designated material box from the shelving area and deliver it to the assembly station." The system first generates a semantic topology map based on the warehouse environment, including the shelving area, main aisle, detour aisle, assembly station, personnel access area, and temporary buffer area, and labels the semantic attributes and access constraints of each area. The robot selects an initial path from the main aisle to the shelving area according to the target task.

[0060] When the robot detects that the main passage is temporarily occupied by a forklift during its movement, the dynamic semantic update module updates the corresponding topological edge status to "temporarily high risk, cannot be passed first" and synchronizes the relevant information to the semantic guidance planning and replanning module. The latter regenerates candidate paths based on the semantic topology map, compares the main passage waiting plan and the detour path plan, and selects the detour path as the replanning path if the detour path, although slightly longer, meets the current load passage restrictions and is more conducive to completing the task on time, even though it is longer.

[0061] Once the robot reaches the shelving area, if it finds temporary stacked boxes blocking the path in front of the target bin, the system can insert auxiliary action nodes such as "adjust position," "switch gripping side," or "wait for obstacle clearing" to ensure that the movement path adjustment is consistent with the robotic arm's operation. This avoids the problem of failing to grasp the target bin despite reaching it, due to improper positioning.

[0062] In warehousing scenarios, different semantic priorities can be set for different types of channels. For example, main channels are suitable for long-distance straight-line transportation, narrow channels are suitable for short-distance replenishment, and manual picking channels are not considered priority paths during specific time periods. In this way, when multiple candidate paths are close in length, the system prioritizes the channel that better matches the current load status and task category, thereby reducing subsequent secondary adjustments.

[0063] Let's take a human-robot collaborative assembly scenario as an example to illustrate the application of this invention. The assembly area simultaneously contains robots, human operators, and mobile tooling carts. When a robot moves to an assembly station, it considers not only the path length but also whether the station is currently under human operation, the density of personnel, and whether there are any temporarily restricted areas. When the system detects that a human operator is working near a station, it can mark the relevant area as "high interaction risk" or "wait required," thereby guiding the robot to prioritize more natural avoidance paths.

[0064] Semantic nodes in an assembly scenario can contain process sequence states. If a workstation has not completed its preceding steps, although it may be geometrically reachable, it does not semantically meet the "currently workable" condition, and the system will not prioritize it as a target node. By writing the process sequence state into semantic nodes, the path planning results can be kept consistent with the on-site process, preventing the robot from prematurely entering workstations that are not yet open.

[0065] The semantic-guided planning and replanning module can also incorporate task priority into path selection. For example, for urgent material replenishment tasks, the weight related to time efficiency is increased; for high-risk handling tasks, the weight of safe passage semantics can be increased; and for collaborative assembly tasks, the weight of personnel interaction semantics and workstation occupancy semantics can be increased. Through adjustable weights, the same semantic topology map can be adapted to different task objectives.

[0066] Semantic nodes can also have task dependencies. For example, a certain workstation only enters the "workable" state after the preceding workstation has completed material unloading, and a certain area only becomes "accessible" after manual clearing. The system writes these task dependencies as special semantic constraints into the semantic topology map to ensure that the path planning results are consistent with the process flow.

[0067] Let's take an inspection scenario as an example to illustrate the application of this invention. The robot needs to sequentially visit several inspection points to complete status reading, image acquisition, and anomaly recording. The system can represent "inspection point," "high-risk equipment area," "restricted access area," "temporary maintenance area," and "temporarily closed passage" as different semantic nodes, and automatically update the status of relevant nodes after maintenance personnel enter. When an inspection point is temporarily inaccessible due to maintenance closure, the system can rearrange the inspection order according to alternative access relationships, rather than simply skipping all subsequent nodes.

[0068] The semantic topology map in the inspection scenario can record the semantics of observation requirements. For example, some equipment needs to acquire images from directly in front, some instruments need to be read under specific lighting conditions, and some areas need to be briefly stopped to complete gas detection. The system associates these observation requirements with path nodes, so that the path planning results not only determine "where to go," but also "how to arrive and stop," thereby improving the quality of inspection task completion.

[0069] This invention can also be applied to multi-robot collaborative handling scenarios. Robot A and Robot B are responsible for picking up and delivering goods to different areas, respectively. By sharing a semantic topology map, Robot A can share the relevant status with Robot B when it detects congestion in a certain passage, allowing Robot B to adjust its route in advance. If the two robots need to pass through a narrow passage together, they can also coordinate their timing based on the shared information to reduce conflicts.

[0070] In multi-robot collaborative scenarios, reservation statuses are set for key semantic nodes. When a robot determines that it will enter a narrow passage or workstation within a predetermined time window, it first marks the node as reserved. Other robots then reduce the priority of that node or choose a waiting strategy when prioritizing their paths. This approach can further reduce collision conflicts and repeated yielding among multiple robots in critical areas.

[0071] The semantic topology map maintains both a static layer and a dynamic layer. The static layer stores relatively stable regional topological relationships, workstation layouts, and functional area information, while the dynamic layer stores temporary obstacles, real-time occupancy status, personnel activity status, and risk labels. By combining the static and dynamic layers, the stability of the map representation can be maintained while supporting dynamic updates.

[0072] The system can also generate experience-based labels for certain areas based on historical semantic event logs. For example, if a passageway is frequently occupied during a specific time period, it can be labeled as a "high-congestion probability area"; if a workstation typically experiences high staff activity during shift change times, it can be labeled as a "workstation with high interaction risk." In subsequent planning, these experience-based labels can be used as prior information for path sequencing.

[0073] The system can maintain the correlation between path execution logs and semantic event logs. By associating a path switch, waiting action, detour action, and final execution result with the corresponding semantic change event, it can analyze which semantic changes are most likely to cause task delays, which alternative paths are most stable, and which areas are most suitable as buffer zones. These statistical results can be used for subsequent parameter tuning and map maintenance.

[0074] If the semantic-guided planning and replanning modules fail to find an alternative path that satisfies the constraints multiple times consecutively, the system triggers an exception handling process. This process may include outputting the reason for the failure, indicating that the constraints are not met, suggesting manual intervention, or switching to an alternative task. This prevents the robot from repeatedly searching in an ineffective state.

[0075] The anomaly handling process can also implement different handling strategies based on the cause of failure. When the cause of failure is temporary channel obstruction, the waiting and reconfirmation strategy is prioritized; when the cause of failure is long-term unavailability of the target workstation, the task transfer or workstation switching strategy is prioritized; when the cause of failure is uncertainty in perception, the verification perception or proximity observation strategy is prioritized. By classifying and handling these issues, anomaly handling can be correlated with specific failure mechanisms.

[0076] The motion execution module adopts a hierarchical control structure. The upper layer outputs phased pose targets and operation node targets based on the planned path, while the lower layer chassis controller and robotic arm controller perform trajectory tracking and motion control based on the targets. If a large deviation, approach failure, or local obstacle avoidance failure occurs during execution, the abnormal state is fed back to the dynamic semantic update module and the semantic guided planning and replanning module, triggering further adjustments.

[0077] The motion execution module switches control modes based on the semantic categories of path nodes. When entering a high-precision docking area, it prioritizes fine positioning and low-speed control modes; when entering a mixed-traffic area, it prioritizes safe speed-limiting modes; and when entering a long straight-through passage, it prioritizes efficient cruise modes. Establishing a correspondence between control modes and semantic node categories further improves the consistency between planning and execution results.

[0078] The various modules in the system interact through structured messages. The environment perception module outputs messages that include at least node candidate identifiers, location data, status data, and timestamps; the dynamic semantic update module outputs messages that include at least update type, affected node identifiers, affected edge identifiers, updated risk level, and local revision number; the semantic-guided planning and replanning module outputs messages that include at least path node sequences, candidate alternative path sequences, candidate path scores, and auxiliary action node sequences; and the multi-robot semantic sharing module outputs messages that include at least source identifiers, confidence values, map version numbers, and local revision numbers. Using structured message interaction facilitates system integration and subsequent maintenance.

[0079] The system also maintains task session identifiers. At the start of each task, a unified task session identifier is assigned to the corresponding planning, updating, sharing, and execution processes, enabling multi-module logs to be traced back along the same task chain afterward. This allows for quick identification of whether the problem occurred during semantic extraction, node updates, path sorting, or the execution feedback phase when a task fails or a path anomaly occurs.

[0080] The system maintains a map version number and a local revision number for the semantic topology map. After each dynamic semantic update, a local revision number is added to the affected local map. When the local revisions accumulate to a preset number, affect critical paths, or are confirmed by multiple robots, the global map version number is then synchronously increased. Through version management, multiple robots can clearly identify which version of node relationships and passage constraints are currently being used when sharing the same semantic topology map, thereby reducing path divergences caused by version inconsistencies.

[0081] The system determines whether to perform local or global replanning based on the location of affected nodes in the topology graph, the scope of impact, and changes in candidate path scores. Local replanning can be performed when environmental changes only affect the latter part of the current path and do not involve switching the main pathway. Global replanning can be performed when environmental changes cause the target workstation to fail, the main pathway to become unavailable, multiple key nodes to be affected simultaneously, or the candidate path scores are all below a preset threshold. By distinguishing between the boundaries of these two types of replanning, computational costs can be controlled while ensuring the accuracy of the results.

[0082] This invention can also be applied to hospital delivery, building services, and park logistics scenarios. Taking hospital delivery as an example, semantic nodes include nurses' stations, pharmacies, elevator lobbies, operating areas, restricted access areas, and temporary storage areas; semantic constraints include silent passage requirements, sterile area prohibition requirements, and peak-hour avoidance requirements. In this scenario, the system not only plans movement paths but can also determine whether to wait for elevators, whether to detour around high-frequency patient activity areas, and whether to switch delivery sequences based on regional semantics.

[0083] The semantic-guided planning and replanning modules simultaneously output interpretable result information. This interpretable result information includes at least the key semantic nodes upon which the current path selection was based, candidate path scores, the main reasons for discarding candidate paths, the type of environmental change triggering the replanning, the task constraints satisfied by the ultimately adopted alternative path, and the triggering reasons for inserting auxiliary action nodes. This output can be accessed by operations personnel, the scheduling system, or the host computer interface to quickly understand the system's decision-making basis when waiting, detours, or changes in task order occur.

[0084] The system pre-configures multiple parameter configuration files based on scenario type. These configuration files may include access risk thresholds, shared information confidence thresholds, effective time windows for dynamic states, local replanning trigger thresholds, and path weight parameters for different task categories. During on-site deployment, configurations can be quickly switched according to different application scenarios such as warehousing, assembly, inspection, and service, thereby shortening the debugging cycle and improving system adaptation efficiency. Parameter switching results can also be written to the task log for subsequent auditing and traceability.

[0085] This invention does not limit specific semantic extraction algorithms, topology construction algorithms, and path search algorithms. Any algorithm that can achieve the fusion of environmental semantic information and spatial topological relationships for mapping, dynamic semantic updates triggered by environmental changes, path planning and replanning based on semantic topological information, and semantic sharing among multiple robots falls within the scope of protection of this invention.

[0086] For computer implementation, the method of this invention can be deployed entirely on a single robot controller, or distributed across robot body controllers, edge servers, and cloud platforms. The computer-readable storage medium can be a hard disk, solid-state drive, flash memory, random access memory, read-only memory, or other media capable of storing program code. During program execution, it calls the environment perception module, semantic topology map construction module, dynamic semantic update module, semantic guided planning and replanning module, multi-robot semantic sharing module, and motion execution module to complete semantic topology map construction, local incremental updates, candidate path scoring, alternative travel relationship search, auxiliary action node insertion, versioned local sharing, and motion control output.

[0087] In summary, this invention provides a perception and planning technology solution for embodied composite robots suitable for dynamic environments by constructing a semantic topology map that integrates environmental semantic information and spatial connectivity, and combining it with dynamic semantic updates, semantic-guided path planning and replanning, and a multi-robot semantic sharing mechanism. This solution can improve the consistency between path planning and task semantics, and enhance the robot's adaptability and collaborative capabilities in complex dynamic scenarios.

[0088] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any equivalent substitutions, improvements, modifications, or variations made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0089] Explanation of reference numerals in the attached figures 1. A semantic topology perception and replanning system for embodied composite robots; 11. Environmental perception module; 12. Semantic topology map construction module; 13. Dynamic semantic update module; 14. Semantic guided planning and replanning module; 15. Multi-robot semantic sharing module; 16. Motion execution module; 21. Robot body; 22. Mobile chassis; 23. Robotic arm; 24. End effector; 25. Multimodal sensing component; 26. Controller; 27. Edge server; Figure 1 The “semantic topology map data layer” is a schematic diagram of the logical organization of semantic nodes, topological connection relationships, traffic constraints, status labels, map version numbers and local revision numbers, and does not correspond to independent map labels.

[0090] Figures 2 to 5This is a flowchart. The current attached figure does not have separate labels for semantic extraction unit, topology relationship generation unit, semantic topology fusion unit, semantic nodes, topology connection relationship, passage constraints, and status labels.

Claims

1. A semantic topology perception and replanning system for an embodied composite robot, characterized in that, The system includes an environmental perception module, a semantic topology map construction module, a dynamic semantic update module, a semantic-guided planning and replanning module, a multi-robot semantic sharing module, and a motion execution module. The environmental perception module collects object information, obstacle information, spatial access information, and dynamic change information from the robot's operating environment. The semantic topology map construction module constructs a semantic topology map based on the object information, obstacle information, spatial access information, and dynamic change information, including a set of semantic nodes, a set of topological connections, and a set of access constraints. Semantic nodes represent objects, areas, workstations, or dynamic events in the robot's operating environment and record node type, spatial range, status label, and update time or timeliness information. Topological connections represent traversable relationships or operational connections between semantic nodes and record connecting nodes, travel direction, travel status, risk level, or travel cost. The dynamic semantic update module locates the affected semantic nodes and topological connections when it detects environmental changes that meet preset trigger conditions, and updates the corresponding node attributes and topological connections. The semantic guidance planning and replanning module performs local incremental updates based on relationships, traffic constraints, status labels, risk labels, or timeliness labels. It generates a candidate path set based on the semantic nodes corresponding to the target task and the updated semantic topology map, generates candidate path scores based on semantic constraints and traffic costs, and determines the planned path and alternative path set based on the candidate path scores. When the target area status, traffic conditions, or obstacle distribution in the current path changes, a replanned path is generated based on semantic node relationships, the alternative path set, and alternative traffic relationships, and at least one of the path node sequence, alternative path sequence, and auxiliary action node sequence is output. The multi-robot semantic sharing module performs time alignment, confidence fusion, and conflict resolution on the local update results in the semantic topology map, and assigns a map version number and a local revision number to the shared local update results, enabling multiple robots to obtain consistent local environmental change information. The motion execution module outputs at least one of the following based on the planned path or replanned path: mobile chassis control quantity, robotic arm control quantity, or end effector control quantity.

2. The system according to claim 1, characterized in that, The semantic nodes in the semantic topology map are used to represent at least one of the following: target object, passable area, work station, restricted area, temporary obstacle area, personnel activity area, charging area or cache area. The topology connection relationship is used to represent at least one of the following: connectivity relationship, adjacency relationship, reachability relationship or priority passage relationship between semantic nodes.

3. The system according to claim 1, characterized in that, The semantic topology map construction module includes a semantic extraction unit, a topology relationship generation unit, and a semantic topology fusion unit. The semantic extraction unit is used to extract object categories, object attributes, location semantics, and behavioral semantics from environmental perception data. The behavioral semantics include at least one of personnel activity status, area occupancy status, target object status, or workstation operation status. The topology relationship generation unit is used to generate connectivity and constraint relationships between environmental areas. The semantic topology fusion unit is used to associate the extracted semantic information with the topology relationships to form the semantic topology map.

4. The system according to claim 1, characterized in that, The dynamic semantic update module is used to perform local incremental updates on the affected semantic nodes, the affected topological connection relationships and their adjacent topological connection relationships when at least one of the following trigger conditions is met: the displacement of the target object exceeds a preset displacement threshold, the channel occupancy rate exceeds a preset occupancy threshold, the personnel stay time exceeds a preset duration, the status of the key workstation is switched, or the path risk score exceeds a preset upper limit. The module also adds at least one of the following to the affected semantic nodes and topological edges: status label, risk label or timeliness label.

5. The system according to claim 1, characterized in that, The semantic-guided planning and replanning module is used to generate candidate path scores during path planning based on at least two of the following: target semantic priority, area access semantic, obstacle risk semantic, personnel interaction semantic, and task semantic. The candidate path score is obtained by normalizing at least two of the following: path length cost, congestion cost, risk cost, waiting cost, task matching cost, and operational reachability cost, and then weighting them according to weight parameters corresponding to the task type. This score is used to determine the target path and the set of alternative paths. When the current path fails, an alternative path that meets the task semantic constraints is selected from the set of alternative paths first.

6. The system according to claim 5, characterized in that, When performing replanning, the semantic-guided planning and replanning module is also used to insert auxiliary action nodes. The auxiliary action nodes are associated with the semantic nodes or topological connections that trigger their insertion, and include action type, target semantic node and execution conditions. The auxiliary action nodes include at least one of the following: adjusting position, bypassing temporary obstacles, waiting for personnel to pass, switching target approach direction, switching grasping posture, switching operation sequence, retracting the robotic arm to a safe posture, and verifying perception.

7. The system according to claim 1, characterized in that, The multi-robot semantic sharing module is used to perform time alignment, confidence fusion, and conflict resolution on semantic node information, topological edge state information, and local environmental change information uploaded by multiple robots, and to distribute the shared semantic topological map local update results to each robot. The local update results include a source identifier, timestamp, confidence value, spatial range identifier, map version number, and local revision number, enabling each robot to determine whether to adopt the corresponding local update result based on the timestamp, confidence value, map version number, and local revision number. When the local revision number is higher than the locally recorded local revision number and the confidence value meets a preset confidence threshold, the corresponding local update result is adopted first.

8. A semantic topology perception and replanning method for embodied composite robots, characterized in that, The system applied to any one of claims 1 to 7 comprises the following steps: S1. Acquire information about objects, obstacles, spatial access, and dynamic changes in the robot's operating environment; S2. Construct a semantic topology map based on the acquired information, including a set of semantic nodes, a set of topological connections, and a set of traffic constraints, and associate the traffic constraints with the corresponding semantic nodes or topological connections. S3. When an environmental change that meets the preset trigger conditions is detected, locate the affected semantic nodes, the affected topological connection relationships and their adjacent topological connection relationships, and perform local incremental updates on the corresponding node attributes, topological connection relationships, passage constraints, status labels, risk labels or timeliness labels. S4. Generate a candidate path set based on the semantic nodes corresponding to the target task and the updated semantic topology map. Score and rank the candidate paths according to semantic constraints and passage costs to obtain the planned path and the set of alternative paths. When the target area status, passage conditions or obstacle distribution of the current path changes, generate a replanned path based on the semantic node association, the set of alternative paths and the alternative passage relationship, and output at least one of the path node sequence, alternative path sequence and auxiliary action node sequence. S5. Determine whether the scenario is a multi-robot collaborative scenario; if yes, send the updated at least some semantic node information, topological connection relationship and environmental change information to the multi-robot semantic sharing module, and receive the shared update result with map version number and local revision number returned by the multi-robot semantic sharing module after time alignment, confidence fusion and conflict resolution; if no, retain the local update result of the local semantic topology map. S6. Output at least one of the following based on the planned path or replanned path: mobile chassis control quantity, robotic arm control quantity, or end effector control quantity.

9. The method according to claim 8, characterized in that, In step S4, when scoring and ranking candidate paths, path semantic constraints are determined based on at least two of the following: target semantic priority, area access semantic, obstacle risk semantic, personnel interaction semantic, and task semantic. Then, candidate path scores are generated according to the weight parameters corresponding to the task type after normalization based on at least two of the following: path length cost, congestion cost, risk cost, waiting cost, task matching cost, and operation reachability cost.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of claim 8 or 9.

Images