A robot navigation interactive control method and system based on multi-map fusion

Abstract

The present invention discloses a robot navigation interactive control method and system based on multi-map fusion, which belongs to the field of robot control technology. Existing robot navigation solutions lack an effective cross-scene navigation mechanism, which makes it impossible for robots to accurately realize cross-region navigation. The present invention discloses a robot navigation interactive control method based on multi-map fusion, which constructs a front-end interaction module, a multi-map acquisition module, a multi-map fusion module, and a navigation control module, and fuses multiple point cloud files based on the topological relationship between scenes to obtain a complete cross-scene map; then, based on the cross-scene map and the cross-scene navigation task, path planning is performed to obtain navigation data, thereby forming an efficient, reliable, and easy-to-use cross-scene navigation mechanism. Therefore, the present invention can realize navigation in complex environments or with large scene differences, thereby improving the robot's work inspection efficiency.

Description

Technical Field

[0001] The present invention relates to a robot navigation interactive control method and system based on multi-map fusion, belonging to the technical field of robot control. Background Art

[0002] Chinese patent application (CN116817882A) discloses a robot navigation map management method, which builds a navigation map data center on a cloud server, uses the data center to provide front-end and back-end interaction data and perform data backup, and provides robot update data at the same time. The robot is used as a client, and the robot is used to model the environment and obtain corresponding map files and parameter files. The robot provides a corresponding interface for the front end to call and upload the map files and parameter files to the cloud data center. The front-end operation interface is built in a web form. Through the front end and the cloud data center interact, the robot modeling of the environment and the process of obtaining map files and parameter files can be remotely viewed. The robot position is determined according to the robot's real-time coordinates, and the robot navigation is guided by a multi-layer processing method based on the map files and parameter files. The robot navigation map is managed through the cloud and remote web end, which is conducive to improving the robot navigation efficiency.

[0003] This solution uses a multi-layer processing method to guide robot navigation. When encountering areas that require different logical processing during navigation, the front end uses the original map as the base to construct transparent layers of the same size. Each area corresponds to a transparent layer, and all transparent layers are superimposed and displayed on the front end. The front end also adds, deletes, or modifies areas based on the map information of the base.

[0004] Although the above solution uses a multi-layer processing method to achieve robot navigation, it is only applicable to navigation in similar scenes (areas) and cannot achieve navigation in complex environments or with large differences in scenes. Therefore, the above solution lacks an effective cross-scene navigation mechanism, resulting in the robot being unable to accurately complete cross-region navigation. Manual intervention may be required, greatly reducing work efficiency.

[0005] The information disclosed in this Background Art is only for understanding the background of the present inventive concept and therefore it may include information that does not constitute prior art. Summary of the Invention

[0006] In response to the above problem or one of the above problems, an object of the present invention is to provide a robot navigation interactive control method and system based on multi-map fusion. By constructing a front-end interaction module, a multi-map acquisition module, a multi-map fusion module, and a navigation control module, multiple point cloud files are fused based on the topological relationship between scenes to obtain a complete cross-scene map; then, based on the cross-scene map and the cross-scene navigation task, path planning is performed to obtain navigation data, thereby forming an effective cross-scene navigation mechanism, which can realize navigation in complex environments or with large scene differences.

[0007] In response to the above problem or one of the above problems, the second purpose of the present invention is to provide a robot navigation interactive control method and system based on multi-map fusion, which associates navigation-related point cloud maps and cross-scene navigation tasks through the topological relationship between scenes, so as to accurately complete the robot's cross-regional navigation. The entire process does not require human intervention, thereby effectively simplifying the cross-map navigation process, improving operational efficiency, and enhancing the robot's work inspection efficiency.

[0008] To achieve one of the above purposes, the first technical solution of the present invention is:

[0009] A robot navigation interactive control method based on multi-map fusion includes the following contents:

[0010] Receive users' cross-scenario navigation tasks through pre-built front-end interaction modules;

[0011] Using the pre-built multi-map acquisition module, multiple point cloud files for recording cross-scene information are obtained according to the cross-scene navigation task;

[0012] Using the pre-built multi-map fusion module, multiple point cloud files are fused based on the topological relationship between scenes to obtain a complete cross-scene map;

[0013] A pre-built navigation control module is used to perform path planning based on cross-scene maps and cross-scene navigation tasks, obtain navigation data, and feed the navigation data back to the front-end interaction module to achieve closed-loop interaction of robot navigation.

[0014] The present invention constructs a front-end interaction module, a multi-map acquisition module, a multi-map fusion module, and a navigation control module, and fuses multiple point cloud files based on the topological relationship between scenes to obtain a complete cross-scene map; then, based on the cross-scene map and cross-scene navigation tasks, path planning is performed to obtain navigation data, thereby forming an efficient, reliable, and easy-to-use cross-scene navigation mechanism. Therefore, the present invention can realize navigation in complex environments or with large scene differences.

[0015] Furthermore, the present invention associates navigation-related point cloud maps and cross-scene navigation tasks through the topological relationship between scenes, so as to accurately complete the cross-regional navigation of the robot. The entire process does not require human intervention, thereby effectively simplifying the cross-map navigation process, improving operational efficiency, and enhancing the robot's work inspection efficiency. The solution is scientific, reasonable, and feasible.

[0016] As preferred technical measures:

[0017] The method for receiving users' cross-scenario navigation tasks through the pre-built front-end interaction module is as follows:

[0018] Utilizing the front-end display module, receiving user input information, wherein the input information at least includes a destination;

[0019] The input information is passed to the communication instruction generation module; and the communication instruction generation module is used to process the input information and generate cross-scenario control instructions;

[0020] The cross-scene control instructions are then transmitted to the multi-map acquisition module and the navigation control module as cross-scene navigation tasks, and the processing results are fed back to the front-end interaction module to achieve complete closed-loop interaction.

[0021] As preferred technical measures:

[0022] Using the pre-built multi-map acquisition module, the method for obtaining multiple point cloud files for recording cross-scene information based on the cross-scene navigation task is as follows:

[0023] According to the cross-scene navigation task, multiple target areas to be navigated are obtained;

[0024] Collect lidar data on multiple target areas;

[0025] The LiDAR data is processed by a simultaneous positioning and mapping algorithm to obtain 3D point cloud information for each target area. The 3D point cloud information includes at least the coordinate values ​​and color information of the points.

[0026] Based on the 3D point cloud information of the target area, multiple point cloud files are generated.

[0027] As preferred technical measures:

[0028] The method for fusing multiple point cloud files using the pre-built multi-map fusion module based on the topological relationships between scenes to obtain a complete cross-scene map is as follows:

[0029] Perform dimensionality reduction and mapping processing on multiple point cloud files to obtain multiple two-dimensional raster maps;

[0030] Based on the topological relationship between scenes, key points and paths are extracted from each raster map to obtain several topological points and edge information; topological points include location coordinates and attribute information, and edge information is the path information connecting two topological points.

[0031] Based on several topological points, edge information and coordinate transformation relationships between each point cloud file, a topological map file is constructed to fuse multiple point cloud files;

[0032] Use the topology map file as the topology structure of the navigation path to generate a complete cross-scene map.

[0033] As preferred technical measures:

[0034] The method of performing dimensionality reduction and mapping on the point cloud file to obtain a two-dimensional raster map is as follows:

[0035] Acquire multiple point cloud files, each point cloud file is a three-dimensional point cloud set, which includes a number of three-dimensional points;

[0036] Each 3D point is segmented using the vertical coordinate threshold segmentation method and projected onto a 2D plane to obtain a projected 2D point set.

[0037] At the same time, set the coordinates of the lower left corner of the grid map and the size of each grid, that is, the map resolution;

[0038] Based on the coordinates of the lower left corner and the map resolution, calculate the grid coordinates of each 2D coordinate point in the 2D point set;

[0039] According to several grid coordinates, the state of the grid is marked to obtain the state matrix;

[0040] Based on the neighborhood offset, the state matrix is ​​processed to obtain a two-dimensional grid map.

[0041] As preferred technical measures:

[0042] Based on the neighborhood offset, the state matrix is ​​processed to obtain a two-dimensional grid map as follows:

[0043] Obtaining state information of a state matrix, which includes an occupied state, an idle state, and an unknown state;

[0044] Determine the neighborhood offset based on the grid coordinates of the current grid;

[0045] Based on the neighborhood offset, the neighborhood grid is obtained, and the status of each neighborhood grid is judged to obtain occupied grids, idle grids and unknown grids;

[0046] For idle grids and unknown grids, values ​​are assigned based on the grid information of the neighborhood to obtain the expanded grid;

[0047] Smoothing the expanded grid to obtain a smoothed grid;

[0048] Based on the state information, color information is assigned to the smoothed grid to obtain grid pixels;

[0049] Color information includes primary color, secondary color, and tertiary color;

[0050] The first color indicates that there are obstacles in the smooth grid, the second color indicates that the smooth grid is a passable area, and the third color indicates that the smooth grid is an unknown area;

[0051] Several grid pixels are coupled to obtain a two-dimensional grid map.

[0052] As preferred technical measures:

[0053] Using the pre-built navigation control module, we perform path planning based on the cross-scene map and cross-scene navigation tasks. The method for obtaining navigation data is as follows:

[0054] Using the measurements from the LiDAR and IMU, the robot’s position in the cross-scene map is obtained.

[0055] Generate the optimal navigation path based on positioning information, cross-scene maps, and cross-scene navigation tasks;

[0056] Convert the optimal navigation path into robot motion instructions;

[0057] Determine navigation data based on the optimal navigation path and robot motion instructions.

[0058] As preferred technical measures:

[0059] The method for obtaining the robot's positioning information in the cross-scene map using the measurement information obtained from the lidar and inertial measurement unit is as follows:

[0060] Using a laser radar and an inertial measurement unit, measurement information is obtained, which includes at least robot position, velocity, and inertial measurement unit deviation information;

[0061] Based on the correlation between the robot's posture, velocity, and inertial measurement unit deviation information, variable factors are constructed, including the inertial measurement unit pre-integration factor, the radar scan data and map matching factor, and the foot odometry factor;

[0062] Through the graph optimization method, the errors and residual sum of squares of all variable factors are optimized to construct the calculation formula of the positioning process;

[0063] Solve the calculation formula of the positioning process to obtain the current optimal posture and positioning information of the robot in the cross-scene map.

[0064] As preferred technical measures:

[0065] The method for generating the optimal navigation path based on positioning information, cross-scene maps, and cross-scene navigation tasks is as follows:

[0066] Get several topological points and topological edges in the cross-scene map;

[0067] Set priority information for each topological point;

[0068] Set path length and time constraint information for each topological edge;

[0069] Construct a cost function between topological points based on priority information, path length and time constraint information;

[0070] Based on positioning information and cross-scenario navigation tasks, multiple topological point combinations are selected in the cross-scenario map. For each topological point combination, the task sequence and task parameters are set. Task parameters include dwell time, execution action, and collection data type.

[0071] Using the cost function, the cost of each topological point combination is solved to obtain the cost value;

[0072] Sort and search several cost values ​​to get the lowest cost value;

[0073] According to the minimum cost value, determine the corresponding topological point combination;

[0074] Based on the combination of topological points, the optimal navigation path is determined.

[0075] To achieve one of the above purposes, the second technical solution of the present invention is:

[0076] A robot navigation interactive control system based on multi-map fusion, comprising:

[0077] one or more processors;

[0078] a storage device for storing one or more programs;

[0079] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned robot navigation interactive control method based on multi-map fusion.

[0080] Compared with the existing technical solutions, the present invention has the following beneficial effects:

[0081] The present invention constructs a front-end interaction module, a multi-map acquisition module, a multi-map fusion module, and a navigation control module, and fuses multiple point cloud files based on the topological relationship between scenes to obtain a complete cross-scene map; then, based on the cross-scene map and cross-scene navigation tasks, path planning is performed to obtain navigation data, thereby forming an effective cross-scene navigation mechanism. Therefore, the present invention can realize navigation in complex environments or with large scene differences.

[0082] Furthermore, the present invention associates navigation-related point cloud maps and cross-scene navigation tasks through the topological relationship between scenes, so as to accurately complete the cross-regional navigation of the robot. The entire process does not require human intervention, thereby effectively simplifying the cross-map navigation process, improving operational efficiency, and enhancing the robot's work inspection efficiency. The solution is scientific, reasonable, and feasible. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 A schematic flow chart of the robot navigation interactive control method of the present invention;

[0084] Figure 2 A schematic diagram of a framework of the robot navigation interactive control method of the present invention;

[0085] Figure 3 A schematic diagram of a framework of the file management system of the present invention. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0087] Rather, the present invention encompasses any alternatives, modifications, equivalents, and solutions that fall within the spirit and scope of the present invention as defined by the claims. Furthermore, to facilitate a better understanding of the present invention, certain specific details are described in detail below in the detailed description of the present invention. Those skilled in the art will be able to fully understand the present invention without these details.

[0088] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention pertains. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention.

[0089] like Figure 1 As shown, the first specific embodiment of the robot navigation interactive control method based on multi-map fusion of the present invention is:

[0090] A robot navigation interactive control method based on multi-map fusion includes the following contents:

[0091] Receive users' cross-scenario navigation tasks through pre-built front-end interaction modules;

[0092] Using the pre-built multi-map acquisition module, multiple point cloud files for recording cross-scene information are obtained according to the cross-scene navigation task;

[0093] Using the pre-built multi-map fusion module, multiple point cloud files are fused based on the topological relationship between scenes to obtain a complete cross-scene map;

[0094] A pre-built navigation control module is used to perform path planning based on cross-scene maps and cross-scene navigation tasks, obtain navigation data, and feed the navigation data back to the front-end interaction module to achieve closed-loop interaction of robot navigation.

[0095] like Figure 2 As shown, the second specific embodiment of the robot navigation interactive control method based on multi-map fusion of the present invention is as follows:

[0096] A robot navigation interactive control method based on multi-map fusion includes the following contents:

[0097] The front-end interaction module receives user input and passes it to the communication system; at the same time, it receives information feedback from the system for interface display or user prompts.

[0098] It receives user input through the communication system and communicates with other modules. It can also distribute data and feed back processing results to the front end to achieve complete closed-loop interaction.

[0099] The navigation control module receives instructions forwarded by the communication system and sends real-time information feedback back to the communication system to support the execution of navigation-related tasks.

[0100] Use the file management system to manage related file contents and provide file content information to the navigation control module.

[0101] A specific embodiment of a robot navigation interactive control system using the method of the present invention:

[0102] A robot navigation and interactive control system based on multi-map fusion adopts a modular design and divides the system into four modules: front-end interaction system, navigation control system, file management system and communication system.

[0103] The front-end interactive system is built on the front-end interactive module. It is primarily responsible for user interface display and operation processing. It adopts a responsive design and supports full touchscreen operation while also being compatible with PC keyboard and mouse operation. The system includes a map display module, a topology map editing module, a task editing module, and a device control module.

[0104] The map display module supports the display of point cloud maps, raster maps and topological maps, as well as operations such as rotation, translation, zooming in and out;

[0105] The topology map editing module supports operations such as adding, deleting, and modifying topological points and edges; the task editing module supports the setting, sorting, and task planning of inspection points; and the equipment control module provides robot status monitoring and control functions.

[0106] The navigation control system is built around the navigation control module, which is primarily responsible for robot positioning, path planning, and motion control. It includes a positioning module, a path planning module, and a motion control module. The positioning module uses LiDAR and an inertial measurement unit (IMU) to precisely locate the robot within a point cloud map. The path planning module generates the optimal navigation path based on a topological map and grid map. The motion control module converts the planned path into robot motion commands. The positioning process can be represented as follows:

[0107]

[0108] Where, It is a set of state variables of the system, including robot posture, velocity, inertial measurement unit (IMU) deviation, etc. 、 、 They are the inertial measurement unit IMU pre-integration factor, the radar scan data and map matching factor, and the foot odometer factor. is the optimal posture.

[0109] Then, by optimizing the errors of all factors and the residual sum of squares through graph optimization methods, the current optimal pose can be obtained.

[0110] The path planning module gradually updates the shortest path between any two topological points in the topological graph through the idea of ​​dynamic programming. The core idea of ​​the algorithm is to consider an intermediate topological point. To update any two topological points and The shortest path between them, if it passes through the topological point The path is faster than directly from arrive If the path is shorter, update the path. The process can be expressed as follows:

[0111]

[0112] Where, For the set of intermediate topological points allowed to pass through Under the premise of arrive The shortest path length, is the set of intermediate topological points , from the topological point arrive The shortest path length; is the set of intermediate topological points , from the topological point arrive The shortest path length; is the set of intermediate topological points , from the topological point arrive The shortest path length.

[0113] Recursively, Allows the use of prefix in paths As the intermediate topological point, by comparing the two paths, update the shortest path. , then keep the original ; If it passes through the topological point , the path is updated to .

[0114] The file management system is primarily responsible for storing, accessing, and associating various map files. It includes modules for managing point cloud files, raster map files, topology files, visualization information files, and task description files. The system uses unified file naming and storage rules, and the relationships between different file types can be captured through topology file information.

[0115] The communication system primarily handles data exchange between the front-end and the navigation system, utilizing the full-duplex WebSocket protocol to enable real-time data transmission and status updates. This system includes a data transmission module, a data reception module, and a status synchronization module, ensuring that front-end operations are promptly reflected in the navigation system and that the navigation system's status is also reflected in real-time on the front-end interface.

[0116] like Figure 3As shown, the file management system in this embodiment includes point cloud PCD map files, raster map files, topology files, visualization information files and control information files. Among them, the topology file (topology file) is the core of the entire file management system, supporting basic operations such as adding, deleting, checking and modifying, and has editing and maintenance capabilities. It is associated with the point cloud PCD map, raster map, front-end visualization information, task description information, control information, etc., and data query for display, positioning, control and task execution can be achieved through the topology file. The topology file stores the topology required for navigation. The topology includes point or edge information. The attributes stored in each point include task description, specific location, map to which it belongs, etc. Each edge stores gait control, connection relationship, etc. Therefore, the point cloud PCD map and raster map required for navigation can be associated through the information stored in the topology file, and front-end visualization information and navigation control information can be provided to form a complete description information required for the navigation task.

[0117] The topology file records the topological structure of the navigation path, including the information of topological points and edges. Expressed as:

[0118]

[0119] in, It is a topological point set, representing several topological points; It is an edge set, which represents the path information connecting topological points.

[0120] Each topological point Contains location coordinates ,property ,in 、 、 Represents the position coordinates of the point in three-dimensional space, Indicates the type of the topological point, such as ordinary point, charging point, door point, elevator point, etc. Therefore, a topological point can be represented as a tuple containing location and custom attributes. Its expression is as follows:

[0121]

[0122] Similarly, each topological edge It can be expressed as:

[0123]

[0124] in and are two topological points connected by a topological edge, It is the topological edge attribute, such as normal gait, visual gait, walking speed, etc.

[0125] Topological points record location coordinates and attributes (such as ordinary points, charging points, door points, elevator points, etc.), and edges record the path information connecting two points.

[0126] Topology files are stored in JSON format. There are multiple ways to create a topology: one is manual creation, where the user directly adds points and edges on the interface; the other is recording generation, where the user controls the robot's movement, and the system records the trajectory and generates a topology.

[0127] In addition to basic location information, topological point attributes also include the point type (such as regular point, charging point, door point, elevator point, etc.), behavior (such as dwell time, execution action, etc.), and constraints (such as speed limit, accuracy requirements, etc.). These attributes guide the robot's behavior during navigation and improve navigation effectiveness. Topological edges also have a rich set of attributes, including edge type (such as straight line, curve, recorded trajectory, etc.), direction (unidirectional or bidirectional), weight (influencing path planning priority), and constraints (such as speed limit, width requirement, etc.). In particular, for recorded trajectories, the system records the robot's actual motion path, including position, speed, posture, and other information, to ensure that the robot can accurately reproduce the path.

[0128] The point cloud (PCD) file is the basic data file of the system, recording the three-dimensional point cloud information of the environment for robot positioning and navigation. Each PCD file corresponds to the point cloud data of an area. The file format adopts the standard PCD format and contains information such as the coordinates and color of the points. In a multi-map scene, each PCD file has a corresponding visualization information file, which mainly stores the offset and rotation relationship of the PCD point cloud based on the world coordinate system. Therefore, multiple PCD files can be spliced ​​together to form a complete large-scene PCD using the following formula. The expression is as follows:

[0129]

[0130] Where, For a complete cross-scene map, For the A point cloud collection of PCD files, is the number of PCDs, For the The transformation relationship of the point cloud set of a PCD file in the world coordinate system.

[0131] There are two ways to generate point cloud files: one is through autonomous mapping by the robot, and the other is by importing external point cloud data. In autonomous mapping mode, the system controls the robot to move within the target area while collecting LiDAR data and generating a point cloud map using the simultaneous localization and mapping (SLAM) algorithm.

[0132] The raster map file is a two-dimensional raster map generated based on point cloud data, which is used for path planning and obstacle avoidance. The raster map generation process includes three steps: point cloud projection, rasterization and post-processing. Assume that a three-dimensional point cloud set is , each point After segmentation using the Z coordinate threshold, the image is projected onto a two-dimensional plane. The projected two-dimensional point set is:

[0133]

[0134] in, , 、 are coordinates in three-dimensional space.

[0135] The two-dimensional coordinate point The process of mapping to a raster map can be expressed as:

[0136] ,

[0137] in, are grid coordinates, and is the coordinate of the lower left corner of the raster map, and Represents the size of each grid map, that is, the map resolution. At this time, the grid map can be The state matrix It is expressed as follows:

[0138]

[0139] Where, For the occupied state, In idle state, In order to expand the range of obstacles, ensure safe distance, reduce noise impact, and fill unknown areas, the grid map is processed as follows:

[0140]

[0141]

[0142]

[0143] Where, 、 、 They are the grid maps after expansion, smoothing and filling, For the current grid The surrounding neighborhood offset.

[0144] Raster maps are stored in PNG format, with each pixel representing a grid in the environment. Black represents obstacles, white represents traversable areas, and gray represents unknown areas. The system supports multi-resolution raster maps, allowing maps of varying accuracy to be generated based on demand. High-resolution maps (e.g., 5 cm / grid) are used for precise navigation and obstacle avoidance, while low-resolution maps (e.g., 20 cm / grid) are used for global path planning, balancing accuracy and computational efficiency. The system also supports manual editing of raster maps, allowing users to modify the map directly within the interface, such as adding virtual walls and marking prohibited areas.

[0145] The task description file records the task information, including the task point sequence, execution mode (single or cyclic), time interval, etc. The task description file is stored in JSON format, and each line records the information of a task point (topological point), including the topological map to which it belongs, point identification ID, execution order, etc. The creation process of the task description file is the core of task planning. The user first selects the task point from the topological map, and then sets the task sequence and parameters, such as dwell time, execution action, collection data type, etc. The system supports manual sorting and automatic optimization sorting. Automatic optimization will consider path length, task priority, and time constraints to generate the optimal task path. The task priority is divided into different levels from 1 to 10, and the larger the value, the higher the priority. The path length , Task Priority , time constraints The normalization can be expressed as:

[0146]

[0147]

[0148]

[0149] in, is the Euclidean distance between two task points, The actual distance between the two mission points along the planned route, is the task priority, The Euclidean distance between two task points at the maximum speed of the robot The time spent, It is the time taken by the robot to pass through two task points based on the planned route and actual speed.

[0150] Assume the topology is , each point Attributes All contain priority Information, each topological edge Attributes Both include path length and time constraint information , the task is topological points, then from point Arrive Cost function It can be expressed as:

[0151]

[0152] in 、 、 are weights, taking values ​​of 0.35, 0.25, and 0.4 respectively.

[0153] Assume that the task point set is , then the sorted recursive search process It can be expressed as:

[0154]

[0155] in is the set of visited path points, is the new cumulative cost, is the starting position, is the end position, is the cost function, The next neighboring point with the lowest cost, For search results.

[0156] The task description file also supports conditional branching and exception handling, such as selecting a different task path based on sensor data or external signals, or automatically adjusting the task plan when encountering obstacles, low battery, or other abnormal conditions. These advanced features make tasks more flexible and intelligent, able to adapt to complex and changing environments and needs.

[0157] In this embodiment, the processing mechanism for special points such as charging points, door points, and elevator points is as follows:

[0158] Charging points support both laser positioning and QR code positioning. With laser positioning, the user moves the robot to the charging location and then records the point cloud features of that location. The system associates these features with the charging point and uses them for precise location positioning during subsequent navigation. With QR code positioning, the user simply installs a standard QR code near the charging location, and the system recognizes the code to locate the charging location.

[0159] For door points, a door status monitoring and control interface has been designed. When the robot needs to pass through a door, the system first checks the door's status. If the door is open, the robot passes through directly; if the door is closed, the system sends a door-opening request and waits for the door to open before passing. The system supports multiple door-opening methods, such as sending a door-opening command over the network, manual door opening via voice prompts, and using a robotic arm to press a door-opening button, to meet the needs of different scenarios.

[0160] For the elevator points, a complete elevator control process has been designed, including operations such as calling the elevator, selecting a floor, and opening and closing the doors. Each elevator floor consists of an elevator standby point and an elevator point. The elevator standby point is located outside the elevator door and is used to wait for the elevator to arrive; the elevator point is located inside the elevator and indicates the robot's position in the elevator. When using the elevator, the robot first navigates to the elevator standby point and sends an elevator call request. After the elevator arrives and the doors open, the robot navigates to the elevator point and sends a request for the target floor. After the elevator arrives and the doors open, the robot exits the elevator. The entire process communicates via the full-duplex WebSocket protocol, obtaining real-time elevator status and sending requests.

[0161] Special points utilize a unified data structure and processing flow. Each point includes information such as type, location, attributes, and operation methods. The system provides standardized interfaces and protocols to support integration with external systems, such as access control and elevator control systems. Special point processing also includes exception handling mechanisms, such as automatic retry for charging failures, door switch status monitoring, and elevator response timeout handling, enhancing system robustness and reliability. Furthermore, the system records the usage of special points and any abnormal events, facilitating subsequent analysis and optimization.

[0162] The task planning method in this embodiment includes the following contents:

[0163] The task planning method of the present invention provides a flexible task planning function, supports the addition, deletion, modification and sorting of task points, and provides an automatic optimization sorting function. At the same time, it supports two modes of single execution and cyclic execution, as well as pause, resume and cancel operations during the task process. The system supports setting the attributes and parameters of the task points, such as dwell time, collected data type, execution action, etc., to meet the requirements of different tasks. During the execution of the task, the system provides real-time monitoring and remote control functions to display the robot's position, status and task progress. At the same time, it supports exception handling and automatic recovery, such as automatic detour when encountering obstacles, automatic return to charging when the battery is low, and automatic recovery after task interruption. After the task is completed, the system will generate a detailed task report, including task path, time records, abnormal events, collected data, etc., to facilitate subsequent analysis and evaluation.

[0164] The front-end interaction mechanism in this embodiment includes the following:

[0165] The front-end interaction mechanism of the present invention is optimized for touch operation and is compatible with PC keyboard and mouse operation. The interface adopts a layered structure, modularizes complex functions, and provides an intuitive operation method. For example, in terms of map display, it supports single-point touch, finger dragging, two-finger zooming in and out and other operations; in terms of topology map editing, it provides functions such as clicking the screen to generate topology points and dragging to modify positions. The interface layout adopts an adaptive design, which automatically adjusts the size and position of elements according to the screen size and resolution. The main interface is divided into a top menu bar, a left function area, and a central display area with a clear layout and distinct levels. The operation process design follows user habits, reduces operation steps, and improves efficiency. The interface also provides rich visual feedback and prompt information, such as operation prompts, status indicators, error warnings, etc., to help users understand the system status and operation results. At the same time, it supports customized interface themes and layouts to meet the personalized needs of different users.

[0166] In terms of map display, the system of the present invention supports multiple interactive operations, including the following:

[0167] 1. Single-touch functionality for selecting elements or locations;

[0168] 2. Finger drag function, used to pan the view;

[0169] 3. Two-finger zoom in and out function to adjust the view ratio;

[0170] 4. Finger drag plus specific key function for combined operations, such as holding down a specific key while dragging to achieve functions such as rotation;

[0171] 5. Select the drag function to move the selected element.

[0172] In terms of topology map editing, the system of the present invention provides a variety of operating functions, including the following:

[0173] 1. Point operations, including adding points (based on the robot's current position or manually entering coordinates), selecting points (modifying coordinates, deleting, setting properties, etc.), and dragging points (directly adjusting positions).

[0174] 2. Edge operations, including adding edges (connecting two points), selecting edges (modifying properties, deleting, etc.), and recording edges (recording the robot's actual motion trajectory as the edge path);

[0175] 3. Special point settings, including charging points (laser positioning or QR code positioning), door points, elevator points, etc.

[0176] In terms of task editing, the system of the present invention provides an intuitive operation method, including the following:

[0177] 1. Task point selection function, used to select points from the topology map as task points;

[0178] 2. Task point sorting function, used to manually drag and adjust the order or automatically optimize the sorting;

[0179] 3. Task setting function, used to set the execution mode (single or loop), time interval, etc.

[0180] 4. Task control function, used to start, pause, resume, cancel and other operations.

[0181] Therefore, the navigation interactive control system provided by the present invention simplifies cross-map navigation and task processes and improves operational efficiency by associating navigation-related PCD maps, grid maps, task description information, control information, etc. through topology file information.

[0182] At the same time, it improves user operating efficiency: it is optimized for touch operation, the interface is intuitive and easy to use, the operation is simple and efficient, which lowers the learning threshold and improves operating efficiency.

[0183] Furthermore, the present invention adopts a unified file system and special point processing mechanism, which enhances the adaptability of the system in different environments and provides good scalability, making it easy to add new functions and support new scenarios.

[0184] Furthermore, the present invention enhances the processing capability of special scenarios by designing a unified mechanism for processing special points, including charging points, door points, and elevator points, thereby improving the system's processing capability and reliability in special scenarios. The present invention provides a flexible task planning function that supports the addition, sorting, and task setting of task points, optimizes task paths and execution strategies, and improves task efficiency and quality.

[0185] Furthermore, the present invention employs unified file naming and storage rules, simplifying file management, reducing association errors, and lowering system maintenance costs. The present invention's interface adopts a responsive design, supports full touchscreen operation, and is compatible with PC keyboard and mouse operation, providing a good user experience. Furthermore, the system design of the present invention takes into account future technological development and application expansion needs, providing a good foundation for the integration of new technologies and new application scenarios, and promoting technological innovation and application expansion.

[0186] An embodiment of a device applying the method of the present invention:

[0187] An electronic device comprising:

[0188] one or more processors;

[0189] a storage device for storing one or more programs;

[0190] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned robot navigation interactive control method based on multi-map fusion.

[0191] A computer medium embodiment of the method of the present invention:

[0192] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned robot navigation interactive control method based on multi-map fusion.

[0193] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, optical storage, etc.) containing computer-usable program code.

[0194] The present application is described in terms of flowcharts or / and block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process or / and block in the flowchart or / and block diagram, as well as the combination of processes or / and blocks in the flowchart or / and block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0195] The module in this application is an object that objectively describes the morphological structure with the help of physical or virtual representation. The object is not equal to the physical body and is not limited to physical and virtual. It can be a data processing function, software program, processing mode, usage method, operation method, workflow, application process, electronic hardware, circuit module, processing system, system imitation or simulation object.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field can still modify or replace the specific implementation methods of the present invention. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A robot navigation interactive control method based on multi-map fusion, characterized by: Includes the following: Receive users' cross-scenario navigation tasks through pre-built front-end interaction modules; Using the pre-built multi-map acquisition module, multiple point cloud files for recording cross-scene information are obtained according to the cross-scene navigation task; Using the pre-built multi-map fusion module, multiple point cloud files are fused based on the topological relationship between scenes to obtain a complete cross-scene map; A pre-built navigation control module is used to perform path planning based on cross-scene maps and cross-scene navigation tasks, obtain navigation data, and feed the navigation data back to the front-end interaction module to achieve closed-loop interaction of robot navigation.

2. The robot navigation interactive control method based on multi-map fusion according to claim 1, characterized in that: The method for receiving users' cross-scenario navigation tasks through the pre-built front-end interaction module is as follows: Utilizing the front-end display module, receiving user input information, wherein the input information at least includes a destination; The input information is passed to the communication instruction generation module; and the communication instruction generation module is used to process the input information and generate cross-scenario control instructions; The cross-scene control instructions are then transmitted to the multi-map acquisition module and the navigation control module as cross-scene navigation tasks, and the processing results are fed back to the front-end interaction module to achieve complete closed-loop interaction.

3. The robot navigation interactive control method based on multi-map fusion according to claim 1, characterized in that: Using the pre-built multi-map acquisition module, the method for obtaining multiple point cloud files for recording cross-scene information based on the cross-scene navigation task is as follows: According to the cross-scene navigation task, multiple target areas to be navigated are obtained; Collect lidar data on multiple target areas; The LiDAR data is processed by a simultaneous positioning and mapping algorithm to obtain 3D point cloud information for each target area. The 3D point cloud information includes at least the coordinate values ​​and color information of the points. Based on the 3D point cloud information of the target area, multiple point cloud files are generated.

4. The robot navigation interactive control method based on multi-map fusion according to claim 1, characterized in that: The method for fusing multiple point cloud files using the pre-built multi-map fusion module based on the topological relationships between scenes to obtain a complete cross-scene map is as follows: Perform dimensionality reduction and mapping processing on multiple point cloud files to obtain multiple two-dimensional raster maps; Based on the topological relationship between scenes, key points and paths are extracted from each raster map to obtain several topological points and edge information; topological points include location coordinates and attribute information, and edge information is the path information connecting two topological points. Based on several topological points, edge information and coordinate transformation relationships between each point cloud file, a topological map file is constructed to fuse multiple point cloud files; Use the topology map file as the topology structure of the navigation path to generate a complete cross-scene map.

5. The robot navigation interactive control method based on multi-map fusion according to claim 4, characterized in that: The method of performing dimensionality reduction and mapping on the point cloud file to obtain a two-dimensional raster map is as follows: Acquire multiple point cloud files, each point cloud file is a three-dimensional point cloud set, which includes a number of three-dimensional points; Each 3D point is segmented using the vertical coordinate threshold segmentation method and projected onto a 2D plane to obtain a projected 2D point set. At the same time, set the coordinates of the lower left corner of the grid map and the size of each grid, that is, the map resolution; Based on the coordinates of the lower left corner and the map resolution, calculate the grid coordinates of each 2D coordinate point in the 2D point set; According to several grid coordinates, the state of the grid is marked to obtain the state matrix; Based on the neighborhood offset, the state matrix is ​​processed to obtain a two-dimensional grid map.

6. The robot navigation interactive control method based on multi-map fusion according to claim 5, characterized in that: Based on the neighborhood offset, the state matrix is ​​processed to obtain a two-dimensional grid map as follows: Obtaining state information of a state matrix, which includes an occupied state, an idle state, and an unknown state; Determine the neighborhood offset based on the grid coordinates of the current grid; Based on the neighborhood offset, the neighborhood grid is obtained, and the status of each neighborhood grid is judged to obtain occupied grids, idle grids and unknown grids; For idle grids and unknown grids, values ​​are assigned based on the grid information of the neighborhood to obtain the expanded grid; Smoothing the expanded grid to obtain a smoothed grid; Based on the state information, color information is assigned to the smoothed grid to obtain grid pixels; Color information includes primary color, secondary color, and tertiary color; The first color indicates that there are obstacles in the smooth grid, the second color indicates that the smooth grid is a passable area, and the third color indicates that the smooth grid is an unknown area; Several grid pixels are coupled to obtain a two-dimensional grid map.

7. The robot navigation interactive control method based on multi-map fusion according to claim 1, characterized in that: Using the pre-built navigation control module, we perform path planning based on the cross-scene map and cross-scene navigation tasks. The method for obtaining navigation data is as follows: Using the measurements from the LiDAR and IMU, the robot’s position in the cross-scene map is obtained. Generate the optimal navigation path based on positioning information, cross-scene maps, and cross-scene navigation tasks; Convert the optimal navigation path into robot motion instructions; Determine navigation data based on the optimal navigation path and robot motion instructions.

8. The robot navigation interactive control method based on multi-map fusion according to claim 7, characterized in that: The method for obtaining the robot's positioning information in the cross-scene map using the measurement information obtained from the lidar and inertial measurement unit is as follows: Using a laser radar and an inertial measurement unit, measurement information is obtained, which includes at least robot position, velocity, and inertial measurement unit deviation information; Based on the correlation between the robot's posture, velocity, and inertial measurement unit deviation information, variable factors are constructed, including the inertial measurement unit pre-integration factor, the radar scan data and map matching factor, and the foot odometry factor; Through the graph optimization method, the errors and residual sum of squares of all variable factors are optimized to construct the calculation formula of the positioning process; Solve the calculation formula of the positioning process to obtain the current optimal posture and positioning information of the robot in the cross-scene map.

9. The robot navigation interactive control method based on multi-map fusion according to claim 7, characterized in that: The method for generating the optimal navigation path based on positioning information, cross-scene maps, and cross-scene navigation tasks is as follows: Get several topological points and topological edges in the cross-scene map; Set priority information for each topological point; Set path length and time constraint information for each topological edge; Construct a cost function between topological points based on priority information, path length and time constraint information; Based on positioning information and cross-scenario navigation tasks, multiple topological point combinations are selected in the cross-scenario map. For each topological point combination, the task sequence and task parameters are set. Task parameters include dwell time, execution action, and collection data type. Using the cost function, the cost of each topological point combination is solved to obtain the cost value; Sort and search several cost values ​​to get the lowest cost value; According to the minimum cost value, determine the corresponding topological point combination; Based on the combination of topological points, the optimal navigation path is determined.

10. A robot navigation interactive control system based on multi-map fusion, characterized by: It includes: one or more processors; a storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a robot navigation interactive control method based on multi-map fusion as described in any one of claims 1-9.

Images