Robot interactive local map construction method and system based on depth camera

Abstract

The invention provides a robot interactive local map construction method and system based on a depth camera, and relates to the technical field of robot real-time map construction. Comprising the following steps: acquiring a depth image and a color image through a depth camera, and acquiring IMU data through an IMU; the color image and IMU data are input into a global speedometer module, and the pose of the robot in a global coordinate system is estimated in real time; an elevation grid map with the robot as the center is constructed according to the global pose and the depth image, the topographic feature and the passing cost of each grid are calculated, and a local trafficability map is constructed; and obtaining a to-be-modified area selected by an operator in the color image, obtaining a map grid corresponding to the to-be-modified area in the color image based on the image alignment module, and modifying the passing cost of the corresponding grid to realize interactive modification of the local passability map. According to the method, the data of the depth camera and the operation intention of a human operator are combined, and interactive construction and modification of the map are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time map construction of robots, and particularly to a method and system for constructing an interactive local map of a robot based on a depth camera. Background Art

[0002] With the development of technology, mobile robots have been widely used in fields such as service, agriculture, and healthcare due to their versatility and flexibility. At the same time, their potential value in fields such as urban security, national defense, and space exploration has become increasingly prominent. By integrating technologies such as intelligent navigation, environmental perception, and autonomous decision-making, mobile robots can operate flexibly in unstructured environments, greatly expanding the application scenarios of robots. Especially in human-robot collaboration, mobile robots have shown unique advantages and achieved safe and efficient collaboration with humans through advanced environmental perception and path planning capabilities.

[0003] When a mobile robot performs complex tasks (such as autonomous navigation) in an unknown environment, it needs to have the ability to sense its own state and environmental information through sensors, as well as the ability to accurately locate and construct an environmental map. However, traditional map construction methods have obvious limitations. These methods usually directly process the original point cloud data, and the constructed map can only express the basic geometric and topological information of the environment, and cannot obtain and describe the semantic information and other higher-dimensional terrain features in the environment. This results in the robot being able to establish a geometric understanding of the surrounding environment, but unable to truly understand the content and meaning of the environment. In other words, the robot knows "what the environment looks like", but does not know "what the environment is" and "what impact the environment has on me", which leads to the robot being unable to perform higher-dimensional analysis of the surrounding environment and severely limits the robot's ability to perform advanced tasks in complex environments. To solve this problem, it is necessary to integrate human perception of the environment into the map.

[0004] Real-time interactive map construction, as a systematic discipline technology involving many scientific research fields such as computer vision, sensor data processing, map construction, and human-computer interaction, is one of the popular research directions in the current field of robotics. The core task of map construction is to obtain the three-dimensional point cloud data of the environment in real time through a depth camera and dynamically update the environmental map in combination with the motion information of the robot. During the construction process, the map needs to be interactive, that is, it can adjust and optimize the map content in real time according to environmental changes or operator requirements. For example, when dynamic obstacles or structural changes occur in the environment, the map can be quickly updated to reflect these changes, thereby providing accurate navigation support for the robot.

[0005] As a core sensor, the depth camera can provide high-resolution 3D point cloud data to capture detailed information about the environment. However, the depth camera may have problems with data noise or reduced accuracy in strong light or long-distance scenarios. Therefore, to achieve high-precision map construction in complex environments, it is usually necessary to combine other sensor data (such as inertial measurement unit IMU or lidar) for multi-source data fusion to improve the robustness and accuracy of the map. In addition, an interactive map not only needs to update environmental information in real time but also needs to support operators to perform operations such as map editing, annotation, or path planning through an interactive interface to meet diverse application requirements. By integrating the 3D perception ability of the depth camera with real-time data processing technology, the interactive map construction method can achieve high-precision and high-robustness map construction and update in a dynamic environment, providing strong technical support for robot navigation, environmental perception, and human-machine interaction. The application of this technology will greatly promote the autonomy and intelligence development of intelligent robots in complex environments.

[0006] Chinese Patent No. 201710244943.2 discloses a method for restricting the motion space of a mobile robot based on an interaction method. This method is based on visual markers and specifies a custom virtual boundary by switching markers to restrict the motion space of the mobile robot. This system can only delimit a virtual boundary in an already constructed map and cannot correct the map while constructing it.

[0007] Chinese Patent No. 202310845919.X discloses a human-machine interaction method for service robots. This method understands the operator's instructions through a sequence model and can achieve fixed-point navigation control in home or office scenarios. However, this method lacks the ability to interact with the map and still uses traditional methods in map construction, unable to incorporate the operator's intentions into the map construction process.

[0008] Chinese Patent No. 202410851954.7 discloses a method for robot terrain perception and autonomous exploration in complex environments. This method uses lidar as a sensor, outputs a point cloud map through the A-LOAM algorithm, calculates an elevation grid map, combines various terrain feature data to obtain passable map information, and finally conducts autonomous exploration on this map. However, this method lacks the ability to interact with the map, still uses traditional methods in map construction, and uses lidar as a sensor, resulting in a high cost.

[0009] The Chinese patent with the application number 202411578428.4 discloses an obstacle map marking method and system combined with an unmanned vehicle. This system can understand the operator's instructions through a sequence model and can achieve fixed-point navigation control in home or office scenarios. However, this method lacks the ability to interact with the map and still uses traditional methods in map construction, and cannot incorporate the operator's intentions into the map construction process. Summary of the Invention

[0010] To solve the problem that it is difficult for existing real-time interactive map construction systems to effectively integrate manual experience, embodiments of the present invention provide a method and system for robot-interactive local map construction based on a depth camera. The technical solutions are as follows:

[0011] On the one hand, a method for robot-interactive local map construction based on a depth camera is provided. This method is implemented by a robot-interactive local map construction system based on a depth camera. The system includes an operator, an interactive operation platform, and a mobile robot. The method includes:

[0012] S1. The depth camera of the mobile robot acquires a depth image and a color image, and the IMU of the mobile robot acquires IMU data, and sends the depth image, color image, and IMU data to the interactive operation platform.

[0013] S2. The interactive operation platform estimates the pose of the robot in the global coordinate system in real time according to the color image, IMU data, and global odometry module.

[0014] S3. The interactive operation platform constructs an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and the depth image, calculates the terrain features and traversal costs of each grid, and then constructs a local traversability map centered on the robot, and feeds back the local traversability map centered on the robot to the operator through the local traversability map display module.

[0015] S4. The operator selects the area to be modified in the color image through the interactive image display window modification module of the interactive operation platform. The interactive operation platform obtains the corresponding map grid of the area to be modified in the local traversability map centered on the robot according to the image alignment module of the depth camera, and modifies the traversal cost of the grid to realize the operator's real-time modification of the map and generate a map available for robot navigation.

[0016] Optionally, constructing an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and the depth image, calculating the terrain features and traversal costs of each grid, and then constructing a local traversability map centered on the robot in S3 includes:

[0017] S31. Set the traversability weight according to the traversability of the robot on different terrain features.

[0018] S32. Obtain an elevation grid map centered on the robot based on the pose of the robot in the global coordinate system and the depth image.

[0019] S33. Filter the elevation grid map using a Savitzky-Golay filter to obtain a filtered elevation grid map.

[0020] S34. Calculate the parameters for describing the terrain features of each grid for the filtered elevation grid map.

[0021] S35. Normalize the terrain features to obtain the traversal cost of the robot for each type of terrain.

[0022] S36. Obtain a local traversability map centered on the robot based on the traversability weight and the traversal cost.

[0023] Optionally, the parameters for describing the terrain features of each grid in S33 include:

[0024] Calculate the slope as shown in the following formula (1):

[0025]

[0026] In the formula, slope represents the slope, Gx represents the height gradient in the x direction, and Gy represents the height gradient in the y direction. Among them, Gx is calculated according to the following formula (2):

[0027]

[0028] In the formula, h(i,j) represents the height of any point in the window, i represents the abscissa of the grid, j represents the ordinate of the grid, and resolution represents the resolution of the map.

[0029] Gy is calculated according to the following formula (3):

[0030]

[0031] Calculate the roughness as shown in the following formula (4):

[0032]

[0033] In the formula, roughness represents the roughness, h(i,j) represents the height of any point in the window, h_center represents the height of the center point, and n represents the number of grids in the window.

[0034] Calculate the steepness, as shown in Equation (5) below:

[0035]

[0036] In the formula, stepness represents the steepness, h max represents the maximum height value within the window, h min represents the minimum height value within the window, and resolution represents the resolution of the map grid.

[0037] Calculate the slope change rate, as shown in Equation (6) below:

[0038]

[0039] In the formula, slope_change_rate represents the slope change rate, dsx represents the change rate of the slope in the x direction, and dsy represents the change rate of the slope in the y direction. Among them, dsx is calculated according to Equation (7) below:

[0040]

[0041] dsy is calculated according to Equation (8) below;

[0042]

[0043] Optionally, the cost of the robot for each type of terrain in S34 is as shown in Equation (9) below:

[0044] cost = 1 - e (-decay_factor*normalized_feature_value) (9)

[0045] In the formula, cos t represents the cost of passage, decay_factor represents an adjustable parameter for controlling the rate of cost increase, and normalized_feature_value represents the result of normalizing the feature value to the range [0, 1].

[0046] Optionally, the image alignment module of the depth camera is used to establish a pixel-level mapping relationship between the color image and the depth image, and align the depth image and the color image according to the pixel-level mapping relationship.

[0047] Among them, the two-dimensional coordinates in the depth image are restored to the three-dimensional coordinates in the depth camera coordinate system through Equation (10) below:

[0048]

[0049] In the formula, P d represents the three-dimensional space coordinates corresponding to a certain pixel in the depth image, K d represents the internal parameter matrix of the depth lens in the depth camera, (ud , v d ), where (u, v) represents the pixel coordinates of each pixel in the depth image, and d represents the depth value corresponding to each pixel.

[0050] Project the 3D points in the depth image into the pixel coordinate system of the color image through the following formula (11):

[0051]

[0052] In the formula, (u c , v c ) represents the pixel coordinates of the pixel block in the color image, K c represents the internal parameter matrix of the color lens in the depth camera, R represents the rotation matrix between the color camera and the depth camera, and T trans represents the translation vector between the color camera and the depth camera.

[0053] Optionally, the method further includes:

[0054] Determine the physical threshold range of the terrain feature parameters according to the mechanical structure characteristics and kinematic constraints of the robot.

[0055] Construct a weighted evaluation matrix, and quantify the influence weight coefficients of each terrain feature on the motion energy consumption, safety risk and passing efficiency of the robot through the analytic hierarchy process.

[0056] On the other hand, a robot interactive local map construction system based on a depth camera is provided. This system is applied to the robot interactive local map construction method based on a depth camera. The system includes an operator, an interactive operation platform and a mobile robot, where:

[0057] The operator is used to select the area to be modified in the color image through the modification module of the interactive image display window of the interactive operation platform.

[0058] The interactive operation platform is used to estimate the pose of the robot in the global coordinate system in real time according to the color image, IMU data and the global odometer module; construct an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and the depth image, calculate the terrain features and passing costs of each grid, and then construct a local passability map centered on the robot, and feedback the local passability map centered on the robot to the operator through the local passability map display module; obtain the map grid corresponding to the area to be modified in the local passability map centered on the robot according to the image alignment module of the depth camera, and modify the passing cost of the grid to realize the operator's real-time modification of the map and generate a map available for robot navigation.

[0059] A mobile robot obtains a depth image and a color image through the depth camera of the mobile robot, obtains IMU data through the IMU of the mobile robot, and sends the depth image, the color image, and the IMU data to an interactive operation platform.

[0060] Optionally, the interactive operation platform is further configured to:

[0061] S31. Set a traversability weight according to the traversability of the robot for different terrain features.

[0062] S32. Obtain an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and the depth image.

[0063] S33. Filter the elevation grid map using a Savitzky-Golay filter to obtain a filtered elevation grid map.

[0064] S34. Calculate parameters for describing the terrain features of each grid for the filtered elevation grid map.

[0065] S35. Normalize the terrain features to obtain the traversal cost of the robot for each type of terrain.

[0066] S36. Obtain a local traversability map centered on the robot according to the traversability weight and the traversal cost.

[0067] Optionally, the interactive operation platform is further configured to:

[0068] Calculate the slope as shown in the following formula (1):

[0069]

[0070] In the formula, slope represents the slope, Gx represents the height gradient in the x direction, and Gy represents the height gradient in the y direction. Among them, Gx is calculated according to the following formula (2):

[0071]

[0072] In the formula, h(i,j) represents the height of any point within the window, i represents the abscissa of the grid, j represents the ordinate of the grid, and resolution represents the resolution of the map.

[0073] Gy is calculated according to the following formula (3):

[0074]

[0075] Calculate the roughness as shown in the following formula (4):

[0076]

[0077] In the formula, roughness represents roughness, h(i,j) represents the height of any point within the window, h_center represents the height of the center point, and n represents the number of grid cells within the window.

[0078] Calculate the steepness as shown in the following formula (5):

[0079]

[0080] In the formula, stepness represents steepness, h max represents the maximum height value within the window, h min represents the minimum height value within the window, and resolution represents the resolution of the map grid cells.

[0081] Calculate the slope change rate as shown in the following formula (6):

[0082]

[0083] In the formula, slope_change_rate represents the slope change rate, dsx represents the change rate of the slope in the x direction, and dsy represents the change rate of the slope in the y direction. Among them, dsx is calculated according to the following formula (7):

[0084]

[0085] dsy is calculated according to the following formula (8);

[0086]

[0087] Optionally, the cost for the robot to traverse each type of terrain is as shown in the following formula (9):

[0088] cost = 1 - w (-decay_factor*normalized_feature_value) (9)

[0089] In the formula, cost represents the traversal cost, decay_factor represents an adjustable parameter for controlling the rate of cost increase, and normalized_feature_value represents the result of normalizing the feature value to the range [0, 1].

[0090] Optionally, the image alignment module of the depth camera is used to establish a pixel-level mapping relationship between the color image and the depth image, and align the depth image and the color image according to the pixel-level mapping relationship.

[0091] Among them, the two-dimensional coordinates in the depth image are restored to the three-dimensional coordinates in the depth camera coordinate system through the following formula (10):

[0092]

[0093] In the formula, P d represents the three-dimensional space coordinates corresponding to a certain pixel in the depth image, and K d represents the internal parameter matrix of the depth lens in the depth camera. (u d , v d ) represents the pixel coordinates of each pixel in the depth image, and d represents the depth value corresponding to each pixel.

[0094] The three-dimensional points in the depth image are projected into the pixel coordinate system of the color image through the following formula (11):

[0095]

[0096] In the formula, (u c , v c ) represents the pixel coordinates of the pixel block in the color image, K c represents the internal parameter matrix of the color lens in the depth camera, R represents the rotation matrix between the color camera and the depth camera, and T trans represents the translation vector between the color camera and the depth camera.

[0097] Optionally, it further includes:

[0098] Determine the physical threshold range of the terrain feature parameters according to the mechanical structure characteristics and kinematic constraints of the robot.

[0099] Construct a weighted evaluation matrix, and quantify the influence weight coefficients of each terrain feature on the motion energy consumption, safety risk and passing efficiency of the robot through the analytic hierarchy process.

[0100] On the other hand, a robot-interactive local map construction device is provided. The robot-interactive local map construction device includes: a processor; a memory, and computer-readable instructions are stored on the memory. When the computer-readable instructions are executed by the processor, any one of the methods in the above-mentioned robot-interactive local map construction method based on a depth camera is implemented.

[0101] On the other hand, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the methods in the above-mentioned robot-interactive local map construction method based on a depth camera.

[0102] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0103] In the present invention, a system capable of realizing real-time interactive map construction is provided. The system is equipped with a depth camera. Through the image data transmitted back by the depth camera and in combination with the operation intention of a human operator, the interactive construction and modification of the map are realized. The embodiments of the present invention support real-time modification and dynamic update during the map construction process, can adapt to complex dynamic environments, and expand the application scope of the robot. Using the depth camera as the main sensor reduces the cost of hardware devices. At the same time, through the visual interface and the hybrid decision-making mechanism combined with the operator's behavior, the semantic understanding ability of the robot for the map is improved, and the reliability and usability of the map are significantly enhanced. It provides a solid foundation for the subsequent navigation logic of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0105] Figure 1 It is a flowchart of a method for robot-interactive local map construction based on a depth camera provided by an embodiment of the present invention;

[0106] Figure 2 It is a flowchart of a robot system for all-weather autonomous following of a specific person provided by an embodiment of the present invention;

[0107] Figure 3 It is a schematic diagram of an interactive map construction module provided by an embodiment of the present invention;

[0108] Figure 4 It is a schematic diagram of an interactive interface provided by an embodiment of the present invention;

[0109] Figure 5 It is a flowchart of map modification of an interactive interface provided by an embodiment of the present invention;

[0110] Figure 6 It is a schematic diagram of the alignment effect of a depth image and a color image provided by an embodiment of the present invention;

[0111] Figure 7 It is a block diagram of a robot-interactive local map construction system based on a depth camera provided by an embodiment of the present invention;

[0112] Figure 8 It is a schematic structural diagram of a device for robot-interactive local map construction provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0113] The technical solutions in the present invention will be described below with reference to the accompanying drawings.

[0114] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to represent examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0115] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they convey are the same. "Of", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when their differences are not emphasized, the meanings they convey are the same.

[0116] In the embodiments of the present invention, sometimes a subscript such as W 1 may be written in a non-subscript form such as W1. When their differences are not emphasized, the meanings they convey are the same.

[0117] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0118] The embodiments of the present invention provide a method for constructing an interactive local map of a robot based on a depth camera. This method can be implemented by an interactive local map construction device of the robot, and the interactive local map construction device of the robot can be a terminal or a server. As Figure 1 、 Figure 2 shown in the flowchart of the method for constructing an interactive local map of a robot based on a depth camera, the processing flow of this method can include the following steps:

[0119] S1. The depth camera of the mobile robot acquires a depth image and a color image, and the IMU of the mobile robot acquires IMU data, and sends the depth image, color image and IMU data to the interactive operation platform.

[0120] S2. The interactive operation platform estimates the pose of the robot in the global coordinate system in real time according to the color image, IMU data and the global odometry module.

[0121] In a feasible implementation, in order to ensure that the constructed map can accurately reflect the information of the robot's surrounding environment, it is necessary to determine the pose of the robot in the global environment. The global odometer is a basic module for robot positioning and map construction. Its core task is to estimate the pose (position and attitude) of the robot in the global coordinate system in real time through multi-sensor data fusion, providing a spatio-temporal reference for subsequent map construction, path planning, and human-robot interaction.

[0122] In the present invention, the VINS-MONO algorithm is used as the algorithm of the global odometer module. By tightly coupling and optimizing monocular vision and IMU (Inertial Measurement Unit) data, it outputs the high-precision robot pose in real time, providing a spatio-temporal reference for the point cloud alignment of the depth camera, dynamic map construction, and operator interaction, and ensuring the positioning robustness of the system in complex environments. By fusing monocular camera and IMU data, the pose estimation of the robot is realized, and finally the pose estimation of the robot in the global environment is output. Through the high-precision pose estimation of the VINS-MONO algorithm, the point cloud data of the depth camera can be aligned with the robot pose information, thus providing an accurate spatio-temporal reference for dynamic map construction and operator interaction, and ensuring the positioning robustness of the system in complex environments.

[0123] S3. The interactive operation platform constructs an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and the depth image, calculates the terrain features and traversal costs of each grid, and then constructs a local traversability map centered on the robot, and feeds back the local traversability map centered on the robot to the operator through the local traversability map display module.

[0124] Optionally, the above step S3 may include the following steps S31-S36:

[0125] S31. Set the traversal ability weight according to the traversal ability of the robot for different terrain features.

[0126] S32. Obtain an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and the depth image.

[0127] In a feasible implementation, the traversability map generated by the present invention is a local map centered on the robot. Compared with maintaining a global map, the construction method of the local map has higher computational efficiency and is suitable for quickly perceiving and responding in an unknown environment. The elevation grid map, as a dynamic local elevation map, is specifically used to create and maintain a local elevation map centered on a micro-miniature mobile platform. This method effectively avoids the huge computational and storage overhead brought by the global map and is more suitable for real-time application scenarios.

[0128] Furthermore, the system will accept the original point cloud collected by the sensor and the odometer information of the platform to generate an elevation grid map centered on the robot. Subsequently, the system will process the elevation data in the elevation grid map. According to the received elevation map data, the calculation of terrain features and corresponding costs, the creation of the original cost map, and the generation of the final filtered map are realized. The process is as Figure 3 shown.

[0129] S33. Filter the elevation grid map using the Savitzky-Golay filter to obtain the filtered elevation grid map.

[0130] In a feasible implementation, due to the limitations of platform movement and sensor characteristics, the original data often has problems such as noise, holes, and discontinuities, resulting in discontinuous invalid data in the generated elevation grid map. The Savitzky-Golay filter is used to filter the elevation grid map. This filter can retain the high-order derivative information of the signal while smoothing the noise, avoiding the over-smoothing caused by traditional moving average filtering and is particularly suitable for processing terrain elevation data with obvious peaks and valleys:

[0131]

[0132] where D′(x,y) is the height value of the filtered grid. c i,j is the filtering coefficient obtained by the least squares method. w represents the window size of the two-dimensional filter. represents the original height data within the window.

[0133] S34. For the filtered elevation grid map, calculate the parameters used to describe the terrain features of each grid.

[0134] In a feasible implementation, in order to calculate the terrain passability, the terrain features of each grid are described by calculating the following parameters:

[0135] 1. Calculate the slope:

[0136] Use a 3×3 Sobel operator to calculate the height gradients in the X and Y directions. The gradient in the X direction is calculated according to Equation (2):

[0137]

[0138] The gradient in the Y direction is calculated according to Equation (3):

[0139]

[0140] The final slope value is the Euclidean norm of the gradients in the two directions:

[0141]

[0142] 2. Calculate roughness:

[0143] Calculate the standard deviation of the height values within a local window of a specified size (default 5×5). For each point within the window, calculate the sum of the squares of the height differences from the center point. The calculation formula is shown in Equation (5):

[0144]

[0145] In the formula, roughness represents roughness, h(i,j) represents the height of any point within the window, h_center represents the height of the center point, and n represents the number of grid cells within the window.

[0146] 3. Calculate steepness:

[0147] Calculate the ratio of the maximum height difference to the resolution within a 3×3 local window. The calculation formula is shown in Equation (6):

[0148]

[0149] In the formula, stepness represents steepness, h max represents the maximum height value within the window, h min represents the minimum height value within the window, and resolution represents the resolution of the map grid.

[0150] 4. Calculate slope change rate:

[0151] Calculate the change rates of the slope in the X and Y directions for each grid cell. Change rate in the X direction:

[0152]

[0153] Change rate in the Y direction:

[0154]

[0155] The final slope change rate is the modulus of the change rate in the X direction and the change rate in the Y direction:

[0156]

[0157] In the formula, slope_change_rate represents the slope change rate, dsx represents the change rate of the slope in the x direction, and dsy represents the change rate of the slope in the y direction.

[0158] S35. Normalize the terrain features to obtain the traversal cost of the robot for each type of terrain.

[0159] In a feasible implementation, according to the traversal capabilities of the robot for different terrain features, the operator can set the traversal capability weights of the robot for different terrain features in the interactive interface. Normalize all the terrain features to obtain the traversal cost of the robot for each type of terrain:

[0160] cost = 1 - e (-decay_factor*normalized_feature_value) (10)

[0161] In the formula, cost represents the traversal cost, decay_factor represents an adjustable parameter for controlling the rate of cost increase, and normalized_feature_value represents the result of normalizing the feature value to the range [0, 1].

[0162] S36. Obtain the local traversability map centered on the robot according to the traversal capability weights and the traversal cost.

[0163] In a feasible implementation, the weighted sum of the terrain traversal cost and the terrain traversal capability weight of the map is the total traversal cost of the map. Thus, a local traversability map centered on the robot can be obtained.

[0164] S4. The operator selects the area to be modified in the color image through the modification module of the interactive image display window of the interactive operation platform. The interactive operation platform obtains the corresponding map grid of the area to be modified in the local traversability map centered on the robot according to the image alignment module of the depth camera, and modifies the traversal cost of the grid, so as to realize the operator's real-time modification of the map and generate a map that can be used for robot navigation.

[0165] In a feasible implementation, the present invention adopts a modular architecture design. The interactive modification of the map is mainly realized by the interactive interface and the traversal cost calculation module. Through the collaborative work of the multimodal human-computer interaction interface and the intelligent decision-making algorithm, the adaptive map construction and optimization in a dynamic environment are realized. The system front-end develops a cross-platform visual interaction interface based on the QT framework, integrates a remote communication protocol stack, and supports the operator to establish a low-latency bidirectional data link with the mobile robot through the TCP / IP protocol, effectively ensuring the real-time interaction requirements in the remote operation scenario.

[0166] During the system initialization phase, the user needs to complete the motion characteristic modeling of the mobile platform through the parameter configuration panel. Specifically, according to the mechanical structure characteristics and kinematic constraints of the target robot, the physical threshold ranges of multi-dimensional terrain feature parameters need to be input in the interactive interface, including but not limited to key performance indicators such as the maximum climbable slope, ground roughness tolerance, and vertical obstacle crossing height. At the same time, a weighted evaluation matrix needs to be constructed to quantify the influence weight coefficients of various terrain features on the robot's motion energy consumption, safety risk, and passing efficiency through the Analytic Hierarchy Process (AHP), providing a mathematical basis for subsequent passability modeling.

[0167] As Figure 4 shown in the interface, the human-machine interaction interface adopts a three-view layout design: on the left is a two-dimensional grid map visualization window constructed based on SLAM technology, which real-time displays the local passability map centered on the robot; the central area integrates an image annotation module, supporting the visualization and interactive annotation of the raw data of the depth camera; the right control panel contains functional units such as motion instruction issuance, map correction confirmation, and system status monitoring, forming a complete map closed-loop correction system.

[0168] The interactive modification of the map is mainly achieved through the operator's enclosure operation and corresponding buttons in the interactive image display window. Clicking the mouse in the interactive image display window can fix the current image, and the system will record the global pose information of the robot at the current moment, and then the enclosure operation can be performed on this image.

[0169] By clicking or dragging the mouse, a region can be selected in the color image, and then the operator can change the attributes of the enclosed region according to their own needs (such as considering the enclosed region as a puddle). The system will preset terrain attributes such as roads, grasslands, muddy lands, and obstacles. After enclosing a region in the image, the system will, through the image alignment module, find the map grid corresponding to the enclosed image region and establish the mapping between the image and the grid map. After the operator selects the terrain attribute, the system will comprehensively consider the previously input robot passing ability and the terrain characteristics of the enclosed region, and recalculate the passing cost of the enclosed region. And the change in the passing cost of the enclosed region in the image will be reflected in the constructed passability map, changing the cost value of the map grid corresponding to the enclosed image region. In this way, the interactive modification of the passability map is realized.

[0170] All manually corrected areas are recorded in the system. When the robot subsequently perceives the same environmental area, the system will preferentially use the manually corrected data to overwrite the real-time perception results, thus ensuring the effective inheritance and persistent application of human experience. This hybrid decision-making mechanism not only retains the environmental adaptability of the autonomous mapping system but also incorporates the domain knowledge of human operators, significantly improving the map reliability in complex scenarios. The process of interactive map modification is as Figure 5 shown.

[0171] Optionally, an image alignment module of the depth camera is used to establish a pixel-level mapping relationship between the color image and the depth image, and align the depth image and the color image according to the pixel-level mapping relationship.

[0172] In a feasible implementation, in the depth camera system, the core task of image alignment is to establish a pixel-level mapping relationship between the color image (RGB) and the depth image (Depth), ensuring that the color information and depth value of the same scene point correspond precisely. This process is the basis for subsequent 3D reconstruction and interactive map construction. The alignment effect is as Figure 6 shown.

[0173] The depth camera can capture a pair of depth images and color images in the same frame. For the camera, its internal parameter matrix K can be expressed as:

[0174]

[0175] In the formula, dx and dy respectively represent the physical sizes of each pixel on the image plane, (u 0 , v 0 ) is the origin coordinate in the image coordinate system, and f is the camera focal length. Let the internal parameter matrix of the depth lens in the depth camera be K d , and the internal parameter matrix of the color lens be K c . The external parameter matrix between the color camera and the depth camera is T, which includes the rotation matrix R and the translation vector T trans , and can be expressed as:

[0176]

[0177] To restore the two-dimensional coordinates in the depth image to the three-dimensional coordinates in the camera coordinate system, the following formula can be used:

[0178]

[0179] In the formula, P d represents the three-dimensional space coordinate corresponding to a certain pixel in the depth image, K d represents the internal parameter matrix of the depth lens in the depth camera, (u d , v dIndicates the pixel coordinates of each pixel in the depth image, and d represents the depth value corresponding to each pixel, that is, the distance between the pixel and the camera.

[0180] To map the corresponding three-dimensional points in the depth image to the color image, the following formula is needed:

[0181]

[0182] In the formula, P d is the three-dimensional point coordinates in the depth image (obtained by converting the depth value), R and T trans represent the rotation matrix and translation vector between the color camera and the depth camera, and K c represents the internal parameter matrix of the color lens, and (u c , v c ) represents the pixel coordinates of the pixel block in the color image.

[0183] After the image alignment operation, the pixels in the color image will establish a corresponding mapping relationship with the pixels in the depth image, and finally the depth value corresponding to each pixel in the color image will be obtained, providing a basis for subsequent interactive map annotation by the user. Obtaining accurate camera internal parameter matrix K and external parameter T matrix through camera calibration is a prerequisite for implementing the present invention.

[0184] In the embodiment of the present invention, a system capable of realizing real-time interactive map construction is provided. The system is equipped with a depth camera, and through the image data transmitted back by the depth camera, combined with the operation intention of the human operator, the interactive construction and modification of the map are realized. The example of the present invention supports real-time modification and dynamic update during the map construction process, can adapt to complex dynamic environments, and expands the application scope of the robot. Using the depth camera as the main sensor reduces the cost of hardware devices. At the same time, through the visual interface and the hybrid decision-making mechanism combined with the operator's behavior, the semantic understanding ability of the robot for the map is improved, and the reliability and usability of the map are significantly improved. It provides a solid foundation for the subsequent navigation logic of the robot.

[0185] Figure 7 is a block diagram of a robot-interactive local map construction system based on a depth camera shown according to an exemplary embodiment. This system is used for a robot-interactive local map construction method based on a depth camera. Referring to Figure 7 , the system includes an operator, an interactive operation platform, and a mobile robot, where:

[0186] The operator is used to select the area to be modified in the color image through the modification module of the interactive image display window of the interactive operation platform.

[0187] An interactive operation platform is used to estimate the pose of a robot in the global coordinate system in real time based on color images, IMU data, and a global odometry module; construct an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and depth images, calculate the terrain features and traversability costs of each grid, and then construct a local traversability map centered on the robot, and feedback the local traversability map centered on the robot to the operator through a local traversability map display module; obtain the map grid corresponding to the area to be modified in the local traversability map centered on the robot according to the image alignment module of the depth camera, modify the traversability cost of the grid, realize real-time modification of the map by the operator, and generate a map that can be used for robot navigation.

[0188] A mobile robot obtains depth images and color images through the depth camera of the mobile robot, obtains IMU data through the IMU of the mobile robot, and sends the depth images, color images, and IMU data to the interactive operation platform.

[0189] Optionally, the interactive operation platform is further used for:

[0190] S31. Set the traversability ability weight according to the traversability ability of the robot for different terrain features.

[0191] S32. Obtain an elevation grid map centered on the robot according to the pose of the robot in the global coordinate system and depth images.

[0192] S33. Filter the elevation grid map using a Savitzky-Golay filter to obtain a filtered elevation grid map.

[0193] S34. Calculate the parameters for describing the terrain features of each grid for the filtered elevation grid map.

[0194] S35. Normalize the terrain features to obtain the traversability cost of the robot for each type of terrain.

[0195] S36. Obtain a local traversability map centered on the robot according to the traversability ability weight and traversability cost.

[0196] Optionally, the interactive operation platform is further used for:

[0197] Calculate the slope as shown in the following formula (1):

[0198]

[0199] In the formula, slope represents the slope, Gx represents the height gradient in the x direction, and Gy represents the height gradient in the y direction. Among them, Gx is calculated according to the following formula (2):

[0200]

[0201] Wherein, h(i,j) represents the height of any point within the window, i represents the abscissa of the grid, j represents the ordinate of the grid, and resolution represents the resolution of the map.

[0202] Gy is calculated according to the following formula (3):

[0203]

[0204] Calculate the roughness, as shown in the following formula (4):

[0205]

[0206] Wherein, roughness represents the roughness, h(i,j) represents the height of any point within the window, h_center represents the height of the center point, and n represents the number of grids within the window.

[0207] Calculate the steepness, as shown in the following formula (5):

[0208]

[0209] Wherein, stepness represents the steepness, h max represents the maximum height value within the window, h min represents the minimum height value within the window, and resolution represents the resolution of the map grid.

[0210] Calculate the slope change rate, as shown in the following formula (6):

[0211]

[0212] Wherein, slope_change_rate represents the slope change rate, dsx represents the change rate of the slope in the x direction, and dsy represents the change rate of the slope in the y direction. Among them, dsx is calculated according to the following formula (7):

[0213]

[0214] dsy is calculated according to the following formula (8);

[0215]

[0216] Optionally, the traversal cost of the robot for each type of terrain is as shown in the following formula (9):

[0217] cost = 1 - e (-decay_factor*normalized_feature_value) (9)

[0218] Wherein, cos t represents the passing cost, decay_factor represents an adjustable parameter for controlling the rate of cost increase, and normalized_feature_value represents the result of normalizing the feature value to the range of [0, 1].

[0219] Optionally, an image alignment module of the depth camera is configured to establish a pixel-level mapping relationship between the color image and the depth image, and align the depth image and the color image according to the pixel-level mapping relationship.

[0220] Among them, the two-dimensional coordinates in the depth image are restored to the three-dimensional coordinates in the depth camera coordinate system through the following formula (10):

[0221]

[0222] In the formula, P d represents the three-dimensional space coordinates corresponding to a certain pixel in the depth image, K d represents the internal parameter matrix of the depth lens in the depth camera, (u d , v d ) represents the pixel coordinates of each pixel in the depth image, and d represents the depth value corresponding to each pixel.

[0223] The three-dimensional points in the depth image are projected into the pixel coordinate system of the color image through the following formula (11):

[0224]

[0225] In the formula, (u c , v c ) represents the pixel coordinates of the pixel block in the color image, K c represents the internal parameter matrix of the color lens in the depth camera, R represents the rotation matrix between the color camera and the depth camera, and T trans represents the translation vector between the color camera and the depth camera.

[0226] Optionally, it further includes:

[0227] Determine the physical threshold range of the terrain feature parameters according to the mechanical structure characteristics and kinematic constraints of the robot.

[0228] Construct a weighted evaluation matrix, and quantify the influence weight coefficients of each terrain feature on the motion energy consumption, safety risk and passing efficiency of the robot through the analytic hierarchy process.

[0229] In an embodiment of the present invention, a system capable of realizing real-time interactive map construction is provided. The system is equipped with a depth camera. Through the image data transmitted back by the depth camera and in combination with the operation intention of a human operator, the interactive construction and modification of the map are realized. The example of the present invention supports real-time modification and dynamic update during the map construction process, can adapt to complex dynamic environments, and expands the application scope of the robot. Using the depth camera as the main sensor reduces the cost of hardware devices. At the same time, through the visual interface and the hybrid decision-making mechanism combined with the operator's behavior, the semantic understanding ability of the robot for the map is improved, and the reliability and usability of the map are significantly enhanced. This provides a solid foundation for the subsequent navigation logic of the robot.

[0230] Figure 8 FIG. is a schematic structural diagram of a robot-interactive local map construction device provided by an embodiment of the present invention, as Figure 8 shown, the robot-interactive local map construction device may include the above Figure 7 shown robot-interactive local map construction system based on a depth camera. Optionally, the robot-interactive local map construction device 410 may include a first processor 2001.

[0231] Optionally, the robot-interactive local map construction device 410 may further include a memory 2002 and a transceiver 2003.

[0232] Wherein, the first processor 2001, the memory 2002, and the transceiver 2003 may be connected through a communication bus, for example.

[0233] Next, in combination with Figure 8 each component of the robot-interactive local map construction device 410 will be specifically introduced:

[0234] Among them, the first processor 2001 is the control center of the robot-interactive local map construction device 410, and may be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or may be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0235] Optionally, the first processor 2001 can perform various functions of the robot-interactive local map construction device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0236] In a specific implementation, as an example, the first processor 2001 may include one or more CPUs, such as Figure 8 the CPU0 and CPU1 shown in

[0237] In a specific implementation, as an example, the robot-interactive local map construction device 410 may also include multiple processors, such as Figure 8 the first processor 2001 and the second processor 2004 shown in

[0238] Among them, the memory 2002 is used to store software programs for implementing the solution of the present invention and is controlled by the first processor 2001 for execution. The specific implementation manner can refer to the above method embodiments and will not be elaborated here.

[0239] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 8 not shown in

[0240] The transceiver 2003 is used to communicate with network devices or terminal devices.

[0241] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 8 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0242] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and is coupled to the first processor 2001 through an interface circuit ( Figure 8 not shown) of the robot-interactive local map construction device 410. The embodiments of the present invention do not make specific limitations on this.

[0243] It should be noted that Figure 8 the structure of the robot-interactive local map construction device 410 shown in

[0244] does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0245] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and this processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or this processor may also be any conventional processor, etc.

[0246] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0247] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be solid-state drives.

[0248] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship. The specific meaning can be understood by referring to the context.

[0249] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0250] It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0251] Those of ordinary skill in the art will realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0252] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the devices, apparatuses, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0253] In several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0254] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0255] In addition, the functional units in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0256] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs.

[0257] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A method for constructing a robot interactive local map based on a depth camera, characterized in that: The method is implemented by a robot interactive local map construction system based on a depth camera, the system comprising an operator, an interactive operation platform and a mobile robot, and the method comprises: S1, the depth camera of the mobile robot obtains a depth image and a color image, the IMU of the mobile robot obtains IMU data, and the depth image, color image and IMU data are sent to the interactive operation platform; S2, the interactive operation platform estimates the position and posture of the robot in the global coordinate system in real time according to the color image, IMU data and global odometer module; S3, the interactive operation platform constructs an elevation grid map centered on the robot according to the position and posture of the robot in the global coordinate system and the depth image, calculates the terrain features and the travel cost of each grid, and then constructs a local traversability map centered on the robot, and feeds back the local traversability map centered on the robot to the operator through a local traversability map display module; S4. The operator selects the area to be modified in the color image through the interactive image display window modification module of the interactive operation platform. The interactive operation platform obtains the map grid corresponding to the area to be modified in the local passability map centered on the robot according to the image alignment module of the depth camera, and modifies the pass cost of the grid, so that the operator can modify the map in real time and generate a map that can be used for robot navigation.

2. The method for constructing a robot interactive local map based on a depth camera according to claim 1, characterized in that: The step S3 constructs an elevation grid map centered on the robot according to the position of the robot in the global coordinate system and the depth image, calculates the terrain features and the travel cost of each grid, and then constructs a local traversability map centered on the robot, including: S31, setting a traffic capacity weight according to the robot's traffic capacity for different terrain features; S32, obtaining an elevation grid map centered on the robot according to the position and posture of the robot in the global coordinate system and the depth image; S33, filtering the elevation grid map using a Savitzky-Golay filter to obtain a filtered elevation grid map; S34, calculating parameters for describing the terrain features of each grid for the filtered elevation grid map; S35, normalizing the terrain features to obtain the travel cost of the robot for each type of terrain; S36: Obtain a local traversability map centered on the robot according to the traversability weight and the traversability cost.

3. The method for constructing a robot interactive local map based on a depth camera according to claim 2, characterized in that: The parameters calculated in S33 for describing the terrain characteristics of each grid include: Calculate the slope as shown in the following formula (1): In the formula, slope represents the slope, Gx represents the height gradient in the x direction, and Gy represents the height gradient in the y direction; Gx is calculated according to the following formula (2): In the formula, h(i,j) represents the height of any point in the window, i represents the horizontal coordinate of the grid, j represents the vertical coordinate of the grid, and resolution represents the resolution of the map; Gy is calculated according to the following formula (3): The roughness is calculated as shown in the following formula (4): In the formula, roughness represents the roughness, h(i,j) represents the height of any point in the window, h_center represents the height of the center point, and n represents the number of grids in the window; The steepness is calculated as shown in the following formula (5): In the formula, stepness represents the steepness, h max Indicates the maximum height value within the window, h min Indicates the minimum height value in the window, and resolution indicates the resolution of the map grid; Calculate the slope change rate as shown in the following formula (6): Where slope_change_rate represents the slope change rate, dsx represents the slope change rate in the x direction, and dsy represents the slope change rate in the y direction; Wherein, dsx is calculated according to the following formula (7): dsy is calculated according to the following formula (8):

4. The method for constructing a robot interactive local map based on a depth camera according to claim 2, characterized in that: The travel cost of the robot in S34 for each type of terrain is shown in the following formula (9): cost=1-e (-decay_factor*normalized_feature_value) (9) In the formula, cost represents the travel cost, decay_factor represents the adjustable parameter used to control the rate of cost growth, and normalized_feature_value represents the result of normalizing the feature value to the range of [0,1].

5. The method for constructing a robot interactive local map based on a depth camera according to claim 1, characterized in that: The image alignment module of the depth camera is used to establish a pixel-level mapping relationship between the color image and the depth image, and align the depth image and the color image according to the pixel-level mapping relationship; The two-dimensional coordinates in the depth image are restored to the three-dimensional coordinates in the depth camera coordinate system by the following formula (10): Where P d Indicates the three-dimensional space coordinates corresponding to a pixel in the depth image, K d Represents the intrinsic parameter matrix of the depth lens in the depth camera, (u d ,v d ) represents the pixel coordinates of each pixel in the depth image, and d represents the depth value corresponding to each pixel; The 3D points in the depth image are projected into the pixel coordinate system of the color image using the following equation (11): In the formula, (u c ,v c ) represents the pixel coordinates of the pixel block in the color image, K c represents the intrinsic parameter matrix of the color lens in the depth camera, R represents the rotation matrix between the color camera and the depth camera, T trans Represents the translation vector between the color camera and the depth camera.

6. The method for constructing a robot interactive local map based on a depth camera according to claim 1, characterized in that: The method further comprises: According to the mechanical structure characteristics and kinematic constraints of the robot, the physical threshold range of the terrain feature parameters is determined; A weighted evaluation matrix was constructed, and the weight coefficients of the impact of terrain features on robot motion energy consumption, safety risks and passing efficiency were quantified through the hierarchical analysis method.

7. A robot interactive local map construction system based on a depth camera, wherein the robot interactive local map construction system based on a depth camera is used to implement the robot interactive local map construction method based on a depth camera as claimed in any one of claims 1 to 6, characterized in that: The system includes an operator, an interactive operation platform and a mobile robot, wherein: The operator is used to select a region to be modified in the color image through the interactive image display window modification module of the interactive operation platform; The interactive operation platform is used to estimate the position and posture of the robot in the global coordinate system in real time according to the color image, IMU data and global odometer module; construct an elevation grid map centered on the robot according to the position and posture of the robot in the global coordinate system and the depth image, calculate the terrain features and the travel cost of each grid, and then construct a local passability map centered on the robot, and feed back the local passability map centered on the robot to the operator through the local passability map display module; obtain the map grid corresponding to the area to be modified in the local passability map centered on the robot according to the image alignment module of the depth camera, modify the travel cost of the grid, enable the operator to modify the map in real time, and generate a map that can be used for robot navigation; The mobile robot obtains a depth image and a color image through a depth camera of the mobile robot, obtains IMU data through an IMU of the mobile robot, and sends the depth image, color image and IMU data to the interactive operation platform.

8. The robot interactive local map construction system based on a depth camera according to claim 7, characterized in that: According to the position and posture of the robot in the global coordinate system and the depth image, an elevation grid map centered on the robot is constructed, the terrain features and the travel cost of each grid are calculated, and then a local traversability map centered on the robot is constructed, including: S31, setting a traffic capacity weight according to the robot's traffic capacity for different terrain features; S32, obtaining an elevation grid map centered on the robot according to the position and posture of the robot in the global coordinate system and the depth image; S33, filtering the elevation grid map using a Savitzky-Golay filter to obtain a filtered elevation grid map; S34, calculating parameters for describing the terrain features of each grid for the filtered elevation grid map; S35, normalizing the terrain features to obtain the travel cost of the robot for each type of terrain; S36: Obtain a local traversability map centered on the robot according to the traversability weight and the traversability cost.

9. A robot interactive local map construction device, characterized in that: The robot interactive local map building device comprises: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 6.

Images