Mapping method and device based on depth meter and laser radar, equipment and medium

Abstract

The application relates to a mapping method and device based on a depth gauge and a laser radar, equipment and a storage medium. The method comprises the following steps: acquiring inertial measurement unit data, depth gauge data and laser radar data of a robot, converting the measurement unit data, the depth gauge data and the laser radar data into posture information, depth information and target point cloud information of the robot respectively, fusing the posture information and the depth information to obtain three-dimensional position posture information of the robot, performing a rotation compensation operation on the target point cloud information according to the three-dimensional position posture information to obtain a local three-dimensional grid map, and performing a fusion operation on the local three-dimensional grid map and a pre-constructed global map to obtain a target three-dimensional map. The application can improve the accuracy and robustness of underwater three-dimensional mapping.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to a mapping method, apparatus, device, and storage medium based on depth gauges and lidar. Background Technology

[0002] Currently, most pool cleaning robots rely on a single sensor for environmental perception and path planning. However, factors such as severe underwater light attenuation, texture loss, water surface reflection, and water flow disturbance degrade the performance of visual and laser sensors. This makes it difficult for existing mapping methods to accurately represent the three-dimensional structures of the pool bottom, such as steps, slopes, and drains, resulting in geometrical map distortion. Furthermore, local point clouds are often distorted due to robot posture fluctuations. If these distortions are not corrected and effectively integrated with the global map, they will lead to map ghosting, misalignment, or information conflicts, severely impacting the integrity of path planning and obstacle avoidance safety.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] In view of the above, this application provides a mapping method, apparatus, device and storage medium based on depth gauge and lidar, the purpose of which is to solve the above-mentioned technical problems.

[0005] In a first aspect, this application provides a mapping method based on depth measurement and lidar, the method comprising: Acquire data from the robot's inertial measurement unit, depth gauge, and lidar. The measurement unit data, the depth gauge data, and the lidar data are respectively converted into robot posture information, depth information, and target point cloud information; The posture information and the depth information are fused to obtain the robot's three-dimensional position and posture information; Based on the three-dimensional position and attitude information, a rotation compensation operation is performed on the target point cloud information to obtain a local three-dimensional raster map; The local 3D raster map is fused with the pre-built global map to obtain the target 3D map.

[0006] Secondly, this application provides a mapping device based on a depth sensor and a lidar, the mapping device comprising: Acquisition module: Used to acquire data from the robot's inertial measurement unit, depth gauge, and lidar. Conversion module: used to convert the measurement unit data, the depth gauge data and the lidar data into robot posture information, depth information and target point cloud information respectively; Fusion module: used to fuse the posture information with the depth information to obtain the robot's three-dimensional position and posture information; Compensation module: used to perform rotation compensation operation on the target point cloud information based on the three-dimensional position and attitude information to obtain a local three-dimensional raster map; Generation module: used to merge the local 3D raster map with the pre-built global map to obtain the target 3D map.

[0007] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the steps of the mapping method based on depth gauges and lidar as described in any embodiment of the first aspect.

[0008] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the mapping method based on depth gauges and lidar as described in any embodiment of the first aspect.

[0009] The technical solutions provided in this application have the following advantages compared with the prior art: This application constructs a six-DOF pose estimation system with reliable vertical constraints by fusing data from inertial measurement units, depth gauges, and lidar. Rotation compensation and voxelization of the target point cloud effectively eliminate point cloud distortion caused by robot tilt. By weighted fusion of the aligned local 3D grid map and the global map in overlapping areas based on occupancy probability, observation timestamps, and confidence levels, a geometrically consistent and structurally complete 3D target map can be generated. This improves the accuracy and robustness of underwater 3D mapping, accurately reconstructing complex pool bottom terrain and providing a reliable environmental awareness foundation for pool cleaning robots to achieve high coverage operations, safe obstacle avoidance, and task reuse. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a preferred embodiment of the mapping method based on depth gauges and lidar in this application; Figure 2 This is a schematic diagram of a preferred embodiment of the mapping device based on depth gauge and lidar of this application; Figure 3 This is a schematic diagram of a preferred embodiment of the electronic device of this application; The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0014] It should be noted that the use of terms such as "first" and "second" in this application is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of those features. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, such a combination of technical solutions should be considered non-existent and not within the scope of protection claimed in this application.

[0015] Reference Figure 1 The diagram shown is a flowchart illustrating an embodiment of the mapping method based on depth sensors and lidar according to this application. The method is executed by an electronic device, which can be implemented by a software system and / or a hardware system. The mapping method based on depth sensors and lidar includes: Step S10: Acquire the robot's inertial measurement unit data, depth gauge data, and lidar data; Step S20: Convert the measurement unit data, the depth gauge data, and the lidar data into robot posture information, depth information, and target point cloud information, respectively; Step S30: Fuse the posture information with the depth information to obtain the robot's three-dimensional position and posture information; Step S40: Perform rotation compensation operation on the target point cloud information based on the three-dimensional position and pose information to obtain a local three-dimensional grid map; Step S50: Perform a fusion operation between the local 3D raster map and the pre-built global map to obtain the target 3D map.

[0016] This embodiment uses a swimming pool cleaning robot as an example. When performing autonomous cleaning tasks, the swimming pool cleaning robot needs to perceive the underwater environment in real time and construct an accurate map to achieve comprehensive coverage and safe obstacle avoidance. Since the bottom of the pool has non-planar structures such as steps, slopes, and drains, relying solely on LiDAR for 2D mapping or using only visual sensors for target recognition will lead to map distortion, thus affecting the integrity and safety of path planning. Therefore, this embodiment proposes a mapping method that integrates inertial measurement unit (IMU), depth gauge, and LiDAR data. By introducing reliable vertical height constraints, attitude correction is performed on the LiDAR point cloud, and closed-loop fusion of local and global maps is achieved, thereby constructing a geometrically consistent and structurally complete 3D environmental map, providing the swimming pool cleaning robot with high-precision spatial cognition capabilities.

[0017] After the robot starts up and enters mapping mode, the main controller periodically reads acceleration and angular velocity signals from the inertial measurement unit (IMU), analog or digital signals reflecting water depth from the depth gauge, and raw point cloud data obtained from a single frame scan from the lidar via the internal communication bus. Understandably, all sensor data acquisition is performed under a unified time reference. Through hardware synchronization signals or software timestamp alignment mechanisms, it is ensured that the three types of data correspond to the same physical moment in time, avoiding pose estimation errors caused by asynchronous sampling.

[0018] The attitude information generation is handled by the attitude calculation submodule. This submodule receives the acceleration and angular velocity signals output by the inertial measurement unit, performs zero-bias correction and noise suppression on the raw signals, and then recursively estimates the attitude based on a multi-sensor fusion algorithm. During the estimation process, the sensitivity of the acceleration signal to the direction of gravity under low dynamic conditions is utilized to periodically correct the attitude drift generated by the integral of angular velocity. This allows the robot's pitch, roll, and yaw angles relative to the world coordinate system at the current moment to be obtained, thus forming the attitude information.

[0019] The generation of depth information is handled by the depth processing submodule. This submodule receives the raw electrical signal output from the depth gauge, converts it into a preliminary depth value according to the factory calibration parameters, and then calls temperature compensation logic to correct the preliminary depth value. To eliminate abnormal jumps caused by water surface fluctuations, bubble adhesion, or instantaneous disturbances, a sliding window filter can also be performed to retain smooth and stable depth readings. Combined with robot body structural parameters (e.g., the relative relationship between the sensor installation position and the robot's center of mass), the corrected depth value is converted into a vertical position reference of the robot in the world coordinate system, forming the depth information.

[0020] The generation of target point cloud information is handled by a point cloud preprocessing submodule. Upon receiving the raw point cloud from the lidar, this submodule first removes invalid points that are too close or too far away to eliminate interference from sensor blind spots or beyond the measurement range. Through multi-frame point cloud comparison, dynamic points caused by water surface swaying, floating object movement, etc., are identified and removed. Then, isolated or sparsely distributed outliers are removed, preserving stable structural features in the environment. To reduce computational load and improve subsequent processing efficiency, the remaining point cloud is spatially downsampled to reduce the number of points while maintaining geometric details. The point cloud output after these processes is the target point cloud information.

[0021] Since attitude information only describes the robot's orientation and depth information only represents the vertical height, it cannot reflect horizontal displacement and rotation. Therefore, the directional constraints provided by attitude information and the vertical position constraints provided by depth information can be fused to form a complete six-degree-of-freedom pose estimation. A six-degree-of-freedom pose representation containing three-dimensional position and three-axis attitude (pitch, roll, yaw) can be constructed as a unified spatial reference for subsequent point cloud correction and map construction.

[0022] Since the point cloud acquired by LiDAR is based on the robot itself as the reference frame, when the robot's position shifts due to buoyancy, water flow impact, or terrain undulations, the point cloud may exhibit tilting or distortion, failing to accurately reflect the environmental geometry. Directly using such point clouds for mapping would lead to problems such as misjudgment of step heights and distortion of slope angles. Therefore, this embodiment performs coordinate transformation on the point cloud based on 3D position and attitude information, correcting it to the world coordinate system to form a local 3D grid map suitable for fusion. Specifically, a coordinate transformation relationship is constructed using 3D position and attitude information to transform the target point cloud information from the robot coordinate system to the world coordinate system, eliminating geometric distortions caused by robot tilt. Within a local spatial range centered on the robot's current position, the corrected point cloud is divided into voxels, discretizing the continuous point cloud into a regular 3D grid structure, with each voxel marked as being occupied by an obstacle.

[0023] Since the local 3D grid map only covers the robot's current field of view, it needs to be spatially aligned with a pre-built global map. The optimal registration relationship between the two is determined through feature matching and pose optimization. Then, for voxels in overlapping areas, the occupancy status is fused and updated based on factors such as observation time and sensor confidence. The updated results are written to the global map storage area, generating a target 3D map reflecting the current environmental state. The pre-built global map refers to a 3D map covering the pool area that is gradually accumulated by the robot during its historical operations.

[0024] In one embodiment, fusing the pose information with the depth information to obtain the robot's three-dimensional position and pose information includes: A coordinate transformation matrix is ​​constructed using the pitch and roll angles in the attitude information. Based on the coordinate transformation matrix, the depth information is rotated in the coordinate system to obtain the vertical depth component in the world coordinate system. Based on the yaw angle in the attitude information, determine the rotational disturbance value measured by the depth gauge caused by the robot's rotational motion; Based on the rotational disturbance value, the vertical depth component is compensated to obtain the compensated vertical depth component. The compensated vertical depth component is fused with the attitude angle in the attitude information to generate the robot's three-dimensional position and attitude information.

[0025] The pitch and roll angles at the current moment are extracted from the attitude information, and a three-dimensional rotation transformation matrix from the robot's body coordinate system to the world coordinate system is constructed according to a preset rotation sequence (e.g., pitching around the X-axis first, then rolling around the Y-axis). Since the depth gauge measures the distance along its own mounting direction, which deviates from the Z-axis of the world coordinate system when the robot tilts, the depth information is treated as a vector along the robot's Z-axis. This vector is projected onto the world coordinate system using the aforementioned three-dimensional rotation transformation matrix, and the Z-axis component is extracted as the vertical depth component. This vertical depth component reflects the robot's true altitude reference in the world coordinate system, eliminating geometric projection errors caused by body tilt.

[0026] The robot's current yaw rate is calculated by monitoring the rate of change of the yaw angle over time. A rotational disturbance model is constructed by combining the installation position of the depth gauge on the robot body (the lateral offset distance relative to the robot's rotation center). This model maps the angular velocity and offset distance into an equivalent linear disturbance in the depth gauge measurement direction. The equivalent disturbance is used as the rotational disturbance value, which is used to characterize the depth reading deviation caused by yaw rotation.

[0027] While the vertical depth component corrected for pitch and roll angles can eliminate geometric projection errors caused by static tilt to some extent, in actual robot operation, when it continuously rotates around the vertical axis (i.e., the yaw direction), if the depth gauge's installation position deviates from the robot's rotation center (e.g., offset to one side of the housing), the depth gauge itself will move in a circular motion with the robot. This motion causes a tangential velocity component in the depth gauge at the moment of measurement, which, due to factors such as water flow disturbance, sensor response delay, or installation structure, introduces additional dynamic measurement deviations. These deviations cannot be eliminated by static coordinate transformation. If not corrected, the vertical depth component will exhibit periodic fluctuations when the robot turns or rotates in place, affecting the stability of height estimation. Therefore, this embodiment dynamically compensates the vertical depth component based on the rotational disturbance value to suppress such motion coupling errors. By applying dynamic compensation to the vertical depth component that matches the rotational disturbance value, depth measurement drift caused by robot yaw motion can be suppressed, especially when the robot performs typical pool cleaning actions such as turning, circling obstacles, or patrolling along the pool wall, improving the robustness and consistency of vertical height estimation under dynamic conditions.

[0028] By fusing the compensated vertical depth component with the attitude angle, a six-degree-of-freedom three-dimensional position and attitude information with complete structure, consistent coordinates, and time alignment is generated. This three-dimensional position and attitude information has both reliable absolute height constraints and accurate direction description, overcoming the defects of easy drift in pure inertial calculation and lack of height in pure laser mapping, and providing a high-precision spatial reference for subsequent point cloud rotation compensation and local map generation.

[0029] Further, determining the rotational disturbance value measured by the depth gauge due to the robot's rotational motion based on the yaw angle in the attitude information includes: The yaw angle in the attitude information is differentiated over time to obtain the instantaneous angular velocity of the robot about the vertical axis; Multiply the instantaneous angular velocity by the sampling period of the depth gauge to calculate the horizontal arc length displacement caused by yaw rotation within a single sampling interval; Based on the horizontal arc length displacement and the lateral offset distance from the depth gauge installation position to the robot's rotation center, a rotational disturbance geometric model is constructed. The rotational disturbance value is obtained by mapping the horizontal arc length displacement to an equivalent disturbance in the depth gauge measurement direction using the rotational disturbance geometric model.

[0030] Since the yaw angle itself only reflects the robot's current orientation and cannot directly indicate whether it is rotating or how fast it is rotating, motion disturbances only occur when the robot is rotating due to the depth gauge deviating from the center of rotation. Therefore, dynamic features can be extracted from the temporal change of the yaw angle. Specifically, between two consecutive time points when attitude information is acquired, the difference between the current yaw angle and the previous yaw angle is compared, and combined with the time interval between the two time points, the change in yaw angle per unit time, i.e., the instantaneous angular velocity, is calculated. This operation is only performed when the attitude information is updated and the time interval is valid. Its output directly characterizes whether the robot is currently rotating around the vertical axis and the speed and direction of rotation, which is a prerequisite for determining whether there is a rotational disturbance.

[0031] Since the depth gauge completes one data acquisition within a fixed sampling period, if the robot has an angular velocity during this period, the depth gauge probe actually moves along an arc. This movement distance is the source of disturbance, so the angular velocity can be converted into an actual spatial displacement. Specifically, using the instantaneous angular velocity obtained in the previous step, multiplied by the depth gauge's inherent sampling period (which is determined by the hardware configuration and loaded during system initialization), the horizontal arc length displacement traversed by the depth gauge probe due to yaw rotation during a single depth measurement is estimated. This operation is only performed when the instantaneous angular velocity is non-zero and the sampling period is known. Its output quantifies the lateral movement amplitude of the depth gauge within the measurement window and serves as an intermediate quantity connecting rotational dynamics and spatial disturbance.

[0032] Since the horizontal arc length displacement itself is not directly equal to the depth error, it is necessary to consider the physical layout of the depth gauge on the robot body to determine how this displacement affects depth measurement. Furthermore, because the depth gauge is usually installed off-center from the robot's rotation center, its lateral offset distance is a fixed structural parameter, determined through calibration at the robot's factory and stored in the system configuration. Therefore, it is necessary to combine the dynamic arc length displacement with the static installation offset to establish a geometric model that reflects the coupling of horizontal motion to the depth direction. Specifically, the installation offset distance of the depth gauge can be used as a radius reference, and the arc length displacement can be considered as a tangential disturbance input. The combined effect of this disturbance on the depth measurement direction through hydrodynamic effects and structural elastic deformation can be analyzed to construct a mapping function that describes the equivalent depth direction deviation that may occur under a given horizontal displacement.

[0033] The aforementioned geometric model is merely an abstract relationship; it must be applied to the actual arc-length displacement to output specific values ​​suitable for compensation. Specifically, the horizontal arc-length displacement calculated in the second step is substituted into the rotational disturbance geometric model constructed in the third step. Through the model's internal mapping logic, an equivalent disturbance value matching the current rotational state is directly output. This disturbance value is the rotational disturbance value; its magnitude reflects the disturbance intensity, and its sign indicates the disturbance direction (positive indicates a larger depth reading, negative indicates a smaller reading). This operation is performed immediately after the model is constructed and the arc-length displacement is valid, and its output serves as the input for subsequent depth compensation. Compared to existing technologies that ignore rotational effects or use empirical fixed compensation, this embodiment can adaptively adjust the disturbance estimation based on the robot's real-time motion state, improving the reliability of depth information under dynamic conditions. The obtained rotational disturbance value is used for subsequent depth component compensation processing, enabling the robot to maintain high-precision height estimation even during typical pool cleaning actions such as turning, circling, or adjusting orientation.

[0034] Further, the step of fusing the compensated vertical depth component with the attitude angle in the attitude information to generate the robot's three-dimensional position and attitude information includes: The compensated vertical depth component is used as the robot's Z-axis position coordinate in the world coordinate system. Combined with the preset initial horizontal position, an initial pose value containing the three-dimensional position is constructed. Based on the pitch, roll and yaw angles in the attitude information, a set of three-axis attitude angles is generated and aligned with the position portion of the initial pose value using timestamps. Based on the set of three-axis attitude angles, an orientation matrix of the robot body is constructed, and the orientation matrix and the Z-axis position coordinates are jointly encoded into a six-degree-of-freedom pose representation. Based on the attitude change rate at adjacent time points, Z-axis motion constraints are constructed, and the vertical position in the six-degree-of-freedom pose representation is checked and corrected for consistency, thereby generating the robot's three-dimensional position and pose information.

[0035] When the robot starts, the starting point is set as the global origin or an initial horizontal position is set according to the task plan. This position remains unchanged as a horizontal reference throughout the operation or is incrementally updated by other modules (e.g., laser odometry). Based on this, the compensated vertical depth component is used as the Z coordinate and combined with the initial horizontal position to form a three-dimensional position vector containing (X, Y, Z), which serves as a preliminary estimate of the current pose.

[0036] Since attitude information and position information from different points in time will lead to inconsistencies in the pose representation, which in turn will cause distortion in the point cloud transformation, the currently valid pitch angle, roll angle and yaw angle are extracted from the attitude information to form a complete three-axis attitude angle set. The timestamp of this set is checked to see if it is consistent with the timestamp of the position part in the initial pose value. If there is a deviation, it is aligned by interpolation or nearest neighbor matching to ensure that the position and attitude data correspond to the same physical time.

[0037] Since attitude angles are angular parameters, and the orientation matrix, as a standard rotation transformation tool, can efficiently map points from the robot coordinate system to the world coordinate system, the three-axis attitude angle set can be converted into a 3×3 orthogonal rotation matrix, i.e., the robot's orientation matrix, according to a preset rotation sequence (e.g., pitch around the X-axis, roll around the Y-axis, and yaw around the Z-axis). This orientation matrix is ​​then combined with the Z-axis position coordinates (along with the X and Y coordinates) determined in the first step to form a six-DOF pose representation containing rotation and translation information. The six-DOF pose representation retains complete orientation information and possesses a clear spatial position.

[0038] Since the vertical motion of a robot underwater typically exhibits physical continuity—meaning its Z-axis velocity and acceleration are limited by buoyancy, propulsion, and fluid resistance, abrupt changes are unlikely—the rate of change of attitude angles between the current and previous moments is calculated. For example, the trend of pitch angle change. Combined with the robot's dynamic characteristics, a reasonable maximum allowable displacement range for the Z-axis is derived, forming a Z-axis motion constraint. The Z-axis position in the six-degree-of-freedom pose representation is then compared with this constraint. If it exceeds a preset range, it is considered an outlier and smoothed using the Z-value from the previous moment and its trend. If it does not exceed the preset range, it is retained. The output, verified and corrected six-degree-of-freedom pose, serves as the robot's three-dimensional position and attitude information. The three-dimensional position and attitude information generated in this embodiment provides an accurate spatial reference for point cloud rotation compensation, enabling the local three-dimensional grid map to exhibit higher geometric fidelity at complex structures such as slopes, drain outlets, and pool wall corners. This improves the pool cleaning robot's high-coverage operation and obstacle avoidance capabilities across all scenarios.

[0039] In one embodiment, the step of performing a rotation compensation operation on the target point cloud information based on the three-dimensional position and pose information to obtain a local three-dimensional raster map includes: The robot's attitude rotation matrix is ​​extracted from the three-dimensional position and attitude information. The attitude rotation matrix is ​​composed of pitch angle, roll angle and yaw angle in a preset rotation order. The attitude rotation matrix is ​​used to characterize the rotation transformation relationship from the world coordinate system to the robot coordinate system. The attitude rotation matrix is ​​transposed to obtain the point cloud compensation rotation matrix; The coordinates of each three-dimensional point in the target point cloud information are transformed using the point cloud compensation rotation matrix, and the transformed three-dimensional point is added to the position component in the three-dimensional position and attitude information to obtain the translated point cloud data. Within a preset spatial range centered on the location component, the translated point cloud data is voxelized at a preset voxel resolution to generate a local three-dimensional raster map.

[0040] While 3D position and attitude information contains complete six-degree-of-freedom parameters, point cloud transformations require rotation transformations to be expressed in matrix form, and attitude angles themselves cannot be directly used for linear algebraic operations. Therefore, after obtaining updated 3D position and attitude information each time, the pitch, roll, and yaw angles at the current moment can be extracted. Following a pre-defined rotation order (e.g., pitch around the X-axis, roll around the Y-axis, and yaw around the Z-axis), corresponding rotation sub-matrices are constructed sequentially. These three sub-matrices are then multiplied to form a 3×3 orthogonal matrix, i.e., the attitude rotation matrix. The attitude rotation matrix represents any vector in the world coordinate system; after this matrix transformation, its representation in the robot's body coordinate system can be obtained.

[0041] Since each point in the target point cloud is located in the robot coordinate system, and mapping requires transforming it to the world coordinate system, and since the attitude rotation matrix describes the transformation from the world coordinate system to the robot coordinate system, its inverse transformation (i.e., from the robot coordinate system to the world coordinate system) is exactly equal to its transpose (due to the orthogonality of rotation matrices). Therefore, by performing a transpose operation on the attitude rotation matrix, the point cloud compensation rotation matrix can be obtained. The function of the point cloud compensation rotation matrix is ​​to rotate the point cloud in the robot coordinate system back to the world coordinate system, thereby eliminating the geometric distortion caused by the tilt of the robot body.

[0042] The rotated point cloud remains centered at the origin, while the robot is actually located at a specific position in the world coordinate system. Therefore, each 3D point in the target point cloud information can be traversed, treated as a column vector, and multiplied left by the point cloud compensation rotation matrix to complete the orientation correction from the robot coordinate system to the world coordinate system. Then, the X, Y, and Z position components are extracted from the 3D position and attitude information to form a translation vector, which is then added point-by-point to the rotated points to achieve spatial translation. After this operation, each point is accurately mapped to its true position in the world coordinate system, forming the translated point cloud data.

[0043] Because raw point clouds are unstructured data, they are difficult to use directly for map matching and fusion. Voxelization, on the other hand, can discretize continuous space into a regular grid, making it easier to store, compare, and update. To this end, the position component in the 3D position and pose information can be used as the center of a cube to define a local spatial region with a fixed side length (e.g., covering an area of ​​1.5 meters in front and to both sides). Within this region, a 3D grid is divided according to a preset voxel resolution (e.g., each voxel has a side length of 5 centimeters). Then, the translated point cloud data is traversed, and the voxel to which each point falls is marked as "occupied," while the remaining voxels are marked as "free" or "unknown." The combination of the states of all voxels forms a structured local 3D raster map.

[0044] In one embodiment, the step of fusing the local 3D raster map with a pre-built global map to obtain a target 3D map includes: Based on the pose transformation parameters corresponding to the local 3D raster map, the local 3D raster map is transformed to the global coordinate system to obtain aligned local voxel data; Extract a subset of global voxels that spatially overlap with the aligned local voxel data from the pre-constructed global map; For voxels in the aligned local voxel data that have the same spatial coordinates as those in the global voxel subset, the occupancy probability, observation timestamp, and confidence level of the voxels with the same spatial coordinates are read respectively, and the weighted fusion occupancy probability is calculated. The fused occupancy probability is written into the corresponding voxel position in the pre-built global map to generate the target 3D map.

[0045] Since the local 3D grid map is generated in the robot's local coordinate system, with its voxel positions centered on the robot's current position, while the global map uses a unified world coordinate system, the two coordinate systems are inconsistent and cannot be directly compared or fused. Therefore, after generating a new local 3D grid map, its corresponding pose transformation parameters (which include rotation and translation components) need to be called to perform a rigid body transformation on the spatial coordinates of each voxel in the local map. This involves first applying a rotation transformation to correct the direction, and then superimposing a translation vector to adjust the position, thereby mapping all voxels from the local coordinate system to the global coordinate system. After the transformation, the resulting set of voxels is the aligned local voxel data, and its spatial position is now within the same reference frame as the global map.

[0046] Since the global map may cover a local or global area of ​​the pool, and the aligned local voxel data only occupies a small portion of that area, traversing and fusing the entire global map would result in a large amount of unnecessary computation. Therefore, we can first determine the minimum bounding box of the aligned local voxel data in the global space, and then retrieve all voxels falling within this bounding box in the pre-built global map, forming a spatially constrained global voxel subset. This subset only includes regions that are potentially related to local observations, thereby reducing the data scale for subsequent processing while ensuring that all potentially overlapping voxels are included in the fusion range.

[0047] It iterates through all voxel pairs that completely overlap in spatial coordinates (i.e., one from aligned local voxel data and the other from a subset of global voxels), and reads the occupancy probability (representing the likelihood that the location is occupied by an obstacle), observation timestamp (recording the last time the voxel was updated), and confidence level (reflecting the impact of sensor type, observation angle, or environmental conditions on the reliability of the observation) stored in both. Then, based on the freshness of the timestamp and the confidence level, a weighting coefficient is constructed, assigning greater weight to newer observations and data with higher confidence levels. Using this weighting coefficient, a weighted average of the two occupancy probabilities is obtained, yielding a fused occupancy probability.

[0048] The calculated occupancy probabilities are directly written to the voxel storage units at the same locations in the pre-built global map according to their corresponding spatial coordinates, overwriting the original data. For voxels that exist in the local map but were originally "unknown" in the global map, they are directly initialized to the new occupancy state. For non-overlapping areas, the global map remains unchanged. After updating all overlapping voxels, the resulting map is a target 3D map reflecting the latest state of the current environment.

[0049] In one embodiment, the method further includes: Save the target 3D map as a map in a preset format.

[0050] The completed 3D map of the target is saved in a file format that can be called by the path planning module. The file format can be PGM or PCD. After the mapping task is completed, the robot can enter standby mode or switch to cleaning mode.

[0051] Reference Figure 2 The diagram shown is a functional module schematic of the mapping device 100 based on depth gauge and lidar of this application.

[0052] The mapping device 100 based on depth sensors and lidar described in this application is installed in an electronic device. Depending on its functions, the mapping device 100 includes an acquisition module 110, a conversion module 120, a fusion module 130, a compensation module 140, and a generation module 150. These modules can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, and are stored in the memory of the electronic device.

[0053] In this embodiment, the functions of each module / unit are as follows: Acquisition module 110: used to acquire inertial measurement unit data, depth gauge data and lidar data of the robot; Conversion module 120: used to convert the measurement unit data, the depth gauge data and the lidar data into robot posture information, depth information and target point cloud information respectively; Fusion module 130: used to fuse the posture information with the depth information to obtain the robot's three-dimensional position and posture information; Compensation module 140: used to perform rotation compensation operation on the target point cloud information based on the three-dimensional position and attitude information to obtain a local three-dimensional grid map; Generation module 150: used to perform a fusion operation between the local 3D raster map and the pre-built global map to obtain the target 3D map.

[0054] The specific implementation of the mapping device based on depth gauge and lidar in this application is largely the same as the specific implementation of the mapping method based on depth gauge and lidar described above, and will not be repeated here.

[0055] Reference Figure 3 The diagram shown is a schematic representation of a preferred embodiment of the electronic device of this application.

[0056] The electronic device includes a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. The memory 113 is used to store computer programs, such as mapping programs based on depth sensors and lidar; In some embodiments, the processor 111 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 111 is typically used to control the overall operation of the electronic device, such as performing data interaction or communication-related control and processing. In this embodiment, the processor 111 is used to run program code stored in the memory 113 or process data.

[0057] The communication interface 112 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The communication interface 112 may also be used to establish a communication connection between the electronic device and other electronic devices.

[0058] The memory 113 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 113 may be an internal storage unit of the electronic device, such as the hard disk or memory of the electronic device. In other embodiments, the memory 113 may also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. of the electronic device. Of course, the memory 113 may include both internal storage units and external storage devices of the electronic device. In this embodiment, the memory 113 is typically used to store the operating system and various computer programs installed on the electronic device, such as the program code of mapping programs based on depth sensors and lidar. In addition, the memory 113 can also be used to temporarily store various types of data that have been output or will be output.

[0059] Figure 3 Only an electronic device having a processor 111, a communication interface 112, a memory 113 and a communication bus 114 is shown. However, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0060] In one embodiment of this application, when the processor 111 executes the program stored in the memory 113, it implements the mapping method based on depth gauge and lidar provided in any of the foregoing method embodiments, including: Acquire data from the robot's inertial measurement unit, depth gauge, and lidar. The measurement unit data, the depth gauge data, and the lidar data are respectively converted into robot posture information, depth information, and target point cloud information; The posture information and the depth information are fused to obtain the robot's three-dimensional position and posture information; Based on the three-dimensional position and attitude information, a rotation compensation operation is performed on the target point cloud information to obtain a local three-dimensional raster map; The local 3D raster map is fused with the pre-built global map to obtain the target 3D map.

[0061] For a detailed explanation of the above steps, please refer to the above. Figure 1 A flowchart illustrating an embodiment of a mapping method based on depth gauges and lidar.

[0062] Furthermore, this application also proposes a computer-readable storage medium that is both non-volatile and volatile. This computer-readable storage medium is any one or any combination of several of the following: hard disk, multimedia card, SD card, flash memory card, SMC, read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, etc. The computer-readable storage medium includes a data storage area and a program storage area. The program storage area stores a mapping program based on depth sensing and LiDAR. When the depth sensing and LiDAR mapping program is executed by a processor, it performs the following operations: Acquire data from the robot's inertial measurement unit, depth gauge, and lidar. The measurement unit data, the depth gauge data, and the lidar data are respectively converted into robot posture information, depth information, and target point cloud information; The posture information and the depth information are fused to obtain the robot's three-dimensional position and posture information; Based on the three-dimensional position and attitude information, a rotation compensation operation is performed on the target point cloud information to obtain a local three-dimensional raster map; The local 3D raster map is fused with the pre-built global map to obtain the target 3D map.

[0063] The specific implementation of the computer-readable storage medium in this application is largely the same as the specific implementation of the mapping method based on depth gauge and lidar described above, and will not be repeated here.

[0064] It should be noted that the sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0065] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware simulation platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device to execute the methods described in the various embodiments of this application.

[0066] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A mapping method based on depth measurement and lidar, characterized in that, The method includes: Acquire data from the robot's inertial measurement unit, depth gauge, and lidar. The measurement unit data, the depth gauge data, and the lidar data are respectively converted into robot posture information, depth information, and target point cloud information; The posture information and the depth information are fused to obtain the robot's three-dimensional position and posture information; Based on the three-dimensional position and attitude information, a rotation compensation operation is performed on the target point cloud information to obtain a local three-dimensional raster map; The local 3D raster map is fused with the pre-built global map to obtain the target 3D map.

2. The mapping method based on depth gauge and lidar as described in claim 1, characterized in that, The process of fusing the pose information with the depth information to obtain the robot's three-dimensional position and pose information includes: A coordinate transformation matrix is ​​constructed using the pitch and roll angles in the attitude information. Based on the coordinate transformation matrix, the depth information is rotated in the coordinate system to obtain the vertical depth component in the world coordinate system. Based on the yaw angle in the attitude information, determine the rotational disturbance value measured by the depth gauge caused by the robot's rotational motion; Based on the rotational disturbance value, the vertical depth component is compensated to obtain the compensated vertical depth component. The compensated vertical depth component is fused with the attitude angle in the attitude information to generate the robot's three-dimensional position and attitude information.

3. The mapping method based on depth gauge and lidar as described in claim 2, characterized in that, The step of determining the rotational disturbance value measured by the depth gauge due to the robot's rotational motion based on the yaw angle in the attitude information includes: The yaw angle in the attitude information is differentiated over time to obtain the instantaneous angular velocity of the robot about the vertical axis; Multiply the instantaneous angular velocity by the sampling period of the depth gauge to calculate the horizontal arc length displacement caused by yaw rotation within a single sampling interval; Based on the horizontal arc length displacement and the lateral offset distance from the depth gauge installation position to the robot's rotation center, a rotational disturbance geometric model is constructed. The rotational disturbance value is obtained by mapping the horizontal arc length displacement to an equivalent disturbance in the depth gauge measurement direction using the rotational disturbance geometric model.

4. The mapping method based on depth gauge and lidar as described in claim 2, characterized in that, The step of fusing the compensated vertical depth component with the attitude angle in the attitude information to generate the robot's three-dimensional position and attitude information includes: The compensated vertical depth component is used as the robot's Z-axis position coordinate in the world coordinate system. Combined with the preset initial horizontal position, an initial pose value containing the three-dimensional position is constructed. Based on the pitch, roll and yaw angles in the attitude information, a set of three-axis attitude angles is generated and aligned with the position portion of the initial pose value using timestamps. Based on the set of three-axis attitude angles, an orientation matrix of the robot body is constructed, and the orientation matrix and the Z-axis position coordinates are jointly encoded into a six-degree-of-freedom pose representation. Based on the attitude change rate at adjacent time points, Z-axis motion constraints are constructed, and the vertical position in the six-degree-of-freedom pose representation is checked and corrected for consistency, thereby generating the robot's three-dimensional position and pose information.

5. The mapping method based on depth gauge and lidar as described in claim 1, characterized in that, The step of performing rotation compensation on the target point cloud information based on the three-dimensional position and pose information to obtain a local three-dimensional raster map includes: The robot's attitude rotation matrix is ​​extracted from the three-dimensional position and attitude information. The attitude rotation matrix is ​​composed of pitch angle, roll angle and yaw angle in a preset rotation order. The attitude rotation matrix is ​​used to characterize the rotation transformation relationship from the world coordinate system to the robot coordinate system. The attitude rotation matrix is ​​transposed to obtain the point cloud compensation rotation matrix; The coordinates of each three-dimensional point in the target point cloud information are transformed using the point cloud compensation rotation matrix, and the transformed three-dimensional point is added to the position component in the three-dimensional position and attitude information to obtain the translated point cloud data. Within a preset spatial range centered on the location component, the translated point cloud data is voxelized at a preset voxel resolution to generate a local three-dimensional raster map.

6. The mapping method based on depth gauge and lidar as described in claim 1, characterized in that, The step of fusing the local 3D raster map with a pre-constructed global map to obtain the target 3D map includes: Based on the pose transformation parameters corresponding to the local 3D raster map, the local 3D raster map is transformed to the global coordinate system to obtain aligned local voxel data; Extract a subset of global voxels that spatially overlap with the aligned local voxel data from the pre-constructed global map; For voxels in the aligned local voxel data that have the same spatial coordinates as those in the global voxel subset, the occupancy probability, observation timestamp, and confidence level of the voxels with the same spatial coordinates are read respectively, and the weighted fusion occupancy probability is calculated. The fused occupancy probability is written into the corresponding voxel position in the pre-built global map to generate the target 3D map.

7. The mapping method based on depth gauge and lidar as described in claim 1, characterized in that, The method further includes: Save the target 3D map as a map in a preset format.

8. A mapping device based on depth gauge and lidar, characterized in that, The device includes: The apparatus includes a module that performs the mapping method based on depth gauges and lidar as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the mapping method based on depth gauge and lidar as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the mapping method based on depth measurement and lidar as described in any one of claims 1 to 7.

Images