Method for constructing environmental map, robot control device, apparatus, and storage medium

Abstract

The application discloses a method for constructing an environmental map, a robot control device, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring a first original point cloud through a sensor carried in a robot, and mapping the first original point cloud into a first global point cloud according to a first prior pose of the robot; determining geometric constraint features of a surrounding environment according to a first point cloud map and the first global point cloud; correcting the first global point cloud according to the geometric constraint features; and constructing a second point cloud map according to the corrected first global point cloud and the first point cloud map, wherein the second point cloud map is a map of the surrounding environment in a first sampling frame in a global coordinate system. According to the geometric constraint features, the first global point cloud in different degeneration degree scenes can be subjected to differential pose correction and map optimization, so that the positioning accuracy and map consistency of the robot in a geometric degeneration scene are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for constructing an environmental map, a robot control device, an equipment, and a storage medium. Background Technology

[0002] SLAM (simultaneous location and mapping) technology refers to the integrated technology by which a robot, in an unknown environment, acquires environmental data through its onboard sensors and completes real-time localization and map construction of the surrounding environment without needing to provide map information in advance. SLAM technology is the core foundation for autonomous mobile robots to achieve intelligent navigation.

[0003] In related technologies, SLAM methods mostly rely on geometric features in the surrounding environment. For example, LOAM (LidarOdometry and Mapping) extracts corner points and planar features from the surrounding environment for inter-frame matching, and combines motion estimation to solve for its own pose and update the map; or PP-ICP (Point-to-Plane Iterative Closest Point), PL-ICP (Point-to-Line Iterative Closest Point), NDT (Normal Distributions Transform), etc., through iterative optimization, minimize the distance error between point clouds in different frames to achieve dynamic map updates.

[0004] However, when the above methods are applied to degraded scenarios such as corridors and tunnels where geometric features are degraded, problems such as severe pose estimation drift, decreased point cloud registration accuracy, and easy distortion in map construction occur due to the lack of rich geometric features in the surrounding environment. Summary of the Invention

[0005] This application provides a method for constructing an environmental map, a robot control device, an equipment, and a storage medium to solve the technical problems existing in related technologies. It includes the following technical solutions: In a first aspect, this application provides a method for constructing an environmental map, the method comprising: acquiring a first original point cloud through sensors mounted on a robot, and mapping the first original point cloud to a first global point cloud based on a first prior pose of the robot, wherein the first original point cloud indicates the positional relationship between the robot and the surrounding environment in the sensor coordinate system at a first sampling time, the first prior pose is the estimated pose of the robot in the global coordinate system at the first sampling time, and the first global point cloud indicates the positional relationship between the robot and the surrounding environment in the global coordinate system at the first sampling time; determining geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud, wherein the first point cloud map is a map of the surrounding environment in the global coordinate system at a second sampling time, and the second sampling frame is the previous sampling time of the first sampling time; correcting the first global point cloud based on the geometric constraint features; and constructing a second point cloud map based on the corrected first global point cloud and the first point cloud map, wherein the second point cloud map is a map of the surrounding environment in the global coordinate system at the first sampling time.

[0006] In some possible implementations, the first global point cloud includes a plurality of first spatial points, the first point cloud map includes a plurality of local planes, and the step of determining the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud includes: determining a plurality of target local planes that correspond one-to-one with the plurality of first spatial points among the plurality of local planes; and determining the geometric constraint features based on the plane normal vectors of the plurality of target local planes.

[0007] In some possible implementations, the plurality of target local planes includes effective local planes, and determining the plurality of target local planes that correspond one-to-one with the plurality of first spatial points includes: filtering effective spatial points among the plurality of first spatial points according to plane fitting conditions; and determining the effective local planes that correspond to the effective points among the plurality of local planes.

[0008] In some possible implementations, the first global point cloud includes first geometrical spatial information and first point cloud feature information. The step of correcting the first global point cloud based on the geometrical constraint features includes: if the geometrical constraint features satisfy a degradation condition, then correcting the first global point cloud based on the first point cloud map, the first geometrical spatial information, the first point cloud feature information, and the first prior pose; if the geometrical constraint features do not satisfy the degradation condition, then correcting the first global point cloud based on the first point cloud map, the first geometrical spatial information, and the first prior pose.

[0009] In some possible implementations, correcting the first global point cloud based on the first point cloud map, the first geometrical space information, the first point cloud feature information, and the first prior pose includes: determining a first residual based on the first point cloud map and the first geometrical space information, the first residual indicating the geometric deviation between the first global point cloud and the first point cloud map; determining a second residual based on the first point cloud map and the first point cloud feature information, the second residual indicating the point cloud feature deviation between the first global point cloud and the first point cloud map; optimizing the first prior pose based on the first residual and the second residual; and remapping the first original point cloud to the global coordinate system based on the optimized first prior pose to obtain the corrected first global point cloud.

[0010] In some possible implementations, optimizing the first prior pose based on the first residual and the second residual includes: obtaining a first weighting coefficient for the first residual and the second residual, wherein the first weighting coefficient is used to adjust the contribution of the first residual and the second residual in the optimization process of the first prior pose; and optimizing the first prior pose based on the first weighting coefficient, the first residual, and the second residual.

[0011] In some possible implementations, mapping the first original point cloud to a first global point cloud based on the robot's first prior pose includes: downsampling the first original point cloud if the first original point cloud meets the downsampling conditions; and mapping the downsampled first original point cloud to a first global point cloud based on the first prior pose.

[0012] In a second aspect, this application provides a robot control device, the device comprising an acquisition module, a determination module, a correction module, and a map generation module; the acquisition module is configured to acquire a first raw point cloud through sensors mounted on the robot, and map the first raw point cloud to a first global point cloud based on a first prior position of the robot, wherein the first raw point cloud indicates the positional relationship between the robot and the surrounding environment in the sensor coordinate system at a first sampling time, the first prior position indicates the estimated pose of the robot in the global coordinate system at the first sampling time, and the first global point cloud indicates the positional relationship between the robot and the surrounding environment at the first sampling frame. The system describes the positional relationship in the global coordinate system; the determining module is configured to determine the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud, wherein the first point cloud map is a map of the surrounding environment in the global coordinate system at the second sampling time, and the second sampling frame is the previous sampling frame of the first sampling frame; the correcting module is configured to correct the first global point cloud based on the geometric constraint features; the map generating module is configured to construct a second point cloud map based on the corrected first global point cloud and the first point cloud map, wherein the second point cloud map is a map of the surrounding environment in the global coordinate system at the first sampling time.

[0013] In some possible implementations, the first global point cloud includes a plurality of first spatial points, the first point cloud map includes a plurality of local planes, and the determining module is configured to: determine a plurality of target local planes that correspond one-to-one with the plurality of first spatial points in the plurality of local planes; and determine the geometric constraint features based on the plane normal vectors of the plurality of target local planes.

[0014] In some possible implementations, the plurality of target local planes includes effective local planes, and the determining module is configured to: filter effective spatial points among the plurality of first spatial points according to plane fitting conditions; and determine the effective local plane among the plurality of local planes corresponding to the effective points.

[0015] In some possible implementations, the first global point cloud includes first geometrical spatial information and first point cloud feature information. The correction module is configured to: if the geometrical constraint features satisfy the degradation condition, then correct the first global point cloud according to the first point cloud map, the first geometrical spatial information, the first point cloud feature information, and the first prior pose; if the geometrical constraint features do not satisfy the degradation condition, then correct the first global point cloud according to the first point cloud map, the first geometrical spatial information, and the first prior pose.

[0016] In some possible implementations, the correction module is configured to: determine a first residual based on the first point cloud map and the first geometrical information, the first residual indicating the geometrical deviation between the first global point cloud and the first point cloud map; determine a second residual based on the first point cloud map and the first point cloud feature information, the second residual indicating the point cloud feature deviation between the first global point cloud and the first point cloud map; optimize the first prior pose based on the first residual and the second residual; and remap the first original point cloud to the global coordinate system based on the optimized first prior pose to obtain the corrected first global point cloud.

[0017] In some possible implementations, the correction module is configured to: obtain a first weighting coefficient for the first residual and the second residual, the first weighting coefficient being used to adjust the contribution of the first residual and the second residual in the optimization process of the first prior pose; and optimize the first prior pose based on the first weighting coefficient, the first residual, and the second residual.

[0018] In some possible implementations, the acquisition module is configured to: downsample the first original point cloud if the first original point cloud meets the downsampling conditions; and map the downsampled first original point cloud to a first global point cloud based on the first prior pose.

[0019] In a third aspect, this application provides an electronic device for constructing an environmental map, comprising: a memory storing at least one program instruction for constructing an environmental map; and a processor, wherein when the program instruction is executed by the processor, the electronic device implements the method of the first aspect of this application or any possible embodiment of the first aspect.

[0020] In a fourth aspect, this application provides a computer program (product) comprising computer program / instructions that are executed by a processor to enable a computer to implement the method of the first aspect of this application or any possible implementation thereof.

[0021] In a fifth aspect, this application provides a computer-readable storage medium having stored thereon program instructions for constructing an environment map, which, when executed by one or more processors, cause a computer to implement the method of the first aspect or any possible implementation of the first aspect.

[0022] The beneficial effects of the technical solution provided in this application include at least the following: The technical solution provided in this application embodiment, by using the first global point cloud of the robot in the first sampling frame and the first point cloud map of the previous sampling frame, can determine the geometric constraint features used to characterize the constraint strength of the surrounding environment, and thus determine the degree of geometric degradation of the surrounding environment. Based on this, this application embodiment can perform differentiated pose correction and map optimization on the first global point cloud under different degradation scenarios according to the geometric constraint features, thereby improving the robot's positioning accuracy and ground motion in geometrically degraded scenarios. Figure 1 Consistency is beneficial for suppressing pose drift and enhancing the stability and robustness of SLAM systems in environments with weak geometric features. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an implementation scenario provided in the embodiments of this application; Figure 2 This is a flowchart of the environmental map construction method provided in the embodiments of this application; Figure 3 This is a flowchart of the degradation scene identification method provided in the embodiments of this application; Figure 4 This is a flowchart of the point cloud map update method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the robot control device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of an electronic device for constructing an environmental map provided in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] Figure 1 This is a schematic diagram of an implementation scenario provided in an embodiment of this application. (Reference) Figure 1 The implementation scenario provided in this application embodiment is, for example, a robot in a mapping and positioning scenario, or a mapping and positioning process. The robot provided in this application embodiment is, for example, equipped with a sensor 11 and a control unit 12.

[0028] The mapping and localization scenario of a robot is the process by which a robot continuously perceives and explores its surrounding environment through its onboard sensors in order to build a map of the surrounding environment in global space and determine its own pose.

[0029] Sensor 11 is used, but is not limited to, to collect point cloud data, pose observation data, and motion state data of the robot's surrounding environment.

[0030] Optionally, sensor 11 may include, for example, a lidar sensor for acquiring three-dimensional laser point clouds of the surrounding environment, providing a data source for point cloud matching, plane extraction, and geometric constraint calculation; an IMU (Inertial Measurement Unit) for measuring the robot's angular velocity and acceleration, assisting in motion estimation and initial pose prediction; and an odometer for outputting the robot's motion displacement and attitude changes, providing prior information for inter-frame pose recursion.

[0031] The control unit 12 communicates with the sensor 11 via wired or wireless means. It is used, but not limited to, to receive, parse and process various observation data collected by the sensor 11, and to perform operations such as point cloud registration, geometric constraint calculation, degradation scene determination and pose optimization, so that the control unit 12 can achieve stable laser SLAM positioning and map building based on the environmental perception data and motion data output by the sensor 11.

[0032] Those skilled in the art should understand that the above-described sensor 11 and control unit 12 are merely examples, and other existing or future sensors and control units that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0033] Figure 2 This is a flowchart of an environmental map construction method provided in an embodiment of this application. This method can, for example, be constructed by... Figure 1The control unit in the system executes this. See also Figure 2 The method for constructing an environmental map provided in this application embodiment may include steps S210-S240 as shown below.

[0034] Step S210: Obtain the first original point cloud through the sensors mounted on the robot, and map the first original point cloud into the first global point cloud according to the robot's first prior pose.

[0035] Among them, the first original point cloud indicates the positional relationship between the robot and the surrounding environment in the sensor coordinate system at the first sampling time, the first prior pose is the estimated pose of the robot in the global coordinate system at the first sampling time, and the first global point cloud indicates the positional relationship between the robot and the surrounding environment in the global coordinate system at the first sampling time.

[0036] For example, the sensors mounted on the robot include, for instance, LiDAR, depth cameras, 3D vision sensors, or any other type of sensor capable of acquiring 3D point cloud data of the environment. The robot's mapping and localization process is a continuous process involving multiple sampling frames, each corresponding to a different sampling time. The sampling frequency of the multiple sampling frames can be adjusted according to the actual application scenario, and this application does not impose any limitations in this regard.

[0037] Based on this, the original point cloud refers to the set of three-dimensional points obtained by the robot at any sampling time after sensing and detecting the surrounding environment using LiDAR or other types of sensors for three-dimensional environmental perception. For example, the first original point cloud collected at the first sampling time (hereinafter referred to as the first time).

[0038] The sensor coordinate system refers to a local reference coordinate system with the sensor's detection center as the origin, which changes synchronously with the robot's pose. The raw point cloud acquired by the sensors on the robot is a set of three-dimensional points obtained by the robot at any sampling moment, with the sensor's detection center as the origin, after perceiving and detecting the surrounding environment. In other words, it represents the perceived positional relationship between the robot and its surroundings from the robot's perspective, i.e., the spatial distribution between the robot and its environment. Because the origin of the sensor coordinate system changes with the robot's pose, the positional relationship described by the raw point cloud is a relative positional relationship. For example, the positional relationship indicated by the first raw point cloud is the relative positional relationship between the robot and its surroundings in the sensor coordinate system at the first moment.

[0039] A global coordinate system, for example, is a reference coordinate system with a fixed origin during robot mapping and localization. It is used, but not limited to, to provide a unified spatial reference for environmental perception, robot pose, and the global map. For instance, at the starting point of the robot mapping process, this starting point is used as the origin of the global coordinate system, and this origin remains unchanged once initialized. The position of the origin of the global coordinate system can be adjusted according to the actual application scenario, and this application does not impose any restrictions in this regard.

[0040] The prior pose is a prediction of the robot's position and orientation in the global coordinate system at any given sampling time. It is used, but is not limited to, to indicate the translational and / or rotational components of the robot in the global coordinate system between any given sampling time and the previous sampling time. It can serve as a coordinate transformation basis connecting the sensor coordinate system and the global coordinate system, mapping the original point cloud in the sensor coordinate system to the global point cloud in the global coordinate system. For example, the first prior pose is the predicted pose of the robot in the global coordinate system at a first moment, used to map the first original point cloud to the first global point cloud.

[0041] Based on this, the global point cloud obtained from the mapping of the original point cloud and the prior pose refers to the set of three-dimensional points obtained after transforming the original point cloud in the sensor coordinate system to the global coordinate system through the translation and / or rotation components indicated by the prior pose, i.e., the indicated coordinate transformation relationship. This set is used to characterize the absolute spatial distribution of the robot and its surrounding environment under a globally unified reference, that is, the absolute positional relationship between the robot and its surrounding environment in the global coordinate system. For example, the positional relationship indicated by the first global point cloud is the absolute positional relationship between the robot and its surrounding environment in the global coordinate system at the time corresponding to the first sampling frame.

[0042] In some embodiments, a method for mapping a first raw point cloud to a first global point cloud based on a first prior pose of the robot includes, for example, determining a first transformation matrix based on the first prior pose, and mapping the first raw point cloud to the first global point cloud based on the first transformation matrix. The first transformation matrix is, for example, a coordinate transformation matrix containing rotational and / or translational components, used to indicate the spatial transformation relationship from the sensor coordinate system to the global coordinate system.

[0043] Optionally, the method for determining the robot's first prior pose is, for example, to recursively estimate the first prior pose by combining the robot's prior pose from the previous sampling time to the first sampling time with the motion increment between the previous sampling time and the first sampling time. The motion increment between the previous sampling time and the first sampling time can be obtained, for example, from changes in the robot's position and attitude acquired by sensors such as the inertial measurement unit and odometry mounted on the robot.

[0044] If the previous sampling time is the initial sampling time among multiple sampling times, then the origin of the global coordinate system is taken as the prior position of the previous sampling time.

[0045] Step S220: Determine the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud. The first point cloud map is the map of the surrounding environment in the global coordinate system at the second sampling time. The second sampling time is the sampling time before the first sampling time.

[0046] For example, the first sampling time is any one of multiple sampling times, and the second sampling frame is the previous sampling time that is located before and adjacent to the first sampling time among the multiple sampling frames. Accordingly, the first point cloud map is the point cloud map constructed at the second sampling time (hereinafter referred to as the second time); the method of constructing the first point cloud map is the same as the method of constructing the point cloud map at the current time, i.e., the second point cloud map.

[0047] Geometric constraint features refer to the distribution characteristics of geometric constraints imposed on the robot's pose by the surrounding environment within the global space corresponding to the global coordinate system. Specifically, geometric constraints are the spatial structures in the surrounding environment that restrict changes in the robot's pose and hinder its penetrating movement.

[0048] For example, in a global coordinate system with the vertical direction of the robot's movement (hereinafter referred to as the lateral direction) as the X-axis, the parallel direction of the robot's movement (hereinafter referred to as the longitudinal direction) as the Y-axis, and the direction perpendicular to the horizontal plane (hereinafter referred to as the vertical direction) as the Z-axis, the planar structures in the surrounding environment that the robot cannot pass through in the corresponding global spatial directions (X-axis, Y-axis, and Z-axis) constitute the geometric constraints that hinder the robot's movement. The more spatial dimensions the planar structures cover and the more evenly the planar structures are distributed in each direction, the higher the sufficiency of the geometric constraints in the surrounding environment; conversely, the lower the sufficiency of the geometric constraints in the surrounding environment.

[0049] For example, when the robot is in a closed room, the surrounding environment has effective planar structures along the X, Y, and Z axes in global space, indicating a high degree of geometric constraint sufficiency. However, when the robot is in a tunnel and moves along its extension direction, the surrounding environment has effective planar structures along the X and Z axes in global space, but lacks effective planar structures along the Y axis, indicating a low degree of geometric constraint sufficiency.

[0050] As mentioned earlier, the geometric constraints of the surrounding environment refer to the geometric constraints existing in the global space and their distribution. For example, the planar structures in the global space and their distribution.

[0051] Based on this, if the first global point cloud includes multiple first spatial points and the first point cloud map includes multiple local planes, a method for determining the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud may include, for example, determining multiple target local planes that correspond one-to-one with the multiple first spatial points in the multiple local planes; and determining the geometric constraint features based on the plane normal vectors of the multiple target local planes.

[0052] For example, the point cloud map is a set of three-dimensional spatial points, on which any one of the multiple local planes is fitted, for example, by at least three non-collinear points from a plurality of second spatial points of the first point cloud map.

[0053] In this case, a method for determining multiple target local planes that correspond one-to-one with multiple first spatial points in multiple local planes is, for example, to determine at least three first target points among multiple second spatial points that are closest to each first spatial point and are not collinear; and to obtain multiple target local planes corresponding to multiple first spatial points by fitting the target local planes corresponding to each first spatial point based on the first target points.

[0054] In some embodiments, the plane normal vectors include multiple planes that correspond one-to-one with multiple target local planes. A method for determining the geometric constraint features of the surrounding environment based on the plane normal vectors of the multiple target local planes is, for example, to calculate the cross product of any plane normal vector among the multiple plane normal vectors; and to construct a constraint matrix based on the cross product of the multiple plane normal vectors; to perform eigenvalue decomposition on the constraint matrix to obtain multiple eigenvalues, and to take the smallest eigenvalue among the multiple eigenvalues ​​as the geometric constraint feature of the surrounding environment.

[0055] The constraint matrix is ​​a multi-dimensional matrix used to characterize the strength distribution of geometric constraints in various directions of the surrounding environment within the global space. The dimension of the constraint matrix is ​​the same as the robot's pose degrees of freedom and the plane normal vector. For example, if the plane normal vector is a three-dimensional vector (corresponding to the robot's three-dimensional pose degrees of freedom in the X, Y, and Z directions of the global coordinate system), then the constraint matrix is ​​also a three-dimensional matrix. Multiple eigenvalues ​​are used to indicate the strength of the geometric constraints corresponding to different degrees of freedom in the global space. Larger eigenvalues ​​indicate stronger constraints in the corresponding direction, while smaller eigenvalues ​​indicate weaker constraints.

[0056] In the above method, by matching and associating each first spatial point in the first global point cloud with the first point cloud map, the plane where each first spatial point is located and the corresponding plane normal vector can be determined based on the neighboring spatial points in the map, thereby determining the planar structure and the distribution of the planar structure in the surrounding environment in the first moment of global space.

[0057] Considering that in practical applications, the first global point cloud may contain outliers, noise points, and other invalid spatial points, these invalid spatial points can interfere with the identification of the surrounding planar structure and the calculation of geometric constraint features, leading to deviations in the subsequent determination of degraded scenarios. Therefore, the method provided in this application can filter multiple first spatial points through configured planar fitting conditions to eliminate invalid spatial points and retain valid spatial points that can be used for subsequent planar structure identification, planar normal vector calculation, and geometric constraint feature determination, ensuring the accuracy and reliability of identifying whether the surrounding environment is a degraded scenario. Here, a degraded scenario refers to a scenario where the surrounding environment has effective geometric constraints in a certain spatial dimension. For example, the surrounding environment may only have a single or unidirectional planar structure, such as a long corridor, tunnel, narrow corridor, or open flat area that cannot provide sufficient multi-directional geometric constraints for robot pose solving.

[0058] In some possible cases, multiple target local planes include effective local planes. Determining multiple target local planes that correspond one-to-one with multiple first spatial points includes: filtering effective spatial points among multiple first spatial points based on plane fitting conditions; and determining effective local planes that correspond to the effective spatial points among multiple local planes.

[0059] In some embodiments, the plane fitting condition is, for example, whether the first distance from any of the plurality of first spatial points to the corresponding target local plane reaches a first specified distance threshold.

[0060] For example, if the first distance from any first spatial point to the corresponding target local plane reaches a first specified distance threshold, it indicates that there is no target local plane corresponding to that first spatial point among the multiple local planes. That is, the first spatial point does not fall into any of the multiple local planes and cannot be used for the identification of the planar structure of the surrounding environment and the calculation of geometric constraint features. Therefore, the first spatial point is an invalid spatial point. If the first distance from any first spatial point to the corresponding target local plane does not reach the first specified distance threshold, it indicates that there is a target local plane corresponding to that first spatial point among the multiple local planes. That is, the first spatial point falls into any of the multiple local planes and can be used for the identification of the planar structure of the surrounding environment and the calculation of geometric constraint features. Therefore, the first spatial point is a valid spatial point. The value of the first specified distance threshold can be adjusted according to the actual application scenario, and this application does not impose any restrictions in this regard.

[0061] During the initial mapping phase of the system, the robot has not yet built a historical global map. At this time, environmental degradation judgment is not performed, and the global map is directly initialized with the currently perceived global point cloud. In this case, the method for building an environmental map by the robot provided in this application includes: if the first sampling time is the initial sampling time among multiple sampling times, building a second point cloud map based on the first global point cloud.

[0062] Step S230: Correct the first global point cloud based on geometric constraint features.

[0063] In some embodiments, the first global point cloud includes first geometrical spatial information and first point cloud feature information. Correcting the first global point cloud based on geometrical constraint features includes: if the geometrical constraint features satisfy a degradation condition, then correcting the first global point cloud based on the first point cloud map, the first geometrical spatial information, the first point cloud feature information, and the first prior pose; if the geometrical constraint features do not satisfy a degradation condition, then correcting the first global point cloud based on the first point cloud map, the first geometrical spatial information, and the first prior pose.

[0064] For example, geometric spatial information refers to the description information of the three-dimensional pose of each first spatial point in the first global point cloud in the global coordinate system, such as multiple coordinates that correspond one-to-one with multiple first spatial points in the first global point cloud.

[0065] Point cloud feature information refers to the sensor's perception feature information corresponding to each first spatial point, used to help distinguish the material, surface properties, and other information of different environmental points in the surrounding environment. For example, if the first original point cloud is a point cloud acquired by LiDAR, then the point cloud feature information consists of multiple reflection intensities corresponding one-to-one with multiple first spatial points. If the sensor is a millimeter-wave radar, then the point cloud feature information consists of RCS (Radar Cross Section) or echo amplitude information corresponding one-to-one with multiple first spatial points. If the sensor is a vision-based depth sensor, then the point cloud feature information consists of grayscale values, color values, or texture feature information corresponding one-to-one with multiple first spatial points.

[0066] Degradation features are, for example, pre-configured conditions used to determine whether the surrounding environment is a degrading scenario. In some embodiments, if the geometric constraint feature of the surrounding environment is the minimum eigenvalue of the constraint matrix, the degradation condition is, for example, that the minimum eigenvalue is less than a specified eigenvalue threshold. The value of the specified eigenvalue threshold can be adjusted according to the actual application scenario, and this application does not impose any restrictions in this regard.

[0067] In this scenario, if the minimum eigenvalue of the constraint matrix is ​​less than a specified eigenvalue threshold, it indicates that at the time corresponding to the first sampling frame, the robot's surrounding environment is a scene with low geometric constraint sufficiency, i.e., a degenerate scene with weak geometric constraints on the robot. In this case, if the robot relies solely on geometric spatial information to correct the map, deviations are likely to occur. Due to the ease with which pose prediction can drift, the global map will exhibit significant bias. Therefore, when correcting the first global map, combining both the first geometric spatial information and the first point cloud feature information can improve the accuracy and stability of map correction, suppress map deviations caused by pose drift, and ensure the consistency of the global map.

[0068] If the minimum eigenvalue of the constraint matrix is ​​greater than or equal to a specified eigenvalue threshold, it indicates that at the time corresponding to the first sampling frame, the robot's surrounding environment is a scene with a high degree of geometric constraint, i.e., a non-degenerate scene with strong geometric constraints on the robot. At this time, the robot's pose prediction accuracy is high, and the geometric spatial information can meet the map correction requirements. Therefore, when correcting the first global map, only the first geometric spatial information is considered, which simplifies the calculation process, reduces computational overhead, and improves map update efficiency while ensuring map correction accuracy.

[0069] In some embodiments, correcting the first global point cloud based on a first point cloud map, first geometrical spatial information, first point cloud feature information, and a first prior pose includes: determining a first residual based on the first point cloud map and the first geometrical spatial information, the first residual indicating the geometric deviation between the first global point cloud and the first point cloud map; determining a second residual based on the first point cloud map and the first point cloud feature information, the second residual indicating the point cloud feature deviation between the first global point cloud and the first point cloud map; optimizing the first prior pose based on the first residual and the second residual; and remapping the first original point cloud to the global coordinate system based on the optimized first prior pose to obtain the corrected first global point cloud.

[0070] The first residual indicates the geometric deviation between the first global point cloud and the first point cloud map, such as the distance deviation from each first spatial point in the first global point cloud to the corresponding target local plane in the second point cloud map, or the deviation of the angle between the plane normal vectors, etc., representing geometric constraint deviations. The second residual indicates the point cloud feature deviation between the first global point cloud and the first point cloud map, such as the reflection intensity deviation between each first spatial point and the corresponding target local plane. The method for optimizing the first prior pose based on the first and second residuals is, for example, to construct a joint optimization objective function containing the first and second residual constraint terms, and then, by minimizing the weighted sum of squares of the first and second residuals, to obtain the optimal first prior pose, thus achieving the optimized update of the first prior pose.

[0071] In some embodiments, the first point cloud map includes multiple local planes, and any one of the multiple local planes is fitted by at least three non-collinear second spatial points from multiple second spatial points corresponding to the first point cloud map. The first geometrical spatial information includes multiple first coordinates that correspond one-to-one with the multiple first spatial points. The method for determining the first residual based on the first point cloud map and the first geometrical spatial information is, for example, as follows: for each first spatial point, substitute the first coordinate of each first spatial point into the plane equation of the corresponding target local plane, calculate the distance from the first spatial point to the corresponding target local plane, and determine the first residual as the sum or weighted sum of the distances from all first spatial points to the corresponding target local plane.

[0072] In other embodiments, the first point cloud map includes multiple local planes, each of which is fitted by at least three non-collinear second spatial points. The first point cloud feature information includes multiple first feature information corresponding one-to-one with the multiple first spatial points. A method for determining the second residual based on the first point cloud map and the first point cloud feature information includes, for example, finding a target feature point in the first point cloud map that matches each first spatial point for each first spatial point, calculating the feature difference between the first feature information of each first spatial point and the second feature information of the corresponding target feature point, and determining the second residual based on the feature differences corresponding to the multiple first spatial points. The target feature point is, for example, at least three second spatial points fitted to the target local plane corresponding to each first spatial point, and the second feature information of the target feature point is, for example, the average value of the feature information of at least three non-collinear second spatial points.

[0073] Optionally, the method for determining the second residual based on the feature differences corresponding to multiple first spatial points can be, for example, by determining the second residual based on the second weighting coefficients of the multiple first spatial points and the feature differences corresponding to the multiple first spatial points. For example, the feature differences corresponding to the multiple first spatial points can be weighted and summed based on the second weighting coefficients of the multiple first spatial points. The second weighting coefficients are used to adjust the contribution of the multiple first spatial points to the second residual.

[0074] Optionally, optimizing the first prior pose based on the first residual and the second residual includes: obtaining a first weighting coefficient for the first residual and the second residual, wherein the first weighting coefficient is used to adjust the contribution of the first residual and the second residual in the optimization process of the first prior pose; and optimizing the first prior pose based on the first weighting coefficient, the first residual, and the second residual.

[0075] In some embodiments, the first weighting coefficient is determined, for example, based on the degree of degradation of the surrounding environment at the current moment indicated by the geometric constraint features and the degradation condition. For example, if the geometric constraint feature is the minimum eigenvalue of the constraint matrix and the degradation condition is a specified eigenvalue threshold, then the first weighting coefficient is determined based on the ratio of the minimum eigenvalue to the specified eigenvalue threshold.

[0076] For example, the smaller the ratio of the minimum eigenvalue to the specified eigenvalue threshold, the higher the degree of degradation of the surrounding environment at the current moment, such as the first moment, and the more point cloud feature information is needed to assist in correcting the pose and the global point cloud. The larger the ratio of the minimum eigenvalue to the specified eigenvalue threshold, the lower the degree of degradation of the surrounding environment at the current moment, and reliable pose optimization and point cloud correction can be completed mainly with geometric spatial information.

[0077] Step S240: Construct a second point cloud map based on the corrected first global point cloud and the first point cloud map. The second point cloud map is a map of the surrounding environment in the global coordinate system at the first sampling time.

[0078] As mentioned earlier, the method for constructing a second point cloud map based on the corrected first global point cloud and the first point cloud map is, for example, to fuse the corrected first global point cloud of the current frame and overlay it onto the first point cloud map to generate the latest global map after iteration, which is the second point cloud map.

[0079] Based on this, the point cloud map at any sampling time is the result of incremental fusion and iterative update of the point cloud maps at all sampling times before that sampling time. It fully contains all environmental spatial information from the initial sampling time to that sampling time, integrates the global point cloud after correction at each sampling time and the corresponding planar structure and geometric constraint features, realizes the continuous iteration and dynamic update of the point cloud map with the robot's sampling frames, and ensures that the map can reflect the complete three-dimensional structure of the surrounding environment in real time and accurately.

[0080] In some embodiments, a method for mapping a first original point cloud to a first global point cloud based on a robot's first prior pose includes: downsampling the first original point cloud if the first original point cloud satisfies a downsampling condition; and mapping the downsampled first original point cloud to a first global point cloud based on the first prior pose.

[0081] Optionally, the method for downsampling the first original point cloud is, for example, voxel grid downsampling. The downsampling condition is, for example, that the point cloud density of the first original point cloud reaches a specified density threshold. The value of the specified density threshold can be adjusted according to the actual application scenario, and this application does not impose any restrictions in this regard.

[0082] The following will combine Figure 3 and Figure 4The method for constructing the environmental map provided in the embodiments of this application will be further described.

[0083] Figure 3 This is a flowchart of the degradation scene identification method provided in the embodiments of this application.

[0084] Figure 4 This is a flowchart of the point cloud map update method provided in the embodiments of this application.

[0085] like Figure 3 As shown in the embodiment of this application, in the degradation scene recognition method, after the degradation scene recognition is started, if the point cloud density of the first original point cloud reaches a specified density threshold, the first original point cloud is downsampled to reduce the amount of point cloud data, reduce computational redundancy, and improve subsequent computational efficiency.

[0086] Then, based on the robot's first prior pose at the current moment, the downsampled first original point cloud is mapped and converted into the first global point cloud in the global coordinate system, thus completing the global spatial alignment of the single-frame point cloud.

[0087] Then, each first spatial point in the first global point cloud is traversed, and validity checks and matches are performed point by point to determine the valid spatial points among the multiple first spatial points. For example, this includes: a first check to determine whether there are at least three second spatial points in the first point cloud map at the previous sampling time that are adjacent to the current first spatial point and not collinear, such as whether there are at least three second spatial points whose second distance to the current first spatial point is less than a second specified distance threshold. A second check to determine whether the first distance between the first spatial point and the corresponding target local plane is less than the first specified distance threshold.

[0088] In this case, the plane fitting condition is, for example, whether the first distance from any of the multiple first spatial points to the corresponding target local plane reaches a first specified distance threshold; or, for each first spatial point, whether there exists a neighboring point of any of the multiple second spatial points. If there is a neighboring point of any first spatial point, then that first spatial point is a valid spatial point; otherwise, it is an invalid spatial point. Here, a neighboring point is used to indicate a second spatial point whose second distance from any first spatial point is less than a second distance threshold.

[0089] If the first check or any of the checks in the first check is not satisfied, then skip the current first spatial point.

[0090] Based on all valid spatial points obtained after the first and second verifications and their corresponding target local planes, a constraint matrix of the surrounding environment is constructed, and the minimum eigenvalue of the constraint matrix is ​​solved to quantitatively characterize the sufficiency of the overall geometric constraints of the current environment.

[0091] If the minimum eigenvalue is less than the specified eigenvalue threshold, it is determined that the system has entered a degradation scenario; if the minimum eigenvalue is not less than the eigenvalue threshold, it is determined that the system has not entered a degradation scenario.

[0092] like Figure 4 As shown, in the point cloud map update method provided in this application embodiment, the method for determining geometric constraint features is similar to... Figure 3 The same applies, and will not be repeated here. The point cloud map update method provided in this application embodiment includes, for example, completing... Figure 3 After identifying the degradation scene shown, it is determined whether to enter the degradation scene based on whether the geometric constraint features meet the preset degradation conditions.

[0093] If it is determined that the robot is currently in a degenerate scenario, such as a corridor, tunnel, or narrow passage where the sufficiency of geometric constraints in a single direction is low, then a two-dimensional joint constraint correction mechanism is entered: the first residual is calculated by combining the first geometric spatial information of the historical first point cloud map and the current first global point cloud, and the second residual is calculated by combining the first point cloud map and the first point cloud feature information carried by the current first global point cloud, such as the intensity of laser reflection.

[0094] Then, based on the first residual and the second residual, and using the first weighting coefficient, the first prior pose of the robot in the current frame is subjected to weighted joint optimization. Through the dual residual collaborative constraint, the drift error of the predicted pose is corrected.

[0095] After pose optimization is completed, based on the optimized first prior pose, the first original point cloud in the sensor coordinate system is remapped to the global coordinate system to obtain the corrected first global point cloud.

[0096] Then, the corrected first global point cloud is incrementally fused and stitched with the first point cloud map at the previous sampling time, and the second point cloud map at the current sampling time is generated iteratively to complete the real-time map update in the degradation scenario.

[0097] The technical solution provided in this application embodiment, by using the first global point cloud of the robot in the first sampling frame and the first point cloud map of the previous sampling frame, can determine the geometric constraint features used to characterize the constraint strength of the surrounding environment, and thus determine the degree of geometric degradation of the surrounding environment. Based on this, this application embodiment can perform differentiated pose correction and map optimization on the first global point cloud under different degradation scenarios according to the geometric constraint features, thereby improving the robot's positioning accuracy and ground motion in geometrically degraded scenarios. Figure 1 Consistency is beneficial for suppressing pose drift and enhancing the stability and robustness of SLAM systems in environments with weak geometric features.

[0098] In some other possible implementations, this application also provides a robot control device. Figure 5 This is a schematic diagram of the control structure of the robot provided in the embodiments of this application. See also: Figure 5 The robot control device provided in this application embodiment includes: an acquisition module 510, a determination module 520, a correction module 530, and a map generation module 540.

[0099] The acquisition module 510 is configured to acquire a first raw point cloud through sensors mounted on the robot, and map the first raw point cloud to a first global point cloud based on the robot's first prior position. The first raw point cloud indicates the positional relationship between the robot and the surrounding environment in the sensor coordinate system at the first sampling time, the first prior position indicates the estimated pose of the robot in the global coordinate system at the first sampling time, and the first global point cloud indicates the positional relationship between the robot and the surrounding environment in the global coordinate system at the first sampling frame.

[0100] The determination module 520 is configured to determine the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud. The first point cloud map is a map of the surrounding environment in the global coordinate system at the second sampling time, and the second sampling frame is the previous sampling frame of the first sampling frame.

[0101] The correction module 530 is configured to correct the first global point cloud based on geometric constraint features.

[0102] The map generation module 540 is configured to construct a second point cloud map based on the corrected first global point cloud and the first point cloud map. The second point cloud map is a map of the surrounding environment in the global coordinate system at the first sampling time.

[0103] In some possible implementations, the first global point cloud includes multiple first spatial points, the first point cloud map includes multiple local planes, and the determination module 520 is configured to: determine multiple target local planes that correspond one-to-one with the multiple first spatial points in the multiple local planes; and determine geometric constraint features based on the plane normal vectors of the multiple target local planes.

[0104] In some possible implementations, the plurality of target local planes include effective local planes, and the determining module 520 is configured to: filter effective spatial points among a plurality of first spatial points according to plane fitting conditions; and determine the effective local planes among the plurality of local planes corresponding to the effective points.

[0105] In some possible implementations, the first global point cloud includes first geometrical spatial information and first point cloud feature information. The correction module 530 is configured to: if the geometrical constraint features meet the degradation conditions, then correct the first global point cloud according to the first point cloud map, the first geometrical spatial information, the first point cloud feature information, and the first prior pose; if the geometrical constraint features do not meet the degradation conditions, then correct the first global point cloud according to the first point cloud map, the first geometrical spatial information, and the first prior pose.

[0106] In some possible implementations, the correction module 530 is configured to: determine a first residual based on a first point cloud map and first geometrical information, the first residual indicating the geometric deviation between the first global point cloud and the first point cloud map; determine a second residual based on the first point cloud map and first point cloud feature information, the second residual indicating the point cloud feature deviation between the first global point cloud and the first point cloud map; optimize a first prior pose based on the first and second residuals; and remap the first original point cloud to the global coordinate system based on the optimized first prior pose to obtain a corrected first global point cloud.

[0107] In some possible implementations, the correction module 530 is configured to: obtain a first weighting coefficient for the first residual and the second residual, the first weighting coefficient being used to adjust the contribution of the first residual and the second residual in the optimization process of the first prior pose; and optimize the first prior pose based on the first weighting coefficient, the first residual, and the second residual.

[0108] In some possible implementations, the acquisition module 510 is configured to: downsample the first original point cloud if the first original point cloud meets the downsampling conditions; and map the downsampled first original point cloud to a first global point cloud according to the first prior pose.

[0109] It should be understood that the robot control device and the environmental map construction method provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the environmental map construction method embodiments.

[0110] In some other possible implementations, this application also provides an electronic device for constructing environmental maps. Figure 6 This is a schematic diagram of the structure of an electronic device for constructing an environmental map provided in an embodiment of this application. See also... Figure 6 The electronic device for constructing an environmental map provided in this application embodiment includes: The memory 610 stores at least one program instruction for constructing an environment map.

[0111] When the processor 620 executes the above program instructions, it enables the robot to achieve the above combination. Figure 2The steps of the described method and its various embodiments are described below. Depending on the implementation, the processor 420 may be one or more types of processors, including but not limited to DSP (digital signal processor), ASIC (application specific integrated circuit), FPGA (field-programmable gate array), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and the number of such devices may be determined according to actual needs.

[0112] In some other possible implementations, this application also provides a computer program (product) comprising computer programs / instructions, which are executed by a processor to enable the computer to perform the above-described combination. Figure 2 The steps of the described method and its various embodiments.

[0113] In some other possible implementations, this application also provides a computer-readable storage medium storing program instructions for constructing an environment map, which, when executed by one or more processors, implement the above-described combination. Figure 2 The steps of the described method and its various embodiments are described. The computer-readable storage medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical memory, magnetic memory, or any suitable combination thereof.

[0114] In the embodiments of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0115] The term "and / or" in the embodiments of this application is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0116] The above description is only for the purpose of enabling those skilled in the art to understand the technical solution of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for constructing an environmental map, characterized in that, The method includes: The robot acquires a first raw point cloud using sensors mounted on it, and maps the first raw point cloud to a first global point cloud based on the robot's first prior pose. The first raw point cloud indicates the positional relationship between the robot and the surrounding environment in the sensor coordinate system at the first sampling time. The first prior pose is the estimated pose of the robot in the global coordinate system at the first sampling time. The first global point cloud indicates the positional relationship between the robot and the surrounding environment in the global coordinate system at the first sampling time. The geometric constraint features of the surrounding environment are determined based on the first point cloud map and the first global point cloud. The first point cloud map is a map of the surrounding environment in the global coordinate system at the second sampling time, and the second sampling time is the sampling time before the first sampling time. The first global point cloud is corrected based on the geometric constraint features; A second point cloud map is constructed based on the corrected first global point cloud and the first point cloud map. The second point cloud map is a map of the surrounding environment in the global coordinate system at the first sampling time.

2. The method according to claim 1, characterized in that, The first global point cloud includes multiple first spatial points, and the first point cloud map includes multiple local planes. Determining the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud includes: Determine a plurality of target local planes that correspond one-to-one with the plurality of first spatial points among the plurality of local planes; The geometric constraint features are determined based on the plane normal vectors of the plurality of target local planes.

3. The method according to claim 2, characterized in that, The plurality of target local planes includes effective local planes, and determining the plurality of target local planes that correspond one-to-one with the plurality of first spatial points includes: Valid spatial points are selected from the plurality of first spatial points based on the plane fitting conditions; Determine the effective local plane that corresponds to the effective spatial point among the plurality of local planes.

4. The method according to claim 1, characterized in that, The first global point cloud includes first geometric spatial information and first point cloud feature information. The step of correcting the first global point cloud based on the geometric constraint features includes: If the geometric constraint features satisfy the degradation condition, then the first global point cloud is corrected according to the first point cloud map, the first geometric space information, the first point cloud feature information, and the first prior pose. If the geometric constraint features do not meet the degradation condition, the first global point cloud is corrected based on the first point cloud map, the first geometric space information, and the first prior pose.

5. The method according to claim 4, characterized in that, The step of correcting the first global point cloud based on the first point cloud map, the first geometric spatial information, the first point cloud feature information, and the first prior pose includes: A first residual is determined based on the first point cloud map and the first geometric space information. The first residual is used to indicate the geometric deviation between the first global point cloud and the first point cloud map. A second residual is determined based on the first point cloud map and the first point cloud feature information. The second residual is used to indicate the point cloud feature deviation between the first global point cloud and the first point cloud map. The first prior pose is optimized based on the first residual and the second residual; Based on the optimized first prior pose, the first original point cloud is remapped to the global coordinate system to obtain the corrected first global point cloud.

6. The method according to claim 5, characterized in that, The optimization of the first prior pose based on the first residual and the second residual includes: Obtain the first weighting coefficients of the first residual and the second residual, and use the first weighting coefficients to adjust the contribution of the first residual and the second residual in the optimization process of the first prior pose; The first prior pose is optimized based on the first weighting coefficient, the first residual, and the second residual.

7. The method according to any one of claims 1-6, characterized in that, The step of mapping the first original point cloud to a first global point cloud based on the robot's first prior pose includes: If the first original point cloud meets the downsampling conditions, the first original point cloud is downsampled; Based on the first prior pose, the downsampled first original point cloud is mapped to the first global point cloud.

8. A robot control device, characterized in that, The device includes an acquisition module, a determination module, a correction module, and a map generation module; The acquisition module is configured to acquire a first raw point cloud through sensors mounted on the robot, and map the first raw point cloud to a first global point cloud based on the robot's first prior position. The first raw point cloud indicates the positional relationship between the robot and the surrounding environment in the sensor coordinate system at the first sampling time. The first prior position indicates the estimated pose of the robot in the global coordinate system at the first sampling time. The first global point cloud indicates the positional relationship between the robot and the surrounding environment in the global coordinate system at the first sampling frame. The determining module is configured to determine the geometric constraint features of the surrounding environment based on the first point cloud map and the first global point cloud, wherein the first point cloud map is a map of the surrounding environment in the global coordinate system at the second sampling time, and the second sampling frame is the previous sampling frame of the first sampling frame. The correction module is configured to correct the first global point cloud based on the geometric constraint features; The map generation module is configured to construct a second point cloud map based on the corrected first global point cloud and the first point cloud map, wherein the second point cloud map is a map of the surrounding environment in the global coordinate system at the first sampling time.

9. An electronic device, characterized in that, include: Memory, which stores program instructions used to build environment maps; as well as A processor, when the program instructions are executed by the processor, causes the electronic device to perform the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, It stores program instructions for constructing an environment map, which, when executed by one or more processors, cause the device to implement the method described in any one of claims 1-7.

Images