Map processing method, medium and program product

Abstract

The invention provides a map processing method, a medium and a program product, and relates to the technical field of laser mapping, according to the embodiment, under the condition of triggering loopback detection, a first relative pose between a current frame and a historical key frame is determined based on the current frame and the historical key frame obtained by a robot through scanning in a current scene; if the loopback detection is successful, determining whether the current scene is a gallery scene based on the point cloud of the historical key frame; if it is confirmed that the current scene is the gallery scene, according to the angle of the gallery in the world coordinate system and the mapping relation between the world coordinate system and the coordinate system where the gallery is located, the error in the first relative pose in the gallery direction is eliminated, and a second relative pose is obtained; according to the second relative pose, the contour map of the current scene is constructed, errors in the gallery direction (namely, parallel to the gallery direction) can be eliminated, at the moment, errors accumulated at the front end can be eliminated, and a correct contour map can be established.

Description

Technical Field

[0001] The present application relates to the field of laser mapping technology, and in particular to a map processing method, medium and program product. Background Art

[0002] In some scenarios where robots are used, such as commercial cleaning, industrial logistics, and commercial food delivery, contour maps of robot activity scenes can be pre-built for robot positioning and navigation in these scenarios. Related technologies perform inter-frame matching at the front end to obtain the pose of the current frame; perform loop detection at the back end, and after determining the loop, add the loop detection results as constraints to the back end optimization for global optimization. In the above scenarios, if there are constraints in all directions, the error of the loop detection results is very small when performing loop detection. Adding the loop detection results as constraints to the back end optimization can eliminate the accumulated errors of the front end and establish a correct contour map. However, there are still scenarios that lack effective constraints in the above scenarios, such as long corridor scenes. If the scenes are not distinguished during loop detection, the accuracy of the loop detection results will be very poor. Such loop detection results will often only introduce large errors to the back end optimization. If the corridor is very long, continuously adding loop detection results with poor accuracy to the back end optimization will cause the contour map to be misplaced and the map construction will fail. Summary of the invention

[0003] In view of this, the embodiments of the present application are committed to providing a map processing method, medium and program product to solve the problem in the related technology that due to the lack of effective constraints in the corridor scene, the front-end accumulated error is added to the back-end optimization, which will introduce a large error and cause the failure of constructing the contour map.

[0004] In a first aspect, an embodiment of the present application provides a map processing method, the method comprising: under the condition of triggering loop detection, determining a first relative pose between the current frame and the historical key frame based on the current frame and the historical key frame scanned by the robot in the current scene; if the loop detection is successful, confirming whether the current scene is a corridor scene based on the point cloud of the historical key frame; if the current scene is confirmed to be a corridor scene, eliminating the error along the corridor direction in the first relative pose according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, to obtain a second relative pose; according to the second relative pose, constructing a contour map of the current scene.

[0005] In a second aspect, an embodiment of the present application provides a map processing device, which includes: a posture determination module, which is used to determine a first relative posture between a current frame and a historical key frame based on a current frame and historical key frames scanned by a robot in a current scene under the condition of triggering loop detection; a scene confirmation module, which is used to confirm whether the current scene is a corridor scene based on the point cloud of the historical key frame if the loop detection is successful; an error optimization module, which is used to eliminate the error along the corridor direction in the first relative posture according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, to obtain a second relative posture if the current scene is confirmed to be a corridor scene; and a map construction module, which is used to construct a contour map of the current scene according to the second relative posture.

[0006] In a third aspect, an embodiment of the present application provides a computing device, which includes: a processor and a memory for storing processor executable instructions; the processor is used to execute the map processing method described in the first aspect above.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program for executing the map processing method described in the first aspect above.

[0008] In a fifth aspect, an embodiment of the present application provides a program product, including computer program instructions, which, when executed by a processor, enable the processor to implement the map processing method described in the first aspect.

[0009] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0010] 1) In the embodiment of the present application, under the condition of triggering loop detection, a first relative pose between the current frame and the historical key frame is determined based on the current frame and the historical key frame in the current scene; when the loop detection is successful, whether the current scene is a corridor scene is confirmed based on the point cloud of the historical key frame; when the current scene is confirmed to be a corridor scene, according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, the error in the first relative pose along the corridor direction is eliminated to obtain a second relative pose, and a contour map of the current scene is constructed based on the second relative pose, which can eliminate the error along the corridor direction (i.e., parallel to the corridor direction). At this time, the accumulated error in the front end can be eliminated, and a correct contour map can be established.

[0011] 2) When confirming whether the current scene is a long corridor scene, the embodiment of the present application filters out points with incorrect normal angle matching in the point cloud of the historical key frame according to the first relative pose, and then performs principal component analysis on the normal angles of all matching points (i.e., matching points with correct normal angle matching), and determines the main direction and the normal direction with the largest angle difference with the main direction, and determines the angle difference between the normal direction and the main direction; and confirms whether the current scene is a long corridor scene based on the determined angle difference, thereby enabling accurate identification of the long corridor scene.

[0012] 3) The embodiment of the present application utilizes the first relative posture to project each point, and based on the matching result between the normal angle of each projection point and the normal angle of the second adjacent point, each point whose matching result is the first matching result is used as the matching point in the point cloud, thereby filtering out the points in the point cloud with incorrect normal angle matching based on the first relative posture.

[0013] 4) The embodiment of the present application uses the angle of the corridor in the world coordinate system to project the first relative posture to the coordinate system where the corridor is located, and obtains the third relative posture. The origin of the coordinate system where the corridor is located is located at any position of the corridor, and the horizontal axis is parallel to the direction of the corridor, and the vertical axis is perpendicular to the direction of the corridor; next, the value of the horizontal coordinate of the third relative posture in the coordinate system where the corridor is located is set to 0 to obtain the fourth relative posture; next, the angle of the corridor in the world coordinate system is used to project the fourth relative posture to the world coordinate system to obtain the second relative posture, thereby removing the error in the first relative posture along the direction of the corridor.

[0014] 5) The embodiment of the present application uses the second relative posture as a constraint condition to construct a loop edge; then, the error function of the loop edge and the error of the Jacobian matrix in the direction perpendicular to the corridor are optimized to obtain the optimized error function and Jacobian matrix; then, the current posture is corrected using the optimized error and Jacobian matrix, thereby optimizing the error perpendicular to the corridor direction, and a correct contour map can be established.

[0015] 6) The embodiment of the present application utilizes the angle of the corridor in the world coordinate system to project the third relative posture into the coordinate system where the corridor is located, and corrects the error of the projected error function in the horizontal axis direction to 0, and corrects the Jacobian matrix of the projected error function in the horizontal axis direction to 0, thereby obtaining the optimized error function and Jacobian matrix, thereby optimizing the error function and Jacobian matrix of the loop edge in the direction perpendicular to the corridor, and obtaining the optimized error function and Jacobian matrix.

[0016] 7) The embodiment of the present application matches the local map where the current frame is located with the local map where the historical key frame is located to determine the first relative pose between the historical key frame and the current frame, thereby improving the robustness of the loop detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the architecture of the application scenario of the map processing method provided in the embodiment of the present application;

[0018] Figure 2 is a flowchart of a map processing method provided in an embodiment of the present application;

[0019] Figure 3 is a structural schematic diagram of a map processing device provided in an embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of the structure of the computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0022] There are still some scenes in the related technologies that lack effective constraints. For example, in the long corridor scene, when performing loop detection, due to the lack of effective constraints parallel to the direction of the corridor, the error of the loop detection result is very large. Adding the loop detection result as a constraint to the back-end optimization will introduce a large error. If the corridor is very long, continuously adding loop detection results with large errors will cause the contour map to be misaligned and the map construction to fail.

[0023] The map processing method provided in the embodiment of the present application determines, under the condition of triggering loop detection, a first relative pose between the current frame and the historical key frame in the current scene based on the current frame and the historical key frame; when the loop detection is successful, confirms whether the current scene is a corridor scene based on the point cloud of the historical key frame; when confirming that the current scene is a corridor scene, according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, eliminates the error along the corridor direction in the first relative pose to obtain a second relative pose, and constructs a contour map of the current scene based on the second relative pose, which can eliminate the error along the corridor direction (i.e., parallel to the corridor direction). At this time, the accumulated error in the front end can be eliminated, and a correct contour map can be established.

[0024] For ease of understanding, this article first introduces the application scenarios to which the map processing method provided by this application is applicable. Figure 1, the computing device 110 is electrically connected to the sensor group 130. The data storage system 120 can store the data that the computing device 110 needs to process. The data storage system 120 can be integrated on the computing device 110, or placed on the cloud or other network servers. After the sensor group 130 collects the current frame, it transmits the collected current frame to the computing device 110, and the computing device 110 performs the following operations:

[0025] (1) Under the condition of triggering loop detection, based on the current frame and the historical key frames scanned by the robot in the current scene, determine the first relative pose between the current frame and the historical key frames;

[0026] (2) If the loop detection is successful, then the point cloud of the historical keyframe is used to confirm whether the current scene is a long corridor scene;

[0027] (3) If it is confirmed that the current scene is a corridor scene, then according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, the error along the corridor direction in the first relative pose is eliminated to obtain the second relative pose;

[0028] (4) Based on the second relative pose, construct a contour map of the current scene.

[0029] The computing device 110 may be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable wearable devices. In the embodiment of the present application, it may be a computing unit in a robot.

[0030] like Figure 2 As shown, the embodiment of the present application provides a map processing method, which is applied to Figure 1 The computing device in the example is used to illustrate, including the following steps:

[0031] Step S201, under the condition of triggering loop detection, based on the current frame and the historical key frames scanned by the robot in the current scene, determine the first relative posture between the current frame and the historical key frames.

[0032] Step S202: If the loop detection is successful, it is confirmed whether the current scene is a long corridor scene based on the point cloud of the historical key frame.

[0033] Step S203: If it is confirmed that the current scene is a corridor scene, the error in the direction of the corridor in the first relative posture is eliminated according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, so as to obtain the second relative posture.

[0034] Step S204: construct a contour map of the current scene according to the second relative posture.

[0035] It can be seen that, under the condition of triggering loop detection, the present application determines the first relative pose between the current frame and the historical key frame based on the current frame and the historical key frame of the robot in the current scene; when the loop detection is successful, it is confirmed whether the current scene is a corridor scene based on the point cloud of the historical key frame; when the current scene is confirmed to be a corridor scene, according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, the error along the corridor direction in the first relative pose is eliminated to obtain the second relative pose, and a contour map of the current scene is constructed based on the second relative pose, which can eliminate the error along the corridor direction (i.e., parallel to the corridor direction). At this time, the accumulated error at the front end can be eliminated, and a correct contour map can be established.

[0036] The following is a detailed description of each step in the above process and the effects that can be further produced in conjunction with the embodiments. It should be noted that the limitations such as "first" and "second" involved in this application do not have limitations in terms of size, order, and quantity, and are only used to distinguish them in name. For example, "first relative posture" and "second relative posture" are used to distinguish two postures in name.

[0037] First, in conjunction with the embodiment, the above step S201, namely, "under the condition of triggering loop detection, determining the first relative posture between the current frame and the historical key frames based on the current frame and the historical key frames scanned by the robot in the current scene" is described in detail.

[0038] In the embodiment of the present application, sensors (for example, Figure 1 The sensor group 130 shown in the figure), during the movement of the current scene, the robot will obtain the laser frame of the latest position through the sensor scanning as the current frame; then, under the condition of triggering loop detection, the first relative posture between the current frame and the historical key frame is determined based on the current frame and the historical key frame. Among them, the sensor can include various laser radars, etc. The conditions for triggering loop detection are usually related to the movement state of the robot in the environment and its ability to perceive the environment. The conditions for triggering loop detection in the embodiments of the present application include but are not limited to the following: (1) the cumulative error reaches a certain level; (2) the similarity between the current and historical detected environments must exceed a certain threshold, which is usually reflected in the feature similarity between the current frame and the historical key frame exceeding a certain threshold; (3) the moving distance must exceed a certain threshold; (4) the time from the last loop detection exceeds a certain threshold.

[0039] When the robot moves a certain distance (i.e., the moving distance exceeds a certain threshold) or a certain time has passed (i.e., the moving time exceeds a certain threshold) without detecting a loop, loop detection can be triggered. This is to ensure that the robot can correct the accumulated errors through loop detection after moving for a long time or a long distance.

[0040] Among them, the historical key frame is a laser frame marked as a historical key frame at intervals and distances during the robot's movement in the current scene according to the richness of the laser information. The richness of the laser information can be considered from the dimensions of information quantity, information accuracy, and information diversity.

[0041] The historical key frame in the embodiment of the present application refers to a representative frame in a series of continuous frames in SLAM (Simultaneous localization and mapping). It contains enough information to describe the characteristics of the surrounding environment while reducing the redundancy of information. In SLAM, the selection of historical key frames is crucial to improving positioning accuracy, reducing the amount of calculation, and optimizing map construction.

[0042] In one example, determining a first relative pose between a current frame and a historical key frame may include:

[0043] Match the current frame with the local map where the historical key frame is located to determine the first relative pose between the current frame and the historical key frame; or, match the local map where the current frame is located with the historical key frame to determine the first relative pose between the current frame and the historical key frame; or, match the local map where the current frame is located with the local map where the historical key frame is located to determine the first relative pose between the current frame and the historical key frame; or, match the current frame with the historical key frame to determine the first relative pose between the current frame and the historical key frame.

[0044] Here, relative posture includes position and attitude. In two-dimensional space, the position is represented by (x, y) coordinates and the attitude is represented by azimuth.

[0045] Correspondingly, in this example, the local map can be a map constructed within a preset time period, such as a local map created within the past 30 seconds; the local map can also be a local map constructed for all laser frames within a preset distance, such as saving all laser frames within a preset distance as a local map with a resolution of 0.01.

[0046] Before matching the current frame with the historical keyframe, the historical keyframe and the current frame can be transferred to the same coordinate system (e.g., the world coordinate system or the coordinate system of the corridor) to ensure that the historical keyframe and the current frame are consistent in spatial position. In this way, when matching, comparison and analysis can be based on the same spatial reference, thereby improving the accuracy of matching.

[0047] The above step S202, i.e., "if the loop detection is successful, confirming whether the current scene is a corridor scene based on the point cloud of the historical key frame" is described in detail below in conjunction with the embodiments.

[0048] In the embodiment of the present application, when confirming that the loop detection is successful, it is confirmed whether the current frame is a corridor scene based on the point cloud of the historical key frame.

[0049] The success of loop detection in the embodiment of the present application is determined based on the feature similarity between the current frame and the historical key frame. For example, the Euclidean distance between the features of the current frame and the historical key frame is determined, and if the Euclidean distance is less than the set distance value, the loop detection is confirmed to be successful.

[0050] In a long corridor scene, there are usually two walls, and the two boundary lines between the two walls and the ground are parallel to each other. In the process of identifying the long corridor scene, whether the current scene where the robot is located is a long corridor scene is mainly determined by identifying whether the point cloud of the historical key frame belongs to the points on the two boundary lines.

[0051] In the embodiment of the present application, the long corridor scene usually has some specific geometric features, such as two parallel walls, a long corridor length, etc. Therefore, the long corridor scene can be identified by extracting these features.

[0052] In one example, confirming whether the current frame is a long corridor scene based on the point cloud of the historical key frame may include the following steps:

[0053] Step 1: Use the point cloud segmentation algorithm to segment the point cloud of the historical keyframe into different geometric structures, such as walls, floors, ceilings, etc.

[0054] Step 2: Extract possible wall features from the segmented point cloud and calculate their normal angles.

[0055] Step 3: Check whether the wall features are parallel and whether the distance between them meets the characteristics of the corridor.

[0056] Among them, a point cloud is a set of points discretely distributed in a three-dimensional space, each point usually contains its three-dimensional coordinates (X, Y, Z) and possible additional information such as color, normal, intensity, etc. In SLAM and three-dimensional reconstruction, a point cloud is generated by extracting feature points (such as corner points, edge points, etc.) from an image and then converting these feature points into points in a three-dimensional space using depth information (such as obtained through stereo vision, laser scanning, structured light, etc.).

[0057] The point cloud of historical keyframes refers to the set of points extracted from the historical keyframe images and converted into three-dimensional space. These point clouds are used in SLAM systems to build three-dimensional maps of the environment, and also help with the positioning and navigation of robots. Since the selection of historical keyframes reduces redundant information, the point cloud of historical keyframes is more efficient in building maps.

[0058] The following is a detailed description of “confirming whether the current scene is a corridor scene based on the point cloud of historical key frames” in conjunction with an embodiment.

[0059] In an embodiment of the present application, the normal angle of each point is determined for the first adjacent point of each point in the point cloud of the historical key frame, and the normal angle of each point is the angle between the normal at each point and the target axis of the world coordinate system; then, the normal angles of the matching points in the point cloud that match correctly are determined according to the first relative pose; then, the normal angles of these matching points (i.e., the normal angles of the matching points that match correctly) are subjected to principal component analysis, and the main direction and the normal direction with the largest angle difference with the main direction are determined, and the angle difference between the normal direction and the main direction is determined, and whether the current scene is a long corridor scene is confirmed according to the determined angle difference. Among them, the normal direction can be defined by an angle range, such as the direction corresponding to the angle range of [-89,-79]. Among them, the normal angle in point cloud processing usually refers to the angle between the normal at each point and a reference direction (such as the world coordinate system direction), wherein the world coordinate system direction can be the direction corresponding to the target axis, and the target axis can be the X-axis, Y-axis or Z-axis, and the world coordinate system direction can also be a specific direction in the plane formed by these axes.

[0060] Specifically, in the first step, the normal angle of each point is calculated using the first adjacent point using the storage order of the laser data; in the second step, clustering is performed according to the matching results: each point is projected using the first relative posture, and whether the normal angle of the point is retained is determined based on the matching result between the normal angle of each projection point and the normal angle of the second adjacent point; the match is confirmed to be successful when the angle between the normal angle of the second adjacent point and the normal angle of each projection point is less than a preset angle threshold; in the third step, the main direction and the normal direction with the largest angle difference from the main direction are obtained using the results of principal component analysis, and then the angle difference between the main direction and the normal direction is determined. If the angle difference is less than a certain threshold, the current scene is considered to belong to a corridor scene.

[0061] Here, the above-mentioned successful matching may also be that a second adjacent point adjacent to each projection point can be found within a preset range. The preset range may be set according to the accuracy of the identifiable corridor and / or prior knowledge.

[0062] It should also be noted that after finding the second adjacent point, the match is confirmed to be successful only when the angle between the normal angle of the second adjacent point and the normal angle of each projection point is less than the preset angle threshold, which can improve the accuracy of identifying the long corridor scene.

[0063] In one example, before executing the third step, the map processing method further includes: performing principal component analysis on the normal angles of the matching points, and counting the number of matching points that fall into each normal direction; filtering out the normal directions whose number of matching points exceeds a first threshold value from all normal directions. The covariance matrix of the normal angles of the matching points is calculated, and the main direction corresponds to the eigenvector corresponding to the maximum eigenvalue of the covariance matrix. In the long corridor scene, the main direction is the direction perpendicular to the long corridor direction.

[0064] Correspondingly, in this example, PCA (principal components analysis) is used to determine each normal direction (for example, each normal direction in the main direction and other normal directions), and the number of normal angles in each normal direction is counted by a histogram. Among them, the number of other normal directions is at least one.

[0065] Correspondingly, in this example, the number of matching points falling into each normal direction is counted, including: the normal angle of each matching point can be mapped to each normal direction, and the number of normal angles falling into each normal direction is counted respectively, and then the number of matching points in each normal direction is determined according to the number of normal angles falling into each normal direction.

[0066] [89°, -89°] is divided into 18 angle ranges, each of which corresponds to the above normal direction.

[0067] Here, the normal directions whose number of matching points exceeds the first quantity threshold are screened out from each normal direction by using the first quantity threshold. The first quantity threshold can be set according to the accuracy of the contour map and / or prior knowledge.

[0068] The embodiment of the present application performs principal component analysis on the normal angles of the matching points, and counts the number of matching points falling into each normal direction; and based on a first quantity threshold, screens out the normal directions whose number of matching points exceeds the quantity threshold; next, based on the screened directions, executes the third step to confirm whether the current scene is a long corridor scene. Based on the first quantity threshold, the normal directions whose number of matching points is less than the first quantity threshold can be filtered out, and the normal angles falling into the normal method can be filtered out accordingly, thereby avoiding the interference of outliers.

[0069] In one example, the slope of each point may be determined based on the coordinates of the first neighboring point of each point in the point cloud of the historical key frame; then, the normal angle of each point may be determined according to the slope of each point.

[0070] It should be noted that the method of determining the slope of each point only according to the coordinates of the first adjacent point is to reduce the amount of calculation to improve processing efficiency, and this is not the only method. For example, the slope of each point can also be determined by the slope of the first adjacent point.

[0071] Here, the first adjacent point may be a point adjacent to each point, for example, two adjacent front and back points.

[0072] Continuing from the above, determining the slope of each point based on the coordinates of the first adjacent point of each point in the point cloud of the historical key frame may include: for each point, calculating the difference between the horizontal axis coordinate and the vertical axis coordinate of the two adjacent points before and after the point, and finding their ratio as the slope.

[0073] Correspondingly, the slope of each point can be determined by the average of the slopes of the adjacent preceding and following points. Specifically, for each point, the slopes of the two preceding and following points are average filtered to obtain the slope of the point. This can reduce the impact of noise.

[0074] Correspondingly, the slope of each point can be used to query a preset query table to obtain the normal angle of each point; wherein the query table includes the corresponding relationship between the tangent angle value and the normal angle.

[0075] Specifically, the tangent value closest to the slope is searched from a preset query table through the slope, and the normal angle corresponding to the found tangent value is output as the normal angle of each point.

[0076] Correspondingly, for the slope of each point, the atctan (inverse tangent) of the slope can be calculated to calculate the angle, and the angle is used as the normal angle of the point.

[0077] In one example, determining, according to the first relative pose, each matching point in the point cloud with a correct normal angle may include:

[0078] Based on the first relative pose, each point is projected to the same coordinate system, and the matching result between the normal angle of the second adjacent point of each projection point and the normal angle of each projection point is determined; each point whose matching result is the first matching result is used as each matching point in the point cloud; wherein the first matching result is used to indicate that the angle difference between the normal angle of each projection point and the normal angle of the second adjacent point meets the preset angle threshold. The same coordinate system can be a world coordinate system or a coordinate system where the long corridor is located.

[0079] For example, each point is projected to the world coordinate system using the first relative pose to obtain the projection point of each point in the world coordinate system; next, the matching result between the normal angle of the second adjacent point of each projection point and the normal angle of each projection point is determined; next, each point with directional consistency (i.e., the angle difference between the normal angle of each projection point and the normal angle of the second adjacent point meets the preset angle threshold) is used as each matching point in the point cloud. The preset angle threshold can be set based on the accuracy of the contour map and / or prior knowledge.

[0080] In an embodiment of the present application, the first relative posture is used to project each point in the point cloud to determine the matching result, and whether the normal angle of the point is correct is judged based on the matching result. Only the correctly matched matching points (that is, the angle difference between the normal angle of the second adjacent point and the normal angle of each projection point is less than a preset angle threshold) are used to determine whether the current scene is a long corridor scene.

[0081] The above step S203, i.e., "if it is confirmed that the current scene is a corridor scene, then according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, the error along the corridor direction in the first relative posture is eliminated to obtain the second relative posture" is described in detail in conjunction with the embodiments.

[0082] In an embodiment of the present application, when it is confirmed that the current scene is a corridor scene, the error along the corridor direction in the first relative posture is eliminated according to the angle of the corridor in the world coordinates and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, so as to obtain the second relative posture.

[0083] It should be noted that eliminating the error along the corridor direction in the first relative pose means that the error along the corridor direction is not processed during the back-end optimization (i.e., the error along the corridor direction is removed), and only the error perpendicular to the corridor direction is optimized.

[0084] For example, the coordinate value along the corridor direction (corresponding to the horizontal axis) in the coordinate system where the corridor is located is set to 0 to eliminate the error along the corridor direction in the first relative pose, so that there is no need to process the error along the corridor direction during the back-end optimization.

[0085] In the embodiment of the present application, as the moving distance increases, the front end will continue to accumulate errors. When the robot returns to the vicinity of the position it has passed at a historical moment, loop detection is used to determine that the robot is in the same environment, and then loop edges are added to eliminate the accumulated errors of the front end. However, in the special environment of a long corridor, the loop detection result has a large error in the direction parallel to the corridor, which in turn affects the back-end optimization, resulting in the inability to eliminate the accumulated errors, map errors, overlapping images, and other phenomena.

[0086] The embodiment of the present application uses a mapping relationship between coordinate systems (i.e., a mapping relationship between the world coordinate system and the coordinate system where the corridor is located) to transform the error accumulated in the front end of the first relative posture through the coordinate system to eliminate the error in the first relative posture along the corridor direction.

[0087] In one example, according to the angle of the corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the corridor is located, eliminating the error along the corridor direction in the first relative posture to obtain the second relative posture can include: using the angle of the corridor in the world coordinate system, projecting the first relative posture to the coordinate system where the corridor is located, to obtain a third relative posture; wherein the origin of the coordinate system where the corridor is located is located at an arbitrary position of the corridor, the horizontal axis of the coordinate system where the corridor is located is parallel to the direction of the corridor, and the vertical axis of the coordinate system where the corridor is located is perpendicular to the direction of the corridor; setting the value of the horizontal coordinate of the third relative posture in the coordinate system where the corridor is located to 0, to obtain a fourth relative posture; using the angle of the corridor in the world coordinate system, projecting the fourth relative posture to the world coordinate system, to obtain the second relative posture.

[0088] For example, the first relative posture is projected to the coordinate system where the corridor is located using the angle ɑ of the corridor in the world coordinate system to obtain the third relative posture, and the value of the horizontal coordinate of the third relative posture in the coordinate system where the corridor is located is set to 0 to obtain the fourth relative posture. Finally, the fourth relative posture is projected to the world coordinate system using the angle ɑ to obtain the second relative posture.

[0089] The above steps can eliminate the error along the long corridor direction, that is, the relative posture along the long corridor direction is not added to the back-end optimization, thereby removing the influence of the error along the long corridor direction on the back-end optimization.

[0090] The second relative position in the embodiment of the present application refers to the relative movement of the robot from one position to another position through the environmental information captured by the sensor. This is usually achieved by analyzing continuous sensor data (such as image frames or laser scanning data).

[0091] The above step S204, namely "constructing a contour map of the current scene according to the second relative posture", is described in detail below in conjunction with the embodiments.

[0092] In one example, building a contour map of the current scene may include the following steps: Initialize the map: Before the robot starts moving, an initial map or a blank map is usually required to represent the unknown environment; Update the map: As the robot moves, update the contour map according to the second relative pose. This usually involves fusing new sensor data (such as point clouds or images) into the map to represent the environment around the robot; Optimize the map: In order to improve the accuracy and consistency of the map, the map can be optimized. This includes eliminating noise, smoothing edges, merging similar features, etc.; Loop detection: During the movement of the robot, it may return to areas that have been visited before; through loop detection, these repeatedly visited areas can be identified and the map can be updated to eliminate accumulated errors.

[0093] After the cumulative error passes through step S203, the error of the coordinate system where the long corridor is located in the direction perpendicular to the long corridor can also be optimized to further eliminate the influence of the cumulative error on the back-end optimization.

[0094] Next, the following method is used to optimize the error of the coordinate system where the corridor is located in the direction perpendicular to the corridor: the second relative posture is used as a constraint condition, a loop edge is constructed, and it is added to the back-end optimization. At this time, the error function and Jacobian matrix of the loop edge are processed as follows: specifically: the second relative posture is used as a constraint condition to construct a loop edge; optimize the error of the error function and Jacobian matrix of the loop edge in the direction perpendicular to the corridor, and obtain the optimized error function and Jacobian matrix; use the optimized error and Jacobian matrix to correct the current posture. Among them, the loop edge represents the direct constraint relationship between the current frame and the historical key frame in space.

[0095] In one example, optimizing the error function and Jacobian matrix of the loop edge in the direction perpendicular to the corridor to obtain the optimized error function and Jacobian matrix can include: using the angle of the corridor in the world coordinate system to project the error function to the coordinate system where the corridor is located to obtain the projected error function; correcting the error of the projected error function in the horizontal axis direction to 0, and correcting the Jacobian matrix of the projected error function in the horizontal axis direction to 0, to obtain the optimized error function and Jacobian matrix.

[0096] It should be noted that, by correcting the error in the horizontal axis direction to 0 and correcting the Jacobian matrix of the projected error function in the horizontal axis direction to 0, the error along the corridor direction can also be eliminated; the processing for eliminating the error along the corridor direction here can be used as a further supplement to the above-mentioned step S203, and the error along the corridor direction can be eliminated multiple times to completely eliminate the error along the corridor direction before the back-end optimization, thereby solving the influence of the error along the corridor direction on the back-end optimization.

[0097] For example, assuming that the error function of the loop edge is f(x), use the angle ɑ of the corridor in the world coordinate system to project the error function to the corridor coordinate system to obtain the error function f(x)_2, set the error in the horizontal axis direction to 0, and set the Jacobian matrix in the horizontal axis direction to 0.

[0098] In the embodiment of the present application, the constraints along the corridor direction are eliminated in each iteration of the back-end optimization. In this way, in the corridor scenario, the problem in the related art that if the loop detection results with large errors are added as constraints to the back-end optimization, large cumulative errors will be introduced due to the lack of effective constraints parallel to the corridor direction.

[0099] Here, the contour map may include a corrected current posture; wherein the posture is represented as position and attitude, and in two-dimensional space, the position is represented by (x, y) coordinates, and the attitude is represented by azimuth.

[0100] In the embodiment of the present application, through the application of SLAM technology and the precise calculation of the second relative posture, an accurate and consistent contour map can be constructed, providing strong support for the robot's autonomous navigation.

[0101] Here, the contour map is used to provide navigation for the robot's movement in the current scene. It is similar to the reflective column navigation and QR code navigation in related technologies, avoiding environmental modification and complex maintenance in the later stage. Among them, reflective column navigation refers to navigation using reflective columns artificially placed in the environment; reflective columns refer to cylinders that can reflect back most of the energy of lasers. QR code navigation refers to navigation using QR codes artificially posted in the environment.

[0102] It should be noted that when robots are currently used to construct contour maps, the sensors used are usually odometers, IMUs (Inertial measurement units), and two-dimensional laser sensors. The odometer and IMU provide the initial pose of the two frames of lasers. The two frames of lasers are precisely matched using the initial pose to obtain the precise pose between the two frames of lasers.

[0103] Combination of the above Figure 2 , describes the method embodiment of the present application in detail, and the following is combined with Figure 3 to Figure 4 , describes the device embodiment of the present application in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous method embodiment.

[0104] Figure 3 FIG. 1 is a schematic diagram of the structure of a map processing device provided by an embodiment of the present application. Figure 3 As shown, the map processing device provided by the embodiment of the present application includes a posture determination module 301, a scene confirmation module 302, an error optimization module 303 and a map construction module 304. Among them, the posture determination module 301 is used to determine the first relative posture between the current frame and the historical key frame based on the current frame and the historical key frame scanned by the robot in the current scene under the condition of triggering loop detection; the scene confirmation module 302 is used to confirm whether the current scene is a long corridor scene based on the point cloud of the historical key frame if the loop detection is successful; the error optimization module 303 is used to eliminate the error along the long corridor direction in the first relative posture according to the angle of the long corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the long corridor is located, and obtain the second relative posture; the map construction module 304 is used to construct a contour map of the current scene according to the second relative posture.

[0105] In one embodiment, the scene confirmation module 302 includes:

[0106] An angle determination unit (not shown in the figure), used to determine the normal angle of each point according to the first adjacent point of each point in the point cloud of the historical key frame;

[0107] A matching point determination unit (not shown in the figure), used to determine each matching point in the point cloud with a correct normal angle according to the first relative posture;

[0108] An angle difference determination unit (not shown in the figure), used to perform principal component analysis on the normal angle of the matching point, and determine the main direction and the normal direction with the largest angle difference with the main direction, and determine the angle difference between the normal direction and the main direction;

[0109] The scene confirmation unit (not shown in the figure) is used to confirm whether the current scene is a long corridor scene according to the determined angle difference.

[0110] In one embodiment, the matching point determination unit is specifically used to:

[0111] Projecting each point to the same coordinate system based on the first relative pose, and determining a matching result between a normal angle of a second adjacent point of each projection point and a normal angle of each projection point;

[0112] Each point whose matching result is the first matching result is respectively used as each matching point in the point cloud; wherein the first matching result is used to indicate that the angle difference between the normal angle of each projection point and the normal angle of the second adjacent point satisfies a preset angle threshold.

[0113] In one embodiment, the error optimization module 303 is specifically used to:

[0114] Using the angle of the corridor in the world coordinate system, the first relative posture is projected to the coordinate system where the corridor is located to obtain the third relative posture; wherein the origin of the coordinate system where the corridor is located is located at any position of the corridor, the horizontal axis of the coordinate system where the corridor is located is parallel to the direction of the corridor, and the vertical axis of the coordinate system where the corridor is located is perpendicular to the direction of the corridor;

[0115] The value of the horizontal coordinate of the third relative posture in the coordinate system where the long corridor is located is set to 0, to obtain a fourth relative posture;

[0116] The fourth relative posture is projected into the world coordinate system using the angle of the corridor in the world coordinate system to obtain the third relative posture.

[0117] In one embodiment, the map construction module 304 includes:

[0118] A construction unit (not shown in the figure), configured to construct a loop edge using the second relative posture as a constraint condition;

[0119] An error optimization unit (not shown in the figure) is used to optimize the error function of the loop edge and the error of the Jacobian matrix in the direction perpendicular to the long corridor to obtain the optimized error function and Jacobian matrix;

[0120] The correction unit (not shown in the figure) is used to correct the current posture using the optimized error and Jacobian matrix.

[0121] In one embodiment, the error optimization unit is specifically used to: project the error function to the coordinate system where the long corridor is located using the angle of the long corridor in the world coordinate system to obtain the projected error function;

[0122] The error of the projected error function in the horizontal axis direction is corrected to 0, and the Jacobian matrix of the projected error function in the horizontal axis direction is corrected to 0, so as to obtain the optimized error function and Jacobian matrix.

[0123] In one embodiment, the posture determination module 301 is specifically used to: match the local map where the current frame is located with the local map where the historical key frame is located to determine the first relative posture between the current frame and the historical key frame; or,

[0124] Match the current frame with the historical key frame to determine the first relative pose between the current frame and the historical key frame; or,

[0125] Match the local map where the current frame and the historical key frame are located to determine the first relative pose between the current frame and the historical key frame; or,

[0126] The local map where the current frame is located is matched with the historical key frame to determine the first relative pose between the current frame and the historical key frame.

[0127] Below, reference Figure 4 To describe a computing device according to an embodiment of the present application. Figure 4 Shown is a schematic diagram of the structure of a computing device provided in one embodiment of the present application.

[0128] like Figure 4 As shown, the computing device includes one or more processors 401 and memory 402 .

[0129] The processor 401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in a computing device to perform desired functions.

[0130] The memory 402 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement functions such as the map processing method of each embodiment of the present application mentioned above. Various contents such as point clouds may also be stored in the computer-readable storage medium.

[0131] In one example, the computing device may further include: an input device 403 and an output device 404 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0132] The input device 403 may include, for example, a keyboard, a mouse, etc.

[0133] The output device 404 can output various information to the outside, including contour maps, etc. The output device 404 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0134] Of course, to simplify, Figure 4Only some of the components in the computing device related to the present application are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the computing device may further include any other appropriate components according to specific application scenarios.

[0135] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the map processing method according to various embodiments of the present application described above in this specification.

[0136] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0137] In addition, an embodiment of the present application may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps of the map processing method according to various embodiments of the present application described above in this specification.

[0138] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0139] The basic principles of the present application are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present application are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present application. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, not for limitation, and the above details do not limit the present application to being implemented by adopting the above specific details.

[0140] The block diagrams of the devices, apparatuses, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0141] It should also be noted that in the apparatus, device and method of the present application, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0142] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0143] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

[0144] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A map processing method, characterized in that: The method comprises: Under the condition of triggering loop detection, based on the current frame and the historical key frame scanned by the robot in the current scene, determine a first relative posture between the current frame and the historical key frame; If the loop detection is successful, confirm whether the current scene is a long corridor scene based on the point cloud of the historical key frame; If it is confirmed that the current scene is a long corridor scene, then according to the angle of the long corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the long corridor is located, the error in the first relative posture along the direction of the long corridor is eliminated to obtain a second relative posture; According to the second relative posture, a contour map including the current scene is constructed.

2. The method according to claim 1, characterized in that The step of confirming whether the current scene is a long corridor scene based on the point cloud of the historical key frame includes: Determine the normal angle of each point according to the first adjacent point of each point in the point cloud of the historical key frame, where the normal angle of each point is the angle between the normal at each point and the target axis of the world coordinate system; Determine, according to the first relative posture, each matching point in the point cloud with a correct normal angle; Performing principal component analysis on the normal angles of the matching points, and determining a main direction and a normal direction having the largest angle difference with the main direction, and determining an angle difference between the normal direction and the main direction; It is determined whether the current scene is a long corridor scene according to the determined angle difference.

3. The method according to claim 2, characterized in that The determining, according to the first relative posture, each matching point in the point cloud with a correct normal angle, comprises: Projecting each point to the same coordinate system based on the first relative posture, and determining a matching result between a normal angle of a second adjacent point of each projection point and a normal angle of each projection point; Each point whose matching result is the first matching result is respectively used as the matching point in the point cloud; wherein the first matching result is used to indicate that the angle difference between the normal angle of the second adjacent point and the normal angle of each projection point satisfies a preset angle threshold.

4. The method according to claim 1, characterized in that Eliminating the error in the first relative posture along the direction of the long corridor according to the angle of the long corridor in the world coordinate system and the mapping relationship between the world coordinate system and the coordinate system where the long corridor is located to obtain the second relative posture includes: Using the angle of the long corridor in the world coordinate system, the first relative posture is projected to the coordinate system where the long corridor is located, so as to obtain a third relative posture; wherein the origin of the coordinate system where the long corridor is located is located at any position of the long corridor, the horizontal axis of the coordinate system where the long corridor is located is parallel to the direction of the long corridor, and the vertical axis of the coordinate system where the long corridor is located is perpendicular to the direction of the long corridor; The value of the abscissa of the third relative posture in the coordinate system where the long corridor is located is set to 0, to obtain a fourth relative posture; The fourth relative posture is projected onto the world coordinate system using the angle of the long corridor in the world coordinate system to obtain a second relative posture.

5. The method according to any one of claims 1 to 4, characterized in that Constructing a contour map including the current scene according to the second relative posture includes: Using the second relative posture as a constraint condition to construct a loop edge; Optimizing the error function and Jacobian matrix of the loop edge in a direction perpendicular to the long corridor to obtain an optimized error function and Jacobian matrix; The current posture is corrected using the optimized error and Jacobian matrix.

6. The method according to claim 5, characterized in that The step of optimizing the error function and Jacobian matrix of the loop edge in a direction perpendicular to the long corridor to obtain an optimized error function and Jacobian matrix includes: Using the angle of the long corridor in the world coordinate system, the error function is projected to the coordinate system where the long corridor is located to obtain a projected error function; The error of the projected error function in the horizontal axis direction is corrected to 0, and the Jacobian matrix of the projected error function in the horizontal axis direction is corrected to 0, so as to obtain an optimized error function and Jacobian matrix.

7. The method according to any one of claims 1 to 4, characterized in that: The determining of a first relative posture between the current frame and the historical key frame comprises: Matching the local map where the current frame is located with the local map where the historical key frame is located to determine a first relative position between the current frame and the historical key frame; or, Matching the current frame with the historical key frame to determine a first relative posture between the current frame and the historical key frame; or, Matching the local map where the current frame and the historical key frame are located to determine a first relative position between the current frame and the historical key frame; or, The local map where the current frame is located is matched with the historical key frame to determine a first relative posture between the current frame and the historical key frame.

8. A computing device, characterized in that including memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 7 by running the program in the memory.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, cause the processor to implement the method according to any one of claims 1 to 7.

Images