Map processing method and device, and readable storage medium

Abstract

Embodiments of the present application provide a map processing method and device, and a readable storage medium, relating to the technical field of mapping. In the method, in the process of mapping by a robot, a display device is controlled to display a first display line, the first display line corresponding to a first entity line in a physical environment to be mapped. In response to detecting that an included angle between a second entity line in the physical environment to be mapped and a normal direction of the first entity line is less than or equal to a first angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, the second display line being orthogonal to the first display line. Alternatively, in response to detecting that the included angle between the second entity line in the physical environment to be mapped and the normal direction of the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, the second display line being orthogonal to the first display line. In this way, the quality of a map can be improved.

Description

Technical Field

[0001] This invention relates to the field of mapping technology, and in particular to a map processing method, apparatus, and readable storage medium. Background Technology

[0002] Typically, robots collect data from their physical environment using onboard environmental sensors, and then build maps of that environment for later use. For example, navigation, path planning, and localization can be performed based on these maps.

[0003] In related technologies, the mapping process is directly processed according to the default mapping workflow. In this method, due to detection errors, the entities that are originally perpendicular or nearly perpendicular to each other are skewed in the displayed map, resulting in poor map quality. Summary of the Invention

[0004] This application provides a map processing method, apparatus, and readable storage medium to solve the problem of poor map quality caused by skewed entity structures in the displayed map.

[0005] In a first aspect, embodiments of this application disclose a map processing method, the method comprising: During the robot mapping process, the control display device displays a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; or In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, the second display line being orthogonal to the first display line.

[0006] This application also discloses a map processing apparatus, which includes a memory and a processor; The memory is used to store program code; The processor calls the program code, which, when executed, performs the following operations: During the robot mapping process, the control display device displays a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; or In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, the second display line being orthogonal to the first display line.

[0007] This application also discloses a readable storage medium storing instructions that, when executed by one or more processors, cause the processors to perform the methods described above.

[0008] This application also discloses a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The embodiments of this application include the following advantages: In the map processing method provided by the embodiments of the present invention, during the robot mapping process, the display device is controlled to display a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, which is orthogonal to the first display line. Alternatively, in response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, which is orthogonal to the first display line. Thus, for the first entity lines and second entity lines of two mutually perpendicular or nearly perpendicular entity structures, the second display line displayed by the display device is orthogonal to the first display line, which can avoid the situation where mutually perpendicular or nearly perpendicular entity structures are displayed skewed or non-orthogonal in the map, thereby improving map quality. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. The drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1This is a flowchart of the steps of a map processing method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the system architecture provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a processing flow provided in an embodiment of the present invention; Figure 4 This is a block diagram of a map processing device provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0013] First, an exemplary application scenario related to the embodiments of this application will be described. The map processing method provided in the embodiments of this application can be applied to robot mapping scenarios. For example, the robot can be a wheeled robot, a legged robot, etc. To facilitate the robot's navigation, path planning, and other operations, the robot will perform Simultaneous Localization and Mapping (SLAM) in the physical environment. Specifically, the robot will build a map of the physical environment based on data perceived by environmental sensors. The environmental sensors can be sensors used to collect the data required for mapping; for example, environmental sensors can be LiDAR, cameras, etc., and the data perceived by the environmental sensors can be point cloud data or images.

[0014] In related technologies, robots typically use data perceived by environmental sensors and combine it with odometry, Iterative Closest Point (ICP) matching, graph optimization, and other processes to complete mapping. In the physical environment to be mapped, such as regular environments like indoor offices, corridors, factories, and supermarkets, there are usually nearly parallel or orthogonal solid structures, such as walls, passageways, and curbs. These structures often exhibit parallel or orthogonal structural patterns. During robot mapping, odometry errors, matching errors, partial occlusion, or dynamic object points can affect the mapping process. For mutually perpendicular or nearly perpendicular solid structures, the lines representing these structures on the map may appear skewed or non-orthogonal, leading to poor map quality.

[0015] For example, when constructing walls based on design drawings, the walls in the drawings are typically parallel or perpendicular to each other. Tools such as laser levels are usually used to ensure that the constructed walls are horizontal or perpendicular, minimizing construction errors compared to the detection errors during robot mapping. Therefore, due to detection errors, the originally perpendicular or nearly perpendicular solid structures appear skewed in the final map.

[0016] Therefore, this application provides a map processing method. The map processing method will be described in detail below.

[0017] Figure 1 This is a flowchart of the steps of a map processing method provided in an embodiment of this application, such as... Figure 1 As shown, the method may include: Step 101: During the robot mapping process, the control display device displays the first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped.

[0018] Step 102: In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the orthogonal direction of the first entity line is less than or equal to a first angle threshold, control the display device to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line.

[0019] Step 103, or, in response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, the second display line being orthogonal to the first display line.

[0020] In this embodiment, the robot can begin mapping upon receiving a mapping instruction. For example, when the robot first enters a physical environment, it can be controlled to use the current physical environment as the physical environment to be mapped and begin mapping.

[0021] The first entity line and the second entity line can be any entity line of a physical structure in the physical environment to be mapped. The first entity and the second entity can be two entities or two parts of a single entity. The entity lines in the physical environment to be mapped are lines possessed by the physical structures within that environment. For example, entity lines can be lines possessed by physical structures such as walls, passageways, wardrobes, and desks. When displaying an entity structure on a map, it can be represented by lines displayed on the display interface (i.e., entity lines). The first display line and the second display line can be lines displayed on the display interface by a display device. Specifically, the first display line displayed on the display interface represents the first entity line in the physical environment to be mapped, and the second display line displayed on the display interface represents the second entity line in the physical environment to be mapped. The display device can be a mobile terminal with display functionality connected to the robot, or it can be a display device integrated into the robot body; this embodiment does not limit this.

[0022] In some embodiments, the first entity line and the second entity line are lines extending along a specific direction. For example, the first entity line and the second entity line are lines extending horizontally on a solid structure.

[0023] In some embodiments, the display device may also display a 3D map, in which the wall surface corresponding to the solid wall is displayed. It should be noted that since the wall surface is also composed of lines, when the wall surface corresponding to the solid wall is displayed, the line corresponding to the solid wall is necessarily displayed as well; that is, the area defined by the displayed line also includes the displayed surface.

[0024] After displaying the first display line for the first entity line in the physical environment to be mapped, any subsequent entity line can be used as a second entity line, and the angle between the orthogonal direction of the second entity line and the first entity line is detected to be less than or equal to a first angle threshold. The orthogonal direction of the first entity line is a direction that differs from the corresponding direction of the first entity line by 90°. The first display line corresponding to the first entity line can be directly displayed based on data including the first entity line collected by environmental sensors, or it can be displayed based on the main environmental direction of the physical environment to be mapped. The main environmental direction is typically the direction of a long indoor wall or corridor. In some embodiments, the main environmental direction is the structural direction with the largest cumulative length among multiple frames of data collected by environmental sensors. In practical applications, for two entity structures that appear perpendicular to each other to the user's eye, the entity lines corresponding to these two entity structures are usually perpendicular. The first angle threshold can be preset. For example, the first angle threshold is greater than 0 degrees and less than or equal to 10 degrees, and can be any value from (0°, 10°). For example, the first angle threshold can be 10°, 5°, 3°, or 2°. This avoids the problem of the first angle threshold being too large, which could lead to the wrong display of lines corresponding to non-orthogonal entity lines as orthogonal, thus reducing map accuracy. If the angle between the detected second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold, it means that the second entity line and the first entity line are in a perpendicular or nearly perpendicular state in the physical environment to be mapped. The entity structures corresponding to the second entity line and the first entity line are visually perpendicular to each other when viewed by the user's eye. Therefore, the display device can be controlled to display the second display line corresponding to the second entity line in a way that is orthogonal to the first display line. This makes the line structure displayed in the map more regular, avoiding the situation where mutually perpendicular entity structures are skewed or non-orthogonal when displayed in the map, thereby improving map quality.

[0025] Alternatively, the system can detect whether the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold. The second and third angle thresholds can be preset. For example, the second angle threshold can be greater than or equal to 80 degrees and less than 90 degrees, and the third angle threshold can be greater than or equal to 90 degrees and less than or equal to 100 degrees. A value can be randomly selected from [80°, 90°] as the second angle threshold and from [90°, 100°] as the third angle threshold. For example, the second angle threshold can be 80°, 85°, 87°, or 88°. Similarly, the third angle threshold can be 92°, 93°, 95°, or 100°. This avoids the problem of mistakenly displaying lines corresponding to non-orthogonal entity lines as orthogonal, thus preventing a decrease in map accuracy. If the angle between the detected second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, it indicates that the second entity line and the first entity line in the physical environment to be mapped are perpendicular or nearly perpendicular. The corresponding entity structures of the second and first entity lines appear perpendicular to each other when viewed by the user's eye. Therefore, the display device can be controlled to display the second display line corresponding to the second entity line in a manner orthogonal to the first display line. For example, an edge can be directly displayed based on data collected by the environmental sensor, including the second entity line. This data can be the image, point cloud data, etc., corresponding to the second entity line. Then, the displayed edge is rotated to a position orthogonal to the first display line according to the minimum angle between the displayed edge and the orthogonal direction of the first display line, thereby controlling the second display line to be orthogonal to the first display line. When directly displaying an edge based on data collected by the environmental sensor, including the second entity line, it can be displayed in a transparent style. After rotation, the edge is adjusted to a non-transparent style for user viewing. This makes the rotation process invisible to the user, thus avoiding any interference.

[0026] Specifically, the operations of detecting whether the angle between the orthogonal directions of the second entity line and the first entity line is less than or equal to a first angle threshold, and detecting whether the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, can be performed by measuring and detecting the angle between the second entity line and the first entity line using a laser goniometer mounted on the robot. Alternatively, the angle between the second entity line and the first entity line can be detected based on collected point cloud data or images. Specific detection methods will be explained in detail later.

[0027] In summary, in the map processing method provided by this embodiment of the invention, during the robot mapping process, the display device is controlled to display a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, which is orthogonal to the first display line. Alternatively, in response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, which is orthogonal to the first display line. Thus, for the first and second entity lines of two mutually perpendicular entity structures, the display device controls the corresponding second display line to be orthogonal to the first display line, which avoids the situation where mutually perpendicular entity structures are displayed skewed or non-orthogonal in the map, thereby improving map quality.

[0028] Optionally, embodiments of the present invention may further include the following steps: Step S21: In response to detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold, control the display device to display the second display line corresponding to the second entity line, wherein the second display line is parallel to the first display line.

[0029] The fourth angle threshold can be preset. For example, the fourth angle threshold can be greater than 0 degrees and less than or equal to 10 degrees, and can be any value selected from (0°, 10°). For example, the fourth angle threshold can be 2°, 3°, 5°, or 10°. This avoids the problem of mistakenly displaying lines corresponding to non-parallel entities as parallel, which would lead to a decrease in map accuracy.

[0030] In practical applications, for two seemingly parallel solid structures as viewed by the human eye, the angle between their corresponding solid lines typically does not exceed the fourth angle threshold. Correspondingly, if the angle between the second solid line and the first solid line is less than or equal to the fourth angle threshold, it indicates that the second solid line and the first solid line are nearly parallel in the physical environment of the map being constructed, and the corresponding solid structures are visually parallel to each other as viewed by the human eye. Therefore, the display device can be controlled to display the second display line corresponding to the second solid line in a manner parallel to the first display line.

[0031] In this embodiment of the invention, in response to detecting that the angle between the second entity line and the first entity line in the physical environment to be mapped is less than or equal to a fourth angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, and the second display line is parallel to the first display line. This avoids the situation where parallel entity structures are displayed skewed or non-parallel on the map, making the line structure displayed on the map more regular and the map more consistent with the actual building layout. This further improves map quality, readability, and subsequent user experience.

[0032] Optionally, embodiments of the present invention may further include the following steps: Step S31: Obtain the virtual line corresponding to the first physical line and the virtual line corresponding to the second physical line.

[0033] Accordingly, the detection that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is less than or equal to the first angle threshold may specifically include: detecting that the angle between the orthogonal direction of the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to the first angle threshold.

[0034] The above-mentioned detection that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to the second angle threshold and less than or equal to the third angle threshold may specifically include: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is greater than or equal to the second angle threshold and less than or equal to the third angle threshold.

[0035] The above-mentioned detection that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to the fourth angle threshold may specifically include: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to the fourth angle threshold.

[0036] In this context, virtual lines are those used in the robot's backend processing. The virtual lines corresponding to the physical lines are those extracted from the data collected by environmental sensors during backend processing, representing the physical lines. When the environmental sensor is a LiDAR, the point cloud data obtained by scanning the first physical line can be used as the first point cloud frame. Edge points are extracted from the first point cloud frame. Specifically, based on the neighborhood of each point in the first point cloud frame, the normal vector of the neighborhood of each point is calculated as the normal vector corresponding to each point. For any point in the first point cloud frame, if the angle between the normal vector corresponding to that point and the normal vector corresponding to its neighboring points exceeds a preset angle threshold, then that point is considered an edge point. Neighboring points are points located within the neighborhood of that point. Alternatively, the curvature corresponding to each point in the first point cloud frame can be calculated, and points with curvature greater than a preset curvature threshold can be considered edge points. Alternatively, a plane fitting can be performed on the first point cloud frame, and the points on the intersection lines between planes in the first point cloud frame can be considered edge points. Then, the edge points in the first point cloud frame are clustered to obtain the first point cluster. Line segment fitting is performed on the first point cluster to obtain the virtual line corresponding to the first solid line. If there are multiple first point clusters and multiple line segments are obtained through fitting, the longest line segment can be used as the virtual line corresponding to the first solid line.

[0037] The point cloud data obtained by scanning the second entity line is used as the second point cloud frame, and edge points are extracted from the second point cloud frame. The edge points in the second point cloud frame are clustered to obtain second point clusters. Line segment fitting is performed on the second point clusters to obtain the virtual line corresponding to the second entity line. If multiple second point clusters exist and multiple line segments are obtained through fitting, the longest line segment can be used as the virtual line corresponding to the second entity line. The implementation method for extracting edge points in the second point cloud frame can refer to the implementation method for extracting edge points in the first point cloud frame, and will not be repeated here. In this implementation method, by extracting edge points and fitting line segments from the point cloud data, the virtual lines corresponding to the first and second entity lines can be obtained respectively. This provides a basis for subsequent judgment on whether the angle between the orthogonal directions of the second and first entity lines exceeds the corresponding angle threshold, making subsequent judgment operations convenient.

[0038] When the environmental sensor is a camera, the image captured for the first entity line can be used as the first image. Line segments are extracted from the first image. For example, an edge detection algorithm can be used to traverse the first image to obtain a binary edge image. Then, line segments can be extracted from the binary edge image based on a line segment detection algorithm, such as the LSD (Line Segment Detector) algorithm. If multiple line segments are extracted, the longest line segment can be used as the virtual line corresponding to the first entity line. Further, the image captured for the second entity line can be used as the second image. Line segments are extracted from the second image. For example, an edge detection algorithm can be used to traverse the second image to obtain a binary edge image. Then, line segments can be extracted from the binary edge image based on a line segment detection algorithm. If multiple line segments are extracted, the longest line segment can be used as the virtual line corresponding to the second entity line. In this implementation, by performing edge detection and line segment detection on the image, the virtual lines corresponding to the first and second entity lines can be obtained. This provides a basis for judging whether the angle between the second entity line and the first entity line exceeds the corresponding angle threshold, thus enabling convenient subsequent judgment operations.

[0039] Furthermore, the direction vector of the virtual line corresponding to the first entity line can be calculated based on the endpoint coordinates of the virtual line corresponding to the first entity line, and the direction vector of the virtual line corresponding to the second entity line can be calculated based on the endpoint coordinates of the virtual line corresponding to the second entity line. Next, the dot product of these two direction vectors is calculated, which serves as the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line (hereinafter referred to as the first angle). The dot product of these two direction vectors plus π is then calculated, serving as the angle between the orthogonal directions of the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line (hereinafter referred to as the second angle).

[0040] If the second included angle is less than or equal to the first angle threshold, then it is determined that the angle between the orthogonal direction of the second entity line and the first entity line is less than or equal to the first angle threshold. If the first included angle is greater than or equal to the second angle threshold and less than or equal to the third angle threshold, then it is determined that the angle between the second entity line and the first entity line is greater than or equal to the second angle threshold and less than or equal to the third angle threshold. If the first included angle is less than or equal to the fourth angle threshold, then it is determined that the angle between the second entity line and the first entity line is less than or equal to the fourth angle threshold.

[0041] In this embodiment of the invention, by obtaining the virtual line corresponding to the first physical line and the virtual line corresponding to the second physical line, and calculating the angle between the orthogonal direction of the virtual line corresponding to the second physical line and the virtual line corresponding to the first physical line, and the angle between the virtual line corresponding to the second physical line and the virtual line corresponding to the first physical line, it is possible to detect whether the angle between the orthogonal direction of the second physical line and the first physical line is less than or equal to a first angle threshold, whether the angle between the second physical line and the first physical line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, and whether the angle between the second physical line and the first physical line is less than or equal to a fourth angle threshold. Therefore, detection efficiency can be ensured to a certain extent.

[0042] The embodiments of the present invention may further include the following steps: Step S41: Obtain the images corresponding to the first entity line and the second entity line.

[0043] Accordingly, the detection that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the orthogonal direction of the first entity line is less than or equal to the first angle threshold may specifically include: determining, based on the image, that the angle between the second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold.

[0044] The above-mentioned detection that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold may specifically include: determining, based on the image, that the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold.

[0045] The detection mentioned above, where the angle between the second entity line and the first entity line in the physical environment of the map to be constructed is less than or equal to a fourth angle threshold, can specifically include: determining, based on the image, that the angle between the second entity line and the first entity line is less than or equal to a fourth angle threshold.

[0046] The image corresponding to the first entity line is an image obtained by taking a picture of the first entity line (i.e., the first image mentioned above), and the image corresponding to the second entity line is an image obtained by taking a picture of the second entity line (i.e., the second image mentioned above). The image can be an RGB or RGBD image.

[0047] An angle prediction model can be pre-trained using a first training sample. This first training sample can include sample image pairs and actual angles. The sample image pairs consist of images captured by a camera of different entity lines, and the actual angle is the angle between the two entity lines. The sample image pairs can be input into the neural network model to be trained. Based on the model's output and the actual angles in the first training samples, the model loss value is calculated. The model parameters are optimized based on the loss value until the model converges, obtaining the final angle prediction model. Correspondingly, the image corresponding to the first entity line and the image corresponding to the second entity line can be used as a prediction sample pair and input into the angle prediction model. The model's output is used as the angle between the second entity line and the first entity line. The model's output + π is used as the angle in the orthogonal direction between the second entity line and the first entity line. In this way, the angle can be conveniently obtained through model prediction, thereby improving detection efficiency to some extent. Detection is achieved by comparing the angle between the second solid line and the first solid line, and the angle between the second solid line and the orthogonal direction of the first solid line, with the corresponding angle thresholds (i.e., the first angle threshold, the second angle threshold, the third angle threshold, and the fourth angle threshold mentioned above).

[0048] In this embodiment of the invention, by acquiring images corresponding to the first entity line and the second entity line, since the acquired images can represent the first entity line and the second entity line, based on the acquired images, it is detected whether the angle between the orthogonal direction of the second entity line and the first entity line is less than or equal to a first angle threshold, whether the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, and whether the angle between the second entity line and the first entity line is less than or equal to a fourth angle threshold. Therefore, the detection accuracy can be ensured to a certain extent.

[0049] In another implementation, a second training sample can be used to pre-train the angle prediction model. The second training sample can include sample point cloud pairs and actual angles. The sample point cloud pairs include point cloud frames obtained by LiDAR scanning different entity lines, and the actual angle is the angle between these two entity lines. The sample point cloud pairs can be input into the neural network model to be trained. Based on the model's output and the actual angle in the first training sample, the model loss value is calculated. The model parameters are optimized based on the model loss value until the model converges, obtaining the final angle prediction model. Correspondingly, the point cloud frames corresponding to the first entity line and the second entity line can be used as a pair of samples to be predicted and input into the angle prediction model. The output of this model is used as the angle between the second entity line and the first entity line. The output of the model plus π is used as the angle in the orthogonal direction between the second entity line and the first entity line. In this way, the angle can be conveniently obtained through model prediction, thereby improving detection efficiency to some extent. Detection is achieved by comparing the angle between the second solid line and the first solid line, and the angle between the second solid line and the orthogonal direction of the first solid line, with the corresponding angle threshold.

[0050] Optionally, the step of controlling the display device to display the first display line in the embodiments of the present invention may specifically include: Step 1011: Determine the main direction of the physical environment of the map to be built based on the multiple structural line segments of the physical environment to be built, wherein the angles between the multiple structural line segments and the first entity line are all less than or equal to a preset angle threshold.

[0051] Step 1012: Control the display device to display the first display line according to the main direction of the environment, wherein the direction of the first display line is the same as the main direction of the environment.

[0052] In this embodiment of the invention, multiple structural line segments of the physical environment to be mapped can be determined based on multi-frame data collected by an environmental sensor. When the environmental sensor is a lidar, multiple point cloud frames can be acquired, and edge points in each of these point cloud frames can be extracted. The method for extracting edge points from point cloud frames can refer to the previously described method for extracting edge points from the first point cloud frame, and will not be repeated here. Next, the edge points in each point cloud frame are clustered, and line segment fitting is performed on each cluster to obtain multiple virtual lines corresponding to the multiple point cloud frames.

[0053] When the environmental sensor is a camera, multiple frames of images captured by the environmental sensor can be acquired, and line segments in each of these multiple frames can be extracted as multiple virtual lines corresponding to the multiple image frames. Among them, one of the multiple virtual lines represents a structural line segment (i.e., a solid line) of the physical environment to be mapped. The implementation method for extracting line segments from the images can refer to the implementation method for extracting line segments from the first image described above, and will not be repeated here.

[0054] Next, for any virtual line among the extracted virtual lines, the direction vector of the virtual line corresponding to the first entity line can be calculated based on the endpoint coordinates of the virtual line corresponding to the first entity line. The dot product of these two direction vectors is then calculated as the angle between the structure segment represented by the virtual line and the first entity line. If this angle is less than or equal to a preset angle threshold, the virtual line is added to the line segment set. The preset angle threshold can be pre-set; for example, it can be greater than 0 degrees and less than or equal to 10 degrees, and can be any value from (0°, 10°). If the angle is less than or equal to the preset angle threshold, it indicates that the structure segment is parallel to the first entity line. When the number of virtual lines included in the line segment set reaches a preset number threshold, the entity lines represented by the virtual lines included in the line segment set are considered as multiple structure line segments of the physical environment to be mapped.

[0055] Directional clustering can be performed based on the direction vectors of the virtual lines included in the line segment set, and the direction represented by the center vector of the cluster containing the most lines can be taken as the main direction of the environment. Alternatively, the mean of the direction vectors of the virtual lines included in the line segment set can be calculated, and the direction represented by the average direction vector can be taken as the main direction of the environment. Furthermore, the display device can be controlled to display the first display line corresponding to the first physical line along the main direction of the environment, so that the direction of the first display line is the same as the main direction of the environment.

[0056] In this embodiment of the invention, the main direction of the physical environment to be mapped is determined based on multiple structural line segments, wherein the angles between the multiple structural line segments and the first solid line are all less than or equal to a preset angle threshold. Next, based on the main direction of the environment, the display device is controlled to display a first display line, ensuring that the direction of the first display line is the same as the main direction of the environment. This ensures that the displayed first display line aligns with the main direction of the physical environment to be mapped, thereby making the created map more regular.

[0057] Optionally, embodiments of the present invention may further include: Step S51: In response to detecting that the angle between the second entity line and the first entity line in the orthogonal direction is greater than the first angle threshold, and the angle between the second entity line and the first entity line is greater than the fourth angle threshold, directly control the display device to display the second display line corresponding to the second entity line, wherein the second display line is not parallel and not orthogonal to the first display line.

[0058] Alternatively, in step S52, in response to detecting that the angle between the second entity line and the first entity line is less than the second angle threshold or greater than the third angle threshold, and the angle between the second entity line and the first entity line is greater than the fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is not parallel and not orthogonal to the first display line.

[0059] In this embodiment of the invention, the angle between the orthogonal directions of the second entity line and the first entity line, and the angle between the second entity line and the first entity line can be determined with reference to the foregoing description. If the angle between the orthogonal directions of the second entity line and the first entity line is greater than a first angle threshold, and the angle between the second entity line and the first entity line is greater than the fourth angle threshold, it indicates that the second entity line and the first entity line are neither perpendicular nor parallel in the physical environment of the map to be constructed. The corresponding entity structures of the second entity line and the first entity line are visually non-perpendicular and non-parallel when viewed by the user's eye. Therefore, the display device can be controlled to display the second display line corresponding to the second entity line in a manner that is neither parallel nor orthogonal to the first display line.

[0060] If the angle between the second entity line and the first entity line is less than a second angle threshold or greater than a third angle threshold, and the angle between the second entity line and the first entity line is greater than a fourth angle threshold, this indicates that the second entity line and the first entity line in the physical environment to be mapped are neither perpendicular nor parallel. The corresponding entity structures of the second and first entity lines appear to be neither perpendicular nor parallel to the user's eye. Therefore, the display device can be controlled to display the second display line corresponding to the second entity line in a way that is neither parallel nor orthogonal to the first display line. This ensures that the non-parallel and non-orthogonal entity structures are correctly displayed on the map, ensuring the map's accuracy.

[0061] Optionally, in another implementation, the step of controlling the display device to display the first display line in the embodiments of the present invention may specifically include: Step 1013: According to the line segment direction angle of the line segment in the multiple key point cloud frames, divide the line segment into the angle interval to which the line segment direction angle belongs.

[0062] Step 1014: Determine the total length of the line segments included in each angle interval, and use it as the cumulative length of each angle interval.

[0063] Step 1015: When the maximum cumulative length and the second largest cumulative length meet the preset length requirements, determine the main environmental direction of the physical environment to be mapped based on the angle represented by the angle interval with the largest cumulative length.

[0064] Step 1016: Control the display device to display the first display line according to the main direction of the environment, wherein the direction of the first display line is the same as the main direction of the environment.

[0065] In this context, key point cloud frames are representative point cloud frames selected from point cloud frames acquired by LiDAR during the mapping process for map construction. The specific number of multiple key point cloud frames can be preset as needed, and this embodiment of the invention does not impose any limitation on this. For example, let m represent the specific number, and a sliding window can be used to traverse the key point cloud frame sequence. Traversal can begin when the number of existing key point cloud frames is greater than or equal to m. The length of the sliding window is greater than 2, meaning that one window includes multiple key point cloud frames, and the key point cloud frame sequence includes all key point cloud frames accumulated during the mapping process. For the multiple key point cloud frames currently included in the sliding window, a preset line segment extraction algorithm can be used to extract the line segments included in each key point cloud frame. Then, the line segment orientation angle of each line segment can be calculated, and the line segments are binned according to the line segment orientation angle. Each orientation bin corresponds to an angle interval, and the angle resolution of the orientation bin is... It can be the interval length of the angle interval, for example, It can be 0.1°. For the angle range of 0° to 180°, it can start from 0° and increment by 0.1°. Divide it into an angular interval. This represents the line segment direction angle of the i-th line segment (segment i), denoted by . The length of the i-th line segment is represented by the number of the directional bucket containing line segment i.

[0066] For any direction bucket, the sum of the lengths of all line segments within that direction bucket can be calculated as the cumulative length for each of the aforementioned angle intervals. For any direction bucket Steering bucket The cumulative length can be expressed as:

[0067] Indicates the opposite direction bucket Find the sum of the lengths of all line segments in the given equation.

[0068] The cumulative lengths can be sorted in descending order, with the first-ranked cumulative length being the maximum and the second-ranked cumulative length being the second-largest. Alternatively, a histogram can be constructed based on the cumulative lengths, with the cumulative length represented by the tallest rectangular bar in the histogram being the maximum and the cumulative length represented by the second-tallest rectangular bar in the histogram being the second-largest cumulative length.

[0069] The direction bucket corresponding to the maximum cumulative length is the main direction bucket (i.e., the angle range with the largest cumulative length). The main direction bucket can be:

[0070] In this embodiment of the invention, a display constraint on the main direction of the environment is added. Only if the maximum cumulative length and the second largest cumulative length meet a preset length requirement is the main direction determined based on the multiple keypoint cloud frames currently included in the sliding window considered reliable. The main direction of the physical environment to be mapped is then determined based on the angle represented by the main direction bucket. This ensures, to a certain extent, the accuracy of the determined main direction of the environment, thereby ensuring the mapping effect. Specifically, the angle represented by the main direction bucket can be calculated: ,Will The indicated direction is determined as the main environmental direction of the physical environment of the map to be constructed. Among them, The direction represented can be an angle between the horizontal and the vertical. The direction.

[0071] Conversely, if the maximum cumulative length and the second largest cumulative length do not meet the preset length requirements, it indicates that the main direction determined based on these multiple keypoint cloud frames is unreliable. Therefore, the sliding window can be moved, and the main direction of the environment can be re-determined based on the multiple keypoint cloud frames currently included in the sliding window, until the main direction of the environment is determined. In practical application scenarios, stable long line segments can represent the direction of building structures, and the longer the line segment, the better it represents the building structure. Therefore, in this embodiment of the invention, the cumulative length of line segments is used as the basis for finding the main direction of the environment, which can reduce the influence of short line noise, local structure of door frames, and dynamic objects to a certain extent.

[0072] Furthermore, the display device can be controlled to display the first display line corresponding to the first entity line along the main direction of the environment, so that the direction of the first display line is the same as the main direction of the environment.

[0073] In this embodiment of the invention, line segments are divided into angle intervals according to their direction angles in multiple keypoint cloud frames. The cumulative length of each angle interval is calculated based on the total length of the line segments included in each angle interval. When the maximum and second-largest cumulative lengths meet a preset length requirement, the main environmental direction of the physical environment to be mapped is determined based on the angle represented by the angle interval with the largest cumulative length. Thus, by calculating the main environmental direction of the physical environment to be mapped using multiple keypoint cloud frames, the main environmental direction more closely matches the actual environment of the physical environment to be mapped. Furthermore, based on the main environmental direction, the display device is controlled to display a first display line, ensuring that the direction of the first display line is the same as the main environmental direction. This further ensures that the displayed first display line matches the main direction of the physical environment to be mapped, resulting in a more regular map.

[0074] Furthermore, no additional hardware components are required. Based on the key point cloud frames collected by the existing LiDAR, the main direction of the environment can be identified, and the display device can be controlled to display the first display line according to the main direction of the environment. Therefore, the implementation cost is more controllable.

[0075] Optionally, in this embodiment of the invention, if the maximum cumulative length is greater than or equal to a preset length threshold, and the ratio of the maximum cumulative length to the second largest cumulative length is greater than or equal to a preset length ratio threshold, it can be determined that the maximum cumulative length and the second largest cumulative length meet the preset length requirement.

[0076] The preset length threshold can also be called the minimum cumulative length in the principal direction, and the preset length ratio threshold can also be called the confidence ratio of the primary and secondary peaks. The specific values ​​of the preset length threshold and the preset length ratio threshold can be set as needed, and this embodiment of the invention does not impose any restrictions on them. For example, the preset length threshold can be 6.0m, and the preset length ratio threshold can be 1.2.

[0077] by This indicates a preset length threshold. This indicates a preset length ratio threshold. Indicates the maximum cumulative length. Let represent the second largest cumulative length, then satisfying: , In this case, it can be determined that the maximum cumulative length and the second largest cumulative length meet the preset length requirements.

[0078] If the maximum cumulative length is greater than or equal to a preset length threshold, it indicates that the maximum cumulative length is sufficiently large. If the ratio of the maximum cumulative length to the second largest cumulative length is greater than or equal to a preset length ratio threshold, it indicates that the direction corresponding to the maximum cumulative length is significantly stronger than other directions. Therefore, in this embodiment of the invention, when the maximum cumulative length is greater than or equal to the preset length threshold, and the ratio of the maximum cumulative length to the second largest cumulative length is greater than or equal to the preset length ratio threshold, determining that the maximum and second largest cumulative lengths meet the preset length requirement ensures the reliability of the subsequent determination of the main environmental direction based on the angle represented by the angle interval with the largest cumulative length.

[0079] Optionally, embodiments of the present invention may further include the following steps: Step S61: Obtain the endpoint coordinates of each line segment in the multiple key point cloud frames.

[0080] Step S62: Calculate the length of each line segment based on the endpoint coordinates of each line segment.

[0081] Step S63: Based on the endpoint coordinates of each line segment, calculate the undirected direction angle of each line segment, and use it as the line segment direction angle of each line segment.

[0082] In this embodiment of the invention, line segments can be directly extracted from each of the multiple key point cloud frames to obtain the endpoint coordinates of each extracted line segment. Alternatively, for any key point cloud frame among the multiple key point cloud frames, the quality characterization information of the key point cloud frame can be obtained; the quality characterization information includes at least one of laser intensity, point cloud density, and matching confidence. If the quality characterization information of the key point cloud frame meets a preset quality requirement, then the endpoint coordinates of each line segment in the key point cloud frame are obtained.

[0083] Specifically, the value of the intensity representation field in the key point cloud frame can be read as the laser intensity of that key point cloud frame. The local matching module performs point cloud matching between this key point cloud frame and previously identified key point cloud frames, reading the point cloud density and matching score output by the matching algorithm. The matching score is the matching confidence level. If the key point cloud frame satisfies one or more of the following: laser intensity greater than or equal to a preset intensity threshold, point cloud density greater than or equal to a preset density threshold, and matching confidence level greater than or equal to a preset confidence threshold, then the quality characterization information of the key point cloud frame meets the preset quality requirements. Correspondingly, a preset line segment extraction algorithm can be used to extract the line segments included in each of these multiple key point cloud frames, as well as the endpoint coordinates of each line segment. Specifically, line segments whose feature quality meets the standards can be extracted. Understandably, in scenarios with higher quality requirements for key point cloud frames, the quality characterization information of the key point cloud frame can be determined to meet the following conditions: laser intensity greater than or equal to a preset intensity threshold, point cloud density greater than or equal to a preset density threshold, and matching confidence greater than or equal to a preset confidence threshold. In this implementation, the quality characterization information of the key point cloud frame is obtained and jointly evaluated. Only when the quality characterization information of the key point cloud frame meets the preset quality requirements are the endpoint coordinates of each line segment in the key point cloud frame obtained. This avoids the problem of low-quality point cloud frames participating in the main direction recognition, thereby reducing the reliability of the main direction recognition.

[0084] For any obtained endpoint coordinates, these coordinates can first be transformed to a map coordinate system. For example, using... Represents the endpoint coordinates of the line segment endpoints in the keypoint of the k-th frame, in ( , , () represents the global pose of the keypoint in the k-th frame. , Indicates the first The position of each keyframe in the map coordinate system Indicates the first The heading angles of each keyframe in the map coordinate system, and the global pose matrix of the keypoint data in the k-th frame are:

[0085] The endpoint coordinates can be compensated using the following formula:

[0086] in, It can The coordinate values ​​included in the coordinate system are used as endpoint coordinates for transformation to the map coordinate system.

[0087] For any line segment, after the transformation is completed, the length of the line segment can be calculated based on the coordinates of its endpoints, and the undirected direction angle of the line segment can be calculated as the line segment direction angle based on the coordinates of its endpoints.

[0088] Taking line segment i as an example, line segment i can be any one of multiple line segments. , Let represent the coordinates of the two endpoints of line segment i, then the length of line segment i can be: The original direction angle of line segment i can be: The original orientation angle can be represented as the angle between the vector (i.e., line segment i) pointing from P1 to P2 and the X-axis of the map coordinate system.

[0089] Since the direction of a line segment is not distinguished by positive or negative direction, it can be... Normalization to Obtain the undirected direction angle of line segment i:

[0090] In this embodiment of the invention, the endpoint coordinates of each line segment in multiple key point cloud frames are obtained. Based on the endpoint coordinates of each line segment, the line segment length and direction angle can be calculated, which can ensure computational efficiency to a certain extent.

[0091] Optionally, embodiments of the present invention may further include the following steps: Step S71: Control the display device to display the first display line parallel to a boundary of the display interface.

[0092] The display interface can be provided by a display device for displaying a map. The display interface can be rectangular and may include four boundaries: top, bottom, left, and right. The display device can be controlled to display the first display line parallel to one of the four boundaries. For example, the first display line can be rotated to a direction parallel to one of the boundaries. It is understood that the left and right boundaries of the display interface have the same direction, and the top and bottom boundaries have the same direction. Therefore, when the display device displays the first display line parallel to one boundary, it can actually make the first display line parallel to two boundaries of the display interface.

[0093] In this embodiment of the invention, the direction of the first display line is the same as the main direction of the environment. Furthermore, by controlling the display device to make the first display line parallel to one of the boundaries of the display interface, the main direction of the environment on which the displayed map is based is parallel to one of the boundaries of the display interface. This can avoid the problem of the map being tilted as a whole in the final display interface, making the map more in line with the user's actual viewing habits.

[0094] Optionally, embodiments of the present invention may further include the following steps: Step S81: After completing the drawing, control the display device to display the correction confirmation option.

[0095] Step S82: In response to receiving a cancellation operation for the correction confirmation option, control the display device to display the first display line as non-parallel to the boundary.

[0096] The specific form of the correction confirmation option can be set as needed, and this embodiment of the invention does not impose any restrictions on it. For example, the correction confirmation option can be in the form of a pop-up window, including the text "Please confirm whether to correct" and a confirm button and a cancel button. Alternatively, it can include the text "Please confirm whether to correct" and an accept button and an undo button.

[0097] The cancellation operation for the correction confirmation option can be a click, long press, or other action on the cancel / undo button. Correspondingly, if a cancellation operation for the correction confirmation option is received, the display device can be controlled to display the first display line in a direction that is not parallel to the boundary of the display interface. For example, the first display line can be rotated in the opposite direction to be parallel to the boundary. Suppose that during correction, the first display line is rotated 5° clockwise to make it parallel to the right boundary of the display interface; then, in response to receiving a cancellation operation, the display device can be controlled to rotate the first display line 5° counterclockwise to make it non-parallel to the right boundary of the display interface. After the first display line is restored, the created map is saved.

[0098] Of course, in response to receiving a confirmation action for the correction confirmation option, the created map can be saved directly. The cancellation action for the correction confirmation option can be a click, long press, or other similar action on the confirm / accept button.

[0099] In this embodiment of the invention, after mapping is completed, the control display device displays a correction confirmation option. In response to receiving a cancellation operation for the correction confirmation option, the control display device displays the first display line as non-parallel to the boundary. This allows the user to conveniently choose whether to correct the first display line to be parallel to the boundary, thereby improving operational convenience and ensuring that the final map better meets the user's actual needs.

[0100] Optionally, the step of controlling the display device to display the first display line parallel to a boundary of the display interface in the above embodiments of the present invention may specifically include: Step S81a: Select the candidate reference angle with the smallest angular difference from the direction represented by the first display line; the candidate reference angle includes at least the different angles corresponding to the boundaries of the display interface.

[0101] Step S81b: Based on the selected candidate reference angle, control the display device to display the first display line so that the first display line is parallel to a boundary of the display interface.

[0102] In this embodiment of the invention, the direction represented by the first display line is the main direction of the environment, and the direction represented by the first display line corresponds to the above. The different angles corresponding to the boundaries of the display interface can include 0 and... There are two types of candidate reference angles, which can represent the horizontal coordinate axis and the vertical coordinate axis, respectively, corresponding to the horizontal boundary (e.g., the upper boundary and the lower boundary) and the vertical boundary (e.g., the left boundary and the right boundary).

[0103] For any candidate reference angle, it can be based on the candidate reference angle and... The undirected angular difference between the candidate reference angle and the angle representing the direction indicated by the first display line is calculated. For example, the two undirected directional angles... The undirected angular difference between them can be:

[0104] Furthermore, a candidate reference angle with the smallest angular difference can be selected. Based on the selected candidate reference angle, the display device is controlled to display a first display line, such that the first display line is parallel to the boundary corresponding to the selected candidate reference angle. The angle corresponding to the coordinate axis closest to the direction represented by the first display line. It can be represented as:

[0105] Right now From 0, The value selected is the one that minimizes the angle difference.

[0106] Assuming the selected candidate reference angle is 0, the display device can be controlled to display the first display line along the directions of the upper and lower boundaries, making the first display line parallel to the upper and lower boundaries. Assuming the selected candidate reference angle is... This allows the display device to display the first display line along the left and right boundaries, making the first display line parallel to the left and right boundaries. This ensures that the first display line on the map is parallel to the boundaries of the real-world interface, and the second display line corresponding to the entity line parallel or orthogonal to the first display line is displayed parallel or orthogonal to the first display line. Compared to simply rotating the entire map on the display end after map creation, which only changes the viewing angle and doesn't modify the direction of the displayed lines, this embodiment of the invention makes the final map layout more regular, thus ensuring map quality to a certain extent.

[0107] In this embodiment of the invention, the candidate reference angles include at least the different angles corresponding to the boundaries of the display interface. The candidate reference angle with the smallest angle difference from the direction represented by the first display line is selected, and the display device is controlled to display the first display line according to the selected candidate reference angle, so that the displayed first display line is parallel to the boundary corresponding to the selected candidate reference angle. In this way, by displaying the first display line based on the candidate reference angle with the smallest angle difference, the adjustment amount of the first display line is minimized while ensuring that the first display line is parallel to the boundary in the display device.

[0108] Optionally, in this embodiment of the invention, the step of controlling the display device to display the first display line based on the selected candidate reference angle, so that the first display line is parallel to a boundary of the display interface, may specifically include: Step S81b1: Determine the amount of rotation based on the selected candidate reference angle and the angle of the direction represented by the first display line.

[0109] Step S81b2: Correct the pose of the existing key point cloud frame according to the rotation amount so that the first display line is parallel to the boundary corresponding to the selected candidate reference angle.

[0110] In this embodiment of the invention, the difference between the selected candidate reference angle and the angle represented by the first display line can be calculated, and the minimum rotation amount required to rotate from the direction represented by the first display line to the direction represented by the selected candidate reference angle can be determined based on this difference. For example, the minimum rotation amount... It can be as follows:

[0111] in, Used to convert the angle difference to the minimum rotation amount in the sense of an undirected line segment.

[0112] Furthermore, the existing keypoint cloud frames include all keypoint cloud frames accumulated during the mapping process, and the currently displayed map is created based on these existing keypoint cloud frames. The poses of the existing keypoint cloud frames are corrected according to the rotation amount, so that the map is created with the minimum rotation amount. Rotation, and minimum amount of rotation. It is the amount of rotation required to rotate from the direction represented by the first display line to the direction represented by the selected candidate reference angle. Therefore, through pose correction, the first display line in the map is made parallel to the boundary corresponding to the selected candidate reference angle, thus achieving automatic alignment.

[0113] In this embodiment of the invention, the rotation amount is determined based on the selected candidate reference angle and the angle of the direction represented by the first display line. The pose of the existing keypoint cloud frame is corrected according to the rotation amount, ensuring that the first display line is parallel to the boundary corresponding to the selected candidate reference angle. This ensures that the final displayed first display line is parallel to the boundary, and that the displayed map better matches the user's actual viewing habits.

[0114] Optionally, the step of correcting the pose of the existing keypoint cloud frame according to the rotation amount may specifically include: Step S81b21: For any keypoint cloud frame in the existing keypoint cloud frames, based on the first coordinate, the deviation of the second coordinate relative to the first coordinate, and the rotation amount, generate a corrected second coordinate for the keypoint cloud frame; the second coordinate is the coordinate in the pose of the keypoint cloud frame, and the first coordinate is the coordinate in the pose of the first keypoint cloud frame.

[0115] Step S81b23: Based on the heading angle in the pose of the key point cloud frame and the rotation amount, generate a corrected heading angle for the key point cloud frame.

[0116] In this embodiment of the invention, the existing key point cloud frames can be divided into the first key point cloud frame and non-first key point cloud frames. The first coordinate represents the pose of the first keypoint cloud frame (i.e., the first coordinate). The first keypoint cloud frame can be used as the anchor point. Let p represent the pose of any existing keypoint cloud frame. The corrected second coordinate... It can be:

[0117] in, , All include x-values ​​and y-values. This represents a two-dimensional rotation matrix constructed based on rotation amounts. It can be understood that, for the first keypoint cloud frame in an existing keypoint cloud dataset, The value of and Similarly, the coordinates of the first keypoint cloud frame remain the same after the correction as before the correction; that is, the coordinates of the first keypoint cloud frame remain unchanged.

[0118] For any key point cloud frame in the existing key point cloud frames, This represents the heading angle in the pose of the keypoint cloud frame, and the corrected heading angle. It can be:

[0119] in, This represents the heading angle in the pose of the keypoint cloud frame. With rotation amount The sum is normalized to map to the interval [-π, π].

[0120] In this embodiment of the invention, the first keypoint cloud frame serves as the anchor point, and its position remains unchanged before and after rotation, while the heading angle increases by the same amount of rotation. To maintain the relative orientation relationship with other key point cloud frames.

[0121] In this embodiment of the invention, a corrected second coordinate is generated for the keypoint cloud frame based on the deviation of the first coordinate and the second coordinate relative to the first coordinate, as well as the rotation amount. A corrected heading angle is generated for the keypoint cloud frame based on the heading angle and rotation amount in the pose of the keypoint cloud frame. In this way, the coordinates of the first keypoint cloud frame are used as anchor points, and the coordinates of the keypoint cloud frame are corrected according to the rotation amount, thereby avoiding unnecessary overall translation during rotation.

[0122] Optionally, embodiments of the present invention may further include the following steps: Step S91: For any newly added key point cloud frame, calculate the angle difference between the longest line segment of the key point cloud frame and the orthogonal direction represented by the first display line as the first angle difference, and calculate the angle difference between the longest line segment and the direction represented by the first display line as the second angle difference.

[0123] Step S92: If the first angle difference is less than or equal to the first angle threshold, then it is determined that the angle between the detected second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold.

[0124] Step S93: If the second angle difference is less than or equal to the fourth angle threshold, then it is determined that the angle between the second entity line and the first entity line is less than or equal to the fourth angle threshold.

[0125] The newly added key point cloud frames can be selected from point cloud frames acquired by LiDAR. Line segments can be extracted from the newly added key point cloud frames, and the length of each line segment can be calculated. The longest line segment is selected based on its length. The direction angle of the longest line segment is denoted as... The angle of the direction represented by the first display line is Correspondingly, the second angular difference can be The first angle difference can be .

[0126] In this implementation, if the first angle difference is less than or equal to the first angle threshold, it can be determined that the angle between the detected second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold. If the second angle difference is less than or equal to the fourth angle threshold, it can be determined that the angle between the detected second entity line and the first entity line is less than or equal to the fourth angle threshold.

[0127] In this embodiment of the invention, by calculating the angle difference between the longest line segment of the key point cloud frame and the orthogonal direction represented by the first display line, and by calculating the angle difference between the longest line segment and the direction represented by the first display line, the relative positional relationship between the second entity line and the first entity line can be detected based on the relationship between the calculated angle difference and the first angle threshold and the fourth angle threshold, which can ensure detection efficiency to a certain extent.

[0128] Optionally, embodiments of the present invention may further include the following steps: Step S94: For any newly added key point cloud frame, select the minimum angle difference from the first angle difference and the second angle difference.

[0129] Step S95: Correct the pose of the key point cloud frame based on the minimum angle difference, and insert the key point cloud frame into the map according to the corrected pose, so that the displayed second display line is orthogonal to the first display line or the second display line is parallel to the first display line.

[0130] In one implementation, the first angle threshold and the fourth angle threshold can be set to be the same. The first angle threshold and the fourth angle threshold can be regarded as preset angle thresholds for triggering the adsorption direction. , The specific value can be set as needed; for example, It can be Degrees. Accordingly, if the first angle difference is less than or equal to the first angle threshold or the second angle difference is less than or equal to the fourth angle threshold, then it means that the first angle difference and the second angle difference satisfy the directional adsorption condition: Therefore, the pose of the newly added key point cloud frame can be corrected based on the minimum angle difference to achieve directional snapping, so that lines close to the direction represented by the first display line or a direction orthogonal to the direction represented by the first display line are adjusted to a direction parallel to or orthogonal to the first display line. In this embodiment of the invention, it is possible to determine After that, record This is for use during subsequent orientation snapping. If the orientation snapping conditions are not met, it can be processed according to the conventional mapping process to avoid forcibly snapping all lines, which could lead to incorrect map corrections for irregular scenes (e.g., non-orthogonal walls, curved walls, open areas).

[0131] For example, Figure 2 This is a schematic diagram of the system architecture provided in an embodiment of the present invention, such as... Figure 2 As shown, after the LiDAR acquires a frame of point cloud data, it sends the acquired point cloud data to the local matching module, which estimates the pose of the current point cloud data based on the pose of the previous frame. Then, the global matching module determines whether to select the current frame as a key point cloud frame based on the quality index of the point cloud data (which can be generated by the local matching module), the robot's movement distance, and a preset keyframe selection strategy. The line segment feature extraction module extracts the endpoint coordinates of line segments from multiple key point cloud frames and converts them to the map coordinate system, calculating the line segment length and direction angle based on the endpoint coordinates. The main direction statistics module divides line segments into direction buckets according to the line segment direction angles of multiple key point cloud frames, determining the cumulative length of each direction bucket. The main direction reliability judgment module determines whether the maximum and second-largest cumulative lengths meet preset length requirements. If they do, the main direction determined based on the multiple key point cloud frames currently included in the sliding window is reliable. Therefore, the main direction of the physical environment to be mapped can be determined based on the angle represented by the main direction bucket. If the conditions are not met, it means that the main direction determined based on the multiple key point cloud frames currently included in the sliding window is not reliable. You can return to the line segment feature extraction module, move the sliding window, and reprocess the multiple key point cloud frames currently included in the sliding window to continue searching for the main direction of the environment.

[0132] The orthogonal coordinate system automatic alignment module calculates the minimum rotation required to rotate the principal orientation of the environment to the coordinate axis represented by the closest candidate reference angle. The correction module corrects the pose of existing keypoint cloud frames according to this minimum rotation. Next, the keyframe long-line snapping module selects the minimum angle difference and corrects the pose of newly added keypoint cloud frames based on this minimum angle difference. Finally, the map building module inserts the newly added keypoint cloud frames into the map according to the corrected poses to generate a point cloud map or a raster map.

[0133] In this embodiment of the invention, for any newly added keypoint cloud frame, the minimum angle difference is selected from the first angle difference and the second angle difference. The pose of the keypoint cloud frame is corrected based on the minimum angle difference, and the newly added keypoint cloud frame is inserted into the map according to the corrected pose. In this way, the correction operation continues to act on subsequent mapping processes, thereby ensuring the mapping effect.

[0134] Optionally, in this embodiment of the invention, the step of correcting the pose of the key point cloud frame based on the minimum angle difference may specifically include: Step S95a: Determine the target heading angle for the key point cloud frame based on the heading angle in the pose of the key point cloud frame and the minimum angle difference.

[0135] Step S95b: Use the target heading angle and the preset constraint matrix as input to the preset optimizer, and control the preset optimizer to output a heading angle that matches the target heading angle for the key point cloud frame.

[0136] by This represents the heading angle in the pose of the newly added keypoint cloud frame, in order to... To represent the minimum angular difference, the target heading angle can be:

[0137] The preset optimizer can be the global optimizer used in the mapping process. The global optimizer is used to optimize the pose of keypoint cloud frames in the global optimization graph. A node in the global optimization graph represents the pose of a keypoint cloud frame. Each time the global matching module selects a new keypoint cloud frame, it can insert that keypoint cloud frame into the global optimization graph. For example, the heading angle in the pose of a newly added keypoint cloud frame can be adjusted to the target heading angle before its pose is inserted into the global optimization graph, thus providing input to the preset optimizer.

[0138] The preset constraint matrix can be a pre-set information matrix of unary constraints, also known as a confidence matrix of graph optimization constraints; a larger value indicates a stronger constraint. In this embodiment of the invention, a constraint matrix representing weak translation constraints and strong angle constraints can be set to instruct the global optimizer to adjust the coordinates in the pose of the newly added keypoint cloud frame, while minimizing the adjustment of the heading angle. This ensures that the preset optimizer outputs a heading angle that matches the target heading angle for the newly added keypoint cloud frame. This improves the regularity of the displayed lines while avoiding excessive stretching, thus preventing local distortion.

[0139] Preset optimization, the preset constraint matrix can be:

[0140] in, The standard deviation representing the translation constraint can, for example, be: Translation dimension , A larger value indicates a stronger translation constraint. The standard deviation of the angular constraint can, for example, be represented as... Degree. Angle and dimension. A larger value indicates a stronger angular constraint.

[0141] In this embodiment of the invention, a target heading angle is determined for the keypoint cloud frame based on the heading angle and minimum angle difference in the pose of the keypoint cloud frame. The target heading angle and a preset constraint matrix are used as inputs to a preset optimizer, which outputs a heading angle that matches the target heading angle for the keypoint cloud frame. This adjusts the heading angle of the newly added keypoint cloud frame to the target heading angle, thereby causing the second display line ultimately displayed based on the newly added keypoint cloud frame to be snapped to a direction parallel or orthogonal to the first display line. This results in a more regular line structure displayed on the map and improves map quality.

[0142] In this embodiment of the invention, the corrected pose of the existing key point cloud frame is written back to the key frame buffer area, the local optimizer, and the global optimizer, which record the pose, to achieve synchronous updates, so that subsequent steps are processed based on the corrected pose.

[0143] Figure 3 This is a schematic diagram of a processing flow provided by an embodiment of the present invention, such as... Figure 3 As shown, it can receive point cloud data collected by LiDAR and select key point cloud frames from it. When the number of key point cloud frames is greater than or equal to m, the main orientation of the physical environment to be mapped is determined. Based on the angle corresponding to the main orientation and candidate reference angles, a minimum rotation amount is determined, and the pose of existing key point cloud frames is corrected according to the rotation amount so that the first display line is parallel to the boundary corresponding to the selected candidate reference angle. Then, for any newly added key point cloud frame, if the newly added key point cloud frame satisfies the orientation snapping condition, the minimum angle difference is determined, and the pose of the newly added key point cloud frame is corrected based on the minimum angle difference. After the mapping is completed, the final map is obtained.

[0144] During the mapping process, the pose of key point cloud frames may deviate due to factors such as low-cost LiDAR, cumulative errors of wheeled odometers, local occlusion, and dynamic objects. In this embodiment of the invention, the pose of key point cloud frames is corrected to make the pose more accurate. At the same time, directional snapping is performed on newly added key point cloud frames to maintain the continuity of correction and avoid local drift problems after automatic alignment.

[0145] Optionally, in this embodiment of the invention, the candidate reference angle also includes an angle corresponding to a specified direction; wherein, the angle corresponding to the specified direction can be set as needed, for example, the angle corresponding to the specified direction may include a 30° direction, a 45° direction, etc. When the selected candidate reference angle is the angle corresponding to the specified direction, the first display line is parallel to the specified direction. In this way, by setting more candidate reference angles, more directions available for alignment are expanded, thereby enabling the solution to adapt to complex scenarios, such as L-shaped corridors, multi-floor connecting areas, or partially non-orthogonal room scenarios.

[0146] Optionally, the embodiments of the present invention may further include: in response to the robot's current position in the physical environment to be mapped, where the distance from the starting position is less than or equal to a preset distance threshold, controlling the display device to merge the displayed third display line and the fourth display line into a fifth display line; wherein the third display line is the line corresponding to the current position, the fourth display line is the line corresponding to the starting position, and the third display line and the fourth display line are parallel.

[0147] In this embodiment of the invention, the starting position can be the position where the robot begins mapping. If the distance between the robot's current position in the physical environment to be mapped and the starting position is less than or equal to a preset distance threshold, a loop can be considered to have occurred. The third and fourth display lines may be the same as the previously displayed first display line or the same as the second display line. Since the first display line displayed on the map is in the same direction as the main direction, when the second entity line is orthogonal to the first entity line, the displayed second display line will be controlled to be orthogonal to the first display line; when the second entity line is parallel to the first entity line, the displayed second display line will be controlled to be parallel to the first display line. Therefore, when a loop is triggered, the line corresponding to the starting position is often parallel to the line corresponding to the current position. Accordingly, the parallel third and fourth display lines can be merged into one line (i.e., the fifth display line). For example, the starting point of the third display line can be connected to the ending point of the fourth display line to achieve merging. Alternatively, when the third and fourth display lines are parallel in the horizontal direction, the starting point and ending point of one of the third and fourth display lines can be moved to a position aligned with the other line and connected to achieve merging.

[0148] In this embodiment of the invention, if a loop closure is detected, the main orientation of the environment can be re-determined based on all key point cloud frames, and the pose of all key point cloud frames can be re-optimized based on the main orientation of the environment to achieve full-image optimization.

[0149] In this embodiment of the invention, in response to the robot's current position in the physical environment to be mapped, if the distance between the robot and the starting position is less than or equal to a preset distance threshold, the control display device merges the third and fourth display lines into a fifth display line. This can avoid the problem of abnormally broken lines in the map and further improve the map quality.

[0150] Figure 4 This is a block diagram of a map processing apparatus provided in an embodiment of this application, the apparatus comprising: Memory 201 and processor 202; The memory 201 is used to store program code; The processor 202 calls the program code, which, when executed, performs the following operations: during the robot mapping process, controls the display device to display a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped; In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; or In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, the second display line being orthogonal to the first display line.

[0151] In summary, in the map processing apparatus provided in this application embodiment, during the robot mapping process, the display device is controlled to display a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, which is orthogonal to the first display line. Alternatively, in response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, which is orthogonal to the first display line. Thus, for the first and second entity lines of two mutually perpendicular entity structures, the second display line displayed by the display device is orthogonal to the first display line, which avoids the situation where mutually perpendicular entity structures are displayed skewed or non-orthogonal in the map, thereby improving map quality.

[0152] Optionally, the processor 202 is further configured to perform: In response to detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, the second display line being parallel to the first display line.

[0153] Optionally, the first angle threshold is greater than 0 degrees and less than or equal to 10 degrees; The second angle threshold is greater than or equal to 80 degrees and less than 90 degrees; The third angle threshold is greater than or equal to 90 degrees and less than or equal to 100 degrees; The fourth angle threshold is greater than 0 degrees and less than or equal to 10 degrees.

[0154] Optionally, the processor 202 is further configured to perform: Obtain the virtual line corresponding to the first physical line and the virtual line corresponding to the second physical line; The step of detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is less than or equal to a first angle threshold includes: detecting that the angle between the orthogonal direction of the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to the first angle threshold; the step of detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold includes: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold; the step of detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold includes: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to a fourth angle threshold. or, Optionally, the processor 202 is further configured to perform: Obtain the images corresponding to the first entity line and the second entity line; The step of detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is less than or equal to a first angle threshold includes: determining, based on the image, that the angle between the orthogonal direction of the second entity line and the first entity line is less than or equal to the first angle threshold; the step of detecting that the angle between the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold includes: determining, based on the image, that the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold; the step of detecting that the angle between the second entity line and the first entity line in the physical environment to be mapped is less than or equal to a fourth angle threshold includes: determining, based on the image, that the angle between the second entity line and the first entity line is less than or equal to a fourth angle threshold.

[0155] Optionally, the processor 202 is further configured to perform: Based on multiple structural line segments of the physical environment to be built, the main direction of the physical environment to be built is determined, wherein the angles between the multiple structural line segments and the first entity line are all less than or equal to a preset angle threshold. Based on the main direction of the environment, the display device is controlled to display the first display line, wherein the direction of the first display line is the same as the main direction of the environment.

[0156] Optionally, the processor 202 is further configured to perform: In response to detecting that the angle between the second entity line and the orthogonal direction of the first entity line is greater than the first angle threshold, and the angle between the second entity line and the first entity line is greater than a fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is neither parallel nor orthogonal to the first display line; or In response to detecting that the angle between the second entity line and the first entity line is less than the second angle threshold or greater than the third angle threshold, and the angle between the second entity line and the first entity line is greater than the fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is not parallel and not orthogonal to the first display line.

[0157] Optionally, the processor 202 is further configured to perform: The display device is controlled to display the first display line parallel to a boundary of the display interface.

[0158] Optionally, the processor 202 is further configured to perform: After the mapping is completed, the display device is controlled to display the correction confirmation option; In response to receiving a cancellation operation for the correction confirmation option, the display device is controlled to display the first display line as non-parallel to the boundary.

[0159] Optionally, the processor 202 is further configured to perform: The candidate reference angle with the smallest angular difference from the direction represented by the first display line is selected; the candidate reference angle includes at least the different angles corresponding to the boundaries of the display interface. Based on the selected candidate reference angle, the display device is controlled to display the first display line so that the first display line is parallel to a boundary of the display interface.

[0160] Optionally, the processor 202 is further configured to perform: The amount of rotation is determined based on the selected candidate reference angle and the angle of the direction represented by the first display line; The pose of the existing key point cloud frame is corrected according to the rotation amount so that the first display line is parallel to the boundary corresponding to the selected candidate reference angle.

[0161] Optionally, the processor 202 is further configured to perform: For any keypoint cloud frame in the existing keypoint cloud frames, a corrected second coordinate is generated for the keypoint cloud frame based on the first coordinate, the deviation of the second coordinate relative to the first coordinate, and the rotation amount; the second coordinate is the coordinate in the pose of the keypoint cloud frame, and the first coordinate is the coordinate in the pose of the first keypoint cloud frame. Based on the heading angle in the pose of the key point cloud frame and the rotation amount, a corrected heading angle is generated for the key point cloud frame.

[0162] Optionally, the processor 202 is further configured to perform: For any newly added key point cloud frame, calculate the angle difference between the longest line segment of the key point cloud frame and the orthogonal direction represented by the first display line as the first angle difference, and calculate the angle difference between the longest line segment and the direction represented by the first display line as the second angle difference. If the first angle difference is less than or equal to the first angle threshold, then it is determined that the angle between the detected second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold. If the second angle difference is less than or equal to the fourth angle threshold, then it is determined that the angle between the second entity line and the first entity line is less than or equal to the fourth angle threshold.

[0163] Optionally, the processor 202 is further configured to perform: For any newly added key point cloud frame, select the minimum angle difference from the first angle difference and the second angle difference; The pose of the key point cloud frame is corrected based on the minimum angle difference, and the key point cloud frame is inserted into the map according to the corrected pose, so that the second display line is orthogonal to the first display line or the second display line is parallel to the first display line.

[0164] Optionally, the processor 202 is further configured to perform: Based on the heading angle in the pose of the key point cloud frame and the minimum angle difference, the target heading angle is determined for the key point cloud frame; The target heading angle and the preset constraint matrix are used as inputs to the preset optimizer, which is then controlled to output a heading angle that matches the target heading angle for the key point cloud frame.

[0165] Optionally, the processor 202 is further configured to perform: Based on the line segment direction angle of the line segment in multiple key point cloud frames, the line segment is divided into the angle interval to which the line segment direction angle belongs; The total length of the line segments included in each angle interval is determined as the cumulative length of each angle interval; When the maximum cumulative length and the second largest cumulative length meet the preset length requirement, the main environmental direction of the physical environment to be mapped is determined based on the angle represented by the angle interval with the largest cumulative length. Based on the main direction of the environment, the display device is controlled to display the first display line, wherein the direction of the first display line is the same as the main direction of the environment.

[0166] Optionally, the processor 202 is further configured to perform: Obtain the endpoint coordinates of each line segment in the multiple key point cloud frames; Calculate the length of each line segment based on the endpoint coordinates of each line segment; Based on the endpoint coordinates of each line segment, the undirected direction angle of each line segment is calculated and used as the line segment direction angle of each line segment.

[0167] Optionally, the processor 202 is further configured to perform: If the maximum cumulative length is greater than or equal to a preset length threshold, and the ratio of the maximum cumulative length to the second largest cumulative length is greater than or equal to a preset length ratio threshold, then the maximum cumulative length and the second largest cumulative length are determined to meet the preset length requirement.

[0168] Optionally, the processor 202 is further configured to perform: For any one of the plurality of key point cloud frames, obtain the quality characterization information of the key point cloud frame; the quality characterization information includes at least one of laser intensity, point cloud density and matching confidence. If the quality characterization information of the key point cloud frame meets the preset quality requirements, then the endpoint coordinates of each line segment in the key point cloud frame are obtained.

[0169] Optionally, the candidate reference angle may also include the angle corresponding to the specified direction; When the selected candidate reference angle is the angle corresponding to the specified direction, the first display line is parallel to the specified direction.

[0170] Optionally, the processor 202 is further configured to perform: In response to the robot's current position in the physical environment of the map to be built, if the distance from the starting position is less than or equal to a preset distance threshold, the display device is controlled to merge the third and fourth display lines into a fifth display line; The third display line is the line corresponding to the current position, the fourth display line is the line corresponding to the starting position, and the third display line is parallel to the fourth display line.

[0171] This application provides an electronic device, including a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface communicate with each other via the communication bus; the memory stores at least one executable instruction, which causes the processor to execute the map processing method described in the preceding embodiments.

[0172] This application provides a readable storage medium storing instructions that, when executed by one or more processors, enable the processors to perform the map processing method described in the foregoing embodiments. The readable storage medium can be one or more.

[0173] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the map processing method described in the foregoing embodiments.

[0174] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0175] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0176] It should be noted that all actions involving the acquisition of signals, information, or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization granted by the owner of the relevant device.

[0177] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0178] The various component embodiments of this application can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors can be used in practice to implement some or all of the functions of some or all of the components in the control terminal according to the embodiments of this application. This application can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such an implementation of this application can be stored on a computer-readable medium, or can take the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0179] This disclosure provides an embodiment, including the following examples.

[0180] Example 1: A map processing method, the method comprising: During the robot mapping process, the control display device displays a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; or In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, the second display line being orthogonal to the first display line.

[0181] Example 2: According to the method described in Example 1, the method further includes: In response to detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, the second display line being parallel to the first display line.

[0182] Example 3: According to the method described in Example 2, the first angle threshold is greater than 0 degrees and less than or equal to 10 degrees; The second angle threshold is greater than or equal to 80 degrees and less than 90 degrees; The third angle threshold is greater than or equal to 90 degrees and less than or equal to 100 degrees; The fourth angle threshold is greater than 0 degrees and less than or equal to 10 degrees.

[0183] Example 4: According to the method described in Example 2, the method further includes: obtaining the virtual line corresponding to the first physical line and the virtual line corresponding to the second physical line; The step of detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is less than or equal to a first angle threshold includes: detecting that the angle between the orthogonal direction of the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to the first angle threshold; the step of detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold includes: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold; the step of detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold includes: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to a fourth angle threshold. or, The method further includes: acquiring images corresponding to the first entity line and the second entity line; The step of detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is less than or equal to a first angle threshold includes: determining, based on the image, that the angle between the orthogonal direction of the second entity line and the first entity line is less than or equal to the first angle threshold; the step of detecting that the angle between the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold includes: determining, based on the image, that the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold; the step of detecting that the angle between the second entity line and the first entity line in the physical environment to be mapped is less than or equal to a fourth angle threshold includes: determining, based on the image, that the angle between the second entity line and the first entity line is less than or equal to a fourth angle threshold.

[0184] Example 5: According to the method described in Example 1, the control display device displays a first display line, including: Based on multiple structural line segments of the physical environment to be built, the main direction of the physical environment to be built is determined, wherein the angles between the multiple structural line segments and the first entity line are all less than or equal to a preset angle threshold. Based on the main direction of the environment, the display device is controlled to display the first display line, wherein the direction of the first display line is the same as the main direction of the environment.

[0185] Example 6. According to the method described in Example 1, the method further includes: In response to detecting that the angle between the second entity line and the orthogonal direction of the first entity line is greater than the first angle threshold, and the angle between the second entity line and the first entity line is greater than a fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is neither parallel nor orthogonal to the first display line; or In response to detecting that the angle between the second entity line and the first entity line is less than the second angle threshold or greater than the third angle threshold, and the angle between the second entity line and the first entity line is greater than the fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is not parallel and not orthogonal to the first display line.

[0186] Example 7. According to the method described in Example 1, the method further includes: The display device is controlled to display the first display line parallel to a boundary of the display interface.

[0187] Example 8. According to the method described in Example 7, the method further includes: After the mapping is completed, the display device is controlled to display the correction confirmation option; In response to receiving a cancellation operation for the correction confirmation option, the display device is controlled to display the first display line as non-parallel to the boundary.

[0188] Example 9: According to the method described in Example 7, controlling the display device to display the first display line parallel to a boundary of the display interface includes: The candidate reference angle with the smallest angular difference from the direction represented by the first display line is selected; the candidate reference angle includes at least the different angles corresponding to the boundaries of the display interface. Based on the selected candidate reference angle, the display device is controlled to display the first display line so that the first display line is parallel to a boundary of the display interface.

[0189] Example 10. According to the method described in Example 9, controlling the display device to display the first display line based on the selected candidate reference angle, so that the first display line is parallel to a boundary of the display interface, includes: The amount of rotation is determined based on the selected candidate reference angle and the angle of the direction represented by the first display line; The pose of the existing key point cloud frame is corrected according to the rotation amount so that the first display line is parallel to the boundary corresponding to the selected candidate reference angle.

[0190] Example 11: According to the method described in Example 10, the step of correcting the pose of the existing keypoint cloud frame according to the rotation amount includes: For any keypoint cloud frame in the existing keypoint cloud frames, a corrected second coordinate is generated for the keypoint cloud frame based on the first coordinate, the deviation of the second coordinate relative to the first coordinate, and the rotation amount; the second coordinate is the coordinate in the pose of the keypoint cloud frame, and the first coordinate is the coordinate in the pose of the first keypoint cloud frame. Based on the heading angle in the pose of the key point cloud frame and the rotation amount, a corrected heading angle is generated for the key point cloud frame.

[0191] Example 12: According to the method described in Example 2, the method further includes: For any newly added key point cloud frame, calculate the angle difference between the longest line segment of the key point cloud frame and the orthogonal direction represented by the first display line as the first angle difference, and calculate the angle difference between the longest line segment and the direction represented by the first display line as the second angle difference. If the first angle difference is less than or equal to the first angle threshold, then it is determined that the angle between the detected second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold. If the second angle difference is less than or equal to the fourth angle threshold, then it is determined that the angle between the second entity line and the first entity line is less than or equal to the fourth angle threshold.

[0192] Example 13. According to the method described in Example 12, the method further includes: For any newly added key point cloud frame, select the minimum angle difference from the first angle difference and the second angle difference; The pose of the key point cloud frame is corrected based on the minimum angle difference, and the key point cloud frame is inserted into the map according to the corrected pose, so that the second display line is orthogonal to the first display line or the second display line is parallel to the first display line.

[0193] Example 14: According to the method described in Example 13, the step of correcting the pose of the key point cloud frame based on the minimum angle difference includes: Based on the heading angle in the pose of the key point cloud frame and the minimum angle difference, the target heading angle is determined for the key point cloud frame; The target heading angle and the preset constraint matrix are used as inputs to the preset optimizer, which is then controlled to output a heading angle that matches the target heading angle for the key point cloud frame.

[0194] Example 15: According to the method described in Example 1, the control display device displays a first display line, including: Based on the line segment direction angle of the line segment in multiple key point cloud frames, the line segment is divided into the angle interval to which the line segment direction angle belongs; The total length of the line segments included in each angle interval is determined as the cumulative length of each angle interval; When the maximum cumulative length and the second largest cumulative length meet the preset length requirement, the main environmental direction of the physical environment to be mapped is determined based on the angle represented by the angle interval with the largest cumulative length. Based on the main direction of the environment, the display device is controlled to display the first display line, wherein the direction of the first display line is the same as the main direction of the environment.

[0195] Example 16. According to the method described in Example 15, the method further includes: Obtain the endpoint coordinates of each line segment in the multiple key point cloud frames; Calculate the length of each line segment based on the endpoint coordinates of each line segment; Based on the endpoint coordinates of each line segment, the undirected direction angle of each line segment is calculated and used as the line segment direction angle of each line segment.

[0196] Example 17. According to the method described in Example 15, the method further includes: If the maximum cumulative length is greater than or equal to a preset length threshold, and the ratio of the maximum cumulative length to the second largest cumulative length is greater than or equal to a preset length ratio threshold, then the maximum cumulative length and the second largest cumulative length are determined to meet the preset length requirement.

[0197] Example 18: According to the method described in Example 16, obtaining the endpoint coordinates of each line segment in the plurality of key point cloud frames includes: For any one of the plurality of key point cloud frames, obtain the quality characterization information of the key point cloud frame; the quality characterization information includes at least one of laser intensity, point cloud density and matching confidence. If the quality characterization information of the key point cloud frame meets the preset quality requirements, then the endpoint coordinates of each line segment in the key point cloud frame are obtained.

[0198] Example 19: According to the method described in Example 10, the candidate reference angle further includes the angle corresponding to the specified direction; When the selected candidate reference angle is the angle corresponding to the specified direction, the first display line is parallel to the specified direction.

[0199] Example 20: According to the method described in Example 1, the method further includes: In response to the robot's current position in the physical environment of the map to be built, if the distance from the starting position is less than or equal to a preset distance threshold, the display device is controlled to merge the third and fourth display lines into a fifth display line; The third display line is the line corresponding to the current position, the fourth display line is the line corresponding to the starting position, and the third display line is parallel to the fourth display line.

[0200] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The terms "an embodiment," "embodiment," or "one or more embodiments" as used herein mean that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of this application. Furthermore, please note that the phrase "in one embodiment" does not necessarily refer to the same embodiment in all instances.

[0201] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of this application may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0202] In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A map processing method, characterized in that, The method includes: During the robot mapping process, the control display device displays a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped. In response to detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be constructed and the first entity line is less than or equal to a first angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; or In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; The candidate reference angle with the smallest angular difference from the direction represented by the first display line is selected; the candidate reference angle includes at least the different angles corresponding to the boundaries of the display interface. The amount of rotation is determined based on the selected candidate reference angle and the angle of the direction represented by the first display line; The pose of the existing key point cloud frame is corrected according to the rotation amount so that the first display line is parallel to the boundary corresponding to the selected candidate reference angle.

2. The method according to claim 1, characterized in that, The method further includes: In response to detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold, the display device is controlled to display a second display line corresponding to the second entity line, the second display line being parallel to the first display line.

3. The method according to claim 2, characterized in that, The first angle threshold is greater than 0 degrees and less than or equal to 10 degrees; The second angle threshold is greater than or equal to 80 degrees and less than 90 degrees; The third angle threshold is greater than or equal to 90 degrees and less than or equal to 100 degrees; The fourth angle threshold is greater than 0 degrees and less than or equal to 10 degrees.

4. The method according to claim 2, characterized in that, The method further includes: obtaining the virtual line corresponding to the first physical line and the virtual line corresponding to the second physical line; The step of detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is less than or equal to a first angle threshold includes: detecting that the angle between the orthogonal direction of the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to the first angle threshold; the step of detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold includes: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold; the step of detecting that the angle between the second entity line and the first entity line in the physical environment of the map to be built is less than or equal to a fourth angle threshold includes: detecting that the angle between the virtual line corresponding to the second entity line and the virtual line corresponding to the first entity line is less than or equal to a fourth angle threshold. or, The method further includes: acquiring images corresponding to the first entity line and the second entity line; The step of detecting that the angle between the orthogonal direction of the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is less than or equal to a first angle threshold includes: determining, based on the image, that the angle between the orthogonal direction of the second entity line and the first entity line is less than or equal to the first angle threshold; the step of detecting that the angle between the second entity line corresponding to the second entity in the physical environment to be mapped and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold includes: determining, based on the image, that the angle between the second entity line and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold; the step of detecting that the angle between the second entity line and the first entity line in the physical environment to be mapped is less than or equal to a fourth angle threshold includes: determining, based on the image, that the angle between the second entity line and the first entity line is less than or equal to a fourth angle threshold.

5. The method according to claim 1, characterized in that, The control display device displays the first display line, including: Based on multiple structural line segments of the physical environment to be built, the main direction of the physical environment to be built is determined, wherein the angles between the multiple structural line segments and the first entity line are all less than or equal to a preset angle threshold. Based on the main direction of the environment, the display device is controlled to display the first display line, wherein the direction of the first display line is the same as the main direction of the environment.

6. The method according to claim 1, characterized in that, The method further includes: In response to detecting that the angle between the second entity line and the orthogonal direction of the first entity line is greater than the first angle threshold, and the angle between the second entity line and the first entity line is greater than a fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is neither parallel nor orthogonal to the first display line; or In response to detecting that the angle between the second entity line and the first entity line is less than the second angle threshold or greater than the third angle threshold, and the angle between the second entity line and the first entity line is greater than the fourth angle threshold, the display device is directly controlled to display the second display line corresponding to the second entity line, wherein the second display line is not parallel and not orthogonal to the first display line.

7. The method according to claim 1, characterized in that, The method further includes: After the mapping is completed, the display device is controlled to display the correction confirmation option; In response to receiving a cancellation operation for the correction confirmation option, the display device is controlled to display the first display line as non-parallel to the boundary.

8. The method according to claim 1, characterized in that, The step of correcting the pose of the existing key point cloud frame according to the rotation amount includes: For any keypoint cloud frame in the existing keypoint cloud frames, a corrected second coordinate is generated for the keypoint cloud frame based on the first coordinate, the deviation of the second coordinate relative to the first coordinate, and the rotation amount; the second coordinate is the coordinate in the pose of the keypoint cloud frame, and the first coordinate is the coordinate in the pose of the first keypoint cloud frame. Based on the heading angle in the pose of the key point cloud frame and the rotation amount, a corrected heading angle is generated for the key point cloud frame.

9. The method according to claim 2, characterized in that, The method further includes: For any newly added key point cloud frame, calculate the angle difference between the longest line segment of the key point cloud frame and the orthogonal direction represented by the first display line as the first angle difference, and calculate the angle difference between the longest line segment and the direction represented by the first display line as the second angle difference. If the first angle difference is less than or equal to the first angle threshold, then it is determined that the angle between the detected second entity line and the orthogonal direction of the first entity line is less than or equal to the first angle threshold. If the second angle difference is less than or equal to the fourth angle threshold, then it is determined that the angle between the second entity line and the first entity line is less than or equal to the fourth angle threshold.

10. The method according to claim 9, characterized in that, The method further includes: For any newly added key point cloud frame, select the minimum angle difference from the first angle difference and the second angle difference; The pose of the key point cloud frame is corrected based on the minimum angle difference, and the key point cloud frame is inserted into the map according to the corrected pose, so that the displayed second display line is orthogonal to the first display line or parallel to the first display line.

11. A map processing apparatus, characterized in that, The device includes a memory and a processor; The memory is used to store program code; The processor calls the program code, and when the program code is executed, it is used to perform the following operations: during the robot mapping process, the display device is controlled to display a first display line, which corresponds to the first entity line corresponding to the first entity in the physical environment to be mapped; In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the orthogonal direction of the first entity line is less than or equal to a first angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; or In response to detecting that the angle between the second entity line corresponding to the second entity in the physical environment of the map to be built and the first entity line is greater than or equal to a second angle threshold and less than or equal to a third angle threshold, the display device is controlled to display the second display line corresponding to the second entity line, wherein the second display line is orthogonal to the first display line; Select the candidate reference angle with the smallest angular difference from the direction represented by the first display line; The candidate reference angles include at least the different angles corresponding to the boundaries of the display interface; The amount of rotation is determined based on the selected candidate reference angle and the angle of the direction represented by the first display line; The pose of the existing key point cloud frame is corrected according to the rotation amount so that the first display line is parallel to the boundary corresponding to the selected candidate reference angle.

12. A readable storage medium, characterized in that, It stores instructions that, when executed by one or more processors, cause the processors to perform the method as described in any one of claims 1 to 10.

Images