Positioning navigation method and device, robot and storage medium

Abstract

The invention provides a positioning and navigation method and device, a robot and a storage medium, the positioning and navigation method comprises the following steps: firstly, obtaining a first map and a second map corresponding to a target area, the first map being generated by point cloud data of the target area, the second map comprising semantic annotation information of the target area; the first map and the second map are converted into files in a first preset format, a first image and a second image are correspondingly obtained, the first image and the second image are superposed and fused to obtain a fused map, and finally positioning navigation in the target area is performed based on the fused map. According to the method, the positioning navigation in the target area is carried out based on the fusion map fusing the point cloud information and the semantic annotation information, so that intelligent navigation can be realized based on the environment information and the semantic information, and the method can adapt to different use scenes, thereby ensuring the intelligence, reliability and applicability of the positioning navigation.

Description

Technical Field

[0001] This invention relates to the field of positioning equipment technology, and in particular to a positioning and navigation method, device, robot, and storage medium. Background Technology

[0002] Indoor navigation in construction sites faces two major challenges: On the one hand, laser SLAM (Simultaneous Localization and Mapping) technology can scan the environment in real time and generate accurate point cloud maps, but these maps are only geometric representations of physical obstacles and lack semantic information; on the other hand, construction sites themselves have CAD (Computer Aided Design) drawings, which are marked with detailed semantic information such as area divisions, column grid numbers, and construction zones, but the CAD drawings and the actual environment have problems such as coordinate system differences, inconsistent scales, rotation angle deviations, and construction accuracy errors, and cannot be directly superimposed and used.

[0003] Existing methods either use SLAM for pure geometric navigation or manually annotate semantic information, which are costly and cannot cope with dynamic changes on the construction site. They also cannot automatically integrate real-time SLAM point cloud maps with CAD design drawings, resulting in construction robots being unable to accurately locate themselves and understand construction tasks, leading to poor intelligence in the navigation of construction robots. Summary of the Invention

[0004] The present invention aims to at least solve one of the technical problems existing in the prior art. Therefore, the object of the present invention is to provide a positioning and navigation method, device, robot, and storage medium.

[0005] The present invention proposes a positioning and navigation method, comprising the following steps: acquiring a first map and a second map corresponding to a target area, wherein the first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area; converting the first map and the second map into files of a first preset format respectively, thereby obtaining a first image and a second image; overlaying and fusing the first image and the second image to obtain a fused map; and performing positioning and navigation within the target area based on the fused map.

[0006] According to the positioning and navigation method of the present invention, a first map and a second map corresponding to the target area are first obtained, wherein the first map is generated from the point cloud data of the target area and the second map includes the semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability and applicability of positioning and navigation.

[0007] In addition, the positioning and navigation method according to embodiments of the present invention may also have the following additional technical features: Furthermore, obtaining the first map corresponding to the target area includes: scanning the target area based on a lidar sensor to obtain multiple frames of point cloud data; processing the point cloud data using a preset positioning and mapping algorithm to construct a three-dimensional point cloud map, and using the three-dimensional point cloud map as the first map; thereby accurately obtaining the first map corresponding to the target area, which facilitates obtaining an accurate first image based on the first map in the future.

[0008] Furthermore, converting the first map into a file of a first preset format to obtain a corresponding first image includes: projecting the three-dimensional point cloud map into a two-dimensional raster map to obtain the first image; thereby converting the first map into a file of the first preset format to obtain a precise first image, which is convenient for subsequent overlay and fusion based on the first image.

[0009] Further, the step of projecting the 3D point cloud map into a 2D raster map to obtain the first image includes: filtering the Z-axis data of the 3D point cloud map to retain point cloud data with heights within a preset height range; projecting the filtered 3D point cloud map onto a 2D horizontal plane to obtain a top-view point set; dividing the top-view point set into a grid according to a preset raster resolution; marking the grid and outputting the first image in the first preset format, wherein grid cells with point cloud projections are marked as occupied, and grid cells without point clouds are marked as idle; thereby, the 3D point cloud map can be projected into a 2D raster map to obtain an accurate first image, which facilitates subsequent overlay and fusion based on the first image.

[0010] Furthermore, before processing the point cloud data using a preset positioning and mapping algorithm, the method further includes: acquiring the pose data of the lidar sensor; compensating the point cloud data based on the pose data; and constructing the three-dimensional point cloud map based on the compensated point cloud data. Constructing the three-dimensional point cloud map based on the point cloud data after compensating the point cloud data using the pose data ensures the accuracy and reliability of the three-dimensional point cloud map.

[0011] Furthermore, when scanning the target area based on the lidar sensor, the method further includes: if the area of ​​the target area exceeds a preset threshold, dividing the target area into multiple sub-regions; scanning multiple sub-regions to obtain multiple sets of point cloud data; saving the multiple sets of point cloud data as a second preset format file; and stitching the multiple second preset format files together to obtain the three-dimensional point cloud map. In this way, a three-dimensional point cloud map can be accurately and reliably constructed for a large target area.

[0012] Further, the three-dimensional point cloud map is obtained by stitching together multiple second preset format files, including: calculating the normal vector corresponding to each second preset format file; performing ICP coarse registration between each second preset format file and the reference point cloud based on a first distance threshold; after coarse registration, performing ICP fine registration between each second preset format file based on a second preset distance threshold to obtain a transformation matrix, wherein the first preset distance threshold is greater than the second preset distance threshold; rotating and translating each second preset format file according to the transformation matrix to transform each second preset format file to the reference coordinate system, and merging all point cloud data in each second preset format file to obtain the three-dimensional point cloud map; in this way, multiple second preset format files can be stitched together to obtain a three-dimensional point cloud map, thereby enabling accurate and reliable construction of a three-dimensional point cloud map for large target areas.

[0013] Furthermore, obtaining the second map corresponding to the target area includes: obtaining a drawing in a third preset format corresponding to the target area, and using the drawing as the second map, wherein the drawing includes semantic annotation information of the target area; this allows for accurate acquisition of the second map corresponding to the target area, facilitating the subsequent acquisition of an accurate second image based on the second map.

[0014] Further, converting the second map into a file of a first preset format to obtain a corresponding second image includes: converting the second map into an image of a fourth preset format, wherein the image of the fourth preset format includes the original scale of the map and the semantic annotation information; while keeping the image pixel size and grayscale value unchanged, converting the image of the fourth preset format into a file of the first preset format to obtain the second image; thereby, the second map can be converted into a file of the first preset format to obtain a precise second image, which is convenient for subsequent overlay and fusion based on the second image.

[0015] Furthermore, the step of overlaying and fusing the first image and the second image to obtain a fused map includes: selecting target feature points on the first image and the second image respectively; calculating a rigid transformation relationship from the coordinate system of the second image to the coordinate system of the first image based on the target feature points, wherein the rigid transformation relationship includes at least one of rotation, scaling, and translation; automatically aligning the first image and the second image based on the rigid transformation relationship; and fusing and overlaying the aligned first image and the second image to obtain the fused map. This effectively and accurately overlays and fuses the first image and the second image to obtain a precise fused map, facilitating subsequent positioning and navigation based on the precise fused map.

[0016] Furthermore, selecting corresponding feature points on the first image and the second image respectively includes: using multiple identical feature points on the first image and the second image as the target feature points; this allows for accurate fusion and overlay of the first image and the second image, which helps to obtain an accurate fused map.

[0017] Furthermore, after automatically aligning the first image and the second image based on the rigid transformation relationship, the method further includes: receiving a fine-tuning instruction for the second map, wherein the fine-tuning instruction includes adjustment information for the position of the second map; responding to the fine-tuning instruction, fine-tuning the position of the second map to further improve the alignment between the first image and the second image; thereby further improving the alignment between the first image and the second image, ensuring the accuracy of fusing and overlaying the first image and the second image, and helping to obtain an accurate fused map.

[0018] Furthermore, the step of fusing and overlaying the aligned first and second images to obtain the fused map includes: using the first image as a base, overlaying the lines and semantic annotation information from the second image onto it, and covering the corresponding positions in the first image with the pixels containing content from the second image to form the fused map; this ensures that the fused map has both the physical geometric information of the first image and the semantic annotation information of the second image, thereby obtaining an accurate fused map.

[0019] Furthermore, the positioning and navigation within the target area based on the fused map includes: acquiring real-time point cloud data of the moving target object; and performing at least one of real-time positioning, obstacle detection, path planning, dynamic obstacle avoidance, and repositioning on the target object within the target area based on the real-time point cloud data of the target object and the fused map. This enables positioning and navigation within the target area based on the fused map, performing at least one of real-time positioning, obstacle detection, path planning, dynamic obstacle avoidance, and repositioning, thereby ensuring the intelligence of the positioning and navigation.

[0020] Furthermore, the positioning and navigation method further includes: receiving semantic control instructions for a target moving object, the semantic control instructions including at least one of the following: target position, target path planning, real-time position prompts of the target moving object, and task execution feedback information; responding to the semantic control instructions, controlling the target moving object to perform corresponding actions based on the fused map; thereby using the positioning and navigation method to control a target moving object can further ensure the intelligence, reliability, and applicability of the positioning and navigation.

[0021] To address the aforementioned problems, this invention also proposes a positioning and navigation device, comprising: an acquisition module for acquiring a first map and a second map corresponding to a target area, wherein the first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area; a conversion module for converting the first map and the second map into files of a first preset format, respectively, to obtain a first image and a second image; a fusion module for overlaying and fusing the first image and the second image to obtain a fused map; and a processing module for performing positioning and navigation within the target area based on the fused map.

[0022] According to the positioning and navigation device of the present invention, the positioning and navigation method of the above embodiments of the present invention is executed. First, a first map and a second map corresponding to the target area are obtained. The first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability and applicability of positioning and navigation.

[0023] To address the aforementioned problems, the present invention also proposes a robot, comprising: a positioning and navigation device as described in the above embodiments of the present invention; or, a processor, a memory, and a positioning and navigation program stored in the memory and executable on the processor, wherein the positioning and navigation program, when executed by the processor, implements the positioning and navigation method as described in the above embodiments of the present invention.

[0024] According to embodiments of the present invention, a robot executing the positioning and navigation method of the above embodiments of the present invention first acquires a first map and a second map corresponding to a target area. The first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability, and applicability of positioning and navigation.

[0025] To address the aforementioned problems, the present invention also proposes a computer-readable storage medium storing a positioning and navigation program, which, when executed by a processor, implements the positioning and navigation method as described in the above embodiments of the present invention.

[0026] According to an embodiment of the present invention, when a positioning and navigation program stored thereon is executed by a processor, the positioning and navigation method of the above embodiment of the present invention is executed. First, a first map and a second map corresponding to the target area are obtained. The first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability, and applicability of positioning and navigation.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a positioning and navigation method according to an embodiment of the present invention; Figure 2 This is a flowchart of a positioning and navigation method according to another embodiment of the present invention; Figure 3 This is a flowchart of a positioning and navigation method according to a specific embodiment of the present invention; Figure 4 This is a structural block diagram of a positioning and navigation device according to an embodiment of the present invention.

[0029] Figure label: 100 - Positioning and navigation device; 110 - Acquisition module; 120 - Conversion module; 130 - Fusion module; 140 - Processing module. Detailed Implementation

[0030] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. The embodiments of the present invention are described in detail below.

[0031] The following is for reference. Figures 1-4 A positioning and navigation method, apparatus, robot, and storage medium according to embodiments of the present invention are described.

[0032] Figure 1 This is a flowchart of a positioning and navigation method according to an embodiment of the present invention. Figure 1As shown, a positioning and navigation method according to an embodiment of the present invention includes the following steps: Step S1: Obtain the first map and the second map corresponding to the target area. The first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area.

[0033] In a specific embodiment, a first map corresponding to the target area is obtained. The first map is generated from the point cloud data of the target area. Specifically, the first map is, for example, a three-dimensional point cloud map generated using SLAM (Simultaneous Localization and Mapping) technology. The format of the three-dimensional point cloud map is, for example, a PLY (Polygon File Format) file. The PLY file records the three-dimensional coordinates of all scanned points in the environment, including all objects such as walls, substations, equipment, and temporary material piles.

[0034] In a specific embodiment, a second map corresponding to the target area is obtained, and the second map includes semantic annotation information of the target area. Specifically, the second map is, for example, a CAD (Computer Aided Design) drawing.

[0035] Specifically, according to the positioning and navigation method of the present invention, a first map and a second map corresponding to the target area are first obtained. The first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area, which facilitates the subsequent fusion of the first map and the second map.

[0036] Step S2: Convert the first map and the second map into files of the first preset format respectively to obtain the first image and the second image.

[0037] In a specific embodiment, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Specifically, the first preset format is, for example, PGM (Portable Gray Map) format.

[0038] Specifically, according to the positioning and navigation method of the present invention, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image, which facilitates the subsequent fusion of the first image and the second image.

[0039] Step S3: Overlay and fuse the first image and the second image to obtain a fused map.

[0040] In a specific embodiment, the first image and the second image are overlaid and fused to obtain a fused map. Specifically, the overlay and fusion methods include, but are not limited to, feature point alignment, rigid transformation, coordinate transformation, and feature fusion.

[0041] Specifically, according to the positioning and navigation method of the present invention, the first image and the second image are then superimposed and fused to obtain a fused map, which facilitates subsequent positioning and navigation within the target area based on the fused map.

[0042] Step S4: Perform positioning and navigation within the target area based on the fused map.

[0043] In a specific embodiment, positioning and navigation within the target area are performed based on a fused map. Specifically, positioning and navigation includes, but is not limited to, real-time positioning, obstacle detection, path planning, dynamic obstacle avoidance, and relocation.

[0044] Specifically, according to the positioning and navigation method of the present invention, positioning and navigation within the target area is finally performed based on a fused map, that is, based on a fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability and applicability of positioning and navigation.

[0045] Therefore, the positioning and navigation method according to the embodiments of the present invention first obtains a first map and a second map corresponding to the target area, wherein the first map is generated from the point cloud data of the target area and the second map includes the semantic annotation information of the target area. Then, the first map and the second map are respectively converted into files of a first preset format to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability and applicability of positioning and navigation.

[0046] In one embodiment of the present invention, step S1, obtaining the first map corresponding to the target area, includes: scanning the target area based on a lidar sensor to obtain multiple frames of point cloud data; processing the point cloud data using a preset positioning and mapping algorithm to construct a three-dimensional point cloud map, and using the three-dimensional point cloud map as the first map.

[0047] In a specific embodiment, the target area is first scanned using a lidar sensor to obtain multiple frames of point cloud data. Then, a preset localization and mapping algorithm is used to process the point cloud data to construct a three-dimensional point cloud map, which is then used as the first map. Specifically, the lidar sensor can be mounted on a mobile device for mobile scanning. The point cloud data includes, for example, three-dimensional coordinates and reflection intensity information. The preset localization and mapping algorithm is, for example, SLAM. SLAM can eliminate accumulated errors by matching the features of consecutive frame point clouds, thus ensuring the accuracy of the first map.

[0048] Specifically, according to the positioning and navigation method of the present invention, the target area is first scanned based on a lidar sensor to obtain multiple frames of point cloud data. Then, a preset positioning and mapping algorithm is used to process the point cloud data to construct a three-dimensional point cloud map, which is used as the first map. In this way, the first map corresponding to the target area can be accurately obtained, which facilitates the subsequent acquisition of an accurate first image based on the first map.

[0049] In one embodiment of the present invention, step S2 converts the first map into a file of a first preset format to obtain a first image, including: projecting the three-dimensional point cloud map into a two-dimensional raster map to obtain the first image.

[0050] In a specific embodiment, the 3D point cloud map is projected into a 2D raster map to obtain the first image. Specifically, the 3D point cloud can be projected onto a 2D plane, that is, onto the XY horizontal plane, discarding the Z coordinate, and converted into a raster map in PGM format.

[0051] Specifically, according to the positioning and navigation method of the present invention, a three-dimensional point cloud map is projected into a two-dimensional grid map to obtain a first image; thereby, the first map can be converted into a file of a first preset format to obtain a precise first image, which is convenient for subsequent overlay and fusion based on the first image.

[0052] In one embodiment of the present invention, projecting a three-dimensional point cloud map into a two-dimensional raster map to obtain a first image includes: filtering the Z-axis data of the three-dimensional point cloud map to retain point cloud data with heights within a preset height range; projecting the filtered three-dimensional point cloud map onto a two-dimensional horizontal plane to obtain a top-view point set; dividing the top-view point set into a grid according to a preset raster resolution; marking the grid and outputting a first image in a first preset format, wherein grid cells with point cloud projections are marked as occupied and grid cells without point clouds are marked as idle.

[0053] In a specific embodiment, the Z-axis data of the 3D point cloud map is first filtered, retaining point cloud data with heights within a preset height range. Then, the filtered 3D point cloud map is projected onto a 2D horizontal plane to obtain a top-view point set. Next, the top-view point set is divided into a grid according to a preset raster resolution. Finally, the grid is marked, and a first image in a first preset format is output. Specifically, the preset height range is, for example, 0.5 meters to 2 meters, meaning that ceilings and floors are filtered out, retaining only obstacles and building structures. The preset raster resolution is, for example, one centimeter per grid. Grids with point cloud projections are marked as occupied (e.g., black), and grids without point clouds are marked as idle (e.g., white), generating a grayscale image in PGM format. PGM format is a simple raster image format where each pixel represents the occupied state of a grid, suitable for image registration operations.

[0054] Specifically, according to the positioning and navigation method of the present invention, the Z-axis data of the three-dimensional point cloud map is first filtered to retain point cloud data with heights within a preset height range. Then, the filtered three-dimensional point cloud map is projected onto a two-dimensional horizontal plane to obtain a top view point set. Next, the top view point set is divided into a grid according to a preset grid resolution. Finally, the grid is marked and a first image in a first preset format is output. In this way, the three-dimensional point cloud map can be projected into a two-dimensional grid map to obtain an accurate first image, which is convenient for subsequent overlay and fusion based on the first image.

[0055] In one embodiment of the present invention, before processing the point cloud data using a preset positioning and mapping algorithm, the method further includes: acquiring the pose data of a lidar sensor; compensating the point cloud data based on the pose data; and constructing a three-dimensional point cloud map based on the compensated point cloud data.

[0056] In a specific embodiment, the pose data of the LiDAR sensor is first acquired, then the point cloud data is compensated based on the pose data, and finally a 3D point cloud map is constructed based on the compensated point cloud data. Specifically, since the target area may be large or the environment may be complex, the LiDAR sensor needs to scan multiple times or move to scan. Therefore, when constructing a 3D point cloud map, not only the scanned point cloud data but also the pose data of the LiDAR sensor is needed to compensate for the point cloud data.

[0057] Specifically, according to the positioning and navigation method of the present invention, the pose data of the lidar sensor is first acquired, then the point cloud data is compensated based on the pose data, and finally a three-dimensional point cloud map is constructed based on the compensated point cloud data. By constructing a three-dimensional point cloud map based on the point cloud data after compensating the point cloud data with pose data, the accuracy and reliability of the three-dimensional point cloud map can be ensured.

[0058] In one embodiment of the present invention, when scanning the target area based on the lidar sensor, the method further includes: if the area of ​​the target area exceeds a preset threshold, dividing the target area into multiple sub-regions; scanning the multiple sub-regions to obtain multiple sets of point cloud data; saving the multiple sets of point cloud data as a second preset format file; and stitching the multiple second preset format files together to obtain a three-dimensional point cloud map.

[0059] In a specific embodiment, if the area of ​​the target region exceeds a preset threshold, the target region is divided into multiple sub-regions; multiple sets of point cloud data are obtained by scanning the multiple sub-regions; the multiple sets of point cloud data are saved as a second preset format file; and the multiple second preset format files are stitched together to obtain a 3D point cloud map. Specifically, the preset threshold is, for example, the area that a LiDAR sensor can scan in a single scan, and the second preset format file is, for example, a PLY format file.

[0060] Specifically, according to the positioning and navigation method of the present invention, if the area of ​​the target area exceeds a preset threshold, the target area is first divided into multiple sub-areas, and then multiple sets of point cloud data are obtained by scanning multiple sub-areas. Then, the multiple sets of point cloud data are saved as a second preset format file, and finally, the multiple second preset format files are stitched together to obtain a three-dimensional point cloud map. In this way, a three-dimensional point cloud map can be accurately and reliably constructed for a large target area.

[0061] In one embodiment of the present invention, multiple second preset format files are stitched together to obtain a three-dimensional point cloud map, including: calculating the normal vector corresponding to each second preset format file; performing ICP coarse registration between each second preset format file and a reference point cloud based on a first distance threshold; after the coarse registration is completed, performing ICP fine registration between each second preset format file based on a second preset distance threshold to obtain a transformation matrix, wherein the first preset distance threshold is greater than the second preset distance threshold; rotating and translating each second preset format file according to the transformation matrix to transform each second preset format file to the reference coordinate system, and merging all point cloud data in each second preset format file to obtain a three-dimensional point cloud map.

[0062] In a specific embodiment, the normal vectors corresponding to each second preset format file are first calculated. Then, based on a first distance threshold, each second preset format file is coarsely registered with the reference point cloud using ICP (Iterative Closest Point Algorithm). After coarse registration, fine ICP registration is performed on each second preset format file based on the second preset distance threshold to obtain a transformation matrix. Next, each second preset format file is rotated and translated according to the transformation matrix to transform it to the reference coordinate system. All point cloud data from each second preset format file are then merged to obtain a 3D point cloud map. Specifically, the first and second distance thresholds are set as needed; for example, the first distance threshold is 1 meter, and the second distance threshold is 1 centimeter. The normal vectors provide geometric constraints for ICP registration. Coarse ICP registration is used to quickly find approximate rotation and translation relationships, while fine ICP registration uses a point-to-surface error metric to improve registration accuracy.

[0063] In a specific embodiment, the ICP algorithm achieves spatial coordinate alignment by iteratively solving for the rotation matrix R and translation vector t. Specifically, the ICP algorithm optimizes the objective function based on the least squares method, and its core process includes steps such as corresponding point search, centroid calculation, and singular value decomposition to obtain transformation parameters. The mathematical basis of the ICP algorithm is to optimize the spatial transformation parameters between two point sets using the least squares method. The objective function is defined as minimizing the sum of squared Euclidean distances between corresponding points. Centroid processing is used to decouple the calculation of translation and rotation. The optimal rotation matrix is ​​solved by singular value decomposition or quaternion method. The iteration termination condition is usually set to the change in transformation parameters or the error function value being lower than a threshold.

[0064] Specifically, according to the positioning and navigation method of the present invention, the normal vectors corresponding to each second preset format file are first calculated. Then, based on a first distance threshold, each second preset format file and the reference point cloud are coarsely registered using ICP. After the coarse registration is completed, each second preset format file is finely registered using ICP based on the second preset distance threshold to obtain a transformation matrix. Then, each second preset format file is rotated and translated according to the transformation matrix to transform each second preset format file to the reference coordinate system. All point cloud data in each second preset format file are then merged to obtain a three-dimensional point cloud map. In this way, multiple second preset format files can be stitched together to obtain a three-dimensional point cloud map, thereby enabling the accurate and reliable construction of a three-dimensional point cloud map for a large target area.

[0065] In one embodiment of the present invention, step S1 of obtaining a second map corresponding to the target area includes: obtaining a drawing in a third preset format corresponding to the target area, and using the drawing as the second map, wherein the drawing includes semantic annotation information of the target area.

[0066] In a specific embodiment, a drawing in a third preset format corresponding to the target area is obtained, and the drawing is used as a second map. Specifically, the drawing includes semantic annotation information of the target area, and the drawing is, for example, a CAD drawing, and the third preset format is, for example, DWG (Drawing) format.

[0067] Specifically, according to the positioning and navigation method of the present invention, a third preset format drawing corresponding to the target area is obtained, and the drawing is used as a second map; in this way, the second map corresponding to the target area can be accurately obtained, which facilitates the subsequent acquisition of an accurate second image based on the second map.

[0068] In one embodiment of the present invention, step S2 converts the second map into a file of a first preset format to obtain a second image, including: converting the second map into an image of a fourth preset format, the image of the fourth preset format including the original scale and semantic annotation information of the map; and converting the image of the fourth preset format into a file of the first preset format while keeping the image pixel size and grayscale value unchanged to obtain the second image.

[0069] In a specific embodiment, the second map is first converted into an image in a fourth preset format. Then, without changing the image pixel size and grayscale value, the image in the fourth preset format is converted into a file in a first preset format to obtain the second image. Specifically, the image in the fourth preset format includes the original scale and semantic annotation information of the drawing. The first preset format is, for example, PGM format, and the fourth preset format is, for example, PNG (Portable Network Graphics) format.

[0070] Specifically, according to the positioning and navigation method of the present invention, the second map is first converted into an image of a fourth preset format, and then the image of the fourth preset format is converted into a file of a first preset format without changing the image pixel size and grayscale value, so as to obtain the second image; in this way, the second map can be converted into a file of the first preset format, corresponding to an accurate second image, which is convenient for subsequent overlay and fusion based on the second image.

[0071] In one embodiment of the present invention, step S3, which involves overlaying and fusing the first image and the second image to obtain a fused map, includes: selecting target feature points on the first image and the second image respectively; calculating a rigid transformation relationship from the coordinate system of the second image to the coordinate system of the first image based on the target feature points, wherein the rigid transformation relationship includes at least one of rotation, scaling, and translation; automatically aligning the first image and the second image based on the rigid transformation relationship; and fusing and overlaying the aligned first image and the second image to obtain a fused map.

[0072] In a specific embodiment, target feature points are first selected on the first image and the second image respectively. Then, based on the target feature points, a rigid transformation relationship from the coordinate system of the second image to the coordinate system of the first image is calculated. Next, the first image and the second image are automatically aligned based on the rigid transformation relationship. Finally, the aligned first image and the second image are fused and superimposed to obtain a fused map. Specifically, the target feature points are, for example, the same feature points on the first image and the second image. The rigid transformation relationship includes, for example, rotation, scaling, and translation. Rigid transformation can handle angular deviations, scale differences, and positional offsets between CAD drawings and the actual environment. Rigid transformation first calculates the centroids of the two sets of feature points, then solves for the rotation matrix and scaling factor, and finally calculates the translation vector. Singular Value Decomposition (SVD) can also be used to ensure the orthogonality of the rotation matrix and ensure the stability of the transformation.

[0073] Specifically, according to the positioning and navigation method of the present invention, target feature points are first selected on the first image and the second image respectively. Then, based on the target feature points, a rigid transformation relationship from the coordinate system of the second image to the coordinate system of the first image is calculated. Next, the first image and the second image are automatically aligned based on the rigid transformation relationship. Finally, the aligned first image and the second image are fused and superimposed to obtain a fused map. In this way, the first image and the second image can be effectively and accurately superimposed and fused to obtain an accurate fused map, which is convenient for subsequent positioning and navigation based on the accurate fused map.

[0074] In one embodiment of the present invention, selecting corresponding feature points on the first image and the second image respectively includes: using multiple identical feature points on the first image and the second image as target feature points.

[0075] In a specific embodiment, multiple identical feature points on the first and second images are used as target feature points. Specifically, identical feature points include clearly identifiable locations such as the center of a pillar, a corner of a wall, and a door frame. For example, at least five corresponding feature points can be selected on each image. Selecting multiple feature points can improve the robustness of registration and avoid single-point errors affecting the overall alignment accuracy.

[0076] Specifically, according to the positioning and navigation method of the present invention, multiple identical feature points on the first image and the second image are used as target feature points; thereby, the first image and the second image can be accurately fused and superimposed, which helps to obtain an accurate fused map.

[0077] In one embodiment of the present invention, after automatically aligning the first image and the second image based on a rigid transformation relationship, the method further includes: receiving a fine-tuning instruction for the second map, wherein the fine-tuning instruction includes adjustment information for the position of the second map; and, in response to the fine-tuning instruction, fine-tuning the position of the second map to further improve the alignment between the first image and the second image.

[0078] In a specific embodiment, a fine-tuning instruction for the second map is received. In response to the fine-tuning instruction, the position of the second map is fine-tuned to further improve the alignment between the first and second images. Specifically, the fine-tuning instruction includes adjustment information for the position of the second map. The fine-tuning instruction can be issued by an operator or the control system. The operator can use the mouse to drag or the keyboard arrow keys to fine-tune the position of the second map (CAD drawing) until the two images are perfectly aligned. The overlay effect can also be displayed in real time during the fine-tuning process, allowing the operator to clearly see whether features such as walls and columns are precisely aligned.

[0079] Specifically, according to the positioning and navigation method of the present invention, a fine-tuning instruction for a second map is received, and in response to the fine-tuning instruction, the position of the second map is fine-tuned to further improve the alignment of the first image and the second image; thereby further improving the alignment of the first image and the second image, ensuring the accuracy of the fusion and overlay of the first image and the second image, and helping to obtain an accurate fused map.

[0080] In one embodiment of the present invention, the aligned first image and the second image are fused and superimposed to obtain a fused map, including: using the first image as a base, superimposing the lines and semantic annotation information in the second image onto it, and covering the corresponding positions in the first image with the pixels containing content in the second image to form a fused map.

[0081] In a specific embodiment, a first image is used as a base map, and lines and semantic annotation information from a second image are overlaid on it. Pixels containing content from the second image are then overlaid onto corresponding positions in the first image to form a fused map. Specifically, for example, a PGM map converted from SLAM point clouds can be used as a base map (providing real-time obstacle information). Lines and annotations from a CAD drawing are overlaid on it, and pixels containing content from the CAD drawing are overlaid onto corresponding positions in the SLAM map to form a fused map. This fused map contains both the physical geometry information from SLAM and the semantic annotation information from CAD. Furthermore, the fused map can be saved in PGM format and directly used in navigation systems.

[0082] Specifically, according to the positioning and navigation method of the present invention, a first image is used as a base image, and lines and semantic annotation information from a second image are superimposed on it, and pixels containing content in the second image are overlaid on the corresponding positions in the first image to form a fused map; in this way, it can be ensured that the fused map has both the physical geometric information of the first image and the semantic annotation information of the second image, thereby obtaining an accurate fused map.

[0083] In one embodiment of the present invention, positioning and navigation within a target area based on a fused map includes: acquiring real-time point cloud data of a moving target object; and performing at least one of real-time positioning, obstacle detection, path planning, dynamic obstacle avoidance, and relocation of the target object within the target area based on the real-time point cloud data of the target object and the fused map.

[0084] In a specific embodiment, real-time point cloud data of the target moving object is first acquired. Then, based on the real-time point cloud data and the fused map, at least one of the following is performed on the target object within the target area: real-time localization, obstacle detection, path planning, dynamic obstacle avoidance, and relocalization. Specifically, the target moving object is, for example, a construction robot.

[0085] In a specific embodiment, real-time positioning includes: the construction robot continuously scans its surroundings using LiDAR; the SLAM algorithm matches the current scan with an existing map to calculate the robot's precise position and orientation on the map. Specifically, the positioning accuracy can reach the centimeter level, meeting the precision requirements of construction operations.

[0086] In a specific embodiment, obstacle detection includes: real-time detection of surrounding obstacles by LiDAR. Specifically, for static obstacles already present in the map (such as walls and pillars), the system confirms their location through matching; for dynamic obstacles not present in the map (such as temporary material piles, mobile equipment, and construction workers), the SLAM algorithm can identify and mark them in real time.

[0087] In a specific embodiment, path planning includes: based on the physical layer information of the fused map, the navigation algorithm plans the optimal path from the current location to the target location. Specifically, the planning process considers static obstacles, dynamic obstacles, and available space to generate a safe and feasible path.

[0088] In a specific embodiment, dynamic obstacle avoidance includes: when the construction robot moves along the planned path, it continuously monitors the environment ahead. If a new obstacle appears on the path, it performs local path replanning in real time, bypasses the obstacle, and continues to move forward, ensuring the robustness of navigation.

[0089] In a specific embodiment, relocation includes: when the construction robot restarts or fails to locate, using the ICP algorithm to perform global matching between the current laser scan and the map, quickly finding its position on the map and restoring navigation capabilities.

[0090] Specifically, according to the positioning and navigation method of the present invention, real-time point cloud data of the target moving object is first acquired, and then, based on the real-time point cloud data of the target object and the fused map, at least one of the following is performed on the target object within the target area: real-time positioning, obstacle detection, path planning, dynamic obstacle avoidance, and repositioning. In this way, positioning and navigation can be performed based on the fused map within the target area, thereby ensuring the intelligence of positioning and navigation.

[0091] Figure 2 This is a flowchart of a positioning and navigation method according to another embodiment of the present invention. Figure 2 As shown, in another embodiment of the present invention, the positioning and navigation method further includes: step S5: receiving a semantic control command for a target moving object, the semantic control command including at least one of the following: target position, target path planning, real-time position prompt of the target moving object, and task execution feedback information; step S6: responding to the semantic control command, controlling the target moving object to perform corresponding actions based on the fused map.

[0092] In a specific embodiment, semantic control commands for a target moving object are received, and in response to the semantic control commands, the target moving object is controlled to perform corresponding actions based on the fused map. Specifically, the target moving object includes, but is not limited to, a construction robot, and the semantic control commands include, but are not limited to, the target's position, target path planning, real-time position prompts for the target moving object, and feedback information on task execution progress.

[0093] Specifically, the positioning and navigation method according to the embodiments of the present invention can also receive semantic control commands for a target moving object, and in response to the semantic control commands, control the target moving object to perform corresponding actions based on the fused map; thereby, using the positioning and navigation method to control the target moving object can further ensure the intelligence, reliability and applicability of the positioning and navigation.

[0094] The positioning and navigation method of the present invention described above will be further explained below with reference to a specific embodiment. In this specific embodiment, a positioning and navigation method is provided.

[0095] Figure 3 This is a flowchart of a positioning and navigation method according to a specific embodiment of the present invention, such as... Figure 3 As shown in this specific embodiment, the positioning and navigation method includes: Step S10: Hardware equipment configuration.

[0096] Step S20: Laser SLAM mapping.

[0097] Step S30: Multi-segment point cloud stitching and fusion.

[0098] Step S40: Point Cloud Ground Figure 2 Dimensionalization.

[0099] Step S50: CAD drawing preprocessing.

[0100] Step S60: Feature point registration and fusion.

[0101] Step S70: Physical layer navigation.

[0102] Step S80: Semantic navigation application.

[0103] In this specific embodiment, the hardware equipment in step S10 includes: a construction robot, comprising: a lidar for scanning the surrounding environment and generating a 3D point cloud; an IMU (Inertial Measurement Unit) for assisting the SLAM algorithm, providing motion estimation during the intervals of the laser scan, and improving the accuracy and stability of mapping and localization; a computing unit for running the SLAM algorithm and map fusion program; and a mobile chassis for moving and collecting data at the construction site.

[0104] In this specific embodiment, step S20, laser SLAM mapping, includes: the robot equipped with a lidar collects point cloud data at the construction site, generates a 3D point cloud map using the SLAM algorithm, and saves it as a PLY format file. Specifically, this includes: 1. Point cloud acquisition: the lidar continuously scans the environment during robot movement, collecting multiple frames of point cloud data per second. Each point contains 3D coordinates and reflection intensity information; 2. Real-time SLAM mapping: the SLAM algorithm processes the point cloud sequence, calculates the robot's pose in real time, and simultaneously constructs a 3D point cloud map of the environment. The algorithm eliminates accumulated errors and ensures map accuracy by matching the features of consecutive frame point clouds; 3. Saving the point cloud map: after mapping is completed, the point cloud map is saved as a PLY format file. The PLY file records the 3D coordinates of all scanned points in the environment, including all objects such as walls, substructures, equipment, and temporary material piles.

[0105] In this specific embodiment, step S30, multi-segment point cloud stitching and fusion, includes: if the construction area is large and needs to be scanned in segments, the ICP algorithm is used to stitch multiple PLY point clouds into a complete map. The multi-segment stitching method can handle construction areas of any size, ensuring the continuity and consistency of the overall map. Specifically, this includes: 1. Loading multiple PLY files: Reading the PLY files of each scan segment, each point cloud segment has its own local coordinate system; 2. Calculating normal vectors: Calculating normal vector information for each point cloud segment to provide geometric constraints for ICP registration; 3. Coarse ICP registration: Using a large distance threshold (meter level), coarsely registering the point cloud to be stitched with the reference point cloud to quickly find the approximate rotation and translation relationship; 4. Fine ICP registration: Based on the coarse registration, using a smaller distance threshold (centimeter level), fine registration is performed to obtain a high-precision transformation matrix. The ICP algorithm uses a point-to-surface error metric to improve registration accuracy; 5. Point cloud transformation and merging: Rotating and translating the point cloud to be stitched according to the calculated transformation matrix to transform it to the reference coordinate system, and then merging all point clouds to generate a complete construction site point cloud map.

[0106] In this specific embodiment, step S40 is the point cloud. Figure 2 3D point cloud mapping involves projecting the 3D point cloud onto a 2D plane and converting it into a PGM format raster map for easy registration with CAD drawings. Specifically, this includes: 1. Z-axis filtering: retaining point clouds within a certain height range (e.g., half a meter to two meters above ground), filtering out ceilings and floors, and retaining only obstacles and building structures; 2. 2D projection: projecting the filtered point cloud onto the XY horizontal plane, discarding the Z coordinate, and obtaining a top-view point set; 3. Rasterization: setting the raster resolution (e.g., one centimeter per grid), dividing the plane into a grid, marking grids with point cloud projections as black (occupied) and grids without point clouds as white (empty), generating a grayscale image in PGM format. PGM format is a simple raster image format where each pixel represents the occupancy status of a grid, suitable for image registration operations.

[0107] In this specific embodiment, step S50, CAD drawing preprocessing, includes: exporting the CAD DWG file as a PNG image, and then converting it to a PGM format raster image. Specifically, this includes: 1. Exporting CAD to PNG: Exporting the architectural design's DWG format CAD drawing as a PNG image in AutoCAD software, maintaining the original scale and annotation information of the drawing during export; 2. Converting PNG to PGM: Using an image processing program, converting the color or grayscale PNG image to the standard PGM format, maintaining the image's pixel size and grayscale values ​​during the conversion process to ensure that CAD lines are clearly visible. The converted CAD-PGM image is consistent with the SLAM-PGM image format and can be registered and fused.

[0108] In this specific embodiment, step S60, feature point registration and fusion, includes: selecting corresponding feature points on the PGM map generated by SLAM and the PGM image of CAD, calculating the rigid transformation relationship, and achieving automatic alignment of the two maps. The alignment accuracy is optimized through interactive fine-tuning, and finally the CAD image is overlaid onto the SLAM map to generate a fused map. Specifically, this includes: 1. Interactive feature point selection: The system simultaneously displays two PGM images, namely the point cloud map generated by SLAM and the drawing converted from CAD. The operator clicks on the same feature points on both images, such as the center of a column, a corner of a wall, a door frame, or other clearly identifiable locations. At least five corresponding feature points are selected on each image. Selecting multiple feature points can improve the robustness of registration and avoid single-point errors affecting the overall alignment accuracy; 2. Rigid transformation calculation: Based on two sets of corresponding feature points, the rigid transformation from the CAD coordinate system to the SLAM coordinate system is calculated. The rigid transformation includes three types of transformations: rotation, scaling, and translation. It can handle angular deviations, scale differences, and positional offsets between the CAD drawing and the actual environment. The algorithm first calculates the centroids of the two sets of feature points, then solves for the rotation matrix and scaling factor, and finally calculates the translation vector. Singular value decomposition (SVD) is used to ensure the orthogonality of the rotation matrix and ensure the stability of the transformation; 3. CAD Image Transformation: The PGM image of the CAD file is rotated, scaled, and translated according to the calculated transformation matrix to align it with the SLAM point cloud map. The transformed CAD image and the SLAM map are in the same coordinate system. 4. Interactive Fine-tuning: Although rigid transformation can achieve initial alignment, there may be slight deviations. The system provides an interactive fine-tuning function: the transformed CAD image is overlaid on the SLAM map. The operator can use the mouse to drag or the keyboard arrow keys to fine-tune the position of the CAD image until the two images are perfectly aligned. The overlay effect is displayed in real time during the fine-tuning process, and the operator can clearly see whether features such as walls and columns are precisely aligned. 5. Map Fusion: After alignment, the CAD image is fused with the SLAM point cloud map. The fusion strategy is: using the PGM image converted from the SLAM point cloud as the base map (providing real-time obstacle information), the lines and annotations in the CAD image are overlaid on it. Pixels with content in the CAD image are overlaid on the corresponding positions in the SLAM map to form a fused map. The fused map contains both the physical geometric information of SLAM and the semantic annotation information of CAD. It is saved in PGM format and can be directly used in the navigation system. 6. Coordinate transformation relationship storage: The transformation matrix calculated during the fusion process is saved. When the robot is localizing later, the SLAM coordinates can be converted into CAD coordinates in real time, so that it knows its position on the CAD drawing and understands the semantic information of the surroundings.

[0109] In this specific embodiment, step S70, physical layer navigation, includes: the underlying layer of the fused map is a SLAM point cloud map, providing precise positioning and obstacle avoidance capabilities in the physical environment. Specifically, it includes: 1. Real-time positioning: The robot continuously scans the surrounding environment using LiDAR, and the SLAM algorithm matches the current scan with the existing map to calculate the robot's precise position and orientation in the map. 1. **Positioning accuracy down to the centimeter level:** This meets the precision requirements of construction operations. 2. **Obstacle detection:** The LiDAR system detects surrounding obstacles in real time. For static obstacles already present on the map (such as walls and pillars), the system confirms their location through matching. For dynamic obstacles not present on the map (such as temporary material piles, mobile equipment, and construction personnel), the SLAM algorithm can identify and mark them in real time. 3. **Path planning:** Based on the physical layer information of the fused map, the navigation algorithm plans the optimal path from the current location to the target location. The planning considers static obstacles, dynamic obstacles, and passage space to generate a safe and feasible path. 4. **Dynamic obstacle avoidance:** As the robot moves along the planned path, it continuously monitors the environment ahead. If new obstacles appear on the path, it performs real-time local path replanning to bypass the obstacles and continue moving forward, ensuring the robustness of navigation. 5. **Relocalization function:** When the robot restarts or fails to locate, the ICP algorithm is used to globally match the current LiDAR scan with the map, quickly recovering its position on the map and restoring navigation capabilities.

[0110] In this specific embodiment, step S80 semantic navigation application includes: based on the fused map, the robot can both use SLAM point clouds for precise positioning and recognize semantic annotations in CAD drawings to achieve semantic navigation tasks. Specifically, it includes: 1. Semantic instruction understanding: When receiving the instruction "go to Zone 3 for pouring", the system searches for the area marked "Zone 3" in the CAD layer of the fused map and obtains the coordinate boundary of the area; 2. Semantic path planning: The navigation algorithm plans a feasible path based on the current SLAM positioning position and the target area coordinates. The path planning considers not only physical obstacles but also semantic constraints (such as information marked in CAD annotations such as certain areas being closed or priority passages); 3. Real-time semantic positioning: During the robot's movement, SLAM provides physical coordinates, which are converted into CAD coordinates through a transformation matrix. The system informs the robot in real time "You are now in Zone 2 and are heading to Zone 3", achieving semantic-level position awareness; 4. Task execution feedback: After reaching the target area, the system confirms whether the designated position has been reached by comparing the SLAM coordinates and the CAD area boundary, and reports the task completion status to the upper-level task system.

[0111] Therefore, in this specific embodiment, the positioning and navigation method has the following significant advantages compared with the prior art: 1. Automatic fusion of heterogeneous maps: It realizes the automatic fusion of laser SLAM point cloud maps and CAD vector drawings, solving the problem of coordinate alignment between two maps of different sources and formats. Through feature point registration and rigid transformation, accurate fusion can be achieved without manual measurement; 2. Provides precise physical layer positioning capability: The physical layer navigation based on laser SLAM provides centimeter-level positioning accuracy, supports real-time obstacle detection and dynamic obstacle avoidance, ensures the robot's safe movement in complex construction environments, and the repositioning function ensures the system can recover quickly in unexpected situations; 3. Makes full use of existing CAD resources: Construction sites already have design drawings. This method directly utilizes existing CAD drawings without additional drawing or annotation, significantly reducing the cost of semantic map construction; 4. Achieves dual-layer navigation of physical and semantic layers: The fused map retains the physical obstacle information scanned in real time by SLAM and integrates the semantic annotations of CAD drawings. The robot can accurately avoid obstacles and understand construction tasks, meeting the dual needs of the construction scene. The physical layer ensures positioning accuracy and safety, while the semantic layer enables task understanding and intelligent planning; 5. Adapting to dynamic construction environments: SLAM continuously updates changes in the physical environment (such as adding material stacks or mobile equipment), and CAD provides stable building structure and zoning information. The combination of the two enables the robot to cope with dynamic obstacles and follow the construction zoning plan; 6. Providing interactive fine-tuning capabilities: The registration process supports manual interactive fine-tuning, allowing manual optimization based on automatic registration, ensuring that the alignment accuracy meets the requirements of actual applications and improving system reliability.

[0112] In summary, the positioning and navigation method according to embodiments of the present invention first acquires a first map and a second map corresponding to the target area. The first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental and semantic information but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability, and applicability of the positioning and navigation.

[0113] Further embodiments of the present invention disclose a positioning and navigation device. Figure 4 This is a structural block diagram of a positioning and navigation device according to an embodiment of the present invention, such as... Figure 4 As shown, the positioning and navigation device 100 includes: an acquisition module 110, a conversion module 120, a fusion module 130, and a processing module 140.

[0114] Specifically, the acquisition module 110 is used to acquire a first map and a second map corresponding to the target area, wherein the first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area; the conversion module 120 is used to convert the first map and the second map into files of a first preset format, respectively, to obtain a first image and a second image; the fusion module 130 is used to overlay and fuse the first image and the second image to obtain a fused map; and the processing module 140 is used to perform positioning and navigation within the target area based on the fused map.

[0115] In one embodiment of the present invention, the acquisition module 110 acquires a first map corresponding to the target area, including: scanning the target area based on a lidar sensor to obtain multiple frames of point cloud data; processing the point cloud data using a preset positioning and mapping algorithm to construct a three-dimensional point cloud map, and using the three-dimensional point cloud map as the first map.

[0116] In one embodiment of the present invention, the conversion module 120 converts the first map into a file of a first preset format to obtain a first image, including: projecting a three-dimensional point cloud map into a two-dimensional raster map to obtain the first image.

[0117] In one embodiment of the present invention, the conversion module 120 projects a three-dimensional point cloud map into a two-dimensional raster map to obtain a first image, including: filtering the Z-axis data of the three-dimensional point cloud map and retaining point cloud data with heights within a preset height range; projecting the filtered three-dimensional point cloud map onto a two-dimensional horizontal plane to obtain a top view point set; dividing the top view point set into a grid according to a preset raster resolution; marking the grid and outputting a first image in a first preset format, wherein grid cells with point cloud projections are marked as occupied and grid cells without point clouds are marked as idle.

[0118] In one embodiment of the present invention, before processing the point cloud data using a preset positioning and mapping algorithm, the acquisition module 110 is further configured to: acquire the pose data of the lidar sensor; compensate the point cloud data based on the pose data; and construct a three-dimensional point cloud map based on the compensated point cloud data.

[0119] In one embodiment of the present invention, when scanning a target area based on a lidar sensor, the acquisition module 110 is further configured to: if the area of ​​the target area exceeds a preset threshold, divide the target area into multiple sub-regions; scan multiple sub-regions to obtain multiple sets of point cloud data; save the multiple sets of point cloud data as a second preset format file; and stitch together the multiple second preset format files to obtain a three-dimensional point cloud map.

[0120] In one embodiment of the present invention, the acquisition module 110 stitches together multiple second preset format files to obtain a three-dimensional point cloud map, including: calculating the normal vector corresponding to each second preset format file; performing ICP coarse registration between each second preset format file and the reference point cloud based on a first distance threshold; after the coarse registration is completed, performing ICP fine registration between each second preset format file based on the second preset distance threshold to obtain a transformation matrix, wherein the first preset distance threshold is greater than the second preset distance threshold; rotating and translating each second preset format file according to the transformation matrix to transform each second preset format file to the reference coordinate system, and merging all point cloud data in each second preset format file to obtain a three-dimensional point cloud map.

[0121] In one embodiment of the present invention, the acquisition module 110 acquires a second map corresponding to the target area, including: acquiring a drawing in a third preset format corresponding to the target area, and using the drawing as the second map, wherein the drawing includes semantic annotation information of the target area.

[0122] In one embodiment of the present invention, the conversion module 120 converts the second map into a file of a first preset format to obtain a second image, including: converting the second map into an image of a fourth preset format, the image of the fourth preset format including the original scale and semantic annotation information of the map; and converting the image of the fourth preset format into a file of the first preset format while keeping the image pixel size and grayscale value unchanged to obtain the second image.

[0123] In one embodiment of the present invention, the fusion module 130 superimposes and fuses a first image and a second image to obtain a fused map, including: selecting target feature points on the first image and the second image respectively; calculating a rigid transformation relationship from the coordinate system of the second image to the coordinate system of the first image based on the target feature points, wherein the rigid transformation relationship includes at least one of rotation, scaling and translation; automatically aligning the first image and the second image based on the rigid transformation relationship; and fusing and superimposing the aligned first image and the second image to obtain a fused map.

[0124] In one embodiment of the present invention, the fusion module 130 selects corresponding feature points on the first image and the second image respectively, including: taking multiple identical feature points on the first image and the second image as target feature points.

[0125] In one embodiment of the present invention, after automatically aligning the first image and the second image based on a rigid transformation relationship, the fusion module 130 is further configured to: receive a fine-tuning instruction for the second map, wherein the fine-tuning instruction includes adjustment information for the position of the second map; and, in response to the fine-tuning instruction, fine-tune the position of the second map to further improve the alignment of the first image and the second image.

[0126] In one embodiment of the present invention, the fusion module 130 fuses and overlays the aligned first image and the second image to obtain a fused map, including: using the first image as a base, overlaying the lines and semantic annotation information in the second image onto it, and covering the corresponding positions in the first image with the pixels containing content in the second image to form a fused map.

[0127] In one embodiment of the present invention, the processing module 140 performs positioning and navigation within a target area based on a fused map, including: acquiring real-time point cloud data of a moving target object; and performing at least one of real-time positioning, obstacle detection, path planning, dynamic obstacle avoidance, and relocation of the target object within the target area based on the real-time point cloud data of the target object and the fused map.

[0128] In one embodiment of the present invention, the processing module 140 is further configured to: receive a semantic control instruction for a target moving object, the semantic control instruction including at least one of the following: target position, target path planning, real-time position prompt of the target moving object, and task execution feedback information; and respond to the semantic control instruction by controlling the target moving object to perform corresponding actions based on the fused map.

[0129] It should be noted that the specific implementation of the positioning and navigation device 100 in this embodiment of the invention is similar to the specific implementation of the positioning and navigation method described in the above embodiment of the invention. For details, please refer to the description of the positioning and navigation method section. To reduce redundancy, it will not be repeated here.

[0130] According to the positioning and navigation device 100 of the present invention, the positioning and navigation method of the above embodiment of the present invention is executed. First, a first map and a second map corresponding to the target area are obtained. The first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability and applicability of positioning and navigation.

[0131] A further embodiment of the present invention also discloses a robot.

[0132] In some embodiments, the robot includes a positioning and navigation device 100 as described in the above embodiments of the present invention.

[0133] In other embodiments, the robot includes a processor, a memory, and a positioning and navigation program stored in the memory and executable on the processor, wherein the positioning and navigation program, when executed by the processor, implements the positioning and navigation method as described in any of the above embodiments of the present invention.

[0134] It should be noted that the specific implementation of the robot in this embodiment of the invention is similar to the specific implementation of the positioning and navigation method described in the above embodiments of the invention. For details, please refer to the description in the positioning and navigation method section. To reduce redundancy, it will not be repeated here.

[0135] According to an embodiment of the present invention, a robot is used to execute the positioning and navigation method of the above embodiments of the present invention. First, a first map and a second map corresponding to a target area are obtained. The first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability, and applicability of positioning and navigation.

[0136] Further embodiments of the present invention disclose a computer-readable storage medium storing a positioning and navigation program, which, when executed by a processor, implements the positioning and navigation method as described in any of the above embodiments of the present invention.

[0137] It should be noted that the specific implementation of the computer-readable storage medium in the embodiments of the present invention is similar to the specific implementation of the positioning and navigation method described in the above embodiments of the present invention. For details, please refer to the description in the positioning and navigation method section. In order to reduce redundancy, it will not be repeated here.

[0138] According to an embodiment of the present invention, when a positioning and navigation program stored thereon is executed by a processor, the positioning and navigation method of the above embodiment of the present invention is executed. First, a first map and a second map corresponding to the target area are obtained. The first map is generated from point cloud data of the target area, and the second map includes semantic annotation information of the target area. Then, the first map and the second map are converted into files of a first preset format, respectively, to obtain a first image and a second image. Next, the first image and the second image are superimposed and fused to obtain a fused map. Finally, positioning and navigation within the target area are performed based on the fused map. That is, positioning and navigation within the target area are performed based on the fused map that integrates point cloud information and semantic annotation information. This not only enables intelligent navigation based on environmental information and semantic information, but also adapts to different usage scenarios, thereby ensuring the intelligence, reliability, and applicability of positioning and navigation.

[0139] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0140] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A positioning and navigation method, characterized in that, Includes the following steps: Obtain a first map and a second map corresponding to the target area, wherein the first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area; The first map and the second map are converted into files of a first preset format, respectively, to obtain the first image and the second image. The first image and the second image are overlaid and fused to obtain a fused map; Positioning and navigation within the target area are performed based on the fused map.

2. The positioning and navigation method according to claim 1, characterized in that, Obtain the first map corresponding to the target area, including: The target area is scanned using a lidar sensor to obtain multiple frames of point cloud data. The point cloud data is processed using a preset positioning and mapping algorithm to construct a three-dimensional point cloud map, which is then used as the first map.

3. The positioning and navigation method according to claim 2, characterized in that, Converting the first map into a file of a first preset format to obtain the corresponding first image includes: The three-dimensional point cloud map is projected into a two-dimensional raster map to obtain the first image.

4. The positioning and navigation method according to claim 3, characterized in that, The step of projecting a 3D point cloud map into a 2D raster map to obtain the first image includes: The Z-axis data of the three-dimensional point cloud map is filtered to retain point cloud data with heights within a preset height range; The filtered 3D point cloud map is projected onto a 2D horizontal plane to obtain a top view point set; The top view point set is divided into a grid according to a preset grid resolution; The grid is marked, and a first image in the first preset format is output, wherein grid cells with point cloud projections are marked as occupied, and grid cells without point clouds are marked as idle.

5. The positioning and navigation method according to claim 2, characterized in that, Before processing the point cloud data using a preset localization and mapping algorithm, the process also includes: Acquire the pose data of the lidar sensor; The point cloud data is compensated based on the pose data; The 3D point cloud map is constructed based on the compensated point cloud data.

6. The positioning and navigation method according to claim 2, characterized in that, When scanning the target area based on a lidar sensor, the method further includes: If the area of ​​the target region exceeds a preset threshold, the target region is divided into multiple sub-regions; Multiple sets of point cloud data were obtained by scanning multiple sub-regions; Save the multiple sets of point cloud data as a second preset format file; Multiple files in the second preset format are stitched together to obtain the three-dimensional point cloud map.

7. The positioning and navigation method according to claim 6, characterized in that, The three-dimensional point cloud map is obtained by stitching together multiple files in the second preset format, including: Calculate the normal vectors corresponding to each of the second preset format files; Based on the first distance threshold, each of the second preset format files and the reference point cloud are coarsely registered using ICP. After coarse registration is completed, ICP fine registration is performed on each of the second preset format files based on the second preset distance threshold to obtain a transformation matrix, wherein the first preset distance threshold is greater than the second preset distance threshold; Each of the second preset format files is rotated and translated according to the transformation matrix to transform each of the second preset format files to the reference coordinate system, and all point cloud data in each of the second preset format files are merged to obtain the three-dimensional point cloud map.

8. The positioning and navigation method according to claim 1, characterized in that, Obtain the second map corresponding to the target area, including: Obtain a drawing in a third preset format corresponding to the target area, and use the drawing as the second map, wherein the drawing includes semantic annotation information of the target area.

9. The positioning and navigation method according to claim 8, characterized in that, Convert the second map into a file of a first preset format to obtain the corresponding second image, including: The second map is converted into an image in a fourth preset format, the image in the fourth preset format including the original scale of the map and the semantic annotation information; Without changing the image pixel size and grayscale value, the image in the fourth preset format is converted into a file in the first preset format to obtain the second image.

10. The positioning and navigation method according to claim 1, characterized in that, The step of overlaying and fusing the first image and the second image to obtain a fused map includes: Select target feature points on the first image and the second image respectively; Based on the target feature points, calculate the rigid transformation relationship from the coordinate system of the second image to the coordinate system of the first image, wherein the rigid transformation relationship includes at least one of rotation, scaling and translation; The first image and the second image are automatically aligned based on the rigid transformation relationship; The aligned first and second images are then merged and overlaid to obtain the merged map.

11. The positioning and navigation method according to claim 10, characterized in that, The step of selecting corresponding feature points on the first image and the second image respectively includes: Multiple identical feature points on the first image and the second image are used as the target feature points.

12. The positioning and navigation method according to claim 10, characterized in that, After automatically aligning the first image and the second image based on the rigid transformation relationship, the process further includes: Receive a fine-tuning instruction for the second map, wherein the fine-tuning instruction includes adjustment information for the position of the second map; In response to the fine-tuning command, the position of the second map is fine-tuned to further improve the alignment between the first image and the second image.

13. The positioning and navigation method according to claim 10, characterized in that, The step of fusing and overlaying the aligned first and second images to obtain the fused map includes: Using the first image as a base, the lines and semantic annotation information from the second image are overlaid on it, and the pixels containing content in the second image are overlaid onto the corresponding positions in the first image to form the fused map.

14. The positioning and navigation method according to claim 1, characterized in that, The positioning and navigation within the target area based on the fused map includes: Acquire real-time point cloud data of the target moving object; Based on the real-time point cloud data of the target object and the fused map, the target object is subjected to at least one of the following in the target area: real-time localization, obstacle detection, path planning, dynamic obstacle avoidance, and relocalization.

15. The positioning and navigation method according to claim 1, characterized in that, Also includes: Receive semantic control instructions for a target moving object, the semantic control instructions including at least one of the following: target position, target path planning, real-time position prompt of the target moving object, and task execution feedback information; In response to the semantic control command, the target moving object is controlled to perform corresponding actions based on the fused map.

16. A positioning and navigation device, characterized in that, include: The acquisition module is used to acquire a first map and a second map corresponding to the target area, wherein the first map is generated from the point cloud data of the target area, and the second map includes the semantic annotation information of the target area; The conversion module is used to convert the first map and the second map into files of a first preset format, respectively, to obtain a first image and a second image. The fusion module is used to overlay and fuse the first image and the second image to obtain a fused map; The processing module is used for positioning and navigation within the target area based on the fused map.

17. A robot, characterized in that, include: The positioning and navigation device as described in claim 16; or, A processor, a memory, and a positioning and navigation program stored in the memory and executable on the processor, wherein the positioning and navigation program, when executed by the processor, implements the positioning and navigation method as described in any one of claims 1-15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a positioning and navigation program, which, when executed by a processor, implements the positioning and navigation method as described in any one of claims 1-15.

Images