A map construction method, device and storage medium

Abstract

The application discloses a kind of map construction method, equipment and storage medium, the method comprises: the image of multi-view of target environment is collected, the bird's eye view of the target environment is generated;Current frame semantic point cloud corresponding to the target environment is obtained based on the bird's eye view;At least part of the space points in the current frame semantic point cloud is as current frame target semantic point cloud, and based on the current frame target semantic point cloud and the motion state data of robot current, the target pose of the robot is determined;According to the target pose, the current frame semantic point cloud is mapped to each grid cell under map coordinate system;Based on the state parameters of each grid cell, the grid map of the target environment is generated.

Description

Technical Field

[0001] This application relates to the field of robot positioning and navigation technology, specifically to a map building method, device, and storage medium. Background Technology

[0002] Mapping technology is widely used in core fields such as autonomous driving, intelligent robot navigation, geographic information mapping, indoor positioning services, and virtual reality (VR) / augmented reality (AR) spatial modeling. Among them, grid maps, with their advantages of intuitive environmental representation, adaptability to low-cost sensors, and ease of integration with path planning algorithms, are widely used in indoor robot navigation, low-speed unmanned vehicle driving, and warehouse AGV scheduling.

[0003] The core objective of grid map construction is to accurately depict the distribution of environmental obstacles and spatial characteristics through sensor-sensed data, providing reliable support for the positioning and navigation decisions of downstream devices. The accuracy of its construction directly determines the operational safety and task completion efficiency of intelligent devices. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides a map building method, apparatus, and storage medium to improve the accuracy of map building.

[0005] According to one embodiment of this application, a map construction method is provided, including: A bird's-eye view of the target environment is generated by acquiring images from multiple perspectives of the target environment. Based on the bird's-eye view, obtain the semantic point cloud of the current frame corresponding to the target environment; At least a portion of the spatial points in the current frame semantic point cloud are used as the current frame target semantic point cloud, and the target pose of the robot is determined based on the current frame target semantic point cloud and the robot's current motion state data. Based on the target pose, the semantic point cloud of the current frame is mapped to each grid cell in the map coordinate system; A raster map of the target environment is generated based on the state parameters of each raster unit.

[0006] To solve the above-mentioned technical problems, one technical solution adopted in this application is to provide an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, it is used to implement the map construction method in the above-mentioned technical solution.

[0007] To solve the above-mentioned technical problems, one technical solution adopted in this application is to provide a computer-readable storage medium for storing a computer program, which, when executed by a processor, is used to implement the map construction method in the above-mentioned technical solution.

[0008] Through the above scheme, this application generates a bird's-eye view of the target environment by acquiring images from multiple perspectives, thereby breaking the occlusion and perspective limitations of single-view images on environmental information. This allows the semantic point cloud of the current frame obtained based on the bird's-eye view to cover a more comprehensive spatial structure and scene details of the target environment. Furthermore, at least some spatial points in the semantic point cloud of the current frame are used as the target semantic point cloud of the current frame. Based on the target semantic point cloud of the current frame and the robot's current motion state data, the robot's target pose is determined. Then, based on the target pose, the semantic point cloud of the current frame is mapped to each grid cell in the map coordinate system. Based on the state parameters of each grid cell, a grid map of the target environment is generated, thereby generating a grid map that integrates geometric and semantic information, thus improving the accuracy of map construction. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating an embodiment of the map construction method provided in this application; Figure 2 This is a flowchart illustrating another embodiment of the map construction method provided in this application; Figure 3 This is a schematic diagram of the distortion correction and bird's-eye view transformation effect provided in this application; Figure 4 This is a schematic diagram of noise in a semantic segmentation result provided in this application; Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application; Figure 6 This is a schematic diagram of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation

[0010] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the application. Similarly, the following embodiments are only some, not all, embodiments of the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0011] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0012] It should be noted that the terms "first," "second," etc., used in this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0013] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the map construction method provided in this application. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that result. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this embodiment includes: S110: Generate a bird's-eye view of the target environment by acquiring images from multiple perspectives of the target environment.

[0014] Multi-view image acquisition is a set of images containing local or global information about a target environment, acquired from different spatial orientations at the same height using an acquisition device. Because of the spatial perspective differences between multi-view images, they can complementaryly cover information that is difficult to capture from a single viewpoint, such as occluded areas and complex spatial structures. In one embodiment, four cameras (front, back, left, and right) installed at the same height and tilted downwards are used to acquire a set of multi-view images in the target environment.

[0015] A bird's-eye view (BEV), also known as a top-down view, is a global image presented from directly above or nearly directly above the target environment. It can intuitively reflect the spatial relationships, layout structure, and scale of various objects and areas in the environment, breaking through the limitations of conventional perspectives.

[0016] In one embodiment, distortion correction and inverse perspective transformation are performed on each of the multi-view acquired images to obtain a bird's-eye view image corresponding to each view acquired image. The multiple bird's-eye view images are then stitched together to obtain a bird's-eye view of the target environment.

[0017] In one embodiment, based on an image acquisition module onboard the robot, such as a monocular color camera, continuous or discrete images of the target environment are acquired at preset intervals and angles during robot movement or through multiple preset fixed acquisition points. The acquisition pose corresponding to each image is recorded synchronously during the acquisition process. Before generating a bird's-eye view from the multi-view acquired images, preprocessing can be performed on the acquired multi-view images, including distortion correction, exposure correction, feature point matching, and image registration, to eliminate geometric distortion and pixel misalignment between different viewpoints. Finally, using multi-view geometric algorithms such as perspective projection transformation, the pixel information of the registered multi-view images is mapped to a preset bird's-eye view coordinate system, and a bird's-eye view covering the target environment is generated, realizing the conversion from multi-view local observation to a unified top-down perspective.

[0018] S120: Obtain the semantic point cloud of the target environment in the current frame based on the bird's-eye view.

[0019] The current frame semantic point cloud refers to the point cloud data corresponding to the robot's current operation time (or the current data acquisition cycle), which contains the three-dimensional spatial information and semantic attributes of the target environment. Its core feature is that each three-dimensional spatial point is associated with a clear semantic category label, such as obstacle, passable area, road marking, target object, etc. It can reflect the geometric spatial structure of the environment and distinguish the categories of objects in the environment.

[0020] In one embodiment, a preset semantic segmentation model is used to perform semantic segmentation on the bird's-eye view to obtain a semantically labeled bird's-eye view. Based on the semantically labeled bird's-eye view, a spatial dimension transformation is performed to obtain the semantic point cloud of the current frame corresponding to the target environment. The semantic labels of the pixels in the semantically labeled bird's-eye view can be used to represent the category of the environmental object to which the pixel belongs.

[0021] In one implementation, a pre-trained deep learning semantic segmentation model is used to perform semantic segmentation on the generated bird's-eye view of the target environment. Based on a preset semantic category system, each pixel in the bird's-eye view is semantically labeled, and a semantic segmentation mask containing pixel-level semantic labels is output. Further, combining the camera intrinsic and extrinsic parameters used in generating the bird's-eye view and the definition of the bird's-eye view coordinate system, a defined transformation matrix is ​​used to map the two-dimensional coordinates of each pixel in the bird's-eye view to three-dimensional space, obtaining the corresponding three-dimensional spatial point coordinates. Finally, the semantic category label of each pixel in the semantic segmentation mask is bound one-to-one with the three-dimensional spatial point coordinates obtained by mapping that pixel, forming a point cloud dataset containing three-dimensional spatial coordinates and semantic labels, which is the semantic point cloud of the current frame corresponding to the target environment.

[0022] In one embodiment, since semantic segmentation results may produce certain false detections and noise, some abnormal noise points may exist, which may affect the subsequent registration and mapping results. To further improve the accuracy of the semantic bird's-eye view, this embodiment uses edge detection and contour extraction. By extracting the closed contour of the segmentation result, the area of ​​the closed contour is used to determine whether it is an abnormal noise point. For example, the closed contour in the semantic bird's-eye view is extracted, the contour area of ​​the image region where the closed contour is located is calculated, and in response to the contour area being less than a preset threshold, the semantic labels of each pixel within the closed contour are removed.

[0023] S130: Take at least a portion of the spatial points in the current frame semantic point cloud as the current frame target semantic point cloud, and determine the robot's target pose based on the current frame target semantic point cloud and the robot's current motion state data.

[0024] The target semantic point cloud of the current frame refers to a set of three-dimensional spatial points selected from the semantic point cloud of the current frame that correspond to the main semantic label, including several target semantic points. The main semantic label is determined based on multiple semantic labels (such as lane lines, left turn signs and other ground marking related labels) obtained after semantic segmentation of the semantic point cloud of the current frame, specifically the semantic label that has the dominant proportion among the multiple semantic labels.

[0025] The robot's current motion state data refers to the set of kinematic parameters acquired by the robot at the current moment or during the current data acquisition period through its own sensors, such as inertial measurement units (IMUs), odometry, and wheel speed sensors. These parameters include, but are not limited to, the robot's linear velocity, angular velocity, acceleration, cumulative odometry displacement, and attitude change, and are used to reflect the robot's real-time motion trend and preliminary pose estimation results. In one embodiment, the robot's current motion state data includes the robot's initial predicted pose, which is currently predicted by the robot's inertial measurement unit.

[0026] The target pose of a robot refers to the precise three-dimensional position and orientation of the robot in a preset map coordinate system. It is used to determine the robot's spatial coordinates and orientation in the environment, providing a positioning reference for subsequent map updates and path planning.

[0027] In one embodiment, target semantic points in the target semantic point cloud are registered with reference semantic points in the reference sub-map to determine the optimal pose change. The initial predicted pose and the optimal pose change are then fused to determine the target pose. The target semantic points and reference semantic points are spatial points with preset semantic labels.

[0028] In one embodiment, a reference sub-map is constructed based on a pre-set number of target semantic point clouds. The target semantic point cloud of the current frame is transformed to the map coordinate system according to the initial predicted pose. For each target semantic point in the target semantic point cloud of the current frame, a reference semantic point matching the target semantic point is found in the reference sub-map to form a matching point pair. By minimizing the spatial error between each matching point pair, the optimal pose change is determined. Then, the initial predicted pose and the optimal pose change are fused to determine the target pose.

[0029] In one example, the main semantic label point cloud, such as road markings, is extracted from the semantic point cloud of the current frame based on preset filtering rules. Motion parameters are collected in real time by sensors such as the robot's IMU (Inertial Measurement Unit) and odometry. The data is preprocessed to reduce noise and obtain stable motion state data. Based on this data, the robot's initial pose is predicted by a kinematic model.

[0030] The current frame target semantic point cloud is matched with the historical main semantic label point cloud stored in the map to obtain the pose estimation result based on semantic features. Then, through a multi-source data fusion algorithm, the semantic feature pose estimation result is fused with the initial pose predicted by motion state data to correct the estimation error of a single data source. Finally, an accurate and robust robot target pose is output.

[0031] By filtering key semantic point clouds to remove redundant interference and fusing robot motion state data to achieve multi-source constraints, the accuracy and robustness of robot pose estimation are effectively improved, providing a reliable positioning benchmark for subsequent semantic point cloud map mapping.

[0032] S140: Map the semantic point cloud of the current frame to each grid cell in the map coordinate system according to the target pose.

[0033] Semantic point cloud mapping refers to the process of transforming the semantic point cloud of the current frame to the map coordinate system based on the robot's target pose, and then assigning the semantic and geometric information of the point cloud to the corresponding grid cells.

[0034] In one embodiment, the preset map coordinate system is defined including the origin, coordinate axis directions, range, and the scale of the grid cells, while the determined robot target pose is used as the reference for coordinate transformation. For each 3D spatial point in the current frame's semantic point cloud, a rigid transformation matrix from the local coordinate system (the robot's own coordinate system) to the map coordinate system is constructed based on the robot target pose. Matrix operations are then used to convert the local coordinates of the point into global coordinates in the map coordinate system. Next, based on the global coordinates in the map coordinate system, the target grid cell corresponding to each transformed semantic point cloud is determined using grid index calculation rules. Finally, the semantic labels, spatial confidence scores, and other information of each semantic point cloud are bound and stored with the corresponding grid cell. If multiple semantic point clouds are mapped to the same grid cell, the core semantic attributes and state parameters of the grid cell can be determined through a voting mechanism (such as semantic label frequency statistics) or a confidence-weighted fusion algorithm.

[0035] In this embodiment, the precise target pose of the robot is used as a spatial reference. Through coordinate transformation and point cloud matching, the semantic point cloud in the local coordinate system is transformed into the semantic-geometric fusion information of each grid unit in the map coordinate system. This enables the precise alignment and structured storage of local environmental data to the global grid map, thereby providing precise data support at the unit level for subsequent grid map generation.

[0036] S150: Generate a raster map of the target environment based on the state parameters of each raster cell.

[0037] The state parameters of a raster cell refer to the probability data (such as occupancy probability, idle probability, confidence level of a specific semantic category, etc.) that quantifies the "occupancy," "idleness," or "semantic category affiliation" of the spatial area covered by that cell. State parameters can reduce the impact of single-frame data noise on the map through probability quantization, thereby improving the reliability and stability of the map. In one embodiment, the state parameters include the probability values ​​of the raster cells.

[0038] A grid map of a target environment refers to a global / local map formed by integrating the probability values ​​of each grid cell using discrete grid cells as the basic unit. It can quantitatively represent the spatial occupancy status and semantic attributes of the target environment, and has both a discretized representation of geometric space and a probabilistic description of state information.

[0039] In one embodiment, the occupancy category of each grid cell is determined, including occupied and idle categories. In response to a grid cell being occupied, the probability value of the grid cell is updated using a first update probability method; in response to a grid cell being idle, the probability value of the grid cell is updated using a second update probability method.

[0040] In one implementation, a blank raster map is constructed based on a preset map coordinate system range and raster scale, and the initial probability value of all raster units is set to a default value, such as 0.5. The probability data of each raster unit in the semantic point cloud information is acquired, and combined with a Bayesian update rule or a probability-weighted fusion algorithm, the raster probability value of the current frame is fused with the historical probability values ​​stored in the map to eliminate the influence of single-frame data noise or random errors. A sensor observation probability threshold is set; for example, when the occupancy probability > 0.7, the raster is determined to be an occupied raster; when the occupancy probability < 0.3, the raster is determined to be an idle raster; and raster units in between are still considered unknown. The state of each raster unit is clearly determined based on the fused probability value. Finally, the determined raster map is post-processed and optimized to generate a complete, accurate, and reliable target environment raster map.

[0041] This embodiment transforms the probability information of discrete grid cells into a structured grid map that can accurately represent the environmental occupancy status and semantic attributes by initializing grid probability values, fusing multiple frames, and determining the state. This improves the reliability and accuracy of the map's description of the environment.

[0042] It's important to note that in the original image, objects far from the camera (such as the far end of the ground) have a smaller pixel footprint and lower resolution due to perspective projection. When inverse perspective transformation (IPM) is performed to convert it into a bird's-eye view, the pixels in these areas are significantly stretched, resulting in blurred edges. Consequently, areas near the edges are susceptible to uneven ground and IPM stitching errors, leading to a deviation between pixel scale and reality. This can cause missed detections in semantic segmentation tasks, where objects with semantic categories may exist in blurred areas, while false positives may occur due to scale errors in edge segmentation caused by stitching.

[0043] In one embodiment, a first image region and a second image region are determined in the bird's-eye view. The first image region is an image region of a preset size and shape, and a preset position point of the first image region coincides with a preset position point of the bird's-eye view. For a first grid cell corresponding to a spatial point within the first image region, in response to the first grid cell's occupancy category being occupied, the probability value of the first grid cell is updated with a first probability value; in response to the first grid cell's occupancy category being idle, the probability value of the first grid cell is updated with a second probability value. For a second grid cell corresponding to a spatial point within the second image region, in response to the second grid cell's occupancy category being occupied, the probability value of the second grid cell is updated with a third probability value; in response to the second grid cell's occupancy category being idle, the probability value of the second grid cell is not updated. The third probability value is less than the first probability value.

[0044] In one embodiment, a rectangular region defined as the area bounded by the center of the bird's-eye view image (n1 above, n2 below, n3 to the left, and n4 to the right) is designated as the effective region (corresponding to the first image region), and the area outside this region is designated as the low-confidence region (corresponding to the second image region). The update probability when an observation occupies the effective region is defined as p. occ The update probability during observation idle time is p. free The update probability when low-confidence region observations occupy a certain period is p. low_occ The observation is not updated during idle periods, where p low_occ Greater than 0.5 and less than p occ Among them, p occ p free and p low_occ The effective area can be set according to actual historical experience values. In practical applications, the division of the effective area can be designed with arbitrary shapes such as ellipse and circle, depending on the performance of the sensor.

[0045] Please see Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the map construction method provided in this application. This embodiment uses an automated guided vehicle (AGV) in a warehouse scenario to construct a ground-marked map, serving as an auxiliary positioning method in complex scenarios such as those with numerous dynamic factors or dirty QR codes. It should be noted that if substantially the same result is achieved, this embodiment does not necessarily reflect that outcome. Figure 2 The sequence of processes shown is limited.

[0046] In this embodiment, the mobile robot used includes a mobile chassis, multiple color cameras at a certain height above the ground with a shared field of view, an inertial measurement unit (IMU), etc. The mobile chassis includes a motion controller, motors, a battery, an embedded computer, etc. For forklift robots in warehouse scenarios, a multi-camera surround-view system acquires environmental images in real time, generates a bird's-eye view from a top-down perspective through inverse perspective transformation (IPM), and constructs a probabilistic raster semantic map using semantic segmentation fusion technology. During the dynamic map construction process, the robot gradually eliminates single-view occlusion and measurement noise through multi-frame observation fusion and temporal probability updates, ultimately outputting a more accurate static environmental map with semantic annotations. Figure 2 As shown, this embodiment includes: Please see Figure 3 , Figure 3 This is a schematic diagram illustrating the distortion correction and bird's-eye view transformation effect provided in this application. Original images acquired from multiple perspectives by sensors often contain distortion. To improve the accuracy of subsequent processing, eliminate lens distortion, and restore true geometry, distortion correction is needed to map the image onto an ideal imaging plane.

[0047] In this embodiment, the Kannala-Brand (KB, a method for modeling and calibrating geometric distortion of fisheye / wide-angle cameras) model is used to achieve nonlinear transformation of pixel coordinates using the following formula (1), where, , , , These are the calibrated distortion parameters. The distance from a pixel to the center of the image. The angle of incidence of the light ray is denoted as α.

[0048] Formula (1) After distortion correction of the original multi-view image, inverse perspective transformation is performed using the following formulas (2) and (3) to transform the corrected image into a bird's-eye view (BEV) perspective. For any pixel in the BEV perspective... Formula (2) can be used to transform the pixel coordinates of a camera in a multi-view perspective. .

[0049]

[0050] Formula (2) Formula (3) Where b is defined as the front Left The vehicle body coordinate system is defined. For the right back The vehicle body coordinate system. for The transformation matrix from the vehicle coordinate system to BEV pixel coordinates, assuming that the Z-axis coordinates of all pixels within the field of view are zero, and the camera extrinsic parameters are... Simplified to , As shown in formula (2), the homography matrix of camera pixels transformed into BEV pixel coordinates can be obtained based on the camera intrinsic and extrinsic parameters. Finally, the multi-view BEV images are stitched together and the overlapping parts are weighted to obtain a panoramic multi-view bird's-eye view.

[0051] After obtaining the bird's-eye view of the current frame, it is input into the semantic reasoning module. Based on a preset semantic category system, pixel-level semantic label reasoning is performed, and the corresponding semantic segmentation result is output. It should be noted that during semantic segmentation, factors such as blurred environmental textures and changes in lighting can easily lead to false detections and noise. For example, please refer to [link to relevant documentation]. Figure 4 , Figure 4This is a schematic diagram of noise in a semantic segmentation result provided in this application. The area highlighted in the diagram represents tiny noise. If such noise is directly used for subsequent semantic point cloud generation, feature registration, and raster map construction, it will lead to point cloud noise accumulation, feature matching misjudgment, and ultimately reduce map construction accuracy and positioning reliability.

[0052] In this embodiment, noise reduction is achieved using edge detection, contour extraction, and area thresholding. For example, firstly, in the edge detection stage, a second-order Laplacian operator is used for edge localization. The edge positions are extracted by leveraging the zero-crossing property of its second derivative at the edge, and a binarized edge map is generated through thresholding. Subsequently, in the contour extraction stage, the Suzuki85 boundary tracking algorithm (a fully automatic contour extraction and topology analysis algorithm for binarized images) is used to perform topological analysis on the binarized edges, extracting the pixel set corresponding to the closed contour and establishing contour hierarchy relationships. Finally, in the area calculation stage, Green's theorem discretization algorithm is applied to the obtained closed contour point set, and the precise area of ​​the region enclosed by the contour is accurately calculated through coordinate integration to eliminate pixel sets with areas smaller than a preset threshold.

[0053] The pixels with effective semantic values ​​in the semantically segmented bird's-eye view are then processed using formula (4). Transform to actual vehicle coordinate system points By traversing all pixels sequentially, semantic point clouds of different categories can be obtained.

[0054] Formula (4) In this embodiment, a primary semantic label, primarily based on ground-marking semantic labels, is selected to predict the robot's initial pose. Based on IMU data aligned with the current frame's point cloud timestamp, interpolation is used to obtain the initial pose prediction result. Further, precise pose calibration is performed using the point cloud corresponding to the primary semantic label as the core. Specifically, for the i-th frame's primary semantic point cloud P, it is first transformed to the world coordinate system based on the IMU-predicted pose. Then, a sub-map M is constructed by stitching together the primary semantic point clouds from the previous N frames. For each point p in point cloud P... i Search for the nearest neighbor m in the submap M. i Form point pairs (p) i , m i The optimal pose change is solved by formula (5) to minimize the spatial distance error between the two sets of point clouds. Based on the pose change, the spatial position of point cloud P is transformed again. The above matching-solving-transformation process is iteratively executed until the pose change is less than the preset threshold. Finally, the target pose (R, t) of the robot is output, where R is the rotation matrix and t is the translation vector.

[0055] Formula (5) After obtaining the robot's target pose, using this pose as the transformation reference, the complete semantic point cloud of the i-th frame is mapped to the world coordinate system through coordinate transformation, obtaining semantic point cloud data in the world coordinate system, and storing it in a global raster map with a preset resolution of r. For two-dimensional planar points in the world coordinate system... ( z =0 corresponds to a ground-view perspective, consistent with the two-dimensional planar layout of the raster map, and it is related to the raster index in the raster map. The corresponding transformation relationship follows formula (6), where and These are the minimum coordinates of the global raster map in the X and Y axes of the world coordinate system, respectively, used to determine the starting boundary and index offset of the raster map.

[0056] Formula (6) Set the initial occupancy probability of each grid cell in the global raster map. The value is set to 0.5, and two key observation probability parameters are preset. (The probability that a grid is actually occupied when the sensor observes it as "occupied") and (The probability that a grid is actually occupied when the sensor observes it as "idle"), among which The value is greater than 0.5 and less than 1. The value is greater than 0 and less than 0.5.

[0057] To avoid numerical underflow and computational complexity issues caused by direct product of probabilities, this embodiment uses odd probability. grid (The logarithmic ratio of the probability) The grid probability is updated as shown in formula (7): When the sensor observes the grid as "occupied", the grid occupancy probability is updated iteratively through formula (8) and then the updated occupancy probability is mapped; when the sensor observes the grid as "idle", the occupancy probability is updated and the probability is mapped through formula (9) and finally the grid occupancy probability is dynamically calibrated to ensure that the grid state gradually converges to the real environment state with multiple frames of observation data.

[0058] Formula (7) Formula (8) Formula (9) Meanwhile, this embodiment also sets confidence levels for different regions of the bird's-eye view. High-sensitivity updates are used within the effective areas to ensure that even if real objects are missed by the semantic model, they can still be quickly corrected through sensor observation. In low-confidence areas, only the occupied state is conservatively updated to avoid introducing false obstacles due to edge misdetection or stitching errors. This solves the problems of missed detection accumulation and false detection spread caused by uniform updates in traditional grid maps.

[0059] At this point, the pose optimization, world coordinate system mapping, and grid probability update processes for the semantic point cloud of frame i are all completed, thus completing the construction of a single-frame grid map. The updated grid map has integrated the environmental semantic and geometric information of the current frame. As the robot continues to move in the target environment, new multi-view images and IMU data will be collected synchronously. Following the process provided in this embodiment, the data processing loop for frame i+1 will begin. Through iterative accumulation and probability fusion of data from multiple frames, the accurate convergence and dynamic updating of the global grid map will be gradually achieved.

[0060] Please see Figure 5 , Figure 5 This is a schematic diagram of an embodiment of the electronic device provided in this application. The electronic device 60 includes a memory 61 and a processor 62 that are interconnected. The memory 61 is used to store a computer program. When the computer program is executed by the processor 62, it is used to implement the map building method in the above embodiment.

[0061] The methods described in the above embodiments can exist in the form of a computer program; therefore, this application proposes a computer-readable storage medium. Please refer to [link / reference needed]. Figure 6 , Figure 6 This is a schematic diagram of an embodiment of a computer-readable storage medium provided in this application. The computer-readable storage medium 80 is used to store a computer program 81, which can be executed to implement the map construction method in the above embodiment.

[0062] The computer-readable storage medium 80 can be any medium capable of storing program code, such as a server, USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0063] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A map construction method, characterized in that, The method includes: A bird's-eye view of the target environment is generated by acquiring images from multiple perspectives of the target environment. Based on the bird's-eye view, obtain the semantic point cloud of the current frame corresponding to the target environment; At least a portion of the spatial points in the current frame semantic point cloud are used as the current frame target semantic point cloud, and the target pose of the robot is determined based on the current frame target semantic point cloud and the robot's current motion state data. Based on the target pose, the semantic point cloud of the current frame is mapped to each grid cell in the map coordinate system; A raster map of the target environment is generated based on the state parameters of each raster unit.

2. The method according to claim 1, characterized in that, The step of generating a bird's-eye view of the target environment by acquiring images from multiple perspectives includes: Each of the multi-view acquired images is subjected to distortion removal and inverse perspective transformation to obtain a bird's-eye view image corresponding to each of the multi-view acquired images. Multiple bird's-eye view images are stitched together to obtain the bird's-eye view of the target environment.

3. The method according to claim 1, characterized in that, The step of obtaining the semantic point cloud of the target environment in the current frame based on the bird's-eye view includes: The bird's-eye view is semantically segmented using a preset semantic segmentation model to obtain a semantically labeled bird's-eye view. Based on the semantic bird's-eye view, the semantic point cloud of the current frame corresponding to the target environment is obtained through spatial dimension transformation.

4. The method according to claim 3, characterized in that, The semantic labels of pixels in the semantic bird's-eye view represent the category of the environmental objects to which the pixel belongs; And / or, after performing semantic segmentation processing on the bird's-eye view using a preset semantic segmentation model to obtain a semantically labeled bird's-eye view, the method further includes: Extract the closed contours from the semantic bird's-eye view; Calculate the area of ​​the image region containing the closed contour; In response to the contour area being less than a preset threshold, the semantic labels of each pixel within the closed contour are cleared.

5. The method according to claim 1, characterized in that, The current frame target semantic point cloud includes several target semantic points, and the robot's current motion state data includes the robot's initial predicted pose; determining the robot's target pose based on the current frame target semantic point cloud and the robot's current motion state data includes: The target semantic points in the target semantic point cloud are registered with the reference semantic points in the reference sub-map to determine the optimal pose change, wherein the target semantic points and the reference semantic points are spatial points with preset semantic labels; The target pose is determined by fusing the initial predicted pose and the optimal pose change.

6. The method according to claim 5, characterized in that, The initial predicted pose is currently predicted by the robot's inertial measurement unit; And / or, the step of registering the target semantic points in the target semantic point cloud with the reference semantic points in the reference sub-map to determine the optimal pose change includes: The reference sub-map is constructed based on a pre-set number of target semantic point clouds; Based on the initial predicted pose, the semantic point cloud of the target in the current frame is transformed to the map coordinate system; For each target semantic point in the target semantic point cloud of the current frame, a reference semantic point that matches the target semantic point is found in the reference sub-map to form a matching point pair; The optimal pose change is determined by minimizing the spatial error between each pair of matching points.

7. The method according to claim 1, characterized in that, The state parameters include the probability values ​​of the grid cells. Generating a grid map of the target environment based on the state parameters of each grid cell includes: Determine the occupancy category of each grid cell, the occupancy category including occupied and idle; In response to the grid cell's occupancy category being "occupancy", the probability value of the grid cell is updated using a first update probability method; In response to the grid cell's occupancy category being "idle", the probability value of the grid cell is updated using a second update probability method.

8. The method according to claim 1, characterized in that, The method further includes: A first image region and a second image region are determined in the bird's-eye view. The first image region is an image region of a preset size and shape, and the preset position point of the first image region coincides with the preset position point of the bird's-eye view. For the first grid cell corresponding to a spatial point within the first image region, when the occupancy category of the first grid cell is occupied, the probability value of the first grid cell is updated with a first probability value; when the occupancy category of the first grid cell is idle, the probability value of the first grid cell is updated with a second probability value. For the second grid cell corresponding to a spatial point within the second image region, when the occupancy category of the second grid cell is occupied, the probability value of the second grid cell is updated with a third probability value; when the occupancy category of the second grid cell is idle, the probability value of the second grid cell is not updated; wherein, the third probability value is less than the first probability value.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being coupled to the memory, the processor being configured to perform one or more steps of the map building method according to any one of claims 1 to 8 based on instructions stored in the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the steps of the map construction method as described in any one of claims 1 to 8.

Images