A semantic grid map construction method and system with automatic obstacle removal function and a medium

Abstract

This application relates to a semantic grid map construction method, system, and medium with automatic obstacle removal function, belonging to the field of lawnmower technology. The semantic grid map construction method includes: acquiring original point cloud data and original image data of the lawn according to the target projection resolution, and converting them to the same view plane; temporally synchronizing the original point cloud data and original image data to obtain a three-dimensional point cloud and a two-dimensional image, and transmitting them to a data queue; spatially transforming the three-dimensional point cloud, and constructing fused image data by combining object semantic labels with the two-dimensional image in a layered fusion mechanism; generating a lawn grid based on the fused image data, and marking obstacle nodes using a ray tracing mechanism to obtain a lawn grid map; accurately distinguishing obstacle types (static / dynamic, passable / impassable) through category information; and dynamically clearing grids in obstacle-free areas through a ray tracing mechanism, reducing false positive obstacles caused by limited detection field of view.

Description

Technical Field

[0001] This application relates to the field of lawnmower technology, and in particular to a semantic grid map construction method, system and medium with automatic obstacle removal function. Background Technology

[0002] With the continuous advancement of technology, robots are gradually replacing humans in many repetitive tasks, such as sweeping, food delivery, patrolling, and warehousing. Due to their advantages such as low long-term operating costs, ability to work continuously around the clock, high efficiency, and ease of centralized management, robots have been widely used in recent years in areas such as restaurant delivery, cargo handling, and factory inspection, showing promising market prospects.

[0003] For robots, real-time localization and mapping (SLAM) technology is the core foundation for their autonomous operation. In order to complete higher-level tasks such as path planning and real-time obstacle avoidance in dynamic environments, robots need to have the ability to quickly and accurately build maps of their surrounding environment.

[0004] This technology also has significant application value in open, semi-structured environments such as lawns. For example, in areas such as lawn inspection in parks, golf course maintenance, and management of large public green spaces, robots equipped with SLAM systems can autonomously construct maps of lawn areas, enabling tasks such as fixed-point patrols, weed identification, and automatic watering or mowing. Because lawn environments typically feature undulating terrain, vegetation cover, and varying light levels, SLAM systems must address challenges such as sparse feature points and interference from dynamic obstacles; therefore, their robustness and adaptability are crucial for successful technology implementation.

[0005] Existing patents disclose a method and system for constructing a local obstacle grid map based on an octree structure. This method involves obtaining prior information about the local obstacle map from a pre-loaded map, generating an actual local obstacle map based on collected data, and fusing the two to provide accurate and reliable obstacle information around the robot. Compared to previous map construction methods, this invention has advantages in scenarios with high pedestrian traffic and frequently changing environments: ground detection transforms point clouds with a certain slope onto the xOy plane, facilitating height threshold judgment and obstacle point cloud processing; the octree structure dynamically updates the original local point cloud based on the probability of point cloud hits and passes, providing strong adaptability to scenarios with many people and dynamically changing environments; by storing only the local grid map around the robot, memory usage is reduced, ensuring that resource consumption does not increase linearly with runtime; and compared to LiDAR solutions, this invention offers the advantage of low cost.

[0006] The existing technical solutions mentioned above have the following drawbacks: 1. Because point cloud data lacks semantic information, it is impossible to distinguish the categories of different objects. All scanned points can only be regarded as obstacles, which limits the intelligence and flexibility of navigation. 2. The ray tracing logic fails to adequately consider scenes at infinity. When the LiDAR sensor is facing an unobstructed distance, it cannot effectively remove historical obstacle information left in the map, resulting in map data redundancy and distortion. 3. The existing mechanism lacks the ability to automatically clear obstacle information. When positioning drift occurs, incorrectly marked obstacles cannot be corrected in time, which seriously affects the robustness and long-term stability of the system in dynamic environments. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this application is to provide a semantic raster map construction method, system, and medium with automatic obstacle removal functionality. This method more accurately distinguishes obstacle types (static / dynamic, passable / impassable) through category information (such as semantic segmentation results); dynamically removes raster costs in obstacle-free areas through ray tracing mechanisms, reducing false positive obstacles caused by sensor field-of-view limitations; and flexibly adapts to the safety distance requirements of different obstacles by setting different expansion radii and attenuation rates based on object categories through an adaptive expansion strategy.

[0008] This was achieved using the following technical solutions: Firstly, this application provides a semantic raster map construction method with automatic obstacle removal function, including: The original point cloud data and original image data of the lawn are collected according to the preset target projection resolution and converted to the same view plane. The format conversion and timing synchronization of the original point cloud data and original image data are used to obtain 3D point cloud and 2D image data, which are then transmitted to the data queue. Spatial alignment and transformation of 3D point clouds, and layered fusion of object semantic labels and 2D images according to the layered fusion mechanism to construct fused image data; A lawn raster is generated based on the fused image data, and obstacle nodes are marked using a ray tracing mechanism to obtain a lawn raster map.

[0009] By adopting the above technical solutions, based on point cloud registration and image fusion algorithms, and through spatiotemporal synchronization, semantic layering and ray tracing technologies, automated and high-precision modeling and obstacle recognition of the 3D environment of the lawn are achieved, which significantly improves the efficiency of environmental perception and the accuracy of map construction.

[0010] This application is further configured to: acquire raw point cloud data and raw image data of the lawn according to a preset target projection resolution, and convert them to the same viewpoint plane, including: Based on the preset target projection resolution, multi-dimensional data collection of the lawn is performed to obtain raw point cloud data and raw image data. Based on the physical distribution range of the lawn and the sampling field of view, a virtual view plane of the target is constructed; The virtual view plane of the target is divided according to the target projection resolution, and the planar grid sampling points are determined to form a plane with the same view. Perform perspective transformation on the original point cloud data based on the same view plane, calculate the coordinates of the projection intersection points, and quantize them into a point cloud projection map; The distortion is corrected based on the distortion coefficient to obtain a distortion-free image, which is then mapped back to a plane with the same viewpoint to form a planar projection image.

[0011] By adopting the above technical solution, based on perspective transformation and distortion correction algorithms, and through virtual view plane construction and coordinate mapping, the lawn point cloud and image data collected from multiple sources are uniformly converted to the same reference plane, realizing accurate alignment and efficient preprocessing of multimodal data, and significantly improving the accuracy and automation level of subsequent fusion processing.

[0012] This application further includes: format conversion and time-series synchronization of original point cloud data and original image data to obtain 3D point cloud and 2D image, and transmission to a data queue, including: Based on the data type, the original point cloud data and the original image data are filtered and denoised respectively to obtain noise-free point cloud data and noise-free image data. Missing values ​​are filled and duplicate values ​​are removed from noiseless point cloud data and noiseless image data to obtain noiseless point cloud data and noiseless image data. Based on the timestamp and time tolerance, the point cloud data and image data without focus are aligned and normalized to obtain standard point cloud data and standard image data. Based on the data dimension, standard point cloud data and standard image data are converted into formats to obtain three-dimensional point clouds and two-dimensional images; The 3D point cloud and 2D image are encoded and encapsulated to generate standard data packets, which are then transmitted to the data queue.

[0013] By adopting the above technical solution, the original point cloud and image data are cleaned, completed and synchronized based on algorithms such as filtering, interpolation and timestamp alignment. Then, standardized 3D point cloud and 2D image data packets are generated through format conversion and encoding and transmitted to the queue. This realizes automated, highly consistent preprocessing and efficient transmission of multi-source heterogeneous data, and improves the quality of data fusion and system processing efficiency.

[0014] This application further specifies: spatial alignment and transformation of 3D point clouds, and hierarchical fusion of object semantic labels and 2D images according to a hierarchical fusion mechanism to construct fused image data, including: The 3D point cloud is reduced in dimension by projection based on the static transformation matrix, and distortion is removed and normalized by combining distortion parameters to obtain distortion-free projected coordinates. Depth boundary filtering is performed on the distortion-free projection coordinates to obtain the effective projection coordinates, and a projection point coordinate map is constructed. The effective projected coordinates are matched and verified with the coordinates of the projection intersection points, and the matching degree of the projection points is calculated. If the matching degree of the projected points is less than the preset matching degree threshold, the coordinate map of the current projected points is mapped and fused according to the point cloud projection map to obtain the standard projection coordinate map. No, then the current projection point coordinate map is determined to be a standard projection coordinate map; Based on the object category, feature extraction is performed on the standard projected coordinate map to obtain the object shape features, which are then matched with a pre-set semantic feature mapping library to calculate the shape matching degree. If the shape matching degree is greater than the preset shape similarity threshold, then the object semantic label corresponding to the shape feature of the current object is assigned. If not, the object's morphological features are decomposed and combined for matching, and the corresponding combined object semantic tags are assigned. The planar projection map is layered according to the layered fusion mechanism, and combined with the semantic labels of objects to form a data core layer; the data core layer includes a color space layer, a semantic label layer, a depth information layer, a point cloud attribute layer, and a fusion weight layer. The data fusion layer is obtained by directly filling the data core layer and performing neighborhood interpolation based on the standard projected coordinate map. Based on the two-dimensional image, the data fusion layers are correlated and semantically encapsulated to generate fused image data.

[0015] By adopting the above technical solution, spatial alignment and dimensionality reduction of 3D point clouds are achieved based on perspective transformation and point cloud projection algorithms. Semantic labels are assigned to objects through feature matching and semantic mapping algorithms. Then, multi-source information (color, semantics, depth, point cloud attributes) are integrated into a unified data structure using hierarchical fusion and interpolation algorithms to construct fused image data containing rich environmental attributes, which significantly improves the completeness of scene understanding, semantic accuracy and efficiency of subsequent analysis.

[0016] This application further specifies a layered fusion mechanism, including: Create a layer array based on the dimensions of the planar projection and according to layer priority, and assign an independent raster array; The planar projection map is analyzed based on the frequency of data changes to determine the static layer, barrier layer, and expansion layer; Configure parameters for the independent raster arrays based on the layer type to determine the layer raster fusion parameter group; The coordinate system of the real-time acquired raw point cloud data is transformed according to the coordinate transformation matrix to obtain the same system point cloud data. Based on the preset detection range, distance and angle filtering are performed on the same-system point cloud data to obtain effective point cloud data; Cluster the effective point cloud data, calculate the density of neighboring points, and identify outlier noise points to obtain aggregated point cloud data; The aggregated point cloud data is time-series synchronized and aligned based on the timestamp to obtain synchronized point cloud data. Based on the synchronized point cloud data, the layer raster fusion parameter group of the obstacle layer is initially fused to determine the effective obstacle points and calculate the raster index. Based on the spatial boundaries of the planar projection, determine the boundary grid frame and judge the grid index; If the raster index is located within the boundary raster frame, the current raster status is determined to be pending update; The static layer's raster blending parameter group is updated and blended according to the preset update frequency to construct a new layer raster data group; Statistical analysis is performed on the effective point cloud data to calculate the slope and roughness, and the weighted calculation is used to obtain the terrain update cost. Dynamic detection is performed on the planar projection map to generate temporary obstacle areas, which are then merged and updated by combining the new layer raster data group and the terrain update cost to obtain the total layer update area; The layer raster of the expansion layer is traversed and distances are calculated to extract the critical obstacle raster and obtain the obstacle raster distance values; Based on a pre-defined distance cost mapping table, the distance values ​​of obstacle grids are matched, and the grid cost value is calculated: Based on the total layer update area and according to layer priority, the raster cost value is fused pixel by pixel and the cost area is smoothed to obtain the total fused cost map; Spatiotemporal consistency filtering is performed on the total fusion cost map to record the cost of abnormal obstacles and determine the anomaly type by combining the preset cost range. Based on data type and anomaly type, the total fusion cost map is layered in multiple dimensions to determine the core data layer.

[0017] By adopting the above technical solutions, based on frequency domain analysis, clustering algorithms and spatiotemporal synchronization technology, and through dynamic tomography, cost map fusion and anomaly detection, the intelligent integration and updating of multi-layer environmental data is achieved, which significantly improves the completeness, real-time performance and decision accuracy of scene understanding.

[0018] This application further specifies: generating a lawn raster based on the fused image data, and combining it with a ray tracing mechanism to mark obstacle nodes, thereby obtaining a lawn raster map, including: Feature analysis is performed on the fused image data to extract the depth information layer, semantic label layer, and point cloud attribute layer; Binarization and morphological operations are performed on the semantic tag layer based on the binary mask to construct a lawn mask; The lawn mask and the depth information layer are fused to obtain a three-dimensional point set of the lawn; Based on the spatial boundary of the lawn area and the grid resolution, coordinate mapping is performed on the three-dimensional point set of the lawn to construct the lawn grid. The lawn grid is traversed and detected based on the semantic tags of objects to determine the coordinates of obstacles. Rays are projected onto the coordinates of obstacles using ray tracing, the obstacle connection grid is calculated, and obstacle nodes are marked. The basic cost and category expansion radius are determined by combining obstacle nodes with object semantic labels; Based on the distance decay function and the category expansion radius, the cost of the obstacle connection grid is calculated, and the final grid value is determined by combining the base cost value. Based on the data timestamps, the final generation value of the raster is decayed over time to construct a time-series raster map; The temporal raster map is transformed and filled based on map metadata to generate a lawn raster map.

[0019] By adopting the above technical solution, the lawn area is extracted and a grid is constructed based on multi-source data fusion (depth, semantics, point cloud) and image processing algorithms (such as binarization and morphological operations). The obstacle is marked and the dynamic cost is calculated using ray tracing and cost propagation algorithms. Then, a high-precision lawn grid map is generated through a time-series decay model. This achieves automated and refined modeling of complex lawn environments and real-time obstacle perception, significantly improving the map's practicality, accuracy and dynamic adaptability.

[0020] Secondly, this application also provides a semantic raster map construction system with automatic obstacle removal function, which adopts the following technical solution: A semantic raster map construction system with automatic obstacle removal function, and a semantic raster map construction method for implementing it, including: The acquisition and conversion module is used to acquire the original point cloud data and original image data of the lawn according to the preset target projection resolution, and convert them to the same view plane; The synchronization processing module is used for format conversion and time-series synchronization of raw point cloud data and raw image data to obtain 3D point cloud and 2D image, which are then transmitted to the data queue. The transformation and fusion module is used for spatial alignment and transformation of 3D point clouds, and for layered fusion with 2D images according to the layered fusion mechanism to construct fused image data; The map building module is used to generate a lawn raster based on the fused image data, and combine it with a ray tracing mechanism to mark obstacle nodes, thus obtaining a lawn raster map.

[0021] By adopting the above technical solutions, multi-source data unification is achieved based on perspective transformation, temporal alignment and point cloud registration algorithms. Semantic hierarchical fusion and ray tracing technology are used to construct a lawn grid map with dynamic costs, realizing high-precision and automated modeling of the lawn environment and real-time obstacle perception, which significantly improves the accuracy and practicality of environmental perception.

[0022] Thirdly, this application also provides an electronic device, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the methods in the above scheme.

[0023] Fourthly, this application also provides a storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the semantic raster map construction method with automatic obstacle removal function as described above.

[0024] In summary, the beneficial technical effects of this application are as follows: Based on perspective transformation, temporal alignment and point cloud registration algorithms, multi-source data unification is achieved. Semantic hierarchical fusion and ray tracing technology are used to construct a lawn grid map with dynamic cost, realizing high-precision and automated modeling of the lawn environment and real-time obstacle perception, which significantly improves the accuracy and practicality of environmental perception. Based on frequency domain analysis, clustering algorithms, and spatiotemporal synchronization technology, this system achieves intelligent integration and updating of multi-layer environmental data through dynamic tomography, cost map fusion, and anomaly detection, significantly improving the completeness, real-time performance, and decision-making accuracy of scene understanding. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the semantic raster map construction method in this application; Figure 2 This is a flowchart illustrating step S3 in the semantic raster map construction method of this application; Figure 3 This is a schematic diagram of the specific network structure of the layered fusion mechanism in this application; Figure 4 This is a schematic diagram of the semantic raster map construction system in this application. Detailed Implementation

[0026] The present application will be further described in detail below with reference to the accompanying drawings.

[0027] Reference Figure 1 This application discloses a semantic raster map construction method with automatic obstacle removal function, comprising: S1: Collect the original point cloud data and original image data of the lawn according to the preset target projection resolution, and convert them to the same view plane; S2: Format conversion and time-series synchronization of raw point cloud data and raw image data to obtain 3D point cloud and 2D image, and transmit them to the data queue; S3: Spatial alignment and transformation of 3D point clouds, and hierarchical fusion mechanism combining object semantic labels with 2D images to construct fused image data; S4: Generate a lawn raster based on the fused image data, and use ray tracing to mark obstacle nodes to obtain a lawn raster map.

[0028] The implementation principle of this embodiment is as follows: When performing intelligent turf maintenance and management on a large golf course, the system first collects the original point cloud data and original high-resolution image data of the entire golf course turf simultaneously using a laser radar and multispectral camera mounted on a drone and a ground mobile robot, based on a preset target projection resolution of 0.5 cm. Then, it uses a unified GPS-IMU fusion positioning system to convert the two types of data to the same northeast-sky coordinate system plane with the center of the golf course as the origin in real time.

[0029] Subsequently, the system denoises and registers the original point cloud, performs distortion correction and white balance processing on the original image, and achieves millisecond-level temporal synchronization between the two through hardware trigger signals and unified timestamps, forming a 3D point cloud with color information and a geometrically accurate 2D orthophoto, which is then transmitted to a distributed data queue for processing. Next, based on feature point matching and ICP algorithm, the 3D point cloud is further spatially aligned and its coordinates transformed. At the same time, a layered fusion mechanism is adopted—first, semantic labels are added to objects in the 2D image (such as trees, sand pits, and slope change areas) through a deep learning model, and then the image layer with semantic information is fused with the depth and reflection intensity information of the 3D point cloud at the pixel level to construct fused image data that combines geometric accuracy, visual detail, and semantic information.

[0030] Then, based on the fused data, a lawn grid is generated with 5-centimeter units. Each grid contains multi-dimensional attributes such as height, vegetation index, and land surface type. A ray tracing mechanism is introduced to simulate lighting conditions at different times of day, marking dynamic and static obstacle nodes that may cause shadow interference or affect mechanical passage. Finally, a high-precision lawn grid map is output, which includes detailed terrain modeling, obstacle distribution, passable area labeling, and semantic layering. This provides a reliable environmental perception foundation for subsequent fully autonomous maintenance operations such as automatic mowing, precision irrigation, and path planning.

[0031] Preferably, step S1 includes: Based on the preset target projection resolution, multi-dimensional data collection of the lawn is performed to obtain raw point cloud data and raw image data. Based on the physical distribution range of the lawn and the sampling field of view, a virtual view plane of the target is constructed; The virtual view plane of the target is divided according to the target projection resolution, and the planar grid sampling points are determined to form a plane with the same view. Perform perspective transformation on the original point cloud data based on the same view plane, calculate the coordinates of the projection intersection points, and quantize them into a point cloud projection map; The distortion is corrected based on the distortion coefficient to obtain a distortion-free image, which is then mapped back to a plane with the same viewpoint to form a planar projection image.

[0032] In this embodiment, a preset "sensor configuration parameter file" is loaded, which contains: LiDAR parameters: horizontal / vertical field of view (FOV), angular resolution, installation height, and elevation angle.

[0033] Camera parameters: image resolution (e.g., 1920x1080), horizontal / vertical field of view, focal length, distortion coefficient.

[0034] Spatial alignment parameters: The "extrinsic parameter matrix" of the lidar to the camera (containing the rotation matrix R and the translation vector t) describes the precise relative position and orientation relationship between the two.

[0035] Initialize the TF (Coordinate Transformation) buffer listener, load the static coordinate transformation relationship between the robot or carrier chassis, LiDAR mount, and camera mount, and form a complete sensor coordinate system chain.

[0036] Load the "Segmentation and Processing Configuration", such as defining the accessibility flags for the "Lawn" category, map expansion radius, etc.

[0037] Set a uniform "target projection resolution" (e.g., 0.05 m / pixel), which will determine the grid density of the final generated view plane.

[0038] Send a synchronization trigger signal (or hardware synchronization) to the lidar and camera to ensure that both acquire scene data at the same time.

[0039] LiDAR: Captures a frame of 3D point cloud data, where each point contains (x, y, z, intensity) information, and the coordinates are based on the LiDAR's own coordinate system.

[0040] Camera: Captures a frame of a two-dimensional RGB image, with pixel coordinates based on the image coordinate system.

[0041] Add high-precision, synchronized timestamps to point cloud frames and image frames to ensure that subsequent processing is based on the same physical moment.

[0042] Choose a virtual plane with the camera's optical center as the origin and along the camera's optical axis (Z-axis) at a distance d. This plane is perpendicular to the camera's optical axis, simulating the camera's imaging plane, but extending to the measurement range of the lidar; d is usually set according to the effective range of lawn detection (e.g., 10 meters) to ensure that the target lawn area is located in front of this plane.

[0043] A two-dimensional regular grid is generated on the virtual plane based on the camera's horizontal and vertical field of view and the "target projection resolution".

[0044] Calculate the 3D coordinates (X_plane, Y_plane, Z_plane) corresponding to each grid point (u, v), where Z_plane = d (a constant), and X_plane and Y_plane are expanded according to the viewpoint and resolution. This set of grid points is the "plane with the same viewpoint"; Each point in the original LiDAR point cloud is transformed from the LiDAR coordinate system to the camera coordinate system through a static transformation from LiDAR to the camera.

[0045] Project the point cloud, located in the camera coordinate system, onto the virtual plane defined in the previous step. Calculate the coordinates (X_proj, Y_proj) of the intersection point of each point in the point cloud and this plane. Since the plane is perpendicular to the optical axis, this projection is equivalent to "projecting" the 3D points onto the plane along the line connecting them to the camera's optical center.

[0046] The projected (X_proj, Y_proj) coordinates are quantized into indices of the virtual planar mesh, and the original height (z value) or intensity information of the point at that location is preserved or calculated to form a point cloud projection map with height / intensity channels at the same resolution as the virtual planar mesh.

[0047] Using camera intrinsic parameters and distortion coefficients, the original RGB image is distorted to obtain a distortion-free image.

[0048] Planar projection texture mapping: For each pixel (u_img, v_img) on ​​the distorted image, based on the distance d between the camera intrinsics and the virtual plane, calculate the corresponding 3D point on the virtual plane. Then, assign the color (RGB value) of this 3D point to the corresponding position (u_plane, v_plane) in the planar mesh.

[0049] After completing all pixel mapping, a "planar projected RGB image" is obtained that is strictly aligned with the virtual planar mesh. The viewpoint of this image is exactly the same as that of the point cloud projection, eliminating perspective differences.

[0050] Preferably, step S2 includes: Based on the data type, the original point cloud data and the original image data are filtered and denoised respectively to obtain noise-free point cloud data and noise-free image data. Missing values ​​are filled and duplicate values ​​are removed from noiseless point cloud data and noiseless image data to obtain noiseless point cloud data and noiseless image data. Based on the timestamp and time tolerance, the point cloud data and image data without focus are aligned and normalized to obtain standard point cloud data and standard image data. Based on the data dimension, standard point cloud data and standard image data are converted into formats to obtain three-dimensional point clouds and two-dimensional images; The 3D point cloud and 2D image are encoded and encapsulated to generate standard data packets, which are then transmitted to the data queue.

[0051] In this embodiment, firstly, statistical filtering (neighborhood point number threshold ≥ 5) and bilateral filtering are performed on the original point cloud data collected by the UAV LiDAR and the original image data acquired by the multispectral camera, respectively, according to the data type, to obtain noise-free point cloud data and noise-free image data after removing flying noise points and image noise; then, KNN-based missing value interpolation (search radius ≤ 0.1m) and Euclidean distance deduplication (point spacing tolerance ≤ 0.02m) are performed on both to obtain noise-free point cloud data and noise-free image data; then, based on the hardware synchronization timestamp and combined with a preset time tolerance (≤ 50 milliseconds), The two types of data are time-series aligned and numerically normalized (intensity and pixel values ​​are normalized to [0,1]) to form standard point cloud data and standard image data. Then, the standard point cloud data is converted into a 3D point cloud in PCD format and the standard image data is converted into a 2D orthophoto in GeoTIFF format according to the data dimension. Finally, the two are encoded using H.264 and Zlib and encapsulated into a standard data package containing coordinate system, acquisition time, and resolution metadata, which is then transmitted to the subsequent fusion processing node through a message queue to provide high-quality input for the construction of the lawn grid map.

[0052] Preferably, refer to Figure 2 Step S3 includes: The 3D point cloud is reduced in dimension by projection based on the static transformation matrix, and distortion is removed and normalized by combining distortion parameters to obtain distortion-free projected coordinates. Depth boundary filtering is performed on the distortion-free projection coordinates to obtain the effective projection coordinates, and a projection point coordinate map is constructed. The effective projected coordinates are matched and verified with the coordinates of the projection intersection points, and the matching degree of the projection points is calculated. If the matching degree of the projected points is less than the preset matching degree threshold, the coordinate map of the current projected points is mapped and fused according to the point cloud projection map to obtain the standard projection coordinate map. No, then the current projection point coordinate map is determined to be a standard projection coordinate map; Based on the object category, feature extraction is performed on the standard projected coordinate map to obtain the object shape features, which are then matched with a pre-set semantic feature mapping library to calculate the shape matching degree. If the shape matching degree is greater than the preset shape similarity threshold, then the object semantic label corresponding to the shape feature of the current object is assigned. If not, the object's morphological features are decomposed and combined for matching, and the corresponding combined object semantic tags are assigned. The planar projection map is layered according to the layered fusion mechanism, and combined with the semantic labels of objects to form a data core layer; the data core layer includes a color space layer, a semantic label layer, a depth information layer, a point cloud attribute layer, and a fusion weight layer. The data fusion layer is obtained by directly filling the data core layer and performing neighborhood interpolation based on the standard projected coordinate map. Based on the two-dimensional image, the data fusion layers are correlated and semantically encapsulated to generate fused image data.

[0053] In this embodiment, the static transformation matrix from the lidar coordinate system to the camera coordinate system is queried and obtained through a TF listener. This matrix is ​​obtained by pre-completion manual or automatic calibration and includes the rotation matrix R and the translation vector t, denoted as T_lidar_to_cam.

[0054] Load the camera intrinsic parameter matrix K (including focal lengths fx, fy and principal points cx, cy) and distortion coefficients D.

[0055] For each point in the cleaned PCL format 3D point cloud P_lidar={(x_l,y_l,z_l)}, apply an extrinsic parameter matrix to transform it from the lidar coordinate system to the camera coordinate system: P_cam=R·P_lidar+t or equivalently in homogeneous coordinate form: P_cam=T_lidar·P_lidar.

[0056] The coordinates (x_c, y_c, z_c) of each point in the point cloud represent the three-dimensional position of that point in the camera coordinate system, where z_c is the depth (positive value) of the point from the camera optical center.

[0057] Projecting the 3D point P_cam in the camera coordinate system onto the normalized camera plane yields normalized coordinates (x_n, y_n): x_n = x_c / z_c, y_n = y_c / z_c. Using the camera distortion coefficients D, distortion correction is performed on the normalized coordinates (x_n, y_n) to obtain distortion-free normalized coordinates (x_u, y_u).

[0058] Apply the camera intrinsic parameter matrix K to convert the distortion-free normalized coordinates into image pixel coordinates (u,v): [u,v,1]^T=K·[x_u,y_u,1]^T.

[0059] Retain points with depth z_c > 0 (points in front of the camera). Retain points whose pixel coordinates (u,v) fall within the valid image range (e.g., [0,width]x[0,height]). Record the mapping relationship for each valid projection point: point cloud index i ↔ pixel coordinates (u_i,v_i) ↔ depth value z_i.

[0060] Read the synchronized, segmented image converted to OpenCV format. Each pixel location (u,v) in the image stores a semantic category label (e.g., 0-background, 1-road (passable), 2-obstacle (impassable), 3-lawn, etc.).

[0061] Iterate through all valid projection points. For a point i in the point cloud projected onto pixel (u_i, v_i), query the category label label_i of the segmented image at that pixel location.

[0062] According to the preset semantic-attribute mapping rules, assign an attribute label to the point cloud point. For example: label_i=1 (road) → the point attribute is marked as passable.

[0063] label_i=2(obstacle) → The point attribute is marked as impassable.

[0064] label_i=3 (lawn) → The point attribute is marked as conditional access.

[0065] Add the tagged attribute information (such as an integer code) back into the PCL point cloud data structure as a new field (such as semantic_label).

[0066] Create a multi-layered data structure with the same resolution as the original 2D image (e.g., a multi-channel image or a data structure containing multiple 2D grids). The core layer includes: Layer 1: Raw RGB layer: Color information of the raw camera image (optional, for visualization).

[0067] Layer 2: Semantic Label Layer: Pixel-level semantic categories from the segmented image.

[0068] Layer 3: Depth information layer: Stores the depth value z_c after projection and interpolation from the point cloud.

[0069] Layer 4: Point Cloud Attribute Layer: Stores semantically labeled attribute information (such as "accessibility") transferred from the point cloud. Pixels without point cloud projection can be labeled as "unknown".

[0070] Layer 5: Confidence / Fusion Weight Layer (Optional): Based on point cloud density, projection distance, segmentation model confidence, etc., it provides a reliability weight for the fusion information of each pixel.

[0071] For pixels (u_i, v_i) with accurate point cloud projection, directly fill the corresponding depth value z_i and point cloud attributes into the corresponding positions in Layer2 and Layer3.

[0072] For pixels without direct point cloud projection, but whose surrounding effective projection points form a local region, nearest neighbor or bilinear interpolation methods can be used to infer their depth and attributes from nearby effective projection points and fill Layer 2 and Layer 3. This helps generate denser, continuous fusion information.

[0073] Ensure that Layer1 (semantics), Layer2 (depth), and Layer3 (attributes) are strictly aligned in space. For example, the depth information (Layer2) and accessibility (Layer3) of a pixel should both correspond to the same physical point of that pixel in the image.

[0074] All the above layers (Layer 1-5) are packaged into a unified, multi-dimensional fused image data object. This object not only contains appearance (RGB) and semantic information, but more importantly, it integrates precise geometric (depth) information and derived spatial attributes (accessibility) from the 3D point cloud.

[0075] This data structure is also associated with the original point cloud (with semantic attributes already labeled) and coordinate transformation parameters, allowing subsequent modules to flexibly call information at any level.

[0076] Preferably, refer to Figure 3 The layered integration mechanism includes: Create a layer array based on the dimensions of the planar projection and according to layer priority, and assign an independent raster array; The planar projection map is analyzed based on the frequency of data changes to determine the static layer, barrier layer, and expansion layer; Configure parameters for the independent raster arrays based on the layer type to determine the layer raster fusion parameter group; The coordinate system of the real-time acquired raw point cloud data is transformed according to the coordinate transformation matrix to obtain the same system point cloud data. Based on the preset detection range, distance and angle filtering are performed on the same-system point cloud data to obtain effective point cloud data; Cluster the effective point cloud data, calculate the density of neighboring points, and identify outlier noise points to obtain aggregated point cloud data; The aggregated point cloud data is time-series synchronized and aligned based on the timestamp to obtain synchronized point cloud data. Based on the synchronized point cloud data, the layer raster fusion parameter group of the obstacle layer is initially fused to determine the effective obstacle points and calculate the raster index. Based on the spatial boundaries of the planar projection, determine the boundary grid frame and judge the grid index; If the raster index is located within the boundary raster frame, the current raster status is determined to be pending update; The static layer's raster blending parameter group is updated and blended according to the preset update frequency to construct a new layer raster data group; Statistical analysis is performed on the effective point cloud data to calculate the slope and roughness, and the weighted calculation is used to obtain the terrain update cost. Dynamic detection is performed on the planar projection map to generate temporary obstacle areas, which are then merged and updated by combining the new layer raster data group and the terrain update cost to obtain the total layer update area; The layer raster of the expansion layer is traversed and distances are calculated to extract the critical obstacle raster and obtain the obstacle raster distance values; Based on a pre-defined distance cost mapping table, the distance values ​​of obstacle grids are matched, and the grid cost value is calculated: Based on the total layer update area and according to layer priority, the raster cost value is fused pixel by pixel and the cost area is smoothed to obtain the total fused cost map; Spatiotemporal consistency filtering is performed on the total fusion cost map to record the cost of abnormal obstacles and determine the anomaly type by combining the preset cost range. Based on data type and anomaly type, the total fusion cost map is layered in multiple dimensions to determine the core data layer.

[0077] In this embodiment, a LayerManager is initialized to uniformly manage all layers; a layer array is created and stored in priority order: static layer (lowest) → obstacle layer → terrain layer (optional) → dilation layer (highest); each layer is assigned an independent two-dimensional raster array (the same size as the main map) as its private workspace. Static layer: Loads pre-built static map files (such as PGM), parses them into raster data (0-255 values), and marks static obstacles; Obstacle layer: Configure the list of associated sensors (such as LiDAR ID, depth camera ID), data update frequency, and effective detection range; Inflation Layer: Configure inflation parameters: robot profile radius (or safety distance), maximum inflation distance (e.g., 2.0 meters), cost decay function (linear / exponential / quadratic), and whether to use a robot footprint model (for non-circular robots). Create two raster arrays of the same size as a double buffer: ActiveBuffer: Used for layer blending calculations in the current cycle; PublishedBuffer: Stores the cost map that has been calculated in the previous cycle for the path planning module to read; sets up a grid locking mechanism to prevent read / write conflicts.

[0078] Obtain the transformation chain from sensor to map coordinate system from the TF tree: sensor coordinate system → robot base coordinate system → map coordinate system; apply the transformation matrix to each sensor data point to unify it to the map coordinate system; Distance filtering: Removes points outside the effective range (e.g., 0.1-30 meters); Angle filtering: Removes points outside the sensor's field of view; Statistical filtering: Removes outlier noise points based on neighbor density. A high-precision timestamp is appended to each data point. For multi-sensor data, the data from other sensors is aligned by interpolation, using the time of the primary sensor (such as LiDAR) as the reference. For each valid obstacle point P (map coordinates x, y): calculate the grid index (grid_x, grid_y) = ((x - origin_x) / resolution, (y - origin_y) / resolution)); If the index is within the map boundary: set the obstacle layer raster value to 255 (LETHAL_OBSTACLE), and record the raster to the "Current Period Update List"; After the initial load, it is usually not updated (unless relocation or map change). Configurable low-frequency updates (e.g., 1Hz) for creating new static maps that are integrated with SLAM systems; Terrain layer: Slope and roughness are calculated based on point computing and converted into cost value (0-254); Social layer: Temporary obstacle areas are generated around people based on personnel detection; Each layer records the updated grid area bounding box in this cycle; For the obstacle layer, the updated area sector / rectangle is calculated based on the sensor field of view angle; The updated areas of all layers are merged to obtain the total area that needs to be recalculated in this cycle; The fatal obstacle grids (grids with a value of 255) of the obstacle layer and static layer are merged as the source of expansion, using a fast distance transformation algorithm (such as the Felzenszwalb algorithm): First scan (from left to right, from top to bottom): For each grid cell (i,j): If it is not an obstacle: dist[i][j] = min(previous distance + 1, previous row distance + 1, diagonal distance + √2 (if using Euclidean distance)); Second scan (from right to left, from bottom to top): Similar update, ensuring the global optimum of the distance value.

[0079] Based on the distance value d, calculate the cost (cost(d)) for each grid cell: if d <= robot_radius: cost = 255 / / Areas the robot cannot enter; otherwise, if d <= inflation_radius: / / Cost-decreasing region Linear decay: cost = 255 * (inflation_radius - d) / (inflation_radius - robot_radius); Exponential decay: cost = 255 * exp(-λ * (d - robot_radius)); Secondary attenuation: cost = 255 * (1 - (d - robot_radius) / (inflation_radius - robot_radius))^2; Otherwise, cost = 0 / / safe zone.

[0080] If a non-circular robot profile is used, the robot's orientation and shape need to be considered during the calculation; For each grid cell, calculate the distance on the contour of the robot at that position and orientation to the nearest point on the obstacle; use this minimum distance to calculate the cost, ensuring that the robot can safely pass through the entire contour.

[0081] For each raster within the area that needs updating, merge them according to layer priority: Initialize the fusion cost to 0; and process them from low to high priority: fusion_cost = max(fusion_cost, static layer cost); fusion_cost = max(fusion_cost, barrier layer cost); fusion_cost = max(fusion_cost, terrain layer cost); fusion_cost = max(fusion_cost, social layer cost); fusion_cost = max(fusion_cost, cost of expansion layer); Write fusion_cost to the corresponding raster in the working buffer.

[0082] Static barriers (static layers) retain their original cost value under all circumstances; the fatal barrier (255) of dynamic barriers (barrier layers) takes precedence over any expansion cost; the expansion cost does not affect the static / dynamic barrier itself, but only its surrounding area; For transient sensor noise (such as flying insects or drifting objects), a time decay mechanism is used; Only obstacles that are detected in multiple consecutive frames (e.g., 3 frames) are confirmed as real obstacles; Check whether the locations of obstacles in each layer are reasonable (e.g., obstacles should not appear inside walls); for conflict situations (e.g., obstacles in static layers are not detected by sensors), record them as potential map errors. Limit all cost values ​​to the range [0, 255]; set values ​​below the minimum cost threshold (e.g., 10) to 0 (complete freedom); set values ​​above the maximum cost threshold (e.g., 245) to 255 (complete barrier). Gaussian filtering or median filtering is applied to the fused cost map to eliminate raster jagged edges; the filtering is not applied to the fatal obstacle (255) itself, but only to the intermediate cost region.

[0083] Preferably, step S4 includes: Feature analysis is performed on the fused image data to extract the depth information layer, semantic label layer, and point cloud attribute layer; Binarization and morphological operations are performed on the semantic tag layer based on the binary mask to construct a lawn mask; The lawn mask and the depth information layer are fused to obtain a three-dimensional point set of the lawn; Based on the spatial boundary of the lawn area and the grid resolution, coordinate mapping is performed on the three-dimensional point set of the lawn to construct the lawn grid. The lawn grid is traversed and detected based on the semantic tags of objects to determine the coordinates of obstacles. Rays are projected onto the coordinates of obstacles using ray tracing, the obstacle connection grid is calculated, and obstacle nodes are marked. The basic cost and category expansion radius are determined by combining obstacle nodes with object semantic labels; Based on the distance decay function and the category expansion radius, the cost of the obstacle connection grid is calculated, and the final grid value is determined by combining the base cost value. Based on the data timestamps, the final generation value of the raster is decayed over time to construct a time-series raster map; The temporal raster map is transformed and filled based on map metadata to generate a lawn raster map.

[0084] In this embodiment, the fused image data is first analyzed to extract a depth information layer, a semantic label layer containing categories such as "lawn", "trees", and "sandpits", and a point cloud reflection intensity attribute layer. A binary mask is generated based on the semantic label "lawn" and optimized by morphological closing operation (kernel size 5×5) to construct a complete lawn mask. The mask is then fused with the depth information layer to filter out the three-dimensional point set belonging to the lawn area.

[0085] Next, based on the spatial boundaries of the surveyed court and a grid resolution of 5 cm, the coordinates of the point set are mapped to construct an initial lawn grid. The system traverses the grid and determines the coordinates of obstacles based on non-lawn semantic labels (such as "trees" and "sprinklers"). Subsequently, a ray tracing mechanism is activated to project rays from the sensor's perspective to the obstacle coordinates (256 samples), calculate the occluded connecting grids, and mark them as obstacle nodes. Based on the node's semantic category, a base cost value (255 for trees and 200 for sprinklers) and a category expansion radius (3 grids for trees and 1 grid for sprinklers) are set. A linear distance decay function is used to calculate the cost of surrounding grids centered on the obstacle node according to the expansion radius. The base value is then added to determine the final cost value of each grid.

[0086] Finally, a time decay factor (half-life of 30 minutes) is introduced to update the value of the data based on the timestamp, constructing a time-series raster map that reflects the timeliness of obstacles, and filling in metadata such as coordinate system and resolution to generate a high-precision lawn raster map for autonomous navigation.

[0087] Reference Figure 4 A semantic raster map construction system with automatic obstacle removal function, applied to a semantic raster map construction method, including: The acquisition and conversion module is used to acquire the original point cloud data and original image data of the lawn according to the preset target projection resolution, and convert them to the same view plane; In this embodiment, a virtual camera view plane is first calculated based on a preset target projection resolution (e.g., 0.05 m / pixel) and sensor intrinsic and extrinsic parameters. Then, the LiDAR and camera are simultaneously triggered to acquire raw point cloud data and raw image data. Through a coordinate transformation chain, the LiDAR point cloud is converted to the camera coordinate system, and the camera model is used to project both onto the aforementioned virtual plane. This process generates a strictly spatially aligned 2.5D point cloud projection map (including height information) and a planar projected RGB image, fundamentally eliminating viewpoint deviations caused by differences in sensor pose and achieving preliminary geometric fusion of heterogeneous data.

[0088] The synchronization processing module is used for format conversion and time-series synchronization of raw point cloud data and raw image data to obtain 3D point cloud and 2D image, which are then transmitted to the data queue. In this embodiment, the pre-aligned point cloud and image streams are subscribed to. A near-time synchronization strategy is used to pair data pairs whose timestamps are within a tolerance window, ensuring that scene snapshots from the same moment are being processed. Subsequently, the raw data is cleaned and filtered to remove invalid points and noise, and the data is converted to a standard internal format: point clouds are converted to PCL format, and images are converted to OpenCV format. The processed 3D point cloud and 2D image data pairs are packaged into a structure with unified timestamps, coordinate systems, and other metadata, and sent to a thread-safe data queue, providing a stable and ordered data source for asynchronous processing.

[0089] The transformation and fusion module is used for spatial alignment and transformation of 3D point clouds, and for layered fusion with 2D images according to the layered fusion mechanism to construct fused image data; In this embodiment, fine spatial alignment is first performed: the 3D point cloud is precisely transformed to the camera coordinate system using a calibrated extrinsic parameter matrix, and then each 3D point is projected onto the pixel coordinates of the 2D image using camera intrinsic parameters and a distortion correction model. Next, based on a layered fusion mechanism, a multi-layered fused image data structure is constructed: the bottom layer is the original image layer, on which a semantic label layer (from image segmentation), a depth information layer (from the projected point cloud), and a point cloud attribute layer (such as accessibility based on semantic tags) are superimposed. Through pixel-by-pixel association, this module outputs a fused image data object integrating appearance, semantics, precise 3D geometry, and attributes, providing rich contextual information for environmental understanding.

[0090] The map building module is used to generate a lawn raster based on the fused image data, and combine it with a ray tracing mechanism to mark obstacle nodes, thus obtaining a lawn raster map.

[0091] In this embodiment, lawn area and obstacle information are first extracted from the fused data to initialize a lawn grid map. A key step is the application of ray tracing: rays are emitted from the sensor location towards each detected obstacle point, marking the grid along the ray path as passable areas. This clears the cost of the blank areas behind obstacles, restoring the true passable space. Subsequently, based on the semantic category of the obstacle (e.g., static objects, people, animals), a multi-level inflation strategy is used to set different safety radii, and obstacles and their inflation costs are calculated and marked. The module runs in an independent thread, supports time decay to clear expired dynamic obstacles, and continuously publishes the final cost map containing obstacles, inflation areas, and passable lawns, along with its visualization information, completing the closed-loop construction from multimodal perception to a navigation-usable environment model. Example

[0092] In the automated maintenance and intelligent management of a large golf course, a swarm of drones equipped with high-precision lidar and multispectral cameras is driven by a data acquisition and conversion module to scan the entire lawn at a preset target projection resolution of 0.8 cm to obtain raw point cloud data and raw image data. Then, with the help of a GNSS-RTK positioning system, all data are uniformly converted to the same northeast celestial coordinate view plane with the main control station of the golf course as the origin.

[0093] Subsequently, the synchronization processing module removes outliers and registers the original point cloud, performs radiometric and geometric correction on the original image, achieves millisecond-level temporal alignment through hardware synchronization signals, and converts it into a PCD format 3D point cloud with RGB information and a GeoTIFF format 2D orthophoto, which is then pushed to the Kafka data queue. Next, the transform fusion module performs spatial alignment and coordinate transformation on the 3D point cloud using the ICP algorithm, and based on a hierarchical fusion mechanism, performs pixel-level fusion of the depth layer and reflection intensity layer of the point cloud with the semantic segmentation layer of the 2D image (which has been identified by a deep learning model as categories such as lawn, green, sandpit, tree, sprinkler head, etc.), constructing fused image data that combines geometric, spectral, and semantic information.

[0094] Ultimately, the map building module generates a 5-centimeter resolution lawn grid based on the fused data. Each grid is assigned attributes such as height, humidity index, and vegetation type. At the same time, it combines a ray tracing mechanism to simulate the light and shadow at different times of day, accurately marking dynamic and static obstacle nodes, and outputting a high-precision lawn grid map containing passable areas, obstacle distribution, semantic partitioning, and multi-dimensional physical property information. This provides a complete dynamic environmental perception and operation planning foundation for unmanned lawnmowers, precision irrigation systems, and inspection robots.

[0095] An electronic device, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the methods in the above scheme.

[0096] A storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the semantic raster map construction method as described above.

[0097] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for constructing a semantic raster map with automatic obstacle removal function, characterized in that, include: The original point cloud data and original image data of the lawn are collected according to the preset target projection resolution and converted to the same view plane. The original point cloud data and the original image data are converted in format and synchronized in time to obtain a three-dimensional point cloud and a two-dimensional image, which are then transmitted to the data queue. The three-dimensional point cloud is spatially aligned and transformed, and then fused with the two-dimensional image according to the hierarchical fusion mechanism by combining the semantic tags of the object; thus constructing fused image data. A lawn grid is generated based on the fused image data, and obstacle nodes are marked using a ray tracing mechanism to obtain a lawn grid map.

2. The semantic raster map construction method with automatic obstacle removal function according to claim 1, characterized in that, The process of acquiring raw point cloud data and raw image data of the lawn according to a preset target projection resolution and converting them to the same viewpoint plane includes: Based on the preset target projection resolution, multi-dimensional data collection of the lawn is performed to obtain raw point cloud data and raw image data. Based on the physical distribution range of the lawn and the sampling field of view, a virtual view plane of the target is constructed; The target virtual view plane is divided according to the target projection resolution, and planar grid sampling points are determined to form planes with the same view. Based on the same view plane, perform perspective transformation on the original point cloud data, calculate the coordinates of the projection intersection points, and quantize them into a point cloud projection map; The original image data is distorted according to the distortion coefficient to obtain a distortion-free image, which is then mapped inversely to the plane with the same viewpoint to form a planar projection image.

3. The semantic raster map construction method with automatic obstacle removal function according to claim 1, characterized in that, The format conversion and timing synchronization of the original point cloud data and the original image data to obtain a 3D point cloud and a 2D image are then transmitted to the data queue, including: Based on the data type, the original point cloud data and the original image data are filtered and denoised respectively to obtain noise-free point cloud data and noise-free image data. The noise-free point cloud data and the noise-free image data are filled with missing values ​​and removed with duplicate values ​​to obtain noise-free point cloud data and noise-free image data. Based on the timestamp and time tolerance, the point cloud data and the image data without overlap are aligned and normalized to obtain standard point cloud data and standard image data. The standard point cloud data and the standard image data are converted according to the data dimension to obtain a three-dimensional point cloud and a two-dimensional image. The 3D point cloud and the 2D image are encoded and encapsulated to generate a standard data packet, which is then transmitted to the data queue.

4. The semantic raster map construction method with automatic obstacle removal function according to claim 1, characterized in that, The spatial alignment and transformation of the 3D point cloud, combined with the hierarchical fusion mechanism and the object semantic labels, are used to construct fused image data, including: The 3D point cloud is reduced in dimension by projection based on the static transformation matrix, and distortion is removed and normalized by combining distortion parameters to obtain distortion-free projected coordinates. The distortion-free projection coordinates are filtered by depth boundaries to obtain valid projection coordinates, and a projection point coordinate map is constructed. The effective projection coordinates and the coordinates of the projection intersection point are matched and verified, and the matching degree of the projection points is calculated. If the matching degree of the projection point is less than the preset matching degree threshold, then the current projection point coordinate map is mapped and fused according to the point cloud projection map to obtain the standard projection coordinate map; No, then the current projection point coordinate map is determined to be a standard projection coordinate map; Based on the object category, feature extraction is performed on the standard projected coordinate map to obtain the object shape features, which are then matched with a preset semantic feature mapping library to calculate the shape matching degree. If the shape matching degree is greater than the preset shape similarity threshold, then the object semantic label corresponding to the shape feature of the current object is assigned. If not, the object's morphological features are decomposed and combined for matching, and the corresponding combined object semantic tags are assigned. The planar projection map is layered according to the layered fusion mechanism, and combined with the semantic tags of the objects to form a data core layer; the data core layer includes a color space layer, a semantic tag layer, a depth information layer, a point cloud attribute layer, and a fusion weight layer. The data core layer is directly filled and its neighborhood interpolated according to the standard projection coordinate map to obtain the data fusion layer. Based on the two-dimensional image, the data fusion layers are correlated and semantically encapsulated to generate fused image data.

5. The semantic raster map construction method with automatic obstacle removal function according to claim 1, characterized in that, The layered fusion mechanism includes: Create a layer array based on the dimensions of the planar projection and according to layer priority, and assign an independent raster array; The planar projection map is analyzed based on the frequency of data changes to determine the static layer, barrier layer, and expansion layer. Configure the parameters of the independent raster array according to the layer type to determine the layer raster fusion parameter group; The coordinate system of the real-time acquired raw point cloud data is transformed according to the coordinate transformation matrix to obtain the same system point cloud data. The homologous point cloud data is filtered by distance and angle according to the preset detection range to obtain effective point cloud data; Cluster the effective point cloud data, calculate the density of neighboring points, and identify outlier noise points to obtain aggregated point cloud data; The aggregated point cloud data is time-series synchronized and aligned based on the timestamp to obtain synchronized point cloud data.

6. The semantic raster map construction method with automatic obstacle removal function according to claim 1 or 5, characterized in that, The layered fusion mechanism also includes: Based on the synchronized point cloud data, the layer raster fusion parameter group of the obstacle layer is initially fused to determine the effective obstacle points and calculate the raster index. Based on the spatial boundaries of the planar projection, determine the boundary grid frame and judge the grid index; If the raster index is located within the boundary raster frame, then the current raster state is determined to be pending update; The static layer's raster blending parameter group is updated and blended according to the preset update frequency to construct a new layer raster data group; Statistical analysis is performed on the effective point cloud data to calculate the slope and roughness, and the weighted calculation is used to obtain the terrain update cost. The planar projection map is dynamically detected to generate a temporary obstacle zone, which is then merged and updated with the new layer raster data group and the terrain update cost to obtain the total layer update area. The layer raster of the expansion layer is traversed and distances are calculated to extract the critical obstacle raster and obtain the obstacle raster distance values; Based on a preset distance cost mapping table, the distance values ​​of the obstacle grid are matched, and the grid cost value is calculated: Based on the total layer update area and according to layer priority, the raster cost value is fused pixel by pixel and the cost area is smoothed to obtain the total fused cost map; Spatiotemporal consistency filtering is applied to the total fusion cost map to record the cost value of abnormal obstacles and determine the anomaly type by combining it with a preset cost value range. Based on the data type and the anomaly type, the total fusion cost map is layered in multiple dimensions to determine the core data layer.

7. The semantic raster map construction method with automatic obstacle removal function according to claim 1, characterized in that, The process of generating a lawn raster based on the fused image data, and using a ray tracing mechanism to mark obstacle nodes to obtain a lawn raster map, includes: Feature analysis is performed on the fused image data to extract the depth information layer, semantic label layer, and point cloud attribute layer; Binarization and morphological operations are performed on the semantic tag layer based on the binary mask to construct a lawn mask; The lawn mask and the depth information layer are fused to obtain a three-dimensional point set of the lawn; Based on the spatial boundary of the lawn area and the grid resolution, coordinate mapping is performed on the three-dimensional point set of the lawn to construct a lawn grid. The lawn grid is traversed and detected based on the semantic tags of objects to determine the coordinates of obstacles. Rays are projected onto the coordinates of the obstacles using a ray tracing mechanism to calculate the obstacle connection grid and mark the obstacle nodes; Based on the obstacle nodes and the object semantic tags, the basic cost and category expansion radius are determined; Based on the distance decay function and the category expansion radius, the cost of the obstacle connection grid is calculated, and combined with the base cost value, the final cost value of the grid is determined. Based on the data timestamps, the final generation value of the raster is decayed over time to construct a time-series raster map; The temporal raster map is transformed and filled based on map metadata to generate a lawn raster map.

8. A semantic raster map construction system with automatic obstacle removal function, used to implement the semantic raster map construction method as described in any one of claims 1-7, characterized in that, include: The acquisition and conversion module is used to acquire the original point cloud data and original image data of the lawn according to the preset target projection resolution, and convert them to the same view plane; The synchronization processing module is used for format conversion and time synchronization of the original point cloud data and the original image data to obtain three-dimensional point cloud and two-dimensional image, which are then transmitted to the data queue. The transformation and fusion module is used for spatial alignment and transformation of the three-dimensional point cloud, and for layer-by-layer fusion with the two-dimensional image according to the layer-by-layer fusion mechanism to construct fused image data; The map building module is used to generate a lawn raster based on the fused image data, and to mark obstacle nodes using a ray tracing mechanism to obtain a lawn raster map.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the semantic raster map construction method as described in any one of claims 1 to 7.

Images