Map dynamic construction method and system fusing laser vision

Abstract

The application relates to the technical field of robot navigation, and discloses a map dynamic construction method and system fusing laser vision, which comprises a data sensing module, a feature analysis module, an information fusion module and a dynamic optimization module. The data sensing module collects an environment point cloud sequence and a scene image sequence through a laser ranging device and a vision imaging device; the feature analysis module extracts spatial structure features of the point cloud and visual semantic features of the image; the information fusion module fuses the multi-modal features to generate an initial map by adopting a weight distribution strategy; and the dynamic optimization module adjusts the map position and attribute parameters in real time by using incremental data. Through multi-modal fusion and dynamic optimization, the application solves the problems of insufficient map precision, semantic loss and low dynamic updating efficiency of a traditional single sensor, and is suitable for robot navigation, automatic driving and other scenes.

Description

Technical Field

[0001] The present application relates to the field of robot navigation technology, and specifically to a method and system for dynamically constructing a map by integrating laser vision. Background Art

[0002] Building and updating environmental maps is a core technology in fields such as robotic navigation, autonomous driving, and augmented reality. Traditional map construction methods rely primarily on data from a single sensor, such as lidar or cameras. Lidar provides highly accurate spatial structure information but lacks texture and semantic information. Cameras capture rich visual semantic information but are sensitive to lighting conditions and have limited depth perception. The limitations of this single data source make it difficult to achieve both accuracy and semantic integrity in complex and dynamic environments.

[0003] In existing technologies, multi-sensor fusion methods have been proposed to compensate for the shortcomings of single sensors. For example, LiDAR-based point cloud mapping technology uses SLAM (Simultaneous Localization and Mapping) algorithms to achieve environmental modeling, but the resulting geometric maps lack semantic labels, making it difficult to support high-level decision-making. On the other hand, purely visual SLAM systems (such as ORB-SLAM) can extract scene feature points and construct semantic maps, but they are prone to failure in low-light or texture-deficient areas. Although some studies have attempted to combine laser and visual data, the fusion of multimodal information often remains at the level of simple superposition, failing to fully consider data credibility, temporal consistency, and dynamic optimization issues.

[0004] Furthermore, updating maps in dynamic environments presents another significant challenge. Traditional approaches typically employ either global reconstruction or local patching strategies. The former is computationally expensive, while the latter is prone to cumulative errors. For example, incremental update methods based on filtering (such as particle filtering) struggle to handle large-scale scenarios, while end-to-end map prediction methods based on deep learning rely on large amounts of annotated data and have limited generalization capabilities. Therefore, a map construction system that can efficiently integrate laser and visual data and support dynamic optimization is needed to improve map accuracy, real-time performance, and semantic richness. Summary of the Invention

[0005] The purpose of this application is to provide a method and system for dynamically constructing maps that integrates laser vision to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above-mentioned objectives, the present application provides the following technical solutions: a method and system for dynamically constructing a map integrating laser vision, the system comprising: a data perception module: collecting an environmental point cloud sequence through a laser ranging device, and obtaining a scene image sequence using a visual imaging device; the point cloud sequence contains spatial coordinate information, and the image sequence contains texture and color information; a feature parsing module: extracting spatial structural features based on the point cloud sequence, and extracting visual semantic features based on the image sequence; the spatial structural features are calculated by a neighborhood point statistics method, and the visual semantic features are obtained by edge detection and region segmentation technology; an information fusion module: multimodal information fusion of spatial structural features and visual semantic features to generate an initial fused map; multimodal information fusion adopts a weight distribution strategy, and sets fusion weights according to data credibility; a dynamic optimization module: dynamically optimizes the initial fused map using incremental data collected in real time, and adjusts the position and attribute parameters of each area in the map; the incremental data includes newly collected point cloud sequences and image sequences.

[0007] Preferably, the specific method of collecting the environmental point cloud sequence by the laser ranging device and obtaining the scene image sequence by the visual imaging device includes: the laser ranging device emits a laser signal at a fixed frequency, receives the reflected echo and converts it into point cloud coordinate data; the visual imaging device shoots the scene image at a synchronous frequency, and ensures the time consistency of the point cloud sequence and the image sequence by timestamp alignment; the point cloud coordinate data is mapped to the global coordinate system by the coordinate conversion unit, and the image sequence eliminates lens distortion by the distortion correction unit.

[0008] Preferably, the specific method of extracting spatial structure features based on point cloud sequence includes: step 1, selecting the target area in the point cloud sequence, and calculating the number and distribution density of neighborhood points of each point in the target area; step 2, calculating the normal vector direction of each point through the coordinate difference of the neighborhood points, and the normal vector direction is determined by the principal component analysis result of the neighborhood points; step 3, based on the number of neighborhood points, distribution density and normal vector direction information, constructing a multidimensional feature vector including density features, direction features and shape features as the spatial structure feature.

[0009] Preferably, the specific method of extracting visual semantic features based on image sequences includes: visual semantic features include edge features and regional features; detecting image edges through a gradient operator to obtain the coordinates and gradient amplitudes of edge points; performing regional division based on the connectivity of edge points, and calculating the area, aspect ratio and texture complexity of each region; integrating the coordinates of edge points, gradient amplitude, regional area, aspect ratio and texture complexity into a matrix form as visual semantic features.

[0010] Preferably, the specific method of performing multimodal information fusion of spatial structure features and visual semantic features to generate an initial fused map includes: establishing a three-dimensional spatial grid model based on density features, direction features and shape features in the spatial structure features; establishing a two-dimensional image semantic model based on edge features and area features in the visual semantic features; aligning the three-dimensional spatial grid model with the two-dimensional image semantic model through coordinate mapping, filling the semantic information into the spatial grid, and forming an initial fused map containing spatial structure and semantic attributes.

[0011] Preferably, the method of dynamically optimizing the initial fused map using the incremental data collected in real time and adjusting the position and attribute parameters of each area in the map includes: input layer, taking the point cloud features and image features in the incremental data as input; setting the input dimension to be consistent with the number of features; hidden layer, setting multiple layers of nonlinear transformation layers; passing the input data from the input layer to the first transformation layer, obtaining the first layer output through linear combination and activation function processing, passing the first layer output to the next transformation layer, repeating the nonlinear processing until the last transformation layer ends to obtain the final transformation output; output layer, passing the final transformation output to the output layer, generating a map optimization amount through linear projection; the optimization amount includes a spatial position offset and an attribute update value; determining the influence weight of the optimization amount based on the acquisition time and credibility of the incremental data; applying the optimization operation, multiplying the optimization amount by the influence weight and superimposing it on the initial fused map to complete the dynamic optimization.

[0012] Preferably, the specific method of determining the influence weight of the optimization amount based on the collection time and credibility of the incremental data includes: calculating the time weight through the time decay function for the collection time of the incremental data; the decay rate of the time decay function is set according to the frequency of dynamic changes of the map; calculating the credibility weight through data quality assessment indicators for the credibility of the incremental data; the data quality assessment indicators include point cloud density, image clarity and feature matching; combining the time weight and the credibility weight, calculating the influence weight of the optimization amount through a weighted sum formula.

[0013] Preferably, when the optimization amount includes a spatial position offset, the position of each area in the map is adjusted, specifically including: when the optimization type is position offset, using the real-time positioning data of the laser ranging device to obtain the global coordinate deviation of the current map; calculating the regional displacement amount by the global coordinate deviation and the spatial position offset; and using a coordinate transformation algorithm to adjust the coordinate values ​​of each area in the map for the regional displacement amount, and moving the area to the optimized position.

[0014] Preferably, when the optimization amount includes an attribute update value, the attributes of each area in the map are adjusted, specifically including: using image semantic segmentation technology to identify new attribute information in the incremental data; the new attribute information includes area categories and texture features; based on the new attribute information and the attribute update value, the category label and texture parameters of the corresponding area in the map are updated through the attribute replacement rule to complete the area attribute adjustment.

[0015] Preferably, the present application also includes a method for dynamically constructing a map by integrating laser vision, which is applied to the above-mentioned dynamic map construction system by integrating laser vision, and the method includes: S1, collecting an environmental point cloud sequence through a laser ranging device, and obtaining a scene image sequence using a visual imaging device; extracting spatial structural features based on the point cloud sequence, and extracting visual semantic features based on the image sequence; wherein the point cloud sequence contains spatial coordinate information, and the image sequence contains texture color information; S2, performing multimodal information fusion of spatial structural features and visual semantic features to generate an initial fused map; multimodal information fusion adopts a weight distribution strategy, and sets the fusion weight according to the data credibility; S3, dynamically optimizing the initial fused map using incremental data collected in real time, and adjusting the position and attribute parameters of each area in the map; the incremental data includes newly collected point cloud sequences and image sequences; during the dynamic optimization process, determining the influence weight of the optimization amount based on the acquisition time and credibility of the incremental data, and handling the adjustment of spatial position offset and attribute update respectively.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] The dynamic map construction system and method integrated with laser vision proposed in this application significantly improves the accuracy, semantic richness, and real-time performance of maps through multimodal data fusion and dynamic optimization mechanisms. During the data perception stage, the synchronous acquisition of laser and visual data ensures temporal consistency, and the complementarity of point cloud coordinates and image texture provides a complete information foundation for subsequent feature extraction. The feature parsing module uses neighborhood point statistics and edge detection technology to extract highly discriminative structural features and semantic features from point clouds and images, respectively, overcoming the limitations of a single sensor in specific scenarios.

[0018] The information fusion module dynamically adjusts the contributions of laser and visual data through a weighted allocation strategy, avoiding the error amplification problem caused by traditional simple overlay. For example, in low-light conditions, the system automatically downweights visual data, prioritizing the high-precision spatial information of the laser point cloud. In texture-rich areas, the system upweights visual semantic features to enhance map interpretability. This adaptive fusion mechanism significantly improves map robustness.

[0019] The dynamic optimization module enables real-time map updates through incremental data processing. The influence weights of optimization variables, based on a time decay function and data quality assessment, ensure the rational use of new data, avoiding interference from outdated information and the computational burden of frequent updates. A coordinate transformation algorithm for position offsets, combined with semantic segmentation technology for attribute updates, enables the map to accurately reflect environmental changes, such as the trajectory of moving objects or the attributes of newly added obstacles.

[0020] Experiments have shown that this system reduces map construction errors and significantly improves semantic annotation accuracy in dynamic indoor and outdoor scenarios, while also meeting the real-time requirements of robot navigation. Furthermore, the system's modular design facilitates expansion and adaptability to diverse sensor configurations and application scenarios, providing reliable technical support for practical deployment in areas such as autonomous driving and intelligent warehousing. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A diagram showing the working principle of the laser vision-integrated map dynamic construction system provided in an embodiment of the present application;

[0022] Figure 2 A design diagram for extracting spatial structure features from point clouds provided in an embodiment of the present application;

[0023] Figure 3 A design diagram for multimodal information fusion provided in an embodiment of the present application;

[0024] Figure 4 A design diagram of the dynamic optimization module provided in an embodiment of the present application;

[0025] Figure 5 A design diagram for calculating the influence weight of the optimization quantity provided in the embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0027] See also Figure 1-Figure 5 The present application relates to a laser vision-integrated map dynamic construction system, which includes: a data perception module, a feature analysis module, an information fusion module, and a dynamic optimization module. The specific implementation steps are as follows:

[0028] The data perception module uses a laser ranging device to emit laser signals at a fixed frequency, receives reflected echoes and converts them into point cloud coordinate data, thereby collecting an environmental point cloud sequence, which contains spatial coordinate information. At the same time, a visual imaging device is used to capture scene images at a synchronous frequency, and timestamp alignment is used to ensure the temporal consistency between the point cloud sequence and the image sequence, to obtain a scene image sequence, which contains texture and color information. The point cloud coordinate data is mapped to the global coordinate system through a coordinate conversion unit, and the image sequence is eliminated of lens distortion through a distortion correction unit.

[0029] The feature parsing module extracts spatial structural features based on point cloud sequences. Specifically, it selects a target region within the point cloud sequence and calculates the number and distribution density of neighboring points for each point within the target region. It calculates the normal direction of each point by using the coordinate difference of the neighboring points. The normal direction is determined by the principal component analysis results of the neighboring points. Based on the number of neighboring points, distribution density, and normal direction information, it constructs a multidimensional feature vector containing density features, direction features, and shape features. It also extracts visual semantic features based on the image sequence. Visual semantic features include edge features and regional features. Image edges are detected using gradient operators, and the coordinates and gradient amplitudes of edge points are obtained. Regions are divided based on the connectivity of the edge points. The area, aspect ratio, and texture complexity of each region are calculated, and this information is integrated into a matrix form.

[0030] The information fusion module performs multimodal information fusion of spatial structural features and visual semantic features to generate an initial fusion map. Specifically, a three-dimensional spatial grid model is established based on the density features, direction features, and shape features in the spatial structural features. A two-dimensional image semantic model is established based on the edge features and area features in the visual semantic features. The three-dimensional spatial grid model and the two-dimensional image semantic model are aligned through coordinate mapping to fill the semantic information into the spatial grid.

[0031] The dynamic optimization module uses the incremental data collected in real time to dynamically optimize the initial fused map and adjust the position and attribute parameters of each area in the map. The incremental data includes the newly collected point cloud sequence and image sequence.

[0032] Example 1:

[0033] The data perception module in this embodiment is specifically implemented as follows: Its core function is to synchronize the acquisition of environmental point cloud sequences and scene image sequences, and to preprocess the acquired data to ensure accuracy and consistency during subsequent map construction. This module primarily comprises a laser rangefinder, a visual imaging device, a timestamp alignment unit, a coordinate conversion unit, and a distortion correction unit, working together to complete data acquisition and preprocessing.

[0034] When operating, a laser ranging device continuously emits a laser signal at a pre-set fixed frequency. This fixed frequency is determined based on the specific application scenario. For example, in indoor environments, the frequency can be set to 100 times per second, taking into account scene complexity and data processing requirements. In open outdoor environments, the frequency may be higher to obtain more comprehensive environmental information. When the laser signal is emitted, it is reflected by objects in the environment. The laser ranging device's receiving unit captures these reflected echoes. Upon receiving the echo signal, the receiving unit performs a series of signal processing, including signal amplification and filtering, to remove noise interference. Then, using a built-in algorithm, the echo signal is converted into point cloud coordinate data. This point cloud coordinate data contains the spatial position information of objects in the environment. Each point cloud data point is typically represented as three-dimensional coordinates (X, Y, Z), forming an environmental point cloud sequence.

[0035] The visual imaging device and the laser ranging device need to work synchronously and capture scene images at a synchronized frequency. The synchronization frequency is set to match the emission frequency of the laser ranging device. For example, when the laser ranging device transmits signals at a frequency of 100 times per second, the visual imaging device also captures images at a speed of 100 frames per second. This ensures that the point cloud sequence and the image sequence correspond one-to-one in time. To ensure this temporal consistency, the system uses timestamp alignment. Specifically, each time the laser ranging device transmits a laser signal, a corresponding timestamp is generated to record the precise time of the signal emission; similarly, when the visual imaging device captures each frame of an image, it also adds a timestamp to the frame. Subsequently, the point cloud data and image data at the same time point are associated through timestamp matching to avoid data misalignment caused by time asynchrony.

[0036] After the point cloud coordinate data is generated, it needs to be mapped to the global coordinate system through the coordinate conversion unit. This is because when the laser ranging device collects point cloud data at different times, its own position may have moved. If the point cloud data in the local coordinate system of the device is used directly, the point cloud data at different times will not be correctly superimposed in the same space. The coordinate conversion unit will first obtain the real-time posture information of the laser ranging device when collecting point cloud data. The posture information includes parameters such as the position coordinates and rotation angle of the device. Then, based on this posture information, the point cloud coordinate data in the local coordinate system of the device is converted to the global coordinate system through the coordinate transformation matrix, so that all point cloud data are in the same reference frame, providing a unified spatial reference for subsequent feature extraction and map construction.

[0037] During the acquisition process of the image sequence, due to the physical characteristics of the lens of the visual imaging device, the captured images often have distortions, such as radial distortion and tangential distortion. These distortions will cause the shape of the object in the image to deviate from the actual shape, affecting the subsequent extraction of visual semantic features. Therefore, it is necessary to process the image sequence through a distortion correction unit. The distortion correction unit first needs to calibrate the visual imaging device and obtain the distortion parameters of the lens, such as radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2, etc. When correcting the image, for each pixel coordinate in the image, the corrected coordinate position is calculated based on its original coordinates and distortion parameters, and then the corrected image is pixel-filled through an interpolation algorithm to eliminate lens distortion and obtain an undistorted scene image sequence.

[0038] In actual operation, the laser ranging device and the visual imaging device require strict time synchronization and spatial calibration. Time synchronization can be achieved through hardware synchronization trigger signals to ensure that the starting points of the two operating cycles are consistent. Spatial calibration is to determine the relative positional relationship between the laser ranging device and the visual imaging device, namely the extrinsic parameter matrix, including the rotation matrix R and the translation vector T. Spatial calibration enables precise spatial matching of point cloud data and image data, for example, mapping a point in the point cloud data to a pixel in the image. This lays the foundation for subsequent information fusion.

[0039] When collecting data, the data perception module also needs to consider the impact of environmental factors. For example, in dimly lit environments, the image quality of visual imaging devices may degrade. In this case, image quality can be improved by increasing the camera's sensitivity or adding fill lighting. In bright light environments, overexposure may occur, requiring adjustment of the camera's exposure parameters. For laser ranging devices, the reflection of the laser signal may be affected when encountering transparent or reflective objects, resulting in missing or inaccurate point cloud data. This can be compensated by averaging multiple measurements or combining data from other sensors.

[0040] The data perception module also features data caching and transmission. The collected point cloud and image sequences are first stored in a cache and then transmitted via high-speed data transmission interfaces (such as USB 3.0 and Ethernet) to the subsequent feature analysis module for processing. To ensure stable and efficient data transmission, data compression techniques are typically used during the transmission process to reduce data volume without compromising data quality. Error checking mechanisms are also implemented to prevent data loss or errors during transmission.

[0041] Example 2:

[0042] The specific implementation of the feature parsing module in this embodiment for extracting spatial structural features from point cloud sequences is as follows: The core task of the feature parsing module is to extract key features from point cloud sequences that reflect the spatial structure of the environment, providing structured data support for subsequent information fusion and map construction. This process primarily involves four key steps: target region selection, neighborhood point statistics, normal vector calculation, and multidimensional feature vector construction. These steps are interconnected and collectively enable the precise extraction of spatial structural features.

[0043] Selecting a target region within a point cloud sequence is the first step in feature extraction. Point cloud sequences typically contain a large amount of point cloud data. Directly processing all of this data consumes significant computing resources and is unnecessary. Therefore, it is necessary to delineate a suitable target region within the point cloud sequence based on actual needs and the current processing focus. Target region selection can be performed in a variety of ways, such as manually defining a region of interest (ROI) or automatically capturing the region within a preset spatial range. Automatic selection can determine a specific region centered on the laser rangefinder based on its current position and scanning range, such as a sphere with a radius of 5 meters or a cube with a side length of 10 meters. Furthermore, dynamic adjustment can be performed based on the device's motion state. When the device is stationary, the target region can remain relatively fixed; when the device moves, the target region moves with the device, ensuring that point cloud data within the current field of view is always processed.

[0044] After selecting the target area, it is necessary to calculate the number and distribution density of neighboring points for each point in the area. For each point in the target area, determine its neighborhood range. The size of the neighborhood range can be set according to the density of the point cloud. In areas with high point cloud density, the neighborhood radius can be set to a smaller value, such as 0.1 meters; in areas with low point cloud density, the neighborhood radius can be appropriately increased, such as 0.5 meters. Then, with the point as the center, search for all neighboring points within the set neighborhood radius and count the number of neighboring points. This number is the number of neighboring points for the point. The number of neighboring points reflects the density of points around the point and is an important indicator for measuring the local characteristics of spatial structure.

[0045] Based on the number of neighborhood points, the distribution density of the neighborhood points is further calculated. The calculation of distribution density needs to take into account the volume or area of ​​the neighborhood. For three-dimensional point cloud data, the point cloud density within the neighborhood sphere is usually calculated, that is, the ratio of the number of neighborhood points to the volume of the neighborhood sphere. The distribution density can more intuitively reflect the distribution of the point cloud in space. For example, in flat areas such as walls, the point cloud distribution density is relatively uniform; while in areas such as corners and object edges, the point cloud distribution density may suddenly change. By calculating the number of neighborhood points and the distribution density, we can preliminarily determine the spatial structure type of each point in the target area, such as plane, edge, or corner point.

[0046] The normal direction of each point is calculated by using the coordinate differences of neighboring points. The normal vector is an important parameter for describing the orientation of a spatial surface and can reflect the surface orientation of a local area in a point cloud. For each point in the target area, principal component analysis (PCA) is performed on the coordinate data of the neighboring points within its neighborhood to determine the normal direction. The basic principle of PCA is to transform the coordinate data of the neighboring points into a new orthogonal basis. This orthogonal basis is arranged from largest to smallest according to the data variance, and the direction with the smallest variance is the approximate direction of the normal vector.

[0047] During the specific implementation process, the covariance matrix of the neighborhood points is first constructed. The covariance matrix is ​​used to describe the correlation and variance distribution of the neighborhood point coordinate data in various dimensions. For each neighborhood point, its coordinate value is centered relative to the point, that is, the coordinate value of the point is subtracted to obtain the centralized coordinate data. Then, the covariance matrix is ​​calculated based on the centralized coordinate data. By performing eigenvalue decomposition on the covariance matrix, the eigenvalues ​​and corresponding eigenvectors are obtained. The size of the eigenvalue reflects the variance of the data in the direction of the corresponding eigenvector, and the eigenvector corresponding to the smallest eigenvalue is the direction of the normal vector. In order to ensure the consistency of the normal vector direction, it is usually necessary to normalize and adjust the direction of the normal vector according to the actual situation of the scene, such as uniformly pointing to the outside or inside of the scene.

[0048] After calculating the number of neighborhood points, their distribution density, and the direction of their normal vectors, this information needs to be integrated into a multidimensional feature vector, which serves as a representation of the spatial structure. The construction of this multidimensional feature vector requires comprehensive consideration of the physical meaning and representational capabilities of each feature to ensure that the feature vector can fully and accurately describe the spatial structure.

[0049] Specifically, the multidimensional feature vector contains density features, direction features, and shape features. The density feature is directly composed of the number of neighborhood points and the distribution density. For example, the number of neighborhood points and the distribution density can be used as two independent dimensions in the feature vector. The direction feature is mainly composed of the direction parameters of the normal vector. The normal vector can be represented by three direction cosines in three-dimensional space, that is, the cosine values ​​of the angles between the normal vector and the X, Y, and Z axes. These three values ​​can be used as the three dimensions in the feature vector. The shape feature needs to be extracted in combination with the distribution of neighborhood points and the normal vector information. For example, the shape of the local area can be reflected by calculating the eigenvalue ratio in the principal component analysis of the neighborhood points, such as plane, linear or spherical. The eigenvalue ratio can be defined as the ratio of the minimum eigenvalue to the maximum eigenvalue, the ratio of the intermediate eigenvalue to the maximum eigenvalue, etc. These ratios serve as other dimensions in the feature vector.

[0050] In addition, to improve the robustness and distinguishability of features, each feature can be normalized and mapped to the same numerical range to avoid the impact caused by different feature dimensions. At the same time, according to actual application requirements, other auxiliary features can be added to the feature vector, such as the curvature value of the point. The curvature value can be calculated by the distance from the neighborhood point to the fitting plane, reflecting the degree of curvature of the local area.

[0051] In practical applications, the feature parsing module's processing needs to account for noise and outliers in point cloud data. Due to measurement errors in laser ranging equipment or environmental factors, noise or outliers may exist in the point cloud sequence, interfering with feature extraction. Therefore, before performing neighborhood point statistics and normal vector calculations, it is often necessary to filter the point cloud data within the target area. Methods such as voxel grid filtering and statistical outlier filtering can be used to remove noise and outliers and improve feature extraction accuracy.

[0052] At the same time, in order to improve processing efficiency, the feature analysis module can use parallel computing technology to process the point cloud data in the target area in blocks, and multiple processing units can extract features from point cloud data in different blocks at the same time, thereby shortening the overall processing time and meeting real-time requirements.

[0053] Example 3:

[0054] In this embodiment, the feature parsing module extracts visual semantic features from image sequences as follows: The feature parsing module extracts key features from the image sequence that reflect the scene's semantic information. These features transform edge and region information within the image into structured data, supporting subsequent fusion with point cloud features and the construction of map semantic attributes. This process primarily involves four core steps: edge feature extraction, region segmentation, feature calculation, and matrix integration. These steps collaborate through algorithms to achieve comprehensive extraction of visual semantic features.

[0055] When extracting visual semantic features, edge features and region features are two key components. Edge features reflect the outlines and structural boundaries of objects in an image, while region features reflect the attributes of different semantic regions. The combination of the two can more completely describe the semantic information of an image.

[0056] Edge feature extraction is achieved through the gradient operator. An image is essentially a two-dimensional pixel matrix. The gradient operator can detect locations where pixel values ​​change dramatically, i.e., edges. Common gradient operators include the Sobel operator and the Prewitt operator. For example, the Sobel operator uses two 3×3 convolution kernels to perform convolution operations on the rows and columns of the image, respectively. It calculates the horizontal and vertical gradient components, then takes the square root of the square root to obtain the gradient magnitude, and uses the inverse tangent function to obtain the gradient direction.

[0057] In specific implementation, the gradient operator is applied pixel by pixel to each frame of the image sequence. For each pixel, its gradient value in the horizontal direction (Gx) and vertical direction (Gy) is calculated, and the gradient amplitude is , gradient direction When the gradient magnitude is greater than the first preset threshold, the pixel is determined to be an edge point, and its coordinates (x, y) and gradient magnitude G are recorded. The setting of the first preset threshold needs to be adjusted according to the noise level and edge strength of the image. For example, in an image with less noise, the first preset threshold can be set to 50, and when the noise is higher, it can be increased to 100 to reduce false detections and missed detections.

[0058] After edge point detection, region segmentation is performed based on edge point connectivity. Connectivity refers to the spatial proximity of edge points and is typically defined as connectivity within four (up, down, left, and right) or eight (including diagonal) neighborhoods. Using a connected domain labeling algorithm, such as the double-scan method, edge points in the image are grouped and interconnected into connected domains. Each connected domain corresponds to a contour boundary in the image, and different regions are then partitioned around these boundaries.

[0059] The specific steps are as follows: The first scan assigns a temporary label to each pixel. When encountering unlabeled edge points, a new label is assigned, and label conflicts between adjacent pixels are recorded. The second scan merges equivalent labels based on label conflicts, ultimately determining a unique region label for each pixel. The divided regions can be complete objects, background parts, or other semantic units, and each region consists of a series of consecutive pixels.

[0060] After the region division is complete, the area, aspect ratio, and texture complexity of each region need to be calculated. The area of ​​the region is obtained by counting the number of pixels within the region and directly reflects the size of the region. The aspect ratio is determined by calculating the ratio of the length to the width of the region's circumscribed rectangle and can describe the region's shape characteristics, such as rectangular, circular, or irregular. Texture complexity is used to measure the degree of variation in the pixel grayscale values ​​within the region. It can be characterized by calculating texture feature parameters such as contrast, entropy, and energy using the gray-level co-occurrence matrix (GLCM), or by calculating the variance of the pixel grayscale values ​​within the region.

[0061] Taking grayscale variance as an example, for all pixels in a region, the average grayscale value is calculated. The average of the sum of the squares of the differences between each pixel's grayscale value and the average is then calculated. A larger value indicates a more complex texture within the region. For example, grass is typically more complex than a wall, so the grayscale variations in grass pixels are more dramatic.

[0062] After obtaining information such as edge point coordinates, gradient magnitude, region area, aspect ratio, and texture complexity, this information needs to be integrated into a matrix format as visual semantic features. The matrix structure design must consider the feature dimension and the convenience of subsequent processing. Typically, a multidimensional feature matrix is ​​used, with each row corresponding to a feature unit (such as an edge point or region) and each column corresponding to a different feature dimension.

[0063] Specifically, for edge features, each row of the matrix records the coordinates (x, y) and gradient magnitude G of an edge point, forming an N×3 matrix, where N is the number of edge points. For region features, each row records the area, aspect ratio, and texture complexity parameters of a region, forming an M×3 matrix, where M is the number of regions. To associate edge features with region features, an index field can be added to the matrix to identify the region to which the edge point belongs.

[0064] Furthermore, to facilitate subsequent fusion with point cloud features, the feature matrix must be normalized to map features of different dimensions to the same numerical range, such as the interval [0, 1]. This normalization method can employ minimum-maximum normalization, where eigenvalue = (original feature value - minimum value) / (maximum value - minimum value), to avoid fusion effects affected by differences in feature dimensions.

[0065] In practical applications, image sequences may be affected by factors such as illumination variations and noise, leading to errors in edge detection and region segmentation. To improve the robustness of feature extraction, image preprocessing can be performed before edge detection, such as using Gaussian filtering to reduce noise interference on edge detection. To address uneven illumination, methods such as histogram equalization can be used to adjust image contrast and enhance edge features.

[0066] For semantic regions in complex scenes, such as images containing multiple objects, after region division, it may be necessary to combine prior knowledge or machine learning methods for semantic annotation. For example, the region can be classified into semantic categories such as "wall", "ground", and "furniture" through a trained semantic segmentation model. However, in this embodiment, the focus is on the basic process of feature extraction, and semantic annotation can be used as a subsequent expansion step.

[0067] To improve processing efficiency, the feature parsing module can employ a pipelined approach, processing each frame of the image sequence in parallel. It also leverages the GPU to accelerate computationally intensive gradient operators and region labeling algorithms to meet real-time processing requirements. Furthermore, for image sequences in dynamic scenes, inter-frame differencing or optical flow methods can be combined to detect moving regions, prioritizing feature extraction in dynamic areas to improve feature effectiveness.

[0068] Example 4:

[0069] In this embodiment, the information fusion module performs multimodal information fusion of spatial structural features and visual semantic features to generate an initial fused map. The core function of the information fusion module is to organically combine spatial structural features extracted from point cloud sequences with visual semantic features extracted from image sequences. By establishing a mapping relationship between a 3D spatial model and a 2D semantic model, the module generates an initial fused map that incorporates both spatial structure and semantic attributes. This process primarily involves establishing a 3D spatial grid model, establishing a 2D image semantic model, aligning the models, and filling in semantic information. Each step achieves multimodal information fusion through data association and coordinate transformation.

[0070] When establishing a three-dimensional spatial grid model, it is necessary to base it on the density features, direction features, and shape features of the spatial structure features. The overall size of the three-dimensional grid model is determined based on the spatial distribution range of the point cloud data. For example, if the spatial range covered by the point cloud data is 10 meters long, 8 meters wide, and 3 meters high, the size of the three-dimensional grid model is set to be slightly larger than the range, such as 12 meters long, 10 meters wide, and 4 meters high, to ensure that all point cloud data can be included in the grid. Then, the three-dimensional space is divided into several uniform grid cells, and the size of the grid cells is determined according to the point cloud density and map accuracy requirements. In areas with high point cloud density, such as places with rich surface details of objects, the grid cells can be set to 0.1 meters × 0.1 meters × 0.1 meters to retain more details; in areas with low point cloud density, such as empty corridors, the grid cells can be appropriately increased to 0.5 meters × 0.5 meters × 0.5 meters to reduce the amount of data.

[0071] For each grid cell, its corresponding spatial structure features need to be filled. Specifically, the point cloud data points that fall into the grid cell are counted, and the density features (such as point cloud density), direction features (such as the average value of the normal vector) and shape features (such as the eigenvalue ratio obtained by principal component analysis) of these points are calculated. For example, a grid cell contains 100 point cloud data points. By calculating the neighborhood distribution density of these points, the density features of the grid are obtained; the normal vectors of these points are normalized and averaged to obtain the directional features of the grid; principal component analysis is performed on these points to obtain the eigenvalue ratio as the shape feature of the grid. In this way, each grid cell is given characteristic parameters that can reflect its spatial structure, thereby constructing a complete three-dimensional spatial grid model.

[0072] When building a 2D image semantic model, the edge features and region features of visual semantic features are used. First, each frame in the image sequence is used as the basis for the 2D semantic model. For each frame, the image is divided into different semantic regions using the previously extracted edge features and region features. For example, edge features are used to outline the contours of objects in the image, such as tables, chairs, and walls. Then, based on parameters such as area, aspect ratio, and texture complexity in the region features, a preliminary semantic definition is made for each region.

[0073] Each semantic region is assigned a corresponding semantic label and attribute parameters. Semantic labels can be common scene element categories such as "wall", "ground", and "furniture", while attribute parameters include the texture characteristics and color characteristics of the region. For example, after analysis, a region with low texture complexity, large area, and straight edges may be labeled as a "wall" and the RGB mean of its texture is recorded as an attribute parameter. In this way, the two-dimensional image is converted into a regional model containing semantic information. Each region has a clear semantic label and attribute description, forming a two-dimensional image semantic model.

[0074] The 3D spatial grid model needs to be aligned with the 2D image semantic model. The key to this process is establishing a coordinate mapping relationship between the two models, ensuring a one-to-one spatial correspondence between points in 3D space and pixel coordinates in the 2D image. First, the laser ranging device and the visual imaging device must be precisely calibrated for internal and external parameters, obtaining the rotation matrix and translation vector between them, as well as the camera's intrinsic parameter matrix. These parameters are used to establish the transformation relationship between 3D spatial coordinates and 2D image coordinates.

[0075] Let's use a specific example to illustrate the alignment process: Assume there's a point P in 3D space, with coordinates (X, Y, Z) in the global coordinate system. Using the camera's extrinsic matrix, the coordinates of point P are converted to coordinates in the camera coordinate system (Xc, Yc, Zc). Then, using the camera's intrinsic matrix, these coordinates in the camera coordinate system are projected onto the 2D image plane, yielding the corresponding pixel coordinates (u, v). This establishes a correspondence between point P in 3D space and the pixel coordinates (u, v) in the 2D image. Using this projection relationship, each grid cell in the 3D space grid model is matched to the corresponding region in the 2D image semantic model, achieving coordinate alignment between the two models.

[0076] After model alignment, the semantic information from the 2D image semantic model is added to the 3D spatial grid model to form the initial fused map. Specifically, for each grid cell in the 3D spatial grid model, the corresponding pixel region in the 2D image semantic model is found, the semantic label and attribute parameters of the region are obtained, and this information is added to the grid cell.

[0077] Continuing with the above example, grid cell A in 3D space corresponds to region B in the 2D image. Region B is labeled "wall" and has specific texture properties. In the initial fused map, grid cell A, in addition to its original spatial structural features, is also assigned the semantic label "wall" and the corresponding texture attribute parameters. In this way, each grid cell fuses spatial structural features and visual semantic features, forming a 3D map unit containing rich information.

[0078] In practical applications, one may encounter the problem of matching multiple frames of imagery with a three-dimensional spatial grid model. For example, when laser ranging equipment and visual imaging devices move, they capture multiple frames of imagery, each corresponding to a scene from a different angle. In this case, the semantic information of these multiple frames needs to be fused into the three-dimensional spatial grid model to improve the semantic integrity and accuracy of the map. A weighted fusion approach can be used to assign different weights to the semantic information of different images based on the shooting angle and distance of each frame. Images with closer distances and more positive angles have higher weights, and their semantic information has a greater impact on the grid cells.

[0079] The consistency and conflict resolution of semantic information also need to be considered. When conflicting semantic labels appear for the same grid cell in different frames, such as a grid cell labeled "table" in one frame and "chair" in another, conflict resolution needs to be performed using pre-set rules or machine learning methods. For example, the semantic label with the most occurrences can be selected as the final label based on the majority voting principle; or a deep learning model can be used to combine spatial structural features with multi-frame semantic information to make more accurate semantic predictions.

[0080] To improve fusion efficiency, the information fusion module can employ parallel computing techniques to process the 3D spatial grid model in blocks, while simultaneously filling in semantic information within each block. Furthermore, data structures such as octrees can be used to organize and manage the 3D grid model, improving data access and processing efficiency. This can significantly reduce computing resource consumption, especially when processing large-scale scenarios.

[0081] Example 5:

[0082] In this embodiment, the dynamic optimization module utilizes real-time incremental data to dynamically optimize the initial fused map as follows: The core function of the dynamic optimization module is to iteratively optimize the initial fused map based on real-time incremental data (including new point cloud sequences and image sequences). By adjusting the location and attribute parameters of each region within the map, the map can reflect dynamic changes in the environment in real time. This process encompasses five key steps: input layer data processing, hidden layer feature transformation, output layer optimization quantity generation, optimization quantity influence weight calculation, and optimization operation application. Each step achieves dynamic map updates through data flow and algorithm processing.

[0083] Taking an indoor corridor scene as an example, when a robot equipped with a laser ranging device and a visual imaging device moves in the corridor, the incremental data collected in real time will be continuously input into the dynamic optimization module. The input layer needs to take the point cloud features and image features in the incremental data as input, and the input dimension and number of features must be consistent. For example, the newly collected point cloud features may contain multidimensional vectors such as the density features and directional features of a certain area, while the image features may contain matrix data such as the coordinates of edge points and the complexity of regional textures. The input layer will organize these features into a unified input format according to the preset dimensions to ensure that the data can be processed by the subsequent hidden layers.

[0084] The hidden layer is configured with multiple nonlinear transformation layers. For example, three hidden layers are used, each containing several neurons. When input data is passed from the input layer to the first transformation layer, it is linearly combined with the weight matrix of that layer and then nonlinearly processed using an activation function (such as the ReLU function), returning negative values ​​to zero while retaining positive features, thereby obtaining the output of the first layer. For example, if the input to the first layer is a combined vector of point cloud density features and image edge features, after linear weighting and ReLU activation, the responses of density anomalies and strong edge features are highlighted. When the output of the first layer is passed to the second transformation layer, the above nonlinear processing is repeated to further extract high-order correlations between features. For example, the normal vector direction of the point cloud is mapped to the texture direction of the image area to enhance the expression of structured features. Until the last transformation layer is completed, the final transformation output containing spatial position offsets and attribute update cues is output. This output captures the difference between the incremental data and the existing map.

[0085] The output layer transforms the final output through linear projection to generate map optimizations, including spatial position offsets and attribute updates. For example, in a corridor scene, if newly collected point cloud data indicates that the front wall is 0.2 meters offset from the initial map, the output layer will generate the corresponding spatial position offset. Simultaneously, if image features indicate a new semantic label for "fire hydrant" in a certain area, the output layer will generate attribute updates for that area, such as updating the category label from "wall" to "firefighting facility."

[0086] When calculating the impact weight of the optimization amount, the collection time and credibility of the incremental data must be comprehensively considered. Taking collection time as an example, assuming that the map changes dynamically at a frequency of 1 update per minute, the time decay function can be set to an exponential decay form. The weight of the data collected at the current moment is 1, and the weight of the data collected 1 minute ago decreases at a decay rate of 0.8, that is, the weight is 0.8, and the weight of the data collected 2 minutes ago is 0.64, and so on, ensuring that the latest data has a greater impact on map optimization. The credibility weight is calculated based on indicators such as point cloud density, image clarity, and feature matching: If the newly collected point cloud density is 100 points per cubic meter, which is higher than the second preset threshold of 80 points, the point cloud density index score is 0.9; image clarity is calculated by calculating the pixel gradient variance. If the variance value is 200, which is higher than the standard value of 150, the clarity score is 0.8; feature matching is calculated by comparing the degree of fit between the incremental data features and the existing map features. If the matching rate is 90%, the score is 0.9. The three indicators are weighted at 0.4, 0.3, and 0.3, and the resulting credibility weight is 0.87. The final optimization weight is the weighted sum of the time weight and the credibility weight. For example, the time weight of 0.9 and the credibility weight of 0.87 are weighted at 0.5:0.5 to obtain 0.885.

[0087] When applying an optimization operation, the optimization amount is multiplied by the impact weight and then superimposed on the initial fused map. Regarding spatial position adjustment, when the optimization type is position offset, the real-time positioning data from the laser ranging device indicates a global coordinate deviation of 0.3 meters for the current map. Combined with the 0.2-meter spatial offset generated by the output layer, the calculated regional displacement is 0.5 meters. Using a coordinate transformation algorithm, the corridor wall coordinates in the map are shifted 0.5 meters in the offset direction to align the wall position with the actual environment.

[0088] In terms of attribute updates, image semantic segmentation technology is used to identify new attribute information in incremental data. For example, a region in an image sequence is identified as a "fire hydrant" through semantic segmentation, and its texture features match the template of a fire hydrant in the database by 95%. The new attribute information includes the category label "fire hydrant" and specific texture parameters. Based on the attribute update value, the category label of the corresponding area in the initial map is updated from "wall" to "fire hydrant" through the attribute replacement rule, and the texture parameters are replaced to complete the regional attribute adjustment. Among them, the attribute replacement rule is: when the newly identified category is different from the original category, and the texture feature matching degree exceeds 90%, the category label of the area in the initial map is replaced with the new category, and the texture parameters are replaced.

[0089] During the dynamic optimization process, it is also necessary to address the temporal consistency of multi-source data. For example, when the acquisition frequencies of the laser ranging device and the visual imaging device are 10Hz and 30Hz, respectively, it is necessary to align the incremental data of different frequencies to the same time axis through timestamp interpolation to ensure the temporal correlation of the data when calculating the optimization quantity. For moving objects in a dynamic environment, such as people walking in a corridor, their motion trajectory can be detected by the feature changes of multiple frames of data, avoiding updating the position of the moving object to the map and only optimizing the static environment structure.

[0090] In addition, the optimization module maintains a historical optimization record database, recording information such as the time, optimization amount, and weight of each optimization. If an area is optimized multiple times within a short period of time (e.g., 5 minutes), the system automatically triggers a stability check. If the fluctuation range of the optimization amount exceeds a third preset threshold (e.g., position offset fluctuation exceeds 0.1 meter), the system deems that data anomalies or rapid environmental changes may exist in that area, necessitating increased sampling frequency or cross-validation with other sensor data to improve optimization reliability.

[0091] In terms of computing resource allocation, for nonlinear transformation operations in the hidden layer, the parallel computing capabilities of the GPU can be utilized to allocate calculations of different feature dimensions to different threads for processing. For example, the transformations of point cloud features and image features can be processed by different GPU cores respectively, thereby improving computing efficiency and ensuring that map optimization can keep up with the speed of environmental changes during the real-time movement of the robot.

[0092] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0093] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A dynamic map construction system integrating laser vision, characterized by: include: Data perception module: collects environmental point cloud sequences through laser ranging equipment and obtains scene image sequences using visual imaging devices; the point cloud sequences contain spatial coordinate information, and the image sequences contain texture and color information; Feature parsing module: Extracts spatial structural features based on point cloud sequences and visual semantic features based on image sequences; spatial structural features are calculated using neighborhood point statistics, and visual semantic features are obtained using edge detection and region segmentation techniques; Information fusion module: This module fuses spatial structural features with visual semantic features to generate an initial fusion map. Multimodal information fusion adopts a weight distribution strategy, setting fusion weights based on data credibility. Dynamic Optimization Module: This module uses real-time collected incremental data to dynamically optimize the initial fused map and adjust the location and attribute parameters of each area in the map. The incremental data includes newly collected point cloud sequences and image sequences. The specific method of dynamically optimizing the initial fused map using the incremental data collected in real time and adjusting the position and attribute parameters of each area in the map includes: The input layer takes the point cloud features and image features in the incremental data as input; the input dimension is set to be consistent with the number of features; Hidden layer, set up multiple nonlinear transformation layers; pass the input data from the input layer to the first transformation layer, obtain the first layer output through linear combination and activation function processing, pass the first layer output to the next transformation layer, repeat the nonlinear processing until the last transformation layer is completed, and obtain the final transformation output; The output layer passes the final transformation output to the output layer and generates map optimization quantities through linear projection; the optimization quantities include spatial position offsets and attribute update values; Determine the impact weight of the optimization amount based on the collection time and credibility of the incremental data; Apply the optimization operation, multiply the optimization amount by the influence weight and add it to the initial fusion map to complete the dynamic optimization; The specific method of performing multimodal information fusion of spatial structural features and visual semantic features to generate an initial fused map includes: Based on the density, direction and shape features of the spatial structure, a three-dimensional spatial grid model is established; Based on the edge features and regional features in visual semantic features, a two-dimensional image semantic model is established; The 3D spatial grid model is aligned with the 2D image semantic model through coordinate mapping, and the semantic information is filled into the spatial grid to form an initial fusion map containing spatial structure and semantic attributes. The specific method of determining the influence weight of the optimization amount based on the collection time and credibility of the incremental data includes: The time weight is calculated based on the acquisition time of the incremental data through the time decay function. The decay rate of the time decay function is set according to the frequency of dynamic changes of the map. The credibility weight of incremental data is calculated using data quality assessment indicators, including point cloud density, image clarity, and feature matching. Combining the time weight and the credibility weight, the influence weight of the optimization amount is calculated through the weighted sum formula.

2. The laser vision-integrated map dynamic construction system according to claim 1 is characterized in that: The specific method of collecting the environment point cloud sequence by the laser ranging device and obtaining the scene image sequence by the visual imaging device includes: The laser ranging device emits laser signals at a fixed frequency, receives reflected echoes and converts them into point cloud coordinate data; the visual imaging device captures scene images at a synchronous frequency, and ensures the temporal consistency of the point cloud sequence and the image sequence through timestamp alignment; the point cloud coordinate data is mapped to the global coordinate system through a coordinate conversion unit, and the image sequence eliminates lens distortion through a distortion correction unit.

3. The laser vision-integrated map dynamic construction system according to claim 2 is characterized in that: The specific method of extracting spatial structure features based on point cloud sequence includes: Step 1: Select the target area in the point cloud sequence and calculate the number and distribution density of neighboring points of each point in the target area; Step 2: Calculate the normal vector direction of each point by the coordinate difference of the neighboring points. The normal vector direction is determined by the principal component analysis result of the neighboring points. Step 3: Based on the number of neighborhood points, distribution density, and normal vector direction information, a multidimensional feature vector containing density features, direction features, and shape features is constructed as the spatial structure feature.

4. The laser vision-integrated map dynamic construction system according to claim 3 is characterized in that: The specific method of extracting visual semantic features based on image sequences includes: Visual semantic features include edge features and regional features; Detect image edges through gradient operators and obtain the coordinates and gradient amplitude of edge points; Based on the connectivity of edge points, the region is divided and the area, aspect ratio and texture complexity of each region are calculated; The coordinates of edge points, gradient amplitude, region area, aspect ratio and texture complexity are integrated into a matrix form as visual semantic features.

5. The laser vision-integrated map dynamic construction system according to claim 1 is characterized in that: When the optimization amount includes spatial position offset, adjust the position of each area in the map, including: When the optimization type is position offset, the real-time positioning data of the laser ranging device is used to obtain the global coordinate deviation of the current map; the regional displacement is calculated by the global coordinate deviation and the spatial position offset; based on the regional displacement, the coordinate values ​​of each area in the map are adjusted using the coordinate transformation algorithm to move the area to the optimized position.

6. The laser vision-integrated map dynamic construction system according to claim 5 is characterized in that: When the optimization includes attribute update values, adjust the attributes of each area in the map, including: Image semantic segmentation technology is used to identify new attribute information in incremental data; the new attribute information includes regional categories and texture features; based on the new attribute information and attribute update values, the category labels and texture parameters of the corresponding areas in the map are updated through attribute replacement rules to complete regional attribute adjustment.

7. A method for dynamically constructing a map by integrating laser vision, which is applied to the system for dynamically constructing a map by integrating laser vision according to any one of claims 1 to 6, characterized in that: include: S1. Collect an environmental point cloud sequence using a laser ranging device and obtain a scene image sequence using a visual imaging device; extract spatial structural features based on the point cloud sequence and extract visual semantic features based on the image sequence; wherein the point cloud sequence contains spatial coordinate information and the image sequence contains texture and color information; S2, multimodal information fusion of spatial structural features and visual semantic features to generate an initial fusion map; multimodal information fusion adopts a weight distribution strategy, and the fusion weight is set according to the data credibility; S3. Dynamically optimize the initial fused map using incremental data collected in real time, adjusting the position and attribute parameters of each area in the map. Incremental data includes newly collected point cloud sequences and image sequences. During the dynamic optimization process, the influence weight of the optimization amount is determined based on the acquisition time and credibility of the incremental data, and the adjustment of spatial position offset and attribute update is handled separately. The specific method of dynamically optimizing the initial fused map using the incremental data collected in real time and adjusting the position and attribute parameters of each area in the map includes: The input layer takes the point cloud features and image features in the incremental data as input; the input dimension is set to be consistent with the number of features; Hidden layer, set up multiple nonlinear transformation layers; pass the input data from the input layer to the first transformation layer, obtain the first layer output through linear combination and activation function processing, pass the first layer output to the next transformation layer, repeat the nonlinear processing until the last transformation layer is completed, and obtain the final transformation output; The output layer passes the final transformation output to the output layer and generates map optimization quantities through linear projection; the optimization quantities include spatial position offsets and attribute update values; Determine the impact weight of the optimization amount based on the collection time and credibility of the incremental data; Apply the optimization operation, multiply the optimization amount by the influence weight and superimpose it on the initial fusion map to complete the dynamic optimization.

Images