A scene-prior-based slam method, device, and medium

Abstract

The application discloses a scene-prior-based SLAM method, device and medium, relates to the technical field of instant positioning and map construction, and comprises the following steps: extracting feature parameters of mapping data, performing feature matching and loop detection on the feature parameters, and determining pose transformation constraints; identifying the scene category of a mapping environment, querying the prior weight coefficients corresponding to the mapping data, and constructing a prior weight matrix; converting a degradation index vector into a weight fine-tuning matrix through a preset multi-layer perception machine; calculating a weight coefficient matrix according to the prior weight matrix and the weight fine-tuning matrix; fusing the weight coefficient matrix and an information matrix, determining an optimization objective function, and obtaining a factor graph through factor graph optimization of LVI-SAM; and determining a pose estimation sequence and a map corresponding to the mapping environment through point cloud processing according to the factor graph and the pose transformation constraints. The application solves the technical problem that the prior SLAM method cannot be adaptively adjusted according to the underground extreme environment.

Description

Technical Field

[0001] This application relates to the field of real-time localization and mapping technology, and in particular to a scene-prior-based SLAM method, device and medium. Background Technology

[0002] Simultaneous localization and mapping (SLAM) technology is widely used in autonomous vehicles, robots, and many other fields. Its development has progressed from single-sensor to multi-sensor systems. For extreme environments such as mines, multi-mode fusion SLAM has overcome the problem of insufficient generalization ability of single sensors. The ability to perform high-precision mapping is the foundation and primary prerequisite for the automation and intelligentization of mining processes.

[0003] The development and innovation of existing SLAM algorithms are mostly aimed at applications in fields such as robotics. General-purpose SLAM systems suffer severe performance degradation in extreme working conditions such as mines. The fundamental reason is that traditional methods employ fixed or simple rule-based sensor fusion strategies, which cannot dynamically respond to the rapidly changing and concurrent demands of sensor confidence in mining environments. Firstly, drastic changes in lighting and signal rejection, such as movement between the surface and underground, cause visual sensors to become unstable due to lighting variations. GPS becomes completely ineffective underground, and fixed weights cannot dynamically shut down failed sensors. Secondly, severe disturbances and feature degradation occur. Bumpy roads cause a surge in IMU measurement noise; dust, water mist, etc., lead to sparse lidar point clouds and abnormal reflectivity, while also blurring visual features. The degradation of a single sensor can contaminate the entire system. Thirdly, dynamic interference and scene switching, such as vehicles and personnel, disrupt the system's performance. Figure 1 In terms of consistency, existing technologies mostly optimize at the algorithm level alone, lacking adaptive adjustments based on sensor status at the fusion decision level. Summary of the Invention

[0004] This application provides a scenario-prior SLAM method, device, and medium, which solves the technical problem that existing SLAM methods cannot adaptively adjust to the needs of extreme underground environments.

[0005] In a first aspect, embodiments of this application provide a scene-prior SLAM method, characterized in that the method includes: acquiring environmental detection data collected by pre-set multimodal sensors in the mapping environment, and performing data preprocessing on the environmental detection data to determine the mapping data of the mapping environment; extracting feature parameters of the mapping data, and performing feature matching and loop closure detection on the feature parameters respectively to determine pose transformation constraints; identifying the scene category of the mapping environment, and querying the prior weight coefficients corresponding to the mapping data based on the scene category to construct a prior weight matrix; normalizing the sensor degradation index of the mapping data to obtain the degradation index vector of the mapping data, and converting the degradation index vector into a weight fine-tuning matrix through a pre-set multilayer perceptron; calculating the weight coefficient matrix according to the prior weight matrix and the weight fine-tuning matrix; fusing the weight coefficient matrix with the information matrix of each sensor factor corresponding to the mapping data to determine the optimization objective function, and obtaining the factor map through factor graph optimization of LVI-SAM based on the optimization objective function; and determining the robot's pose estimation sequence and the map corresponding to the mapping environment in the mapping environment through point cloud processing according to the factor map and pose transformation constraints.

[0006] In one implementation of this application, environmental monitoring data is preprocessed to determine mapping data for the mapping environment. Specifically, this includes: establishing a time series of environmental monitoring data, and performing linear interpolation on the environmental monitoring data based on the timestamps of the time series to obtain a multimodal observation dataset; wherein the environmental monitoring data includes: point cloud data, image data, inertial data, and absolute position data; and performing coordinate system transformation on the multimodal observation dataset to determine mapping data for the mapping environment.

[0007] In one implementation of this application, feature matching and loop closure detection are performed on the feature parameters to determine the pose transformation constraints. Specifically, this includes: matching the distortion-free current point cloud features in the feature parameters with the local map to calculate the inertial odometry pose; and based on the inertial odometry pose, matching the current point cloud in the feature parameters with the local point cloud in the local map, and calculating the root mean square error of the current point cloud and the local point cloud to obtain the pose transformation constraints.

[0008] In one implementation of this application, prior weight coefficients corresponding to the mapping data are queried based on the scene category to construct a prior weight matrix. Specifically, this includes: identifying keyframes corresponding to the mapping data, and determining the current scene based on the keyframes and scene classification probabilities; setting prior weight coefficients corresponding to the mapping data according to the scene category to obtain a prior weight table; and indexing the prior weight matrix corresponding to the scene category corresponding to the current scene in the prior weight table.

[0009] In one implementation of this application, the structure of the multilayer perceptron includes: an input layer, a first fully connected layer, a second fully connected layer, and an output layer. The degradation index vector is transformed into a weight fine-tuning matrix using a pre-defined multilayer perceptron. Specifically, this involves: inputting the degradation index vector into the first fully connected layer through the input layer, and activating and normalizing it using the ReLU function to obtain a 32-dimensional vector; inputting the 32-dimensional vector into the second fully connected layer, and activating the vector output by the second fully connected layer using the Sigmoid function through the output layer to obtain an initial weight fine-tuning matrix; wherein the initial weight fine-tuning matrix consists of fine-tuning coefficients; and constructing a loss function for weight fine-tuning based on the initial weight fine-tuning matrix, and constraining the fine-tuning coefficients using the loss function to obtain the weight fine-tuning matrix.

[0010] In one implementation of this application, the weight coefficient matrix is ​​calculated based on the prior weight matrix and the weight fine-tuning matrix. Specifically, this includes: multiplying the prior weight matrix and the weight fine-tuning matrix element by element to obtain the initial weight coefficient matrix; applying an exponential moving average filter to the initial weight coefficient matrix to determine the smoothed weights of the initial weight coefficient matrix at the corresponding time; and setting lower bound constraints on the diagonal elements of the initial weight coefficient matrix based on the smoothed weights to obtain the weight coefficient matrix.

[0011] In one implementation of this application, the weight coefficient matrix is ​​fused with the information matrix of each sensor factor corresponding to the mapping data to determine the optimization objective function. Specifically, this includes: extracting the four diagonal elements from the weight coefficient matrix, and adaptively adjusting the weights of the information matrix based on the diagonal elements to obtain the adjusted information matrix; and constructing the optimization objective function of the adjusted information matrix based on the adjusted information matrix and the residual vectors of each sensor factor.

[0012] In one implementation of this application, based on the factor map and pose transformation constraints, point cloud processing is used to determine the robot's pose estimation sequence and the map corresponding to the mapping environment. Specifically, this includes: optimizing the robot's pose in the mapping environment based on pose transformation constraints to determine the pose estimation sequence; obtaining a corrected point cloud through motion compensation based on the factor map and pose estimation sequence; transforming each frame of the corrected point cloud from the sensor coordinate system to the global world coordinate system to obtain the local world coordinates of the corrected point cloud; dividing the global map of the mapping environment into a voxel grid and selecting the centroid of the voxel grid as the representative point of the voxel grid to determine the map corresponding to the mapping environment.

[0013] Secondly, embodiments of this application also provide a scene-priority-based SLAM device, characterized in that the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement a scene-priority-based SLAM method.

[0014] Thirdly, embodiments of this application also provide a non-volatile computer storage medium for scene-prior SLAM, storing computer-executable instructions, characterized in that the computer-executable instructions, when executed, implement a scene-prior SLAM method.

[0015] This application provides a scene-prior-based SLAM method, device, and medium. Through adaptive weight adjustment based on scene priors, it learns the features of RGB images to identify the scene and obtains prior weights. Different prior weights are set for different scenes with varying lighting, signal rejection, road bumps, and dynamic factors. The degradation level of the sensor is analyzed, and indicators characterizing sensor performance, such as the number of feature points and signal reflection intensity, are obtained from the sensor data. Based on these indicators, the sensor weights are precisely adjusted. Through factor graph optimization of the weight matrix, the final environmental mapping is obtained. This solves the technical problem that existing SLAM methods cannot adaptively adjust to the needs of extreme underground environments, achieving adaptive mapping based on the scene environment and improving the overall response speed and scene generalization ability of the system. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A flowchart of a scene-prior SLAM method provided in this application embodiment;

[0018] Figure 2 A structural diagram of a scene-prior-based SLAM method provided in this application embodiment;

[0019] Figure 3 A scene-prior-based SLAM system architecture diagram is provided for embodiments of this application;

[0020] Figure 4 This is a schematic diagram of a scene recognition neural network provided in an embodiment of this application;

[0021] Figure 5A schematic diagram of weight fine-tuning matrix generation based on multilayer perceptron provided in this application embodiment;

[0022] Figure 6 This is a schematic diagram of the internal structure of a scene-prior SLAM device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] This application provides a scene-prior-based SLAM method, device, and medium. Through adaptive weight adjustment based on scene priors, it learns the features of RGB images to identify the scene and obtains prior weights. Different prior weights are set for different scenes with varying lighting, signal rejection, road bumps, and dynamic factors. The degradation level of the sensor is analyzed, and indicators characterizing sensor performance, such as the number of feature points and signal reflection intensity, are obtained from the sensor data. Based on these indicators, the sensor weights are precisely adjusted. Through factor graph optimization of the weight matrix, the final environmental mapping is obtained. This solves the technical problem that existing SLAM methods cannot adaptively adjust to the needs of extreme underground environments, achieving adaptive mapping based on the scene environment and improving the overall response speed and scene generalization ability of the system.

[0025] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 A flowchart illustrating a scene-prior-based SLAM method provided in this application embodiment. Figure 1 As shown in the figure, the SLAM method based on scene prior provided in this application embodiment specifically includes the following steps:

[0027] Step 101: Obtain environmental detection data collected by the preset multimodal sensors in the mapping environment, and perform data preprocessing on the environmental detection data to determine the mapping data of the mapping environment.

[0028] For example, this application provides a data foundation for feature matching and loop closure detection of mapping data by performing data preprocessing on environmental monitoring data.

[0029] Specifically, the environmental monitoring data is preprocessed to determine the mapping data for the mapping environment. This includes: establishing a time series of the environmental monitoring data, and performing linear interpolation on the environmental monitoring data based on the timestamps of the time series to obtain a multimodal observation dataset. The environmental monitoring data includes point cloud data, image data, inertial data, and absolute position data. The multimodal observation dataset is then subjected to coordinate system transformation to determine the mapping data for the mapping environment.

[0030] Figure 2 This is a structural diagram of a scene-prior SLAM method provided in an embodiment of this application.

[0031] Figure 3 This is a diagram illustrating a scene-prior SLAM system architecture provided for an embodiment of this application.

[0032] In one embodiment, before starting the mapping process, the relevant model of LiDAR, camera, IMU unit, and GPS unit are activated to collect data at a set frequency. The LiDAR collects point cloud information at the set frequency; a 32-line mechanical rotating LiDAR is used, with the acquisition frequency set to 10Hz. The LiDAR emits a laser beam and receives reflected signals, calculating the position and distance information of the reflecting points. All reflecting points form a three-dimensional point cloud, outputting a point cloud matrix. The point cloud matrix contains information such as reflection intensity, three-dimensional coordinates, the line containing the point, timestamp, and the number of points.

[0033] The camera acquires image information at a set frequency, using a dual-lens RGB camera with a capture frequency set to 20Hz. The acquired image resolution is at least 640×480, the images are in RGB format, and include timestamp information. After feature extraction, information such as the image's illumination intensity and the number of feature points can be obtained.

[0034] The IMU unit continuously outputs acceleration and angular velocity information. A six-axis IMU is used, with the acquisition frequency set to 200Hz. Raw data of linear acceleration and angular velocity are acquired. Each data point acquired by the IMU is recorded with a precise timestamp, and an inertial data sequence is output, containing three-axis acceleration and three-axis angular velocity.

[0035] The GPS unit collects global absolute position information using an RTK-GPS receiver at a frequency of 5Hz, acquiring latitude, longitude, altitude, and positioning accuracy factors. The raw GPS radio frequency signal is processed to output position information, including speed, heading, latitude, longitude, altitude, and positioning accuracy; each piece of information includes a timestamp.

[0036] Using the parameters collected by the sensors, a time series is established with a unified time step. Based on the timestamp, linear interpolation is performed on the image, point cloud, inertial data, and absolute position information respectively, and all sensor data are unified to the 10Hz time base of the lidar to form a synchronous multimodal observation dataset.

[0037] By using the calibrated extrinsic matrix, the point cloud is transformed from the lidar coordinate system to the vehicle coordinate system, i.e., the coordinate system of the vehicle-mounted robot, to ensure that it is consistent with the camera and IMU, and an aligned multimodal observation set is obtained to determine the mapping data of the mapping environment.

[0038] Step 102: Extract the feature parameters of the mapping data, and perform feature matching and loop closure detection on the feature parameters to determine the pose transformation constraints.

[0039] For example, in the mapping process, loop closure detection refers to the process of identifying whether a mobile vehicle has returned to a historical scene during SLAM mapping to correct accumulated odometry drift. The output of loop closure detection information includes three parts: First, Boolean information to determine the validity of the loop closure and output the matching keyframe ID; second, output the 6-DOF relative pose transformation between the matching keyframes, which describes the relative motion from the current frame pose to the stored keyframe, including rotation and translation matrices; third, constraint confidence. Visual loop closure detection outputs a similarity score, while laser loop closure detection outputs the root mean square error of registration. The output loop closure constraint information is added to the factor graph optimization as a loop closure factor to adjust the pose.

[0040] Specifically, feature matching and loop closure detection are performed on the feature parameters to determine the pose transformation constraints, including: matching the distortion-free current point cloud features in the feature parameters with the local map to calculate the excitation odometry pose; based on the odometry pose, matching the current point cloud in the feature parameters with the local point cloud in the local map, and calculating the root mean square error of the current point cloud and the local point cloud to obtain the pose transformation constraints.

[0041] In one embodiment, for each frame of RGB image, a modified FAST corner detector combined with Shi-Tomasi scoring is used for filtering via visual feature extraction. Simultaneously, an LSD line segment detector is employed to extract structured linear features. A BRIEF descriptor is calculated for each feature point. This step is preferably executed in parallel on a GPU to improve speed. Visual inertial odometry (LIO) pose is calculated by combining IMU pre-integration and laser depth.

[0042] Between adjacent keyframes, IMU data is pre-integrated to obtain relative pose changes. Simultaneously, the feature points of the current frame image are projected onto the nearest neighbor LiDAR point cloud map, and the depth value of the feature point is obtained through point cloud search, thereby upscaling the 2D image features into 3D spatial points. The IMU pre-integration results and depth-bearing visual features are fused, and the initial pose estimation of the visual inertial odometry (VIO) is solved through nonlinear optimization.

[0043] Then, preliminary loop closure detection is performed based on visual features. The DBoW2 bag-of-words model is used to convert the image features of the current keyframe into visual word vectors and perform similarity matching with historical keyframes. When the similarity exceeds the threshold, it is regarded as a candidate loop closure, which provides constraints for subsequent factor graph optimization.

[0044] Point cloud feature extraction and accurate loop closure detection: The point cloud data is distorted and features are extracted. Visual features and point cloud features are matched with the initial visual matching values. The laser inertial odometry pose is calculated and accurate loop closure detection is completed.

[0045] The laser point cloud distortion correction and feature extraction process utilizes the high-frequency pose estimation provided by VIO to compensate for point cloud distortion caused by carrier motion within a single frame. It calculates the curvature of each point and extracts two types of features based on the curvature: corner points and planar points. Using the pose estimation provided by VIO as the initial value for laser odometry scanning and matching significantly accelerates the convergence process of the Iterative Closest Point (ICP) algorithm or its variants (such as point-to-surface ICP).

[0046] Finally, the distortion-corrected point cloud features of the current frame are matched with the local map, and the angular velocity information of the IMU is fused to constrain the rotational search space for scanning and matching. The laser inertial odometry (LIO) pose is calculated using the Levenberg-Marquardt optimization algorithm. When a candidate loop closure is detected by vision, local point cloud registration is performed using the point cloud of the current keyframe and the point cloud of the loop closure candidate frame. By calculating the root mean square error (RMSE) of the matching between the two point cloud sets, if the RMS error is below a threshold, the loop closure is confirmed, and a high-precision pose transformation constraint is obtained.

[0047] Step 103: Identify the scene category of the mapping environment, and based on the scene category, query the prior weight coefficients corresponding to the mapping data to construct a prior weight matrix.

[0048] For example, since the sensor degradation levels vary in different underground environments (mining areas, tunnels, etc.), mining areas are classified, and a priori weight coefficient matrix is ​​set. A table is then created that corresponds one-to-one between the scenes and the weight coefficients. To speed up the system's response, this application uses a trained image feature extraction neural network to identify the scenes and then looks up the table to obtain the prior weights.

[0049] Specifically, based on the scene category, the prior weight coefficients corresponding to the mapping data are queried to construct a prior weight matrix. This includes: identifying the keyframes corresponding to the mapping data, and determining the current scene based on the keyframes and scene classification probabilities; setting the prior weight coefficients corresponding to the mapping data according to the scene category to obtain a prior weight table; and indexing the prior weight matrix corresponding to the scene category corresponding to the current scene in the prior weight table.

[0050] Figure 4 This is a schematic diagram of a scene recognition neural network provided in an embodiment of this application.

[0051] In one embodiment, firstly, the mining scene is classified according to the scene characteristics and the degree of sensor degradation. After classification, a prior weight coefficient matrix is ​​set and a table is created.

[0052] Based on the real environment of mining operations, the scenarios are predefined into six categories:

[0053] The first category is open scenes above ground. These scenes have a high proportion of sky, a wide field of view, and uniform natural lighting. A large area of ​​sky appears in the upper part of the image, the ground is open, and the horizon or distant mountains are clearly visible. The neural network learns to detect the color and texture of the sky, as well as the position of the horizon.

[0054] The second category is underground mine tunnel scenes. These scenes lack a sky, feature clearly defined artificial structures, and contain artificial point light sources. The images are devoid of sky; they are filled with regular straight lines and arches, and contain bright spots with strong contrasts between light and dark, resulting in uneven overall lighting. The neural network learns to recognize the geometric structure of the tunnel walls and the bright spots against the dark background.

[0055] The third category is dynamic scenes with multiple factors, such as vehicles and people, moderate sky coverage, and complex backgrounds. In open scenes, objects with specific shapes and colors, such as trucks, excavators, and personnel in safety suits, will appear in the image. Neural networks need strong object detection capabilities to learn and detect semantic targets such as vehicles and people.

[0056] The fourth category is high-dust scenes. These scenes exhibit reduced overall color saturation, decreased contrast, and a hazy image. The image appears yellowish-brown or grayish-white, and distant objects and edges become blurred; this may be accompanied by a dynamic blur effect from rising dust. Neural networks learn and analyze the image's chromaticity histogram, overall sharpness, and the richness of texture details.

[0057] The fifth category is scenes with degraded in-ground features. These scenes have simple and repetitive textures, monotonous colors, and lack prominent corners or edges, such as large areas of ore piles or cement walls. Corners that can be used for feature matching in the image are very scarce. The neural network learns to evaluate the number and evenness of the distribution of feature points in the image, as well as the repetition patterns of local textures.

[0058] The sixth category is tunnel entrance / exit transition scenes, including but not limited to half-sky compositions, strong contrasts between light and dark, and tunnel arch structures. The image is divided into two parts: the upper part is the sky, and the lower part is the dark interior of the tunnel. A light-dark-light gradient is formed at the entrance, and the arc-shaped outline of the tunnel entrance is usually visible. The neural network simultaneously learns to detect the sky and areas with strong brightness gradient changes, and identifies the arc-shaped edge structure.

[0059] The scene classification neural network is trained end-to-end and automatically learns and integrates key visual features, including: the proportion of sky area and illumination distribution, the regularity and density of artificial structure lines, the presence or absence of dynamic semantic targets, the color saturation and contrast of the overall image, the richness and repetition of texture, and the intensity of light and dark transitions. The combination of the above features uniquely represents the visual pattern of each type of mining scene.

[0060] Then, different prior weight coefficients are set according to different scenarios and tabulated, and a 4×4 diagonal weight matrix is ​​preset for each scenario. There are four types of sensors.

[0061] For example: In underground mine tunnels with poor lighting, the visual weight is reduced; in structurally stable areas, the laser weight is increased. In dusty scenes with high point cloud noise, the laser weight is significantly reduced, while the IMU weight is relatively increased. In open scenes above ground, the GPS weight is increased, while the visual weight remains unchanged. A one-to-one prior weight table is created by mapping the scene to the prior weight coefficients.

[0062] To reduce computational load, the proposed image feature extraction neural network takes keyframe data as input, extracts RGB images, and then performs scene recognition. To meet embedded deployment requirements, a lightweight image feature recognition neural network is used for training. From the image sequence generated by the visual inertial odometry module, keyframes are selected based on a pre-defined strategy, and the corresponding original RGB images are extracted.

[0063] The acquired raw images are uniformly scaled to a fixed size of 224 pixels × 224 pixels using a bilinear interpolation algorithm, and each pixel channel is normalized. Normalization involves converting pixel values ​​from an integer range of [0, 255] to a floating-point range of [0, 1], and then standardizing them using statistics from the ImageNet dataset, as explained by the following formula.

[0064] (1)

[0065] in, The original image data after normalization to [0, 1];

[0066] This is the final standardized output data;

[0067] The mean, The standard deviation is denoted as .

[0068] For the neural network architecture of image feature recognition, the lightweight MobileNetV3-Small is adopted as the backbone network. This network achieves excellent feature representation capabilities with an extremely low number of parameters by introducing neural network architecture search and a lightweight attention module, making it particularly suitable for edge computing devices.

[0069] Instead of the native classification layer of MobileNetV3 for 1000 classes of ImageNet data, a global average pooling layer is added after the feature map output by the backbone network. This layer averages all spatial locations (H×W) of each feature channel, thereby converting a feature map of arbitrary size into a feature vector of length 576.

[0070] Following the global average pooling layer, a miniature classification network consisting of three fully connected layers is designed. The first layer has a 576-dimensional input and a 128-dimensional output, activated by ReLU and connected to a Dropout layer. The second layer has a 128-dimensional input and a 64-dimensional output, also activated by ReLU. The third layer has a 64-dimensional input and a 6-dimensional output, corresponding to six types of mining scenes. The Softmax function is used to output the probability distribution of the six scene types, where each element in the vector represents the confidence level of the input image belonging to the corresponding scene category.

[0071] For six predefined scenarios, collect no fewer than 10,000 labeled images in real mining environments or through high-fidelity simulations. To ensure model robustness, the images must cover variations in different time periods (day / night), different weather conditions (sunny / rainy / dusty), and different perspectives.

[0072] To expand the dataset and improve the model's generalization ability, random data augmentation strategies were implemented online during training, including random horizontal flipping, random brightness and contrast adjustments, slight color jitter, and random affine transformations. The labeled dataset was randomly divided into training, validation, and test sets in a 7:2:1 ratio to ensure a balanced proportion of each class.

[0073] The cross-entropy loss function is used, which is a standard method for measuring the difference between the model's output probability distribution and the true label distribution, and is very suitable for multi-class classification tasks.

[0074] The AdamW optimizer was used. The initial learning rate was set to 3e-4, and a cosine annealing scheduling strategy was used to dynamically adjust the learning rate. The batch size was set to 32 or 64 based on the GPU memory. The final trained model was fairly evaluated using a reserved test set, and the overall classification accuracy, precision, and recall for each class were calculated. A confusion matrix was also plotted to analyze which classes the model was prone to confusion with.

[0075] Finally, during the mapping process, the keyframe images of SLAM are extracted, scaled, and input into the trained neural network. The category with the highest probability is taken as the current scene, and the prior weights are retrieved from the prior weight table based on the current scene.

[0076] Scene recognition is performed based on keyframe RGB image information. When selecting keyframes in SLAM, the RGB image of that frame is input into a pre-trained lightweight network to obtain a scene classification probability vector. The category with the highest probability is selected as the current scene, and the corresponding prior weight diagonal matrix is ​​indexed from a predefined weight table based on the identified scene category. .

[0077] Step 104: Normalize the sensor degradation index of the mapping data to obtain the degradation index vector of the mapping data, and transform the degradation index vector into a weight fine-tuning matrix through a preset multilayer perceptron.

[0078] For example, this application normalizes the mapping data using sensor degradation indices to obtain a degradation index vector of the mapping data, and then transforms the degradation index vector into a weight fine-tuning matrix through a preset multilayer perceptron, thereby achieving lightweight and fast calculation of the weight fine-tuning matrix to ensure real-time operation in the main loop of the SLAM system.

[0079] Specifically, the structure of the multilayer perceptron includes: an input layer, a first fully connected layer, a second fully connected layer, and an output layer. Using a pre-defined multilayer perceptron, the degradation index vector is transformed into a weight fine-tuning matrix, including: inputting the degradation index vector into the first fully connected layer through the input layer, and activating and normalizing it using the ReLU function to obtain a 32-dimensional vector; inputting the 32-dimensional vector into the second fully connected layer, and activating the vector output by the second fully connected layer using the Sigmoid function through the output layer to obtain an initial weight fine-tuning matrix; wherein the initial weight fine-tuning matrix consists of fine-tuning coefficients; based on the initial weight fine-tuning matrix, a loss function for weight fine-tuning is constructed, and the fine-tuning coefficients are constrained by the loss function to obtain the weight fine-tuning matrix.

[0080] Figure 5 This is a schematic diagram of a weight fine-tuning matrix generation based on a multilayer perceptron, provided as an embodiment of this application.

[0081] In one embodiment, this application evaluates the degradation level of each sensor in real time at the keyframe level and generates a fine-tuning matrix using a lightweight multilayer perceptron (MLP). Used for prior weights A second revision was made.

[0082] and The dimensions are consistent, all being 4×4 diagonal matrices. The diagonal element q takes values ​​between 0 and 1. The smaller the value, the lower the reliability of the sensor in the current key frame period. The core function of the neural network is to map the real-time degradation evaluation vector of the sensor into fine-tuning coefficients for the weights of each sensor.

[0083] First, for the acquisition and processing of sensor degradation data, key frame data collected by the sensor is extracted, the sliding window method is used for data collection, and normalization is performed.

[0084] The indicators of point cloud reflection intensity, point cloud quantity, RGB image illumination intensity, RGB image feature point quantity, smoke and dust concentration, IMU vibration intensity, and GPS signal strength are calculated, extracted, and normalized again.

[0085] Based on the current keyframe timestamp Centered on the data point, a sliding window is constructed by taking raw data from 0.5 seconds before and after the data point. The total length is 1 second, covering 10 frames of laser data, 20 frames of image data, 200 frames of IMU data, and 5 frames of GPS data. Outlier removal and low-pass filtering are applied to the data within the window before calculating the following seven-dimensional degradation index vector. All indicators are normalized within the range [0,1].

[0086] Seven degradation metrics are extracted and calculated from the data in the current keyframe and the time windows before and after it, and explained by the following formula.

[0087] First, the mean reflectance index of point clouds. Both excessively low (water mist absorption) and excessively high (specular reflection) values ​​are considered degradation. The calculation is performed within the sliding window. Reflection intensity of all laser frames within arithmetic mean ;

[0088] (2)

[0089] The mean index of point cloud reflectance intensity was calculated through normalization. .

[0090] (3)

[0091] Secondly, the effective point cloud quantity index If the value is below a preset threshold (e.g., 5000 points per frame), the normalized value decreases, and the statistics are updated. Average number of valid points per frame Calculate the effective point cloud quantity index .

[0092] (4)

[0093] in, for Average number of valid points per frame; if , Set it directly to 0.5.

[0094] Third, the average illuminance index of RGB images By All images were converted to HSV color space, and the average chroma value of the V channel was taken. Calculate the average light intensity index .

[0095] (5)

[0096] in, This represents the average chromaticity value of the V channel.

[0097] Fourth, the number of feature points in RGB images. The data is compared with the historical average, and ORB features are extracted for each frame to calculate the average number of feature points within the window. Calculate the number of feature points in an RGB image. .

[0098] (6)

[0099] in, The average number of feature points within the window, if , Set it to 0.3.

[0100] Fifth, dust concentration estimation indicators By jointly estimating the image chroma (leaning towards yellowish-brown) and the point cloud scattering, the ratio of the mean values ​​of the R and G channels is calculated for each image channel. .

[0101] (7)

[0102] in, For R channel chroma; For G channel chroma; if It was considered to be yellowish-brown, and was recorded as... And calculate the standard deviation of point cloud reflection intensity. .

[0103] (8)

[0104] in, Reflection intensity, 5 is considered to have severe scattering, denoted as .

[0105] Overall score for ;in, This represents the ratio of the mean values ​​of the R channel and the G channel when the color is yellowish-brown. The standard deviation of point cloud reflectance intensity under severe scattering is given, and the dust concentration estimation index is calculated. .

[0106] Sixth, IMU data vibration intensity index Calculate the variance of the accelerometer readings. Internal acceleration modulus Difference .

[0107] (9)

[0108] in, for Internal acceleration modulus.

[0109] Calculate the variance of the difference series .

[0110] (10)

[0111] in, The difference between the magnitude sequences of accelerometer readings within the window; if They believed it was a violent jolt.

[0112] The vibration intensity index of the IMU data is .

[0113] Seventh, GPS signal strength / accuracy factor index Through calculation The proportion of internal RTK fixed solutions .

[0114] (11)

[0115] in, It is a sliding window; for The proportion of internal RTK fixed solutions.

[0116] Calculate the average level precision factor .

[0117] (12)

[0118] in, This is the horizontal precision factor.

[0119] Calculate the GPS signal strength / accuracy factor index .

[0120] (13)

[0121] in, for The proportion of internal RTK fixed solutions; The average level of precision factor, when there is no fixed solution. .

[0122] Then, to Perform a median filter to eliminate abrupt changes. If a sensor is in If there is no data inside, the corresponding dimension is forcibly set to 0, and a 1-bit valid flag is added to the subsequent input of the multilayer perceptron (MLP).

[0123] By designing a simple multilayer perceptron, the input layer is a 7-dimensional normalized vector. This is followed by two hidden layers, using the ReLU activation function.

[0124] The output layer is a 4-dimensional vector, mapped to (0,1) using the Sigmoid function, representing the weights of the prior layers. The fine-tuning ratio is adjusted. The network output is reconstructed as a 4x4 diagonal fine-tuning matrix. The lightweight neural network has the following structure: the input layer has a 7-dimensional data format. After the input layer, a first fully connected layer is connected, activated and normalized using the ReLU function; the output of this fully connected layer is 32-dimensional. After the first fully connected layer, a second fully connected layer is connected, activated and normalized using the ReLU function; the output of this second fully connected layer is also 32-dimensional. Finally, an output layer is connected, activated using the Sigmoid function, which transforms the 32-dimensional data into 4-dimensional data and outputs it. .

[0125] The basic LVI-SAM was run on the mining SLAM dataset. When the deviation between the system's estimated trajectory and the true value suddenly increased, the seven-dimensional vector of the sensor at that time was recorded as a degraded sample. When the system was stable, it was recorded as a good sample. The mean squared error (MSE) was used as the loss function for supervised training.

[0126] A benchmark LVI-SAM system is run on a mine SLAM dataset containing ground truth trajectories to automatically generate sample pairs (X, Y). For each keyframe time during the run... Simultaneously record and calculate the sensor's 7-dimensional degradation evaluation vector at that moment.

[0127] By automatically generating 7-dimensional ideal fine-tuning coefficient labels corresponding to the sample input, the calculation... Near a time window Within, the moving average of the absolute pose error between the SLAM-estimated trajectory and the high-precision ground truth trajectory.

[0128] If the average APE value at the current time suddenly increases by more than the threshold compared to the previous time window. If the sensor fusion degradation occurs at that moment, it is determined that sensor fusion degradation has occurred. At this time, the system automatically analyzes the raw data and intermediate results of each sensor to identify the main subset of sensors that caused the degradation.

[0129] In label Y, the coefficients corresponding to these degraded sensors are set to 0.1, and the rest are set to 1.0; if the average APE remains stable and below the threshold, label Y is set to a vector of all 1.0. To alleviate the imbalance between positive and negative samples, "good" samples are randomly downsampled to ensure that the ratio of "degraded" to "good" samples in the dataset is approximately 1:2. Mean squared error loss is used, and for a batch of samples, the loss function is interpreted using the following formula.

[0130] (14)

[0131] in, The four-dimensional fine-tuning coefficients are predicted by the weighted fine-tuning network (MLP) based on the input seven-dimensional vector.

[0132] Automatically generated 4D ideal labels;

[0133] The number of samples in a training batch.

[0134] By directly constraining the fine-tuning coefficients of the network output to approximate the ideal value, the Adam optimizer is used, with an initial learning rate set to... .

[0135] Dropout is used for regularization after fully connected layers to prevent overfitting. An early stopping strategy is adopted, which terminates training and rolls back to the optimal model weights when the validation set loss no longer decreases after 10 consecutive complete training cycles.

[0136] Finally, the seven-dimensional vector of the current keyframe is input into the trained multilayer perceptron (MLP) to obtain four fine-tuning coefficients, which form a diagonal matrix. ;in, The diagonal The smaller the value of the nth element, the more likely it is to be the nth element. The more severe the degradation of a sensor is at the current moment.

[0137] Step 105: Calculate the weight coefficient matrix based on the prior weight matrix and the weight fine-tuning matrix.

[0138] Specifically, the weight coefficient matrix is ​​calculated based on the prior weight matrix and the weight fine-tuning matrix, including: multiplying the prior weight matrix and the weight fine-tuning matrix element by element to obtain the initial weight coefficient matrix; applying an exponential moving average filter to the initial weight coefficient matrix to determine the smoothed weights of the initial weight coefficient matrix at the corresponding time; and setting lower bound constraints on the diagonal elements of the initial weight coefficient matrix based on the smoothed weights to obtain the weight coefficient matrix.

[0139] In one embodiment, the final weight coefficient matrix S is calculated based on the prior matrix and the weight fine-tuning matrix, and then fused using the Hadamard product of element-wise multiplication, as explained by the following formula.

[0140] (15)

[0141] in, This is the prior weight coefficient matrix; This is the weight coefficient fine-tuning matrix; the weight coefficient matrix simultaneously reflects prior knowledge of the scene type and the real-time sensor degradation state.

[0142] Then, to avoid system oscillations caused by sudden changes in weights, an exponential moving average filter is applied to S, and the weights after the exponential moving average filter are calculated and explained by the following formula.

[0143] (16)

[0144] in, It is a smoothing factor; The weight at time t; The weights at time t-1.

[0145] Finally, to prevent any sensor from completely failing, Set a lower limit for each element on the diagonal. Even if the sensor is severely degraded, it still retains its minimum weight in the fusion process, preventing the factor graph optimization from diverging due to a lack of constraints.

[0146] Step 106: Fuse the weight coefficient matrix with the information matrix of each sensor factor corresponding to the mapping data to determine the optimization objective function, and obtain the factor map by factor map optimization through LVI-SAM based on the optimization objective function.

[0147] Specifically, the weight coefficient matrix is ​​fused with the information matrix of each sensor factor corresponding to the mapping data to determine the optimization objective function. This includes: extracting the four diagonal elements from the weight coefficient matrix and adaptively adjusting the weights of the information matrix based on the diagonal elements to obtain the adjusted information matrix; and constructing the optimization objective function of the adjusted information matrix based on the adjusted information matrix and the residual vectors of each sensor factor.

[0148] In one embodiment, in the factor graph optimization of LVI-SAM, the residual terms of each sensor factor (visual reprojection factor, IMU pre-integration factor, laser matching factor, GPS factor) are weighted by the inverse of its covariance matrix (information matrix) in the objective function. The core improvement of this invention lies in incorporating the weight matrix S into the information matrix of each factor to calculate the objective function. The following formula will be used to explain this.

[0149] (17)

[0150] in, These represent the residual vectors for the four factors: visual reprojection, laser matching, IMU pre-integration, and GPS position, respectively.

[0151] , , , These represent the four diagonal elements extracted from the final weight matrix S, which serve as dynamic adaptive weight coefficients.

[0152] , , , These represent the original information matrices of each sensor factor, characterizing the inherent, static measurement uncertainty of each sensor.

[0153] Furthermore, for a given sensor factor, the adjusted information matrix... for ;in, It is the diagonal element in the weight matrix S corresponding to the sensor; This is the original information matrix.

[0154] When a sensor degrades in a specific scenario, the weight of its corresponding factor decreases, and the optimizer will place more trust in other sensors, thus achieving adaptive robust fusion. Even with the inclusion of adaptive weights, the solution process for factor graph optimization remains unchanged, still employing nonlinear optimization algorithms such as the Levenberg-Marquardt algorithm.

[0155] Step 107: Based on the factor map and pose transformation constraints, determine the robot's pose estimation sequence and the corresponding map in the mapping environment through point cloud processing.

[0156] Specifically, based on the factor map and pose transformation constraints, point cloud processing is used to determine the robot's pose estimation sequence and the map corresponding to the mapping environment. This includes: optimizing the robot's pose in the mapping environment based on pose transformation constraints to determine the pose estimation sequence; obtaining a corrected point cloud through motion compensation based on the factor map and pose estimation sequence; transforming each frame of the corrected point cloud from the sensor coordinate system to the global world coordinate system to obtain the local world coordinates of the corrected point cloud; dividing the global map of the mapping environment into voxel grids and selecting the centroid of the voxel grid as the representative point of the voxel grid to determine the map corresponding to the mapping environment.

[0157] In one embodiment, for each time t, the original laser point cloud is acquired. First, using the optimized pose Motion compensation is performed using the estimated high-frequency IMU data to eliminate point cloud distortion caused by the robot's own motion within a single frame scan, resulting in the corrected point cloud Pt.

[0158] Subsequently, each frame of point cloud is transformed from its sensor coordinate system to the global world coordinate system. The following formula will be used to explain this.

[0159] (18)

[0160] in, The current frame's laser point cloud after motion compensation; This is the fixed extrinsic transformation matrix from the lidar coordinate system to the robot body coordinate system; This is the transformation matrix from the mining machinery's body coordinate system to the global world coordinate system.

[0161] By maintaining a local point cloud map within a sliding window, typically containing the point clouds corresponding to the most recent N keyframes, this local map is used for laser odometry scan matching and is updated in real time.

[0162] For global map construction and redundancy removal, the point clouds of all keyframes are transformed and added to the global map. To control the map size and improve the efficiency of subsequent localization queries, voxel grid downsampling filtering is applied to the global map.

[0163] Specifically, the three-dimensional space is divided into a voxel mesh with side length r. For all points falling within the same voxel, its centroid (or a weighted average of reflection intensity) is taken as the representative point of that voxel. This method effectively eliminates a large number of redundant points in overlapping areas of the point cloud while preserving the scene structure.

[0164] Finally, the generated globally consistent 3D point cloud map is stored in standard formats such as .pcd or .ply. In long-term operation or applications where mapping and localization are separated, the map can be managed in an octree structure to support efficient dynamic updates, semantic information addition, and collision detection.

[0165] This high-precision map can be directly provided to autonomous mining trucks for point cloud registration and positioning, enabling precise navigation and operations along planned paths. Simultaneously, the map serves as the core data foundation for digital mine management systems, and can be used for reserve estimation, route planning, and safety monitoring.

[0166] The above are embodiments of the method proposed in this application. Based on the same inventive concept, embodiments of this application also provide a scene-priority-based SLAM device, the structure of which is as follows: Figure 6 As shown.

[0167] Figure 6 This is a schematic diagram of the internal structure of a scene-prior-based SLAM device provided in an embodiment of this application. Figure 6 As shown, the device includes:

[0168] At least one processor 601;

[0169] And a memory 602 that is communicatively connected to at least one processor;

[0170] The memory 602 stores instructions executable by at least one processor, which are executed by at least one processor 601 to enable at least one processor 601 to:

[0171] The system acquires environmental detection data collected by pre-set multimodal sensors in the mapping environment and preprocesses the data to determine the mapping data for the mapping environment. It extracts feature parameters from the mapping data and performs feature matching and loop closure detection on these parameters to determine pose transformation constraints. It identifies the scene category of the mapping environment and, based on the scene category, queries the corresponding prior weight coefficients of the mapping data to construct a prior weight matrix. It normalizes the sensor degradation indices of the mapping data to obtain a degradation index vector, and then uses a pre-set multilayer perceptron to transform the degradation index vector into a weight fine-tuning matrix. Based on the prior weight matrix and the weight fine-tuning matrix, it calculates the weight coefficient matrix. It fuses the weight coefficient matrix with the information matrices of each sensor factor corresponding to the mapping data to determine the optimization objective function. Based on the optimization objective function, it uses LVI-SAM factor graph optimization to obtain a factor graph. Based on the factor graph and pose transformation constraints, it uses point cloud processing to determine the robot's pose estimation sequence and the map corresponding to the mapping environment.

[0172] Some embodiments of this application provide corresponding to Figure 1 A non-volatile computer storage medium for scenario-prior SLAM, storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0173] The system acquires environmental detection data collected by pre-set multimodal sensors in the mapping environment and preprocesses the data to determine the mapping data for the mapping environment. It extracts feature parameters from the mapping data and performs feature matching and loop closure detection on these parameters to determine pose transformation constraints. It identifies the scene category of the mapping environment and, based on the scene category, queries the corresponding prior weight coefficients of the mapping data to construct a prior weight matrix. It normalizes the sensor degradation indices of the mapping data to obtain a degradation index vector, and then uses a pre-set multilayer perceptron to transform the degradation index vector into a weight fine-tuning matrix. Based on the prior weight matrix and the weight fine-tuning matrix, it calculates the weight coefficient matrix. It fuses the weight coefficient matrix with the information matrices of each sensor factor corresponding to the mapping data to determine the optimization objective function. Based on the optimization objective function, it uses LVI-SAM factor graph optimization to obtain a factor graph. Based on the factor graph and pose transformation constraints, it uses point cloud processing to determine the robot's pose estimation sequence and the map corresponding to the mapping environment.

[0174] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for IoT devices and media are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0175] The systems, media, and methods provided in this application are one-to-one correspondences. Therefore, the systems and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the systems and media will not be repeated here.

[0176] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0177] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0178] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0179] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0180] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0181] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0182] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0183] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0184] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A scene-prior-based SLAM method, characterized in that, The method includes: The environmental detection data collected by the preset multimodal sensors of the mapping environment is acquired, and the environmental detection data is preprocessed to determine the mapping data of the mapping environment; The feature parameters of the mapping data are extracted, and feature matching and loop closure detection are performed on the feature parameters to determine the pose transformation constraints. Identify the scene category of the mapping environment, and based on the scene category, query the prior weight coefficients corresponding to the mapping data to construct a prior weight matrix; The mapping data is normalized by sensor degradation index to obtain the degradation index vector of the mapping data, and the degradation index vector is transformed into a weight fine-tuning matrix through a preset multilayer perceptron. Calculate the weight coefficient matrix based on the prior weight matrix and the weight fine-tuning matrix; The weight coefficient matrix is ​​fused with the information matrix of each sensor factor corresponding to the mapping data to determine the optimization objective function, and the factor map is obtained by factor map optimization through LVI-SAM based on the optimization objective function. Based on the factor map and the pose transformation constraints, point cloud processing is used to determine the robot's pose estimation sequence in the mapping environment and the map corresponding to the mapping environment.

2. The SLAM method based on scene priors according to claim 1, characterized in that, The environmental monitoring data is preprocessed to determine the mapping data for the mapping environment, specifically including: A time series of the environmental monitoring data is established, and linear interpolation is performed on the environmental monitoring data based on the timestamps of the time series to obtain a multimodal observation dataset; wherein, the environmental monitoring data includes: point cloud data, image data, inertial data, and absolute position data; The coordinate system of the multimodal observation dataset is transformed to determine the mapping data for the mapping environment.

3. The SLAM method based on scene priors according to claim 1, characterized in that, Feature matching and loop closure detection are performed on the feature parameters respectively to determine the pose transformation constraints, specifically including: The distortion-free current point cloud features in the aforementioned feature parameters are matched with the local map to calculate the excitation inertial odometry pose. Based on the inertial odometry pose, the current point cloud in the feature parameters is matched with the local point cloud in the local map, and the root mean square error of the current point cloud and the local point cloud is calculated to obtain the pose transformation constraint.

4. The SLAM method based on scene priors according to claim 1, characterized in that, Based on the scene category, the prior weight coefficients corresponding to the mapping data are queried to construct a prior weight matrix, specifically including: Identify the keyframes corresponding to the mapping data, and determine the current scene based on the keyframes and scene classification probability selection; Based on the scenario category, set the prior weight coefficients corresponding to the mapping data to obtain the prior weight table; For the scene category corresponding to the current scene, index the corresponding prior weight matrix in the prior weight table.

5. The SLAM method based on scene priors according to claim 1, characterized in that, The structure of the multilayer perceptron includes: an input layer, a first fully connected layer, a second fully connected layer, and an output layer; The degradation index vector is transformed into a weight fine-tuning matrix using a pre-defined multilayer perceptron, specifically including: The degradation index vector is input into the first fully connected layer through the input layer, and then activated and normalized by the ReLU function to obtain a 32-dimensional vector. The 32-dimensional vector is input into the second fully connected layer, and then the vector output by the second fully connected layer is activated by the Sigmoid function through the output layer to obtain an initial weight fine-tuning matrix; wherein, the initial weight fine-tuning matrix is ​​composed of fine-tuning coefficients; Based on the initial weight fine-tuning matrix, a loss function for weight fine-tuning is constructed, and the fine-tuning coefficients are constrained by the loss function to obtain the weight fine-tuning matrix.

6. The SLAM method based on scene priors according to claim 1, characterized in that, Based on the prior weight matrix and the weight fine-tuning matrix, the weight coefficient matrix is ​​calculated, specifically including: The prior weight matrix and the weight fine-tuning matrix are multiplied element by element to obtain the initial weight coefficient matrix; The initial weight coefficient matrix is ​​subjected to an exponential moving average filter to determine the smoothed weights of the initial weight coefficient matrix at the corresponding time. Based on the smoothing weights, lower bound constraints are set for the diagonal elements in the initial weight coefficient matrix to obtain the weight coefficient matrix.

7. The SLAM method based on scene priors according to claim 1, characterized in that, The weighting coefficient matrix is ​​fused with the information matrix of each sensor factor corresponding to the mapping data to determine the optimization objective function, specifically including: Extract the four diagonal elements from the weight coefficient matrix, and based on the diagonal elements, perform adaptive weight adjustment on the information matrix to obtain the adjusted information matrix; Based on the adjustment information matrix and the residual vectors of each sensor factor, an optimization objective function for the adjustment information matrix is ​​constructed.

8. The SLAM method based on scene priors according to claim 1, characterized in that, Based on the factor map and the pose transformation constraints, point cloud processing is used to determine the robot's pose estimation sequence and the map corresponding to the mapping environment, specifically including: Based on the pose transformation constraints, the pose of the robot in the mapping environment is constrained and optimized to determine the pose estimation sequence; Based on the factor map and the pose estimation sequence, a corrected point cloud is obtained through motion compensation; The calibration point cloud of each frame is transformed from the sensor coordinate system to the global world coordinate system to obtain the local world coordinates of the calibration point cloud. The global map of the mapping environment is divided into voxel grids, and the centroid of the voxel grid is selected as the representative point of the voxel grid to determine the map corresponding to the mapping environment.

9. A scene-prior-based SLAM device, characterized in that, The device includes: At least one processor; And, a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to implement a scene-priority-based SLAM method as described in claims 1-8.

10. A non-volatile computer storage medium for scene-prior-based SLAM, storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed, they implement a scene-prior-based SLAM method as described in claims 1-8.

Images