Monocular neural rendering slam method and system in complex environment

Abstract

The application relates to a monocular neural rendering SLAM method and system in a complex environment. The method obtains an initial image sequence in a complex environment and carries out pretreatment, then inputs a pre-trained semantic segmentation model to obtain a multi-dynamic object mask; point features and line features are extracted from the pretreated image sequence, and the multi-dynamic object mask is used to eliminate corresponding dynamic features to obtain static features; the static features are subjected to feature matching, a conversion matrix is solved, and an estimated pose is obtained by optimizing a point-line joint re-projection error function; sparse super primitives are obtained, a region to be densified is identified according to point re-projection errors, and a new super primitive set is generated in combination with unmatched feature points; a rendering image is obtained, the rendering image is subjected to multi-scale Gaussian pyramid supervision training, and a final global map is obtained. Compared with the prior art, the application has the advantages of improving the pose estimation accuracy and map representation capability of SLAM in a complex environment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot vision positioning and mapping, and in particular to a monocular neural rendering SLAM method and system in a complex environment. BACKGROUND

[0002] In recent years, mobile robots are increasingly widely used in life. Traditional SLAM systems based on lasers are very mature in indoor positioning and mapping. However, in special application scenarios, dynamic objects interfere or long corridors and other degenerative environments also face great challenges, and the cost of laser sensors is expensive and the detection range is limited. Therefore, more and more SLAM solutions tend to rely on vision to build systems.

[0003] Traditional point feature-based visual simultaneous localization and mapping (SLAM) systems also face great challenges in dynamic environments. Due to the presence of dynamic objects, they often have problems of tracking failure or significant accuracy decline. Many methods use semantic segmentation networks to identify dynamic objects in images and delete dynamic feature points in subsequent tracking processes, but this approach is difficult to promote due to the speed and accuracy of the segmentation network. There are also related methods that remove dynamic points based on the error size of each point, but this method relies on a relatively accurate initial pose and must introduce a new sensor to provide initial value estimation, greatly increasing the complexity of the system. Considering a special indoor scene, when it contains high dynamic moving objects, such as crowds. And there are a lot of weak texture areas in this scene, in this case, many methods based on dynamic prior filtering are difficult to operate normally, because after removing the features of dynamic objects, the remaining static background also cannot extract enough feature points for matching and initializing the map.

[0004] Therefore, how to improve the pose estimation accuracy and map representation ability of SLAM in complex environments, such as complex environments containing dynamic and weak texture areas, has become a problem to be solved in the field. SUMMARY

[0005] The purpose of the present application is to overcome the above-mentioned prior art in complex environments containing dynamic and weak texture areas, inaccurate pose estimation and insufficient map representation, and to provide a monocular neural rendering SLAM method and system in a complex environment.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] According to a first aspect of the present application, a monocular neural rendering SLAM method in a complex environment is provided, comprising the following steps: a data acquisition and preprocessing step: acquiring an initial image sequence in a complex environment and preprocessing; a dynamic object mask acquisition step: inputting the preprocessed image sequence into a pre-trained semantic segmentation model to obtain a plurality of dynamic object masks; a feature extraction and dynamic elimination step: extracting point features and line features from the preprocessed image sequence, and eliminating corresponding dynamic features using the plurality of dynamic object masks to obtain static features; a pose estimation step: performing feature matching on the static features based on the point features and the line features, solving a transformation matrix, and obtaining an estimated pose by optimizing a point-line joint reprojection error function; a geometric mapping step: obtaining sparse super primitives, identifying a region to be densified according to a point reprojection error, and generating a new super primitive set in combination with unmatched feature points; a scene mapping step: obtaining a rendered image, performing multi-scale Gaussian pyramid supervised training on the rendered image, and obtaining a final global map.

[0008] As a preferred technical solution, the method further comprises a loop detection step: performing loop detection using a preset bag-of-words model, and when a loop is detected, adjusting the current frame and the super primitives using a similarity change to correct the pose and correct cumulative errors.

[0009] As a preferred technical solution, the specific implementation process of the dynamic object mask acquisition step comprises: sequentially tracking and segmenting a single dynamic object to obtain a mask corresponding to each dynamic object; and fusing all masks on each frame to obtain a mask of all dynamic objects in each frame image.

[0010] As a preferred technical solution, in the feature extraction and dynamic elimination step, point features are extracted using an ORB detection method, and line features are extracted using an LSD line detection method.

[0011] As a preferred technical solution, in the pose estimation step, triangulation is performed through feature matching to complete initialization of a super primitive map, key frames are selected according to current frame information, and a factor graph is constructed by optimizing co-visible point features, line features, and key frames, the factor graph being used for optimizing a tracking process.

[0012] As a preferred technical solution, the current frame information includes a matching number of point features and line features and a feature tracking situation.

[0013] As a preferred technical solution, the point-line joint reprojection error function is expressed as:

[0014]

[0015] In the formula, the kth key frame pose is T cw,k= [R, t], the observed i-th spatial point p I The reprojection error is Ep k,i , the reprojection error of the jth spatial line observed in the kth key frame is El k,i , and Represent different observation covariances, ρ p and ρ l Represent different robust kernel functions respectively.

[0016] As a preferred technical solution, the scene mapping step specifically includes: performing Gaussian smoothing and downsampling on the input image to obtain a rendered image; constructing a pyramid to perform supervision at different levels of image resolution; constructing photometric loss and SSIM loss to train and optimize the representation of the rendered image.

[0017] As a preferred technical solution, the luminosity loss and SSIM loss are expressed as:

[0018]

[0019] Where, represents the photometric loss and SSIM loss, λ is the preset weight parameter, I r is the rendered image, I gt is the true value of the input image.

[0020] According to the second aspect of the present invention, a monocular neural rendering SLAM system for a complex environment is provided, comprising: a data acquisition and preprocessing module for acquiring and preprocessing an initial image sequence in a complex environment; a dynamic object mask acquisition module for inputting the preprocessed image sequence into a pretrained semantic segmentation model to obtain a multi-dynamic object mask; a feature extraction and dynamic culling module for extracting point features and line features from the preprocessed image sequence, and using the multi-dynamic object mask to eliminate the corresponding dynamic features to obtain static features; a pose estimation module for performing feature matching on the static features based on the point features and the line features, solving the transformation matrix, and obtaining an estimated pose by optimizing the reprojection error function of the point-line union; a geometric mapping module for acquiring sparse super-primitives, identifying the area to be densified according to the reprojection error of the points, and generating a new super-primitive set in combination with unmatched feature points; and a scene mapping module for acquiring a rendered image, performing multi-scale Gaussian pyramid supervised training on the rendered image, and obtaining a final global map.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1、The application utilizes a semantic segmentation model to obtain a multi-dynamic object mask, and utilizes the mask to eliminate the interference of dynamic objects, can realize accurate pose estimation in a dynamic environment, and at the same time, after removing dynamic features, performs feature matching on static features, solves a transformation matrix, and obtains an estimated pose by optimizing a point-line joint re-projection error function, can realize accurate pose estimation under weak texture conditions by introducing point features and line features to minimize the error function, so that the application can sufficiently improve the accuracy of pose estimation in a complex environment containing dynamic and weak texture areas;

[0023] 2、The application introduces neural rendering for a SLAM method after obtaining a sparse super-primitive, i.e., a point cloud set, obtains a rendered image, performs multi-scale Gaussian pyramid supervised training on the image, and obtains a final global map, can realize accurate estimation of camera pose and enhance scene representation under multiple views, and effectively solves the SLAM problem under a dynamic and weak texture scene;

[0024] 3、The application can significantly improve the scalability of a SLAM system, and algorithm verification is performed on multiple data sets, and compared with existing algorithms, the application has obvious improvement in absolute trajectory error (ATE) and peak signal-to-noise ratio (PSNR) and other indicators, proving the effectiveness and practicality of the method. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The application provides a method architecture schematic diagram;

[0026] Figure 2 The application provides a method implementation flow schematic diagram;

[0027] Figure 3 The application provides a multi-dynamic object segmentation result diagram of the method;

[0028] Figure 4 The application provides a SLAM method mapping comparison result based on a PSNR index. DETAILED DESCRIPTION

[0029] For the special indoor scene containing high dynamic moving objects and a large number of weak texture areas, the existing method based on dynamic prior filtering has certain limitations in positioning and map representation, which can easily reduce the accuracy of visual simultaneous localization and mapping (SLAM). In view of this, the present application considers the particularity of the scene, uses a semantic segmentation model to obtain a multi-dynamic object mask to remove the interference of dynamic objects, and introduces a neural rendering technique to improve the SLAM method. Among them, neural rendering, as an emerging technology, combines the ideas of classical computer graphics and machine learning, and reconstructs the scene representation from observations through a differentiable rendering loss, thereby synthesizing photo-realistic images and videos. Due to its fast rendering and high-quality image representation, it can be introduced into SLAM to improve the shortcomings of the existing SLAM mapping method and provide a new type of map representation.

[0030] The present application will be described in detail below in conjunction with the accompanying drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation manner and specific operation process, but the protection scope of the present application is not limited to the following embodiments.

[0031] Embodiment

[0032] As Figure 1 shown, the present embodiment proposes a monocular neural rendering SLAM method in a complex environment. On the one hand, the method uses an open semantic segmentation model to accurately obtain dynamic object priors in multiple scenes, i.e. multi-dynamic segmentation masks; on the other hand, in the traditional SLAM process based on point features, line feature constraints are added to improve the problem of inaccurate pose estimation caused by insufficient number of feature points in weak texture areas; on this basis, a progressive rendering method is used to achieve a photo-realistic mapping result, and scene information under a new perspective can be obtained. Figure 2 One of the implementation processes of the foregoing method is shown, which is as follows:

[0033] Step S1, data acquisition and preprocessing: acquiring an initial image sequence in a complex environment and performing preprocessing.

[0034] Specifically, in actual application, a preset sensor can be used to obtain an initial image sequence, but due to the structure of the sensor itself and the particularity of the scene, the obtained pictures may have problems such as noise, distortion or illumination change, and cannot be directly used for tracking and mapping of the SLAM method and system. Therefore, advanced image processing techniques can be used to preprocess the picture data to improve the effectiveness of the data. Preprocessing includes steps such as denoising, distortion correction, and illumination adjustment, so that the pictures are more suitable for subsequent reconstruction and analysis work.

[0035] Step S2, dynamic object mask acquisition: input the preprocessed image sequence into the pre-trained semantic segmentation model to obtain multiple dynamic object masks.

[0036] Specifically, in a complex scene containing multiple dynamic objects, an open semantic segmentation model SAM2 is used to segment different categories of objects in parallel, and then multiple object masks are fused to obtain accurate multiple dynamic object masks in consecutive frames. The SAM2 segmentation model is used for semantic segmentation of the current frame to obtain the category label id∈{0,1,…N} of each pixel.

[0037] Most existing visual SLAM systems are based on the assumption of static background and cannot be directly applied to dynamic scenes. To overcome this problem, the simplest way is to remove redundant features through dynamic object priori to prevent calculation errors in the matching process. Specifically, a pre-trained visual segmentation model SAM2 is used to provide rough pixel positions of people or other dynamic objects to realize the perception of dynamic objects and obtain the corresponding mask. This method can segment dynamic objects in near real time, and the segmentation result is relatively accurate, which can obtain most of the pixel regions where dynamic objects are located.

[0038] However, due to the simultaneous tracking of multiple objects in actual use, the use of semantic segmentation models often leads to label jumping and other errors in segmentation. Therefore, this method first tracks and segments a single dynamic object to obtain its mask, then segments the remaining objects to obtain the masks corresponding to all dynamic objects, and then fuses the masks of all dynamic objects in each frame to obtain the masks of all dynamic objects in each frame, and then obtains the multiple dynamic object masks, thereby improving the accuracy and robustness of segmentation. As shown in the results in Figure 3 (a) is the image to be segmented, and (b) is the segmentation result: the balloon is segmented first, then the human body is segmented, the masks in each frame are fused, and the final accurate mask containing all dynamic objects is obtained.

[0039] Step S3, feature extraction and dynamic elimination: extract point features and line features from the preprocessed image sequence, and eliminate corresponding dynamic features using the multiple dynamic object masks to obtain static features.

[0040] Among them, the multiple dynamic object masks are used to eliminate the corresponding dynamic features, that is, the dynamic regions corresponding to the multiple dynamic object masks are determined, and the features in the dynamic regions are deleted according to the dynamic masks to filter the points and lines in the static regions.

[0041] Optionally, this step extracts point features using the ORB detection method, i.e., using ORB feature points to obtain point features, and extracts line features using the LSD line detection method, i.e., using LSD line features to extract line features of the scene. After the point features and line features are extracted, the features falling in the multi-object dynamic mask region are removed, i.e., the outliers are deleted, that is, the Mask label of the pixel position corresponding to each feature is determined, if the pixel belongs to a dynamic object, the feature is removed, thereby retaining the static features.

[0042] Steps S2-S3 provide the following ideas for solving the SLAM problem of a complex scene containing multiple types of dynamic objects and accompanied by a weak texture region: first, a semantic segmentation network is used to identify and detect multiple types of dynamic objects in the scene, and dynamic feature points are removed in the subsequent tracking stage (including the pose estimation process); second, in addition to point features, line features that are more robust and easy to identify are introduced for weak texture regions (such as white walls, etc.), to extract more complete geometric information contained in the scene. Based on this, the error model of the SLAM method is redesigned, so that there are more constraints on the pose estimation in the iterative optimization process.

[0043] Step S4, pose estimation: based on the point features and line features, the static features are matched, the transformation matrix is solved, and the estimated pose is obtained by optimizing the re-projection error function of the joint point and line.

[0044] Specifically, the obtained point features and line features are matched with historical frames, the optimal match is found, and the transformation matrix is solved by triangulating the corresponding features, and a factor graph is constructed by optimizing the co-visible point features, line features, and key frames. Among them, the key frames are selected according to the current frame information, which includes the matching number of point features and line features and the feature tracking situation.

[0045] The estimated pose is obtained by optimizing the re-projection error function of the joint point and line, which specifically includes: first, obtaining the error function of the camera pose, improving the error function by combining the point features and line features to obtain the re-projection error function of the joint point and line, and obtaining the estimated pose by optimizing the joint error function. The re-projection error function of the joint point and line can be represented as:

[0046]

[0047] In the formula, the kth key frame pose is T cw,k =[R, t], the re-projection error of the observed ith space point p I is Ep k,i , the re-projection error of the jth space line observed by the kth key frame is El k,i , and respectively represent different observation covariances, and ρp and p l denote different robust kernel functions.

[0048] The feature matching of this step refers to 2D and 3D feature matching for the remaining static features, and a conversion matrix is estimated, so that the hyper-pragma map is initialized by triangulation. The hyper-pragma map contains the point cloud of the ORB feature, rotation, scaling, density and spherical harmonic coefficient, and the tracking process is optimized by a factor graph solver.

[0049] In step S5, geometric mapping: obtain sparse hyper-pragma, identify the area to be densified according to the point re-projection error, and generate a new hyper-pragma set in combination with the unmatched feature points.

[0050] According to the definition of hyper-pragma in Photo-SLAM, it is a set of point clouds with rotation r∈SO(3), scale s∈R+, density d∈R+ and ORB feature information. Body density Spherical harmonic coefficient and ORB feature information. The point re-projection error function is used to determine which areas need to be densified.

[0051] The sparse hyper-pragma map is composed of a point cloud set storing ORB features, rotation, scaling, density and spherical harmonic coefficient, and the area to be densified is identified according to the point re-projection error.

[0052] On this basis, for the 2D feature points that cannot be matched with the corresponding 3D points, the depth of the surrounding neighboring feature points is used to infer the depth, and the new hyper-pragma set is generated by projecting it to the 3D space. The sparse hyper-pragma is created by using the inactive 2D feature points. Specifically, the nearest activated point is found in the inactive 2D feature points, and the depth of the activated point is interpolated to estimate the depth of the inactive point, and the 2D feature is projected to the 3D space to create a hyper-pragma.

[0053] In step S6, scene mapping: obtain a rendered image, perform multi-scale Gaussian pyramid supervised training on the rendered image, and obtain the final global map. Specifically, it includes:

[0054] Firstly, the input image is subjected to Gaussian smoothing and down-sampling to obtain a rendered image; secondly, a pyramid is constructed for supervision at different levels of image resolution; finally, a photometric loss and an SSIM loss are constructed to train and optimize the representation of the rendered image, and the final global map representation is obtained.

[0055] The photometric loss and the SSIM loss can be expressed as:

[0056]

[0057] In the formula, represents the photometric loss and the SSIM loss, and λ is a preset weight parameter, I r is the rendered image, and I gt is the input image ground truth. During the training process, the hyper-primitive map is optimized by minimizing the loss function.

[0058] This step uses a neural rendering method to complete the mapping process, complete the mapping from 3D space features to 2D plane pixels, obtain a spatial model close to the real scene representation and an image representation of a new view in 2D space, and can effectively enrich the map representation information.

[0059] Step S7, loop detection: using a preset bag-of-words model for loop detection, when a loop is detected, adjusting the current frame and the hyper-primitive using a similar change to correct the pose and correct the cumulative error.

[0060] This step aims to further optimize the map representation and reduce ghosting and blurring in the rendering result.

[0061] Next, the algorithm is verified on multiple data sets to prove the effectiveness and practicality of the method. Specifically:

[0062] As shown in Table 1, on the TUM and BONN data sets, scene sequences such as freiburg3_walking_xyz (fr3_w_xyz) are selected for algorithm testing, and the absolute trajectory error (ATE) results are recorded, and compared with existing methods Dyna-SLAM, DS-SLAM, and ORB-SLAM3. Through comparison, it can be seen that compared with existing positioning algorithms, the ATE index of the present method is lower, and the positioning accuracy is greatly improved.

[0063] In addition, in the evaluation of the mapping result, the peak signal-to-noise ratio (PSNR) is calculated by counting the static pixels to reflect the reconstruction quality. The comparison results are shown in Figure 4 , where Figure 4 part (a) of the figure is the mapping result of the DDN-SLAM method, Figure 4 part (b) of the figure is the mapping result of the ESLAM method, Figure 4 part (c) of the figure is the mapping result of the LC_CRFSLAM method, Figure 4 part (d) of the figure is the mapping result of the Orbeez-SLAM method, Figure 4 part (e) of the figure is the mapping result of the present method. Through comparison, it can be seen that the present method has a great improvement in mapping quality compared with existing mapping algorithms.

[0064] Table 1 Comparison of ATE of various SLAM methods on TUM and BONN data sets

[0065] Sequence Dyna DS ORB3 The method TUM fr3_w_xyz 0.0214 0.0248 0.4622 0.0169 fr3_w_static 0.0092 0.0087 0.3243 0.0069 fr3_w_rpy 0.0353 0.4426 0.1683 0.0310 fr3_w_halfsphere 0.0262 0.0343 0.6531 0.0313 fr3_s_xyz 0.0152 0.0017 0.0248 0.0134 fr3_s_static 0.0067 0.0065 0.0146 0.0066 BONN ballon 0.0311 0.0323 0.0921 0.0292 balloon2 0.0297 0.0388 0.2163 0.0299 ballon_tracking 0.0326 0.0341 0.1811 0.0313 crowd 0.0183 0.3436 0.4742 0.0181 crowd2 0.0199 0.1784 0.6617 0.0184 crowd3 0.0254 0.1622 0.4536 0.0311 synchronous 0.0122 0.1136 0.6178 0.0092

[0066] Further, the embodiment also provides a monocular neural rendering SLAM system in a complex environment, comprising a data acquisition and preprocessing module, a dynamic object mask acquisition module, a feature extraction and dynamic elimination module, a pose estimation module, a geometric mapping module and a scene mapping module. Wherein: the data acquisition and preprocessing module is used for acquiring an initial image sequence in a complex environment and performing preprocessing; the dynamic object mask acquisition module is used for inputting the preprocessed image sequence into a pre-trained semantic segmentation model to obtain a plurality of dynamic object masks; the feature extraction and dynamic elimination module is used for extracting point features and line features from the preprocessed image sequence, and eliminating corresponding dynamic features by using the plurality of dynamic object masks to obtain static features; the pose estimation module is used for performing feature matching on the static features based on the point features and the line features, solving a transformation matrix, and obtaining an estimated pose by optimizing a point-line joint reprojection error function; the geometric mapping module is used for acquiring sparse hyperprimitives, identifying an area to be densified according to a point reprojection error, and generating a new hyperprimitive set in combination with unmatched feature points; and the scene mapping module is used for acquiring a rendered image, performing multi-scale Gaussian pyramid supervised training on the rendered image, and obtaining a final global map. The specific execution steps of each module are basically the same as the execution steps of the foregoing method, and will not be described here.

[0067] In summary, the present application proposes a monocular neural rendering SLAM method and system in a complex environment. The present application uses point-line feature construction and makes improvements on the method based on existing dynamic SLAM and neural rendering SLAM systems, which can improve the robustness of the SLAM system in dynamic, weak texture and other complex environments, and the neural rendering method can provide a rendered image close to the real scene from a new perspective, enriching the representation of the SLAM map. In addition, the present method effectively avoids the problems of misidentification and missed identification when existing image segmentation models simultaneously segment multiple objects by using the fusion of single object masks. At the same time, the present method constructs a relationship graph of point-line feature fusion and constructs a joint error function to solve the rotation matrix and translation vector of the camera, thereby improving the problem of insufficient number of point features under weak texture conditions by introducing line features.

[0068] The foregoing describes in detail the preferred embodiments of the present application. It should be understood that those skilled in the art can make many modifications and changes without creative labor based on the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A monocular neural rendering SLAM method in a complex environment, characterized in that, The method comprises the following steps: a data acquisition and preprocessing step: acquiring an initial image sequence under a complex environment and preprocessing; a dynamic object mask acquisition step: inputting the preprocessed image sequence into a pre-trained semantic segmentation model to obtain a plurality of dynamic object masks; a feature extraction and dynamic elimination step: extracting point features and line features from the preprocessed image sequence, and eliminating corresponding dynamic features using the plurality of dynamic object masks to obtain static features; a pose estimation step: based on the point features and the line features, performing feature matching on the static features, solving a transformation matrix, and obtaining an estimated pose by optimizing a point-line joint reprojection error function; a geometric mapping step: acquiring sparse super primitives, identifying an area to be densified according to point reprojection error, and generating a new super primitive set in combination with unmatched feature points; a scene mapping step: acquiring a rendered image, performing multi-scale Gaussian pyramid supervised training on the rendered image, and obtaining a final global map; the point-line joint reprojection error function is expressed as: In the formula, the kth key frame pose is , the reprojection error of the ith space point is , the reprojection error of the jth space line observed by the kth key frame is , and respectively represent different observation covariances, and respectively represent different robust kernel functions; the scene mapping step specifically comprises: performing Gaussian smoothing and downsampling on the input image to obtain a rendered image; constructing a pyramid for supervision at different levels of image resolution; constructing a photometric loss and an SSIM loss to train and optimize the representation of the rendered image; the photometric loss and the SSIM loss are expressed as: In the formula, denotes the luminosity loss and the SSIM loss, is a preset weight parameter, is a rendered image, is an input image true value.

2. The monocular neural-rendered SLAM method in complex environments of claim 1, wherein, The method further comprises a loop detection step: using a preset bag-of-words model to detect loops, and when a loop is detected, adjusting the current frame and the super primitives using a similarity change to correct the pose and correct the cumulative error.

3. The monocular neural-rendered SLAM method in complex environments of claim 1, wherein, The specific implementation process of the dynamic object mask acquisition step comprises: tracking and segmenting individual dynamic objects in sequence to obtain masks corresponding to all dynamic objects respectively; fuse all masks on each frame to obtain the masks of all dynamic objects in each frame image.

4. The monocular neural-rendered SLAM method in complex environments of claim 1, wherein, In the feature extraction and dynamic elimination step, point features are extracted using an ORB detection method, and line features are extracted using an LSD line detection method.

5. The monocular neural-rendered SLAM method in complex environments of claim 1, wherein, In the pose estimation step, triangulation is performed through feature matching to initialize the super primitive map, key frames are selected according to current frame information, and a factor graph is constructed by optimizing co-visible point features, line features and key frames, the factor graph being used to optimize the tracking process.

6. The monocular neural-rendered SLAM method in complex environments of claim 5, wherein, The current frame information includes the number of matched point features and line features and the feature tracking situation.

7. A system employing the monocular neural-rendered SLAM method in complex environments according to any one of claims 1-6, characterized in that, It comprises: a data acquisition and preprocessing module for acquiring an initial image sequence under a complex environment and preprocessing; a dynamic object mask acquisition module for inputting the preprocessed image sequence into a pre-trained semantic segmentation model to obtain a plurality of dynamic object masks; a feature extraction and dynamic elimination module for extracting point features and line features from the preprocessed image sequence, and eliminating corresponding dynamic features using the plurality of dynamic object masks to obtain static features; a pose estimation module for performing feature matching on the static features based on the point features and the line features, solving a transformation matrix, and obtaining an estimated pose by optimizing a point-line joint reprojection error function; The geometry mapping module is configured to obtain sparse hyper primitives, identify an area to be densified according to a point re-projection error, and generate a new hyper primitive set in combination with unmatched feature points; The scene mapping module is configured to obtain a rendering image, perform multi-scale Gaussian pyramid supervised training on the rendering image, and obtain a final global map.

Images