Visual positioning method and device for unmanned vehicle intelligent simulation test under denial condition
By integrating 3D Gaussian Splatting with an unsupervised SLAM method that can be optimized through differential cycles, and using image reconstruction errors for training, the problem of insufficient generalization ability of deep learning SLAM methods in complex scenes is solved, and high-precision unsupervised localization and mapping are achieved.
Patent Information
- Application Number
- CN202510994119.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-18
- Publication Date
- 2025-11-21
AI Technical Summary
Existing deep learning SLAM methods lack generalization ability in complex scenarios, rely on supervised data, resulting in poor mapping accuracy and difficulty in effectively locating and mapping in dynamic, sparse, or unknown environments.
An unsupervised SLAM method using 3D Gaussian Splatting and a differentiable cyclic optimization structure is adopted. The method is trained by image reconstruction error, constructs a differentiable graph optimization framework, and achieves end-to-end unsupervised training by utilizing 3D Gaussian ellipsoid modeling and a differentiable rendering mechanism.
Without requiring real-label data, it significantly improves the system's localization and mapping accuracy and robustness in complex environments, reduces training costs, and enhances cross-scenario adaptability.
Smart Images
Figure CN120997258A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of virtual and real intelligent testing, more particularly, to a visual positioning method and device under denial condition for intelligent simulation testing of unmanned vehicles, and discloses a 3D Gaussian Splatting (3DGS) unsupervised fine-tuning differentiable loop optimization Simultaneous Localization and Mapping (SLAM) method, which utilizes deep learning and unsupervised learning technology to realize high-precision three-dimensional reconstruction and camera pose estimation of a scene, and is suitable for complex environment perception tasks such as autonomous driving and robot navigation. BACKGROUND
[0002] With the rapid development of intelligent systems such as autonomous robots, augmented reality / virtual reality (AR / VR), and unmanned driving, higher requirements are put forward for the precision, real-time performance, and robustness of the Simultaneous Localization and Mapping (SLAM) system. Traditional SLAM methods rely on geometric calculations and feature matching, and although they can achieve relatively stable positioning and mapping in static environments, their performance significantly decreases in complex scenes with occlusions, dynamic objects, and sparse textures. In addition, such methods are difficult to generalize across scenes and often require manual parameter tuning or the design of specific data association strategies.
[0003] In recent years, with the rise of deep learning technology, learning-based SLAM methods have rapidly developed, especially the differentiable loop optimization framework represented by DROID-SLAM, which realizes joint optimization of camera pose and depth through the construction of a differentiable bundle adjustment module (BA), significantly improving the accuracy and robustness of the system. Such methods rely on end-to-end trained feature extraction, optical flow estimation, and optimization modules, and can adapt to real complex environments to some extent. However, such methods generally rely on a large amount of labeled supervised data (such as real depth maps or accurate poses), which limits their promotion and deployment under actual conditions with high data acquisition costs and difficult labeling. SUMMARY
[0004] The present application aims to solve the deficiencies of existing deep learning SLAM methods in generalization ability, dependence on supervised data, and accuracy of complex scene mapping, and proposes an unsupervised SLAM method and system that combines 3D Gaussian Splatting (3DGS) and a differentiable loop optimization structure. By constructing an end-to-end differentiable graph optimization framework, the system can be trained using image reconstruction errors without real depth or pose labels, significantly improving the positioning and mapping capabilities of the system in dynamic, sparse, or unknown environments. The specific technical solutions are as follows: The application provides a visual positioning method under denial condition for intelligent simulation test of unmanned vehicle. Input a plurality of image sequences and perform preprocessing, wherein the plurality of image sequences include target views and source views; Feature extraction is performed on each target view and source view by using a convolution network; Optical flow estimation is performed based on the features extracted from the target views and the source views; A differentiable cyclic optimization structure is constructed, and updated poses and depths of all views are calculated by differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views; A source view Gauss ellipsoid set is constructed based on the poses and depths of the source views, including source view Gauss ellipsoid parameters; The target views are rendered based on the source view Gauss ellipsoid parameters.
[0005] In an embodiment, feature extraction is performed on each target view and source view by using a convolution network, including: The convolution network uses two independent networks to perform feature extraction on the views, and the two independent networks are a feature network and a context network, and image features and context features are obtained respectively.
[0006] In an embodiment, optical flow estimation is performed based on the features extracted from the target views and the source views, including: A correlation volume pyramid is constructed based on the similarity of image features at different pixel positions; A predicted optical flow is obtained according to the correlation volume pyramid, the context features of the source views and the context features of the target views.
[0007] In an embodiment, a predicted optical flow is obtained according to the correlation volume pyramid, the context features of the source views and the context features of the target views, including: The correlation volume pyramid, the context features of the source views and the context features of the target views are input into a gated recurrent unit, and an updated hidden state is obtained by using a convolution operator thereof; An optical flow residual is predicted based on the updated hidden state by using a convolution operation; A predicted optical flow is obtained according to the optical flow output in the previous step and the optical flow residual.
[0008] In an embodiment, updated poses and depths of all views are calculated by differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views, and the calculation is as follows:
[0009]
[0010] wherein , are the focal lengths of the camera axes, are the optical centers on the camera axes, , denotes, is a view relation graph constructed from source views and target views, is an edge of the graph , is a source view pixel coordinate, is a source view corresponding target view pixel coordinate computed by optical flow , is a depth at pixel position in the nth step, is a covariance matrix, is a camera pose at pixel position in the nth step, , are the camera poses in the nth and nth+1 steps, , are the camera depths in the nth and nth+1 steps.
[0011] In an embodiment, rendering a target view based on Gaussian ellipsoid parameters of a source view comprises:
[0012] wherein denotes a Gaussian ellipsoid index used for transparency calculation in alpha blending, is an index of the source view, is a learned color, is a learned opacity, is a product of a 2D Gaussian Gaussian distribution with a projection center , a covariance , is obtained by a 3D Gaussian covariance dimensionality reduction transformation, denotes a pixel.
[0013] In an embodiment, the method further comprises: using photometric error between reconstructed images and real images as a supervision signal, back-propagating to optimize the differentiable recurrent optimization to fine-tune the SLAM network parameters. Based on the same inventive concept, the second aspect of the present application provides a visual positioning device under denial condition for intelligent simulation test of unmanned vehicle, comprising: a data input and preprocessing module configured to input a multi-frame image sequence and perform preprocessing, wherein the multi-frame image sequence comprises target views and source views; a feature extraction module configured to perform feature extraction on each target view and source view using a convolutional network; an optical flow estimation module configured to perform optical flow estimation based on the features extracted from the target views and the source views; a differentiable recurrent optimization structure construction module configured to construct a differentiable recurrent optimization structure for calculating updated poses and depths of all views by differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views; a 3D Gaussian ellipsoid scene modeling module configured to construct a set of source view Gaussian ellipsoids based on the poses and depths of the source view cameras, including source view Gaussian ellipsoid parameters; a differentiable rendering and reconstruction module configured to perform rendering on the target views based on the source view Gaussian ellipsoid parameters.
[0014] Based on the same inventive concept, the third aspect of the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the visual positioning method under the denial condition for intelligent simulation testing of an unmanned vehicle according to the first aspect.
[0015] Based on the same inventive concept, the fourth aspect of the present application provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the visual positioning method under the denial condition for intelligent simulation testing of an unmanned vehicle according to the first aspect when executing the program.
[0016] Compared with the prior art, the present application has the following advantages and beneficial technical effects: This invention discloses a visual localization method under rejection conditions for intelligent simulation testing of unmanned vehicles. This method integrates 3D Gaussian Splatting (3DGS) modeling technology with a differentiable cyclic optimization SLAM framework. It fine-tunes the SLAM system end-to-end using image reconstruction errors without requiring real-world labeled data. The method includes: extracting features from input image pairs, estimating optical flow, and constructing a differentiable factor graph to optimize camera pose and depth; reprojecting source image features based on reconstructed geometry to establish a 3D Gaussian ellipsoidal scene representation and rendering the target view; using the photometric error between the reconstructed image and the real image as a supervision signal, backpropagating to optimize the parameters of the differentiable cyclic optimization SLAM network, resulting in a compact scene 3D reconstruction (including camera pose and depth images) network output. Backpropagation optimizes the network parameters, resulting in a compact scene 3D reconstruction network output. This method has advantages such as being label-free, highly adaptable to different scenes, and capable of end-to-end training, significantly improving the localization and mapping accuracy of SLAM systems in complex dynamic scenes. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart of a visual positioning method under rejection conditions for intelligent simulation testing of unmanned vehicles, as described in an embodiment of the present invention.
[0019] Figure 2 This is a framework diagram of a visual positioning device under rejection conditions for intelligent simulation testing of unmanned vehicles, as described in an embodiment of the present invention. Detailed Implementation
[0020] This invention aims to address the shortcomings of existing deep learning SLAM methods in terms of generalization ability, reliance on supervised data, and accuracy in mapping complex scenes. It proposes an unsupervised SLAM system that integrates 3D Gaussian Splatting (3DGS) with a differentiable cyclic optimization structure. By constructing an end-to-end differentiable graph optimization framework, this system utilizes image reconstruction errors for training without requiring real depth or pose labels, significantly improving its localization and mapping capabilities in dynamic, sparse, or unknown environments.
[0021] Specifically, the method of the present application is based on the framework of microcyclic optimization SLAM, retaining its efficient differentiable cyclic optimization module, while introducing 3DGS for representing and rendering scene geometry information. The system flow includes image input, feature extraction, optical flow estimation, depth and camera pose optimization, feature re-projection and 3DGS modeling, and image reconstruction supervision based on the differentiable rendering results. The core innovation lies in: 3D Gaussian ellipsoid scene modeling based on re-projection geometry: using depth estimation and camera pose to project the pixels of the target view back to the three-dimensional space, and constructing a 3D Gaussian ellipsoid to realize continuous and differentiable scene representation.
[0022] 3DGS high-quality differentiable rendering mechanism: using the 3D Gaussian field generated by re-projection to perform differentiable rendering on the target view image to generate a reconstructed image that approximates the real image, which is used as a supervision signal to fine-tune the deep learning SLAM.
[0023] End-to-end unsupervised fine-tuning mechanism: using image reconstruction error (including L1 loss and structural similarity index SSIM) as loss function, performing gradient backpropagation optimization on network parameters to realize adaptive optimization of the entire SLAM system.
[0024] Embodiment one The embodiment provides a visual positioning method under denial condition for intelligent simulation test of unmanned vehicle, please see Figure 1 , comprising: S1: inputting a plurality of image sequences and performing preprocessing, wherein the plurality of image sequences include target views and source views.
[0025] Specifically, S1 is image input and preprocessing. Specifically, it can be realized by the following way: Input a plurality of image sequences , k is the index of the image sequence, the width is , and the height is Taking three images as an example:
[0026] Wherein is the target view at time , and the error calculated by the radiance field rendering synthesized image is used to fine-tune the depth learning SLAM network parameters, are the source views at time .
[0027] The camera intrinsic matrix parameters are read by calibration.
[0028] are the focal lengths of the camera axis, Respective cameras Optical center on the axis.
[0029] Padding or cropping the image to transform the resolution to a multiple of 8.
[0030] S2: Feature extraction for each target view and source view using a convolutional network.
[0031] Specifically, S2 is feature extraction, extracting the feature of each view respectively.
[0032] In an embodiment, it can be achieved by the following way: The convolutional network uses two independent networks to extract the features of the views respectively, the two independent networks are a feature network and a context network, respectively obtaining image features and context features.
[0033] In the specific implementation process, the system uses a convolutional network to extract features from each view , using two independent networks: a feature network and a context network, respectively generating image features and context features and . Among them, the extracted image features are used to construct a set of correlation volumes, while the context features are injected into a gated recurrent unit (GRU) to obtain a new hidden state each time the operator is applied. The expression of feature extraction is:
[0034] C denotes the channel dimension of the feature.
[0035] S3: Optical flow estimation based on the features extracted from the target view and the source view.
[0036] Specifically, S3 is optical flow estimation, when performing optical flow extraction (calculation), all source views are associated with the target view to construct a view relationship graph , a node in the view relationship graph is an edge , input the RAFT optical flow network pair by pair , and output the optical flow .
[0037]
[0038] Among them, the optical flow estimation based on the features extracted from the target view and the source view includes: constructing a correlation volume pyramid based on the similarity of the image features at different pixel positions; obtaining the predicted optical flow according to the correlation volume pyramid, the context features of the source view, and the context features of the target view.
[0039] In the implementation process, the image features at each pixel position are calculated one by one to obtain the feature vectors of the image features at each pixel position within a radius of , and the similarity of the feature vectors is calculated to form a correlation volume. Then, four levels of average pooling are performed on the last two dimensions of the correlation volume to form a correlation volume pyramid.
[0040]
[0041]
[0042] obtaining the predicted optical flow according to the correlation volume pyramid, the context features of the source view, and the context features of the target view, comprising: inputting the correlation volume pyramid, the context features of the source view, and the context features of the target view into a gated recurrent unit, and using a convolution operator thereof to obtain an updated hidden state; predicting an optical flow residual based on the updated hidden state using a convolution operation; obtaining the predicted optical flow according to the optical flow output in the previous step and the optical flow residual.
[0043] In the implementation process, based on the correlation volume pyramid and the context features of the source view and the target view , a gated recurrent unit (GRU) convolution operator is used to output an iteratively updated hidden state at the first step , and further predict an optical flow residual through a convolution operation, obtain an optical flow at the first iteration step , and obtain a predicted optical flow at the first step , and the specific formula is: .
[0044] S4: constructing a differentiable recurrent optimization structure for calculating the updated poses and depths of all views based on the estimated optical flow and the poses and depths of all views using a differentiable dense bundle adjustment.
[0045] Specifically, S4 constructs a differentiable recurrent optimization structure for calculating the updated poses and depths of the views.
[0046] wherein, based on the estimated optical flow and the poses and depths of all views, the updated poses and depths of all views are calculated by a differentiable dense bundle adjustment, specifically:
[0047]
[0048] wherein , are focal lengths of camera axes, are optical centers of camera axes, denotes a point coordinate in the camera coordinate system, denotes the updated difference value of the Lie algebra form of the camera pose after the th step of optimization, denotes the updated difference value of the depth after the th step of optimization, is a view relationship graph constructed by source views and target views, is an edge of the node , i, j correspond to edge nodes in the view relationship graph, is a source view pixel coordinate, is a target view pixel coordinate corresponding to the source view calculated by the optical flow , is the depth of the pixel position in the th step of optimization, is a covariance matrix, is the camera pose between the edges in the th step of optimization, is the Lie algebra form of the camera pose, are camera poses of all views in the th step and the th step of optimization, respectively, and the camera pose and depth of all views in the th step of optimization are the updated camera pose and depth of all views.
[0049] In the implementation process, given the iterative optical flow of each edge of the view relationship graph, the updated camera pose and depth of all views in the th step of optimization are calculated by a differentiable dense bundle adjustment (DBA). The camera pose and depth of all views are finally outputted through a preset iteration number.
[0050] S5: Constructing a source view Gaussian ellipsoid set according to the pose and depth of the source view camera, including source view Gaussian ellipsoid parameters.
[0051] Specifically, S5 is source target view 3D Gaussian sphere Gaussian ellipsoid scene modeling. A 3D Gaussian sphere represents a three-dimensional scene as a mixture of anisotropic three-dimensional Gaussian distributions, where each distribution is defined by a three-dimensional covariance matrix and a mean :
[0052] wherein, represents a three-dimensional Gaussian variable.
[0053] In the specific implementation process, given the source view camera pose and depth , a source view Gaussian ellipsoid set with a number of and an index of is constructed, which is used to render a target view image :
[0054] For all pixel sets of the target view, the corresponding Gaussian ellipsoid parameters of the target view pixel are obtained by three-dimensional point back-projection pixel.
[0055]
[0056] wherein is the depth of the pixel position.
[0057] The remaining parameters (the scale of the Gaussian ellipsoid and the rotation , color , transparency ) are obtained through pooling multi-view features.
[0058] S6: Rendering the target view based on the Gaussian ellipsoid parameters of the source view.
[0059] Specifically, S6 is differentiable rendering and image reconstruction. It can be implemented in the following way:
[0060]
[0061] wherein represents a Gaussian ellipsoid index for transparency calculation in alpha blending, is an index of a source view, is a learned color, is a learned opacity, and is is a product of a 2D Gaussian Gaussian distribution with a projection center , a covariance , and a covariance is obtained by 3D Gaussian covariance dimensionality reduction transformation, represents a pixel.
[0062] wherein the covariance is calculated in the following way:
[0063] wherein is a Jacobian matrix of an affine approximation transformation of a projection transformation, is a view transformation matrix, which can be calculated by a camera pose . In an embodiment, the method further comprises: Taking photometric error between a reconstructed image and a real image as a supervision signal, the optimization of the differentiable recurrent optimization SLAM network parameters is performed by back propagation, and the recurrent optimization structure is fine-tuned. Specifically, the fine-tuning of the differentiable recurrent optimization SLAM network is performed by using the following image reconstruction loss , and the training scene adaptability is improved.
[0064]
[0065] wherein is a weight hyperparameter, is a structural similarity index, is a real image, is a reconstructed image.
[0066] The photometric error is used as a supervision signal for unsupervised training to fine-tune the deep learning SLAM, so that the network outputs a compact scene three-dimensional reconstruction result, and adaptive optimization of the entire SLAM system is realized.
[0067] Overall, the advantages and beneficial technical effects of the present application are as follows: (1) The dependence on supervised label data is eliminated: The application introduces a 3D Gaussian snowball sputtering (3DGS) modeling and a differentiable rendering mechanism, uses image reconstruction error as a training supervision signal, realizes completely unsupervised system fine-tuning, effectively solves the high dependence on depth or pose labels of traditional differentiable recurrent optimization structure SLAM in actual deployment, and significantly reduces the training cost.
[0068] (2) The geometric modeling capability for complex scenes is improved: by fusing 3DGS for structure prior modeling in the framework of the differentiable recurrent optimization structure SLAM, the geometric details in the scene can be more accurately captured, and the mapping accuracy and integrity of the system in the environment with sparse texture, severe occlusion or complex structure are improved.
[0069] (3) The generalization ability and adaptability of the system are enhanced: the structure perception feature reprojection mechanism and the training method based on photometric consistency of the application enable the network parameters to be adaptively adjusted during the training process, thereby having stronger cross-scene migration ability and robustness under different visual conditions.
[0070] Embodiment two Based on the same inventive concept, the embodiment discloses a visual positioning device under denial condition for intelligent simulation test of unmanned vehicle, which comprises: a data input and preprocessing module for inputting a plurality of image sequences and performing preprocessing, wherein the plurality of image sequences comprise target views and source views; a feature extraction module for extracting features of each target view and source view by using a convolutional network; an optical flow estimation module for estimating optical flow based on the features extracted from the target views and the source views; a differentiable recurrent optimization structure construction module for constructing a differentiable recurrent optimization structure, for calculating updated poses and depths of all views by a differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views; a 3D Gaussian ellipsoid scene modeling module for constructing a set of source view Gaussian ellipsoids according to the poses and depths of the source view cameras, including source view Gaussian ellipsoid parameters; a differentiable rendering and reconstruction module for rendering the target views based on the Gaussian ellipsoid parameters of the source views.
[0071] As Figure 2 shown, it is a whole framework diagram of the visual positioning device under denial condition for intelligent simulation test of unmanned vehicle.
[0072] Since the device in the second embodiment of the present application is the device used for implementing the method in the first embodiment of the present application, the specific structure and the deformation of the device can be understood by the person skilled in the art based on the method introduced in the first embodiment of the present application, and thus will not be described here again. Any device used for the method in the first embodiment of the present application belongs to the scope of the present application.
[0073] Embodiment three Based on the same inventive concept, the present application further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the visual positioning method under the rejection condition for the intelligent simulation test of the unmanned vehicle according to the first embodiment.
[0074] Since the computer readable storage medium introduced in the third embodiment of the present application is the computer readable storage medium used for implementing the visual positioning method under the rejection condition for the intelligent simulation test of the unmanned vehicle according to the first embodiment of the present application, the specific structure and the deformation of the computer readable storage medium can be understood by the person skilled in the art based on the method introduced in the first embodiment of the present application, and thus will not be described here again. Any computer readable storage medium used for the method in the first embodiment of the present application belongs to the scope of the present application.
[0075] Embodiment four Based on the same inventive concept, the present application further provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to the first embodiment when executing the program.
[0076] Since the computer device introduced in the fourth embodiment of the present application is the computer device used for implementing the visual positioning method under the rejection condition for the intelligent simulation test of the unmanned vehicle according to the first embodiment of the present application, the specific structure and the deformation of the computer device can be understood by the person skilled in the art based on the method introduced in the first embodiment of the present application, and thus will not be described here again. Any computer device used for the method in the first embodiment of the present application belongs to the scope of the present application.
[0077] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.
[0078] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions of the flowchart block(s) or step(s) of the flowchart block(s). Figure 1 one or more functions specified by one or more blocks or steps in the flowchart block(s) and / or flowchart step(s). Figure 1 one or more functions specified by one or more blocks or steps in the flowchart block(s) and / or flowchart step(s).
[0079] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they learn of the basic inventive concepts. Such additional variations and modifications have been omitted in order to allow for a clearer and more concise description of the basic Obviously, various modifications and changes can be made to the embodiments of the present application without departing from the spirit and scope of the application. It is understood, therefore, that this application is not intended to be limited to the particular forms disclosed but is to be accorded the broadest interpretation of the claims so as to encompass all such modifications and equivalent arrangements.
Claims
1. A visual positioning method under denial condition for intelligent simulation test of unmanned vehicle, characterized in that, The method comprises the following steps: inputting a multi-frame image sequence and preprocessing, wherein the multi-frame image sequence comprises target views and source views; extracting features of each target view and source view using a convolutional network; estimating optical flow based on the features extracted from the target views and the source views; constructing a differentiable recurrent optimization structure for calculating updated poses and depths of all views by differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views; constructing a source view Gaussian ellipsoid set based on the poses and depths of the source view cameras, including source view Gaussian ellipsoid parameters; rendering the target views based on the Gaussian ellipsoid parameters of the source views.
2. The visual positioning method under denial condition for intelligent simulation test of unmanned vehicle according to claim 1, wherein, The method for extracting features of each target view and source view using a convolutional network comprises the following steps: The convolutional network uses two independent networks to extract features of the views, and the two independent networks are feature networks and context networks, respectively obtaining image features and context features.
3. The visual positioning method under denial condition for intelligent simulation test of unmanned vehicle according to claim 2, wherein, The method for estimating optical flow based on the features extracted from the target views and the source views comprises the following steps: constructing a correlation volume pyramid based on the similarity of image features at different pixel positions; obtaining predicted optical flow based on the correlation volume pyramid, the context features of the source views, and the context features of the target views. 4.The visual positioning method under denial condition for intelligent simulation test of unmanned vehicle according to claim 3, wherein, The method for obtaining predicted optical flow based on the correlation volume pyramid, the context features of the source views, and the context features of the target views comprises the following steps: inputting the correlation volume pyramid, the context features of the source views, and the context features of the target views into a gated recurrent unit to obtain updated hidden states using the convolutional operator thereof; predicting optical flow residuals based on the updated hidden states using a convolutional operation; obtaining predicted optical flow based on the output optical flow and the optical flow residuals of the previous step.
5. The visual positioning method under denial condition for intelligent simulation test of unmanned vehicle according to claim 1, wherein, The method for calculating updated poses and depths of all views by differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views comprises the following steps: wherein , are the optical centers of the cameras on the z-axis, are the optical centers of the cameras on the z-axis, denotes the point coordinates in the camera coordinate system, denotes the depth of the point denotes the updated difference value of the Lie algebra form of the camera pose after the denotes the updated difference value of the depth after the step optimization, is a view relation graph constructed from the source views and the target views, is an edge of the node , is the source view pixel coordinate, is the target view pixel coordinate corresponding to the source view pixel coordinate calculated by the optical flow , is the depth of the point at the pixel position, is the covariance matrix, is the camera pose between the edges in the n-th step, is the Lie algebra form of the camera pose, , are the camera poses of all views in the n-th step and the n+1-th step, , are the camera depths of all views in the n-th step and the n+1-th step, and the camera pose and depth of all views in the n+1-th step are the updated camera pose and depth of all views.
6. The visual positioning method under denial condition for intelligent simulation test of unmanned vehicle according to claim 1, wherein, The method for rendering the target views based on the Gaussian ellipsoid parameters of the source views comprises the following steps: wherein represents a Gaussian ellipsoid index for transparency calculation in alpha blending, is an index of a source view, is a learned color, is a learned opacity, is is a product of a 2D Gaussian Gaussian distribution with a projection center , a covariance , a covariance is obtained by a 3D Gaussian covariance dimensionality reduction transform, represents a pixel.
7. The visual positioning method under denial condition for intelligent simulation test of unmanned vehicle according to claim 1, wherein, The method further comprises the following steps: using the photometric error between the reconstructed image and the real image as a supervision signal to optimize the parameters of the differentiable recurrent optimization SLAM network in a backward propagation manner, and fine-tuning the differentiable recurrent optimization structure. 8.A visual positioning device under denial condition for intelligent simulation test of unmanned vehicle, characterized in that, The method comprises the following steps: a data input and preprocessing module for inputting a multi-frame image sequence and preprocessing, wherein the multi-frame image sequence comprises target views and source views; a feature extraction module for extracting features of each target view and source view using a convolutional network; an optical flow estimation module for estimating optical flow based on the features extracted from the target views and the source views; a differentiable recurrent optimization structure construction module for constructing a differentiable recurrent optimization structure for calculating updated poses and depths of all views by differentiable dense bundle adjustment based on the estimated optical flow and the poses and depths of all views; a 3D Gaussian ellipsoid scene modeling module for constructing a source view Gaussian ellipsoid set based on the poses and depths of the source view cameras, including source view Gaussian ellipsoid parameters; a differentiable rendering and reconstruction module for rendering the target views based on the Gaussian ellipsoid parameters of the source views.
9. A computer-readable storage medium, characterized in that, A computer program product, comprising a computer readable medium having stored thereon a computer program which, when executed by a processor, implements the method for visual localization under denial conditions for intelligent simulation testing of unmanned vehicles according to any one of claims 1 to 7.
10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, A processor implementing the method for visual localization under denial conditions for intelligent simulation testing of unmanned vehicles according to any one of claims 1 to 7 when executing the program.