116 results about "Reprojection error" patented technology

The invention provides a camera calibration method and device, electronic equipment and a storage medium, and belongs to the technical field of data processing, and the method comprises the steps: obtaining laser radar point cloud data, inertial measurement unit data and reference camera image data of a vehicle, and obtaining initial calibration parameters of a to-be-calibrated camera; based on the laser radar point cloud data, the inertial measurement unit data and the reference camera image data, constructing a point cloud map containing color textures and a pose track of a vehicle, and converting the point cloud map into a surface grid model; according to the pose track and the initial calibration parameters, the surface grid model is rendered to a visual angle of the to-be-calibrated camera, and a rendered image and a rendered depth map corresponding to the to-be-calibrated camera are generated; based on the real image data, the rendered image and the rendered depth map, establishing a corresponding relationship between two-dimensional image features and three-dimensional space points in the real image data; and constructing a re-projection error objective function based on the corresponding relationship, and optimizing the external reference and the internal reference of the camera to be calibrated by minimizing the objective function.

The invention discloses a rapid robust monocular vision inertial positioning method and system. The method comprises the following steps: configuring IMU and monocular camera sensor parameters; preprocessing the IMU data; iMU attitude initialization is carried out; image features of the monocular camera are extracted, abnormal matching points are removed, and IMU pre-integration is carried out; monocular vision initialization is carried out; odometer attitude optimization: judging whether the feature points are mismatched or not based on an IMU pre-integration result, constructing a feature point re-projection error jacobian matrix, and performing blocking and diagonalization processing on the re-projection error jacobian matrix to respectively optimize inverse depths and image frame attitudes of the feature points; selecting a key frame based on a key frame identification rule after the current frame attitude update is completed; and judging according to the current frame, and removing a certain frame in the sliding window to reserve a sliding window space for adding the latest frame. According to the method, the inverse depth of the feature points and the image frame attitude are optimized respectively based on the counterweight projection error Jacobian matrix partitioning and diagonalization processing, and the calculation complexity is reduced.

The invention discloses a dynamic visual guidance method and system for cooperative assembly of multiple mechanical arms, and belongs to the technical field of industrial robot visual guidance, and the method comprises the steps: synchronously collecting an image sequence through a binocular visual system, synchronously recording the pose sequence of each mechanical arm through a robot controller, and obtaining an observation data set; obtaining a two-dimensional trajectory flow of vehicle body assembly area feature points and instrument body feature points through an optical flow tracking algorithm, and obtaining a hand-eye matrix correction through a nonlinear optimization algorithm; obtaining an updated hand-eye matrix and a local pixel compensation field through a spatial interpolation algorithm based on the re-projection error of each feature point; according to the method, self-adaptive compensation of flexible deformation of a vehicle body and dynamic reflection of an instrument is realized by adopting unsupervised clustering and spatial interpolation according to a re-projection error, and the precision of cooperative assembly of multiple mechanical arms and the self-adaptive capability of a system in a complex industrial environment are improved.

The invention relates to the field of computer vision and mobile robot navigation, and discloses an object-level semantic vision SLAM method and system based on a mixed attention mechanism target detection network and an ellipsoid model. The system comprises five parallel thread modules, namely a semantic perception module, a visual tracking and repositioning module, a local mapping and fusion module, a loopback detection module and a global consistency optimization module. According to the method, local and global features of an image are extracted in parallel through a target detection network, and high-precision semantic observation is output; when the tracking is lost, the dual geometric constraint of the 2D internally tangent ellipsoid-3D object ellipsoid is utilized, and the P3P algorithm and the IoU cost function are matched to realize rapid relocation. In the mapping process, Gaussian-Wasserstein distance is introduced to measure semantic re-projection errors, and a joint objective function containing map point geometric errors and object semantic errors is constructed to carry out bundle adjustment. According to the method, the problem of feature extraction failure in motion blur and weak texture scenes is effectively solved, and the construction precision of the semantic map and the survivability of the system are remarkably improved.

The invention provides a multi-camera external parameter calibration method. The multi-camera external parameter calibration method comprises the following steps: acquiring an image sequence acquired by each camera in a plurality of cameras of road mobile equipment in a turning time window and an actual pose data sequence of the image sequence; when it is detected that the road mobile device is in the idle state, executing the following steps: obtaining a target space point of any camera and camera coordinate system target pose data corresponding to any camera; obtaining an initial transformation parameter for representing a spatial transformation relationship between a coordinate system of each camera and a coordinate system of the road mobile device; obtaining a cross-camera feature matching relation of any two cameras; and according to the cross-camera feature matching relation of every two cameras, the target space points of all the cameras, the initial transformation parameters and the initial rotation external parameter of each camera, obtaining the rotation external parameter of each camera when the total reprojection error is the minimum value, and taking the rotation external parameter as the calibration rotation external parameter of each camera. According to the method, full-automatic and high-precision rotation external parameter calibration can be realized.

The invention discloses a steel structure three-dimensional reconstruction method based on Gaussian splashing, and belongs to the technical field of three-dimensional computer vision and computer graphics, and the method comprises the steps: obtaining a multi-view image of a steel structure scene, and generating a view alignment image sequence and an image quality mask; generating a visibility weight in combination with the appearance stability score and the occlusion estimation; linear, planar and circular geometric features in the image are recognized, spatial registration is carried out on the geometric features and the visibility weight, and registration structure guiding features are generated; initializing a three-dimensional Gaussian set and projecting the structural features to Gaussian parameters to generate structural correlation parameters; the weighted reprojection error and the structural constraint error are fused to form a joint error index, and splitting, merging and optimization of a Gaussian model are guided; and through boundary consistency evaluation and sharpening processing, a steel structure three-dimensional reconstruction model is generated. By fusing the prior features of the geometric structure in the two-dimensional image and the optimization process of the three-dimensional Gaussian model, the structural accuracy and boundary definition of the reconstructed model can be improved.

The invention discloses a cultivated land non-agrochemical monitoring method based on unmanned aerial vehicle video spatialization. The method comprises the following steps: acquiring a farmland area video stream, positioning and attitude determination system data and digital elevation model data acquired by an unmanned aerial vehicle; extracting a key frame from the video stream based on a double-threshold detection algorithm of the optical stream change rate and the image information entropy; performing spatialization processing on the key frame through a collinear equation optimization solution combined with a digital elevation model, and converting image coordinates into geographic coordinates; fusing the key frame image data with the corresponding geographical coordinate information, and constructing input data of a deep learning model of geographical perception; inputting the fused data into a geographic perception deep learning model to perform ground object interpretation, and extracting spatial features by the model through a built-in geographic coordinate embedding layer to identify a cultivated land non-agrochemical target; post-processing is carried out on the interpretation result, the post-processing comprises morphological closed operation processing and grid-to-vector conversion processing, quality control is carried out by calculating a re-projection error index and an intersection-to-parallel ratio index, and a cultivated land non-agrochemical monitoring report is generated.

The invention discloses a state estimation method based on mixed features and adaptive key frame selection, and belongs to the field of autonomous navigation of unmanned systems. The implementation method comprises the following steps of: acquiring a plurality of images, and synchronously acquiring inertial measurement data of a six-axis IMU (Inertial Measurement Unit); the method comprises the following steps: performing key frame decision on a current frame image according to inertial measurement data, and classifying the image into a key frame image and a non-key frame image: acquiring a key frame feature matching set through an entropy maximization self-supervision feature extractor and an image attention matcher, and constructing a visual reprojection error term based on triangulation map points; non-key frame feature points are extracted by adopting an improved corner detector, feature displacement is obtained through IMU constraint optical flow tracking, and a visual re-projection error and an IMU pre-integration residual term are generated; and based on a vision-inertia tight coupling optimization module solution system, combining a vision reprojection error, an IMU pre-integration residual error and a pneumatic constraint residual error to construct a tight coupling optimization problem, and solving an optimal pose state. The method is beneficial to navigation of the small-sized high-speed unmanned aerial vehicle.

The invention relates to an intelligent multi-source navigation method based on inertia and Beidou / visual information fusion, which comprises the following steps: firstly, establishing a multi-source information system of an inertial navigation system, and embedding an IMU (Inertial Measurement Unit) multi-parameter online calibration mechanism based on carrier dynamics in inertial navigation solution; establishing a tight coupling fusion framework taking an INS resolving result as a reference, and constructing tight combination measurement with the original observed quantity of the BDS and constructing re-projection error measurement with the visual feature points by utilizing INS high-frequency pose information; an intelligent adaptive filter based on inertial dynamics error model constraint is adopted, an INS error equation is used as a state prediction model, a Sage-Husa algorithm is introduced to estimate system noise statistical characteristics in real time, estimated error parameters are fed back to INS calculation through closed-loop correction, real-time correction of navigation results is achieved, and the system noise statistical characteristics are estimated in real time. Through deep tight coupling fusion and intelligent adaptive processing, navigation precision and system reliability in a complex environment are effectively improved.

The invention belongs to the field of three-dimensional scene reconstruction, and particularly relates to a three-dimensional Gaussian reconstruction method based on feature matching and multi-view consistency, and the method comprises the steps: obtaining a feature matching graph and a reliable matching mask graph between image pairs; constructing a surface fitting constraint loss function, and minimizing a component along the z-axis direction in the three-dimensional Gaussian ellipsoid zoom vector to enable the three-dimensional Gaussian ellipsoid zoom vector to become a two-dimensional Gaussian plane; calculating an unbiased depth value of each pixel based on a two-dimensional Gaussian plane normal and a camera line-of-sight direction; calculating a depth re-projection error of the current view angle and the adjacent view angle according to the filtering feature matching graph, and constructing a depth consistency loss function; comparing normal directions of the current view angle and the adjacent view angle according to the filtering feature matching graph, and constructing a normal consistency loss function; and weighting the surface fitting constraint loss function, the depth consistency loss function and the normal consistency loss function to obtain overall optimization loss, and performing back propagation to finally obtain a three-dimensional reconstruction result. The geometric accuracy of a reconstruction result is remarkably improved.

This invention provides an automatic calibration method and apparatus for a ship panoramic surround view system based on UAV collaboration and 3D surface mapping. A virtual 3D surface projection model is established in the ship's coordinate system, and the surface normal vectors of sampling points on this model are mapped outwards to generate the UAV's flight trajectory. The UAV, equipped with an RTK positioning module, acts as a dynamic calibration target with precise three-dimensional coordinates as it flies along this trajectory, its spatial distribution naturally matching the geometry of the surface projection model. During flight, images from the UAV are simultaneously acquired by the ship's onboard camera, and pixel coordinates are extracted using a target detection algorithm. Simultaneously, the time synchronization data between the UAV and the ship's onboard RTK module is used to establish the correspondence between pixel coordinates and three-dimensional coordinates in the ship's coordinate system, minimizing reprojection errors and jointly calculating the extrinsic parameters of each camera. This eliminates the dependence on fixed planar references on shore, enabling automated calibration with precise adaptation to the 3D surface projection model in any water area.

This invention discloses a calibration method for an underwater curved surface camera based on a spherical waterproof cover, belonging to the field of underwater sensing technology. The calibration method includes: establishing a unified camera coordinate system and a dome coordinate system; mechanically adjusting the camera to the center of the dome; collecting calibration data when the two centers coincide in different media; moving the camera to other designated positions inside the glass cover to collect calibration data; processing the collected calibration data; calculating translation compensation between the camera coordinate system and the dome coordinate system; calculating rotation compensation between the camera coordinate system and the dome coordinate system; using Snell's law to calculate the light paths in air, glass, and water media sequentially; and optimizing various calibration parameters. This invention improves the accuracy of curved surface calibration by establishing a unified camera coordinate system and a dome coordinate system for translation and rotation compensation, then using ray tracing technology to derive the expressions for refracted light rays in each medium sequentially, and finally iteratively optimizing to obtain various calibration parameters and reprojection errors.

The application provides a charging port recognition and positioning method and device based on multi-modal perception, which comprises the following steps: acquiring binocular visible light images and infrared depth maps, synchronizing and pixel-level registering based on calibration external parameters to obtain RGBD data; inputting the visible light images into a detection model embedded with coordinate attention to obtain a candidate detection box; combining the geometric parameters of the detection box and local depth features to output two-dimensional correction offsets from a correction network, and correcting at least three two-dimensional feature points; establishing a two-dimensional-three-dimensional correspondence according to three-dimensional template points and outputting rotation parameters and translation parameters by differentiable PnP solving; weighting the reprojection error by using the variance output by the uncertainty branch; when the reprojection error meets the trigger condition, writing the sample into the difficult sample buffer and performing incremental updating on at least part of the parameters of the correction network and / or the uncertainty branch. The technical scheme of the application realizes stable output of the charging port pose, and improves the success rate of automatic charging alignment and plugging.

The application discloses a kind of dynamic ring vision splicing method and system based on image overlap area feature perception, method includes based on the depth information and the feature information dynamic planning splicing path;According to the splicing path of planning, image splicing is executed, including the multi-band image fusion based on depth perception.The present application extracts local and global features of the overlapping area and calculates pixel-level depth information, constructs a geometric transformation model optimized by feature-depth combination, so that panoramic stitching can accurately identify the outline of close-range objects, and dynamically plan the optimal stitching line to bypass prominent objects;By searching the distance splicing template based on the real distance and automatically unifying the focal length of each camera, the field of view range is kept consistent;Through the multi-scale depth estimation network guided by features and joint reprojection error optimization, high-precision registration can still be maintained in scenes with few feature points or low overlap areas.

The application discloses a kind of based on track keeping's unmanned aerial vehicle thermal infrared image splicing method and system, it is related to image splicing technical field, including steps: constructing image set;Obtain the matching feature point of any pair of to-be-spliced unmanned aerial vehicle's thermal infrared image respectively, and the re-projection error of matching feature point is generated alignment item by the similar transformation matrix of to-be-spliced thermal infrared image;The actual distance between the center of reference image and the thermal infrared image of any unmanned aerial vehicle is calculated, and the track trajectory protection item is obtained between the center point of similar transformation matrix and reference image by pixel distance;Alignment item and track trajectory protection item are weighted and summed to construct global optimization objective function, and global optimization objective function is converted into linear least square model with similar transformation constraint to solve, and generate spliced thermal infrared panoramic image.The application minimizes the cumulative error of splicing process, while ensuring the integrity of global structure.

This invention specifically relates to a multi-view consistent image generation method based on retrieval enhancement and cross-view attention constraints, belonging to the field of image generation technology. This invention takes a single-view image as input, retrieves reference images from a 2D image library to construct retrieval enhancement conditions, and supplements prior details. It constructs a cross-view mapping relationship through monocular depth estimation and camera parameters, converting reprojection errors into geometric consistency biases and injecting cross-view attention. Multi-view latent variables are generated in parallel during diffusion-based inverse denoising, and multi-view consistent images are obtained through feature interaction and decoding. This invention improves multi-view structure and texture consistency within a unified framework, alleviates detail drift and invisible region collapse, and is suitable for scenarios such as 3D reconstruction. The generated quality and stability are superior to existing methods.

This invention belongs to the field of image generation, specifically relating to a method and system for generating 360° images of subway vehicles based on image stitching. The method includes: acquiring source images collected along the subway vehicle; obtaining effective matching point pairs through feature extraction matching and spatial topological similarity filtering; selecting the camera pair with the strongest geometric constraints as a benchmark; using incremental triangulation to recover the camera pose and 3D point cloud; in global bundle adjustment, optimization is performed by combining reprojection error and linear regularization terms based on the geometric principal axes of principal component analysis, with the regularization term weights adjusted according to the average distance from the point cloud to the principal axes; after optimization, the source images are projected onto a cylindrical canvas, and the optimal image gradient is selected based on gradient information entropy to construct a gradient field; a seamless 360° panoramic image is generated by solving the Poisson equation. This invention can effectively eliminate mismatches, suppress 3D reconstruction offset, and generate high-quality seamless panoramic images.

The application discloses a visual SLAM method fusing point features and coplanar anchor points. The method comprises the following steps: extracting point features and line segment features in parallel for a current frame image; processing the line segment features extracted from the current frame image, screening and combining to generate coplanar anchor point features; matching the point features and the coplanar anchor point features extracted from the current frame image with features in a visual map established, to obtain a point feature matching pair set and a coplanar anchor point feature matching pair set; constructing a fusion re-projection error model according to the point feature matching pair set and the coplanar anchor point feature matching pair set, optimizing and solving the camera pose of the current frame image, to obtain an optimized camera pose; updating the visual map according to the optimized camera pose, and judging whether the current frame image is added as a new key frame. The application overcomes the deficiency in a low-texture scene, and avoids the explosive growth of calculation caused by complex multi-feature fusion.

The invention discloses an automobile panoramic camera joint calibration method and system. The method comprises two steps of primary calibration and global optimization. According to the primary calibration, calibration cloth with the same number as the cameras is arranged on the ground, based on the plane homography transformation principle, homography matrixes of all the cameras are sequentially solved in an iteration mode, and internal parameters of the cameras are not needed in the whole process. In the global optimization, an initial calibration result is used as an initial value, minimization of re-projection errors of all angular points is taken as a target, and angular point world coordinates of all calibration cloth and homography matrixes of all cameras are synchronously optimized by adopting a nonlinear optimization algorithm, so that accumulated errors caused by chain transmission are effectively eliminated. The invention further protects the electronic equipment for executing the method, a computer storage medium and an automobile panoramic imaging system comprising the method. According to the method, the calibration process is simplified, the requirement for hardware knowledge is reduced, the calibration precision and robustness are remarkably improved, and meanwhile, a comprehensive patent protection system is constructed.

The present application relates to the technical field of computer vision, in particular to a method for fast processing 2D video and color fusion of real scene reconstruction three-dimensional model based on GPU, comprising the following steps: data acquisition and space-time reference calibration preprocessing; real scene three-dimensional basic model construction; GPU parallel space-time joint pose compensation projection mapping; re-projection error calculation and pose compensation iterative optimization; GPU accelerated color fusion optimization output. The present application designs the GPU parallel space-time joint pose compensation projection mapping, combines the acquisition timestamp difference and device motion parameter to calculate the pose compensation amount, matches the re-projection error iterative optimization, adapts the real scene three-dimensional reconstruction dynamic acquisition scene, corrects the space-time asynchronization and pose offset, accelerates the GPU parallel operation and guarantees the projection mapping accuracy; at the same time, relying on the GPU parallel rendering engine to complete the color difference correction, color tomography smoothing and edge transition processing, realizes the uniform fusion of 2D video color and three-dimensional model, and solves the problems of uneven color and connection trace.

This invention discloses a method and apparatus for obtaining a drone path based on video and an oblique photogrammetry model, relating to the field of computer vision technology. The method includes: acquiring a video of an area captured by the drone during flight and an oblique photogrammetry model of that area; loading the model and adjusting the projection matrix of the Cesium camera to match that of the drone's camera; adjusting the Cesium camera using the optimal pose of the drone's camera in the previous video frame so that the 3D scene rendered on the screen using the model is similar to the current video frame, and capturing an image displayed on the screen; extracting and matching feature points from the current video frame and the captured image; acquiring 3D spatial coordinates based on the matching results to construct a reprojection error model; obtaining the optimal pose of the drone's camera in the current video frame by solving the error model; and obtaining the flight path based on the optimal pose. This invention can eliminate the error accumulation problem of SLAM systems and obtain accurate trajectories.

The application discloses a fast robust monocular vision inertial positioning method and system, comprising the following steps: configuring IMU and monocular camera sensor parameters; IMU data preprocessing; IMU attitude initialization; extracting image features of a monocular camera, removing abnormal matching points and performing IMU pre-integration; monocular vision initialization; odometer attitude optimization, comprising the following steps: judging whether a feature point is mis-matched based on an IMU pre-integration result, constructing a feature point re-projection error Jacobian matrix, and optimizing inverse depth of the feature point and image frame attitude through block and diagonalization processing of the re-projection error Jacobian matrix; after completing current frame attitude updating, selecting a key frame based on a key frame identification rule; and judging according to a current frame, removing a certain frame in a sliding window to reserve a sliding window space for adding a latest frame. The application optimizes inverse depth of a feature point and image frame attitude through block and diagonalization processing of a re-projection error Jacobian matrix, and reduces computational complexity.

A tennis 3D trajectory reconstruction method and device based on monocular vision mainly include: real-time acquisition of monocular motion video; extracting the 2D pixel bounding box of the tennis in each frame, and splicing out the complete tennis 2D motion trajectory; taking the tennis space motion physical model containing gravitational acceleration and air resistance as the constraint condition, constructing the re-projection error function from the 3D space coordinate to the 2D pixel coordinate, taking the initial 3D position and the initial velocity vector of the tennis as the joint optimization variable, and adopting the nonlinear optimization algorithm for iterative fitting, so that the re-projection error between the coordinates of the theoretical 3D trajectory re-projected to the 2D plane and the actual tennis 2D motion trajectory is minimized, thereby inversely solving the accurate 3D trajectory. Through the 3D reconstruction method based on 2D trajectory fitting and projection iterative optimization, the tracking and depth estimation problem of high-speed tennis in the terminal with limited computing power can be effectively solved.

The invention discloses a high-fidelity scene reconstruction method based on laser radar-vision-inertial navigation fusion. The method comprises the following steps: carrying out online scanning and segmentation on a laser radar point cloud according to a camera sampling interval; acquiring an initial camera attitude by a laser radar-inertial navigation odometer, and jointly optimizing a re-projection error and a luminosity error through error state iteration Kalman filtering to obtain a refined camera attitude; an octree is adopted to remove map points outside a view cone, survival map points, an image and a refined camera attitude are combined for Gaussian micro rendering, meanwhile, a Gaussian ellipsoid cloning process is improved, the influence of motion blur is eliminated, overlapped fog shadow and back propagation loss are avoided, Gaussian ellipsoid parameters are iteratively optimized, and high-fidelity scene reconstruction is achieved. According to the scene reconstruction method provided by the invention, the problem of insufficient robustness of the traditional SLAM technology in the outdoor environment is improved, and meanwhile, the performance of the 3DGS in the outdoor environment is optimized.

The application provides a multi-modal high-definition camera fusion intelligent driving image re-projection error noise reduction method, comprising the following steps: S1: multi-modal camera joint space-time calibration and error reference library construction; S2: multi-modal image synchronous acquisition and standardized pretreatment; S3: driving scene semantic partitioning and error sensitivity level mapping; S4: bidirectional re-projection error tracing and attribution label generation; S5: pixel-level noise reduction weight adaptive allocation; S6: multi-modal fusion constraint iterative noise reduction and local closed loop verification; S7: error reference library self-updating and global parameter closed loop optimization. The application realizes high-precision re-projection error tracing and adaptive noise reduction, effectively suppresses errors while completely retaining the details of intelligent driving core perception targets, and improves the multi-modal image fusion quality and the reliability of the intelligent driving perception system.

The invention discloses a vehicle-mounted unmanned aerial vehicle recovery entrance relative pose measurement method based on VisionMama and EPnP, and the method comprises the steps: obtaining a current frame image, enabling a camera to participate in an entrance three-dimensional reference point set, cutting an entrance image block according to a search window, and inputting the image block into a VisionMama feature point generation network in combination with a time sequence memory vector; regression is carried out on the entrance two-dimensional feature point set and the shielding visible weight, and the memory vector is updated; a two-dimensional and three-dimensional corresponding relation is established according to a point sequence, a weighted corresponding relation set is generated, an initial relative pose is solved by adopting weighted EPnP, pose validity judgment is carried out based on a re-projection error and an entering channel deviation, and a recovery closed-loop control quantity is generated; updating a next frame of search window parameter by using the effective pose projection; the method can provide continuous pose and effectiveness constraints for mobile platform recovery alignment-entry.

The invention provides a stereoscopic vision-inertia SLAM (Simultaneous Localization and Mapping) implementation method and a stereoscopic vision-inertia SLAM implementation system, which belong to the technical field of indoor robot positioning and mapping and are used for improving the picture composition precision. Performing point feature extraction and line feature extraction on each frame of image, and performing matching between adjacent frames; constructing an initial 3D map based on the carrier motion data, the point features and the line features; and defining a global state variable, and carrying out joint objective function iterative solution by utilizing adjacent frame image data and carrier motion data to update the global state variable, thereby updating the 3D map. According to the method, structural drift can be effectively solved by introducing a line feature reprojection error, and a filtering strategy of multistage geometric constraints fusing neighborhood topology consistency is further proposed to filter line segment matching pairs; the random and isolated mismatching rate caused by environment texture repetition (such as indoor corridors, ceiling lines and floor tile gaps) is obviously reduced.

This application provides a data processing method, apparatus, computer device, and readable storage medium, relating to the field of autonomous driving. The method includes: acquiring an image data packet of an autonomous vehicle's estimated initial trajectory and driving environment, wherein the image data packet includes multiple image data points acquired by the autonomous vehicle's camera at different positions along the estimated initial trajectory; performing image matching on the multiple image data points, determining the coordinates of different image feature points in 3D space based on the image matching results, obtaining multiple 3D space points; constructing a reprojection error function based on the camera's extrinsic parameters and the 3D space coordinates of the 3D space points; and optimizing the reprojection error function to obtain target camera extrinsic parameters and target 3D space coordinates that minimize the reprojection error function.

Methods, systems, and apparatus for performing bundling adjustment using epipolar constraints. A method includes receiving image data from a headset for a particular pose. The image data includes a first image from a first camera of the headset and a second image from a second camera of the headset. The method includes identifying at least one key point in a three-dimensional model of an environment at least partly represented in the first image and the second image and performing bundle adjustment. Bundle adjustment is performed by jointly optimizing a reprojection error for the at least one key point and an epipolar error for the at least one key point. Results of the bundle adjustment are used to perform at least one of (i) updating the three-dimensional model, (ii) determining a position of the headset at the particular pose, or (iii) determining extrinsic parameters of the first camera and second camera.