56 results about "Epipolar geometry" patented technology

The invention discloses an unmanned aerial vehicle camera attitude estimation optimization method and device, and relates to the technical field of unmanned aerial vehicles. The method comprises the following steps: acquiring data by using an unmanned aerial vehicle carrying various sensors, preprocessing, carrying out feature extraction and matching on preprocessed image data, removing mismatching points, and based on an initial point cloud, carrying out sparse reconstruction in an incremental expansion and bundle adjustment optimization stage; integrating the re-projection error term, the flight path constraint, the epipolar geometric constraint and the triangulation constraint into a target function; constructing a model for predicting the pose correction based on the deep residual convolutional neural network, and taking the initial pose of the camera and the corresponding local image as input; and adjusting the weight of each constraint in the target function based on the predicted pose correction and pose confidence, and minimizing the target function to improve the precision of unmanned aerial vehicle camera pose estimation. The problem of reconstruction scene deviation caused by inaccurate camera attitude estimation in the prior art is solved.

The invention discloses a binocular X-ray guide wire three-dimensional positioning method, a binocular X-ray guide wire three-dimensional positioning device and binocular X-ray guide wire three-dimensional positioning equipment based on epipolar geometry and a storage medium. The method comprises the steps that two to-be-processed images collected by two X-ray scanning devices in the biplane digital subtraction angiography system for a target object in a guide wire intervention state are acquired, and the to-be-processed images comprise intervention guide wires; conducting guide wire positioning on the two to-be-processed images based on projection matrixes which are obtained through pre-calibration and respectively correspond to the two X-ray scanning devices, and determining path information of the interventional guide wire, the path information comprising first position information of the interventional guide wire; obtaining an expected path of the interventional guide wire, and determining an offset error of the interventional guide wire based on expected position information of the interventional guide wire in the expected path and the first position information; the expected path of the intervention guide wire is updated based on the offset error, accurate positioning of the intervention guide wire and updating of the expected path are achieved, and accurate data support is provided for subsequent treatment.

The invention relates to a sparse view angle pose-free scene reconstruction method and system based on 3DGS, belongs to the field of computer vision and three-dimensional reconstruction, and solves the problems of easy failure and poor geometric consistency in a sparse view angle or weak texture environment due to dependence on accurate camera pose priori. Constructing a double-flow sensing module containing semantic flow and geometric flow, extracting semantic features by using a visual basic model, and extracting an explicit geometric corresponding relation by using a dense feature matching network, so as to regress relative camera pose under pose-free priori; a geometric guidance depth refinement module combining a potential diffusion model architecture and Pluecker ray coding is introduced, and scale fuzziness of monocular depth estimation is eliminated through a depth residual prediction mechanism; and based on the micronizable Gaussian rasterization, performing end-to-end optimization by using a self-supervised loss function including rendering consistency, reprojection and epipolar geometric constraint. According to the method, high-fidelity three-dimensional reconstruction is realized without supervision of external parameter true values, and geometric stability and rendering quality in a complex scene are improved.

The invention discloses an underwater target identification and positioning method based on fusion of a physical model and deep learning, and belongs to the technical field of computer vision, and the method comprises the steps: carrying out the transmissivity estimation and physical restoration of a collected underwater monocular image according to an underwater light propagation model, and carrying out the image correction; key feature points are extracted, and high-quality matching point pairs are screened in combination with the joint similarity; further deriving a basic matrix through the high-quality matching point pairs meeting the epipolar geometric constraint relation, solving an essential matrix, and obtaining relative attitude parameters between the cameras in combination with weighted re-projection-LM optimization; and finally, refraction correction triangulation is carried out according to the Snell's law, pixel-level fusion is carried out after scale normalization and space alignment are carried out on the refraction correction triangulation and dense depth output by the MiDaS, three-dimensional space coordinates of the target are inverted, and high-precision recognition and positioning of the underwater target are achieved. The system is light in structure, efficient in calculation, suitable for being integrated on various underwater autonomous or remote control robot platforms and used for tasks such as target recognition, tracking and positioning.

The invention discloses a dynamic scene visual SLAM method based on deep learning, and the method comprises the following steps: obtaining an original scene image, and extracting ORB feature points in an image scene; based on the improved target detection network, performing dynamic target detection on the original scene image to obtain a dynamic target frame in the image scene; associating the dynamic target frame and the ORB feature points by adopting a key frame synchronization mechanism, and removing the ORB feature points corresponding to the dynamic target frame; obtaining a feature scene image after primary elimination; performing secondary elimination on unmatched ORB feature points in adjacent feature scene images by adopting epipolar geometric constraint to obtain a static feature scene image; and performing pose estimation and / or map construction based on the ORB feature points corresponding to the static feature scene image. According to the method, the dynamic feature points of the dynamic target are extracted by improving the target detection network, and the dynamic feature points are eliminated twice by adopting the epipolar geometric constraint, so that the positioning precision is improved, and accurate map construction is facilitated.

The invention discloses an SLAM dynamic point semantic filtering method based on DDMA-SAM, and belongs to the technical field of synchronous positioning and mapping. A decoupling distillation mechanism is introduced, an image encoder in an original SAM model is subjected to lightweight optimization, a DDMA-SAM semantic network integrating a multi-scale aggregation detection module and an efficient mask decoding module is constructed, and the SLAM dynamic point semantic filtering method based on DDMA-SAM is obtained. And the segmentation performance is improved while the model parameters are greatly compressed. Based on the semantic network, providing a semantic prior and geometric consistency combined-driven double filtering strategy; based on a semantic mask and a confidence threshold, carrying out preliminary dynamic point identification; in combination with the epipolar geometric constraint and the triangulation reprojection error of random sampling consistency estimation, fine elimination of dynamic feature points is realized, and only static points are reserved to participate in camera pose estimation. According to the method, the real-time performance of the system is kept, and meanwhile, the mapping quality and the track stability in a dynamic scene are effectively improved.

The invention belongs to the technical field of multi-target identity judgment among multiple passive sensors. The invention provides a space-time and track double-constraint multi-target identity judgment method for multiple passive sensors. According to the embodiment of the invention, the time-space criterion based on the continuous multi-frame epipolar geometric constraint and the track criterion based on the track feature cosine similarity are constructed, so that the identity judgment characteristics of multiple targets in different sensors can be described more accurately, and the accuracy and robustness of the identity judgment of the multiple targets are remarkably improved. The problem of weight optimization under different sensor configurations and scene characteristics is solved through an adaptive fusion weight mechanism, and performance degradation of a fixed weight strategy in a complex scene is avoided. The method has good scene adaptive ability and excellent real-time response characteristic, completely meets the rapid online processing requirement in the airborne environment with limited resources, maintains the flexibility and expansibility of cross-platform deployment, and provides reliable technical support for leading-edge applications such as multi-carrier collaborative awareness.

The application provides a feature matching method, system and device based on multi-geometry collaborative learning and a storage medium, and relates to the technical field of robot positioning and navigation. The steps include: acquiring an image pair, extracting two groups of sparse local features in the image pair and inputting the local features into a multi-geometry collaborative network model; enhancing the local features through an attention mechanism and iteratively updating the sparse network through a multi-layer perception and / or linear projection; generating and screening reliable matching points through affine transformation estimation, position coding and key point neighborhood expansion; constraining the matching process through pose information and adaptively adjusting the matching strategy for matching prediction; combining real matching information and homography geometry information for loss calculation to train the multi-geometry network model; and repeating the steps until a preset iteration number is reached. Through collaborative optimization of affine geometry, epipolar geometry and homography geometry, the application realizes mutual promotion of local feature discriminability and geometric consistency, and can achieve a better balance between speed and accuracy in feature matching.

The invention relates to a dynamic SLAM method and system based on lightweight YOLOv8 and an adaptive key frame strategy, and belongs to the technical field of computer vision. According to the method, a dynamic region in an image frame is detected through an improved lightweight network, and extracted dynamic feature points are removed in combination with an epipolar geometry constrained motion consistency detection method; extracting line features from the static region based on an improved LSD algorithm, and compensating the loss of feature points; designing a key frame selection strategy of adaptive weight to delete redundant key frames, and constructing a joint optimization objective function by combining static point features and static line features to perform pose estimation and construction of a point cloud map; and finally, performing closed-loop matching by using visual and semantic features between the key frames, verifying a closed loop, establishing a pose constraint, and performing global graph optimization. According to the method, the influence of a dynamic object in a scene on the mapping quality of the SLAM algorithm can be effectively reduced, and the robustness of the algorithm in a dynamic environment is enhanced.

The application discloses a kind of circular stripe three-dimensional reconstruction methods based on epipolar Markov modeling, steps are as follows:First, the method is aimed at the problem that existing concentric circle stripe projection measurement method is strongly dependent on camera-projection device geometry arrangement, in the camera-projection device system that has been calibrated, the epipolar geometry is combined with concentric circle equal phase model, and the intersection of the projector epipolar line corresponding to camera pixel and equal phase circle is obtained to obtain candidate projection point;Again, candidate point selection modeling is one-dimensional Markov chain sequence energy minimization problem, and candidate branch disambiguation is realized by smoothing constraint, branch switching penalty and gradient consistency constraint;Finally, according to the corresponding relationship of screened camera-projection device, triangulation is carried out to obtain three-dimensional point cloud.The application does not require camera and projector to meet strict parallel optical axis or single direction displacement assumption, and can improve the matching stability and three-dimensional reconstruction robustness of circular stripe structured light under the condition of complex curved surface and geometric configuration change.

This invention relates to the field of image analysis technology, specifically a deep learning-based method for detecting surface defects in tab adapters. The method includes acquiring the three-dimensional geometric curvature parameters of the bending region through multi-view polarization imaging; decoupling specular reflection and diffuse structural features using a photometric distribution model to resolve high-brightness reflection interference; nonlinearly projecting the surface texture onto a two-dimensional plane using a local tangent space mapping matrix to generate an isotropic corrected texture map to compensate for geometric distortion; identifying suspected defect points that disrupt flow continuity through singular value decomposition of the anisotropic structure tensor; and introducing stress distribution priors and epipolar geometric constraints for multi-view spatial consistency verification to effectively filter lighting artifacts. The verified region and corrected texture are fused and input into a deep recognition network with a spatial attention mechanism to analyze the surface defect image. This invention achieves accurate analysis of surface defect images through deep coupling of three-dimensional geometry and a photometric physical model.

The application discloses a dynamic scene SLAM positioning method based on a target detection algorithm YOLOv4 and geometric constraints. Firstly, an image is collected through a visual sensor, a potential moving object area in the image sequence is recognized by using the YOLOv4 target detection algorithm, feature points of a part outside the area are extracted, and the matching feature points are calculated by using an optical flow method. After a transformation matrix of two frames is calculated, whether the feature points are dynamic points is judged by using an epipolar geometry epipolar constraint. When the epipolar constraint is invalid due to degenerate motion, whether the feature points are dynamic feature points is judged by using a boundary constraint. Finally, a dynamic object area is determined in combination with a target detection result, and after the corresponding area is removed, tracking, local mapping and loop detection threads are sequentially performed, a SLAM algorithm for a dynamic scene is realized, and the existing visual SLAM algorithm is inaccurate in pose estimation in a dynamic environment and is prone to be lost. The application significantly improves the visual SLAM pose precision in a dynamic scene while ensuring real-time performance, and can smoothly run on an embedded system.

Techniques are described for identifying correspondences between images to generate a fundamental matrix for the camera positions related to the images. The resultant fundamental matrix enables epipolar geometry to correlate common features among the images. Correspondences are identified by confirming feature matches across images by applying a homography to data representing features across images. Further techniques are described herein for generating a representation of a boundary of a feature of a structure based on a digital image. In one or more embodiments, generating a representation of a boundary of a particular feature in a digital image comprises determining a portion of the image that corresponds to the structure, and determining a portion of the image that corresponds to the particular feature. One more vanishing points are associated with the portion of the image corresponding to the particular feature. The one or more vanishing points are used to generate a set of bounding lines for the particular feature, based on which the boundary indicator for the feature is generated.

The application relates to a monocular vision inertial odometer method based on feature point depth, which comprises the following steps: acquiring input images of two adjacent frames and extracting feature points in the images respectively; matching the extracted feature points; predicting first camera pose changes by using acceleration measurement data and angular rate measurement data output by an inertial measurement sensor; obtaining the depth of each pair of matched feature points based on the first camera pose changes; if the number of feature points meets a preset condition, constructing a weight for each pair of feature points and obtaining a weight matrix; constructing a weighted observation equation set based on an epipolar geometry constraint equation and the weight matrix; constructing a quadratic form based on the weighted observation equation set and solving the quadratic form to obtain an essential matrix of the matched feature points in the least square sense; obtaining a second rotation matrix and a second translation vector by singular value decomposition based on the essential matrix and the second rotation matrix and the second translation vector, and obtaining the representation of actual camera pose changes.

The invention discloses a large-viewing-angle visual positioning method based on geometric constraints of dynamic and static objects, and relates to the technical field of computer vision. The method aims at solving the problem that the pose estimation precision is reduced due to feature matching failure under the large view angle change in the existing feature point-based positioning technology. The method comprises the following steps: identifying an object from an image pair synchronously acquired by a multi-camera system, fitting the contour into an ellipse, and distinguishing a static ellipse from a dynamic ellipse; reconstructing an object motion vector by using the projection change of the dynamic ellipse, establishing a rotation constraint based on the invariance of the object motion vector, and solving the initial estimation of the relative pose between the cameras in combination with a non-rolling constraint; constructing an epipolar geometric space by using the initial estimation, and establishing a corresponding relation by defining an ellipse distance function and matching a static ellipse; and finally, combining geometric constraints of dynamic and static ellipses to construct an optimization model, and performing nonlinear optimization on the relative pose parameters of the camera to obtain a high-precision estimation result. According to the method, robust geometric constraints provided by dynamic and static objects in a scene are utilized, so that the robustness and precision of visual positioning under large visual angle difference are remarkably improved.

The invention discloses a low-altitude unmanned aerial vehicle closed dynamic flight environment semantic modeling method based on deep learning, and belongs to the technical field of low-altitude unmanned aerial vehicles. The method mainly comprises the following four steps: (1) acquiring an airborne RGB-D image and IMU data of the unmanned aerial vehicle, and constructing an IMU pre-integration model; (2) carrying out dynamic and static judgment and dynamic feature elimination on feature points in a scene by utilizing a target detection network in combination with epipolar geometric constraints, and realizing front-end anti-interference; (3) carrying out pixel-level semantic segmentation on the screened key frame based on a DeepLab v3 + network to obtain an environment semantic mask; and (4) through closed-loop detection and global pose map optimization, fusing semantic information and geometric information, and constructing a globally consistent closed dynamic flight environment three-dimensional dense semantic map. According to the method, the problems that the positioning of the low-altitude unmanned aerial vehicle in the closed dynamic flight environment is easily interfered and the mapping accumulative error is large are effectively solved, and high-precision and strong-robustness environment semantic modeling is realized.

The invention discloses a fire safety monitoring system based on computer vision, particularly relates to the field of computer vision, and is used for solving the problems of low fire source identification precision, large illumination interference and difficulty in unified analysis of multi-view monitoring images in existing fire monitoring. A candidate hot spot extraction module is arranged to perform binarization processing on a monitoring image to extract a candidate hot spot region, a morphological enhancement module is utilized to adaptively adjust a nuclear parameter fusion hot spot region, and a standard view angle correction module is utilized to realize perspective correction based on feature point detection and homography matrix calculation; when a multi-view-angle image exists, a view angle fusion module is used for calculating three-dimensional point cloud data of a hot spot based on epipolar geometry, and a real hot spot is identified through time sequence and space consistency verification; and finally, the monitoring output module completes spatial positioning and risk extension trend prediction of real hot spots, outputs fire risk position and trend information, and realizes high-precision fire source identification and real-time early warning in a complex scene.

The application provides a dam and navigation facility ship collision monitoring and early warning method and system, and relates to the technical field of shipping traffic. It comprises: at the main and auxiliary monitoring points, image, message and radar point cloud data are obtained through a camera, an automatic ship identification system and a millimeter wave radar respectively, after time and space synchronization processing, the radar data is projected to the image plane to form a radar image, and then sent to a ship identification network to identify the two-dimensional bounding box and type of the ship; then, the three-dimensional bounding box and type of the ship are obtained by using epipolar geometry processing, and compared with the message data, if they are inconsistent or have no related type, a three-level early warning is issued; at the same time, the future speed and heading of the ship are predicted according to the multiple three-dimensional bounding boxes of the ship, if the set threshold is exceeded or there is a risk of collision with the dam, a two-level early warning is issued; if the ship triggers the three-level and two-level early warnings at the same time, a one-level early warning is issued. The scheme can greatly improve the accuracy of early warning and the perfection of the early warning mechanism.

This invention proposes a robust adaptive VIO navigation and positioning method suitable for GNSS-denied environments. First, it utilizes an IMU and camera to acquire the acceleration, angular velocity, and surrounding environmental information of UGVs. Then, it establishes a measurement constraint model for VIO based on the epipolar geometry and trifocal tensor geometry relationships between multiple images. Next, in the filtering algorithm, the H∞ criterion is introduced into the CMSCKF to improve the robustness of the VIO filtering algorithm. Based on the system uncertainty, a three-stage robust adaptive VIO filtering algorithm is adopted, comprehensively utilizing the standard Kalman filter algorithm and the H∞ filter algorithm to complement each other's advantages, thereby improving the overall performance of the filtering algorithm. This results in VIO integrated navigation not only having high filtering accuracy but also good robustness, thus improving the navigation and positioning performance of UGVs in complex environments.

A method for generating a relief representation model using a plurality of N pairs of stereo images, each of the N pairs of stereo images being associated with a map of photogrammetric disparities which is obtained by correlation calculation based on the pair of stereo images in question, comprises: a calculation of maps of geometric disparities based on a predetermined relief representation model, by projection of the predetermined relief representation model into an epipolar geometry of the pairs of stereo images; a calculation of a cost function, representative of a difference between the geometric and photogrammetric disparities, associated with the predetermined relief representation model; and an updating of the predetermined relief representation model by optimization of the gradient descent type of the cost function, and iteration until a predefined endpoint criterion is reached. Also a computer program and a system.

The application discloses a novel OCT space reconstruction method, which comprises the following steps: acquiring multiple coronary angiography image data at different angles; extracting coronary angiography images at the same moment in different cardiac cycles from the coronary angiography image data acquired at each angle; extracting a blood vessel skeleton according to the coronary angiography images; in the process of extracting the blood vessel skeleton, the coronary angiography images at different angles are included; multi-view stereo matching is performed by using deep learning, and matching correction is performed by using an epipolar geometry algorithm; a three-dimensional blood vessel skeleton is acquired according to the coronary angiography images after the matching correction; coronary OCT sequence images are acquired, and rearrangement is performed in combination with the direction of the three-dimensional blood vessel skeleton; body rendering is performed on the rearranged images, and a three-dimensional space model with a real posture is rendered. The method realizes prediction of invisible parts and guarantees the correctness of three-dimensional reconstruction of visible parts, and provides a data basis for screening of vascular deformity and the like in the later stage.

The invention discloses a non-reference light field image quality evaluation method based on pseudo video sequence state modeling. The method comprises the following steps: S1, preprocessing an original light field distortion image, and extracting a light field pseudo video sequence; s2, extracting and fusing features of a spatial domain, an angle domain and an epipolar geometric domain of the light field pseudo video sequence by adopting a mixed framework of Mama and Transform to obtain a domain fusion feature code; and S3, inputting the domain fusion feature code into a quality prediction module, and outputting an objective quality evaluation score. According to the method, cross-dimension correlation explicit modeling is carried out, so that the quality evaluation precision of immersive experience is improved; and the strong nonlinear mapping capability is adopted, the performance and complexity are efficiently balanced, and the evaluation accuracy is high.

This invention relates to a multimodal image matching method and system based on learned features and epipolar geometric constraints. The method includes edge enhancement processing of the input image using wavelet transform; extraction of multi-scale dense feature maps based on a modified convolutional neural network, combined with principal direction normalization to generate feature descriptors with rotation and scale invariance; preliminary feature matching using the FLANN algorithm and dynamic distance constraints; and the introduction of a fundamental matrix construction and epipolar geometric consistency verification mechanism, combined with a RANSAC affine constraint model to eliminate mismatched point pairs. This invention integrates image enhancement, deep learning, and geometric verification strategies, effectively addressing the radiation nonlinearity and geometric distortion problems caused by differences in imaging mechanisms between multimodal images, improving matching accuracy and robustness. It is suitable for remote sensing applications such as optical-SAR registration, multi-source image fusion, and land surface change detection.

This invention discloses a point-line fusion robot SLAM method and system for dynamic environments, including: acquiring image frame sequences in a dynamic environment; extracting point and line features from the image frame sequences to construct a set of point-line feature pairs; performing dynamic detection on the image frame sequences, removing point features that constitute dynamic regions from the set of point feature pairs; using epipolar geometry to check whether the remaining point features in the set of point feature pairs are dynamic point features, and removing dynamic point features; using the PROSAC algorithm to filter out the remaining static point features in the set of point feature pairs to obtain usable point feature pairs; using a sparse image alignment algorithm to achieve a preliminary estimation of the robot pose by minimizing image photometric error; optimizing the robot pose by minimizing reprojection error based on the usable point feature pairs and line feature pairs; performing point cloud processing through local mapping to obtain a sparse point cloud map; and optimizing the robot pose using loop closure detection.

The application discloses a kind of methods for indoor positioning of uncalibrated camera using Aruco code.The application uses a special two-dimensional code mark-Aruco code to carry out the self-calibration of camera, and the mapping and camera positioning of indoor environment.Through the corresponding relationship of a group of Aruco code corner points under different viewing angles, the distortion coefficient of the camera is solved using epipolar geometry constraint.The camera intrinsic parameters are calculated using the special properties of essential matrix after removing the distortion of the image.In an indoor scene with a group of Aruco codes pasted on a wall and a ceiling, the positional relationship between all codes is calculated using the pose relationship between a single code and the camera, thereby establishing an indoor map and completing the indoor positioning task of the camera.

The application discloses a pose estimation method and system based on a complex dynamic scene and a storage medium, and the pose estimation method comprises the following steps: S1, feature points in an input image are extracted, and a motion consistency detection is performed on a motion state of an object by using a sparse optical flow pyramid method or an epipolar geometry method to obtain a motion consistency detection result; S2, real-time semantic segmentation is performed on the input image by using a lightweight semantic segmentation model to obtain a semantic segmentation result; S3, semantic label classification is performed on the object in the scene according to the semantic segmentation result, and potential dynamic object detection is performed according to the semantic label classification result and a corresponding depth image to obtain a potential dynamic object detection result; S4, dynamic points are adaptively removed based on geometric information and semantic information; and S5, pose estimation is performed based on the reserved feature points. The application can effectively guarantee the accuracy and real-time performance of the pose estimation of a visual odometer in a complex dynamic scene.

The invention provides a dual-camera weak texture multi-target space matching and positioning method and an application system, and belongs to the technical field of computer vision and three-dimensional perception. According to the method, based on a dual-camera epipolar geometric model, on the basis of obtaining a dual-camera synchronous image and performing multi-target detection, four corner points and a central point of a target outer surrounding frame are introduced to construct a multi-key-point combined epipolar constraint, and target space structure information is fully utilized. The multi-key-point combined epipolar constraint is constructed by selecting the four corner points and the center point of the outer surrounding frame of the target, the overall space structure information of the target is introduced into the dual-camera matching process, a matching constraint mechanism of multi-geometric clue fusion is formed, and compared with an epipolar constraint mode only based on a single image point, the matching constraint mechanism is more accurate. The matching ambiguity caused by appearance similarity or deformation is effectively reduced, so that the matching accuracy in a multi-target scene and the anti-interference capability of the system are improved, and the method is particularly suitable for a weak texture environment.

The invention discloses a dynamic scene VSLAM optimization method based on target detection and SuperPoint, and belongs to the field of visual SLAM.According to the method, a front-end visual odometer part of a traditional ORB-SLAM3 algorithm is improved, firstly, a tracking thread of the front-end visual odometer part is improved, and then the tracking thread of the front-end visual odometer part is optimized; a SuperPoint feature extraction algorithm module is adopted to replace a feature extraction algorithm module in an original system; a coding layer is subjected to lightweight processing, so that the feature point detection precision is maintained, the model parameter magnitude is effectively reduced, and the operation speed is increased; on the basis, newly adding a target detection thread which runs in parallel, and introducing a Yo-FastV2 target detection model to detect a dynamic object in an environment; and finally, eliminating interference feature points on the dynamic object in combination with epipolar geometric constraint. The method can improve the pose estimation and positioning precision in a complex dynamic scene, and has a wide application value.

The invention discloses a vehicle-mounted binocular vision calibration method based on scene information understanding, and belongs to the technical field of vehicle-mounted vision monitoring. According to the method, external parameters, namely a rotation matrix and a translation vector, of a camera are calibrated, three-dimensional space re-projection constraint and depth information estimation are introduced in the external parameter optimization process on the basis of a traditional epipolar geometric calibration method, and the modeling capacity for long-distance point space uncertainty is enhanced; meanwhile, aiming at external parameters of multi-frame estimation, a multi-frame result clustering algorithm is designed, and estimation deviation caused by single-frame noise and a bad initial value is effectively reduced. According to the method, the defects that a traditional self-calibration method based on epipolar geometry only depends on two-dimensional projection point information, a long-distance target is difficult to process accurately, and the traditional self-calibration method depends on a better initial value are overcome.

The application relates to the technical field of volume data visualization, and discloses a volume rendering Gaussian representation fast construction method based on visual-volume data alignment, which comprises the following steps: obtaining three-dimensional volume data to be visualized and corresponding multi-view images; inputting the three-dimensional volume data and the multi-view images into a feedforward neural network to directly map a three-dimensional Gaussian splat representation for representing the three-dimensional volume data through the feedforward neural network, wherein the feedforward neural network comprises: a dual Transformer network for jointly modeling two-dimensional appearance information of the multi-view images and three-dimensional geometric information of the three-dimensional volume data; and a volume geometry forcing mechanism for aligning and injecting the two-dimensional appearance information and the three-dimensional geometric information into the three-dimensional Gaussian splat representation based on epipolar geometry constraints; and the application is superior to existing direct volume rendering methods and three-dimensional Gaussian representation methods based on optimization in terms of rendering efficiency and visual quality.