299 results about "Geometric consistency" patented technology

The invention provides a multi-view three-dimensional point cloud reconstruction method and device based on DPE-SE depth estimation, and relates to the technical field of computer vision and three-dimensional reconstruction. The method comprises the following steps: acquiring multi-view image data; preprocessing the image; inputting the preprocessed image into a DPE-SE-based depth estimation model, carrying out key constraint on an edge region through a semantic edge guiding mechanism, carrying out adaptive propagation updating on a weak texture region, realizing accurate depth estimation, and generating a multi-view depth result; then geometric consistency check and multi-scale depth fusion are performed on a multi-view depth result, and a dense depth map is constructed; and finally, performing three-dimensional back projection reconstruction and point cloud optimization processing, and outputting high-quality point cloud data containing three-dimensional coordinates and confidence information. According to the method, the problems of edge mismatching and depth voids are remarkably improved in complex illumination, weak texture and shielding environments, the continuity and structural integrity of the point cloud are improved, and technical support is provided for unmanned aerial vehicle surveying and mapping, building detection and digital twin modeling.

The invention discloses an automobile seat framework machining defect detection method based on machine vision, and particularly relates to the technical field of defect detection. The method comprises the following steps: constructing a multi-angle image acquisition and edge reflection modeling module aiming at complex defects such as weld joint pseudo soldering, microcracks, hole site deviation and collapse deformation, extracting weld joint continuity, edge integrity and hole site geometric consistency characteristics by using a deep neural network, and generating a structural feature vector; defect type recognition and credibility scoring are completed through small sample anomaly modeling and Gaussian mixture model classification, sub-pixel-level coordinate labeling of defect positions is achieved in combination with a Gaussian fitting algorithm, a defect positioning map is output, traceability analysis and severity grading are conducted based on historical data comparison, and the defect positioning accuracy is improved. The method is suitable for industrial online detection and quality closed-loop control.

The invention discloses a three-dimensional scene reconstruction method and system based on monocular depth estimation, and belongs to the technical field of computer vision and three-dimensional reconstruction, and the method comprises the steps: carrying out the multi-scale feature coding of a monocular RGB image through a mixed attention depth coding module, and obtaining the hierarchical depth feature representation; carrying out autoregression depth decoding through a self-adaptive edge perception depth decoding module to generate an initial depth map; a depth confidence map is calculated through a geometric consistency constraint optimization module and is fed back to a coding module for iterative optimization, and a refined depth map is output; and three-dimensional Gaussian ellipsoid scene representation is constructed through the Gaussian ellipsoid scene reconstruction module. According to the invention, high-precision depth estimation and high-quality three-dimensional reconstruction are realized by constructing a depth-coupled closed-loop cooperative system.

The invention discloses a single view reconstruction and rendering method, which belongs to the field of view reconstruction and rendering, and comprises the following steps of: firstly, generating multi-view feature representation with strong geometric consistency from a single input image by introducing an image diffusion module of a cross attention mechanism; a point cloud reconstruction module with self-attention and cross-attention is utilized, and multi-view information is fused to reconstruct an accurate three-dimensional point cloud; and finally, constructing differentiable three-dimensional Gaussian representation based on the point cloud, rendering the differentiable three-dimensional Gaussian representation, and outputting a new view angle image, a normal map and a depth map. According to the system, a staged training strategy is adopted, and a composite loss function including multi-scale bidirectional consistency smooth loss and feature consistency loss is innovatively used for optimization. According to the method, the problems of low geometric accuracy, multi-view inconsistency, detail missing and the like in single-view reconstruction are effectively solved, and the method can be widely applied to the fields of virtual reality, digital twinning, cultural heritage digitization and the like.

The invention provides an injection molding part surface flaw detection method and device based on double-model collaboration, and the method comprises the steps: carrying out the local abnormal reflection feature analysis of a multi-view optical image collected on the surface of an injection molding part based on a first detection model, and obtaining a candidate flaw region; performing spatial positioning in the multi-view optical image based on the candidate flaw area to obtain a multi-view candidate image segment; performing surface geometric continuity analysis on the multi-view candidate image segments based on a second detection model to obtain a three-dimensional surface geometric consistency result; carrying out authenticity discrimination on the candidate flaw area based on a three-dimensional surface geometric consistency result to obtain a real flaw area; wherein the first detection model is used for capturing an optical response model of local abnormal reflection characteristics; the second detection model is a stereoscopic vision model for analyzing the geometric continuity of the multi-view lower surface. According to the invention, the false alarm rate of surface defect detection of the injection molded part under a complex surface condition is reduced.

The invention discloses a virtual space multi-person interaction synchronous control method oriented to an end-cloud collaborative architecture. The invention relates to a computer graphics and three-dimensional modeling technology, and discloses a cross-format lightweight and geometric consistency maintenance method based on a three-dimensional model. Through format-independent geometric representation and a self-adaptive lightweight strategy, efficient compression and precision maintenance of three-dimensional model cross-format conversion are realized. The method specifically comprises the following steps: performing format analysis and geometric feature extraction on an input model, and establishing a unified internal representation; adaptively selecting a multi-level LOD lightweight strategy based on the complexity of the model; the accuracy of key information is ensured through geometric feature keeping and topology consistency detection; the geometric consistency is dynamically maintained by combining error monitoring and an iterative correction mechanism; and generating a target format lightweight model and carrying out quality verification. According to the method, adaptive precision control, multi-level consistency maintenance and format irrelevant processing are combined, the model size and conversion errors are effectively reduced, and cross-platform compatibility and geometric fidelity are improved. The method can be widely applied to the fields of industrial design, game development, virtual reality and the like.

The invention discloses an abdominal CT image multi-view fusion method based on HU physical characteristic guidance, and relates to the technical field of medical image processing. The method comprises the following steps: firstly, constructing a three-dimensional HU volume, generating a semantic mask covering the whole HU range as physical prior, and obtaining two-dimensional slice sequences in the axial direction, the coronal direction and the sagittal direction through three-view projection; on the basis, multi-scale deformable alignment, HU-perceived deformation field fine correction, cross-view attention fusion and regional differentiation decision fusion are sequentially carried out, HU-perceived high-frequency residual enhancement and local contrast self-adaptive processing are carried out on two-dimensional slices, and finally a denser CT slice sequence is generated. According to the method, HU physical partition constraint is introduced in the multi-view alignment and fusion process, abnormal deformation and high-density structure distortion of a gas region are effectively inhibited, and geometric consistency and visual stability of an abdominal CT image in the interpolation reconstruction process are improved.

The invention provides a three-dimensional data generation method and device, and the method comprises the steps: carrying out the planning of a text prompt before generation, optimizing the three-dimensional data in stages, introducing a reflection and correction mechanism in the generation process, recognizing and correcting defects through a multi-modal large model, and introducing a cross-view alignment constraint. And global planning and local fine adjustment of the three-dimensional generation process are realized. According to the method, a complex generation task is decomposed into a closed-loop process of planning-execution-reflection-correction, and multi-view consistency is forced from a feature level, so that geometric consistency and semantic accuracy of generated three-dimensional data can be effectively improved, the problems of structural artifacts and Janus which are easy to occur in the prior art are avoided, and the generation efficiency of the three-dimensional data is improved. And finally, high-quality three-dimensional data with consistent views are generated.

The invention relates to a sparse view angle pose-free scene reconstruction method and system based on 3DGS, belongs to the field of computer vision and three-dimensional reconstruction, and solves the problems of easy failure and poor geometric consistency in a sparse view angle or weak texture environment due to dependence on accurate camera pose priori. Constructing a double-flow sensing module containing semantic flow and geometric flow, extracting semantic features by using a visual basic model, and extracting an explicit geometric corresponding relation by using a dense feature matching network, so as to regress relative camera pose under pose-free priori; a geometric guidance depth refinement module combining a potential diffusion model architecture and Pluecker ray coding is introduced, and scale fuzziness of monocular depth estimation is eliminated through a depth residual prediction mechanism; and based on the micronizable Gaussian rasterization, performing end-to-end optimization by using a self-supervised loss function including rendering consistency, reprojection and epipolar geometric constraint. According to the method, high-fidelity three-dimensional reconstruction is realized without supervision of external parameter true values, and geometric stability and rendering quality in a complex scene are improved.

The invention discloses a cross-modal fusion and multi-frequency coding view angle image generation method in an ultra-low orbit scene, and solves the problem of view angle condition generation and single image three-dimensional reconstruction method based on diffusion prior in the ultra-low orbit scene of an aircraft target. Due to insufficient condition information fusion capability, limited radiation or illumination characterization and insufficient cross-view-angle geometric constraint, a problem that a new view angle image with visual coherence and geometric consistency of texture details is difficult to obtain at the same time under large-range view angle and scale change is solved; according to the method, multi-order angle information of direction distribution is effectively captured through perceptual coding of a light direction vector, then the position of a light starting point coordinate is coded, and finally the pose of a camera pose parameter is coded, so that the influence of a relative pose on depth perception and perspective distortion in an image generation process can be depicted; the three coding results are used for a decoding stage of the potential diffusion model, and geometric consistency of detail recovery under a new view angle is effectively improved.

The invention discloses an attention anti-mask double-branch distillation method for three-dimensional point cloud completion. The method comprises the following steps: firstly, extracting a multi-scale space attention map from a teacher model, and generating an anti-mask point cloud based on the multi-scale space attention map; then constructing a double-branch student network, and respectively processing the intermediate feature point cloud and the anti-mask point cloud of the student model; then, a joint recursive distillation module is adopted to carry out recursive upsampling and feature aggregation under geometric consistency constraint on point cloud features of the two student branches and the teacher model; and finally, training a student model by utilizing a triple supervision optimization module and jointly optimizing CD distance loss, characteristic distillation loss and cross-branch contrast loss. According to the invention, the point cloud completion precision and robustness of a student model in a complex automatic driving scene are effectively improved by constructing an anti-mask mechanism guided by teacher attention, introducing a double-input branch architecture, designing a combined recursive distillation mechanism and constructing a triple distillation loss function at the same time.

The invention discloses a semantic constraint adversarial sample generation method and system. According to the method, firstly, a confrontation sample generation task based on a natural language instruction is constructed, and parameters are initialized; then deploying a proxy model and a diffusion model for double-branch noise estimation; by judging instruction complexity, mask guidance is selectively adopted to realize natural constraints of spatial differentiation; by constructing a residual-guided antagonistic DDIM sampler and combining an adaptive optimization iterative algorithm, the calculation complexity is reduced, and the attack mobility is enhanced at the same time; and aiming at a three-dimensional generation requirement, a three-dimensional Gaussian sputtering rendering model is further integrated, and geometric consistency is ensured through multi-view gradient averaging. According to the method, the problems of inaccurate semantic control, weak migration aggressiveness, poor three-dimensional generation consistency and the like in the prior art are effectively solved, the attack success rate and the visual naturalness are remarkably improved on multiple target models, and an efficient and reliable red team test tool is provided for security evaluation and alignment of a multi-modal large model.

A robot grabbing method based on a geometric topology constraint field and adaptive calibration comprises the following steps: constructing a manifold neighborhood enhancement field based on a target object prior point cloud; carrying out feature coding on an input real-time observation point cloud and a prior point cloud template, and embedding observation features into a manifold neighborhood enhancement field to realize geometric alignment; aiming at the RGB image and the real-time observation point cloud, establishing a preprocessing and symmetry sensing mechanism of a multi-modal feature; constructing a geometric feature re-calibration module, performing adaptive weighting on feature response through a channel attention and space attention mechanism, and performing adaptive calibration on deviation between observation features and prior manifold features; the 6D pose parameters of the target object are regressed based on the re-calibrated global features, and a corresponding geometric consistency measurement value is output; and according to the consistency measurement result, the pose result is subjected to iterative correction in the reasoning stage, and a robot grabbing instruction is output. According to the method, the space postures of different objects can be accurately modeled, and the grabbing precision of the robot in a complex environment is improved.

The invention discloses a dual-path non-cooperative spacecraft pose estimation method based on uncertainty perception, and belongs to the technical field of spacecraft pose estimation. According to the method, a dual-path prediction framework based on a shared backbone network is constructed and comprises a geometric reasoning path and a global context sensing path. In the training stage, prediction results of the two paths are aligned through a geometric consistency loss function, and joint optimization is carried out by combining multi-task losses such as key point regression, classification, rotation and translation regression and the like. In the inference stage, the pose estimation result is adaptively fused according to the predicted variance output by each path, when the variance is low, the output of the global context sensing path is directly adopted, otherwise, the key point projections of the two paths are combined, and the final pose is recovered. The method has higher estimation precision, robustness and interpretability in complex space environments such as shielding, illumination variation and large-scale variation, and provides reliable visual navigation support for on-orbit autonomous tasks.

The invention discloses a hierarchical three-dimensional scene generation method and system based on spatial super-division, and belongs to the technical field of computer graphics, and the method comprises the steps: carrying out the preprocessing of a scene image, and obtaining a high-resolution object image; generating initial rough scene voxels for the scene image, and screening rough voxels and structural latent variables aligned with the high-resolution object image from the initial rough scene voxels to construct a hierarchical scene tree; inputting the high-resolution object image and the rough voxel into a voxel super-resolution model, and generating a fine voxel which keeps geometric consistency with the rough voxel; performing scale alignment and attitude registration based on the rough voxels and the fine voxels; and generating fine voxels of the sub-components recursively by taking the rough voxels of the current node as conditions based on the hierarchical scene tree, and finally assembling to generate a high-resolution three-dimensional scene. According to the method, a high-quality three-dimensional scene with high visual fidelity, fine geometric details and global structure consistency can be efficiently and automatically reconstructed from a single RGB image.

The invention particularly relates to a CAD part feature recognition method based on a graph neural network, and the method comprises the steps: extracting geometric entity information and topological relation information based on C # and NX Open API, and generating standardized AAGJSON format data; converting the geometric data into a graph structure which comprises a topological adjacent matrix and a multi-dimensional node feature vector comprising geometric, topological and shape features; a graph neural network model comprising a graph convolution layer, a graph attention layer and a graph sampling aggregation layer is adopted for training, and feature categories and confidence degrees of holes, grooves, bosses and the like are output; and the prediction result is corrected based on geometric consistency, topological connectivity and context rationality constraint. The method has the characteristics of high recognition precision, strong adaptability and good robustness of complex parts, can be integrated in UG NX to realize real-time recognition and visualization, provides important technical support for design automation and manufacturing industry digitization, and solves the problems of low recognition precision, poor adaptability and insufficient topological information utilization in the existing CAD part feature recognition method.

The invention discloses a stereoscopic vision-based model attitude measurement method, belongs to the technical field of attitude measurement, aims to solve the problems of insufficient precision and expensive depth camera of traditional attitude estimation, realizes accurate measurement of model attitude, and has the core of fusing stereoscopic vision geometric characteristics and deep learning advantages. The method specifically comprises the steps of 1, forming a three-dimensional system by using at least two industrial cameras, calibrating internal and external parameters, and calculating a scene depth map through an improved SGBM algorithm to obtain object depth information, and 2, predicting coordinates of pixels in a target 3D coordinate system by using a pre-trained double-branch fusion network in combination with the depth map and left and right eye RGB images, the method comprises the steps of 1, generating a plurality of groups of candidate poses on the basis of association and spatial association, 2, restoring the candidate poses into virtual objects, comparing the virtual objects with real objects through geometric consistency verification to quantify scores, and 3, selecting the candidate pose with the highest score as a final result through a PoseSelection module. And large view field coverage and high-precision positioning are both considered.

The invention provides an industrial product quality prediction method based on geometry preserving cross-scale difference, and relates to the technical field of industrial product quality prediction.The method comprises the steps that collected time sequence data of industrial process variables are preprocessed, a sample sequence is constructed through a sliding window, and the sample sequence is divided into a training set, a verification set and a test set according to the time sequence; the method comprises the following steps: constructing a double-branch coding architecture to independently process trend features and differential features; a geometric perception attention mechanism is introduced into each branch encoder, it is ensured that hidden layer representation and output target space keep geometric consistency, and the stability and interpretability of the model are enhanced; and deep interaction and adaptive fusion of double-branch information are further realized by adopting cross-scale cross attention, so that the comprehensive modeling capability of long-term trend and short-term dynamic in the industrial process is remarkably improved.

The invention discloses a weak supervision orthopaedic three-dimensional reconstruction method based on geometric constraint and comparative learning regularization, and aims to solve the problems of high labeling cost and poor robustness of a weak supervision model in the existing orthopaedic three-dimensional reconstruction technology. According to the method, a coarse threshold segmentation result automatically generated by a Mimics tool is used as a weak supervision label, manual fine adjustment is not needed, and the labeling cost is remarkably reduced. The method comprises the following steps: acquiring and registering a CT sample; generating an enhanced sample through significance guide mixing and random shielding; feature extraction and segmentation prediction are carried out by using the 3DUNet sharing the weight; designing a multi-loss function fusing consistency regularization loss, geometric constraint loss and threshold guide feature comparison loss, and strengthening geometric consistency and feature distinguishing capability; and finally, high-precision and high-generalization skeleton three-dimensional reconstruction is realized through training optimization and reasoning, and the method is suitable for scenes such as clinical diagnosis and surgical planning.

The invention discloses an underground pipeline detection method and system based on image recognition. The method comprises the following steps: obtaining a preprocessed RGB image sequence and registered laser radar point cloud data; generating a target detection frame set with semantic tags; segmenting the obtained initial two-dimensional semantics into a mask set; generating a two-dimensional semantic segmentation mask set after depth calibration and a semantic point cloud cluster set after depth calibration; obtaining a two-dimensional semantic segmentation mask set after cylinder verification and a semantic point cloud cluster set after cylinder verification; outputting a two-way binding semantic point cloud cluster set; and spatial reconstruction processing is executed based on the two-way bound semantic point cloud cluster set, and underground pipeline spatial recognition detection is completed. According to the method, three-dimensional geometric consistency is introduced as a verification condition on an algorithm structure, so that a segmentation result simultaneously meets semantic rationality and spatial physical authenticity.

The invention belongs to the technical field of image processing, and particularly relates to a pole adaptive positioning method and system based on image processing, and the method comprises the steps: obtaining edge pixel points in a pole gray level image and the gradient direction of the edge pixel points; estimating an illumination distortion main direction according to the gradient direction, and dividing edge pixel points into a positive edge set and a negative edge set; according to the forward edge set, calculating weighted votes of edge pixel points to gradient direction intersection points and accumulating the weighted votes to a forward accumulator, clustering accumulation results to obtain an initial forward candidate center, and finally performing iterative updating through corrected voting weights based on geometric consistency to obtain forward center estimation; obtaining negative center estimation according to the negative edge set; and determining a circle center positioning point of the pole hole by combining the positive center estimation and the negative center estimation. The method effectively overcomes the positioning error caused by uneven illumination and metal reflection, and improves the positioning precision and robustness.

The invention discloses a three-dimensional semantic field construction method and device under sparse view angle input, and belongs to the technical field of computer vision and artificial intelligence. In order to solve the problems that an existing three-dimensional semantic understanding method depends on dense multi-view input, semantic and geometry fusion is insufficient, and cross-scene generalization ability is poor, the invention provides a new method which can complete high-quality three-dimensional semantic reconstruction only through 2-4 sparse RGB images. According to the method, a double-branch feature extraction network is designed, a bottom layer CNN is shared, and color and semantic features are learned through SwinTransform; a camera perception attention mechanism is introduced, and a camera pose is explicitly fused into Transform to enhance geometric consistency; double Gaussian representation of geometric attributes is shared by adopting color Gaussian and semantic Gaussian, and the semantic coherence is improved in combination with regional smooth loss.

The invention discloses a geometric perception multi-view consistency image generation method based on a diffusion model. The geometric perception multi-view consistency image generation method comprises the following steps: S1, multi-modal condition coding: uniformly coding conditions such as texts, reference images and camera postures into fusion features and inputting the fusion features into U-Net; and S2, gating multi-path attention fusion: dynamically fusing the multi-source features in the U-Net through a gating multi-path attention mechanism, and enhancing the multi-view two-dimensional features. And S3, voxel feedback closed-loop refinement: inversely projecting the two-dimensional features to a three-dimensional voxel space, and re-projecting the two-dimensional features back to two dimensions after three-dimensional volume accumulation so as to form closed-loop geometric feedback. And S4, residual error correction autoregressive sampling: generating an image according to autoregressive of a view angle sequence, reprojecting and correcting a current sampling track by using front view angle information, and inhibiting error accumulation. And S5, composite supervision collaborative optimization: carrying out collaborative optimization on the model in combination with multiple loss functions, and freezing a backbone network so as to realize multi-view generation with high geometric consistency and high quality.

The invention discloses a CAD (Computer Aided Design) program generation method based on a multi-view freehand sketch and a cross-modal Transform, which comprises the following steps: acquiring freehand sketch images of three views, and preprocessing each freehand sketch image; the preprocessed freehand sketch images are used as input of a trained CAD program model; according to the method, by designing a CAD program model comprising a visual encoder network, a fusion module, a cross-modal alignment module and a CAD sequence generator, the structural integrity and geometric consistency of freehand sketch image modeling are effectively improved, the generated CAD program sequence conforms to parametric modeling logic, and the reliability and accuracy of a modeling result are improved; according to the method, hierarchical position coding vectors and hierarchical constraint masks are added in a CAD program model training stage, the stability of a generation result in the aspects of structural rationality, geometric consistency and performability is improved, and the method is suitable for freehand sketch input under different drawing habits and imaging conditions and has good robustness and universality.

The invention relates to a self-supervised multi-modal fusion and collaborative optimization method suitable for a curve and ramp scene. The method comprises the following steps: acquiring an original perception image; generating four paths of geometrically consistent recovery images; constructing a multi-modal fusion model, and generating a depth confidence map aligned with the restored image; uniform ground plane parameters are obtained; collecting all coordinate points to construct an initial 3D lane candidate point set; generating a high-density locally enhanced BEV lane representation; and outputting the fused global consistency 3D lane map. And outputting the structured 3D lane line marking data. Non-planar structures such as curve superelevation and longitudinal ramps can be accurately depicted, and geometric consistency in a complex road scene is greatly improved; the problems of single vehicle shielding, sensor noise and the like are effectively relieved, the precision, the integrity and the system-level reliability of 3D lane reconstruction are remarkably improved, meanwhile, dependence on a high-precision map or 3D true value marking is avoided, and the method has high engineering landing value.

The invention belongs to the technical field of computer vision and three-dimensional image processing, and particularly discloses a mismatching mark point correction method and system based on an improved RANSAC algorithm. According to the method, congruent triangle criteria are introduced into the RANSAC algorithm, and potential matching point pairs conforming to geometric consistency are screened. Only when the potential matching point pairs meet the geometric consistency, the transformation matrixes R and T are solved, that is, mismatching points existing in the potential matching point pairs are effectively eliminated, so that on the premise that the point cloud splicing precision and the success rate are not changed, wrong calculation is reduced, the point cloud registration efficiency and robustness are improved, and the point cloud registration accuracy is improved. The method is suitable for various application scenes such as industrial part measurement, medical modeling, cultural relic digital protection and virtual reality.

The invention discloses a scale and structure consistency image generation method fusing three-dimensional geometric prior, and belongs to the technical field of computer vision and image generation. The method comprises the following steps: acquiring a scene image and depth information corresponding to the scene image, reconstructing a scene point cloud containing pixel space coordinates and color information, acquiring a colored point cloud of an object to be inserted, placing the colored point cloud at a specified position of the scene through rigid body transformation, and unifying a coordinate system with the scene point cloud; projecting the transformed object point cloud to an image plane through a camera model, determining a visible point according to the depth, and constructing an insertion region mask to remove pixels of a region corresponding to a scene; then constructing a multi-channel geometric structure priori graph containing color, depth and other information, and splicing the multi-channel geometric structure priori graph with the cleared scene image to form a diffusion type generation model input; and finally, generating an image under geometric prior guidance, ensuring that the inserted object is consistent with the original scene in the aspects of position, scale, perspective and the like, solving the problem of insufficient geometric consistency of the existing method, and realizing image content insertion with an accurate structure and real vision.

The invention discloses a multi-view three-dimensional reconstruction method and device based on reliable anchor point guidance and a medium, and the method comprises the steps: firstly extracting a high-confidence anchor point from a multi-view image, optimizing auxiliary visual angle selection with the visibility of the anchor point as a constraint, and achieving the stable diffusion of a depth initial value through confidence propagation; in the reconstruction stage, deformation parameters of local patches are adaptively adjusted according to scale differences and elevation angle changes between visual angles, so that geometric consistency and detail fidelity of depth estimation are enhanced; and finally, dynamically adjusting a point cloud sampling strategy according to anchor point density distribution and a visual angle overlapping rate, and realizing dense three-dimensional reconstruction with noise suppression and continuous structure. The method can maintain the reconstruction precision and stability under the conditions of complex visual angles and multiple types of images, is suitable for the scenes of engineering surveying and mapping, intelligent monitoring, unmanned aerial vehicle photogrammetry, digital twin model updating and the like, and has the characteristics of high robustness, high precision and good expandability.

The invention discloses a geometric consistency guided deep manifold network parameter initialization method and system, and relates to the technical field of manifold learning in machine learning, and the method comprises the steps: dividing an input feature into a plurality of independent modals, and building a feature-level modal energy distribution model; deriving a geometric consistency constraint formula of learnable parameters based on the energy distribution consistency constraint; the parameters are mapped to a high-degree-of-freedom fixed rank manifold for optimization, and updating of the parameters on the manifold and geometric structure maintaining are achieved through a designed Riemannian projection operator and a pull-back operator; and combining a geometric consistency constraint formula and a manifold optimization strategy, and carrying out iterative training until the model is converged. According to the method, the geometric consistency in the network training process can be effectively maintained under different manifold structures and data distribution, and the convergence speed, the stability and the cross-scene generalization ability of the deep manifold network are remarkably improved.

The invention discloses a dynamic shelter restoration method and system based on continuous streetscape panoramic images, and the method comprises the steps: firstly obtaining a to-be-restored target panoramic image A and a to-be-restored reference panoramic image B, and generating an original pixel-level shelter mask; secondly, extracting matching points among the panoramic images, realizing cross-view geometric alignment of the images, and obtaining a target perspective view C and a reference perspective view D through perspective re-projection; and then inputting the target perspective view C and the reference perspective view D into a three-dimensional reconstruction framework, and performing three-dimensional point cloud reimaging under the camera pose of the target perspective view C by using a depth inspection mechanism. And finally, carrying out image restoration on a three-dimensional point cloud re-imaging result, restoring to a panoramic coordinate system, splicing with an original panoramic image, and outputting a shielding-free panoramic image. According to the method, it is ensured that the repairing result conforms to the authenticity and geometric consistency of the geographic space, and the problems of overlapping conflicts and visual tearing during multi-view projection fusion are effectively solved.