103 results about "Motion recovery" patented technology

The invention provides a panoramic image three-dimensional reconstruction method based on 3DGS, and relates to the technical field of computer vision and image processing, and the method comprises the steps: 1, collecting a panoramic image sequence of a to-be-reconstructed scene, carrying out the unified coding through equidistant cylindrical projection, and obtaining a coding data set; and step 2, executing panoramic motion recovery structure processing on the coded data set, and generating six-degree-of-freedom camera pose parameters of each image and a corresponding sparse three-dimensional point cloud. According to the invention, high-precision and high-efficiency panoramic scene three-dimensional reconstruction is realized, a high-quality panoramic image or a dense three-dimensional model can be output, and the three-dimensional reconstruction quality and efficiency are improved.

The invention discloses a construction site three-dimensional scene reconstruction method based on an unmanned aerial vehicle image and a monitoring video. The method comprises the following steps: acquiring a construction site scene multi-view image; a sparse three-dimensional point cloud and a camera pose are generated through a feature matching and motion recovery structure algorithm, and dense reconstruction is carried out to obtain a global three-dimensional point cloud; aligning the global three-dimensional point cloud with a world coordinate system by using geographic position information; estimating the position of a shooting camera in the three-dimensional point cloud, sampling a candidate view angle and rendering a virtual RGB image; based on two-dimensional feature matching of the shot image and the virtual RGB image, internal parameters and external parameters of the shooting camera are iteratively solved through triangulation and a pose optimization algorithm; monocular depth estimation is carried out on the shot image, and the shot image is converted to a measurement scale through static region depth alignment; and projecting the depth of the shot image to the three-dimensional point cloud, updating the dynamic object in real time, and fusing the dynamic object into a complete three-dimensional scene model. The method has the advantage that the real-time three-dimensional reconstruction of the dynamic scene of the construction site is realized.

The invention relates to the technical field of building structure health monitoring, and particularly discloses a building group safety inspection method based on external structure size displacement detection, and the method comprises the steps: obtaining three-dimensional laser scanning point cloud data of a target building group at a plurality of time points, adding timestamp information, and constructing a time sequence point cloud data set; initial registration is carried out by adopting a feature matching and motion recovery structure technology, and the registration precision and stability in a dynamic environment are improved in combination with a global optimization model fused with space-time consistency constraints; identifying and eliminating pseudo displacement point clusters through track consistency analysis, and extracting a static background structure; finally, key part deformation identification is carried out on the differential point cloud data, and a structure health state evaluation report is generated. According to the invention, full-process automatic processing from data acquisition, optimization registration to intelligent evaluation is realized, the dynamic interference influence is effectively inhibited, the precision and reliability of structural deformation monitoring are improved, and technical support is provided for long-term safety inspection of building groups.

The invention discloses a tunnel surrounding rock multi-source data dynamic fine rapid sensing method, which comprises the following steps: based on a mobile terminal multi-view image sequence, arranging shooting positions according to a preset space interval and an orthogonal angle, generating a sparse point cloud through a motion recovery structure algorithm, and generating a dense point cloud model in combination with multi-view stereo matching optimization; and for the dense point cloud model, delimiting a local neighborhood based on k-nearest neighbor search, resolving a neighborhood point covariance matrix through principal component analysis, extracting a feature vector corresponding to a minimum feature value as a normal vector, and constructing a rock mass surface microscopic geometric feature field. The invention provides a dynamic fine rapid sensing method based on multi-source data, and aims to improve the efficiency, precision and timeliness of tunnel surrounding rock information acquisition and provide accurate surrounding rock information support for tunnel construction and support design.

The invention relates to the technical field of three-dimensional geological modeling and geological mapping data processing, in particular to an oblique photography three-dimensional surface model geological modeling system and method, and the system comprises a data collection module, a scene construction module, a boundary line processing module, a geologic body modeling module, a profile linkage module and a result output module. The data acquisition module fuses the oblique photography image and topographic data, and generates a three-dimensional earth surface model through a motion recovery structure algorithm; the geologic body modeling module constructs a closed geologic body through implicit curved surface reconstruction and constrained triangulation; a profile linkage module calls a graph neural network to predict a stratigraphic intersection line trend, and realizes two-dimensional and three-dimensional real-time synchronization based on event driving and sparse voxel hash mapping; and the result output module executes data verification and generates a multi-dimensional geological report. According to the scheme, dependence of a third-party tool is eliminated, geological mapping efficiency is improved, and the problem of section linkage delay is solved.

The invention provides a crop population three-dimensional reconstruction and organ phenotypic character analysis method, and relates to the field of three-dimensional reconstruction, and the method comprises the steps: collecting cotton envelope image data, carrying out the data preprocessing, and carrying out the motion recovery processing and Gaussian sputtering of the cotton envelope image data, the method comprises the following steps of: preprocessing a group cotton point cloud, removing a ground point cloud, obtaining a complete cotton group point cloud model, segmenting a cotton individual point cloud model and a corresponding organ point cloud model from the complete cotton group point cloud model, automatically analyzing cotton key phenotypic characters, and performing correlation analysis. According to the method, a cotton original habitat three-dimensional point cloud model is constructed based on mobile visual equipment, and key characters such as cotyledon node height, plant height and leaf area can be automatically analyzed, so that the problems that phenotypic character extraction in the prior art is high in cost, low in flux, easy to make mistakes, inconsistent in standard, generally destructive, labor-consuming, time-consuming and the like are solved.

The invention discloses a three-dimensional geographic environment real-time intelligent deduction method based on a multi-modal large model, and relates to the technical field of computer vision. The method is used for solving the technical problem of unified modeling and real-time deduction of a dynamic object and a static environment in a three-dimensional geographical environment. The method comprises the following steps: firstly, resolving a camera pose through a motion recovery structure algorithm to generate a sparse point cloud, and separating a dynamic foreground object in a monitoring video to extract motion features; thirdly, initializing a three-dimensional Gaussian distribution set based on the sparse point cloud, extracting semantic features through a visual encoder, and mapping the semantic features to corresponding Gaussian distribution; thirdly, a topological graph structure of Gaussian distribution is constructed, motion features are used as initial excitation, and coordinate offset and appearance variation of each distribution are iteratively updated through message passing calculation; finally, Gaussian distribution attributes are dynamically updated, a continuous deduction image sequence is synthesized through micro-rasterization rendering, and high-reality real-time simulation of dynamic evolution of the three-dimensional geographical environment is achieved.

The invention discloses an unmanned aerial vehicle bridge detection method based on machine vision. According to the method, an unmanned aerial vehicle is used as an autonomous mobile platform, and full-coverage and high-resolution image data acquisition of structural surfaces such as a bridge deck, a beam body and a bridge pier is realized through systematic task planning. The collected data is processed by a motion recovery structure and a multi-view three-dimensional algorithm to generate a high-precision three-dimensional live-action model and a digital orthophoto map. On the basis, a deep learning target detection algorithm is adopted to perform intelligent analysis on the image, apparent defects such as cracks, spalling and corrosion are automatically identified, positioned and classified, and accurate quantification of parameters such as crack width and spalling area is realized. According to the method, the problems of high risk, low efficiency, high subjectivity, existence of detection blind areas and the like in traditional manual detection are effectively solved, and key technical support is provided for digital and intelligent transformation of bridge operation and maintenance management.

The invention discloses a visual camera calibration method and system applied to large-view-field optical measurement, and belongs to the technical field of visual measurement, and the method comprises the steps: firstly obtaining a multi-view-angle image sequence and a camera three-dimensional position corresponding to each frame of image; obtaining an initial camera external parameter with a real scale and a three-dimensional scene structure through a motion recovery structure and in combination with a camera position; selecting and initializing a large-view-field camera projection model containing an internal reference matrix and a lens distortion parameter; iteratively optimizing lens distortion parameters, camera external parameters and three-dimensional point coordinates; and constructing and minimizing a global cost function containing a re-projection error and camera position constraint, and jointly optimizing all parameters to obtain a final calibration result. According to the method, through comprehensive model optimization, by applying surveying and mapping services except satellite application services, the calibration precision and model completeness of large-view-field complex distortion are remarkably improved, and the problem that distortion correction is insufficient due to dependence on a few points is solved; and the robustness and the intelligent level of calibration and the reliability and the traceability of a result are improved.

The invention provides a disaster site-oriented multi-unmanned aerial vehicle rapid three-dimensional reconstruction and decision-making auxiliary method and related equipment, and belongs to the technical field of emergency surveying and mapping and three-dimensional reconstruction. The method comprises the steps that multiple unmanned aerial vehicles are used for cooperatively collecting image data of a disaster site, and uncertainty indexes used for representing the data missing degree are constructed; taking the uncertainty index as input, selecting an optimal viewpoint to generate a supplementary collection task and executing the supplementary collection task based on the contribution degree of candidate viewpoints to reduction of uncertainty and the comprehensive cost of executing the viewpoints, and obtaining an image data set with coverage integrity; screening a key frame set for three-dimensional reconstruction from the image data set; performing motion recovery structure and space partitioning based on the key frame set, and fusing depth priori and semantic information to obtain an initialized point cloud; and based on a three-dimensional Gaussian sputtering model, extracting structured elements, constructing a risk cost map, and obtaining a rescue route and a disaster situation decision report. According to the invention, emergency command and rescue actions can be supported.

The invention discloses a small sample new view synthesis method based on reinitialization three-dimensional Gaussian splashing. The method comprises the following steps: firstly, acquiring sparse three-dimensional point cloud and camera internal and external parameters from a training view through a motion recovery structure algorithm; then, sampling points are generated in the point cloud bounding box by adopting a spatial expansion hybrid sampling strategy, and a coarse-grained Gaussian set is constructed and optimized; thirdly, obtaining a rendered image through Gaussian initialization and splash rendering, calculating pixel importance through depth errors and transmissivity, generating a fine-grained Gaussian set through back projection, and optimizing the fine-grained Gaussian set; and finally, calculating a sampling probability based on the cross-view contribution degree, screening key Gaussian distribution, and optimizing to form a final Gaussian set, thereby realizing high-quality new view synthesis. According to the method, the problem of sparsity difference is solved by eliminating extended view dependence, the multi-view consistency and local geometric details of a new view scene are improved, and high-quality new view synthesis of sparse data is realized.

This invention relates to the field of intelligent reconstruction technology for 3D models, specifically a method and system for intelligent reconstruction of 3D building models based on deep learning. The method includes acquiring multi-source data of the target ancient building; generating a dense point cloud through motion reconstruction and multi-view stereo matching; identifying structural and non-structural components through semantic segmentation and reconstructing an initial mesh model with semantically labeled information; using a graph neural network to analyze the spatial topological relationships of structural components, and combining this with a historical construction knowledge base to generate hierarchical combination methods and connection constraint rules; adaptively adjusting the pose of mesh nodes according to the connection constraint rules to drive the precise assembly of parameterized components, and fusing them to obtain a structured 3D model of the target ancient building. This invention can restore the structural component connection relationships in hidden areas of ancient buildings under non-contact acquisition conditions, significantly improving the structural rationality and geometric accuracy of the 3D model.

The invention discloses a multi-view three-dimensional reconstruction method for a slender object based on curve guidance. According to the method, geometric prior and a deep learning segmentation network are fused, two-stage foreground extraction, a curve-guided motion recovery structure (SfM) and a surface optimization module capable of differential Poisson reconstruction are provided, and a set of complete processing flow is established for the problem that a slender and weak-texture object is difficult to accurately reconstruct. The method comprises the following steps: acquiring a multi-view image through a consumer-level terminal and performing frame extraction to obtain an input data set; generating a high-quality foreground mask by utilizing depth estimation and semantic segmentation; performing curve-guided SfM initialization based on the foreground skeleton curve to realize joint estimation of the camera pose and the sparse three-dimensional curve; carrying out surface geometric optimization through a micro Poisson equation and differentiable grid rendering; and designing a multi-branch loss function fusing luminosity, mask, regularization and Gaussian rendering supervision to train the model. Finally, the method can output a three-dimensional reconstruction result of the slender structure with high geometric accuracy, strong structural integrity and smooth surface, and has good robustness and application value.

The invention relates to the technical field of three-dimensional reconstruction and rendering, and particularly discloses an underwater long-sight-distance adaptive three-dimensional reconstruction and rendering method based on a Gaussian variable-distance field, and the method comprises the steps: carrying out the initialization point cloud generation based on a motion recovery structure algorithm for RGB image data obtained by underwater image collection equipment; points on the initialized point cloud are combined with a three-dimensional Gaussian distribution model, a three-dimensional Gaussian primitive is defined, and parameter initialization about information such as a Gaussian geometric center, a three-dimensional covariance matrix, opacity and color is carried out on the three-dimensional Gaussian primitive; the method comprises the following steps: establishing a Gaussian cross-space transformation model, performing transformation from a Cartesian coordinate space to a homogeneous coordinate space on a three-dimensional Gaussian primitive, establishing a Gaussian full-parameter space mapping model, mapping color information of the three-dimensional Gaussian primitive after cross-space transformation into a spherical harmonic function, and spatial adaptive three-dimensional reconstruction and rendering in an underwater long-sight-distance environment are realized.

In various examples, methods and systems are provided for estimating depth values for images (e.g., from a monocular sequence). Disclosed approaches may define a search space of potential pixel matches between two images using one or more depth hypothesis planes based at least on a camera pose associated with one or more cameras used to generate the images. A machine learning model(s) may use this search space to predict likelihoods of correspondence between one or more pixels in the images. The predicted likelihoods may be used to compute depth values for one or more of the images. The predicted depth values may be transmitted and used by a machine to perform one or more operations.

The present invention discloses a pavement crack extraction method based on a neural radiation field and a semantic segmentation network, and specifically relates to the field of road engineering technology. The pavement image data of a research area is acquired by an unmanned aerial vehicle and preprocessed to form a required image data set. The preprocessed image data set is three-dimensionally reconstructed by combining a motion recovery structure multi-view stereo vision algorithm and a neural radiation field algorithm to obtain a pavement point cloud model and a point cloud data set. A semantic segmentation network is constructed and the point cloud data set is trained. The pavement crack information is extracted and calculated according to the semantic segmentation result. Compared with the prior art, the detection cost is reduced, the real-time detection requirements are met, and the current status of point cloud processing automation, poor expressiveness and insufficient application is significantly improved. In addition, the road scene with cracks can be intuitively and accurately characterized, thereby providing an underlying platform and data support for the digital representation of pavement information.

The invention discloses a structural semantic guided bridge point cloud Gaussian splash modeling and rendering method, and belongs to the technical field of computer vision and bridge engineering digitization. The method comprises the steps of collecting a multi-view image of a bridge, and obtaining a sparse point cloud and a camera pose through a motion recovery structure technology; dense point clouds are generated, deep learning semantic segmentation is carried out, and key components such as piers, bridge floors and cables are identified; mapping the semantic tag back to the sparse point cloud; performing differential Gaussian element initialization on the sparse point cloud based on semantic tags, and endowing different parts with initial shapes conforming to geometric characteristics of the different parts; a geometric constraint loss function is introduced in the optimization process for joint optimization; and finally, outputting a Gaussian splash model of a high-fidelity and accurate geometric structure. According to the method, the problems of model redundancy and geometric distortion when a traditional Gaussian splashing technology is used for processing a bridge scene are solved, and lightweight and high-precision reconstruction and real-time rendering of the bridge model are realized.

The invention discloses a forest three-dimensional integrated reconstruction method fusing on-forest and under-forest images, and belongs to the technical field of forestry informatization, three-dimensional reconstruction and remote sensing. According to the method, two collection tasks with complementary functions are executed by deploying the same technology; an advanced unmanned aerial vehicle-based cross surrounding route is utilized to perform canopy surveying and mapping, and a crossing unmanned aerial vehicle is innovatively adopted to perform under-forest crossing collection. And carrying out independent reconstruction and high-precision registration on the two groups of homologous image data through a motion recovery structure algorithm, and finally generating a vertical structure integrated forest three-dimensional model. The result shows that the point cloud reconstructed by the CCO route is close to the airborne laser radar in integrity, and the individual tree parameters extracted by the integrated point cloud are highly related to the foundation laser radar. Through an innovative multi-view and multi-level data acquisition and fusion strategy and a single and economic photogrammetry technology, the method has the potential of obtaining comprehensive forest three-dimensional structure information, and a new method is provided for realizing large-scale and high-precision forest dynamic monitoring.

The invention relates to the technical field of computer graphics, in particular to a virtual-real fusion method and system based on Gaussian splash live-action reconstruction, and the method comprises the following steps: S1, carrying out the motion recovery structure processing of an obtained target scene image sequence, and generating a sparse point cloud of a target scene; s2, training the sparse point cloud through a Gaussian splashing technology, generating a live-action three-dimensional Gaussian model, carrying out lightweight processing on the live-action three-dimensional Gaussian model, and outputting a lightweight live-action model; and S3, acquiring an internal parameter matrix, an external parameter matrix and a real-time video stream of the physical camera. According to the method, the spatial mapping relation between the live-action reconstruction model and the artificial fine model is established, the user interaction instruction is responded, and the explicit and implicit states of different detail level models in a unified scene are dynamically controlled, so that smooth switching without context loss from macroscopic to microscopic is realized; the problems of visual angle jump and spatial cognition interruption caused by system switching in a traditional scheme are solved.

The invention discloses a power line corridor intrusion detection method based on 3D Gaussian sputtering. The method comprises the following steps: acquiring an unmanned aerial vehicle video sequence of a power line corridor and preprocessing the unmanned aerial vehicle video sequence; based on the preprocessed data, performing three-dimensional reconstruction by using a motion recovery structure and a 3D Gaussian sputtering technology to obtain a high-precision 3D GS model of the power line corridor; based on a two-dimensional image detection and segmentation result, performing semantic annotation on a three-dimensional Gaussian ellipsoid in the 3D GS model through multi-view back projection and fusion, and identifying a target object at least including the engineering machinery and the power transmission conductor; calculating the minimum space distance between the identified engineering machinery and the transmission conductor in the three-dimensional space; and judging whether intrusion occurs or not according to whether the minimum spatial distance is smaller than a safety threshold, and outputting alarm information. According to the method, the problems of low reconstruction precision, indirect measurement, poor real-time performance and the like in the prior art are solved, and real-time three-dimensional detection and early warning of the power line corridor intrusion target are realized.

The present application relates to the field of rock mass structure plane identification, and in particular to a rock mass three-dimensional structure plane intelligent identification method based on an AI vision large model. The method combines an artificial intelligence vision large model with a motion recovery structure technology and uses data of different photogrammetry devices to realize three-dimensional structure plane identification. The method converts the identification task from point cloud to original image data by using the correlation between image pixels and point cloud in the motion recovery structure (which is often ignored by previous research), thereby significantly reducing the implementation difficulty. Benchmark tests show that the method achieves an average accuracy of 0.91 in a three-dimensional database and identifies 87 structure planes. The results show that the method has high precision and high efficiency and provides a powerful tool for geological data labeling.

The invention relates to a three-dimensional model reconstruction method and device based on a Gaussian sputtering model. The method comprises the following steps: acquiring a scene image of a sparse view angle of a target scene, and determining initialization parameter information of a Gaussian ellipsoid according to the scene image through a motion recovery structure technology and a Gaussian sputtering model; determining a priori depth, a color loss function and a gradient smoothing loss function through a monocular depth model according to a scene image; determining a reference depth according to the scene image through a multi-view three-dimensional depth estimation algorithm; determining a depth consistency loss function according to the prior depth and the reference depth; and updating the initialization parameter information through a Gaussian sputtering model based on a depth consistency loss function, a color loss function and a gradient smooth loss function, determining updated parameter information, and generating a three-dimensional model of the target scene according to the updated parameter information. The reliability and robustness of the three-dimensional model can be improved through depth prior constraint information and a dynamic gradient control method.

The invention discloses a multi-camera motion recovery structure method and system fusing feature point tracks, and the method comprises the steps: carrying out the feature point detection and matching of an input multi-camera image set, calculating the relative pose between image pairs, and constructing a scene graph with camera nodes and feature tracks as edges; defining a multi-camera model, and converting a camera pose estimation problem into a rigid unit pose and internal camera pose estimation problem; by taking the multi-camera model as a constraint, optimizing a problem through a decoupling rotation averaging method to obtain camera rotation parameters; in combination with a distance and angle-based mixed objective function, performing joint optimization on rigid unit translation and internal camera relative translation to obtain camera translation parameters and feature tracks; and by taking the rotation parameter, the translation parameter and the characteristic track as initial values, adjusting and optimizing a global pose and sparse point cloud coordinates through multi-camera joint binding, and obtaining robust and accurate camera parameters and scene sparse point cloud.

The invention discloses a new viewpoint generation method based on point cloud densification and multi-modal collaborative optimization under a sparse viewpoint, which belongs to the field of new viewpoint generation and comprises the following steps of: 1, acquiring a sparse viewpoint image; 2, obtaining a camera pose through a motion recovery structure (SfM), and generating a sparse point cloud; 3, generating a depth map of the input image through a DPT model; 4, generating a high-density and low-noise initial point cloud through a point cloud densification process; 5, initializing the densified point cloud into a 3D Gaussian parameter; 6, through semantic regularization and local depth regularization, 3D Gaussian parameters are optimized in a combined mode; and 7, generating a final new view angle image and a final depth map through a 3D Gaussian sputtering engine. According to the method, through global and local feature perception mechanisms, sparse point cloud densification and multi-modal regularization collaborative optimization, the quality and efficiency of new viewpoint generation under sparse viewpoints are improved.

The application discloses a three-dimensional geographic environment real-time intelligent deduction method based on a multi-modal large model, and relates to the technical field of computer vision. The method is used to solve the technical problem of unified modeling and real-time deduction of dynamic objects and static environment in a three-dimensional geographic environment. First, a camera pose is calculated by a motion recovery structure algorithm to generate a sparse point cloud, and a dynamic foreground object in a monitoring video is separated to extract motion features. Then, a three-dimensional Gaussian distribution set is initialized based on the sparse point cloud, semantic features are extracted by a visual encoder and are mapped to corresponding Gaussian distributions. Next, a topological graph structure of the Gaussian distributions is constructed, and the motion features are used as initial excitation to iteratively update the coordinate offset and appearance change of each distribution by message passing calculation. Finally, the Gaussian distribution attributes are dynamically updated, and a continuous deduction image sequence is rendered by differentiable rasterization to realize high-fidelity real-time simulation of the dynamic evolution of the three-dimensional geographic environment.

The invention discloses an intelligent grading method and system for tunnel face surrounding rock. According to the method, a two-dimensional image, a three-dimensional point cloud and environmental mechanics data of a tunnel face are synchronously collected through a sensor array integrated on tunneling equipment; after space-time registration fusion is carried out on the multi-source data, pixel-level identification and quantitative extraction of geological features are carried out by using a deep learning model, and three-dimensional reconstruction is carried out based on a motion recovery structure algorithm to obtain rock mass structural plane occurrence parameters; visual, geometric and environmental characteristics are fused, and accurate prediction of the surrounding rock grade is realized through an integrated learning model; and finally, dynamic excavation numerical simulation is carried out by combining the three-dimensional geologic model and a prediction result, the surrounding rock stability is analyzed, and safety early warning is generated. According to the method, automation and quantification of the whole process of surrounding rock grading are achieved, and the accuracy and timeliness of tunnel construction safety decision making are effectively improved.

This application discloses a real-time image stitching method and apparatus. The method includes: receiving images uploaded to the cloud by a camera device in real time; extracting feature points from the images and matching feature points between the current frame and the previous frame; calculating the ratio of the homography matrix score to the sum of the homography matrix score and the fundamental matrix score to determine whether to use the homography matrix or the fundamental matrix to recover the image's pose parameters and map points, and initializing the map; estimating the pose of the current frame and optimizing the pose of the current frame using candidate frames; converting the coordinates of the map points to two-dimensional coordinates and performing plane fitting; converting the coordinates of the image corner points in the camera coordinate system to the object coordinate system, calculating perspective transformation parameters and performing geometric transformation on the image; and performing Gaussian pyramid fusion on the generated tiles. This method solves the problems of low efficiency in scene reconstruction using motion reconstruction algorithms and the inability to process real-time photogrammetric data.

This application provides a method, system, medium, and device for human-centered video scene reconstruction and separation. The method includes: for a first-person perspective video sequence, initializing a 3D Gaussian set covering the background, hands, and objects based on a priori knowledge of structure recovery from motion; assigning a learnable dynamic category probability vector to each Gaussian point in the 3D Gaussian set; constructing a dedicated deformation branch; according to the learnable dynamic category probability vector of each Gaussian point, assigning each Gaussian point to the dedicated deformation branch for processing through a preset soft-hard two-stage routing mechanism, determining the Gaussian points processed by the dedicated deformation branch; rendering the Gaussian points processed by the dedicated deformation branch to determine the 4D scene reconstruction image and the decomposed reconstruction images of the background, hands, and objects. This application achieves 4D scene reconstruction of human-centered video and explicit, fine-grained separation of the background, hands, and objects.

This application belongs to the field of computer vision technology and discloses a cross-source point cloud registration method based on adversarial neural networks. The method includes: inputting a multi-frame LiDAR point cloud sequence and a UAV video sequence; segmenting each frame of the LiDAR point cloud into ground points and non-ground points and registering them to obtain a global LiDAR point cloud; performing motion recovery structure reconstruction on the UAV video sequence to generate a dense color point cloud; inputting the global LiDAR point cloud and the SfM dense color point cloud into an MMAlignNet network to construct a shared latent feature space; solving for the rigid transformation matrix from the LiDAR point cloud to the SfM dense color point cloud; and aligning the cross-modal point clouds according to the rigid transformation matrix to obtain a multimodal 3D reconstruction result. This application improves the registration accuracy, completeness, and robustness of large-scale outdoor environment 3D reconstruction by fusing UAV imagery and mobile LiDAR data, combined with cross-modal feature alignment and geometric constraint optimization methods.

The present invention discloses a dynamic scene reconstruction method and device based on multi-scale Gaussian spheres, which relates to the field of computer vision and includes: processing a video frame sequence to be reconstructed by using a structure from motion algorithm to generate a sparse point cloud, initializing the sparse point cloud to generate a 3D Gaussian sphere set; processing the 3D Gaussian sphere set by using a dual-domain deformation model and an adaptive timestamp to obtain a deformed 3D Gaussian sphere set; performing multi-scale Gaussian processing on the deformed 3D Gaussian sphere set to generate a multi-scale Gaussian sphere set; performing Gaussian screening based on pixel coverage rate on the multi-scale Gaussian sphere set to obtain an optimized multi-scale Gaussian sphere set; and performing Alpha blending processing based on the optimized multi-scale Gaussian sphere set to reconstruct an anti-aliased dynamic rendering scene image. The present invention solves the problems of large computational overhead and aliasing effect in current dynamic scene reconstruction.