4081 results about "Pose" patented technology

Disclosed in the present invention is a dynamic human modeling method based on three-dimensional Gaussians. In the present method, a multi-view human video is given firstly, and in order to correct and transform the dynamic human appearance, a latent code and a set of blend weights are added to each Gaussian; and the latent code and a target pose are inputted to an MLP to correct the Gaussian in a canonical space, so as to capture an appearance change under the target pose. The corrected Gaussian is transformed into the target pose by means of the blend weights thereof in a linear blend skinning (LBS) manner. Finally, a realistic human image at a new viewing angle or a new pose can be rendered in real time by means of Gaussian splatting. Compared with an existing method based on an implicit neural radiance field (NeRF), the three-dimensional Gaussian representation of the present invention can better capture high-frequency details while achieving optimal rendering performance.

The invention provides a reconstruction method of a three-dimensional reconstruction model based on two-dimensional Gaussian splashing, which comprises the following steps: S1, carrying out sparse reconstruction on an input image sequence through a multi-view stereoscopic vision algorithm to generate an initial sparse three-dimensional point cloud and a corresponding camera pose parameter; s2, inputting an improved two-dimensional Gaussian radiation field by using the sparse three-dimensional point cloud and the camera pose as information; s3, dynamically screening a visible anchor point subset based on the current view angle parameter, and generating a rendered image through a differentiable rendering pipeline; s4, calculating a loss function of the rendering image of the training track and the input image to optimize a reconstruction scene; and S5, starting a special visualization tool, and inputting a rendering result. According to the method, by introducing a trimmable anchor point parameterization framework and a multi-scale feature fusion mechanism, light-weight and high-precision three-dimensional scene modeling is achieved, and the problems that traditional 2D Gaussian sputtering is insufficient in multi-view geometric consistency, storage overhead and weak texture region reconstruction and an existing 2D Gaussian splashing method is insufficient in self-adaptive mechanism are solved.

In various examples, systems and methods are disclosed relating to generating animatable characters or avatars. The system can assign a plurality of first elements of a three-dimensional (3D) model of a subject to a plurality of locations on a surface of the subject in an initial pose. Further, the system can assign a plurality of second elements to the plurality of first elements, each second element of the plurality of second elements having an opacity corresponding to a distance between the second element and the surface of the subject. Further, the system can update the plurality of second elements based at least on a target pose for the subject and one or more attributes of the subject to determine a plurality of updated second elements. Further, the system can render a representation of the subject based at least on the plurality of updated second elements.

The invention discloses a mechanical arm positioning and grabbing method based on machine vision, and relates to the technical field of machine vision and mechanical arm control, the method comprises the following steps: synchronously acquiring RGB-D images of a target scene through a multi-view camera array, and generating three-dimensional point cloud data through data fusion; an improved LSD algorithm and a PnP algorithm are adopted to calculate the initial pose of the target object, illumination distortion is eliminated in combination with the generative adversarial network, and three-dimensional coordinates are output; a mechanical arm motion error transfer model is constructed based on Monte Carlo simulation, and a candidate grabbing scheme set is generated through reinforcement learning; and an optimal grabbing scheme is screened through a preset priority evaluation rule, and a mechanical arm joint movement track and a control instruction set are generated. Through multi-modal data fusion and a nonlinear optimization algorithm, the technical problems of large target positioning deviation and sensitive illumination interference in a complex environment are solved, and the grabbing precision and robustness of the mechanical arm are improved.

The invention discloses an end-to-end monocular visual odometer method fusing space-time semantic information. According to the method, continuous image sequence frames are collected through a color monocular camera, and a multi-information fusion end-to-end deep learning framework is constructed; a heterogeneous training domain is adopted to set various data set course sharing parameter fusion training, continuous image sequences are input, and the end-to-end deep learning framework is dynamically coupled with hidden state feature vectors output historically, so that a feature mapping relation of time sequence perception is formed; and interpretable feature decoupling of the static background elements and the dynamic entity objects in the scene is realized. After iterative feature fusion, the system outputs sparse depth and camera motion poses which conform to scene geometric constraints, so that a camera trajectory estimation model with high robustness and strong generalization ability in a complex environment is constructed. According to the method, the positioning precision and stability of the monocular vision odometer are remarkably improved.

The invention relates to a dynamic environment semantic SLAM method based on monocular vision and LiDAR fusion, and belongs to the technical field of robots. The method comprises the following steps: 1) performing instance segmentation on point clouds to obtain unknown motion state point clouds, pure static road sign point clouds and ground attribute point clouds, identifying semi-static and dynamic road sign point clouds through initial pose estimation and Kalman filtering, and then performing accurate pose estimation; 2) segmenting and enhancing the image instance, extracting SuperPoint feature points, projecting a dynamic road sign to the segmented image so as to distinguish static and non-static points, tracking static features by adopting a plurality of tracking strategies, and optimizing the pose by utilizing a local map; 3) carrying out heterogeneous sensor attitude fusion; 4) outputting an optimized track by combining two loopback detection results; and 5) constructing a global pure static semantic map. According to the method, the dynamic environment influence is considered, and the positioning precision and the mapping quality of the system in the dynamic complex environment are improved.

The invention discloses an unhooking robot control method and system based on depth visual recognition, and relates to the field of visual recognition, and the method comprises the steps: obtaining an original optical signal and ambient light intensity, carrying out the dynamic focusing of the original optical signal, generating a target pose sequence according to a frequency domain fusion algorithm and a gradient maximization positioning algorithm, and carrying out the dynamic focusing of the original optical signal; meanwhile, the environment light intensity is substituted into adjacent frame pose difference, and a real-time speed vector is obtained; performing modeling on the target pose sequence and the real-time velocity vector by adopting neuromorphic space-time motion to obtain a predicted trajectory of a dynamic coupling target body and a space-time constraint domain of an end effector of the robot; a bionics principle is adopted to simulate a biological motion coordination mechanism, a target motion track and mechanical arm dynamic constraint are jointly described, and the limitation that track prediction and mechanical arm motion ability are disjointed in traditional isolated modeling is broken through.

The invention provides an interaction method and system based on deep learning, and relates to the technical field of computer vision, and the method comprises the steps: collecting a continuous image sequence, extracting feature points, calculating the position variation and gradient variation of the feature points, constructing a two-dimensional feature vector, and carrying out the clustering through a Gaussian mixture model, thereby obtaining a stable feature class. And weighting the feature points based on a feature weight distribution function, calculating feature similarity and constructing a matching relationship. And calculating a dynamic parallax threshold in combination with the vehicle movement speed, screening reliable matching feature pairs, and solving pose transformation parameters. A feature change sequence is extracted through a feature detection network, a motion feature mapping matrix is constructed, and a compensation depth map is generated in combination with depth information. And scene reconstruction is carried out based on the compensation depth map and the pose transformation parameters, the scene confidence score is calculated, the pose parameters are calibrated, a high-precision pose parameter sequence is obtained, and the robustness of feature matching and the accuracy of pose estimation can be improved.

A high-quality digital human avatar is quickly reconstructed from a monocular video and is rendered in real time, the method is widely applied to various fields, and an existing method based on a neural radiation field is high in training and rendering cost. According to the dynamic human body three-dimensional modeling method based on the three-dimensional Gaussian sputtering technology, three-dimensional Gaussian can be expanded to a dynamic human scene, and rapid training and real-time rendering can be achieved. Specifically, the three-dimensional Gaussian of the human body is initialized in a standard space, and the three-dimensional Gaussian is converted from the standard space to a posture space by utilizing linear hybrid skin (LBS) according to posture parameters, so that fine details of the human body are learned at very low calculation cost; a posture parameter refining module and an LBS weight optimization module are designed to improve the problem of inaccurate detection of posture parameters in a real data set, and multi-resolution hash coding is introduced to accelerate the model training speed.

The invention provides a robot grabbing pose optimization method based on visual and tactile fusion, and belongs to the field of robot grabbing pose estimation. According to the method, firstly, an initial grabbing pose is predicted based on target object sparse point cloud obtained through vision initial observation segmentation, then a robot plans and executes grabbing to obtain tactile data of the surface of an object, and target object point cloud information of an area where the initial grabbing pose is located is complemented based on the tactile data with object local information; and finally, the initial grabbing pose is optimized according to the complemented point cloud, so that more stable robot grabbing is achieved.

The invention provides a mechanical arm autonomous grabbing method and system based on image instance segmentation, relates to the field of computer vision and mechanical arm grabbing, and solves the limitation problems of poor generalization and stability and the like in traditional mechanical arm visual grabbing. The method comprises the following steps: performing multi-angle image acquisition on a target object, segmenting and outputting object mask information, adjusting a mechanical arm to an observation position, and obtaining depth point cloud data of the target object; performing principal component analysis on the depth point cloud data, extracting a principal direction vector representing spatial distribution of a target object, and constructing a principal direction coordinate system; generating a candidate grabbing pose set based on the depth point cloud data, and screening out an optimal grabbing pose meeting a preset direction constraint; and based on the optimal grabbing pose, the mechanical arm is driven to execute the grabbing action. The method has the advantages of being small in calculation amount and insensitive to environmental changes, the mechanical arm can autonomously move to the optimal position where the target object is observed and conduct grabbing, and the grabbing accuracy is improved.

The invention discloses a visual inertial positioning method based on dynamic target detection and semantic information constraint, and belongs to the field of motion estimation and dynamic environment processing. According to the method, a dynamic target detection mechanism is introduced, the dynamic target is effectively detected based on the target detection network, inertial navigation information and geometric constraints, the dynamic target is effectively recognized in the image processing process, the corresponding dynamic feature points are screened out, the mismatching rate of the dynamic features is remarkably reduced, and high-quality observation input is provided for back-end optimization. In the back-end sliding window optimization stage, a semantic information consistency constraint method is constructed, and the estimation stability of the system in a weak texture area or a repeated texture area is enhanced by utilizing the consistency of feature points in the same semantic area on a geometric structure. Visual inertia pose estimation is realized based on dynamic target detection and semantic information constraint, and high-robustness and high-precision pose estimation can still be realized in a complex environment with dynamic interference of pedestrians, vehicles and the like and severe scene change.

The invention relates to the technical field of robot control, and discloses a heavy-load robot motion trail method and system based on machine learning. The method comprises the steps that historical movement track data of the heavy-load robot in a working scene are collected, and the data comprise a joint position sequence, an end effector pose sequence and environment obstacle distribution information; the data is preprocessed, track features are extracted, a space-time correlation matrix is constructed, and the matrix is used for representing the dynamic coupling relation between joint movement and the tail end pose; training a trajectory prediction model containing a long and short-term memory network and an attention mechanism based on the matrix, and generating a collaborative mapping relation between a joint position and a tail end pose; obtaining a current task target pose sequence and an environment constraint condition in real time, and outputting a candidate track set meeting dynamic constraint through a model; and adopting a multi-objective optimization algorithm to screen candidate tracks, generating an optimal track instruction and issuing the optimal track instruction to an execution mechanism. The method adapts to the complex characteristics and variable working conditions of the heavy-load robot, and the track adaptability is improved.

The invention provides an interactive robot grabbing method based on a large language model, and the method comprises the steps: obtaining a task operation sequence according to a two-dimensional image of a current scene, dialogue interaction information and the large language model, the task operation sequence comprises a target object, component information of the target object, target grabbing position information and action information; obtaining a target mask area according to the two-dimensional image and the target capture position information in combination with an image segmentation model, and obtaining a three-dimensional point cloud image with a target capture position according to the target mask area and the depth map of the current scene; and according to the action information, the three-dimensional point cloud picture with the target grabbing position and a grabbing prediction model, a target grabbing pose is obtained, and according to the target grabbing pose, a robot is controlled to grab the target object. According to the method, the analysis capability of the robot on the complex semantic instruction is improved, and the adaptability of the robot in a refined operation task is also improved.

The invention discloses a self-adaptive three-dimensional scene reconstruction method and system based on a single panorama, and belongs to the field of computer vision processing. The method comprises the following steps: firstly, generating a depth map through an indoor scene panorama and constructing an initial three-dimensional grid; then generating a multi-view image and a shielding mask thereof based on view conversion, forming a training sample pair for fine tuning of the diffusion completion model, and injecting scene prior information for the model; then extracting a grid boundary contour and calculating a central axis, and adaptively constructing a camera pose set; after the integrity of the grid is optimized through iteration completion, the grid is converted into a 3D Gaussian sputtering field, and Gaussian point parameters are optimized through multi-view data; in the optimization process, an up-sampling strategy of rendering error feedback and edge Gaussian points is adopted, and finally a scene model with complete geometric textures is output. According to the method, a high-quality three-dimensional scene is reconstructed from a single panorama through a structure self-adaptive completion and Gaussian refining technology, and the method is particularly suitable for immersive roaming reconstruction of indoor scenes.

The invention belongs to the technical field of image processing, and particularly relates to an object six-degree-of-freedom pose estimation method based on three-dimensional geometric information registration, which comprises a target detection module, a three-dimensional point cloud generation model and a geometric information registration pose estimation module. According to the method, geometric reconstruction, a geometric information high-precision registration technology and an object pose estimation task are combined, a geometric reconstruction module introduces object geometric feature extraction and feature decoding to realize deep understanding and modeling of an algorithm on object geometric features, the whole process accords with geometric disorder, and the accuracy of object pose estimation is improved. And the accuracy and scene adaptability of the algorithm are further improved. According to the method, accurate estimation of the six-degree-of-freedom pose of the object can be achieved, and powerful support is provided for the fields of industrial automation, robot vision, AR / VR, automatic driving, medical robots and the like.

Systems and techniques are disclosed for generating a three-dimensional (3D) model. For example, a process can include estimating a plurality of features associated with at least a portion of images; inverse warping the plurality of features into reference pose features having a reference pose; generating filtered reference pose features by selecting features from the reference pose features based on a distance of the selected features from corresponding features from the reference pose; generating modified reference pose features by modifying the filtered reference pose features based on a feature grid associated with a reference model associated with the reference pose; projecting the filtered reference pose features into one or more two dimensional (2D) planes; identifying first features associated with the person from the one or more 2D planes; and generating a 3D model of the person having a pose using the first features, the modified reference pose features, and pose information.

The invention provides a lightweight real-time semantic segmentation method for a three-dimensional Gaussian scene, and belongs to the field of computer vision and three-dimensional scene understanding. The process comprises the following steps: S1, data preparation: acquiring multi-modal information data including point cloud, images and poses, and preprocessing the data; s2, carrying out two-dimensional semantic segmentation, and generating a pixel-level semantic segmentation map and a semantic tag for the image by utilizing PP-LiteSeg; s3, performing three-dimensional scene modeling and semantic projection, fusing multi-modal information to complete three-dimensional Gaussian scene reconstruction, and projecting two-dimensional semantics to a Gaussian function to form preliminary Gaussian function semantic features; and S4, semantic optimization and segmentation output: considering image difference, semantic difference and smoothness, establishing a loss function, optimizing semantic features and outputting a segmentation result. According to the method, efficient, real-time and accurate three-dimensional Gaussian semantic segmentation work can be completed, and the method is suitable for scene understanding tasks of light-weight and resource-limited equipment.

The invention relates to the technical field of intelligent manufacturing, and discloses a target structure automatic detection method, device and equipment and a medium, and the method comprises the steps: obtaining scanning path planning data of a target detection structure, driving an ultrasonic probe to execute surrounding scanning motion, and collecting an ultrasonic image sequence and spatial pose data; dynamically adjusting a pressure application angle and a scanning speed based on force feedback information, fusing spatial pose data and an image sequence to perform three-dimensional reconstruction, constructing a three-dimensional geometric model of a target detection structure, extracting feature distribution data by applying an intelligent analysis model, generating a feature decision set, and performing feature extraction; and mapping the feature decision set to a three-dimensional coordinate system to construct an analysis report containing the feature type marks and the topological relation. Through fusion of force control scanning, image reconstruction and intelligent analysis, standardization and intelligence of a detection process are realized, image consistency and structure identification precision are improved, manual dependence is reduced, and comprehensiveness and reliability of lesion identification are enhanced.

The invention discloses an end-to-end underwater three-dimensional reconstruction method and system based on an underwater imaging model, and belongs to the technical field of underwater image processing. According to the method, deep learning pose estimation is combined with an underwater imaging model; firstly, a multi-frame underwater image sequence is collected as input, a deep neural network is constructed, and the network mainly comprises two core sub-modules: a pose estimation network; secondly, a three-dimensional reconstruction network is adopted, dense point cloud or voxel reconstruction is completed according to the predicted pose and image content, pose estimation and the three-dimensional reconstruction process are integrated in the same system, and overall joint optimization is achieved; and in combination with a self-adaptive underwater imaging model, modeling is performed on physical processes such as underwater illumination attenuation and scattering, so that the reality sense and the accuracy of a reconstruction result are improved. According to the method, high-quality three-dimensional reconstruction of images in a complex underwater environment is realized, and the method can be widely applied to ocean engineering, underwater robots, submarine topography surveying and mapping and underwater cultural relic protection.

The invention discloses a building engineering progress automatic identification early warning system based on computer vision, which relates to the field of building engineering informatization and comprises a synchronous calibration module, a joint calibration module, a pose generation module, a mapping construction module, a resampling module, a mapping registration module and a comparison early warning module. Clock synchronization and rolling readout calibration are carried out on a camera and an inertial measurement unit, a continuous time pose is established in a frame, sampling is carried out according to rows, and an equivalent global shutter frame is generated in combination with plane and pixel-level geometric mapping; outputting a camera pose track and a local measurement map by adopting back-end optimization containing a rolling mechanism, and registering with the building information model; component detection, instance segmentation and state discrimination are completed on an equivalent global shutter frame, pixel domain measurement is unified into engineering quantity under pose and registration constraints, the engineering quantity is mapped to a work decomposition structure and a progress plan, deviation is calculated, and graded early warning is output according to a multi-threshold rule. Long-term stable and reliable operation and evidence traceability of the system are guaranteed through whole-course quality monitoring and threshold grading.

The invention provides an editable digital human modeling method and device based on neural texture and three-dimensional Gaussian, and the method comprises the steps: carrying out the modeling of the expression, posture and appearance of a digital human through a three-dimensional Gaussian splashing model according to the motion sequence video data of a target object, and obtaining a Gaussian model of the digital human; modeling the expression, posture and appearance of the digital human through the neural texture model to obtain a neural texture model of the digital human; pruning the Gaussian model, and performing hybrid rendering according to a rendered image output by the pruned Gaussian model and a rendered image output by the neural texture model of the digital human to obtain a hybrid rendered image; and optimizing a Gaussian model of the digital human based on the mixed rendered image and the action sequence video data to obtain an editable digital human model. According to the method, the storage overhead and the structural complexity of the digital human model are reduced, and the high-frequency details and editing efficiency of the skin surface of the digital human model are improved.

The invention provides a three-dimensional human body posture estimation method and system based on multi-view visual information fusion and a storage medium, and the method comprises the steps: 1, designing the front half part of a model into Ender Layers with the same layer number as a Transform decoder at a multi-view feature fusion layer, carrying out the data enhancement of an input multi-view original image, and carrying out the reconstruction of the Ender Layers in the multi-view feature fusion layer; inputting the CNN Backbone with the shared weight to extract an initial feature map; 2, introducing a micro-reprojection optimization mechanism, deeply fusing the multi-view geometric consistency constraint into a model training process, and guiding the model to predict a three-dimensional attitude end to end; and step 3, constructing a dynamic projection compensation module. The method has the beneficial effects that the method is particularly suitable for capturing human body posture information in a multi-person interaction scene, the shielding problem and depth estimation ambiguity in a single view angle can be effectively overcome, and the robustness, precision and efficiency of three-dimensional human body posture estimation are remarkably improved.

The invention discloses a human body posture key point recognition method based on feature enhancement high resolution, and the method comprises the steps: firstly introducing a Res2Net module into a backbone network, constructing a layered similar residual connection structure, achieving the fine-grained multi-scale feature representation, effectively expanding the network receptive field range, and improving the recognition precision of a human body posture key point. According to the structure, multi-scale feature extraction and fusion are achieved in a single residual block, the adaptability of a model to different scale targets is enhanced, meanwhile, the calculation complexity is reduced, then a multi-scale convolution attention MSCA module is embedded, space context information of different scales is captured through multi-branch depth separable convolution, and the multi-scale feature fusion is achieved. According to the method, the key features are adaptively enhanced in combination with a channel attention mechanism, the positioning capability of the key points of the human body is remarkably improved, and finally, richer and more accurate key point information of the human body posture is acquired by fusing multi-scale and deep feature representation, so that accurate recognition of the human body posture is realized.

The invention provides a mechanical arm grabbing method and system based on multi-modal information fusion, and belongs to the technical field of robot intelligent control. Comprising the steps that the conversion relation between a camera coordinate system and a mechanical arm base coordinate system is established through camera calibration, a deep learning neural network is used for conducting grabbing pose estimation on an obtained RGB-D image, and multiple candidate grabbing poses are determined; analyzing a natural language instruction input by a user based on a multi-modal large model, and recognizing a target object region from the RGB-D image by combining a target detection and image segmentation technology; based on the obtained candidate grabbing poses and the target object area, an optimal grabbing pose is screened through a scoring mechanism and mapped to a mechanical arm base coordinate system; and then a dynamic grabbing path is generated by adopting an imitation learning algorithm, and the mechanical arm is controlled to execute grabbing operation. Through multi-modal semantic understanding, accurate grabbing of the mechanical arm in a complex environment can be achieved.

The invention discloses a multi-sensor fusion SLAM (Simultaneous Localization and Mapping) method in a dynamic scene, which belongs to the field of robot perception and comprises the following steps of: preprocessing visual, laser and IMU (Inertial Measurement Unit) data, including time synchronization and space synchronization; a sensor mathematical model is established, a visual inertia subsystem and a laser inertia subsystem are constructed at the same time, and the two systems are fused in a tight coupling mode. The laser inertial subsystem assists the visual inertial subsystem to complete scale recovery, and the visual inertial subsystem assists the laser inertial subsystem to carry out loopback detection. And finally, establishing a global nonlinear optimized factor graph model in a graph optimization mode, and adding a laser odometer factor, an IMU pre-integration factor, a visual factor and a loopback detection factor into a factor graph to obtain global optimal pose estimation and a three-dimensional map. In a dynamic environment, information of a plurality of sensors is fused, so that the influence of a dynamic object on positioning is reduced, and higher positioning precision and robustness are achieved.

The invention provides a 3D + 2D high-dimensional feature fusion-based material tracking multi-target identification method and system. The method comprises the steps of performing target identification on a to-be-tracked material by using a YOLO algorithm; performing multi-target real-time tracking on the material based on a Deep SORT algorithm; generating three-dimensional point cloud data of a target by adopting a multi-view data calibration and alignment algorithm, and carrying out noise reduction, sampling and data enhancement preprocessing operation; performing three-dimensional reconstruction on the preprocessed three-dimensional point cloud data by using a Gaussian Splitting method, and introducing time sequence information at the same time; segmenting a dynamic three-dimensional target and a static background through a Mask R-CNN technology; aligning the position of the target in the two-dimensional image with the coordinates in the three-dimensional space by using a feature fusion method; a PNP algorithm is adopted to estimate attitude and depth information of a target in a three-dimensional space, accurate positioning and state restoration of materials are realized, and efficient and reliable technical support is provided for material management and tracking in an industrial environment.

The invention discloses a hand-eye calibration method, device and equipment for a mechanical arm and a storage medium. The method comprises the steps that a calibration plate is fixed to the tail end of a mechanical arm, and the mechanical arm is controlled to move along a preset track, so that the calibration plate is always located in the view field of a camera; under at least three different postures of the mechanical arm, the calibration plate image is collected, and meanwhile the pose of the calibration plate at the tail end of the mechanical arm under a mechanical arm coordinate system is recorded; calculating the pose of the calibration plate in the camera coordinate system through calibration plate angular points in the calibration plate image; and according to the pose of the corresponding calibration plate in the mechanical arm coordinate system and the pose of the corresponding calibration plate in the camera coordinate system, calculating a rotation matrix and a translation vector between the mechanical arm coordinate system and the camera coordinate system so as to complete hand-eye calibration. According to the technical scheme, high-precision hand-eye calibration is achieved, and the stability of automatic application of the robot is remarkably improved.

The invention relates to the field of robots, and discloses a mapping method based on multi-sensor fusion, electronic equipment and a storage medium. According to the method, firstly, a laser radar, a millimeter-wave radar and a visual and motion inertial odometer are generated by collecting multi-sensor data, and it is ensured that other sensors can still provide effective information when part of the sensors are degraded due to insufficient illumination, noise interference or extreme dynamic scenes. And secondly, through real-time health degree evaluation, unstable data generated due to sensor degradation or abnormity is automatically eliminated, and it is ensured that only health data is used for generating a pose map, so that the robustness of data fusion is improved. And finally, a factor graph is constructed by utilizing head IMU data, global pose optimization is realized by virtue of short-time high-frequency stability and continuity of the factor graph, and even if other sensors are partially degraded, the accuracy and consistency of overall positioning can be maintained.

The invention discloses a cross-source data three-dimensional reconstruction method and system based on improved Gaussian sputtering, and the method comprises the steps: collecting an unmanned plane inclined image and a ground panoramic image of a target region, and constructing a time-space correlation data set; based on multi-view geometric constraints, space-time coding matching point pairs are established through an adaptive feature pyramid, intelligent incremental cross-source data sparse reconstruction is carried out, and point cloud and camera parameters are output; adopting improved Gaussian sputtering, compressing a three-dimensional Gaussian kernel into a two-dimensional Gaussian primitive through double tangent vector constraint, and fitting surface geometry to realize multi-scale reconstruction; and optimizing primitive parameters by using a differentiatable renderer, completing multi-scale fine reconstruction through gradient back propagation, and generating a high-precision three-dimensional model. According to the method, multi-scale accurate geometric prior input and accurate camera poses are provided for three-dimensional reconstruction, the dependence on professional manual operation in a traditional three-dimensional reconstruction method is greatly reduced, and meanwhile, the geometric accuracy and visual fidelity of a reconstruction result are remarkably improved.