221 results about "Gauss point" patented technology

The invention discloses a three-dimensional Gaussian sputtering scene reconstruction method based on structure perception refined Gaussian, and aims to solve the problems of Gaussian drift, edge blur, structure artifacts and the like of a reconstruction model due to the fact that sparse point cloud contains outliers, Gaussian morphology and normal are mismatched and a multi-dimensional optimization target is lacked in an existing three-dimensional Gaussian sputtering reconstruction method. A key frame is extracted by collecting target scene video data, sparse three-dimensional point clouds are reconstructed by using an SfM algorithm, a depth map and a normal map are generated through a Lotus model, three-dimensional Gaussian distribution is initialized after the sparse point clouds are filtered, a Gaussian covariance matrix is adjusted by using a normal consistency regular term, and the sparse point clouds are extracted. And after structure attribute analysis is carried out, a comprehensive scoring function is constructed to screen Gaussian points, and finally, a combined training framework including luminosity, normal consistency and structure continuity loss is adopted to optimize and generate a three-dimensional Gaussian scene model. The method is mainly applied to the field of three-dimensional reconstruction and multi-view rendering, and scene reconstruction precision and geometric consistency can be improved.

The invention relates to the technical field of computer vision, in particular to a point projection type three-dimensional reconstruction and segmentation method and system based on semi-Gaussian pruning. The method comprises the following steps: respectively obtaining an SFM point cloud and a consistency label mask of a cross-view label based on an obtained multi-view image; initializing the obtained SFM point cloud into an identity semi-Gaussian point cloud, and performing rendering optimization by using a differentiable renderer; densifying the initial sparse point cloud by using a localized semi-Gaussian point management method, and identifying a local error region for resetting and repairing; using the obtained consistency label mask to supervise Gaussian identity feature learning by using cross entropy loss, and using unsupervised 3D regularization loss to force spatially adjacent gauss to maintain identity consistency; according to the method, the identity coding semi-Gaussian kernel method is adopted, the inherent representation fuzziness of a single opacity formula is eliminated, and the positive influence on the identity coding precision is generated.

Disclosed in the present invention is a three-dimensional Gaussian splatting optimization method for an unposed input. The method comprises: for an input image, predicting a ray bundle distribution by using a ray prediction model, to obtain distribution features in the form of ray bundles; on the basis of the distribution features in the form of ray bundles, calculating a camera pose; for the ray bundle distribution, performing sampling on the basis of the volumetric density of light rays, to obtain an initial spatial distribution of a three-dimensional Gaussian point cloud focused on a visual center area; for the input image, obtaining a visible shell by means of view frustum projection and object-mask computation; on the basis of the initial spatial distribution of the three-dimensional Gaussian point cloud and the visible shell, performing three-dimensional Gaussian splatting scene training, to obtain a three-dimensional scene reconstruction model satisfying a preset loss function criterion, the loss function comprising a training regularization term for a camera pose parameter. The present invention provides important scene initialization information for three-dimensional Gaussian splatting training, and significantly improves the quality and the richness of detail of the final three-dimensional structure.

The invention discloses rendering equipment based on three-dimensional Gaussian sputtering. The hardware architecture includes a memory interface, a pre-processing engine, a cardinality ordering engine, a rasterization engine, and the like. The preprocessing engine converts the three-dimensional Gaussian point into a two-dimensional representation and generates a key value pair containing a tile identifier and depth information; the cardinal number sorting engine performs staged efficient sorting through a parallel processing mechanism and a BRAM alternate working mode; and the rasterization engine performs rendering processing by adopting a full-pipeline design and an early-stage stopping mechanism. According to the method, parallel execution of depth sequencing and rasterization is achieved, memory access and calculation redundancy are reduced through optimized data flow design, an efficient and low-power-consumption hardware acceleration solution is provided for three-dimensional Gaussian sputtering rendering, and the method is particularly suitable for application scenes such as virtual reality, games and scientific visualization requiring real-time rendering.

The invention discloses a dynamic scene repairing method based on four-dimensional Gaussian sputtering, which is characterized in that efficient dynamic scene space repairing is realized based on a repairing framework of a four-dimensional Gaussian sputtering monocular video reconstruction algorithm and a video repairing model, and by counting the time frame number of Gaussian primitives in a mask area and removing Gaussian points exceeding a threshold value, the dynamic scene space repairing efficiency is improved. According to the method, high-efficiency dynamic object removal is realized, and a mask weighting loss function is introduced to guide the repair area to converge to real texture information in a subsequent time frame, so that the repair quality is improved. The invention provides a dynamic scene repair method based on four-dimensional Gaussian sputtering, and provides a repair framework based on a four-dimensional Gaussian sputtering monocular video reconstruction algorithm and a video repair model for removing and repairing a target object in a dynamic scene to realize three-dimensional repair of the dynamic scene.

The invention provides an infrared scene reconstruction and rendering method and device based on three-dimensional Gaussian splashing, and the method comprises the steps: carrying out the processing of preprocessed infrared image data through employing an SfM algorithm, obtaining a sparse three-dimensional point cloud and a camera pose, and setting a Gaussian ellipsoid at the center position of the sparse three-dimensional point cloud, so as to construct a three-dimensional Gaussian point cloud model; inputting the camera pose into a Gaussian microrasterized renderer to render the three-dimensional Gaussian point cloud model to obtain a two-dimensional image of a corresponding view angle; the two-dimensional image and the infrared image data of the camera pose are compared through a loss function, the model is trained, a trained three-dimensional Gaussian point cloud model is obtained, and the loss function comprises a temperature gradient field, the projection position of a Gaussian ellipsoid in a pixel plane under the current camera visual angle and the main axis projection direction; and processing the trained three-dimensional Gaussian point cloud model by using a three-dimensional graphic engine and a 3D Gaussian rasterization renderer to obtain an infrared simulation image. And the authenticity of reconstruction and rendering of the infrared scene is relatively high.

The embodiment of the invention provides a foreground target segmentation and reconstruction method and device, equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: directly inputting a video frame into a target segmentation model so as to generate a foreground target segmentation mask, associating the foreground target segmentation mask with a corresponding initial Gaussian point cloud as input data, inputting the input data into a space-time Gaussian splash model for data processing, and performing time slice division on the input data so as to obtain a time slice division result; and obtaining a slice Gaussian point cloud corresponding to each time slice, obtaining a two-dimensional Gaussian curved surface corresponding to each slice Gaussian point cloud, and obtaining a rendered image corresponding to each visual angle according to the plurality of two-dimensional Gaussian curved surfaces. And obtaining a foreground target rendering sequence at the processing moment according to the rendered image corresponding to each visual angle, and generating a foreground target reconstruction result according to the foreground target rendering sequences at all acquisition moments. While the dependence on external prior information is reduced, the stability of a time sequence can be ensured, and the accuracy of a rendering result is improved.

The invention discloses a building three-dimensional reconstruction method based on 3D Gaussian sputtering guidance. The building three-dimensional reconstruction method comprises the following steps: S1, acquiring multi-view building image data; s2, sparse point cloud reconstruction of the building is carried out from the multi-view building image based on a motion recovery structure SFM, and then the point cloud of the building is initialized through 3D Gaussian Splitting; s3, performing multi-scale optimization representation on the initialized 3D Gaussian point cloud, accurately capturing local geometric features and performing optimization adjustment on surface fitness so as to enhance representation of a building surface; s4, building patches are generated based on three-dimensional Gaussian distribution; and S5, obtaining a building three-dimensional model and a rendering result. According to the scheme, a highly detailed surface can be recovered, and high-precision and high-fidelity reconstruction of the geometric shape and appearance of a building in a complex scene is realized.

The invention provides an image rendering method, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models, augmented reality and the like, and can be applied to scenes such as three-dimensional reconstruction and the like. According to the specific implementation scheme, a target grid model is generated based on a three-dimensional Gaussian sputtering model, the three-dimensional Gaussian sputtering model comprises a plurality of Gaussian points used for describing an object to be rendered, and the target grid model comprises respective target patches of the Gaussian points; the transparency of the middle area of the target patch in the target mesh model is lower than that of the peripheral area of the target patch; taking the target grid model as a mask to project a shadow, and obtaining a shadow map of the to-be-rendered object under a predetermined resolution; rendering the to-be-rendered object based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; and displaying the rendered target image. The invention further provides an image rendering device, electronic equipment and a storage medium.

The invention discloses a photographing viewpoint planning method for hub surface defect detection, and the method specifically comprises the following steps: carrying out the Gaussian mapping of a triangular patch model of a to-be-detected hub, and forming a Gaussian ball; dividing the triangular patch model into a plurality of independent areas, wherein normal vector included angles corresponding to Gaussian points in the independent areas are close and topologically adjacent; searching adjacent independent areas, wherein the average normal vector included angle between the adjacent independent areas and each independent area is smaller than the field angle of the camera, and combining the found adjacent independent areas with the corresponding independent areas; constructing an OBB bounding box of each independent area, and determining a camera viewpoint of the OBB bounding box surrounded by the camera visual cone model; and detecting the shielded camera viewpoints, and adjusting the positions and poses of the shielded camera viewpoints to obtain a viewpoint set during photographing of the camera. The photographing viewpoint is generated in a manner of partitioning the triangular patch model of the hub to be detected, so that viewpoint redundancy caused by uniform sampling or random sampling on the surface of the triangular patch model is avoided, and the viewpoint generation efficiency is improved.

The invention discloses a plant three-dimensional Gaussian phenotype extraction method based on an improved parameter adaptive fast pose optimization algorithm, and the method comprises the steps: obtaining a multi-view image sequence of a plant through a camera system, estimating the pose of a camera through a global adaptive fast pose optimization algorithm, and generating a sparse three-dimensional point cloud of a scene, pose estimation errors in an agricultural scene can be effectively eliminated, the problem of error accumulation in a traditional method is avoided, and high-precision camera pose estimation is guaranteed; secondly, taking the camera pose and the sparse three-dimensional point cloud as input, converting an object in a scene into a plurality of three-dimensional Gaussian points, performing three-dimensional reconstruction based on a Gaussian representation method, and generating a dense three-dimensional point cloud, thereby solving common shielding and dynamic change problems in an agricultural scene, and improving the quality and efficiency of point cloud reconstruction; and finally, based on the obtained dense three-dimensional point cloud, phenotypic characteristics of the plants are extracted, and more accurate data support is provided for crop phenotypic analysis.

The invention relates to an editable human body reconstruction method and system based on three-dimensional Gaussian splashing, and the method converts a dynamic human body reconstruction problem into a reconstruction problem of a static template human body in a standard space, achieves the respective decoupling of the shape and the appearance through a mode of constructing the mapping of a three-dimensional Gaussian model and a parameterized human body model, and achieves the reconstruction of the human body. Human body reconstruction in an editable form is achieved, and the editable form comprises the steps that geometric editing is achieved by changing a body shape coefficient and a pose coefficient in the parameterized human body model; and replacing the vertex mapped three-dimensional Gaussian point cloud in the editable three-dimensional Gaussian model to realize texture editing. Compared with the prior art, the method solves the problem that personalized editing cannot be carried out on the human body model based on three-dimensional Gaussian splashing in the prior art.

The invention discloses a four-dimensional Gaussian video method and a four-dimensional Gaussian video device combined with motion layering, which realize vivid volume video generation and streaming transmission in a dynamic scene. According to the method, a streaming Gaussian Group (GOG) representation is constructed, the representation reconstructs a dynamic scene by using Gaussian point clouds grouped according to a time sequence, and a two-dimensional segmentation result is promoted to a three-dimensional motion layering in each GOG in combination with a motion layering technology to obtain layering of static points and dynamic points, so that the dynamic scene is reconstructed. And modeling a dynamic portion in the scene by predicting a motion offset of the dynamic point. In addition, the invention provides a self-adaptive static point conversion mechanism for converting static points into dynamic points in motion modeling, so that appearance changes (such as shadow movement and reflection changes) of static objects are effectively processed.

The invention discloses a three-dimensional modeling method, electronic equipment, a storage medium and a computer program product, and relates to the technical field of data processing, and the method comprises the steps: retrieving a four-dimensional bilateral affine grid based on a spatial dimension index and a color dimension index of a three-dimensional Gaussian point of a training view to obtain an affine transformation matrix, adjusting the color dimension index through the affine transformation matrix to obtain an adjusted color dimension index; rendering the training view based on the adjusted color dimension index through a three-dimensional Gaussian splash model to obtain a rendered image; and based on the luminosity loss between the rendering image and the training view, performing parameter optimization on the four-dimensional bilateral affine grid and the three-dimensional Gaussian splash model to obtain an optimized three-dimensional Gaussian splash model, and performing three-dimensional modeling on the multi-view image based on the optimized three-dimensional Gaussian splash model. According to the method, the floating objects in the scene are reduced, and the geometric modeling accuracy of the scene is improved, so that the rendering quality and the reconstruction accuracy in the three-dimensional modeling process are improved.

The invention relates to the technical field of synchronous positioning and mapping, in particular to a positioning mapping method and system based on three-dimensional Gaussian splashing and a depth camera, and the method comprises the steps: collecting the image and depth information of a target scene in real time, and obtaining the data of a current frame and the corresponding Gaussian point cloud data; based on the current frame data and the corresponding Gaussian point cloud data, adopting an Adam optimization algorithm and analyzing a Jacobian matrix to estimate the camera pose of the current frame; selecting a key frame from a plurality of current frames based on the camera pose, and constructing a dynamic key frame window based on the key frame; and according to the key frame data and the corresponding Gaussian point cloud data, adding a Gaussian point cloud in an area with insufficient coverage, and carrying out iterative rendering on the key frame in the dynamic key frame window to complete the construction of the three-dimensional map. According to the invention, rapid and high-quality positioning mapping can be realized.

The invention relates to a real-time digital person generation method and system, and the method comprises the following steps: obtaining a monocular video of a target person, and extracting 3DMM information in the monocular video; performing Gaussian point initialization on the monocular video according to the 3DMM information to obtain Gaussian parameters in a standard space; extracting voice audio features in the monocular video, and inputting the voice audio features into the audio-motion model to obtain a universal face key point motion sequence; converting the universal face key point motion sequence into a target face key point motion sequence through a projection algorithm; inputting the target face key point motion sequence and the Gaussian parameters in the standard space into a face key point guided Gaussian deformation network to obtain Gaussian deformation parameters; and rendering the Gaussian deformation parameter into a corresponding video frame through a Gaussian rasterizer, thereby obtaining a digital person video of the target person. Compared with the prior art, the digital human video generation accuracy and creation flexibility are improved.

The invention relates to a three-dimensional reconstruction-oriented Gaussian point cloud lossless coding and decoding method, a product and a storage medium. The method comprises the following steps: extracting a first Gaussian point cloud corresponding to anhor data in a numpy array format at a coding side; converting the first Gaussian point cloud in the numpy array format into a first Gaussian point cloud in a ply format; meanwhile, through a Morton code-based spatial sorting method, performing Morton code sorting on geometric data and attribute data of the first Gaussian point cloud in the numpy array format, and generating a Morton sequence index table; and carrying out AVS coding on the first Gaussian point cloud in the ply format to obtain compressed data in a binary code stream form. On the decoding side, AVS decoding is carried out when the compressed data in the binary code stream form and the Morton sequence index table are received, and a second Gaussian point cloud in the ply format is obtained through reduction; performing format conversion on the second Gaussian point cloud in the ply format to obtain a second Gaussian point cloud in a numpy array format; and rearranging the attribute data and the geometric data based on the Morton sequence index table to realize one-to-one correspondence of the geometric data and the attribute data between the first Gaussian point cloud and the second Gaussian point cloud before and after coding and decoding. Compared with an existing method, the data consistency before and after coding is improved.

The invention relates to an object attitude estimation method based on scene-level semantic three-dimensional Gaussian spatter, which comprises the following steps of: firstly, constructing scene-level three-dimensional representation according to a multi-view image, and associating semantic embedding of three-dimensional Gaussian points of each target object; positioning a target mask area of a target object described by a language instruction according to the query image, and extracting each target three-dimensional Gaussian point subset through semantic matching and three-dimensional space clustering; secondly, an ICP registration method is guided through two-stage learning, and the initial 6D pose of the target object is obtained; and finally, performing cascade fine optimization by combining pose rendering and similarity comparison under disturbance to obtain a 6D pose of the target object in the target scene. According to the design scheme, target retrieval and instance extraction under open vocabularies are achieved through three-dimensional Gaussian reconstruction of semantic enhancement, and the attitude estimation precision under the conditions of shielding and low texture in a complex scene is effectively improved by combining language prompt and registration and rendering fine optimization guided by two-stage learning.

The invention discloses a dynamic three-dimensional scene representation method based on Gaussian splashing, and belongs to the field of machine vision. A first time feature constructed through a first neural network based on the time index is fused with the Gaussian feature of the original Gaussian point cloud, and then decoding is performed to obtain various parameters of the original Gaussian point cloud; and fusing a second time feature obtained through second neural network coding based on the time index with the feature obtained by the micro renderer, so that time domain information is introduced in the image reconstruction process to assist in constructing the incidence relation of adjacent video frames, and the time domain expression capability of the model is further enhanced. According to the method, the convolutional neural network is combined with Gaussian splash, the modeling capability of a model for a dynamic scene can be effectively enhanced by fusing time domain features, and efficient expression of a dynamic three-dimensional scene is realized; time domain redundancy is eliminated by modeling a time domain incidence relation, and efficient expression of a dynamic scene is achieved.

The embodiment of the invention provides a Gaussian point cloud rendering method and device, equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: acquiring pixel points of foreground targets in two corresponding key frames, generating a double-view-angle Gaussian point cloud based on the pixel points, quantifying double-view-angle Gaussian parameters of the double-view-angle Gaussian point cloud based on different rate-distortion coefficients to obtain quantized Gaussian parameters corresponding to each rate-distortion coefficient, and obtaining the target foreground target in the two key frames according to the quantized Gaussian parameters of the double-view-angle Gaussian point cloud and the quantized Gaussian parameters of the double-view-angle Gaussian point cloud. At least entropy coding is carried out on the quantized Gaussian parameter to obtain a coded Gaussian parameter; and determining a target rate-distortion coefficient according to the definition requirement, selecting a coding Gaussian parameter corresponding to the target rate-distortion coefficient as a target coding Gaussian parameter, and sending at least one target coding Gaussian parameter corresponding to each dual-view sequence to a decoding end. The frame extraction operation is performed on the video frame, so that the data volume can be reduced. Secondly, quantitative Gaussian parameters corresponding to different definitions are obtained according to different rate distortion coefficients, and balance of data transmission and rendering quality is achieved.

The invention relates to the technical field of computer vision, and discloses a sparse view-based three-dimensional image reconstruction method and device and computer equipment, and the method comprises the steps: obtaining a multi-view image of a target human body wearing clothes; fitting a three-dimensional human body template of the target human body; extracting a multi-view clothing image; according to the multi-view garment image and the three-dimensional human body template, generating an initial Gaussian point cloud of the garment; constructing a boundary range for the initial Gaussian point cloud, and removing free Gaussian points outside the boundary range; constructing a loss function of two-dimensional Gaussian sputtering, performing multiple optimization iteration training on the initial Gaussian point cloud, and continuously removing free Gaussian points outside the boundary range in the optimization iteration training process to obtain a reconstructed garment model; generating a three-dimensional human body model with clothes according to the three-dimensional human body template and the reconstructed clothes model; according to the method, the quality and availability of three-dimensional human body model reconstruction are improved.

The invention discloses an object level motion blur three-dimensional scene synthesis method and an object level motion blur three-dimensional scene synthesis device based on three-dimensional Gaussian splash (3D Gaussian splash, 3DGS). According to the method, a multi-view image and corresponding camera parameters are used as input, and three-dimensional Gaussian point scene representation including spatial position, scale, rotation, opacity and appearance characteristics is constructed; generating a binary mask of a moving foreground object and a static background by using an image segmentation model; on the basis, camera exposure time is dispersed into a plurality of time steps, space pose parameters changing along with time are introduced only for three-dimensional Gaussian points corresponding to the foreground object, instantaneous three-dimensional scene representation under each time step is obtained, integration or weighted averaging is carried out on rendering results of the plurality of time steps, and the real-time three-dimensional scene representation of the foreground object is obtained. Generating a motion blur rendering result with real physical significance; meanwhile, a mask-guided foreground-background separation rendering strategy and a partition loss function are adopted, clear rendering supervision is applied to a static background area, fuzzy rendering supervision is applied to a moving foreground area, and joint optimization of three-dimensional Gaussian scene parameters is achieved; in the reasoning stage, the clear background and the fuzzy foreground are fused and output according to the mask, and a three-dimensional scene reconstruction result with the clear background and the reasonable motion fuzzy effect of the foreground is obtained. According to the method, the exposure integral process and the object movement track are explicitly modeled in the three-dimensional geometric space, and object-level controllable fuzzy three-dimensional scene generation is realized.

The invention discloses a multi-view driven three-dimensional generation method, and belongs to the field of computer vision and three-dimensional reconstruction. According to the method, a multi-view image of an object is coded to generate corresponding feature representation, and support is provided for subsequent correction flow guide three-dimensional model optimization; a correction flow model is adopted to gradually convert a three-dimensional sample under prior distribution into implicit three-dimensional representation of an object under the condition of multi-view images of the object, a multi-view Transform network is adopted as a speed prediction model of the correction flow model, and the correction flow model starts from a Gaussian distribution sample and is guided according to a current state and a speed field, so that the speed of the object is predicted. The method comprises the following steps: continuously correcting a sample path, enabling a sample to be stably close to target distribution along a predetermined track, achieving efficient remodeling of an implicit expression of an object, and finally decoding the implicit expression into a three-dimensional Gaussian point cloud through a decoder, thereby realizing three-dimensional reconstruction of the object. The method has the advantages of high speed and high reconstruction quality, and is suitable for the fields of three-dimensional modeling, data simulation and the like.

The invention discloses a text-to-three-dimensional Gaussian shape generation method and device, equipment and a medium, and the method comprises the steps: obtaining an initialized three-dimensional Gaussian model through a pre-trained three-dimensional generator according to the text input for generating a target task; performing multi-view projection to obtain a key view image and a group of auxiliary view images; performing feature extraction on the auxiliary view image, and fusing the extracted features with the features of the key view image through an attention mechanism; performing multi-view joint geometric optimization on the initialized three-dimensional Gaussian model through fractional distillation loss by using the fused features and text input injection diffusion model to obtain a three-dimensional Gaussian model subjected to geometric optimization; and a layered Gaussian constraint mechanism is implemented, reinforced constraint is applied to Gaussian points generated earlier, higher degree of freedom is given to Gaussian points newly added in the later period, and a final three-dimensional Gaussian shape is obtained. According to the method, the geometric semantic consistency and the overall fidelity and fineness of the generated result under different visual angles are improved.

The invention discloses a head reconstruction method based on Gaussian sputtering, and belongs to the field of computer vision. Performing alignment and clone splitting on the initial point cloud model by using the supervision image to obtain a head accurate point cloud of the supervision image, and taking the head accurate point cloud position as a Gaussian point cloud position; projecting the Gaussian point cloud to an image coordinate system, and performing feature extraction by taking the supervised image as convolutional neural network input to obtain a convolutional feature map; sampling the head precise point cloud and the convolution feature map of the image coordinate system to obtain a feature vector, and decoding the feature vector to obtain a Gaussian point cloud parameter so as to obtain a three-dimensional head Gaussian model; a head image of any viewpoint is quickly rendered through the three-dimensional head Gaussian model, pixel-by-pixel loss calculation is performed on the rendered image and a real image, and the three-dimensional head Gaussian model is optimized. According to the method, under the condition of sparse input, the convolutional neural network and Gaussian sputtering are fitted, and the obtained high-fidelity three-dimensional head Gaussian model can be rendered quickly in real time and has generalization.

The invention provides a 3D GS anti-aliasing rendering method fused with adaptive expansion, which comprises the following steps: under a 3D GS framework, before projecting a 3D Gaussian to a screen space, firstly applying a 3D Gaussian low-pass filter to the 3D Gaussian to suppress high-frequency components exceeding a Nyquist frequency and reduce the probability of occurrence of aliasing artifacts; after a three-dimensional Gaussian is projected to a screen space, a resolution self-adaptive expansion coefficient adjusting mechanism is used, and the physical size of a Gaussian point can be automatically adjusted according to the image resolution. And finally, in combination with a super-sampling technology, it is ensured that the colors of the Gaussian primitives can reasonably influence the pixels. According to the multi-guarantee 3DGS anti-aliasing rendering method, the aliasing problem caused by overlarge coverage of Gaussian points is prevented under the low resolution, and detail loss and edge erosion are avoided under the high resolution.

The invention relates to the technical field of photo three-dimensional processing, and discloses a three-dimensional head portrait model generation method based on a single photo, which comprises the following specific steps: photo input processing: firstly, extracting a face region from the single photo by using a face feature point detection algorithm; background segmentation is carried out by using an SAM algorithm, and a face region in the picture is extracted; finally, the depth information of the human face is speculated through a Depth Anything model, and the depth information is used for guiding generation of Gaussian point cloud; according to the three-dimensional head model generation method based on the single photo, the high-precision three-dimensional head model can be generated from the single photo. According to the method, the 360-degree rotating video is firstly generated, and then three-dimensional reconstruction is carried out by utilizing Gaussian splashing, so that the finally generated 3D head model is highly matched with the input picture, and the similarity of the model is relatively high. Besides, the method can retain abundant detail features, and the Gaussian splashing point cloud can accurately restore personalized features of the human face, such as wrinkles, skin colors, hairlines and the like, so that the model is more real and natural.

The invention relates to a semantic dense vision SLAM (Simultaneous Localization and Mapping) method and system based on 3D (Three-Dimensional) Gaussian representation, and belongs to the field of body intelligence. The method comprises the following steps: acquiring image data of a scene through a sensor, constructing a scene model of 3D Gaussian representation, fusing semantic features and appearance features to generate high-dimensional semantic features, optimizing parameters of Gaussian points by combining appearance, geometry and semantic constraints, updating the scene model according to the optimized parameters, and optimizing a camera pose. And generating a rendered image through a differentiable rendering technology, tracking and positioning, detecting a loop by using image features, and optimizing the global map precision. According to the method, the mapping precision and robustness of the SLAM system are remarkably improved, so that the SLAM system can show more excellent performance in a complex environment.

The invention provides a three-dimensional head portrait model construction method, system and device and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: firstly, acquiring multi-view face information of a target person, performing feature extraction on the multi-view face information to obtain a shape parameter, a first posture parameter and a first expression parameter, and determining information of a plurality of anchor points on a face template according to the shape parameter, each piece of anchor point information is used for managing attribute data of three-dimensional Gaussian points on a plurality of associated face templates, adjusting the anchor point information according to the first posture parameter and the first expression parameter to obtain first attribute data of the three-dimensional Gaussian points corresponding to the anchor point information, the first attribute data of the three-dimensional Gaussian point is used for obtaining second attribute data of the three-dimensional Gaussian point, and obtaining a target three-dimensional head portrait according to the second attribute data of the three-dimensional Gaussian point and the face template. The three-dimensional head portrait can highly restore complex expressions and posture changes of the face, and high-frequency details such as fine muscle movement can be accurately captured.

The invention discloses a 3D human avatar method and a 3D human avatar system which are generated based on 3d gaussian patterning and can be driven in real time. The 3D human avatar system comprises a data acquisition module, a Gaussian initialization module and a training rendering module, the data acquisition module acquires a small amount of human body data based on a single camera, predicts human body posture information based on an smplx model and expression information based on a Flame model from two-dimensional human body image data, and makes labels; a Gaussian initialization module carries out adaptive Gaussian initialization on a human body smplx model grid in any frame label to obtain a 3D Gaussian ellipsoid; and the training rendering module is used for training the 3D Gaussian ellipsoid initialized in the step 2 based on an improved 3d Gaussian sputtering model, and rendering a 3D human avatar capable of being driven in real time. According to the method, the topological consistency of the Gaussian point and the human body smplx model can be highly maintained, artifacts are reduced, and high fidelity is improved.