257 results about "Gauss point" patented technology

The invention discloses a three-dimensional Gaussian sputtering scene reconstruction method based on structure perception refined Gaussian, and aims to solve the problems of Gaussian drift, edge blur, structure artifacts and the like of a reconstruction model due to the fact that sparse point cloud contains outliers, Gaussian morphology and normal are mismatched and a multi-dimensional optimization target is lacked in an existing three-dimensional Gaussian sputtering reconstruction method. A key frame is extracted by collecting target scene video data, sparse three-dimensional point clouds are reconstructed by using an SfM algorithm, a depth map and a normal map are generated through a Lotus model, three-dimensional Gaussian distribution is initialized after the sparse point clouds are filtered, a Gaussian covariance matrix is adjusted by using a normal consistency regular term, and the sparse point clouds are extracted. And after structure attribute analysis is carried out, a comprehensive scoring function is constructed to screen Gaussian points, and finally, a combined training framework including luminosity, normal consistency and structure continuity loss is adopted to optimize and generate a three-dimensional Gaussian scene model. The method is mainly applied to the field of three-dimensional reconstruction and multi-view rendering, and scene reconstruction precision and geometric consistency can be improved.

The invention relates to the technical field of computer vision, in particular to a point projection type three-dimensional reconstruction and segmentation method and system based on semi-Gaussian pruning. The method comprises the following steps: respectively obtaining an SFM point cloud and a consistency label mask of a cross-view label based on an obtained multi-view image; initializing the obtained SFM point cloud into an identity semi-Gaussian point cloud, and performing rendering optimization by using a differentiable renderer; densifying the initial sparse point cloud by using a localized semi-Gaussian point management method, and identifying a local error region for resetting and repairing; using the obtained consistency label mask to supervise Gaussian identity feature learning by using cross entropy loss, and using unsupervised 3D regularization loss to force spatially adjacent gauss to maintain identity consistency; according to the method, the identity coding semi-Gaussian kernel method is adopted, the inherent representation fuzziness of a single opacity formula is eliminated, and the positive influence on the identity coding precision is generated.

The invention discloses a volume video figure rendering method, system and equipment based on double-layer Gaussian splashing, a chip and a medium. The method comprises the following steps: S1, acquiring human body pictures with actions from a plurality of visual angles; s2, capturing global motion by using joint Gauss, capturing appearance details by using skin Gauss, and establishing mapping between the skin Gauss and the joint Gauss; s3, performing double-Gaussian optimization by using a coarse-to-fine optimization strategy, and ensuring time consistency; and S4, respectively providing a residual quantization compression method, a picture coding and decoding compression method and a persistent codebook compression method for different Gaussian attributes. According to the invention, through multi-view RGB video input, a double-layer Gaussian splashing and coarse-to-fine optimization strategy is used, so that the rendering quality with extremely high competitiveness is achieved; different Gaussian point cloud attributes are compressed through residual quantization compression, picture coding and decoding compression and a persistent codebook, and the method is superior to the prior art in the aspects of rendering quality, efficiency and storage efficiency.

The invention discloses a three-dimensional reconstruction method based on a three-dimensional Gaussian sputtering technology and related equipment, and belongs to the technical field of three-dimensional reconstruction. Firstly, cyclic progressive Gaussian initialization is adopted, and the density of Gaussian point clouds is increased by utilizing pseudo views in the training process, so that more comprehensive geometric coverage is realized; secondly, depth alignment regularization is implemented, depth information is aligned through a sliding window mechanism in combination with precise depth and monocular depth estimation obtained by initializing the 3D point cloud, and effective geometric constraints are provided; finally, a sparse friendly sampling strategy is introduced, Gaussian ellipsoids which are excessively expanded due to sparse input data are processed in a targeted mode, and the geometric and rendering quality of details is improved by segmenting the ellipsoids.

Disclosed in the present invention is a three-dimensional Gaussian splatting optimization method for an unposed input. The method comprises: for an input image, predicting a ray bundle distribution by using a ray prediction model, to obtain distribution features in the form of ray bundles; on the basis of the distribution features in the form of ray bundles, calculating a camera pose; for the ray bundle distribution, performing sampling on the basis of the volumetric density of light rays, to obtain an initial spatial distribution of a three-dimensional Gaussian point cloud focused on a visual center area; for the input image, obtaining a visible shell by means of view frustum projection and object-mask computation; on the basis of the initial spatial distribution of the three-dimensional Gaussian point cloud and the visible shell, performing three-dimensional Gaussian splatting scene training, to obtain a three-dimensional scene reconstruction model satisfying a preset loss function criterion, the loss function comprising a training regularization term for a camera pose parameter. The present invention provides important scene initialization information for three-dimensional Gaussian splatting training, and significantly improves the quality and the richness of detail of the final three-dimensional structure.

The invention discloses rendering equipment based on three-dimensional Gaussian sputtering. The hardware architecture includes a memory interface, a pre-processing engine, a cardinality ordering engine, a rasterization engine, and the like. The preprocessing engine converts the three-dimensional Gaussian point into a two-dimensional representation and generates a key value pair containing a tile identifier and depth information; the cardinal number sorting engine performs staged efficient sorting through a parallel processing mechanism and a BRAM alternate working mode; and the rasterization engine performs rendering processing by adopting a full-pipeline design and an early-stage stopping mechanism. According to the method, parallel execution of depth sequencing and rasterization is achieved, memory access and calculation redundancy are reduced through optimized data flow design, an efficient and low-power-consumption hardware acceleration solution is provided for three-dimensional Gaussian sputtering rendering, and the method is particularly suitable for application scenes such as virtual reality, games and scientific visualization requiring real-time rendering.

The invention provides a three-dimensional map construction method and device, computer equipment and a storage medium, and belongs to the field of image processing.The method comprises the steps that image information is obtained in real time; estimating pose information of the target camera; performing sub-mapping processing on the image information to obtain multiple pieces of sub-map information; performing dense mapping on each piece of sub-map information by adopting Gaussian sputtering, generating Gaussian point cloud corresponding to each piece of sub-map information, and generating a region mask corresponding to each piece of sub-map information in combination with the image information; determining a high color gradient region and a sparse region according to the region mask; constructing a multi-layer Gaussian pyramid of the key frame based on the high color gradient region and the sparse region; and reducing the number of layers of the pyramid through iterative training, completing rendering of the sub-map information, and obtaining a three-dimensional map corresponding to the image information. Therefore, computing resources are saved, the data processing efficiency and the map rendering speed are improved, and the SLAM performance is improved.

The invention discloses a dynamic scene repairing method based on four-dimensional Gaussian sputtering, which is characterized in that efficient dynamic scene space repairing is realized based on a repairing framework of a four-dimensional Gaussian sputtering monocular video reconstruction algorithm and a video repairing model, and by counting the time frame number of Gaussian primitives in a mask area and removing Gaussian points exceeding a threshold value, the dynamic scene space repairing efficiency is improved. According to the method, high-efficiency dynamic object removal is realized, and a mask weighting loss function is introduced to guide the repair area to converge to real texture information in a subsequent time frame, so that the repair quality is improved. The invention provides a dynamic scene repair method based on four-dimensional Gaussian sputtering, and provides a repair framework based on a four-dimensional Gaussian sputtering monocular video reconstruction algorithm and a video repair model for removing and repairing a target object in a dynamic scene to realize three-dimensional repair of the dynamic scene.

The invention provides an infrared scene reconstruction and rendering method and device based on three-dimensional Gaussian splashing, and the method comprises the steps: carrying out the processing of preprocessed infrared image data through employing an SfM algorithm, obtaining a sparse three-dimensional point cloud and a camera pose, and setting a Gaussian ellipsoid at the center position of the sparse three-dimensional point cloud, so as to construct a three-dimensional Gaussian point cloud model; inputting the camera pose into a Gaussian microrasterized renderer to render the three-dimensional Gaussian point cloud model to obtain a two-dimensional image of a corresponding view angle; the two-dimensional image and the infrared image data of the camera pose are compared through a loss function, the model is trained, a trained three-dimensional Gaussian point cloud model is obtained, and the loss function comprises a temperature gradient field, the projection position of a Gaussian ellipsoid in a pixel plane under the current camera visual angle and the main axis projection direction; and processing the trained three-dimensional Gaussian point cloud model by using a three-dimensional graphic engine and a 3D Gaussian rasterization renderer to obtain an infrared simulation image. And the authenticity of reconstruction and rendering of the infrared scene is relatively high.

The embodiment of the invention provides a foreground target segmentation and reconstruction method and device, equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: directly inputting a video frame into a target segmentation model so as to generate a foreground target segmentation mask, associating the foreground target segmentation mask with a corresponding initial Gaussian point cloud as input data, inputting the input data into a space-time Gaussian splash model for data processing, and performing time slice division on the input data so as to obtain a time slice division result; and obtaining a slice Gaussian point cloud corresponding to each time slice, obtaining a two-dimensional Gaussian curved surface corresponding to each slice Gaussian point cloud, and obtaining a rendered image corresponding to each visual angle according to the plurality of two-dimensional Gaussian curved surfaces. And obtaining a foreground target rendering sequence at the processing moment according to the rendered image corresponding to each visual angle, and generating a foreground target reconstruction result according to the foreground target rendering sequences at all acquisition moments. While the dependence on external prior information is reduced, the stability of a time sequence can be ensured, and the accuracy of a rendering result is improved.

The invention discloses a building three-dimensional reconstruction method based on 3D Gaussian sputtering guidance. The building three-dimensional reconstruction method comprises the following steps: S1, acquiring multi-view building image data; s2, sparse point cloud reconstruction of the building is carried out from the multi-view building image based on a motion recovery structure SFM, and then the point cloud of the building is initialized through 3D Gaussian Splitting; s3, performing multi-scale optimization representation on the initialized 3D Gaussian point cloud, accurately capturing local geometric features and performing optimization adjustment on surface fitness so as to enhance representation of a building surface; s4, building patches are generated based on three-dimensional Gaussian distribution; and S5, obtaining a building three-dimensional model and a rendering result. According to the scheme, a highly detailed surface can be recovered, and high-precision and high-fidelity reconstruction of the geometric shape and appearance of a building in a complex scene is realized.

The invention provides a rendering network training method, a rendering method and a related device, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning and the like, and can be applied to scenes such as content generation based on artificial intelligence. According to the specific implementation scheme, a spherical depth estimation network based on a three-dimensional Gaussian rendering network processes a reference view and a source view to obtain a spherical depth map of the reference view; the reference view and the source view are panoramic images; processing the spherical depth map and the reference view based on a three-dimensional Gaussian model of a three-dimensional Gaussian rendering network to obtain three-dimensional Gaussian points; rendering the three-dimensional Gaussian point to a target visual angle to obtain a rendering result; determining a loss value based on the spherical depth map, the rendering result and a target truth value of the target visual angle; and optimizing model parameters of the three-dimensional Gaussian rendering network based on the loss value.

The invention provides an image rendering method, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models, augmented reality and the like, and can be applied to scenes such as three-dimensional reconstruction and the like. According to the specific implementation scheme, a target grid model is generated based on a three-dimensional Gaussian sputtering model, the three-dimensional Gaussian sputtering model comprises a plurality of Gaussian points used for describing an object to be rendered, and the target grid model comprises respective target patches of the Gaussian points; the transparency of the middle area of the target patch in the target mesh model is lower than that of the peripheral area of the target patch; taking the target grid model as a mask to project a shadow, and obtaining a shadow map of the to-be-rendered object under a predetermined resolution; rendering the to-be-rendered object based on the three-dimensional Gaussian sputtering model and the shadow map to obtain a rendered target image; and displaying the rendered target image. The invention further provides an image rendering device, electronic equipment and a storage medium.

The invention discloses a three-dimensional segmentation method based on three-dimensional Gaussian sputtering and an SAM two-dimensional segmentation model, and belongs to the field of three-dimensional image processing, and the method comprises the following steps: obtaining an RGB image of an existing public data set, and training a three-dimensional Gaussian scene through a three-dimensional Gaussian sputtering method; constructing a new view angle selector, and selecting an optimal segmentation view angle for three-dimensional segmentation through the new view angle selector; a two-dimensional segmentation mask is calculated by means of an SAM two-dimensional segmentation model, a three-dimensional Gaussian scene is initially segmented by adopting a separation algorithm based on three-dimensional Gaussian space information, and a rough three-dimensional segmentation mask is obtained; constructing a background filter combining three-dimensional Gaussian depth and color information, filtering background Gaussian points in the rough three-dimensional segmentation mask, and reserving a foreground segmentation result, thereby obtaining a final three-dimensional Gaussian scene; and rendering by adopting a three-dimensional Gaussian sputtering method to obtain a foreground image after the background is removed. According to the invention, accurate foreground-background segmentation can be realized.

The invention discloses a photographing viewpoint planning method for hub surface defect detection, and the method specifically comprises the following steps: carrying out the Gaussian mapping of a triangular patch model of a to-be-detected hub, and forming a Gaussian ball; dividing the triangular patch model into a plurality of independent areas, wherein normal vector included angles corresponding to Gaussian points in the independent areas are close and topologically adjacent; searching adjacent independent areas, wherein the average normal vector included angle between the adjacent independent areas and each independent area is smaller than the field angle of the camera, and combining the found adjacent independent areas with the corresponding independent areas; constructing an OBB bounding box of each independent area, and determining a camera viewpoint of the OBB bounding box surrounded by the camera visual cone model; and detecting the shielded camera viewpoints, and adjusting the positions and poses of the shielded camera viewpoints to obtain a viewpoint set during photographing of the camera. The photographing viewpoint is generated in a manner of partitioning the triangular patch model of the hub to be detected, so that viewpoint redundancy caused by uniform sampling or random sampling on the surface of the triangular patch model is avoided, and the viewpoint generation efficiency is improved.

The invention discloses a plant three-dimensional Gaussian phenotype extraction method based on an improved parameter adaptive fast pose optimization algorithm, and the method comprises the steps: obtaining a multi-view image sequence of a plant through a camera system, estimating the pose of a camera through a global adaptive fast pose optimization algorithm, and generating a sparse three-dimensional point cloud of a scene, pose estimation errors in an agricultural scene can be effectively eliminated, the problem of error accumulation in a traditional method is avoided, and high-precision camera pose estimation is guaranteed; secondly, taking the camera pose and the sparse three-dimensional point cloud as input, converting an object in a scene into a plurality of three-dimensional Gaussian points, performing three-dimensional reconstruction based on a Gaussian representation method, and generating a dense three-dimensional point cloud, thereby solving common shielding and dynamic change problems in an agricultural scene, and improving the quality and efficiency of point cloud reconstruction; and finally, based on the obtained dense three-dimensional point cloud, phenotypic characteristics of the plants are extracted, and more accurate data support is provided for crop phenotypic analysis.

The invention relates to an editable human body reconstruction method and system based on three-dimensional Gaussian splashing, and the method converts a dynamic human body reconstruction problem into a reconstruction problem of a static template human body in a standard space, achieves the respective decoupling of the shape and the appearance through a mode of constructing the mapping of a three-dimensional Gaussian model and a parameterized human body model, and achieves the reconstruction of the human body. Human body reconstruction in an editable form is achieved, and the editable form comprises the steps that geometric editing is achieved by changing a body shape coefficient and a pose coefficient in the parameterized human body model; and replacing the vertex mapped three-dimensional Gaussian point cloud in the editable three-dimensional Gaussian model to realize texture editing. Compared with the prior art, the method solves the problem that personalized editing cannot be carried out on the human body model based on three-dimensional Gaussian splashing in the prior art.

The invention discloses a four-dimensional Gaussian video method and a four-dimensional Gaussian video device combined with motion layering, which realize vivid volume video generation and streaming transmission in a dynamic scene. According to the method, a streaming Gaussian Group (GOG) representation is constructed, the representation reconstructs a dynamic scene by using Gaussian point clouds grouped according to a time sequence, and a two-dimensional segmentation result is promoted to a three-dimensional motion layering in each GOG in combination with a motion layering technology to obtain layering of static points and dynamic points, so that the dynamic scene is reconstructed. And modeling a dynamic portion in the scene by predicting a motion offset of the dynamic point. In addition, the invention provides a self-adaptive static point conversion mechanism for converting static points into dynamic points in motion modeling, so that appearance changes (such as shadow movement and reflection changes) of static objects are effectively processed.

The invention discloses a three-dimensional modeling method, electronic equipment, a storage medium and a computer program product, and relates to the technical field of data processing, and the method comprises the steps: retrieving a four-dimensional bilateral affine grid based on a spatial dimension index and a color dimension index of a three-dimensional Gaussian point of a training view to obtain an affine transformation matrix, adjusting the color dimension index through the affine transformation matrix to obtain an adjusted color dimension index; rendering the training view based on the adjusted color dimension index through a three-dimensional Gaussian splash model to obtain a rendered image; and based on the luminosity loss between the rendering image and the training view, performing parameter optimization on the four-dimensional bilateral affine grid and the three-dimensional Gaussian splash model to obtain an optimized three-dimensional Gaussian splash model, and performing three-dimensional modeling on the multi-view image based on the optimized three-dimensional Gaussian splash model. According to the method, the floating objects in the scene are reduced, and the geometric modeling accuracy of the scene is improved, so that the rendering quality and the reconstruction accuracy in the three-dimensional modeling process are improved.

The invention relates to the technical field of synchronous positioning and mapping, in particular to a positioning mapping method and system based on three-dimensional Gaussian splashing and a depth camera, and the method comprises the steps: collecting the image and depth information of a target scene in real time, and obtaining the data of a current frame and the corresponding Gaussian point cloud data; based on the current frame data and the corresponding Gaussian point cloud data, adopting an Adam optimization algorithm and analyzing a Jacobian matrix to estimate the camera pose of the current frame; selecting a key frame from a plurality of current frames based on the camera pose, and constructing a dynamic key frame window based on the key frame; and according to the key frame data and the corresponding Gaussian point cloud data, adding a Gaussian point cloud in an area with insufficient coverage, and carrying out iterative rendering on the key frame in the dynamic key frame window to complete the construction of the three-dimensional map. According to the invention, rapid and high-quality positioning mapping can be realized.

The invention provides a scene understanding method and device of a 3D Gaussian point based on semantic knowing. The method comprises the following steps: acquiring picture information of a plurality of camera visual angles in the same scene; the picture information is input into a pre-trained scene understanding model, a semanteme-knowable 3D Gaussian point output by the scene understanding model is obtained, the semanteme-knowable 3D Gaussian point is used for determining a semantic category in a 3D scene, and the scene understanding model is obtained through training based on a semantic presentation loss function and a geometric continuity loss function. According to the scene understanding method and device of the 3D Gaussian points based on semantic knowing provided by the invention, the picture information of multiple camera visual angles of the same scene is input into the pre-trained scene understanding model, and the additional semantic attribute is added to each 3D Gaussian point to record the semantic category to which the Gaussian point belongs; the accuracy of semantic tags can be improved when a 3D scene is understood.

The invention relates to a real-time digital person generation method and system, and the method comprises the following steps: obtaining a monocular video of a target person, and extracting 3DMM information in the monocular video; performing Gaussian point initialization on the monocular video according to the 3DMM information to obtain Gaussian parameters in a standard space; extracting voice audio features in the monocular video, and inputting the voice audio features into the audio-motion model to obtain a universal face key point motion sequence; converting the universal face key point motion sequence into a target face key point motion sequence through a projection algorithm; inputting the target face key point motion sequence and the Gaussian parameters in the standard space into a face key point guided Gaussian deformation network to obtain Gaussian deformation parameters; and rendering the Gaussian deformation parameter into a corresponding video frame through a Gaussian rasterizer, thereby obtaining a digital person video of the target person. Compared with the prior art, the digital human video generation accuracy and creation flexibility are improved.

The invention relates to a three-dimensional reconstruction-oriented Gaussian point cloud lossless coding and decoding method, a product and a storage medium. The method comprises the following steps: extracting a first Gaussian point cloud corresponding to anhor data in a numpy array format at a coding side; converting the first Gaussian point cloud in the numpy array format into a first Gaussian point cloud in a ply format; meanwhile, through a Morton code-based spatial sorting method, performing Morton code sorting on geometric data and attribute data of the first Gaussian point cloud in the numpy array format, and generating a Morton sequence index table; and carrying out AVS coding on the first Gaussian point cloud in the ply format to obtain compressed data in a binary code stream form. On the decoding side, AVS decoding is carried out when the compressed data in the binary code stream form and the Morton sequence index table are received, and a second Gaussian point cloud in the ply format is obtained through reduction; performing format conversion on the second Gaussian point cloud in the ply format to obtain a second Gaussian point cloud in a numpy array format; and rearranging the attribute data and the geometric data based on the Morton sequence index table to realize one-to-one correspondence of the geometric data and the attribute data between the first Gaussian point cloud and the second Gaussian point cloud before and after coding and decoding. Compared with an existing method, the data consistency before and after coding is improved.

The invention provides a three-dimensional scene reconstruction method and system based on slam and 3D Gaussian fusion, and the method comprises the steps: S1, carrying out the shooting of a target indoor scene, obtaining an indoor image, obtaining a dense point cloud A, and carrying out the scanning, and obtaining a dense point cloud B; s2, aligning the dense point cloud B to the dense point cloud A to obtain a fused point cloud; s3, processing the fused point cloud to generate a plurality of Gaussian points, and recording a point cloud data point corresponding to each Gaussian point; s4, aligning the normal of each Gaussian point with the normal of the fusion point cloud based on each point cloud data point; s5, limiting the tangent vector distance between the center of each Gaussian point and the center of the fusion point cloud; s6, limiting the normal vector distance between the center of each Gaussian point and the center of the fusion point cloud; and S7, obtaining a reconstructed three-dimensional scene for measuring an image distance between any two points in the three-dimensional scene and converting the image distance into a real distance. The method has the beneficial effect that the visual effect of the three-dimensional reconstruction scene can be improved.

The invention relates to an object attitude estimation method based on scene-level semantic three-dimensional Gaussian spatter, which comprises the following steps of: firstly, constructing scene-level three-dimensional representation according to a multi-view image, and associating semantic embedding of three-dimensional Gaussian points of each target object; positioning a target mask area of a target object described by a language instruction according to the query image, and extracting each target three-dimensional Gaussian point subset through semantic matching and three-dimensional space clustering; secondly, an ICP registration method is guided through two-stage learning, and the initial 6D pose of the target object is obtained; and finally, performing cascade fine optimization by combining pose rendering and similarity comparison under disturbance to obtain a 6D pose of the target object in the target scene. According to the design scheme, target retrieval and instance extraction under open vocabularies are achieved through three-dimensional Gaussian reconstruction of semantic enhancement, and the attitude estimation precision under the conditions of shielding and low texture in a complex scene is effectively improved by combining language prompt and registration and rendering fine optimization guided by two-stage learning.

The invention discloses a dynamic three-dimensional scene representation method based on Gaussian splashing, and belongs to the field of machine vision. A first time feature constructed through a first neural network based on the time index is fused with the Gaussian feature of the original Gaussian point cloud, and then decoding is performed to obtain various parameters of the original Gaussian point cloud; and fusing a second time feature obtained through second neural network coding based on the time index with the feature obtained by the micro renderer, so that time domain information is introduced in the image reconstruction process to assist in constructing the incidence relation of adjacent video frames, and the time domain expression capability of the model is further enhanced. According to the method, the convolutional neural network is combined with Gaussian splash, the modeling capability of a model for a dynamic scene can be effectively enhanced by fusing time domain features, and efficient expression of a dynamic three-dimensional scene is realized; time domain redundancy is eliminated by modeling a time domain incidence relation, and efficient expression of a dynamic scene is achieved.

The embodiment of the invention provides a Gaussian point cloud rendering method and device, equipment and a storage medium, and relates to the technical field of computer vision. The method comprises the following steps: acquiring pixel points of foreground targets in two corresponding key frames, generating a double-view-angle Gaussian point cloud based on the pixel points, quantifying double-view-angle Gaussian parameters of the double-view-angle Gaussian point cloud based on different rate-distortion coefficients to obtain quantized Gaussian parameters corresponding to each rate-distortion coefficient, and obtaining the target foreground target in the two key frames according to the quantized Gaussian parameters of the double-view-angle Gaussian point cloud and the quantized Gaussian parameters of the double-view-angle Gaussian point cloud. At least entropy coding is carried out on the quantized Gaussian parameter to obtain a coded Gaussian parameter; and determining a target rate-distortion coefficient according to the definition requirement, selecting a coding Gaussian parameter corresponding to the target rate-distortion coefficient as a target coding Gaussian parameter, and sending at least one target coding Gaussian parameter corresponding to each dual-view sequence to a decoding end. The frame extraction operation is performed on the video frame, so that the data volume can be reduced. Secondly, quantitative Gaussian parameters corresponding to different definitions are obtained according to different rate distortion coefficients, and balance of data transmission and rendering quality is achieved.

The invention relates to the technical field of computer vision, and discloses a sparse view-based three-dimensional image reconstruction method and device and computer equipment, and the method comprises the steps: obtaining a multi-view image of a target human body wearing clothes; fitting a three-dimensional human body template of the target human body; extracting a multi-view clothing image; according to the multi-view garment image and the three-dimensional human body template, generating an initial Gaussian point cloud of the garment; constructing a boundary range for the initial Gaussian point cloud, and removing free Gaussian points outside the boundary range; constructing a loss function of two-dimensional Gaussian sputtering, performing multiple optimization iteration training on the initial Gaussian point cloud, and continuously removing free Gaussian points outside the boundary range in the optimization iteration training process to obtain a reconstructed garment model; generating a three-dimensional human body model with clothes according to the three-dimensional human body template and the reconstructed clothes model; according to the method, the quality and availability of three-dimensional human body model reconstruction are improved.

The invention discloses an object level motion blur three-dimensional scene synthesis method and an object level motion blur three-dimensional scene synthesis device based on three-dimensional Gaussian splash (3D Gaussian splash, 3DGS). According to the method, a multi-view image and corresponding camera parameters are used as input, and three-dimensional Gaussian point scene representation including spatial position, scale, rotation, opacity and appearance characteristics is constructed; generating a binary mask of a moving foreground object and a static background by using an image segmentation model; on the basis, camera exposure time is dispersed into a plurality of time steps, space pose parameters changing along with time are introduced only for three-dimensional Gaussian points corresponding to the foreground object, instantaneous three-dimensional scene representation under each time step is obtained, integration or weighted averaging is carried out on rendering results of the plurality of time steps, and the real-time three-dimensional scene representation of the foreground object is obtained. Generating a motion blur rendering result with real physical significance; meanwhile, a mask-guided foreground-background separation rendering strategy and a partition loss function are adopted, clear rendering supervision is applied to a static background area, fuzzy rendering supervision is applied to a moving foreground area, and joint optimization of three-dimensional Gaussian scene parameters is achieved; in the reasoning stage, the clear background and the fuzzy foreground are fused and output according to the mask, and a three-dimensional scene reconstruction result with the clear background and the reasonable motion fuzzy effect of the foreground is obtained. According to the method, the exposure integral process and the object movement track are explicitly modeled in the three-dimensional geometric space, and object-level controllable fuzzy three-dimensional scene generation is realized.

The invention discloses a method for constructing a 3D Gaussian model based on threshold and statistical filtering optimization, which comprises the following steps: firstly, carrying out batch background removal on a reconstructed picture, cutting the size of the picture into half of the original size, and carrying out 3DGS reconstruction by taking the image as input data; and removing black Gaussian ellipsoids and discrete points by adopting a threshold and statistical filtering method, finally optimizing a 3D Gaussian point cloud file format, removing useless parameters in the 3D Gaussian point cloud file, and exporting the optimized point cloud file. According to the method, the background of the picture is firstly removed, the sparse point cloud reconstruction speed is accelerated, only the Gaussian model of the target object is generated, in addition, the black Gaussian ellipsoid and discrete points are removed by adopting a threshold and statistical filtering method, and the accuracy and the stability of the model are improved.