23131results about "3D-image rendering" patented technology

The invention provides a real scene three-dimensional modeling method and system fusing laser point cloud and images, and belongs to the technical field of urban three-dimensional modeling. According to the method, laser point cloud data and multi-view image data of a target scene are acquired through a synchronous acquisition device, and joint calibration is performed on a laser radar and a camera to generate space-time synchronization parameters. And a dynamic weight registration algorithm is utilized to dynamically adjust the registration weight according to the point cloud density distribution and the image texture complexity so as to realize the multi-level registration of the laser point cloud and the image data. The registered data is used for constructing a three-dimensional geometric model and generating a seamless texture map, meanwhile, semantic-driven hole repairing is carried out on a model missing area, and finally a complete three-dimensional model is output. According to the real-scene three-dimensional modeling method and system fusing the laser point cloud and the image, the limitation of single data source modeling is solved, and real-scene three-dimensional modeling with high precision, high integrity and real-time updating is realized.

The invention belongs to the technical field of visual special effects, and discloses an AI visual special effect dynamic generation system fusing multi-modal perception. Through cooperative work of six core modules of multi-modal input processing, importance analysis, parameter configuration, parameterized rendering, parameter optimization and rendering and output, deep fusion of special effects and contents is realized. An image sequence, audio data and scene parameters can be processed at the same time, a multi-modal feature data set is constructed, a scene key visual area is scientifically recognized, a visual importance distribution map is generated, and the special effect parameter configuration and rendering process is guided. An improved neural radiation field technology and a physical constraint model are adopted to ensure the consistency and reality of the special effect at different visual angles; and a multi-dimensional quality evaluation and parameter automatic optimization mechanism is introduced, so that the visual expressive force and the artistic value of the special effect are guaranteed. The creation threshold is remarkably reduced, the cooperative expression ability of the special effect and the content is improved, and the special effect becomes a powerful tool for enhancing the narration and enhancing the emotion.

Disclosed in the present invention is a dynamic human modeling method based on three-dimensional Gaussians. In the present method, a multi-view human video is given firstly, and in order to correct and transform the dynamic human appearance, a latent code and a set of blend weights are added to each Gaussian; and the latent code and a target pose are inputted to an MLP to correct the Gaussian in a canonical space, so as to capture an appearance change under the target pose. The corrected Gaussian is transformed into the target pose by means of the blend weights thereof in a linear blend skinning (LBS) manner. Finally, a realistic human image at a new viewing angle or a new pose can be rendered in real time by means of Gaussian splatting. Compared with an existing method based on an implicit neural radiance field (NeRF), the three-dimensional Gaussian representation of the present invention can better capture high-frequency details while achieving optimal rendering performance.

In some embodiments, a computer system changes a visual prominence of a respective virtual object in response to detecting a threshold amount of overlap between a first virtual object and a second virtual object. In some embodiments, a computer system changes a visual prominence of a respective virtual object based on a change in spatial location of a first virtual object with respect to a second virtual object. In some embodiments, a computer system applies visual effects to representations of physical objects, virtual environments, and / or physical environments. In some embodiments, a computer system changes a visual prominence of a virtual object relative to a three-dimensional environment based on display of overlapping objects of different types in the three-dimensional environment. In some embodiments, a computer system changes a level of opacity of a first virtual object overlapping a second virtual object in response to movement of the first virtual object.

In some embodiments, an electronic device generates virtual lighting effects while presenting a content item. In some embodiments, an electronic device enhances navigation to a respective playback position of a content item. In some embodiments, an electronic device displays media content in a three-dimensional environment. In some embodiments, an electronic device presents media content in different modes of presentation.

The invention provides a reconstruction method of a three-dimensional reconstruction model based on two-dimensional Gaussian splashing, which comprises the following steps: S1, carrying out sparse reconstruction on an input image sequence through a multi-view stereoscopic vision algorithm to generate an initial sparse three-dimensional point cloud and a corresponding camera pose parameter; s2, inputting an improved two-dimensional Gaussian radiation field by using the sparse three-dimensional point cloud and the camera pose as information; s3, dynamically screening a visible anchor point subset based on the current view angle parameter, and generating a rendered image through a differentiable rendering pipeline; s4, calculating a loss function of the rendering image of the training track and the input image to optimize a reconstruction scene; and S5, starting a special visualization tool, and inputting a rendering result. According to the method, by introducing a trimmable anchor point parameterization framework and a multi-scale feature fusion mechanism, light-weight and high-precision three-dimensional scene modeling is achieved, and the problems that traditional 2D Gaussian sputtering is insufficient in multi-view geometric consistency, storage overhead and weak texture region reconstruction and an existing 2D Gaussian splashing method is insufficient in self-adaptive mechanism are solved.

The invention discloses a large-scene three-dimensional reconstruction method based on three-dimensional Gaussian sputtering, and relates to computer graphics. The method comprises the following steps: collecting a multi-view image set of a large scene; obtaining a scene sparse point cloud according to the multi-view image set; performing monocular depth estimation on the multi-view image by using a pre-trained depth prediction network to obtain monocular depth estimation priori; the method comprises the following steps of: performing global training on a scene by utilizing scene sparse point cloud and monocular depth estimation prior to obtain an initial three-dimensional Gaussian model, and performing space grid division on the initial three-dimensional Gaussian model to obtain a plurality of scene blocks with axis alignment bounding boxes; setting image view angle data of each scene block; performing deep supervised training on the Gaussian ellipsoids in the plurality of scene blocks by using a parallel GPU (Graphics Processing Unit); combining the trained scene blocks to obtain a final three-dimensional Gaussian model; in view of low geometric structure reconstruction precision caused by only depending on color information of a multi-view image in large-scene three-dimensional rendering, the method improves the reconstruction precision of large-scene rendering.

The invention discloses a historical block scene three-dimensional reconstruction method and system based on Gaussian sputtering, and the method comprises the steps: collecting a historical block video sequence through a lightweight panorama camera, extracting a multi-frame panorama, and generating an image data set through a projection converter; monocular depth and normal estimation is carried out through a pre-training visual model, and a prior depth and normal graph data set of a historical block scene is constructed; sparse reconstruction is carried out on the multi-view image data set based on the SfM technology, initial point cloud and camera pose information are acquired, and a Gaussian ellipsoid is optimized in combination with prior depth and normal information; dynamically calculating the geometric width estimation value of the street, and guiding and adjusting the adaptive density of the Gaussian ellipsoids of the vertical surfaces on the two sides; rendering the optimized Gaussian ellipsoid through an improved rasterization renderer; and designing a multi-modal loss function and a regularization mechanism to optimize reconstruction and rendering results. According to the method, high-fidelity three-dimensional reconstruction of the historical block scene is realized, and the adaptability of the Gaussian sputtering method to the complex block scene is enhanced.

The invention relates to the technical field of data processing, and discloses a BIM-based building construction quality information intelligent supervision system, which is used for improving the efficiency and accuracy of BIM-based building construction quality information intelligent supervision. The method comprises the following steps: a modeling module performs BIM modeling processing on a building engineering design drawing and building specification data to obtain a building twinborn body; the deployment module carries out field data acquisition system deployment on the building twinborn body to obtain a real-time data acquisition system; the detection module performs anomaly detection processing on the data acquired by the system to obtain a quality anomaly alarm and a priority processing level; the processing module carries out multi-dimensional visualization processing on the quality abnormity alarm and the priority processing level to obtain a visual interface; the decision-making module performs intelligent decision-making analysis on the quality abnormal information in the visual interface to obtain a target processing scheme; and the analysis module performs time sequence analysis and multivariable correlation analysis on the target processing scheme to obtain a dynamic quality evaluation report.

The invention belongs to the technical field of virtual person generation, and particularly relates to a digital person video generation method based on a multi-modal large model, and the method comprises the following steps: 1, constructing a multi-modal data system; 2, multi-modal large model training and adaptation are carried out; 3, constructing a digital human three-dimensional model; step 4, performing semantic analysis and modal mapping; 5, generating a time sequence action and a mouth shape; step 6, building and rendering a virtual scene; step 7, audio and video synchronous rendering and synthesis; step 8, quality optimization and defect repair; and step 9, performing user interaction and iterative optimization. Through technical innovation and engineering, the core pain point in digital human video generation is solved, efficient, vivid and customizable content production capacity is provided for virtual anchors, intelligent customer service, enterprise training and other scenes, and the AI digital human technology is promoted to be applied to large-scale business from experiments.

The invention discloses a three-dimensional Gaussian sputtering scene reconstruction method based on structure perception refined Gaussian, and aims to solve the problems of Gaussian drift, edge blur, structure artifacts and the like of a reconstruction model due to the fact that sparse point cloud contains outliers, Gaussian morphology and normal are mismatched and a multi-dimensional optimization target is lacked in an existing three-dimensional Gaussian sputtering reconstruction method. A key frame is extracted by collecting target scene video data, sparse three-dimensional point clouds are reconstructed by using an SfM algorithm, a depth map and a normal map are generated through a Lotus model, three-dimensional Gaussian distribution is initialized after the sparse point clouds are filtered, a Gaussian covariance matrix is adjusted by using a normal consistency regular term, and the sparse point clouds are extracted. And after structure attribute analysis is carried out, a comprehensive scoring function is constructed to screen Gaussian points, and finally, a combined training framework including luminosity, normal consistency and structure continuity loss is adopted to optimize and generate a three-dimensional Gaussian scene model. The method is mainly applied to the field of three-dimensional reconstruction and multi-view rendering, and scene reconstruction precision and geometric consistency can be improved.

The invention provides a scene reconstruction method based on delayed rendering and three-dimensional Gaussian. The method comprises the following steps: S1, generating initial three-dimensional point cloud data based on a multi-view image; s2, constructing a trainable structural body for three-dimensional Gaussian modeling; s3, normal initialization and residual optimization are carried out on the Gaussian ellipsoid primitives, depth consistency constraint is combined, and a differentiable and learnable normal reconstruction mechanism is realized, so that the geometric expression ability of illumination modeling is enhanced; s4, introducing a reflection training mechanism based on ambient light and a reflection direction, and generating a Gaussian attribute based on a visual angle; and S5, a final image is generated through a differentiable Gaussian sputtering rendering algorithm, and optimization is carried out through pixel loss of the final image and a real image. According to the method, the reality sense and geometric consistency of the Gaussian sputtering model under the complex illumination condition are remarkably improved, and the technical problems of unreal rendering effect, inaccurate surface normal estimation, weak propagation capability and the like of the existing three-dimensional Gaussian sputtering model under the complex illumination condition are solved.

A computer-implemented method for reconstructing a scene in three dimensions from a plurality of images of one or more viewpoints of the scene acquired using an imaging device includes: receiving the plurality of images without receiving extrinsic or intrinsic properties of the imaging device; and processing the plurality of images using a neural network to produce a plurality of pointmaps of the scene that correspond to the plurality of images and that are aligned in a common coordinate frame, where each pointmap is a one-to-one mapping between pixels of one of the plurality of images and three-dimensional points of the scene.

The invention discloses a three-dimensional environment reconstruction optimization method based on multi-sensor fusion data, and relates to the field of three-dimensional environment reconstruction optimization, and the three-dimensional environment reconstruction optimization method based on the multi-sensor fusion data comprises the following steps: S1, collecting multi-source sensor data, and constructing a data set under a unified coordinate system; s2, generating dense visual point cloud, and extracting laser point cloud features to construct a model; s3, establishing a local three-dimensional model, and generating a local environment image; s4, shadow parameters are extracted through shadow geometric analysis, and time sequence optimization is carried out; s5, consistency verification and correction are carried out, and three-dimensional reconstruction data are output; and S6, comparing the reconstruction data with the navigation map database, and carrying out map optimization updating. According to the method, time synchronization and space calibration are carried out on data acquired by the depth camera and the laser radar, complete and accurate three-dimensional information modeling of the target environment is realized, and the geometric precision of environment reconstruction and the image detail reduction capability are improved.

The invention relates to an optimization method and device for sparse view angle three-dimensional Gaussian splash, and belongs to the technical field of three-dimensional reconstruction in computer vision, and the method comprises the steps: collecting a sparse view angle image; a multi-view stereoscopic vision model based on deep learning generates a geometrically consistent depth map for the sparse view image, converts the depth map into point clouds and fuses the point clouds to obtain dense point clouds; sampling dense point clouds by adopting voxel-guided farthest point sampling to obtain initialized point clouds, and constructing a three-dimensional Gaussian field; rendering the three-dimensional Gaussian field through an enhanced geometric renderer to obtain a rendering depth and a rendering normal; constructing a multi-level geometric regularization loss function, and optimizing the three-dimensional Gaussian field; and performing optimization adjustment on the three-dimensional Gaussian field based on a shape-scale constraint criterion and a two-stage adaptive opacity constraint strategy to obtain an optimized three-dimensional Gaussian field. According to the method, the problems of initialization failure, insufficient geometric supervision and element out-of-control of 3D Gaussian splashing under the sparse view angle are solved.

The invention discloses an indoor real scene three-dimensional reconstruction method based on improved 3D Gaussian sputtering, belongs to the technical field of three-dimensional reconstruction, and aims to complement initial point clouds in a self-adaptive manner through a designed feed-forward point cloud densification network, generate a high-density 3D Gaussian radiation field, fill scene missing details and avoid the problem of precision attenuation caused by lack of point cloud constraints. In the cost body optimization stage, a depth regularization loss module is designed by integrating global structure features, depth information is integrated into the optimization process, algorithm reasoning is accelerated, and object structure fuzziness is relieved; and finally, in combination with adaptive density control and Gaussian pruning, radiation field parameters and density distribution are finely adjusted, indoor scene reconstruction with double improvement of visual fidelity and structural integrity is realized, interactive application becomes possible, a real-time solution is provided for indoor scene reconstruction, and application of a three-dimensional reconstruction technology is widened.

A real-time system for the multimodal generation of virtual reality scenes based on artificial intelligence for the creation of immersive three-dimensional environments from natural language narratives, consisting of: a speech capture module configured to continuously record a user's spoken narrative via one or more directional microphones, preprocesses the captured signal by noise reduction and temporal alignment, and outputs a digital speech stream; A speech-to-text processing unit that is operationally coupled to the speech capture module and configured for real-time speech recognition using a continuous neural transformer model. The unit is trained to transcribe natural language utterances into structured text data while maintaining contextual continuity throughout the evolving narrative. a semantic interpretation processing unit that is communicatively linked to the speech recognition unit and configured to perform natural language understanding techniques to extract contextual entities, spatial references, temporal relationships, and object attributes from the transcribed narrative; the engine includes a large language model that is fine-tuned for spatial reasoning tasks; a scene graph generation module configured to transform the interpreted semantic data into a structured, hierarchical representation that defines nodes for identified entities and edges for corresponding relationships, with each node associated with metadata describing geometry, position, orientation, texture, and linking attributes between objects; a multimodal image-language model processor coupled with the scene graph generation module, wherein the processor is configured to retrieve, adapt, or synthesize appropriate three-dimensional elements from a pre-trained visual-lexical embedding space and align these elements with their semantic and spatial definitions derived from the scene graph; a scene assembly and rendering controller configured to create a cohesive virtual scene from the aligned assets, perform real-time rendering using a GPU-accelerated ray tracing pipeline, and produce a stereoscopic visual output that corresponds to the evolving narrative; A head-mounted virtual reality visualization device connected to the rendering engine and configured to display the generated immersive environment to the user in real time. The device features motion sensors and inside-out tracking cameras to detect head and body movements, dynamically updating viewing angles and perspective within the rendered scene; and a bidirectional feedback module integrated into the head-mounted device and connected to the semantic interpretation processing unit; the module is configured to interpret corrective commands, gestures, or supplementary comments from the user to refine or modify specific scene elements without interrupting the real-time visualization; The system continuously updates the virtual scene as the narrative develops, ensuring temporal synchronization between speech input and rendered output below a defined latency threshold, thus enabling a natural, dialogic construction of complex three-dimensional virtual environments.

The invention provides a cloud collaborative three-dimensional building modeling rendering method and system based on WebGL (Web Graphics Library). The method comprises the following steps: acquiring design parameters of a target building, analyzing the design parameters into geometric topological data of a plurality of independent components on the basis of a functional attribute division rule, and generating a multi-stage detail geometric model; adding material physical attributes to the surface vertex data, calculating a diffuse reflection coefficient and specular reflection intensity through a ray tracing algorithm, and generating lightweight component data carrying physical rendering parameters; in response to a view angle switching instruction of a terminal user, according to a space block corresponding to a current view angle, a main GPU node is distributed from a distributed GPU cluster, the main GPU node is divided according to the space block, adjacent blocks of the space block are distributed to edge GPU nodes, parallel ray tracing calculation is executed based on a WebGL protocol, and the parallel ray tracing calculation is executed based on the WebGL protocol. And returning to the main GPU node synthesis terminal display image and displaying. According to the technical scheme provided by the invention, the response speed is remarkably improved while the physical-level rendering effect is kept.

An example operation may include at least one of receiving, via a user interface of a device, an activation input from a user to initiate a session, capturing, by a camera of the device, a scan of a body of the user, wherein the capturing comprises recording at least one image and / or at least one video of the user, processing the at least one image and / or video to generate a three- dimensional model of the user comprising measurements and contours of the body, retrieving, from a database, at least one clothing item associated with the user, the at least one clothing item comprising dimensional attributes and texture attributes, rendering, by a graphics processing unit, the at least one clothing item onto the three-dimensional model to generate a visual representation, wherein the rendering simulates draping behavior, movement, and light interaction of the at least one clothing item relative to the three-dimensional model, and displaying, on the user interface, an interactive visualization comprising the visual representation of the three-dimensional model with the at least one clothing item from multiple viewing angles.

The present invention relates to the field of AR performance optimization, and in particular to an integrated AR device performance optimization method and device based on a multi-core processor. The method includes the following steps: obtaining real-time AR display scene images and real-time user interaction data; performing scene object visual recognition on the real-time AR display scene images, and encapsulating image rendering data to generate multiple AR scene rendering tasks; performing visual rendering computing power calculations on multiple AR scene rendering tasks one by one, and identifying rendering requirements of different scenes to generate rendering requirement data for each task; identifying multi-core processor parameters in the system, performing image rendering computing power perception to generate image rendering computing power for each processor; performing computing power resource pre-allocation matching on the image rendering computing power of each processor according to the rendering requirement data of each task, and constructing a dynamic computing power resource allocation strategy. The present invention realizes dynamic AR device computing power resource allocation and improves the operating performance of AR devices.

The invention provides a virtual reality building roaming method and system based on scene rendering, and relates to the technical field of virtual reality, and the method comprises the steps: dividing building three-dimensional model data through an octree space segmentation algorithm, and determining a detail level threshold according to a viewpoint distance, and constructing a multi-level detail model. And baking the basic illumination chartlet by using a pre-calculation irradiance transmission technology, and performing real-time synthesis in combination with the multi-resolution shadow chartlet. And calculating illumination by adopting a physics-based rendering method, and fusing the illumination with the scene light and shadow synthesis data to generate dynamic illumination rendering data. And determining a current viewpoint based on the spatial position and attitude data of the virtual reality head-mounted display, performing view cone elimination and shielding elimination, and performing GPU instantiation rendering on visible building data. And the rendered image is optimized through high dynamic range processing and a time domain anti-aliasing technology, and a final scene image is generated in combination with an atmospheric scattering algorithm. According to the invention, the rendering efficiency and scene reality sense of virtual reality building roaming can be effectively improved.

Disclosed are systems and methods for generating a 3D model from a single 2D image, the method comprising: receiving a single 2D input image of an object; generating a set of consistent multi-view images based on the single 2D input image using a fine-tuned 2D diffusion model that processes multiple views together in a tiled configuration; constructing a 3D feature volume by projecting 2D patch features from the generated multi-view images using corresponding camera pose information; generating a 3D mesh using a pair of 3D diffusion networks conditioned on the multi-view images, wherein the pair of 3D diffusion networks comprises a first network for generating a coarse occupancy volume and a second network for generating a high-resolution sparse volume; and refining a texture of the generated 3D mesh to produce a textured 3D mesh.

The invention discloses a sparse view three-dimensional reconstruction method and system based on dynamic occlusion perception and multi-prior fusion. The sparse view three-dimensional reconstruction method comprises the steps of obtaining a sparse view; extracting multi-scale geometric features from the sparse view through a feature pyramid network, constructing a probabilistic depth estimation module, and generating an initial sparse point cloud based on the probabilistic depth estimation module; based on the initial sparse point cloud, a dynamic occlusion perception attention mask is introduced, and occlusion area mismatching propagation is inhibited through cross-view visibility analysis and a multi-stage confidence aggregation strategy; and fusing geometric priori, semantic priori and implicit priori, carrying out multi-source priori collaborative constraint through an adaptive weight distribution strategy, enhancing the geometric accuracy of the weak texture region, repairing the occlusion region, and outputting a three-dimensional reconstruction image of the sparse view. According to the invention, the integrity and precision of sparse view reconstruction are improved.

The invention relates to the technical field of intelligent agriculture, and discloses a visual intelligent agricultural planting control system. The system comprises an environment data acquisition module used for acquiring multi-source environment sensing data; the growth feature modeling module is used for extracting crop growth state features through a morphological analysis algorithm; the environment regulation and control decision module is used for generating an environment regulation and control instruction set by utilizing a dynamic threshold matching algorithm; the visual interaction module is used for generating a three-dimensional farmland live-action simulated diagram through multi-dimensional data fusion processing; the strategy execution module is used for driving agricultural facilities to execute actions by adopting a self-adaptive control algorithm; an abnormity early warning module is further arranged, and abnormity early warning signals are generated through correlation analysis. The system realizes comprehensive acquisition and analysis of agricultural planting environment data, precise environment regulation and control, visual and visual display and abnormal early warning, effectively improves the intelligent and precise level of agricultural planting, improves the crop yield and quality, and assists the development of intelligent agriculture.

The invention relates to the technical field of computer vision and three-dimensional reconstruction, and discloses an exhibition hall three-dimensional modeling intelligent optimization system based on multi-modal data fusion, and the system comprises a data collection module which is configured to synchronously obtain laser radar point cloud data, a multispectral image sequence and inertial measurement unit data; the preprocessing module is used for receiving the output of the data acquisition module, aligning a multi-source sensor coordinate system through a space-time calibration algorithm, and separating a static scene from a dynamic interference element by using a dynamic segmentation network; and the multi-modal fusion module is used for receiving the preprocessed data and carrying out adaptive weighted fusion on the geometric features of the laser radar and the visual texture features through a cross-modal attention mechanism. According to the invention, through multi-modal data fusion and a dynamic scene adaptive mechanism, the modeling precision and the real-time updating capability in a complex exhibition hall environment are significantly improved.

A method, apparatus, non-transitory computer readable medium, and system for 3D model generation include obtaining a plurality of input images depicting an object and a set of 3D position embeddings, where each of the plurality of input images depicts the object from a different perspective, encoding the plurality of input images to obtain a plurality of 2D features corresponding to the plurality of input images, respectively, generating 3D features based on the plurality of 2D features and the set of 3D position embeddings, and generating a 3D model of the object based on the 3D features.

The invention discloses a digital twin three-dimensional scene modeling method based on a webGPU, and relates to the technical field of three-dimensional scene modeling and graphic computing, multi-source sensing data are asynchronously sampled from sparse point cloud, a video texture sequence, a scene semantic tag graph and a structure boundary tuple, and a six-dimensional structure unit group is generated through a normalization operator; constructing a structural unit atlas with nodes representing component entities and edges representing constraint relations, and introducing a tension balance mechanism and multi-scale constraints to generate a modeling path prior model; constructing a graph calculation and graph rendering dual-channel assembly line in the WebGPU, and executing parallel texture mapping and boundary fitting operation; a dynamic sensing module is used for capturing scene disturbance and driving atlas response, and incremental reconstruction of the model is achieved; and finally, mapping the model to a Web terminal, and supporting microscopic semantic query and multi-layer data linkage. According to the method, the response speed, semantic consistency and structural adaptability of three-dimensional modeling in a complex environment are improved.