18 results about "Light field rendering" patented technology

The invention discloses a real-time dynamic rendering method and system for holographic interaction display, and relates to the technical field of holographic interaction display and real-time light field rendering, and the method comprises the steps: collecting a multi-modal interaction signal in real time, and dynamically generating an attention thermodynamic diagram through a space-time alignment algorithm; compressible motion primitives in the scene are extracted based on Lie group SE (3) motion manifold analysis; constructing a sparse sampling light field calculation model according to the attention thermodynamic diagram and the motion primitives, and dynamically adjusting the compression ratio; and performing non-rigid deformation correction on a phase error caused by interaction by adopting a dual-channel feedback compensation mechanism. Holographic display is driven based on a multi-mode interaction signal, the motion trajectory prediction precision is improved through Lie group SE (3) motion manifold analysis, and motion errors are reduced; in combination with an attention thermodynamic diagram and a light field compression strategy, optimal allocation of computing resources is realized, and rendering time delay is reduced; the phase error and non-rigid deformation are suppressed through a dual-channel feedback compensation mechanism, the holographic image quality is improved, and the interaction experience is enhanced.

The invention discloses a digital sand table-oriented minimum system and an end-to-end light field rendering method, and compared with a traditional light field rendering method, the digital sand table-oriented minimum system and the end-to-end light field rendering method adopt a light-level end-to-end rendering method based on a neural network, so that viewpoint synthesis and pixel coding steps in a current light field rendering process are saved. According to the invention, only the light under the viewing angle is rendered based on the human eye position, so that the light field rendering process is simplified, and the light field rendering speed is remarkably improved. Besides, a gray NeRF model with small depth and width is adopted to render most of gray light under the viewing angle according to a proportion, and a color NeRF model with large depth and width is adopted to render a small part of color light under the viewing angle according to a proportion, so that compared with a method for rendering the light only by using the color NeRF model, the method has the advantages that the light rendering efficiency is greatly improved; according to the method, the rendering speed is increased, and the method is more in line with human visual watching characteristics.

The present invention discloses a real-time light field stereoscopic rendering method for immersive head-mounted displays, belonging to the technical field of light field rendering, including: uniformly sampling the original light field data in the angular space, transforming the original data to the YUV color space, and generating a multi-viewpoint image pseudo-video sequence by zigzag scanning; compressing the image pseudo-video sequence with an H.264 video encoder and uploading it to the server; the client decompresses the light field data using WebGL and Three.js to generate a three-dimensional texture map; designing a vertex shader and a fragment shader; using a plane in the three-dimensional space as the proxy geometry of the scene, taking this geometric data as the input of the rasterization rendering pipeline, and using the vertex shader and the fragment shader, combined with the three-dimensional map, to render the binocular viewpoint image. By adopting the above method, the present invention improves the real-time efficiency of light field stereoscopic rendering and meets the requirements of immersive head-mounted displays.

A method for light field rendering includes: obtaining an n-dimensional mixture model, with n a natural number equal to or larger than 4, of a light field. The model is made of kernels wherein each kernel represents light information and is expressed by parameter values; mathematically reducing the n-dimensional mixture model into a 2-dimensional mixture model of an image given a certain point of view, wherein the 2-dimensional model is also made of kernels; rendering a view in a pixel domain from the 2-dimensional model made of kernels.

This application discloses a digital sand table system and a light field image rendering method using image rendering. The system reconstructs a 3D scene model using neural radiation fields and performs light field rendering on the scene image. It uses eye tracking to locate the viewer's position and performs directional light rendering, significantly improving processing efficiency compared to rendering images from all angles. Furthermore, the system reproduces the rendered directional image using light field display technology, allowing the stereoscopic image to change with the viewer, enhancing interactivity. In addition, the system can change the content of the stereoscopic image by modifying the 3D scene image and provides interactive processing operations, offering advantages over traditional sand tables in terms of freely changeable stereoscopic image content and superior interactive effects.

The invention provides a multi-modal CT image data fusion three-dimensional reconstruction method for puncture positioning, and the method comprises the steps: constructing a high-dimensional feature space containing an anatomical structure, metabolic activity and time-varying information through the feature coding of spiral CT and multi-phase PET data; modal alignment is realized by using a mutual information-driven dual-constraint registration mechanism, and cognitive fusion of cross-modal features is completed in combination with dynamic weight tensors of patient physiological features, clinical requirements and organ semantics; and finally, through adaptive light field rendering and topology maintenance optimization, a three-dimensional reconstruction model with medical accuracy and clinical guidance is generated, and accurate visual support is provided for puncture surgery.

The application discloses an immersion interactive method and device for appreciating artifacts, and relates to the technical field of human-computer interaction. The method comprises the following steps: in an immersive digital experience cabin, the three-dimensional coordinates of the user's double pupils and the hand skeleton node coordinates are collected, the gesture instruction is analyzed, and the three-dimensional space viewpoint parameter set of the next moment is predicted; in a virtual engine scene, the UV texture coordinates are obtained by ray intersection, the three-dimensional region of interest is demarcated, the candidate tile set and the tile topology feature matrix are retrieved and generated; based on the pre-constructed fitness function, the improved tornado optimization algorithm is used for solving, and the global optimal tile loading sequence is output; the naked-eye light field rendering and the three-dimensional space multi-modal superposition are performed on the global optimal tile loading sequence, and the fusion picture of the artifacts and the suspended interpretation is presented on the naked-eye 3D cabin screen. The application solves the problems of viewpoint prediction lag, texture tile loading leading to video memory overflow or picture damage, and lack of spatial multi-modal fusion in the prior art.

The invention discloses a GPU acceleration light field image generation method based on Image3DModel, and belongs to the technical field of 3D generation and light field rendering. The method comprises the following steps: preprocessing an input single target image to generate a standardized image prompt; inputting into an Image3DModel multi-view diffusion model, optimizing to obtain a three-dimensional neural radiation field model NeRF, and taking a parameter set of the three-dimensional neural radiation field model NeRF as an intermediate representation of a 3D model; through GPU large-scale parallel volume rendering, a 4D light field data array is generated and optimized; and based on a specified target view angle request, performing interpolation calculation on the 4D light field data array through a light field interpolation engine, and outputting a high-resolution light field image after super-resolution processing. According to the invention, through deep fusion of an advanced generation model and an efficient calculation architecture, a high-quality and geometrically consistent light field image is directly generated from a single image prompt.

A method for light field rendering of 3D scene data. The rendering method includes capturing a light field image at a retraction plane parallel to the display plane, decoding the light field camera to produce a holographic element camera for each holographic element at the retraction plane to produce an integrated image, and applying a pixel remapping technique to pixels of the integrated image for display on the light field display. The 3D scene is captured and remapped to adjust the viewing angle, thereby creating an illusion that the image is taken at the display plane, thereby allowing the captured image to be displayed in the inner and outer frustums of the light field display, thereby creating an immersive experience.

The invention discloses a real-time parallax optimization method for adaptive light field rendering, a medium and equipment. The method comprises the following steps: acquiring a dynamic microscopic image sequence and an auxiliary depth map; inputting the data into a double-flow architecture of a spatial-temporal feature extraction network for preprocessing, and outputting a standardized spatial-temporal feature tensor; inputting the standardized spatio-temporal feature tensor into a pre-trained spatio-temporal double-flow cascade network model for processing, and outputting an initial disparity map; a microscope optical parameter and an initial disparity map are input, AO-Net processing is utilized, a polynomial coefficient is generated, a light field propagation model is utilized, the polynomial coefficient is optimized based on the Fresnel diffraction principle, and corrected light field data are output; inputting an initial disparity map, processing the initial disparity map by adopting a grading strategy, and outputting an optimized disparity map; corrected light field data and an optimized disparity map are input, and a physically optimized 3D structure is output through 3D reconstruction and physical constraint; and inputting the physically optimized 3D structure and the optimized disparity map, and executing real-time rendering. According to the invention, the problem of poor 3D imaging effect is solved.

A method for light field rendering of a 3D scene data. The rendering method comprises capturing a light field image at a retracted plane, parallel to the display plane, decoding a light field camera to produce a hogel camera for each hogel at the retraction plane to produce an integral image, and applying a pixel remapping technique to the pixels of the integral image for display on a light field display. The 3D scene is captured and remapped to adjust the perspective giving the illusion the image was taken at the display plane, allowing captured images to be displayed in both the inner frustum and outer frustum of a light field display, creating an immersive experience.

The application provides a multi-modal CT image data fusion three-dimensional reconstruction method for puncture positioning, constructs a high-dimensional feature space containing anatomical structure, metabolic activity and time-varying information through characteristic coding of spiral CT and multi-phase PET data; realizes modality alignment by using a mutual information driven double constraint registration mechanism, and completes cognitive level fusion of cross-modal features by combining a dynamic weight tensor of patient physiological characteristics, clinical requirements and organ semantics; and finally generates a three-dimensional reconstruction model with medical accuracy and clinical guidance by adaptive light field rendering and topology preservation optimization, so as to provide precise visualization support for puncture surgery.

The present invention proposes a freely expandable stereoscopic digital sandbox system and light field rendering method, wherein the system includes a data encoding module, a data decoding module, a human eye tracking module, a human-computer interaction module, a light field image rendering module, a driving module and a display module. On the one hand, the seamless splicing of high-density display screens is achieved through collimated backlight control technology and an optical microstructure array, so that the display size of the digital sandbox can be freely expanded. On the other hand, in conjunction with human eye tracking technology, on the basis of neural radiation field three-dimensional static scene rendering, in conjunction with traditional graphics rendering pipeline technology, the fusion of static and dynamic scenes is further realized, so that the 3D scenes of implicit expression and explicit expression can be rendered and displayed in real time in a unified framework, and at the same time, the amount of image rendering data is effectively reduced by rendering data only at the human eye viewing position. Combining the above two technologies, observers can obtain a better naked-eye stereoscopic immersion at any viewing angle.

The invention relates to the technical field of three-dimensional display, and discloses a method for dynamically adjusting a picture according to a viewing condition based on self-adaptive naked-eye 3D, and the method comprises the following steps: obtaining a current viewer state of a viewer; on the basis of the state and the rendering confidence map of the previous period, predicting and generating a prediction viewpoint of the next moment by fusing scene significance analysis and confidence feedback weighted correction; based on the predicted viewpoint, left and right eye images are generated through neural light field rendering, and a current period rendering confidence map representing the rendering quality is synchronously generated; and finally, based on the generated image and the current period rendering confidence map, adaptive optical compensation and display are carried out through an active microlens array. According to the method, a closed-loop system from intention prediction to rendering quality feedback to optical compensation is constructed, so that active and forward-looking response to viewing condition changes is realized, and the smoothness, the stability and the perception image quality of naked eye 3D display are remarkably improved.

Systems and methods for fast light field rendering from a three-dimensional (3D) representation of a scene. In at least one embodiment, fast light field rendering exploits cached color values of a plurality of color planes corresponding to a reference view and cached transmittance values of a plurality of transmittance planes corresponding to the reference view to composite a light field quilt via a single sweep through a plurality of sampling planes / volume chunks, thereby enhancing computational efficiency during rendering.

Systems and methods for fast light field rendering from a Neural Radiance Field (NeRF), for example, to visualize a three-dimensional (3D) scene represented by the NeRF on a 3D display. In at least one embodiment, fast light field rendering exploits intersection of sampling points in a ray pattern corresponding to an orthographic imaging array, thereby enhancing computational efficiency during rendering.

The invention discloses a multi-field-domain light field rendering method, system and device for AR glasses and a storage medium, and relates to the technical field of augmented reality. In the method, scene data in a display area covered by AR glasses is acquired, and the scene data comprises environmental geometric information and illumination information; performing multi-field domain division on the display area, wherein multiple field domains comprise a user focus field domain and a non-focus field domain; based on the environmental geometric information and the illumination information, respectively generating light features of a focus field domain and a non-focus field domain; rendering each surface area of the first virtual object in the focus field domain by using the light characteristics of the focus field domain, and rendering preset interval surface areas of the second virtual object in the non-focus field domain by using the light characteristics of the non-focus field domain; and combining and outputting rendering results of the first virtual object and the second virtual object to a display area according to a spatial position relationship between the focus field domain and the non-focus field domain. The rendering efficiency of the AR glasses is improved.

The invention discloses a real-time light field three-dimensional rendering method for an immersive head-mounted display, and belongs to the technical field of light field rendering, and the method comprises the steps: carrying out the uniform sampling of an angle space of original light field data, converting the original data into a YUV color space, and generating a multi-view image pseudo video sequence through zigzag scanning; an H.264 video encoder is adopted to compress the image pseudo video sequence, and the compressed image pseudo video sequence is uploaded to a server; the client side adopts WebGL and Three.js to decompress the light field data, and a three-dimensional texture map is generated; designing a vertex shader and a fragment shader; a plane in a three-dimensional space serves as proxy geometry of a scene, geometric data serves as input of a rasterization rendering pipeline, a vertex shader and a fragment shader are adopted, and a binocular viewpoint image is rendered in combination with three-dimensional mapping. By adopting the method, the real-time efficiency of light field three-dimensional rendering is improved, and the requirements of an immersive head-mounted display are met.