Variable Ray Density Volume Rendering for HMD Ultrasound
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional ultrasound imaging systems face challenges in providing real-time, high-framerate volume rendering of ultrasound images at head-mounted displays, leading to increased GPU workload and potential motion sickness due to the computational expense of rendering complex volume data with sufficient spatial resolution.
Innovation Solution
The method and system employ volume ray casting with a higher density of rays near the user's focal position and decreasing ray density further away, along with a 3D MIP map for varying resolution levels, to reduce processing time and prevent aliasing, utilizing a GPU to retrieve voxel data based on local spacing and sampling distance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If volume rendering is performed with high spatial resolution for each eye at 90-120 Hz frame rate, then image quality is improved, but GPU workload increases by at least one order of magnitude
Solution Approach 1:
The patent applies local quality by casting a greater amount of rays near the user focal position and reducing the quantity of rays as distance from the focal position increases. This creates variable resolution where the focal region maintains high spatial resolution while peripheral regions use fewer rays, thereby improving image quality at the focus while reducing overall GPU workload by approximately one order of magnitude.
Solution Approach 2:
The patent implements partial action by selectively applying high-ray-density sampling only to the focal region rather than uniformly across the entire field of view. The sampling distance along rays within the ultrasound image volume increases as distance from the focal position increases, meaning full computational effort is applied only where needed (at the focus) and reduced effort is applied elsewhere, achieving the required image quality with significantly reduced computational expense.
2Productivity
If the quantity of rays is reduced to decrease processing time, then productivity is improved, but spatial resolution deteriorates
Solution Approach 1:
The patent resolves this contradiction by making ray density spatially variable rather than uniform. A greater amount of rays are cast near the user focal position to maintain high spatial resolution, while the quantity of rays is reduced in regions farther from the focal position. This local quality approach ensures that processing time is decreased overall while spatial resolution is preserved at the critical focal region.
Solution Approach 2:
The patent applies partial action by performing high-resolution ray casting only in the focal region and using reduced sampling elsewhere. The sampling distance along rays increases with distance from the focal position, meaning full computational effort (excessive action) is applied only where high resolution is critical, while partial effort suffices in peripheral regions, thereby achieving acceptable processing times without sacrificing essential image quality.
Data Source
AI summary
A processor receives sensor feedback indicating a focal point of an HMD user. The processor casts rays through an image volume based on the sensor feedback. Each of the rays is associated with a pixel of a rendered image. The rendered image has a first plurality of pixels associated with the rays and a second plurality of unassociated pixels. The rays comprise a first portion cast at or near the focal point and a second portion that is cast farther away from the focal point. A first spacing between rays of the first portion is less than a second spacing between rays of the second portion. The processor determines color values corresponding with each of the first plurality of pixels associated with the rays. The processor determines color values corresponding with each of the second plurality of unassociated pixels of the rendered image. The HMD displays the rendered image.