Inpainting Algorithm for Dental Model Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for visualizing virtual three-dimensional dental models in dental augmented reality applications often result in unsatisfactory outcomes due to the 'uncanny valley' effect, where existing teeth are still visible and distract from the realism of the virtual model, especially when the virtual teeth are shorter than the original teeth, leading to psychological rejection.
Innovation Solution
A computer-implemented method using an inpainting algorithm to compute and replace tooth pixels with background pixels, allowing for realistic visualization of virtual dental models by partially removing existing teeth from the image, and overlaying the dental model, which can be performed on a mobile device with limited computing power in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing teeth are kept visible in the image during virtual dental model overlay, then the original dental structure is preserved for reference, but the uncanny valley effect occurs causing psychological rejection and reduced realism
Solution Approach 1:
The patent extracts and removes existing teeth from the image by identifying tooth regions and replacing them with inpainted background pixels, allowing the virtual dental model to be displayed without the disturbing presence of original teeth that cause the uncanny valley effect
Solution Approach 2:
The patent performs preliminary inpainting of the background region before overlaying the virtual dental model, ensuring that the area where virtual teeth will be displayed already has a realistic background appearance, thus preventing the uncanny valley effect from the outset
2Reliability
If inpainting algorithm is used to remove existing teeth from the image, then the uncanny valley effect is reduced and realism is improved, but computational complexity increases
Solution Approach 1:
The patent segments the image processing task into distinct steps: tooth detection, background region identification, inpainting, and virtual model overlay. This segmentation allows each step to be optimized independently and enables real-time processing on mobile devices
Solution Approach 2:
The patent introduces an intermediary inpainting step that generates realistic background pixels to replace removed teeth, serving as a bridge between the original image and the virtual dental model overlay, thereby maintaining realism while enabling efficient processing
3Productivity
If real-time processing is implemented on mobile devices, then user interaction and treatment planning efficiency are improved, but processing power and computational resources are limited
Solution Approach 1:
The patent implements dynamic processing where the inpainting and virtual model overlay are performed in real-time as the user interacts with the system, allowing adjustments to be made immediately without requiring re-scanning or re-processing of the entire image sequence
Solution Approach 2:
The patent performs preliminary processing steps such as tooth detection and background segmentation before the user needs to view the virtual model, and uses cached results from frame-to-frame processing, reducing the computational burden during real-time interaction on mobile devices
Data Source
Figure 1
Figure 2a~2d
Figure 3a~3c
AI summary
Method for realistic visualization of a virtual dental model in a first image of a face comprising at least an inner mouth region of the face, the inner mouth region comprising teeth (31) and a mouth cavity background, the method comprising using an inpainting algorithm to compute, based on background pixels corresponding to the mouth cavity background (35), plausible background pixels for at least a part of tooth pixels of the first image, the tooth pixels corresponding to the teeth, and to replace at least a part of the tooth pixels by plausible background pixels, at least partially removing the teeth (31) from the first image; and visualizing the first image with the at least partially removed teeth (31'), wherein the first image is overlaid with a representation (21) of the dental model.