The invention discloses a medical image intelligent diagnosis and three-dimensional
visualization interaction method and
system based on
artificial intelligence and Unreal Engine. The
system is composed of a
data acquisition module, an
artificial intelligence analysis module, a three-dimensional reconstruction module, a rendering interaction module and a
report generation module. The method comprises the following steps: firstly, carrying out
voxel normalization on a
DICOM / NIfTI image; the improved 3D U-Net is combined with CBAM attention and WGAN-GP training, and a focus initial
density field is output; an AI enhanced implicit
voxel reconstruction (AEIVR)
algorithm is proposed, an enhanced
density field is generated by using position coding + MLP, and grid extraction is completed by using improved Marking Cubes of a dynamic threshold; high-fidelity real-time rendering is realized in combination with a Nand micro polygon and Lumen
global illumination in Unreal Engine, and VR / AR interaction and virtual
surgery (real-time updating of resection
mask driving) are supported. According to the method, the problems of large artifacts and insufficient interactivity and real-time performance of existing two-dimensional output and three-dimensional reconstruction are solved, and the end-to-end segmentation-reconstruction-rendering-interaction-reporting process is realized. On typical CT / MRI data, the segmentation Dice is about 0.90-0.94, the reconstruction
voxel resolution is 0.1-0.5 mm, the artifacts are reduced by about 20%-30%, the PSNR is improved by about 3-7 dB, and the rendering interaction
delay lt is achieved; the
system is suitable for scenes such as
lung CT,
brain MRI, orthopaedic X-
ray and
cardiac ultrasound, and can be expanded to
preoperative planning, remote consultation and teaching.