Multimodal Transformer Medical Data Synthesis for Privacy Protection

The patent generates synthetic medical images that replicate the essential characteristics and diagnostic value of real medical images without containing actual patient data. These synthetic images are created by training a generative model on real medical images and then using the trained model to produce new, fictitious images that maintain statistical properties and anatomical accuracy while eliminating privacy risks associated with real patient data.

The patent introduces an intermediate representation layer where real medical images are first processed to extract key features and patterns, which are then used to train a generative model. This intermediate process allows the system to learn from real data without directly storing or exposing the original patient images, thereby mediating between the need for high-quality training data and privacy protection requirements.

The patent employs a multimodal transformer model that automatically generates and aligns medical images with their corresponding textual descriptions without requiring manual annotation. The model takes medical images as input and autonomously generates accurate radiological reports, eliminating the need for radiologists to manually create text-label pairs and significantly improving data collection efficiency.

The patent pre-trains the generative model on a subset of labeled medical image-text pairs during the training phase, enabling the model to learn the relationship between images and their descriptions. Once trained, the model can rapidly generate new paired data without requiring manual labeling, effectively performing the labeling action in advance during training and eliminating it during deployment.