Natural Language Medical Image Segmentation via Text-Image Embeddings

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional methods for segmenting medical imaging data are imprecise and limited, as pre-defined segmentation tasks fail to accurately represent physician criteria and are restricted to specific tasks, requiring numerous methods that are time-consuming and not comprehensive.

Innovation Solution

A system and method that utilize natural language processing to segment medical imaging data, combining text embeddings with image embeddings generated by AI algorithms, allowing for adaptable and precise segmentation tasks defined in natural language.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If pre-defined segmentation tasks are used, then segmentation can be performed for specific tasks, but the method is imprecise and cannot accurately represent physician criteria

Engineering Contradiction:

Improvesegmentation precisionVSAvoidadaptability to physician criteria

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The system transitions from static pre-defined segmentation tasks to dynamic, flexible segmentation definitions through natural language processing. Physicians can dynamically adjust segmentation criteria by inputting natural language descriptions, allowing the system to adapt to varying physician preferences and specific clinical scenarios without being constrained by fixed task categories.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The invention changes the parameter representation from fixed categorical tasks to continuous natural language parameters. By processing natural language inputs through embedding models, the system converts linguistic descriptions into numerical parameters that can precisely represent nuanced physician criteria, enabling fine-grained control over segmentation behavior.

Inventive Principle:

Principle #35Parameter changes

2Adaptability or versatility

If numerous segmentation methods are developed to cover application cases, then comprehensiveness is improved, but the process becomes time-consuming

Engineering Contradiction:

Improvecoverage of application casesVSAvoidtime for segmentation

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The system implements a universal natural language processing framework that can handle diverse segmentation tasks through a single unified interface. Instead of requiring separate specialized methods for different application cases, the system processes various segmentation requests (lung nodule segmentation, liver lesion segmentation, etc.) through the same natural language understanding pipeline, making one system perform multiple functions.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The invention introduces natural language as an intermediary layer between the user and the segmentation algorithm. This intermediary allows physicians to express complex segmentation requirements in everyday language, which is then translated into computational parameters, eliminating the need for physicians to learn or select from numerous specialized segmentation methods.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Ease of operation

If pre-defined segmentation tasks are used, then the system is simple to operate, but it is restricted to specific tasks and not comprehensive

Engineering Contradiction:

Improveease of useVSAvoidrange of segmentation tasks

Core Design Contradiction:

Ease of operationVSAdaptability or versatility

Solution Approach 1:

The system uses text embedding models to convert natural language descriptions into numerical representations that capture the essence of segmentation criteria. This copying mechanism allows the system to replicate complex physician expertise and segmentation logic through learned embeddings, maintaining simplicity while expanding capability.

Inventive Principle:

Principle #26Copying

Data Source

PatentUS20250104384A1System and method for segmentation of medical imaging data

Publication Date: 2025.03.27 SIEMENS HEALTHINEERS AG

AI summary

A system configured to segment medical imaging data, comprising an input unit configured to receive text data and the medical imaging data, wherein the received text data comprises a segmentation task formulated in natural language with regard to the medical imaging data; a text encoder unit configured to generate a text embedding based on the received text data; an imaging encoder unit configured to generate an image embedding based on the received medical imaging data; a segmentation unit configured to receive the generated text embedding and the generated image embedding and to determine a segmentation of the received medical imaging data via a function trained by an artificial intelligence algorithm; and an output interface configured to output the segmentation of the medical imaging data.