Low Latency Medical Image Segmentation via Server-Client DNN Split

Solution Overview

Problem

Existing web-based medical imaging systems face challenges in providing real-time, interactive segmentation of medical images due to high latency caused by computationally extensive processing tasks, especially when multiple users request 3D organ segmentation.

Innovation Solution

A cloud-deployment architecture that utilizes a deep neural network model comprising an encoder and a decoder network, where the encoder is deployed on the server side to generate a compressed representation of 3D medical images, and the decoder is deployed on the client side to perform interactive segmentation with low latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If computationally extensive 3D segmentation processing is performed on the backend server, then segmentation accuracy is improved, but latency increases and real-time interaction is degraded

Engineering Contradiction:

Improvesegmentation accuracyVSAvoidlatency

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system segments the deep neural network into two parts: an encoder network deployed on the backend server and a decoder network deployed on the client device. The encoder performs computationally intensive feature extraction and generates compressed representations, while the decoder performs lightweight interactive segmentation locally. This segmentation of computation resolves the contradiction by distributing processing tasks according to computational requirements and network constraints.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The encoder network performs preliminary processing by generating compressed representations of the 3D medical image data before transmission to the client. This preliminary action reduces the computational burden on the client side and enables faster real-time interaction during the decoding phase, effectively reducing overall latency while maintaining segmentation accuracy.

Inventive Principle:

Principle #10Preliminary action

2Power

If 3D segmentation processing is performed on the backend server, then processing power is sufficient, but network communication overhead increases latency

Engineering Contradiction:

Improveprocessing powerVSAvoidlatency

Core Design Contradiction:

PowerVSLoss of time

Solution Approach 1:

The system segments the deep neural network into two parts: an encoder network deployed on the backend server and a decoder network deployed on the client device. The encoder performs computationally intensive feature extraction and generates compressed representations, while the decoder performs lightweight interactive segmentation locally. This segmentation of computation resolves the contradiction by distributing processing tasks according to computational requirements and network constraints.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system extracts the interactive segmentation functionality from the backend server and places it on the client device through the decoder network. This extraction eliminates the need for continuous network communication during real-time interaction, as the client can perform segmentation operations locally using the pre-transmitted compressed representations and model weights.

Inventive Principle:

Principle #2Taking out (Extraction)

3Productivity

If a deep neural network model is deployed on the server side for 3D segmentation, then processing capability is improved, but network communication requirements increase latency

Engineering Contradiction:

Improveprocessing capabilityVSAvoidlatency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system segments the deep neural network into two parts: an encoder network deployed on the backend server and a decoder network deployed on the client device. The encoder performs computationally intensive feature extraction and generates compressed representations, while the decoder performs lightweight interactive segmentation locally. This segmentation of computation resolves the contradiction by distributing processing tasks according to computational requirements and network constraints.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system extracts the interactive segmentation functionality from the backend server and places it on the client device through the decoder network. This extraction eliminates the need for continuous network communication during real-time interaction, as the client can perform segmentation operations locally using the pre-transmitted compressed representations and model weights.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS12307674B2Low latency interactive segmentation of medical images within a web-based deployment architecture

Publication Date: 2025.05.20 GE PRECISION HEALTHCARE LLC

AI summary

Techniques are described that facilitate interactive segmentation of medical images with low latency in a cloud-based medical imaging application. According to an embodiment, a system is provided that comprises a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory. The computer executable components comprise a reception component that receives a compressed representation of a three-dimensional (3D) medical image generated from the 3D medical image by an encoder network of a deep neural network model comprising an encoder network and a decoder network, an interaction component that receives user input relative to a displayed portion of the 3D medical image indicating an object included in the displayed portion for segmentation, and a segmentation component that generates a segmentation mask for the object based on the user input using the compressed representation and the decoder network.