Margin Assessment Using Harmonic Generation Microscopy and Deep Learning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional methods struggle to accurately define surgical margins of Extramammary Paget's disease lesions, often leading to incomplete removal due to ill-defined tumor borders and extended tumor spread, which complicates visual assessment and treatment.

Innovation Solution

A margin assessment method combining non-linear harmonic generation microscopy with deep learning, where 3D image groups are generated, labeled, and used to train a deep learning model for accurate classification of malignant and normal skin tissue, enabling real-time, non-invasive margin determination.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional clinical features are used to assess skin cancer margins, then the assessment process is simple, but the accuracy is poor due to ill-defined tumor borders and extended tumor spread

Engineering Contradiction:
Improvemargin assessment accuracyVSAvoidassessment system complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent replaces conventional clinical visual assessment with non-linear harmonic generation microscopy (HGM), which uses optical fields to generate high-resolution 3D images of tissue structures. This substitution enables precise visualization of tumor boundaries and stromal changes that are invisible to clinical examination, achieving accurate margin assessment without requiring complex surgical exploration procedures

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent transitions from 2D clinical surface examination to 3D volumetric imaging using HGM. By capturing multiple focal planes and generating three-dimensional reconstructions of tissue architecture, the system reveals the depth and spatial extent of tumor infiltration that cannot be detected in conventional two-dimensional clinical photos, thereby improving margin definition accuracy

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Reliability

If complete surgical removal is attempted with conventional methods, then treatment thoroughness is improved, but the difficulty in identifying accurate margins leads to incomplete removal

Engineering Contradiction:
Improvecomplete lesion removalVSAvoidtumor border identification
Core Design Contradiction:
ReliabilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent introduces HGM imaging as an intermediary tool between surgical resection and pathological diagnosis. The 3D HGM images serve as a preoperative map that guides surgical margins, allowing surgeons to visualize and respect precise tumor boundaries before cutting, thereby reducing the difficulty of accurate margin identification and improving complete removal rates

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs preliminary 3D imaging and deep learning-based margin prediction before surgical resection. By pre-visualizing the complete extent of tumor infiltration and generating predicted margin maps, the system enables surgeons to plan and execute complete removal more effectively, transforming a postoperative diagnostic challenge into a preoperative planning advantage

Inventive Principle:
Principle #10Preliminary action

3Measurement precision

If deep learning model is trained with labeled 3D image groups, then classification accuracy is improved, but the data processing and model training complexity increases

Engineering Contradiction:
Improvemalignant vs normal tissue classification accuracyVSAvoiddata processing and model training complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent creates digital copies of tissue structures through HGM imaging, generating 3D image groups that serve as training data. These digital representations capture the complete morphological information of both malignant and normal tissues, allowing the deep learning model to learn accurate classification patterns from high-fidelity digital twins of the tissue architecture

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The system transforms conventional 2D histopathological images into 3D volumetric data through HGM, changing the dimensional parameter of the data representation. This transformation provides the deep learning model with richer spatial information and structural context, significantly improving classification accuracy while the automated processing pipeline manages the complexity of data preparation and model training

Inventive Principle:
Principle #35Parameter changes

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

This approach achieves high accuracy in distinguishing malignant EMPD from normal skin with sensitivity of 98.06%, specificity of 93.18%, and accuracy of 95.81%, facilitating precise margin assessment and complete lesion removal.

Implementation Method 1

extracting, by using a harmonic generation microscopy (HGM) imaging system, a plurality of 3D image groups within a range from a surface of a plurality of positions in the predetermined specimen region to a predetermined depth

Methodology Applied
Scientific EffectNon-linear harmonic generation: Second Harmonic Generation

Data Source

PatentUS11899194B2Margin assessment method
Publication Date: 2024.02.13 NAT TAIWAN UNIV
  • US11899194B2 patent drawing
  • US11899194B2 patent drawing
  • US11899194B2 patent drawing

AI summary

A margin assessment method is provided. Under cooperation of harmonic generation microscopy (HGM) and a deep learning method, the margin assessment method can instantaneously and digitally determine whether a 3D image group generated by an HGM imaging system is a malignant tumor or the surrounding normal skin, so as to assist in determining margins of a lesion.