An endoscope system based on augmented reality real-time lesion AR labeling method

By using augmented reality technology to preprocess endoscopic video streams and detect lesions, combined with electromagnetic-visual fusion localization and 3D model construction, the problem of accurate localization and annotation of lesions in endoscopic examinations is solved, improving diagnostic efficiency and accuracy.

CN122290109APending Publication Date: 2026-06-26FIRST HOSPITAL AFFILIATED TO GENERAL HOSPITAL OF PLA
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Applications(China)
Current Assignee / Owner
FIRST HOSPITAL AFFILIATED TO GENERAL HOSPITAL OF PLA
Filing Date
2026-03-24
Publication Date
2026-06-26

AI Technical Summary

Technical Problem

There are problems of invalid area interference and motion blur in the real-time video stream of endoscopy. Lesion identification is easily affected by interference objects in the scene. It is difficult to achieve accurate three-dimensional spatial positioning in two-dimensional field of view. Lesion annotation information cannot be effectively integrated with the real-time endoscopy image, which affects diagnostic efficiency.

Method used

An augmented reality-based real-time lesion annotation method for endoscopic systems is adopted. By removing invalid regions and repairing motion blur through preprocessing, a two-stage cascaded model is used for lesion detection and segmentation. Electromagnetic-visual fusion positioning is combined to obtain endoscopic pose data, and two-dimensional lesion information is mapped to three-dimensional space to construct a three-dimensional lesion model and adjust it according to tissue deformation. The resulting AR annotation information is then fused with the real-time endoscopic image.

Benefits of technology

It improves the accuracy of lesion identification and the diagnostic efficiency of endoscopic examination, and enables real-time AR annotation of lesions, providing intuitive visual reference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122290109A_ABST
    Figure CN122290109A_ABST
Patent Text Reader

Abstract

This invention relates to the field of biomedical technology, and more particularly to a real-time lesion AR annotation method for endoscopic systems based on augmented reality. The method first acquires and preprocesses a real-time video stream from the endoscope; then, a two-stage cascaded model is used to detect and segment lesions in the preprocessed video stream, identifying suspicious lesion areas, generating a segmentation mask, and removing interfering areas in the endoscopic scene; real-time pose data of the endoscope is acquired using an electromagnetic-visual fusion positioning method; based on the segmentation mask and real-time pose data, two-dimensional lesion information is mapped to three-dimensional space to construct a three-dimensional lesion model, which is then completed by combining anatomical structural features and adjusted according to tissue deformation; finally, the adjusted three-dimensional lesion model is back-projected onto the endoscopic field of view to generate AR annotation information, which is then fused with the real-time endoscopic image and output. This invention enables real-time and accurate annotation of endoscopic lesions, providing auxiliary support for endoscopic diagnosis and treatment.
Need to check novelty before this filing date? Find Prior Art