Panoramic endoscope

Abstract

FIELD: medical equipment.SUBSTANCE: endoscope with a panoramic view, comprising a housing, in the distal part of which one main frontal video camera is placed, oriented along the longitudinal axis of the endoscope, and four side video cameras, located along the circumference of the distal part with fixed angular offsets relative to each other, wherein each video camera is equipped with a lighting module with the ability to switch between a wide spectrum mode – white light and a narrowband lighting mode at wavelengths of blue – 415 nm and green – 540 nm. The endoscope additionally contains an image processing unit, configured with the ability to receive video streams from all video cameras and form on their basis a single panoramic image with a viewing angle of 360° in a plane orthogonal to the longitudinal axis of the endoscope.EFFECT: improving the diagnostic accuracy and reliability of endoscopic examination.3 cl, 2 dwg

Description

[0001] The invention relates to medical equipment, namely to endoscopic devices used for visual examination of hollow organs and internal cavities of the human body, such as the gastrointestinal tract (esophagus, stomach, colon), respiratory tract, genitourinary system, as well as in minimally invasive surgery.

[0002] Endoscopic surgery refers to a minimally invasive medical procedure that involves visually examining the internal cavities and hollow organs of the human body (such as the gastrointestinal tract, respiratory tract, and genitourinary system) using a specialized instrument called an endoscope. Unlike many other medical imaging methods (such as X-rays or MRIs), the endoscope is inserted directly into the anatomical area being examined. The device's optical system, using visible or narrowband (e.g., NBI) light, produces images and videos of internal structures in real time. A standard endoscopic procedure involves a thorough examination of the mucosa to identify suspicious areas, such as inflammation, erosions, polyps, tumors, or other signs that may indicate precancerous conditions or malignancies.

[0003] One of the most socially significant applications of endoscopy is colorectal cancer (CRC) screening. CRC ranks among the leading causes of morbidity and mortality worldwide, being the second-third most common cause of cancer death. Early detection and removal of precancerous lesions—colorectal adenomatous polyps—is critical to significantly reducing mortality from this disease. The "gold standard" for CRC diagnosis and prevention is colonoscopy—a procedure that examines the mucosa of the colon and terminal small intestine using an endoscope inserted through the anus. This method not only provides detailed visual diagnostics of pathologies but also allows for immediate biopsy or resection (removal) of detected lesions.

[0004] Despite its high efficiency, traditional colonoscopy has a fundamental limitation: it provides a view only in the direction of the endoscope's movement, leaving "blind spots," particularly behind anatomical folds and curvatures of the intestine. According to studies, the adenoma miss rate (AMR) with standard colonoscopy can reach 20-30%. This means that a significant number of potentially dangerous lesions remain undetected, directly impacting the quality of screening and the patient's long-term prognosis.

[0005] In recent years, computer vision and artificial intelligence (AI) technologies in endoscopy have been rapidly developing to overcome this limitation. Numerous studies have been published on the development of computer-aided detection (CADe) systems based on deep neural networks. However, existing commercial and experimental AI systems focus primarily on the auxiliary task of detecting polyps that have already entered the field of view of the endoscope's single video camera. These systems are designed to draw the physician's attention to a suspicious object, but they do not address the underlying problem: a polyp that was never captured by the video camera due to a "blind spot" cannot be detected by even the most sophisticated image analysis software.

[0006] Thus, the key unsolved problem of modern endoscopy remains the quality of visual coverage. The main challenge is how to radically reduce the number of missed pathologies through a fundamentally new hardware design that eliminates these "blind spots" and integrate this design with an intelligent computer decision-support system. Solving this problem requires a comprehensive approach combining innovations in the optical and mechanical design of endoscopes and the development of specialized algorithms for processing and analyzing panoramic video data.

[0007] From the examined prior art, the applicant has identified an invention according to Russian Federation Patent No. 2770743 "Electronic Endoscope and Electronic Endoscope System". The essence is an electronic endoscope comprising a housing in the form of a tube, at least two lenses with a non-zero angle of view, an image reading unit and a transmission module; wherein the housing in the form of a tube is provided with a light-receiving channel passing through it, the light-receiving channel is configured to accommodating sequentially at least two lenses with a non-zero angle of view, an image reading unit and a transmission module; the optical axes of at least two lenses with a non-zero angle of view are parallel to the axis of passage of the light-receiving channel; the image reading unit is configured to convert the received image, transmitted from at least two lenses with a non-zero angle of view, into an electrical signal and to output the electrical signal;and the transmission module is configured to rotate at least two lenses with a non-zero angle of view around their own optical axes in the same direction; wherein the transmission module comprises a first transmission device, a second transmission device and a transmission coupling means, wherein the second transmission device drives the transmission coupling means, and the transmission coupling means then drives at least two lenses with a non-zero angle of view by means of the first transmission device.

[0008] A drawback of the existing technical solution is the use of rotating lenses with parallel optical axes and a complex transmission mechanism for their rotation. This results in the lateral surface being viewed not simultaneously, but rather over the time interval required to complete a rotation. This inevitably makes the system susceptible to motion artifacts (e.g., due to peristalsis or endoscope displacement), which can distort the generated image and reduce its suitability for real-time analysis. Furthermore, the presence of moving mechanical components directly in the distal portion of the endoscope reduces the overall reliability of the device, complicates its design, increases the diameter of the working part, and creates significant challenges for guaranteed sterilization, increasing risks in clinical practice.

[0009] Furthermore, the existing solution lacks features for processing video data to form a coherent panorama. The system is essentially limited to transmitting sequentially acquired disparate camera angles without addressing the automated synthesis of a single panoramic image with 360-degree coverage, leaving the problem of "blind spots" unresolved. Furthermore, the existing endoscope lacks capabilities for adaptive multimodal visualization and automated analysis of acquired images using artificial intelligence algorithms, significantly limiting its diagnostic potential.

[0010] From the examined prior art, the applicant has identified an invention according to Russian Federation Patent No. 2538626 "Endoscope with a Rotating Prism". The essence is an endoscope with a variable direction of view, the configuration of which allows the device to be introduced into the cranial cavity of a person or an animal; wherein the endoscope includes: an elongated shaft having a proximal end, a distal end and an outer diameter equal to approximately no more than 5 mm; a viewing window located along the shaft at or near the distal end; a rotating prism located inside the shaft closer to the distal end and designed to change the direction of view of the endoscope; and a self-focusing lens located in the shaft and providing automatic focusing of the image appearing in the viewing window as the prism rotates, and a handle connected to the proximal end of the elongated shaft;wherein the handle includes a first rotating scale disk for adjusting the viewing angle of the endoscope by rotating the prism and the first scale disk rotates around the longitudinal axis of the barrel, wherein the prism is installed in a housing functionally coordinated with a rotating axis directed proximally, and the distal part of the axis has a thread for engaging with teeth formed on the housing.

[0011] A drawback of the existing technical solution is its fundamental dependence on mechanically changing the viewing direction of a single optical system, which leads to fundamental limitations in the completeness, speed, and quality of visualization. Despite the use of a self-focusing lens to compensate for changes in prism rotation, the system only provides a sequential, step-by-step inspection of narrow sectors of space, requiring active manual control of the rotating scale disk to redirect the beam. This makes the inspection process labor-intensive, completely operator-dependent, and, critically, does not allow for a complete image of the hollow organ walls at a single moment in time. In fact, this design merely shifts the single "blind zone" circumferentially, but does not eliminate it, leaving a high risk of missing pathological changes when the prism is pointed in the wrong direction.

[0012] Furthermore, the presence of miniature precision mechanics for rotating the prism within the distal end complicates the design, increases its cost, and reduces reliability under conditions requiring continuous sterilization. The solution does not offer any hardware or software mechanism for synthesizing a panoramic view, and its functionality is limited to remotely adjusting the viewing angle without guaranteeing complete coverage.

[0013] Technical result The aim of the claimed invention is to increase the diagnostic accuracy and reliability of endoscopic examination by completely eliminating “blind spots” through providing a one-time panoramic view of 360° around the circumference of the walls of the hollow organ being examined, as well as by expanding the functional capabilities of the device by implementing independent lighting modes for each observation zone and automating the process of identifying pathologies.

[0014] The specified technical result is achieved by the fact thatA panoramic endoscope, comprising a housing and an image processing unit, houses several video cameras in the distal portion of the housing, including a main frontal camera oriented along the longitudinal axis and multiple lateral cameras arranged circumferentially at fixed angular offsets. Each camera is equipped with its own illumination module, independently switching between white light and narrowband illumination modes. Video streams are transmitted via cables to an external processing unit.

[0015] The image processing unit receives video streams from all cameras, calibrates, corrects distortions, and stitches them together to form a single panoramic image with a 360° viewing angle in a plane orthogonal to the endoscope axis and / or a combined 3D image. The technical result is further enhanced by the implementation of at least two display modes (standard and panoramic) in the processing unit and / or the integration of an automatic image analysis module based on artificial intelligence for the detection of pathological changes.

[0016] Automated pathology detection is a module for automatic image analysis based on artificial intelligence (AI). This module, implemented using deep learning algorithms (e.g., convolutional neural networks), continuously processes video data received simultaneously from all cameras. It analyzes both complete panoramic images and individual frames from each camera, ensuring maximum coverage and reliability of analysis. The AI ​​module is pre-trained on extensive labeled endoscopic image databases to detect various pathological changes: polyps, dysplasia, inflammation, vascular anomalies, etc. Upon detection of a suspicious object, the system generates and displays an alarm in real time – a graphic label (e.g., an outline or marker) with a text explanation, superimposed on the panoramic and / or standard image.This provides active assistance to the operator, reduces dependence on the subjective human factor, and significantly increases the likelihood of early detection of pathologies, including in areas traditionally considered difficult to access for visual inspection.

[0017] The essence of the declared technical solutionis an endoscope with a panoramic view, comprising a housing, in the distal part of which one main frontal video camera is placed, oriented along the longitudinal axis of the endoscope, and a plurality of side video cameras, located along the circumference of the distal part with fixed angular offsets relative to each other, wherein each video camera is equipped with a lighting module with the ability to switch between a white light mode and a narrow-band lighting mode, in addition, the endoscope additionally contains an image processing unit, configured with the ability to receive video streams from all video cameras and to form on their basis a single panoramic image with a viewing angle of 360° in a plane orthogonal to the longitudinal axis of the endoscope.

[0018] An endoscope with a panoramic view, in which the image processing unit is designed with the ability to operate in at least two modes: a standard mode, in which only an image from the front video camera is displayed on the display device, and a panoramic mode, in which the specified single panoramic image is displayed.

[0019] A panoramic endoscope that additionally contains an automatic image analysis module based on artificial intelligence, capable of processing video data from all five video cameras to detect pathological changes and output an alarm to the display device.

[0020] The claimed technical solution is illustrated in Fig. 1 – Fig. 2.

[0021] Fig. 1 shows a general view of an endoscope with a panoramic view;

[0022] Fig. 2 shows the overview diagram of each video camera of the endoscope;

[0023] The positions in Fig. 1–Fig. 2 denote:

[0024] 1 – body, 2 – main front video camera, 3 – side video cameras,

[0025] The applicant then provides a description of the claimed technical solution.

[0026] The panoramic endoscope in Fig. 1 comprises an elongated flexible body 1, the distal (working) part of which is inserted into a body cavity.

[0027] A group of video cameras is located in the distal part. The main (frontal) video camera 2 is oriented along the longitudinal axis of the endoscope and provides a view in the direction of movement. Around the frontal video camera, along the circumference of the distal part, are side video cameras 3. Their number is preferably four, with an angular step of 90°, which ensures an optimal balance between the completeness of the view and the complexity of data processing. However, a different number is possible (for example, five, six or seven video cameras) depending on the required panoramic resolution and the diameter of the instrument. The video cameras are rigidly fixed, their optical axes are directed at an angle to the longitudinal axis of the endoscope (for example, in the range from 60° to 120°, preferably about 90°), so that their fields of view cover the side wall. The fields of view of adjacent side video cameras overlap as shown in Fig. 2, which is necessary for accurate image fusion.

[0028] Each video camera (both front and side) is equipped with its own lighting module, containing a light source (for example, white LEDs) and a system of optical filters or separate sources with different spectral characteristics.

[0029] All video cameras are connected via fiber-optic communication lines or electrical conductors to an image processing unit located outside the endoscope body (as part of the workstation). The image processing unit allows for independent switching of illumination modes for each video camera: standard white light or narrowband illumination (e.g., blue and green using NBI technology). This allows, for example, for simultaneous observation of one area in panoramic mode using white light mode and a suspicious area in an adjacent sector using NBI mode for detailed examination of the vascular pattern.

[0030] The image processing unit includes hardware video decoders, graphics processing units (GPUs), and specialized software. The software performs the following key functions: calibration and correction, which eliminates optical distortions from each individual frame. It also combines video streams from the side cameras into a single cylindrical or flat 360° panorama based on camera position data and overlap analysis.

[0031] This endoscope supports at least two display modes. In standard mode, the main monitor displays a highly detailed image from the frontal video camera, making it convenient for navigation. In panoramic mode, the same or an additional monitor displays a 360-degree panorama from the side video cameras, simulating a view "from inside the tube." A combined mode is also possible, displaying both the frontal view and the panoramic sweep simultaneously.

[0032] Additionally, to automate incoming information, there's an automatic image analysis module powered by artificial intelligence, utilizing trained neural networks (convolutional neural networks, U-Net architectures, or similar). The module analyzes video streams from all cameras in real time, identifying potentially pathological areas, such as polyps, areas of inflammation, bleeding, and neoplasia. Detected areas can be outlined with a color contour on the panoramic or frontal image, accompanied by a text label and / or an audible signal to attract the endoscopist's attention.

[0033] Below is an example of the implementation of the stated technical solution.

[0034] The endoscope has a flexible body with a diameter of, say, 14 mm. The distal section houses, for example, five miniature digital CMOS cameras with a resolution of, say, 1920x1080 pixels each. One video camera is located centrally (frontal), and four side video cameras are arranged around the central one in 90° increments. The lenses of the side video cameras are oriented at an angle of, say, 85° to the axis; their field of view is 100° diagonally, ensuring an overlap of at least 20° with the fields of view of adjacent video cameras. Next to the lens of each video camera is a lighting module with two LEDs: one with a broad spectrum (white light), the second with narrowband radiation at wavelengths of, say, 415 nm (blue) and 540 nm (green) for NBI. The lighting modes for each video camera are independently switched using a button on the endoscope handle.

[0035] Video streams are transmitted via cables to an external processing unit containing a multi-channel video input and a high-performance GPU, such as the NVIDIA RTX series, which receives five video streams at 30 fps in real time (with latency <50 ms). The software is implemented using computer vision libraries (such as OpenCV) and deep learning frameworks (such as PyTorch). Using feature-based algorithms (SIFT, ORB) and known calibration parameters, the processor generates a cylindrical panorama of the internal surface of the colon. The panorama is displayed on the main 27-inch medical monitor.

[0036] In hybrid mode, a trained neural network, such as a convolutional neural network (CNN), analyzes frames from all five cameras, highlighting areas with signs of neoplasia. When a probable pathology is detected, the system overlays a translucent colored marker (e.g., a green outline) on the corresponding area in both panoramic and standard views, attracting the physician's attention.

[0037] Below is an example of the use of the declared technical solution. in the examination procedure:

[0038] The endoscope is inserted into the colon. Standard mode is activated by default: the monitor displays the image from the frontal video camera in white light mode, allowing the physician to safely advance the instrument.

[0039] Upon reaching the target section of the intestine, the physician presses the switch, activating panoramic mode. The processing unit immediately (with a latency of less than 100 ms) displays a continuously updated 360° cylindrical panorama of the intestinal wall on the second monitor, synthesized from images from four lateral video cameras. The physician sees the entire circumference of the lumen without having to rotate the endoscope.

[0040] Having noticed a potentially suspicious area on the panoramic image, for example, a small dark spot behind a fold, the physician can, without changing the position of the distal part, switch the illumination mode of the side video camera whose field of view covers this area to NBI. This will allow for a detailed examination of the microvascular component of the examined area.

[0041] The AI ​​module simultaneously analyzes all five video streams. When an image is detected with a high probability of being an adenomatous polyp or any other neoplasm based on morphological characteristics, the program places a translucent red frame around the object in both the panoramic and frontal views (if it is within the frontal camera's field of view). The system also displays a text notification, such as "Polyp detected, 92% probability," and saves a still image to the log.

[0042] The doctor, having received a visual cue, makes a decision to perform a biopsy or immediate polypectomy through the instrument channel of the endoscope.

[0043] The claimed invention allows for a significant improvement in the quality of endoscopic examination by providing a complete, one-time panoramic view, expanding diagnostic capabilities through independent multimodal visualization and automation of primary screening of pathologies, which ultimately contributes to early diagnosis and increased treatment effectiveness.

[0044] Thus, from the above, it can be concluded that the stated technical result has been achieved, namely, the diagnostic accuracy and reliability of endoscopic examination have been increased due to the complete elimination of “blind spots” by providing a one-time panoramic view of 360° around the circumference of the walls of the hollow organ being examined, as well as by expanding the functionality of the device through the implementation of independent lighting modes for each observation zone and automation of the process of identifying pathologies.

[0045] The claimed technical solution meets the patentability requirement of “novelty” for inventions, since, when determining the level of technology, no technical solution was identified that has features identical (that is, coinciding in the function they perform and the form of execution of these features) to the set of features listed in the invention formula, including the characteristic of the purpose.

[0046] The claimed technical solution meets the patentability requirement of “inventive step” required for inventions, since no technical solutions have been identified that have features that coincide with the distinctive features of the claimed invention, and the influence of the distinctive features on the specified technical result has not been established.

[0047] The claimed technical solution meets the patentability requirement of “industrial applicability” required for inventions, since it can be manufactured using known materials, components, standard technical devices and equipment.

Claims

1. An endoscope with a panoramic view, comprising a housing, in the distal part of which one main frontal video camera is placed, oriented along the longitudinal axis of the endoscope, and four side video cameras, located along the circumference of the distal part with fixed angular offsets relative to each other, wherein each video camera is equipped with a lighting module with the ability to switch between a wide-spectrum mode - white light and a narrow-band lighting mode at wavelengths of blue - 415 nm and green - 540 nm, in addition, the endoscope additionally contains an image processing unit, configured with the ability to receive video streams from all video cameras and form on their basis a single panoramic image with a viewing angle of 360° in a plane orthogonal to the longitudinal axis of the endoscope.

2. An endoscope with a panoramic view according to claim 1, characterized in that the image processing unit is designed with the ability to operate in at least two modes: standard, in which only the image from the front video camera is displayed on the display device, and panoramic, in which the specified single panoramic image is displayed.

3. An endoscope with a panoramic view according to claim 1, characterized in that it additionally contains a module for automatic image analysis based on artificial intelligence, configured to process video data from all five video cameras to detect pathological changes and output an alarm to a display device.

Images