SYSTEMS, DEVICES, AND METHODS FOR ENDOSCOPY

Abstract

A portable endoscopic system including an imaging unit for endoscopic procedures, the imaging unit having an imaging coupler for receiving imaging information from an imaging assembly of an endoscope, a display integrated within a housing of the imaging unit, an imaging processor for processing the received imaging information into a time series of images and displaying the images in real time, a motion sensor configured to detect movement of the housing, and a detection processing unit configured to classify at least one anatomical feature in each image of the time series based on an artificial intelligence classifier, determine a confidence evaluation index based on the classification, determine a motion vector based on the detected motion, and display the classification of the at least one anatomical feature, the determined confidence evaluation index, and the determined motion vector together with the corresponding image.

Description

[Technical field]

[0001] (Related Applications) This application claims the benefit of U.S. Patent Application No. 17 / 572,332, filed January 10, 2022, the entire contents of which are incorporated herein by reference for all purposes.

[0002] (Technical field) The present disclosure relates to systems, devices, and methods that utilize endoscopic imaging systems. More particularly, the present disclosure relates to systems, devices, and methods that utilize endoscopic systems to enable various endoscopic procedures (medical and non-medical). For example, the various techniques disclosed herein enable the prevention, prediction, diagnosis, amelioration, monitoring, or treatment of pathologies or non-medical procedures in various mammalian pathologies, such as human prostate pathologies, including but not limited to benign prostatic hyperplasia (BPH), or other human or non-human pathologies. [Background technology]

[0003] There are various prostate diseases. One of these diseases is benign prostatic hyperplasia (BPH), which can be found in human males of almost all ages. In many cases, BPH is the primary cause of lower urinary tract symptoms (LUTS), such as nocturia, frequency, urgency, hesitancy, residual urine, urinary leakage, and dribbling. Generally, 90% of men between the ages of 45 and 80 have some form of LUTS or BPH, and the prevalence increases approximately linearly with age. Although BPH is rarely life-threatening, it can lead to a number of clinical conditions, including urinary retention, renal failure, recurrent urinary tract infections, incontinence, hematuria, and bladder stones. Therefore, in some cases, early intervention may be recommended to improve patient outcomes and quality of life.

[0004] Although several drug therapies are available and are effective in treating BPH, the effects are generally short-lived. Surgical treatments for BPH range from minimally invasive techniques such as prostatic urethral lift devices and various cauterization methods to more invasive resections and even fully invasive prostatectomy. These procedures involve cutting or cauterizing tissue near sensitive structures such as the bladder and verumontanum, which are important for male sexual function. Thus, these procedures require careful cystoscopy practice to identify sensitive structures and estimate the treatment area adjacent to them.

[0005] Minimally invasive procedures offer the advantages of less pain, faster recovery, lower cost, and the use of local anesthesia and mild sedation. Transurethral prostate procedures involve tissue examination of the mucosa of the bladder and urethra using a specialized endoscope called a cystoscope. During the examination, the physician will dilate the urethra and bladder with a clear fluid to visualize the mucosal surfaces of the bladder and urethra. In prostate procedures, the area distal to the bladder neck and proximal to the verumontanum is typically identified as the treatment area. Once the treatment area is identified, the physician may perform the procedure on one or more of the lateral prostate lobes, medial prostate lobes, and anterior prostate lobes.

[0006] Ablation or resection of the prostate requires highly specialized equipment such as microwave, laser, steam, or cryotherapy sources that deliver ablation energy to specialized treatment devices. Furthermore, transurethral ablation procedures tend to be expensive and / or complicated due to the specialized equipment and the need for expertise, and therefore are not practical in outpatient / clinic settings or in parts of the world where such equipment is cost-prohibitive. Recently, transurethral water vapor therapy (TWVT) has gained momentum as a treatment modality with good efficacy for large prostate volumes. Prostatic urethral lift (PUL) procedures have also become a backbone technique for BPH treatment over the last decade by preserving ejaculatory function while at the same time requiring minimal anesthesia. Patients who have undergone PUL procedures report generally better sexual function, improved recovery times, and less interference with daily activities over other treatment modalities. The PUL device is a permanently implantable fixation device, similar to a tack or anchor, that is intended to create a channel in one or more of the prostatic lobes between the bladder neck and the verumontanum to reduce obstruction and improve flow.

[0007] Regardless of which treatment modality, each requires precise localization to achieve optimal and lasting results. For example, in a PUL procedure, four to five implants are typically required for an average-sized prostate to achieve an ideal opening, while larger and / or abnormally shaped prostates may require as many as a dozen implants. A first implant is placed approximately 2 cm distal to the bladder neck, a second implant is placed just anterior to the verumontanum, and additional implants are placed between them to typically form a continuous channel through the front and sides of the prostate. Each implant is stored in a disposable cartridge that must be replaced after placement. The effector handle, cartridge, and cystoscope are then removed to replace the cartridge and introduce a new implant. While the sheath remains placed in the prostatic urethra, the physician must continually repeat this process, which can introduce error into optimal placement of the implant. Additionally, many of the PUL devices require the physician to actuate one or more controls multiple times to fully place the implant, making optimal implantation even more difficult to achieve. Finally, identifying the optimal location for the implant based on each patient's unique anatomy requires a learning curve that can be highly subjective.

[0008] The above obstacles and technical challenges are believed to exist not only in PUL procedures, but in virtually all currently known treatment modalities for typical prostate pathologies. Thus, a need exists in the art for a lightweight, portable imaging platform for endoscopic treatments to identify and track anatomical landmarks and treatment sites in vivo. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] U.S. Patent Application No. 17 / 572,332 Summary of the Invention [Problem to be solved by the invention]

[0010] The present disclosure addresses the above obstacles and solves the above-mentioned technical problems by providing various systems, devices, and methods for endoscopy, whether for medical purposes (e.g., preventing, predicting, diagnosing, ameliorating, monitoring, or treating disease conditions in various mammalian pathologies) or non-medical purposes (e.g., aiding in the visual inspection of narrow, hard-to-reach cavities). These and other features, aspects, and advantages of the present embodiments will become more clearly understood upon consideration of the following detailed description, drawings, and appended claims. [Means for solving the problem]

[0011] In one example of the present disclosure, an imaging unit for an endoscopic procedure is provided. The imaging unit comprises a housing and a display integrated within the housing. An imaging coupler is adapted to receive imaging information from an imaging assembly of an endoscope having a field of view (FoV) including at least a portion of an end effector and a portion of a region of interest (ROI). An imaging processor is configured with instructions to process the received imaging information into pixel values ​​representative of a time series of images and display the images in real time on the display, while a motion sensor is configured to detect movement of the housing during the time series. The imaging unit comprises a detection processing unit (DPU) configured with instructions to classify at least one anatomical feature in each image of the time series based on an artificial intelligence classifier, determine a confidence evaluation index for the classification, determine a motion vector based on the detected motion, and display the classification of the at least one anatomical feature, the determined confidence evaluation index, and the determined motion vector together with the corresponding images.

[0012] In another example of the present disclosure, the motion sensor includes at least a gyroscope configured to generate a gyroscope signal and an accelerometer configured to generate an acceleration signal, and the detection processing unit is further configured to determine a displacement vector based on at least the gyroscope signal and the acceleration signal.

[0013] In another example of the present disclosure, the DPU is configured to display the displacement vectors along with a corresponding classification.

[0014] In another example of the present disclosure, the detection processing unit is configured to display the displacement vector relative to one or more classified anatomical features.

[0015] In another example of the present disclosure, the detection processing unit is configured to display each one of the plurality of displacement vectors relative to a classified unique anatomical feature.

[0016] In another example of the present disclosure, the artificial intelligence classifier is a convolutional neural network configured to compare each image to an anatomical model.

[0017] In another example of the present disclosure, the detection processing unit determines a confidence assessment index based on the comparison.

[0018] In another example of the present disclosure, the detection processing unit is configured to identify at least one treatment site based on the classified at least one anatomical feature, and display the identified at least one treatment site and a relative motion vector between the classified anatomical feature and the identified treatment site together with a corresponding image.

[0019] In another example of the present disclosure, the region of interest includes at least the prostatic urethra, the administered therapy includes a prostate procedure, and the detection processing unit is further configured to classify the prostate pathology.

[0020] In another example of the present disclosure, a method for endoscopic imaging is presented. The method includes operatively coupling an imaging coupler of an imaging unit to a viewing port of an endoscope. Imaging information is received from an imaging assembly of the endoscope. The imaging assembly has a FoV that includes at least a portion of the end effector and a portion of the ROI. The received imaging information is processed into pixel values ​​representing a time series of images. The images are displayed in real time on a display integrated into a housing of the imaging unit, and movement of the housing during the time series is detected. At least one anatomical feature in each image of the time series is classified based on an artificial intelligence classifier. A confidence metric of the classification is determined, a motion vector is determined based on the detected motion, and the classification of the at least one anatomical feature, the determined confidence metric, and the determined motion vector are displayed in real time with the corresponding images.

[0021] In another example of the present disclosure, the step of detecting the movement further includes generating a gyroscope signal and an acceleration signal related to the movement of the housing. A displacement vector is determined based on at least the gyroscope signal and the acceleration signal.

[0022] In another example of the present disclosure, the method further includes displaying the displacement vectors together with a corresponding classification.

[0023] In another example of the present disclosure, the method further includes displaying the displacement vector relative to the classified one or more anatomical features.

[0024] In another example of the present disclosure, the method further includes displaying each one of the plurality of displacement vectors relative to the classified unique anatomical feature.

[0025] In another example of the present disclosure, the artificial intelligence classifier is a convolutional neural network configured to compare each image to an anatomical model.

[0026] In another example of the present disclosure, the confidence metric is based on the above comparison.

[0027] In another example of the present disclosure, the method further includes identifying at least one treatment site based on the classified at least one anatomical feature, and displaying the identified at least one treatment site and the determined motion vector together with a corresponding image.

[0028] In another example of the present disclosure, the region of interest includes at least the prostatic urethra, the administered therapy includes a prostate procedure, and the method further includes classifying the prostate pathology.

[0029] In another example of the present disclosure, a kit for an endoscopic therapeutic procedure is presented. The kit includes an endoscopic imaging unit having a housing and a display integrated within the housing. The endoscopic imaging unit includes an imaging coupler configured to receive imaging information from an imaging assembly of an endoscope having a field of view (FoV) that includes at least a portion of the end effector and a portion of a region of interest (ROI). In addition, the endoscopic imaging unit includes an imaging processor, a motion sensor configured to detect movement of the housing during a time sequence, and a detection processing unit (DPU). Furthermore, the kit includes instructions for performing a method for endoscopic imaging. The method includes the steps of operably coupling an imaging coupler of the imaging unit to a viewing port of the endoscope, receiving imaging information from the imaging assembly, processing the received imaging information into pixel values ​​representing a time series of images, displaying the images in real time on a display, detecting movement of the housing during the time series, classifying at least one anatomical feature in each image of the time series based on an artificial intelligence classifier, determining a confidence evaluation index of the classification, determining a motion vector based on the detected movement, and displaying the classification of the at least one anatomical feature, the determined confidence evaluation index, and the determined motion vector together with the corresponding images.

[0030] In another example of the present disclosure, the step of detecting the movement further includes generating a gyroscope signal and an acceleration signal related to the movement of the housing, and determining a displacement vector based on at least the gyroscope signal and the acceleration signal. The artificial intelligence classifier is a convolutional neural network configured to compare each image to an anatomical model.

[0031] In one embodiment, the method may include receiving, by a processor, images from an endoscope imaging the cavity, depicting anatomical features within the cavity; performing, by the processor, a classification on the anatomical features while the endoscope images the cavity; determining, by the processor, a confidence evaluation index for the classification while the endoscope images the cavity; determining, by the processor, a motion vector for the endoscope imaging the cavity while the endoscope images the cavity; and requesting, by the processor, a display to simultaneously present at least two of the images, the classification, the confidence evaluation index, or the motion vector while the endoscope images the cavity.

[0032] In one embodiment, the method may include receiving, by a processor, images from an endoscope imaging the cavity depicting anatomical features within the cavity; performing, by the processor, a classification on the anatomical features while the endoscope images the cavity; determining, by the processor, a confidence evaluation index for the classification while the endoscope images the cavity; determining, by the processor, a motion vector for the endoscope imaging the cavity while the endoscope images the cavity; and performing, by the processor, an action based on the classification, the confidence evaluation index, and the motion vector.

[0033] So that these and other advantages and objectives of the disclosure may be obtained, a more particular description of the disclosure briefly described above will be rendered by reference to specific embodiments which are illustrated in the accompanying drawings, the disclosure will be described and explained with additional specificity and detail through the use of the accompanying drawings, with the understanding that these drawings depict only typical embodiments of the disclosure and therefore should not be considered as limiting the scope of the disclosure. [Brief description of the drawings]

[0034] [Figure 1] FIG. 1 is a structural diagram of an embodiment of a portable system. [Diagram 2] FIG. 2 is a structural diagram of an embodiment of a portable system positioned within an area of ​​interest. [Diagram 3] 1 is a block diagram of one embodiment of a wireless imaging unit of a portable system. [Figure 4] FIG. 1 is a network diagram illustrating an embodiment of a portable system. [Figure 5A] FIG. 1 is an endoscopic view illustrating an example region of interest displayed on a handheld system. [Figure 5B] FIG. 1 is an endoscopic view illustrating an example region of interest displayed on a handheld system. [Figure 5C] FIG. 1 is an endoscopic view illustrating an example region of interest displayed on a handheld system. [Figure 5D] FIG. 1 is an endoscopic view illustrating an example region of interest displayed on a handheld system. [Figure 6A] 1 is a flow chart of an embodiment for training an anatomical model. [Figure 6B] 1 is a flow chart of an embodiment for training an anatomical model. [Figure 7] 1 is a flow chart of a method for performing an endoscopic procedure. [Figure 8] FIG. 1 is a structural diagram of a kit for performing an endoscopic procedure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0035] In the following, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can immediately carry out the present disclosure. However, it should be noted that the present disclosure is not limited to these embodiments, and can be embodied or carried out in various other ways. In the drawings, parts that are not relevant to the description are omitted for the purpose of simplifying the description, and similar reference numerals represent similar parts throughout this specification.

[0036] It should be noted that various terminology used herein may mean direct or indirect, whole or partial, temporary or permanent, active or inactive. For example, when an element is described as being "on" or "connected" or "coupled" to another element, the element may be directly on, connected to, or coupled to the other element, or intervening elements may be present, including indirect or direct variations. In contrast, when an element is described as being "directly connected" or "directly coupled" to another element, there are no intervening elements present.

[0037] Similarly, as used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X uses A or B" is intended to mean any of the natural inclusive permutations. That is, "X uses A or B" is satisfied if X uses A, if X uses B, or if X uses both A and B, in any of these cases.

[0038] Similarly, as used herein, a referent without a quantifier is intended to include the various singular referents and the various plural referents unless the context clearly dictates otherwise. For example, the phrase "one or more" is used herein, but "a referent" or simply "a referent" is also intended to mean "one or more referents." For example, "one or more" includes 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, tens, hundreds, thousands, or more, as well as all intermediate integer or decimal values ​​between these numbers.

[0039] Additionally, as used herein, "comprises," "includes," or variations thereof, specify the presence of a stated feature, whole, step, operation, element, or component, but do not exclude the presence and / or addition of one or more other features, wholes, steps, operations, elements, components, or groups thereof. Additionally, when the present disclosure describes something as being "based on" another, such a statement refers to a reliance that may also be based on one or more other things. In other words, unless expressly stated otherwise, "based on" as used herein inclusively means "based at least in part on" or "based at least in part on."

[0040] Additionally, although the present specification may use terms such as first, second, and other terms to describe various elements, components, regions, layers, or sections, these elements, components, regions, layers, or sections are not necessarily limited by such terms. In other words, these terms are used to distinguish one element, component, region, layer, or section from another element, component, region, layer, or section. Thus, it is contemplated that a first element, component, region, layer, or section discussed below can be referred to as a second element, component, region, layer, or section without departing from the present disclosure.

[0041] In addition, unless otherwise defined, all terms (including scientific and technical terms) used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure belongs. Therefore, terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning of the term in the context of the relevant art, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

[0042] The various features and aspects of the present disclosure may be best understood by reference to the accompanying drawings as considered in the course of the following description.

[0043] Referring to the drawings, Figure 1 shows the main components of a handheld system 10 used during an endoscopic procedure, which may be a diagnostic or therapeutic procedure (or another type of procedure, which may be medical or non-medical). An endoscope 12 is inserted into a patient 14 (e.g., mammal, human, animal, pet, bird, fish, male, female) to a region of interest (ROI) 16, such as a tissue, organ, body part, or any other in vivo feature, although non-medical applications may use non-patient or inanimate objects, such as tubes, cavities, tunnels, cracks, holes, channels, or other relevant non-patient or inanimate ROIs. The region of interest 16 is illuminated by an external light source 18 that directs incident light along an illumination path, such as optical fibers, that extends along the tube of the endoscope 12 to an illumination lens at the distal tip 14. The illuminated region of interest 16 reflects the incident light back to an imaging lens at the distal tip 14, which transmits the reflected light along an imaging path, such as optical fibers, to an observation port 20, such as an eyepiece. The reflected light is received through viewing port 18 by wireless imaging unit (WIU) 22. WIU 22 may include a digital imaging sensor that converts the reflected light into imaging data that can then be processed and displayed on display 24.

[0044] In other embodiments, the endoscope 12 can be a digital endoscope with a tip-mounted chip arrangement. For example, the endoscope 12 can include one or more light emitting diodes (LEDs) located at the distal tip 14 to illuminate the ROI 16. In this arrangement, there is no external light source 18. The distal tip 14 can also include a digital imaging sensor for generating imaging data of the ROI 16. Instead of illumination and imaging paths, a communication path along the tube of the endoscope 12 can send and receive control signals to control the LEDs and the digital imaging sensor. The WIU 22 can receive imaging data from the digital imaging sensor through a viewing port 20. In one embodiment, the viewing port 20 serves as an optical viewing port such as an eyepiece, whereas in another embodiment, it can take the form of a digital interface such as a digital connector for electronically transmitting imaging data. The viewing port 20 interfaces with the WIU 22 through an imaging coupler 26. In the illustrated embodiment, the imaging coupler 26 optically couples the WIU 22 to the viewing port 20 of the endoscope 12. In the tip-mounted tip embodiment described above, the imaging coupler 26 digitally couples the WIU 22 to the view port 20 through an electrical connector having various data and / or electrical channels for controlling LEDs and / or digital imaging sensors at the distal tip 14. The WIU 22 includes a housing 28 configured to integrate the view port 22, display 24, imaging coupler 26, and light source 18 into a single device while protecting various internal components such as, but not limited to, electronic circuitry, power supplies, thermal management, and the like.

[0045] The portable system 10 includes a treatment device 30 configured to be placed in the ROI 16 in vivo and to administer a treatment (or another action or technique) therein in cooperation with the endoscope 12. For example, the treatment may include prevention, prediction, diagnosis, amelioration, monitoring, or treatment of a condition due to or during placement of the endoscope 12 in the ROI 16 in vivo. Thus, in these situations, the device 30 may be labeled / configured accordingly (e.g., as a diagnostic device 30, a predictive device 30, a preventive device 30, and the like). In non-medical situations, the device 30 may also be configured accordingly. The treatment device 30 includes an end effector 32 that delivers the treatment (or another action or technique), including an actuator 34 for initiating the treatment (or another action or technique). In the illustrated embodiment, the ROI 16 includes at least the prostatic urethra 40, the prostate 42, and the bladder 44, although this inclusion is exemplary and other body parts, organs, or tissues may be used (or inanimate ROI 16 may be used for medical applications). In this embodiment, the treatment device 30 is configured to administer a therapy to treat a condition related to a prostate pathology, such as, but not limited to, benign prostatic hyperplasia (BPH) and the like, although non-prostate pathologies may also be used. The treatment device 30 may be configured to administer one or more of the following treatment procedures: resection, incision, cauterization, thermotherapy, enucleation, implantation, cryotherapy, steam therapy, embolization therapy, and the like. It should be appreciated that while the illustrated embodiment shows the treatment device 30 having a handle 36 and an actuator 34, the treatment device 30 may embody a variety of shapes, sizes, and designs as dictated by the treatment to be delivered. For example, although the treatment device 30 is embodied as a pistol shape with the handle 36, this form factor is not required and other form factors can be used. For example, the handle 36 can be omitted and the actuator 34 can be embodied other than as a lever that pivots toward or away from the handle 36 (e.g., as a depressible / depressible button, a rotating knob, a rotating sleeve).

[0046] The WIU 22 is capable of wireless (e.g., radio frequency, line of sight) communications 46, such as simultaneous or substantially simultaneous direct (or indirect) high speed bidirectional data communications to one or more external devices 48. The external devices 48 are capable of directly (or indirectly) receiving data, such as digital images, digital video, or other information related to a therapeutic procedure. The external devices 48 may also directly (or indirectly) transmit control data or control signals to the WIU 22 for remotely controlling the WIU 22. The external devices 48 may also transmit therapeutic (or other operational or procedure) information related to a therapeutic (or other operational or technique) procedure, such as patient data, in the form of an electronic medical record (EMR) or procedure data, such as instructions for performing the procedure. Examples of external devices 48 include personal computing devices, such as desktop computers, smart devices, smart phones, personal digital assistants, tablet computers, portable devices, such as wrist-worn displays, smart watches, or the like, laptop or portable computers, head-mounted displays, or other computing devices not yet contemplated.

[0047] 2, a close-up of the ROI 16 is illustrated. In an exemplary embodiment, the portable system 10 is configured for the treatment of BPH, but may also be capable of preventing, predicting, diagnosing, ameliorating, monitoring, or treating BPH or non-BPH. Generally, BPH treatment involves reducing the effect that an enlarged prostate 42 has on the prostatic urethra 40. In this exemplary embodiment, the treatment device 30 is configured to place prostatic urethral lift (PUL) implants at various treatment sites along the prostatic urethra 40 to lift or pull prostatic tissue away from the urethral channel 50 to improve flow from the bladder 44, for example. The locations of the treatment sites are chosen at the discretion of the practitioner performing the procedure based on subjective criteria, such as the degree of lift achieved as visualized through the endoscope 12. This subjectivity can result in non-optimal placement of the PUL implant, which can result in costly over-implant placement, insufficient implant placement such that the patient does not achieve the desired outcome, improper placement that damages delicate anatomical structures such as the verumontanum or penetrates the bladder neck resulting in unintended consequences such as sexual and / or bladder dysfunction, and the like. Due to the subjective nature of PUL implantation, practitioners must undergo significant training and supervision before they become proficient enough to perform the procedure properly. No matter how much training a practitioner receives, there is still a chance that the procedure will not be performed optimally to produce lasting results.

[0048] During the procedure, the practitioner introduces the distal tip 14 of the endoscope 12 to identify the ROI 16. The practitioner can introduce the end effector 32 of the treatment device 30 simultaneously while identifying the ROI 16 or after the ROI 16 has been identified. To identify the ROI 16, the practitioner observes on the display 24 real-time imaging information received by the imaging assembly 52 from the ROI 16 illuminated with incident light by the light emitter 54. The imaging assembly 46 detects reflected light from the ROI 16 within a field of view (FoV) 56 that includes at least a portion of the ROI 16 and a portion of the end effector 32 of the treatment device 30.

[0049] Once the distal tip 14 is placed within the identified ROI 16, the practitioner identifies a treatment region 58 between the bladder neck 60 and the verumontanum 62 without damaging the delicate anatomical features mentioned above. The bladder neck 60 is a group of muscles that connect the bladder to the urethra and is primarily responsible for keeping urine in the bladder, and if damaged, can lead to incontinence and other problems. The verumontanum 62 is a protuberance in the prostatic urethral bed and is an important distal landmark that helps identify the entrance to the ejaculatory ducts.

[0050] Once identified, the practitioner backs away from the bladder neck 60 by approximately 2 cm distal to the proximal treatment site 64a, 64b to obtain sufficient proximal opening. However, these proximal treatment sites 64a, 64b vary widely between patients, primarily based on the patient's particular prostate anatomy, such as shape, size, density, and the like. If the bladder neck 60 is damaged, such damage may lead to incontinence, bladder leakage, and other problems. If a site is selected that is too proximal to the bladder neck 60, the practitioner may penetrate the bladder neck 60 and cause such dysfunction. If a site is selected that is too distal to the bladder neck, sufficient proximal opening may not be obtained to alleviate symptoms of BPH, such as urinary retention and / or residual urine. However, even if optimal placement is achieved at the proximal treatment site 64a, 64b and the distal treatment site 66a, 66b, the practitioner may also identify intermediate treatment sites 68a, 68b, which will result in a continuous channel through the anterior surface between the proximal treatment site 64a, 64b and the distal treatment site 66a, 66b. The reason for typically choosing to create a channel through the anterior surface of the prostate is that this anterior surface is typically formed of fibromuscular tissue and is typically absent of delicate glandular tissue. To achieve this anterior channel, the practitioner identifies additional areas of persistent obstruction and places additional implants at these intermediate treatment sites 68a, 68b. In some cases, it may be suggested that the entire extent of the prostate be analyzed to identify the size and shape of the patient's prostate, e.g., tall, long, short obstructed lobes, and the like, before performing the treatment procedure. By visualizing the entire extent of the prostate, the practitioner can simulate the desired anterior channel. However, under certain conditions, this iterative approach can result in implantation errors if no reference point is available when the practitioner attempts to revisit these identified optimal sites. For example, if during the initial analysis the practitioner notes that the patient has a long prostate length and that the optimal location is 2 cm distal to the proximal treatment sites 64a, 64b, but after revisiting these sites the practitioner places the implant 2.5 cm too distal from the bladder neck 60, not achieving the optimal proximal opening in the bladder.Thus, to compensate for a suboptimally placed implant, an implant may have to be placed that should not have been placed.

[0051] The portable system 10 seeks to minimize implantation errors by identifying and tracking various anatomical features 40, 42, 44, 60 within the ROI 16 and determining optimal treatment sites 64a, 64b, 66a, 66b, 68a, 68b based on an artificial intelligence model trained to identify anatomical features within the ROI 16 and a tracking system configured to track the motion vectors of the WIU 22 and thus the movement of the distal tip 14 of the endoscope 12 and / or the end effector 32 of the treatment device 30. By increasing efficiency, shortening procedure times, and reducing non-optimal implantation errors by identifying optimal treatment sites based on the patient's unique anatomy, the present treatment procedures can achieve durable results while keeping costs low.

[0052] Referring to FIG. 3, a block diagram of the WIU 22 is illustrated. The electronic circuitry of the WIU 22 previously described includes a system controller 70 configured to control and power the WIU 22. The system controller 70 includes a number of circuit components responsible for controlling aspects of the WIU 22. The system controller 70 includes a microprocessor 72 that interfaces with several electronic components for sending and receiving instructions to control various aspects of the functionality of the WIU 22. The system controller 70 includes a storage device 74, which is a memory device such as a computer-readable medium (e.g., persistent memory, flash memory, embedded memory, ROM, RAM) for storing program instructions to be executed by the microprocessor 72. In addition to the program instructions, the storage device 74 can store anatomical models, procedure data related to performing a therapeutic procedure, electronic medical record (EMR) related data, and the like. The system controller 70 includes a touch screen input controller 76 configured to receive user input from a touch screen 76 overlaid over or included within the display 24. The practitioner can interact with touch screen 76 to control various aspects of WIU 22 via a user interface displayed on display 24 .

[0053] The system controller 70 further includes an illumination controller 76 that receives instructions from the microprocessor 72 for adjusting the intensity or brightness of the incident light from the light emitter 54 and / or one or more frequency components of the incident light generated from the light emitter 54. It should be appreciated that the light emitter 54 may include one or more LEDs for generating the incident light within the ROI 16, or may be optically coupled to a light source 18 for transmitting the incident light to the light emitter 54.

[0054] The imaging processing unit (IPU) 80 may include or be configured to execute instructions stored in the storage device 74 for performing various imaging related functions. For example, the IPU 80 is configured to receive imaging information from the imaging assembly 52 of the FoV 56. The imaging assembly 52 may be an optical assembly that directs reflected light from the ROI 16 to the viewing port 22 (e.g., the eyepiece) of the endoscope 12. In this illustrative embodiment, the WIU 22 includes an imaging sensor 90 that is integrated into the housing 28 and in direct communication with the IPU 80. In another embodiment (tip-mounted tip arrangement), the imaging assembly 52 includes the imaging sensor 90 and is located at the distal tip 14 of the endoscope 12. In this arrangement, the imaging sensor 90 transmits at least one of an analog signal, a digital signal, or a combination of analog and digital signals related to the imaging information to the viewing port 22, which may be further transmitted to the IPU 80.

[0055] The imaging sensor 90 may include one of a complementary metal oxide semiconductor (CMOS), a charge-coupled device (CCD), or other imaging sensor devices not yet contemplated and developed in the future. The imaging information may be either a digital signal to form pixel values ​​representing a time series of images of the FoV 56, or an analog signal that is converted to a digital signal by an analog-to-digital converter (ADC) of the IPU 80.

[0056] The IPU 80 can also be configured to perform several image processing and post-processing functions on the time series of images in real time or substantially real time. Examples of image processing techniques to enhance image quality include edge detection, geometric transformation, perspective correction, color correction, color calibration, motion compensation, data compression, noise reduction, filtering, and the like. The IPU 80 can also be configured to control functions of the image sensor 90, such as adjusting the depth of focus by controlling the built-in autofocus mechanism, pixel clock, sensitivity, offset, signal amplification, gain, gamma, and the like. The IPU 80 can also be configured to adjust the size of the image displayed on the external device 48 due to differences in screen resolution and screen size between the external device 48 and the display 24. The IPU 80 can also be configured to automatically align images such that the images are centered within the display, independent of the size and / or resolution of the display, whether the display used is the display 24 or the display of the external device 48. The IPU 80 receives display information from the microprocessor 72 and formats the output image accordingly. The post-processed images may then be stored in the image memory 102 for later retrieval for viewing locally on the display 24 or on an external device.

[0057] The WIU 22 may also be configured with a wireless transceiver 100 for communicating directly with an external device 48 via a wireless connection 46, or indirectly via the Internet 102 with a remote device 104, an institutional server 106, a cloud storage system 108, and the like. However, it should be noted that the transceiver 100 may be omitted and a receiver and a transmitter may be provided, or either a receiver or a transmitter may be provided. The remote device 104 may be configured for viewing endoscopic images, videos, or examination data, or for remotely receiving control and / or EMR data. EMR data is a collection of patient and population health information stored electronically in a digital format. EMR data may include various patient information such as demographic information, medical history, medications, allergies, vaccination status, laboratory test results, radiology images, vital signs, personal statistics, billing information, and links. The EMR data may be stored in an institutional server 106, such as one located at a hospital, insurance company, government agency, or the like. The EMR data may also be stored in a cloud storage system 88. The cloud storage system 108 may be a data storage system that includes a logical pool of multiple physical storage media across multiple servers in a distributed manner, often across multiple discrete locations, to ensure redundancy, fault tolerance, and data durability. The institutional server 106 and cloud storage system 108 may include a Picture Archiving and Communication System (PACS) capable of providing storage and access of medical images from multiple imaging modalities using a universal image format, such as the Digital Imaging and Communications in Medicine (DICOM) format.

[0058] It should be noted that in some circumstances, the health institution server 106 and cloud storage system 88 comply with data protection and privacy regulations such as the Health Insurance Portability and Accountability Act (HIPAA) in the United States, the General Data Protection Regulation (GDPR) in the European Union, the Personal Information Protection and Electronic Documents Act (PIPEDA) in Canada, the National Health Portal compliance set by the Insurance Regulatory and Development Authority of India (IRDAI), or other compliance regulations enacted around the world.

[0059] Referring to FIG. 4, a network diagram of an embodiment of the portable system 10 is depicted. The WIU 22 is wirelessly coupled to a local network 110 via a wireless access point 112 using wireless transmission protocols such as 802.11 modulation technique family, IEEE 802.15.4a Ultra Wideband (UWB), and the like (Bluetooth), although suitable wired or waveguide connections / hardware are possible. The local network 110 may include cables, switches, and routers that may utilize Ethernet standards for communication. At least one institution server 106 may be in communication with the local network 110. For example, the institution server 106 may store or have access to EMR data that may be accessed by the WIU 22. Additionally, the local network 104 may be coupled to a picture archiving and communication system (PACS) 114 that may be in communication with the institution server 106 and the WIU 22. At least one external device 48 is in communication with the local network, either by a direct physical connection or wirelessly via a wireless access point 112. In addition, a firewall 116 or other network security technology may be connected to the local network 110 to control access to the Internet 102. For example, a remote device 104 may be authorized to access the local network 110 over the Internet 102 using a secure connection facilitated by the firewall 116. In addition, a cloud storage system 108 may be configured to store and retrieve data, and the cloud storage system 108 may be accessed over the Internet 102, facilitated by the firewall 116.

[0060] 3, the WIU includes a motion processing unit (MPU) 120 that may include or be configured to execute instructions stored in the storage device 74 for receiving motion signals from a motion sensor 122. The motion sensor 122 includes at least one of a gyroscope sensor configured to generate a gyroscope signal and an accelerometer configured to generate an acceleration signal. The motion signals (e.g., the gyroscope signal and the acceleration signal) detect motion of the housing 28 that can be used to estimate motion of the distal tip 14 and / or the end effector 32 during a treatment (or other operation or technique) procedure.

[0061] The WIU 22 includes a detection processing unit (DPU) 130 that can include or be configured to execute instructions stored in a storage device 74 for performing various detection-related functions. For example, these instructions can enable the DPU 130 to compare the images of the time series to an artificial intelligence classifier (AIC) based on an anatomical model of the ROI 16 to classify at least one anatomical feature in each image of the time series. For example, the AIC can be based on an artificial neural network (ANN), which can include a convolutional neural network (CNN), a recurrent neural network (RNN), or other suitable ANN. For example, the storage device 74 can store the AIC locally, thereby enabling edge computing, which can be technically beneficial in various environments that may include poor or no network connectivity. In an illustrative embodiment, the ROI 16 includes anatomical features such as the prostatic urethra 40, the prostate 42, the bladder 44, the verumontanum 52, and the bladder neck 60. However, other urinary tract anatomical features are contemplated, such as, but not limited to, the penile urethra, membranous urethra / external urethral sphincter, bulbar urethra, middle lobe, lateral lobe, ureteral orifice, ureterovesical junction, ureter, ureteropelvic junction, renal pelvis, right / left ureteral orifice, infundibulum, calyx, and the like. Anatomical features may also include pathologies, such as, but not limited to, hypertrophy, trabeculation, tumors / lesions, stones, diverticula, and the like. Similarly, as disclosed herein, anatomical features may be anatomical or non-anatomical, whether medical or non-medical features.

[0062] The DPU 130 receives each image of the time series from the IPU 80, which may be in real time or substantially in real time, compares each received image with an anatomical model, which may be in real time or substantially in real time, and determines a confidence assessment index based on the comparison, which may be in real time or substantially in real time. In this illustrative embodiment, the DPU includes instructions for an AIC-based neural network, such as a deep convolutional neural network (DLCNN), but it should be appreciated that other classifiers are also contemplated, such as, but not limited to, perception, naive Bayes classifiers, decision trees, logistic regression, K-nearest neighbors, support vector machines, CNNs, RNNs, and the like. AIC first trains an anatomical model based on individual frames of the time series captured so far from similar and / or adjacent ROIs. The training can be performed on the WIU 22 itself, but the anatomical model can also be trained on an external device 48, a remote device 104, an institutional service 106, or a cloud storage system 108. The trained anatomical model can then be transferred to the working memory or storage device 74 of the DPU 130 via the wireless transceiver 100, thereby enabling edge computing. The DPU 130 compares each image of the time series to the trained anatomical model in real time or substantially real time, and determines a classification and confidence metric for each image based on the comparison in real time or substantially real time. The DPU 130 is configured to instruct the IPU 80 to display the classified anatomical features on the display 24 along with the corresponding image.

[0063] Furthermore, the DPU 130 can receive the motion signals from the MPU 120 in real time or substantially real time, determine a motion vector of the housing 28 in real time or substantially real time, and then use the motion vector to estimate a motion vector of the distal tip 14 of the endoscope 12 and / or the end effector 32 of the treatment device 30 in real time or substantially real time. The motion vector can be a displacement vector, an acceleration vector, a velocity vector, a rotation vector, or the like. For example, the DPU 130 can estimate the displacement and orientation of a portion of the portable system 10 located within the ROI 16 in real time or substantially real time based on the detected movement of the housing 28. The DPU 130 is configured to instruct the IPU 80 to display the determined motion vectors and / or the classified anatomical features along with the corresponding image on the display 24 in real time.

[0064] 2 and 3, the DPU 130 may be configured to identify a treatment region 58 in real time or substantially real time within the ROI 16. In this illustrative embodiment, the treatment region 58 may be a region proximal to the verumontanum 62 and distal to the bladder neck 60, thus avoiding these anatomical features. Based on the identified treatment region 58 within the ROI 16 and the classified anatomical features 40, 42, 44, 60, 62, the DPU 130 may also be configured to determine one or more treatment sites 64a, 64b, 66a, 66b, 68a, 68b in real time or substantially real time. The DPU 130 is configured to instruct the IPU 80 to display the determined motion vectors, the classified anatomical features 40, 42, 44, 60, 62, and / or the determined treatment sites 64a, 64b, 66a, 66b, 68a, 68b along with corresponding images on the display 24 in real time or substantially real time.

[0065] 5A-5D, an endoscopic view of the ROI 16 of the above exemplary embodiment of the treatment procedure is depicted as displayed on the display 24. The practitioner will view the entire ROI 16 to classify the anatomical features 40, 42, 44, 52, 54 for the designated treatment procedure. The practitioner can interact (e.g., by touch) with the user interface displayed on the display 24 via the touch screen 78 to select the desired treatment procedure, although a default procedure can be selected or no default procedure is selected. The DPU 130 then identifies the desired anatomical model based on the user input and retrieves the anatomical model from any one of the storage device 74, the external device 102, the remote device 104, the institution server 106, or the cloud storage system 108. The practitioner initiates the procedure by a touch command through the user interface or an external command by an assistant. In response, the microcontroller 72 commands the IPU 80 to begin collecting a time series of images and commands the MPU 120 to begin collecting motion signals of the housing 28. The microprocessor 72 can also retrieve procedure data and / or EMR data from one of the storage device 74, the external device 48, the external device 102, the remote device 104, the institutional server 106, or the cloud storage system 108, and display the procedure data on the display 24 for the practitioner to review before beginning the procedure. The practitioner begins the procedure by introducing the portable treatment system into the ROI 16 to image the entire ROI 16 as defined by the procedure data. After the anatomical features 40, 42, 44, 52, 54 are classified in real time or substantially real time, the practitioner can choose whether to perform the procedure manually based on the displayed classified anatomical features 140, the motion vectors 142, and the confidence assessment index 144, or can choose to perform the procedure in a semi-automatic manner based on the determined treatment area 58 and the determined treatment sites 64a, 64b, 66a, 66b, 68a, 68b, which will be described in more detail below.

[0066] During a manual procedure, the practitioner may rely on a relative motion vector 146, such as from the bladder neck 60, to perform a treatment on the proximal treatment sites 56a, 56b. The practitioner would introduce the distal tip 14 and end effector 32 into the ROI 16 until the bladder 54 and bladder neck 60 are displayed in real-time or substantially real-time as classified anatomical features 140, such as when the text indicators of the corresponding anatomical features and the displayed confidence index 144 meet the practitioner's expectations, as illustrated in FIG. 5A. The practitioner may then interact with the touch screen 78 of the display 24 to initiate the relative motion vector 146 therefrom and retract the distal tip 14 and / or end effector 32 until the relative motion vector 146 indicates sufficient displacement and / or rotation to locate the optimal location for the proximal treatment sites 64a, 64b, as illustrated in FIG. 5B. The practitioner would then activate the actuator 34 to deploy the treatment at this location.

[0067] Similarly, the practitioner may repeat this process to deploy treatments at the verumontanum 62 and distal treatment sites 66a, 66b. After deploying treatment at the proximal treatment sites 64a, 64b, the practitioner retracts the distal tip 14 and / or end effector 32 until the verumontanum 62 appears in real time or substantially real time as a classified anatomical feature 140. From there, the practitioner would extend the distal tip 14 and / or end effector 32 until the relative motion vector 146 indicates sufficient translation and / or rotation in real time or substantially real time to seek out the optimal location for the distal treatment sites 66a, 66b, as illustrated in FIG.

[0068] Finally, the practitioner would repeat the above process as necessary to apply treatment to the intermediate treatment sites 68a, 66b. After deploying treatment at the distal treatment sites 66a, 66b, the practitioner would extend the distal tip 14 and / or end effector 32 along the treatment region 58 to identify areas of excessive obstruction and identify one or more intermediate treatment sites 68a, 66b. The practitioner could rely on the relative motion vector 144 to ensure that subsequent intermediate treatment sites 68a, 66b are placed sufficiently far apart to obtain an optimal, uninterrupted channel through the anterior surface of the prostatic urethra 40 without creating unwanted bulges adjacent to the previously treated treatment sites 64a, 64b, 66a, 66b, 68a, 68b.

[0069] During a semi-automatic procedure, the practitioner can rely on the automatically classified treatment region 58 and the treatment sites 64a, 64b, 66a, 66b, 68a, 68b determined in real time or substantially real time by the DPU 130. After the entire ROI 16 is imaged and the anatomical features 40, 42, 44, 60, 62 are classified in real time or substantially real time by the DPU 130, the DPU 130 then determines in real time or substantially real time the optimal location for the treatment site 64a, 64b, 66a, 66b, 68a, 68b. The practitioner reintroduces the distal tip 14 and the end effector into the ROI 16 until the indicated treatment site 146 is reached in real time or substantially real time with a confidence index 144 that the practitioner considers sufficient. Once the optimal location is reached, the practitioner activates the actuator 34 to deploy the treatment at this location. Similar to the manual procedure, the practitioner deploys the treatment first at the proximal treatment sites 64a, 64b, followed by the distal treatment sites 66a, 66b, and then the intermediate treatment sites 68a, 68b until an unbroken channel is achieved through the anterior surface of the prostatic urethra 40.

[0070] Referring to FIG. 6A, a flow chart of an embodiment 200 of a method for training an anatomical model by the DPU 130 is depicted. Images captured in real-time or substantially real-time during a treatment (or other operation or technique) procedure and classified in real-time or substantially real-time are stored in the image memory 82 as training data together with metadata (or other form of data organization) of the corresponding classification and confidence metric (S10). The training data can be used to retrain (or augment or update) the corresponding anatomical model. The training data is binned by the DPU 130 based on a pre-determined confidence metric threshold (S12). Classified images with confidence metrics exceeding (or meeting) the pre-determined confidence metric threshold are considered high-confidence images and can be used to retrain (or augment or update) the corresponding anatomical model without further intervention. However, classified images that do not (or do not) meet the confidence metric threshold are considered low confidence images and are binned for further manual review by a user (e.g., via a physical or virtual keyboard) using the display 24 or external device 48, remote device 104, institutional service 106, or cloud storage system 108. These devices can retrieve the binned low confidence images from the image memory 82 for manual review via the network 104 or the Internet 102. The DPU 130 is configured to retrieve (S14) the classified images that are binned as high confidence images and retrain (or augment or update) (S16) the anatomical model. The retrained (or augmented or updated) anatomical model is then stored (S18) in the storage device 74 for future treatment (or other operational or technical) procedures. It should be appreciated that the retrained (or augmented or updated) anatomical model may also be stored in one or more of the external device 102, the remote device 104, the institutional server 106, or the cloud storage system 108 for future therapeutic (or other operational or technical) procedures.

[0071] Referring to FIG. 3, in another embodiment, the portable system 10 includes a training processing unit (TPU) 150 located in at least one of the external device 102, the remote device 84, the agency service 86, or the cloud storage system 88, which performs retraining (or augmenting or updating) of the anatomical model. Referring to FIG. 6B, another embodiment 202 of a method for training an anatomical model by the TPU 150 is depicted. Training data is retrieved by the TPU 150 of any one of the external device 102, the remote device 84, the agency service 86, or the cloud storage system 88 (S20). Similarly, the TPU 150 bins images of the training data into high-confidence bins and low-confidence bins based on a pre-determined confidence evaluation index threshold (S22). The TPU 150 is configured to retrieve classified images binned as high-confidence images (S24) and retrain (or augment or update) the anatomical model (S26). The retrained (or augmented or updated) anatomical model is then stored (S28) in one of the storage devices, the external device 102, the remote device 104, the institution server 106, or the cloud storage system 108, for future therapeutic (or other operational or technique) procedures. It should be appreciated that the WIU 22 can retrieve the retrained (or augmented or updated) anatomical model stored in one of the external device 102, the remote device 104, the institution server 106, or the cloud storage system 108 and store it in the storage device 64 for future therapeutic (or other operational or technique) procedures.

[0072] 7, a flow chart of a method 300 for performing an endoscopic therapeutic (or other operation or technique) procedure is depicted. A practitioner (e.g., a user, a physician, a technician) operably couples the imaging coupler 26 of the WIU 22 to the viewing port 20 of the endoscope 12 (S30). Through the user interface of the display 24, the practitioner selects a desired therapeutic (or other operation or technique) procedure (S32). Based on the practitioner's input, the DPU 130 of the WIU 22 determines and retrieves an anatomical model corresponding to the desired therapeutic (or other operation or technique) procedure from any one of the storage device 74, the external device 102, the remote device 104, the institutional server 106, or the cloud storage system 108 (S34). The practitioner starts the procedure by a touch command through the user interface, and in response, the microcontroller 72 instructs the IPU 80 to start collecting imaging information and the MPU 120 to start collecting motion signals of the housing 28 (S36). The WIU 22 receives imaging information of the FoV 56 including at least a portion of the end effector 32 of the treatment (or other operation or technique) device 30 and a portion of the ROI 16 (S38), which is converted in real time or substantially in real time into a time series of images and displayed in real time on the display 24. At the same time, the DPU 130 detects the movement of the housing 28 in real time or substantially in real time, and estimates the movement vectors of the endoscope 12 and / or the end effector 32 based on the detected movement signals in real time or substantially in real time (S40). Based on a real-time or substantially real-time comparison of the received imaging information with the retrieved anatomical model, an artificial intelligence classifier in the DPU 130 classifies at least one anatomical feature in each image of the time series in real-time or substantially real-time (S42). The DPU 130 instructs the microprocessor 72 to display the classification of the at least one anatomical feature 140, the determined confidence index 144, and the determined motion vectors 142 with the corresponding images in the time series (S44).

[0073] 8, a kit for performing an endoscopic procedure is illustrated. The kit 400 includes at least the WIU 22 according to any one of the embodiments described above, and instructions 402 for performing at least one of the methods 200 for training an anatomical model by the DPU 130, the method 202 for training an anatomical model by the TPU 150, or the method 300 for performing an endoscopic procedure, which in some circumstances may be an endoscopic therapeutic procedure according to or adapted to any one of the embodiments described above. For example, the kit 400 may include a container (e.g., a box, a plastic bag, a package, a case) that contains the WIU 22 according to any one of the embodiments described above, and instructions for performing an endoscopic procedure according to any one of the embodiments described above, regardless of whether the endoscopic procedure is a medical procedure (e.g., suitable for preventing, predicting, diagnosing, ameliorating, monitoring, or treating a condition in various mammalian pathologies) or a non-medical procedure (e.g., to assist in visual inspection of a narrow and difficult-to-reach cavity). The container may also include at least one of a system controller 70, a light source 18, an observation port 20, an imaging sensor 90, a motion sensor 122, a touch screen 78, an I / O port 156, a display 24, an external device 48, or others, which may or may not be disclosed herein.

[0074] For example, the endoscopic procedures disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat neurological conditions such as epilepsy, headaches / migraines, whether primary or secondary, cluster or tension, neuralgia, stroke, vertigo, dizziness, concussion, aneurysm, paralysis, Parkinson's disease, Alzheimer's disease, or others, as will be understood by those of skill in the art and omitted herein for the sake of brevity only. For example, the endoscopic procedures disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat neurodegenerative diseases such as Alzheimer's disease, Parkinson's disease, multiple sclerosis, postoperative cognitive impairment, and postoperative delirium, or others, as will be understood by those of skill in the art and omitted herein for the sake of brevity only. For example, the endoscopic procedures disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat an inflammatory disease or disorder such as Alzheimer's disease, ankylosing spondylitis, arthritis (osteoarthritis), rheumatoid arthritis (RA), Sjogren's syndrome, temporal arteritis, type 2 diabetes, psoriatic arthritis, asthma, atherosclerosis, Crohn's disease, colitis, dermatitis, diverticulitis, fibromyalgia, hepatitis, irritable bowel syndrome (IBS), systemic lupus erythematosus (SLE), nephritis, fibromyalgia, celiac disease, Parkinson's disease, ulcerative colitis, chronic peptic ulcer, tuberculosis, periodontitis, sinusitis, hepatitis, Graves' disease, psoriasis, pernicious anemia (PA), peripheral neuropathy, lupus, or others that will be understood by those of skill in the art and are omitted herein solely for the purposes of brevity. For example, the endoscopic techniques disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat gastrointestinal conditions such as ileus, irritable bowel syndrome, Crohn's disease, ulcerative colitis, diverticulitis, gastroesophageal reflux disease, or others that will be understood by those of skill in the art and are omitted herein for the sake of brevity only. For example, the endoscopic techniques disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat bronchial disorders such as asthma, bronchitis, pneumonia, or others that will be understood by those of skill in the art and are omitted herein for the sake of brevity only. For example, the endoscopic techniques disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat coronary artery disease, heart attack, arrhythmia, cardiomyopathy, or others that will be understood by those of skill in the art and are omitted herein for the sake of brevity only.For example, the endoscopic techniques disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat voiding disorders such as urinary incontinence, urinalysis, overactive bladder, or others that will be understood by those of skill in the art and are omitted herein for brevity purposes only. For example, the endoscopic techniques disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat cancers such as bladder cancer, breast cancer, prostate cancer, lung cancer, colon or rectal cancer, skin cancer, thyroid cancer, malignant brain tumors, leukemia, liver cancer, lymphoma, pancreatic cancer, or others that will be understood by those of skill in the art and are omitted herein for brevity purposes only. For example, the endoscopic techniques disclosed herein can be used to prevent, diagnose, monitor, ameliorate, or treat metabolic disorders such as diabetes (type 1, type 2, or gestational), Gaucher disease, sickle cell anemia, cystic fibrosis, hemochromatosis, or others that will be understood by those of skill in the art and are omitted herein for brevity purposes only. For example, non-medical endoscopic procedures can be used for visual inspection tasks where the target area is otherwise inaccessible or where accessibility may require disruptive, time-consuming, and / or expensive activities. For example, non-medical endoscopic procedures can be used for non-destructive testing techniques to recognize defects or imperfections (e.g., aircraft engines, gas turbines, steam turbines, diesel engines, automobile engines, truck engines, machined or cast parts, surface finishes, visual inspection of full through holes, forensic applications in law enforcement, building inspections, bore inspection of firearms in firearms manufacturing). In the above situations, whether medical or non-medical, the ROI 16, model, device 30, implant, and associated hardware / software, as well as techniques of manufacture and use, are adapted accordingly.Some embodiments may include a method including receiving, by a processor, images from an endoscope imaging the cavity depicting anatomical features within the cavity, performing, by the processor, a classification on the anatomical features while the endoscope images the cavity, determining, by the processor, a confidence metric for the classification while the endoscope images the cavity, determining, by the processor, a motion vector for the endoscope imaging the cavity while the endoscope images the cavity, and requesting, by the processor, a display to simultaneously present at least two of the images, the classification, the confidence metric, or the motion vector while the endoscope images the cavity. The display may simultaneously present at least three of the images, the classification, the confidence metric, or the motion vector while the endoscope images the cavity. The display may simultaneously present the images, the classification, the confidence metric, and the motion vector while the endoscope images the cavity. The display can simultaneously present the image and at least two of the classification, the confidence index, or the motion vector while the endoscope images the cavity. The display can simultaneously present the image, the classification, and at least one of the confidence index or the motion vector while the endoscope images the cavity.

[0075] Some embodiments may include a method including receiving, by a processor, images from an endoscope imaging the cavity depicting anatomical features within the cavity; performing, by the processor, a classification on the anatomical features while the endoscope images the cavity; determining, by the processor, a confidence metric for the classification while the endoscope images the cavity; determining, by the processor, a motion vector for the endoscope imaging the cavity while the endoscope images the cavity; and performing, by the processor, an action based on the classification, the confidence metric, and the motion vector. The action may include deploying (e.g., moving, extending, adjusting, powering, grasping, cutting) an end effector within the cavity being imaged by the endoscope. For example, the end effector may be a component of a robotic arm (e.g., during a surgical procedure, exploration of the cavity). The action may be relative to (e.g., contacting) the anatomical feature within the cavity. The action may not be to an anatomical feature within the cavity (e.g., to another anatomical feature, an object inside the cavity, an object outside the cavity). The cavity may be a cavity of a mammal or a cavity of an inanimate object. The action may include requesting the display by the processor to simultaneously present at least two of the image, the classification, the confidence assessment index, or the motion vector while the endoscope images the cavity. The display may simultaneously present at least three of the image, the classification, the confidence assessment index, or the motion vector while the endoscope images the cavity. The display may simultaneously present the image, the classification, the confidence assessment index, and the motion vector while the endoscope images the cavity. The display may simultaneously present the image, the classification, the confidence assessment index, or the motion vector while the endoscope images the cavity. The display may simultaneously present the image, the classification, and at least one of the confidence assessment index or the motion vector while the endoscope images the cavity.

[0076] Various embodiments of the present disclosure may be implemented within a data processing system suitable for storing and / or executing program code and including at least one processor coupled directly or indirectly through a system bus to memory elements including, for example, local memory used during the actual execution of the program code, bulk storage, and cache memory that allows for temporary storage of at least some program code to reduce the number of times the code must be retrieved from bulk storage during execution.

[0077] A system may have I / O devices (such as keyboards, displays, pointing devices, DASDs, tapes, CDs, DVDs, thumb drives, and other memory media) coupled to it either directly or through intervening I / O controllers. A system may also have network adapters coupled to it that enable the data processing system to become coupled to other data processing systems, remote printers, or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few of the available types of network adapters.

[0078] The present disclosure may be embodied in a system, a method, and / or a computer program product. The computer program product may include one (or more) computer readable storage mediums having computer readable program instructions for causing a processor to perform aspects of the present disclosure. The computer readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer readable storage media includes portable computer diskettes, hard disks, random access memories (RAM), read only memories (ROM), erasable programmable read only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disk read only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or ridge structures in grooves with instructions recorded thereon, and any suitable combination. The computer readable program instructions described herein can be downloaded from a computer readable storage medium to each computing / processing device or to an external computer or storage device, for example, via the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives the computer readable program instructions from the network and transfers them for storage to a computer readable storage medium within the respective computing / processing device.

[0079] Computer readable program instructions for carrying out the operations of the present disclosure can be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or conventional procedural programming languages, such as object-oriented programming languages, such as Smalltalk, C++, or the like, and conventional procedural programming languages, such as "C" programming language or similar programming languages. A code segment or machine-executable instructions can represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment can be coupled to another code segment or hardware by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. can be passed, forwarded, or transmitted by any suitable means, including memory sharing, message passing, token passing, network transmission, among others. The computer readable program instructions may execute entirely on the user's computer as a stand-alone software package, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the partially or entirely on a remote computer or server scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN), a wide area network (WAN), or an Internet service provider.In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) can execute computer readable program instructions by utilizing state information of the computer readable program instructions to individualize the electronic circuitry to implement aspects of the present disclosure.

[0080] Aspects of the present disclosure have been described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. Each block of the flowchart illustrations and / or block diagrams, and combinations of blocks depicted in the flowcharts and / or block diagrams, may be implemented by computer readable program instructions. Various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in various manners suitable for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0081] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts and block diagrams may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described in the blocks may be performed in an order other than that described in the figures. For example, two blocks shown in succession may actually be executed substantially simultaneously depending on the flowchart illustration, or may be executed in the reverse order in some cases, and combinations of blocks in the block diagrams and / or flowchart illustrations may be implemented by a dedicated hardware-based system that performs the specified functions or acts or a combination of dedicated hardware and computer instructions.

[0082] Words such as "continue," "next," and the like are not intended to limit the order of the steps, but are merely used to guide the reader through the method description. Although a process flow diagram may describe operations as a sequential process, many of these operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, or the like. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

[0083] Features or functions described with respect to a particular exemplary embodiment can be combined and partially combined with various other exemplary embodiments and / or with these embodiments. Also, various aspects and / or elements of the exemplary embodiments disclosed herein can be combined and partially combined in a similar manner. Furthermore, some exemplary embodiments, whether individually and / or collectively, can be components of a larger system, in which the procedures can be succeeded by other procedures and / or the application of the procedures can be modified otherwise. Additionally, some steps may be required before, after, and / or simultaneously with the exemplary embodiments disclosed herein. It should be noted that at least any and / or all of the methods and / or processes disclosed herein can be at least partially implemented in any manner by at least one entity or actor.

[0084] Although the above has been described in some detail for purposes of clarity, it will be appreciated that certain changes and modifications can be made without departing from the principles of the present disclosure. It should be noted that there are many alternative ways of implementing both the processes and the apparatus described herein. Thus, the present embodiments should be considered as illustrative rather than restrictive, and the subject matter of the research and development described herein is not limited to the details presented herein, but may be modified within the scope of the appended claims and their equivalents. [Explanation of symbols]

[0085] 10 Portable Systems 12 Endoscopy 14 patients 16 Region of Interest (ROI) 18 External light source 20 Observation Port 22 Wireless Imaging Unit (WIU) 24 Display 26 Imaging coupler 28 Housing 30 Therapeutic Devices 32 End effector 34 Actuator 36 Handle 40 Prostatic urethra 42 Prostate 44 Bladder 46 Wireless Communications 48 External Devices

Claims

1. An imaging unit for an endoscopic procedure, comprising: Housing and an imaging coupler for receiving imaging information from an imaging assembly of the endoscope having a field of view (FoV) that includes at least a portion of the end effector and a portion of a region of interest (ROI); a display integrated within the housing; an imaging processor configured with instructions for processing the received imaging information into pixel values ​​representing a time series of images and displaying the images in real time on the display; a motion sensor configured to detect motion of the housing during the time sequence; a detection processing unit configured with instructions; Equipped with The instructions of the detection processing unit include: classifying at least one anatomical feature in each image of the time series based on an artificial intelligence classifier; determining a confidence assessment index for the classification; determining a motion vector based on said detected motion; displaying the classification of the at least one anatomical feature, the determined confidence index, and the determined motion vector together with the corresponding image. This is an instruction for the imaging unit.

2. 2. The imaging unit of claim 1, wherein the motion sensor includes at least a gyroscope configured to generate a gyroscope signal and an accelerometer configured to generate an acceleration signal, and the detection processing unit is further configured to determine a displacement vector based on at least the gyroscope signal and the acceleration signal.

3. The imaging unit of claim 2 , wherein the detection processing unit is configured to display the displacement vector together with the corresponding classification.

4. The imaging unit of claim 2 , wherein the detection processing unit is configured to display the displacement vectors for one or more classified anatomical features.

5. The imaging unit of claim 2 , wherein the detection processing unit is configured to display each one of a plurality of displacement vectors for a classified unique anatomical feature.

6. The imaging unit of claim 1 , wherein the artificial intelligence classifier is a convolutional neural network configured to compare each image to an anatomical model.

7. The imaging unit of claim 6 , wherein the detection processing unit determines the confidence assessment index based on the comparison.

8. The detection processing unit further comprises: identifying at least one treatment site based on the classified at least one anatomical feature; displaying the at least one identified treatment site and a relative motion vector between the classified anatomical feature and the identified treatment site together with the corresponding image. The imaging unit according to claim 1 , configured as follows:

9. The imaging unit of claim 1 , wherein the region of interest comprises at least the prostatic urethra, and the administered treatment comprises a prostate procedure, and the detection and processing unit is further configured to classify a prostate pathology.

10. 1. A method for endoscopic imaging, comprising: operatively coupling an imaging coupler of the imaging unit to a viewing port of the endoscope; receiving imaging information from an imaging assembly of the endoscope, the imaging assembly having a FoV that includes at least a portion of an end effector and a portion of a ROI; processing the received imaging information into pixel values ​​representing a time series of images; displaying the image in real time on a display integrated within a housing of the imaging unit; detecting movement of the housing during the time series; classifying at least one anatomical feature in each image of the time series based on an artificial intelligence classifier; determining a confidence assessment metric for the classification; determining a motion vector based on the detected motion; displaying the classification of the at least one anatomical feature, the determined confidence index, and the determined motion vector together with the corresponding image; The method includes:

11. the step of detecting movement further includes generating gyroscope and acceleration signals related to the movement of the housing; The method further includes determining a displacement vector based on at least the gyroscope signal and the acceleration signal. The method of claim 10.

12. The method of claim 11 further comprising displaying the displacement vector along with the corresponding classification.

13. The method of claim 12 , further comprising displaying the displacement vectors for one or more classified anatomical features.

14. The method of claim 1 , further comprising the step of displaying each one of a plurality of displacement vectors for the classified unique anatomical features.

15. 11. The method of claim 10, wherein the artificial intelligence classifier is a convolutional neural network configured to compare each image to an anatomical model.

16. The method of claim 16 , wherein the confidence metric is based on the comparison.

17. identifying at least one treatment site based on the classified at least one anatomical feature; displaying the identified at least one treatment site and the determined motion vector together with the corresponding image; The method of claim 10 further comprising:

18. 20. The method of claim 17, wherein the region of interest includes at least the prostatic urethra and the administered treatment includes a prostate procedure, the method further comprising the step of classifying a prostate pathology.

19. A kit for an endoscopic treatment procedure, comprising: An endoscope imaging unit is provided, The endoscopic imaging unit comprises: Housing and an imaging coupler for receiving imaging information from an imaging assembly of the endoscope having a field of view (FoV) that includes at least a portion of the end effector and a portion of a region of interest (ROI); a display integrated within the housing; An image processing unit; Including, The endoscopic treatment kit further comprises: a motion sensor configured to detect motion of the housing during a time sequence; A detection processing unit; instructions for carrying out a method of: Equipped with The method further comprising: operably coupling the imaging coupler of the imaging unit to a viewing port of the endoscope; receiving the imaging information from the imaging assembly; processing the received imaging information into pixel values ​​representing a time series of images; displaying said image on said display in real time; detecting movement of the housing during the time series; classifying at least one anatomical feature in each image of the time series based on an artificial intelligence classifier; determining a confidence assessment metric for the classification; determining a motion vector based on the detected motion; displaying the classification of the at least one anatomical feature, the determined confidence index, and the determined motion vector together with the corresponding image; Including, Kit for endoscopic treatment procedures.

20. the step of detecting movement further includes generating gyroscope and acceleration signals related to the movement of the housing; The method further includes determining a displacement vector based on at least the gyroscope signal and the acceleration signal; The artificial intelligence classifier is a convolutional neural network configured to compare each image to an anatomical model.

20. The kit of claim 19.

21. 1. A method comprising: receiving, by a processor, an image representative of anatomical features within the cavity from an endoscope imaging the cavity; performing, by the processor, a classification on the anatomical features while the endoscope images the cavity; determining, by the processor, a confidence metric for the classification while the endoscope images the cavity; determining, by the processor, a motion vector for the endoscope imaging the cavity while the endoscope images the cavity; requesting, by the processor, a display to simultaneously present at least two of the images, the classification, the confidence index, or the motion vectors while the endoscope images the cavity; A method comprising:

22. 22. The method of claim 21, wherein the display simultaneously presents at least three of the images, the classifications, the confidence metrics, or the motion vectors while the endoscope images the cavity.

23. 23. The method of claim 22, wherein the display simultaneously presents the images, the classifications, the confidence metrics, and the motion vectors while the endoscope images the cavity.

24. 22. The method of claim 21, wherein the display simultaneously presents the image and at least two of the classification, the confidence assessment index, or the motion vector while the endoscope images the cavity.

25. 22. The method of claim 21, wherein the display simultaneously presents the images, the classification, and at least one of the confidence assessment index or the motion vectors while the endoscope images the cavity.

26. 1. A method comprising: receiving, by a processor, an image representative of anatomical features within the cavity from an endoscope imaging the cavity; performing, by the processor, a classification on the anatomical features while the endoscope images the cavity; determining, by the processor, a confidence metric for the classification while the endoscope images the cavity; determining, by the processor, a motion vector for the endoscope imaging the cavity while the endoscope images the cavity; performing, by the processor, an action based on the classification, the confidence metric, and the motion vector; A method comprising:

26. 26. The method of claim 25, wherein the action includes deploying an end effector within the cavity being imaged by the endoscope.

27. The method of claim 25 , wherein the action is relative to the anatomical feature within the cavity.

28. The method of claim 25 , wherein the action is not relative to the anatomical feature within the cavity.

29. 26. The method of claim 25, wherein the cavity is a mammalian cavity.

30. 26. The method of claim 25, wherein the cavity is an inanimate cavity.

Images