Instrument anomaly detection for medical devices

Abstract

Systems and methods for automatically detecting instrument anomalies in a sampling device during a medical procedure may include obtaining, via a transducer of a medical device, a sequence of ultrasound images of a target site of the medical procedure. The system or method may also include detecting, in the sequence of ultrasound images, an instrument protruding from the medical device and within the field of view of the transducer. A characteristic of an instrument extending from a medical device is determined for each ultrasound image in a sequence of ultrasound images including the instrument extending from the medical device. The system or method may also include generating an alert indicative of a potential anomaly of the instrument extending from the medical device based on the characteristic of the instrument extending from the medical device exceeding a threshold.

Description

Priority Statement

[0001] This patent application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 652,799, filed May 29, 2024, and U.S. Provisional Patent Application Serial No. 63 / 590,620, filed October 16, 2023, pursuant to section 35 USC119(e), both of which are incorporated herein by reference in their entirety. Technical Field

[0002] The examples described herein generally relate to sampling devices. More specifically, the examples described herein relate to automated instrument anomaly detection for sampling devices. Background Technology

[0003] Conventional endoscopes can be used for a variety of clinical procedures, including, for example: illuminating, imaging, detecting, and diagnosing one or more disease states; delivering fluids toward anatomical regions (e.g., delivering saline or other preparations via a fluid channel); providing access to one or more therapeutic devices (e.g., via a working channel) for sampling or processing anatomical regions; and providing aspiration access for collecting fluids (e.g., saline or other preparations). Such anatomical regions may include the gastrointestinal tract (e.g., esophagus, stomach, duodenum, pancreaticobiliary ducts, intestines, colon, etc.), renal regions (e.g., kidneys, ureters, bladder, urethra), and other internal organs (e.g., reproductive system, sinus cavities, submucosal regions, respiratory tract, etc.).

[0004] Sampling devices (which can be an endoscope or can be delivered via the working channel of an endoscope) can be used to sample target tissues within a patient for medical purposes. Such sampling devices can be deployed within the airways of the lungs to capture samples of target nodules (e.g., solitary pulmonary nodules (SPNs) in a patient's lungs). Samples of the target nodules can be extracted to aid in medical procedure planning, diagnosis, etc. Summary of the Invention

[0005] In some examples, the medical system may include a medical device configured to be inserted into a patient during a medical procedure, the medical device including an elongated member having a distal portion. The medical device may also include an instrument configured to extend through the elongated member and laterally from the distal portion of the elongated member, the instrument being configured to interact with a target site during the medical procedure. The medical device may also include a transducer mounted on the distal portion of the elongated member such that the instrument extending from the elongated member is within the transducer's field of view, the transducer being configured to capture medical images of the field of view throughout the medical procedure. Furthermore, the medical device may include a controller including processing circuitry coupled to a memory including instructions configured, when executed by the processing circuitry, to cause the processing circuitry to: determine the extension angle of the instrument from the elongated member via medical images from the transducer, and generate an alarm if the extension angle exceeds a threshold.

[0006] In some examples, a method for automatically detecting device abnormalities in a medical device during a medical procedure may include: obtaining a sequence of ultrasound images of a target site of the medical procedure via a transducer of the medical device; detecting a device extending from the medical device and within the field of view of the transducer in the sequence of ultrasound images; determining the characteristics of the device extending from the medical device for each ultrasound image or at least some of the individual ultrasound images in the sequence of ultrasound images including the device extending from the medical device; and generating an alarm indicating a potential abnormality of the device extending from the medical device based on the characteristics of the device extending from the medical device exceeding a threshold. Attached Figure Description

[0007] Various examples are shown in the accompanying figures. Such examples are illustrative and are not intended to be exhaustive or exclusive examples of the subject matter.

[0008] Figure 1 A schematic diagram of an example sampling device system is shown.

[0009] Figure 2 A schematic diagram illustrating an example of the imaging and control system of the sampling device system is shown.

[0010] Figure 3 A cross-sectional view of a portion of an example sampling device is shown.

[0011] Figure 4 A block diagram of an example method for detecting potential anomalies in a sampling device is shown.

[0012] Figure 5 A cross-sectional view of a portion of an example sampling device is shown.

[0013] Figure 6 A cross-sectional view of a portion of an example sampling device is shown.

[0014] Figure 7 A cross-sectional view of a portion of an example sampling device is shown.

[0015] Figure 8 A cross-sectional view of a portion of an example sampling device is shown.

[0016] Figure 9 A cross-sectional view of a portion of an example sampling device is shown.

[0017] Figure 10 An example field of view of the sampling device is shown.

[0018] Figure 11 An example graphical user interface is shown for use with the example sampling device.

[0019] Figure 12 A schematic diagram of an example sampling device is shown.

[0020] Figure 13 A cross-sectional view of a portion of an example sampling device is shown.

[0021] Figure 14 A schematic diagram of an exemplary computer-based clinical decision support system (CDSS) is shown.

[0022] Figure 15 A block diagram of an example machine on which one or more examples can be implemented is shown. Detailed Implementation

[0023] Ultrasound-enabled sampling devices can be used to obtain biopsy samples from target nodules within a patient. For example, a lung lesion biopsy can be performed by inserting an endobronchial ultrasound (EBUS) sampling device into a patient's lung via the working channel of an endoscope. When the sampling device is inserted into the patient near the target nodule, the sampling instrument (e.g., a flexible biopsy needle, cutting blade, scissors, scraper, or other end actuator) can extend from the sampling device and enter the target nodule. For example, the sampling instrument can extend from a lateral outlet oblique channel through the airway wall and into the target lung nodule. When extending from a lateral outlet oblique channel, the instrument can puncture the patient's tissue, such as the airway wall, like within the lung. As the instrument extends into the patient's tissue, if the sampling device is inadvertently advanced longitudinally forward (i.e., deeper into the airway) or retracted posteriorly within the airway, the instrument may deform or even be damaged (e.g., warped, twisted, broken, etc.), or damage to the airway may occur. For example, if the instrument is a biopsy needle with a helical laser-cut pattern to impart flexibility, and extends from a side-exit bevel through the airway wall into a tissue region outside the airway wall, longitudinal advancement or retraction of the sampling device can cause the tissue protruding within the needle to shift relative to the exit bevel from which the needle protrudes. Ultimately, this can result in the protruding tissue exerting a reaction force on the biopsy needle. These reaction forces can cause the biopsy needle to bend, buckle, or even break, or potentially cause trauma to the patient's tissue.

[0024] Therefore, the inventors of this disclosure have recognized the need for automated instrument anomaly detection for sampling devices. Example systems, apparatus, or methods may include any of the features or techniques described herein.

[0025] For example, to mitigate this problem, a software-based alarm system can be employed. Software (e.g., an ultrasound image analysis module) can monitor ultrasound images to determine when the instrument appears in the field of view of the ultrasound transducer. Once needle movement appears within the transducer's field of view, the ultrasound image analysis module can track the needle movement. An accelerometer can be mounted in the distal tip of the sampling device and can communicate with the ultrasound image analysis module to help track the movement of the sampling device within the patient's body. The ultrasound image analysis module can combine signals from the accelerometer (or other sensors mounted within the sampling device), ultrasound imaging data (from the transducer or image processing unit), or signals from any other component of the system to identify the movement of the medical device or instrument within the transducer's field of view. The measured instrument bending angle (e.g., the angle between the instrument and the sampling device) can be compared to a predetermined threshold at which instrument abnormalities (e.g., kinking, warping, or breakage) or damage to the patient's tissues may occur. For example, if the sampling device inadvertently advances forward during actuation of the biopsy needle to extend it from the oblique channel of the side outlet, causing the angle of the biopsy needle to increase due to the rearward reaction force generated by the tissue already pierced within it, the observed angle of the biopsy needle across the field of view may increase to above a predetermined threshold level. When the measured instrument bending angle approaches or exceeds a predetermined angle, the ultrasound analysis module can generate an alarm, warning, or control signal, any of which can be transmitted to any component of the endoscope system or the sampling device. In this way, if the sampling device moves longitudinally within the airway as the instrument (e.g., the biopsy needle) extends and punctures tissue, a warning or alarm can be presented to the user in cases where such longitudinal movement may have adverse effects on the patient or the instrument.

[0026] The foregoing discussion is intended to provide an overview of the subject matter of this patent application. It is not intended to provide an exclusive or exhaustive description of the invention. The following description is included to provide additional information regarding this patent application.

[0027] Figure 1 This is a schematic diagram of an endoscope system 100, which may include: an imaging and control system 102; and an endobronchial ultrasound sampling device, the endobronchial ultrasound sampling device including an endoscope 104 and a sampling device 136 that can be attached to the endoscope 104, and the sampling device 136 including a distal end portion 144 extending from the distal end portion of the endoscope 104 via a distal working channel port. Figure 1 The system described herein is an illustrative example of an endoscopic system (e.g., a bronchoscope with linearly arranged ultrasound elements) suitable for use with the systems, apparatus and methods described herein.

[0028] Endoscope 104 may be insertable into an anatomical region for imaging, or (e.g., via tethering) attached to one or more sampling devices for biopsy or for treating a disease condition associated with the anatomical region. Endoscope 104 may be engaged or connected to imaging and control system 102. Endoscope 104 is described in this example as a bronchoscope, but other types of endoscopes for use with the features and teachings of this disclosure are contemplated. Imaging and control system 102 may include control unit 106, display unit 108, input unit 110, light source 112, fluid source 114, and suction pump 116.

[0029] The imaging and control system 102 may include various ports for coupling with the endoscope system 100. For example, the control unit 106 may include a data input port for receiving data from the endoscope 104 and a data output port for transmitting data to the endoscope 104. The light source 112 may include an output port for transmitting light, for example, via an optical fiber link, to the endoscope 104. The fluid source 114 may include a port for transmitting fluid to the endoscope 104. The fluid source 114 may include, for example, a pump and a fluid tank, or may be connected to an external tank, container, or storage unit. The suction pump 116 may include a port for evacuating the endoscope 104 to generate suction, for example, for drawing fluid from an anatomical region in which the endoscope 104 is inserted. The display unit 108 and the input unit 110 may be used by an operator of the endoscope system 100 to control the functions of the endoscope system 100 and view the output of the endoscope 104. The control unit 106 may also generate signals or other outputs for treating the anatomical region in which the endoscope 104 is inserted. In the example, the control unit 106 can generate electrical output, acoustic output, fluid output, etc., for treating anatomical areas by means of methods such as cauterization, cutting, or freezing.

[0030] Endoscope 104 may include an insertion section 118, a functional section 120, and a handle section 122, which may be coupled to a cable section 124 and a coupler section 126. The insertion section 118 may extend distally from the handle section 122, and the cable section 124 may extend proximally from the handle section 122. The insertion section 118 may be elongated and include a curved section and a distal end to which the functional section 120 may be attached. The curved section may be controllable (e.g., controllable via a steering controller 128 on the handle section 122) to manipulate the distal end through tortuous anatomical pathways (e.g., stomach, duodenum, kidney, ureter, trachea, lung, etc.). The insertion section 118 may also include one or more working channels (e.g., internal lumens), which may be elongated and may support the insertion of one or more therapeutic instruments (e.g., a bronchoscope) into the functional section 120. The working channel can extend between the handle section 122 and the functional section 120. Additional functions, such as fluid passages, guide wires, and traction wires, can also be provided by the insertion section 118 (e.g., via suction or flushing passages).

[0031] The coupler section 126 can be connected to the control unit 106 to connect the endoscope 104 to various features of the control unit 106, such as the input unit 110, the light source 112, the fluid source 114, and the suction pump 116.

[0032] Handle section 122 may include a steering controller 128 and a port 130. Steering controller 128 may be a knob, lever, or other actuation mechanism for navigating endoscope 104 within the patient's body. Steering controller 128 may be connected to a traction cable or other actuation mechanism extending through insertion section 118. Port 130, and other ports such as port 132, may be configured to couple various cables, guide wires, auxiliary endoscopes, tissue collection devices, fluid lines, etc., to handle section 122, for example, for coupling with insertion section 118. Figure 1 and Figure 2 The example shown is an example of endoscope 104.

[0033] According to the example, the imaging and control system 102 can be mounted on a mobile platform (e.g., a trolley 134) having features for accommodating a light source 112, a suction pump 116, and an image processing unit 202. Figure 2 Shelves for items such as ) . Alternative locations, Figure 1 and Figure 2 The components of the imaging and control system 102 shown can be directly mounted on the endoscope 104, so that the endoscope is "self-contained".

[0034] The functional segment 120 may include components for treating and diagnosing the patient's anatomy. The functional segment 120 may include an imaging device 146 (e.g., a tip-on-chip image sensor based on complementary metal-oxide-semiconductor (CMOS), an illumination device 148 (e.g., a light-emitting diode), and a working channel port 150 located at the distal end of the functional segment 120.

[0035] like Figure 1 As shown, the sampling device 136 can extend from the working channel port 150 at the distal end face of the functional segment 120 of the endoscope 104. The sampling device 136 can be configured to attach to port 132 such that the sampling device 136 extends through the working channel of the endoscope 104 and extends from the distal end of the endoscope 104. The sampling device 136 may include: a sheath extension mechanism 138 for advancing or retracting the insertion segment 118 within the working channel to control how far distally it extends from the distal end of the sampling device 136; an instrument actuator 142 (e.g., for actuating a biopsy needle from a side exit port of the sampling device 136); and a distal end 144. The sheath extension mechanism 138 can be configured to extend the sampling device 136 beyond the distal end of the endoscope 104, for example, to navigate the sampling device 136 to a target area within the patient's body. The instrument actuator 142 can slide along the housing 140 of the sampling device 136. The housing 140 may include markings indicating the amount of extension of the sampling device 136 beyond the distal end of the endoscope 104. An instrument actuator 142 may be configured to extend an instrument from the sampling device 136 to obtain a tissue sample from the patient. The distal end 144 of the sampling device 136 may include a transducer (or other imaging device) and a proximal end of the transducer for guiding the instrument configured to obtain a tissue sample from the patient to a side exit port within the transducer's field of view. The sampling device 136 will be discussed in more detail herein.

[0036] Figure 2 It includes an imaging and control system 102 and an intrabronchial ultrasound device. Figure 1 A schematic diagram of an endoscope system 100, the endobronchial ultrasound device including an endoscope 104 and a sampling device 136 that can extend through the distal working channel port of the endoscope 104. Figure 2 An imaging and control system 102 component coupled to an endoscope 104 is schematically shown.

[0037] The imaging and control system 102 may include a control unit 106, which may include or be coupled to the image processing unit 202, the treatment generator 206 and the drive unit 208, as well as the light source 112, the input unit 110 and the display unit 108. The control unit 106 may include, or be in communication with, an endoscope, surgical instruments and an endoscopic system, which may include means configured to engage tissue and collect and store a portion of that tissue, and an imaging device (e.g., a camera) may be used to observe target tissue via the endoscope, surgical instruments and the endoscopic system through materials and components including optical enhancements. The control unit 106 may be configured to activate the camera to observe target tissue distal to the endoscopic system. Similarly, the control unit 106 may be configured to activate the light source 112 to illuminate a surgical instrument, which may include selected components configured to reflect light in a particular manner, such as a tissue cutter enhanced with reflective particles.

[0038] The imaging and control system 102 may include a light source 112 to illuminate the anatomical region using light of a desired spectrum (e.g., broadband white light, narrowband imaging using a preferred electromagnetic wavelength, etc.). The imaging and control system 102 may be connected (e.g., via an endoscope connector) to an endoscope 104 for signal transmission (e.g., light output from the light source, video signals from the imaging system in the distal end, diagnostic and sensor signals from the diagnostic device, etc.).

[0039] Fluid source 114 ( Figure 1 The endoscope 104 (shown in the diagram) can communicate with control unit 106 and may include one or more air sources, saline sources, or other fluid sources, as well as associated fluid paths (e.g., air passages, flushing passages, suction passages, etc.) and connectors (barbed fittings, fluid seals, valves, etc.). Imaging and control system 102 may also include drive unit 208, which may include a motorized actuator for advancing the distal segment of endoscope 104.

[0040] Coupler section 126 can be connected to control unit 106 to connect endoscope 104 to various features of control unit 106, such as image processing unit 202, treatment generator 206, etc. In this example, port 130 can be used to insert another instrument or device (e.g., a sub-scope or auxiliary scope, or a sampling needle, biopsy needle, ablation instrument, scalpel, etc.) into endoscope 104. Such instruments and devices can be independently connected to control unit 106 via cable section 124. For example, port 132 can connect coupler section 126 to various inputs and outputs, such as video, air, light, and electricity.

[0041] The image processing unit 202, the ultrasound image processing unit 204, and the light source 112 can each be coupled to the endoscope 104 (e.g., at the functional section 120) or the sampling device 136 via wired or wireless connections. The imaging and control system 102 can accordingly illuminate the anatomical region, collect signals representing the anatomical region, process signals representing the anatomical region, and display images representing the anatomical region on the display unit 108.

[0042] In one or more examples, the ultrasound image processing unit 204 may be configured to receive ultrasound signals from the endoscope 104 or sampling device 136, which may be converted into ultrasound images and transmitted to the display unit 108 or any other component of the endoscope system 100. In examples, the ultrasound image processing unit 204 (and other components of the endoscope system 100, such as the control unit 106, etc.) may use edge detection technology to monitor the extension angle of the instrument from the medical device or the shape or curvature found in the instrument as it extends from the medical device. The ultrasound image processing unit 204 may then compare the extension angle or curvature profile in the instrument with known extension angles or curvature profiles to determine when either the extension angle or profile exceeds a predetermined threshold and / or falls within a predetermined threshold range.

[0043] In the example, machine learning components can be trained to monitor the extension angle and bending profile of the instrument and determine when the extension angle or bending profile exceeds a threshold. Regardless, if the ultrasound image processing unit 204 (or other computing device) uses machine learning or edge detection, it can identify potential anomalies in the instrument and generate one or more of a warning, alarm, notification, or control signal in response to the detected anomaly (e.g., which may be transmitted to any component of the endoscope system 100). In some embodiments, the ultrasound image processing unit 204 can detect anomalies where a biopsy needle or other medical instrument traverses the ultrasound field of view at an angle above a first threshold or below a second threshold. For example, if the sampling device 136 has a side exit port with a nominal trajectory angle of 25 degrees measured relative to the central axis 314, the system can define the normal operating parameters of an instrument extending from the side exit port as plus / minus 5 degrees relative to the nominal trajectory angle of 25 degrees. Therefore, in such an embodiment, an alarm can be generated if the actual needle trajectory observed by the ultrasound image processing unit 204 is greater than 30 degrees or less than 20 degrees. Alarms may include indicating that the needle is above or below an acceptable angle range, and instructions / suggestions on how to adjust the needle back to the acceptable angle range (e.g., slowly advancing or retracting the sampling device as needed).

[0044] Figure 3A cross-sectional view of a portion of an example of a medical device 300 (e.g., a sampling device 136 that extends from the working channel port 150 of an endoscope 104) is shown. The medical device 300 can be configured to be inserted into a lumen 340 (e.g., an inner wall defined by an airway or airway of the lung, or any other lumen defined by an inner wall or tissue (e.g., tissue 330 of the patient). In the example, the medical device 300 can be inserted directly into the lumen 340, or it can extend from a mammoscope capable of guiding the medical device 300 toward a target location within the lumen 340. The medical device 300 may include a coupler 302 and a housing 312. The medical device 300 can be a sampling device, an ablation device, or any other medical instrument inserted into a patient to perform a medical procedure.

[0045] Coupler 302 may extend from proximal portion 304 to distal portion 306. Coupler 302 may include a side outlet oblique channel 308. Side outlet oblique channel 308 may extend from proximal portion 304 and pass through the side of coupler 302 within coupler 302. Side outlet oblique channel 308 may guide instrument 310 (e.g., sampling needle, cutting device, light source, liquid source, etc.) through the side of coupler 302 toward tissue in lumen 340.

[0046] The housing 312 may extend along a central axis 314 from a proximal end segment 316 to a distal end segment 318. The housing may include mounting features 320 configured to receive a transducer 322. The transducer 322 may include elements for transmitting ultrasound signals and receiving ultrasound signals reflected by the patient's tissue to generate ultrasound images (e.g., a piezoelectric micromechanical ultrasound transducer (pMUT) element, a capacitive micromechanical ultrasound transducer (CMUT) element, or a bulk PZT element).

[0047] Transducer 322 may be connected to one or more components of medical device 300, such as control unit 106. Figure 1 ), Treatment Generator 206 ( Figure 2 Display unit 108 Figure 1 Input unit 110 Figure 1 Image processing unit 202 Figure 1 ), Ultrasonic image processing unit 204 ( Figure 2 One or more of the following. Transducer 322 can be configured to generate a signal indicating an ultrasound image. In this example, the ultrasound image can be generated by image processing unit 202 ( Figure 2 ) is generated and transmitted by the control unit 106 to the endoscope system 100 ( Figure 1 Another component of the (e.g., ultrasound image processing unit 202). Figure 3 The field of view 324 of the transducer 322 is also shown.

[0048] The housing 312 may also include a sensor 326. For example, the sensor 326 may be mounted in the distal segment 318 of the housing 312. In another example, the sensor 326 may be mounted at any location between the proximal segment 316 and the distal segment 318 of the housing 312. The sensor 326 may be configured to detect the orientation (e.g., x, y, and z positions) of the medical device 300 during a medical procedure. For example, the sensor 326 may be an accelerometer that generates a signal indicating movement of the medical device 300 as it travels within the patient during a medical procedure. The signal from the sensor 326 may be transmitted to the endoscope system 100 (…). Figure 1 ) or any device or component of medical device 300, such as control unit 106 ( Figure 1 ) or ultrasound image processing unit 204 ( Figure 2 ).

[0049] Ultrasound image processing unit 204 ( Figure 2 It can be connected to the control unit 106 and the image processing unit 202. Figure 2 This includes any combination or element of the sensor 326, endoscope system 100, or medical device 300. The ultrasound image processing unit 202 can receive signals from the sensor 326 to determine the positioning and location of the medical device 300, thereby assisting the ultrasound image processing unit 204 in analyzing ultrasound images and determining one or more parameters of the ultrasound images. For example, the ultrasound image processing unit 204 can determine the location of the medical device 300 within the patient's body, select the optimal path that the medical device 300 can take to obtain a sample from the target nodule, and identify any kinking, bending, warping, airway damage, or any other unintended consequences of the sampling needle extending (intentionally or unintentionally) from the side outlet oblique channel 308 during the use of the medical device 300.

[0050] For example, the ultrasound image processing unit 204 can be configured by instruction to determine the extension angle (e.g., extension angle 604) of the instrument 310 from the side outlet oblique channel 308 or the coupler 302 via medical images from the transducer 322. Figure 6 The ultrasound image processing unit 204 can also be configured to generate an alarm if the extension angle exceeds a threshold.

[0051] Figure 4 A block diagram of an example method 400 is shown. Method 400 can be used to detect one or more abnormalities in a medical device. For example, method 400 may include a detection instrument (e.g., instrument 310). Figure 3Anomalies may occur in other parts of the sampling device, during medical procedures, etc. Although example method 400 depicts a specific sequence of operations, this sequence may be changed without departing from the scope of this disclosure. For example, some of the operations described may be performed in parallel or in a different order that does not substantially affect the functionality of method 400. In other examples, various components of the example apparatus or system implementing method 400 may perform their functions substantially simultaneously or in a specific order. Method 400 may optionally include any of operations 402 to 408.

[0052] At operation 402, according to some examples, method 400 may include obtaining a sequence of ultrasound images of the target site of a medical procedure via a transducer of a medical device. For example, ultrasound image processing unit 204 ( Figure 2 The medical device 300 can receive one or more images from its transducer 322. The medical images may include a field of view (e.g., the field of view 324 of the transducer 322, both within...). Figure 3 One or more target nodules within the 350 (middle) area.

[0053] According to some examples, at operation 404, method 400 may include detecting an instrument extending from a medical device and within the transducer's field of view in an ultrasound image sequence. For example, ultrasound image processing unit 204 ( Figure 2 It can detect from the side exit ramp (e.g., side exit ramp 308). Figure 3 )) extends out and in the transducer (e.g., transducer 322 ( Figure 3 The field of view (e.g., field of view 324) Figure 3 The device within (e.g., device 310) Figure 3 )).

[0054] At operation 406, method 400 may optionally include determining characteristics of the instrument extending from the medical device for each ultrasound image in a sequence of ultrasound images, including the instrument extending from the medical device. For example, characteristics may include shape profile, amount of curvature, angle of extension from the side outlet oblique channel, etc.

[0055] According to some examples, at operation 408, method 400 may include generating an alarm indicating a potential abnormality of the instrument extending from the medical device based on a characteristic of the instrument exceeding a threshold. The alarm may be an auditory, visual, tactile, or a combination thereof alarm used to notify medical professionals of potential abnormalities detected by the system. For example, detected abnormalities may include warping of the instrument (e.g., transducer 322), bending of the instrument exceeding a threshold, or an extension angle of the instrument from the side outlet oblique channel greater than expected. Examples of abnormalities will be discussed in more detail herein.

[0056] Figure 5 A cross-sectional view of a portion of an example including medical device 300 and a schematic diagram of a target nodule 350 shown within the field of view 324 of transducer 322 are illustrated. Since instrument 310 is within medical device 300, ultrasound image processing unit 202 ( Figure 2 The ultrasound image processing unit 204 cannot yet track the device 310. However, it can still output a system readiness signal or any other output to components of the endoscope system 100 or medical device 300. For example, the display unit 108 or control unit 106 can send a system readiness signal to indicate that the system (e.g., the ultrasound image processing unit 204) is turned on and ready to track the device 310 when it is deployed from the medical device 300.

[0057] Figure 6 A cross-sectional view of a portion of an example of a medical device 300 is shown. Figure 6 As shown, if the instrument 310 extends from the side outlet oblique channel 308 and enters the patient's tissue 330, while the medical device 300 translates within the lumen 340 along direction 602, this forward or propulsive translation may cause the instrument 310 to bend at a larger angle relative to the central axis 314. When the instrument 310 bends, the extension angle 604 increases. According to the example, the extension angle 604 may be the angle between the instrument 310 and the coupler 302 when the instrument 310 extends from the side outlet oblique channel 308. When the instrument 310 extends from the side outlet oblique channel 308, the instrument 310 may extend into the field of view 324, and the ultrasound image processing unit 204 ( Figure 2 The extension angle 604 can be determined based on the position of the instrument 310 within the field of view 324. For example, if the instrument 310 is positioned higher (e.g., further away from the coupler 302) or closer to the proximal end than expected within the field of view 324, the extension angle 604 can be larger than expected. Conversely, if the instrument 310 is positioned lower or closer to the distal end within the field of view 324 than expected, the extension angle 604 can be smaller than expected.

[0058] Figure 7 A cross-sectional view of a portion of an example including medical device 300 and a schematic diagram of an instrument 310 in a target nodule 350 shown within a field of view 324 extending to transducer 322 are illustrated. Figure 7 As shown, the instrument 310 can begin to bend as it enters the target nodule 350. Therefore, the ultrasound image analysis module can continuously monitor the angle of the instrument 310 relative to the medical device 300, such as the angle of the instrument 310 relative to the coupler 302. Figure 3 ) or side exit inclined passage 308 ( Figure 3The angle of the device 310 is determined by comparing the determined angle with a set of predetermined thresholds when the angle relative to the medical device 300 changes.

[0059] like Figure 7 As shown, when the instrument 310 extends from the side exit oblique channel 308 through the tissue 330 and enters the target nodule 350, the instrument 310 can form a convex shape 702 or contour. In one example, the convex shape 702 may be formed by impacting a rough or hardened material, which may be harder than the tissue 330, causing the extension force of the instrument 310 from the side exit oblique channel 308 to warp the side exit oblique channel 308 and form the convex shape 702. In another example, when the instrument 310 extends from the side exit oblique channel 308 through the tissue 330 and enters the target nodule 350, the convex shape 702 can be caused by translation of the medical device 300 in direction 602. In this example, the ultrasound image processing unit 204 ( Figure 2 It can be configured to detect any cause of the convex shape 702 and generate an alarm for the cause of the convex shape 702 of the communication device 310.

[0060] Figure 8 A cross-sectional view of a portion of an example including medical device 300 and a schematic diagram of an instrument 310 in a target nodule 350 extending into a field of view 324 of transducer 322 are shown. Figure 8 As seen in the image, when the instrument 310 contacts the target nodule 350, a bend 802 can be formed in the instrument 310. In this example, the bend 802 can be formed in the instrument 310 because the material of the target nodule 350 is harder than that of the tissue 330, causing the extension force of the instrument 310 to form the bend 802. The bend 802 can also be caused by translation of the medical device 300 in direction 602, while the instrument 310 extends from the side outlet oblique channel 308 and passes through the tissue 330 and enters the target nodule 350. Therefore, the bend 802 can be caused by movement of the medical device 300 after the instrument 310 is inserted into the target nodule 292 or by accidental deployment of the instrument 310 during movement of the medical device 300. When the instrument 310 bends, the ultrasound image processing unit 204 can continuously monitor the angle between the instrument 310 and the medical device 300 (e.g., the extension angle 604). Figure 6 )) or any bend in the instrument 310 (e.g., convex shape 702 ( Figure 7 (e.g., bending section 802), and generates warning, alarm, or control signals when the extension angle or profile of the device 310 approaches or exceeds a threshold.

[0061] In this example, the ultrasound image processing unit 204 (and other components of the endoscope system 100, such as the control unit 106, etc.) can use edge detection techniques to monitor the extension angle 604 of the instrument 310 from the side exit ramp 308, or any shape or curvature found within the instrument 310. The ultrasound image processing unit 204 can then compare the extension angle or profile of the instrument 310 to known angles or profiles to determine when either exceeds a threshold. In other examples, machine learning components can be trained to monitor the extension angle and curvature of the instrument 310 and determine when either exceeds a threshold. Regardless, if the ultrasound image processing unit 204 (or other computing device) uses machine learning or edge detection, the extension angle 604 of the instrument 310 from the side exit ramp 308, or any shape or curvature found within the instrument 310, can be used to monitor the extension of the instrument 310 from the medical device 300.

[0062] Figure 9 A cross-sectional view of a portion of an example including a medical device 300 and a schematic diagram of an example of a damaged instrument 310 after removal from a target nodule 350 are shown.

[0063] like Figures 6 to 8 As shown, the instrument 310 may be warped, bent, or formed into a concave (or convex) shape. The ultrasound image processing unit 204 may generate warnings or alarms to notify clinicians of potential warping or other abnormalities of the instrument 310, allowing clinicians to intervene to mitigate the possibility of damage to the instrument 310 or injury to the patient's tissues. In this example, the alarm may display an auditory, tactile, or visual notification of potential warping of the instrument 310, or a combination thereof, within the room where the medical procedure is being performed. Furthermore, the control unit 106 may receive the alarm and record the time and location of its generation for reference after the medical procedure. For example, the device may guide a clinician to move the sampling device a predetermined distance. The predetermined distance can be determined by comparing the measured bending angle of the instrument to a threshold value for the bending angle of the instrument.

[0064] In the example, control unit 106 may generate control signals in response to signals received from ultrasound image processing unit 204 to control one or more systems of endoscope system 100 or medical device 300. For example, control unit 106 may limit the supply to endoscope system 100 ( Figure 1 The control unit 106 can also control the power of any component in the medical device 300, set the feed rate of the medical device 300 within the lumen 340, the threshold for the instrument 310 to extend from the side outlet oblique channel 308 of the medical device 300, etc. The control unit 106 can also control the supply from the site of the medical procedure (e.g., via fluid source 114) Figure 1 )) or removal (e.g., via suction pump 116 ( Figure 1 The flushing fluid.

[0065] Figure 10 Examples of a first medical image 1002 and a second medical image 1006 are shown, each medical image showing a sampling device (e.g., sampling device 136). Figure 1 Example field of view 324. Ultrasonic image processing unit (e.g., ultrasonic image processing unit 204) Figure 2 (The endoscope system 100 or any other controller or processor of the medical device 300, etc.) can be configured to detect movement of anatomical structures (e.g., target nodule 350 shown as target nodule 350A in the first position 1004 and target nodule 350B in the second position 1008). Figure 10 As shown, the target nodule 350 can move between medical images within the field of view 324, and the ultrasound image processing unit 204 can be configured to detect such changes (e.g., movement of the patient's anatomical structures or tissues within the field of view 324) and by showing two locations of the anatomical structure (as shown using...). Figure 10 The movement is indicated by target nodules 350A and 350B shown in the image. The ultrasound image processing unit can be configured to alert the user when movement is detected.

[0066] Endoscopic systems (e.g., endoscopic system 100) Figure 1 This could include settings to enable anatomical structure movement detection. For example, if a clinician reaches a target site but must complete additional procedures before finishing the planned task, the clinician (or another medical professional) could activate the movement detection setting to alert the healthcare provider that one or more anatomical structure indicators have moved within the field of view 324. This alert could also include instructions, markings, or guidance on how the medical team can reposition the medical device to re-align with the anatomical features.

[0067] In the example, a first medical image 1002 and an alarm 1106 can be captured from a real-time video stream, for example, to compare two moments in the real-time video stream. In the example, the ultrasound image processing unit 204 can be configured to determine (e.g., from a static medical image or video stream) the frequency at which medical images are captured. In the example, the ultrasound image processing unit 204 can include more than two medical images and can use averaging or other comparison algorithms to detect movement of anatomical structures or other abnormalities in one or more components of the endoscope system 100.

[0068] Figure 11 An example graphical user interface 1100 is shown. The graphical user interface 1100 can be configured to provide a medical team (e.g., a clinician or other medical professional) with a medical device (e.g., an endoscope system 100). Figure 1 Communication between them. In the example, the graphical user interface 1100 may include real-time image feed 1102 and operation parameters 1104.

[0069] Real-time image feed 1102 may include data from an ultrasound image processing unit (e.g., ultrasound image processing unit 204). Figure 2 Ultrasound images from an image processing unit (e.g., image processing unit 202) Figure 2 Real-time feeding of video images or still images. In the example, real-time image feed 1102 may include a split screen having one or more of ultrasound images (e.g., real-time video or still images), video images, still images, etc.

[0070] Operating parameters 1104 may include information that a clinician can select to control the operation of one or more components of the endoscope system 100, and may include information on communication between the endoscope system 100 and the clinician. For example, common operations of the endoscope system 100 may include buttons within operating parameters 1104 for convenient manipulation or control of the endoscope system 100 during medical procedures.

[0071] like Figure 11 As shown, alarm 1106 may include a message indicating characteristics detected during the medical procedure, such as unexpected movement, the discovery of a target nodule, damage to the sampling needle or other instruments, etc. Alarm 1106 may be located on (or above) the real-time image feed 1102, as shown. Figure 11 As shown, it may be located around or above both operating parameter 1104 or real-time image feed 1102 and operating parameter 1104. For example, alarm 1106 may include auditory, tactile, or other noticeable features to alert a medical professional to the detected characteristic. As discussed herein, alarm 1106 may include one or more instructions or suggestions for remediating the defect, such as instructions for moving the medical device, replacing a component of the medical device, etc.

[0072] Figure 12 Sampling device 1202 is a sampling device 136 (e.g., sampling device 136) according to at least one example of the present disclosure. Figure 1 A side view of an example. Sampling device 1202 can be used with insertion device 1228 (e.g., endoscope 104). Figure 1 )) Figure 12(Only a portion is shown in the image) For example, an endoscope or bronchoscope combined with an operation. As previously described, the insertion device 1228 may include an insertion catheter that can be inserted into the body via an orifice or other opening. In the example, the insertion device 1228 may accommodate an elongated instrument 1204, and the elongated instrument 1204 may extend to a desired location via the insertion catheter. For example, the elongated instrument 1204 may be inserted through the working channel of a bronchoscope and extend from a port on the distal end of the bronchoscope. In this way, the elongated instrument 1204 may extend further into the bronchial structure than the bronchoscope because the elongated instrument 1204 has a smaller outer diameter than the bronchoscope. The elongated instrument 1204 may be a sampling probe, which may include an imaging probe (which may be integrated into the distal tip of the elongated instrument) and a sampling needle within a flexible lumen catheter. The elongated instrument 1204 may be insertable via the insertion device 1228 to obtain tissue samples at a desired location in the body. The elongated instrument 1204 may also include a stylet that can be removably inserted into or through a needle, as further described below.

[0073] The sampling device 1202 described herein can be coupled to the insertion device 1228 (e.g., endoscope 104) using a coupler 1206 at the distal end 1208 of the sampling device 1202. Figure 1 An elongated instrument 1204, operable by sampling device 1202, can extend through coupler 1206 and be inserted into the insertion conduit of insertion device 1228. The elongated instrument 1204 can be fixed to actuator 1212, which is movably coupled to housing 1214. Actuator 1212 can move along housing 1214 between the proximal end 1210 and distal end 1208 of sampling device 1202 (which correspond to the proximal and distal ends of housing 1214) to extend and retract the instrument 310 relative to insertion device 1228. Movement of actuator 1212 along housing 1214 in the distal and proximal directions can respectively cause the elongated instrument 1204 to extend distally or retract into the port at the distal end of insertion device 1228. The anti-warping device can be housed within the housing 1214 to provide lateral support to the elongated instrument 1204 as the actuator 1212 moves the elongated instrument 1204 through the housing 1214.

[0074] The flexible lumen conduit of the elongated instrument 1204 can be secured to the actuator 1212, and the needle can be received within the flexible lumen conduit via the actuator 1212. In some embodiments, the proximal port 1216 can be configured to receive and secure an imaging probe, such as a radial endobronchial ultrasound (EBUS) probe, configured to generate real-time ultrasound images of tissue surrounding the distal end of the elongated instrument 1204. The needle inlet guide 1218 can be configured to receive and engage the needle actuator 1220, to which the sampling needle can be secured. The needle inlet conduit 1218 and the needle actuator 1220 can be movably coupled at a directional interface 1222. The directional interface 1222 can be configured to maintain the orientation of the needle actuator 1220 relative to the needle inlet conduit 1218 to control the orientation of the sampling needle, as further described below. The needle actuator 1220 may removably accommodate an end cap 1224, which may be coupled to a core needle and used to releasably secure the core needle within the sampling needle. Depending on the location of the target tissue within the patient's anatomy, the core needle may be used to prevent the sampling needle from collecting non-target tissue. For example, when the operator is targeting tissue several millimeters or centimeters beyond the airway wall, the operator may advance the sampling needle through the non-target tissue while fully inserting the core needle into the sampling needle. Then, when the operator sees on a real-time image generated by the imaging probe that the needle has reached or is close to reaching the target tissue, the core needle may be withdrawn to allow the target tissue to enter the sampling needle core. The needle actuator may also include a release mechanism 1226 that the operator may actively engage to advance the sampling needle into the sampling position, as further described below.

[0075] Figure 13This is a cross-sectional view of the proximal port 1216 of the actuator 1212 of a sampling device 1202 according to at least one example of the present disclosure. The sampling device 1202 may include an imaging probe 1300. The proximal port 1216 of the actuator 1212 may be configured to receive and guide the imaging probe 1300 into a first lumen 1310 of a flexible lumen conduit 1306. The flexible lumen conduit 1306 may include a proximal end portion 1308 that may be coupled to the actuator 1212. The flexible lumen conduit 1306 may define a second lumen 1312 configured to receive a sampling needle 1304. The second lumen 1312 of the flexible lumen conduit 1306 may extend into the first lumen 1310 and is configured to remotely engage the sampling needle 1304 from the imaging probe 1300. In other examples, the flexible lumen catheter 1306 may define only a single lumen configured to receive the sampling needle 1304, and the distal end of the elongated instrument 1204 may include an imaging element (e.g., a linear ultrasound transducer) integrated into its distal tip in a bevel adjacent to a side outlet port, the bevel being configured to guide the sampling needle 1304 into the field of view of the imaging element. As further described below, the sampling needle 1304 may be coupled to and controlled by a needle actuator 1220.

[0076] A sampling needle 1304 may extend between a base and a tip 1314. The sampling needle 1304 may also include a lumen 1302. The lumen 1302 can be used to extract samples from a patient. A needle actuator 1220 may be slidably mounted on a needle inlet catheter 1218 (which may also be further described below). The sampling needle 1304 may extend from the needle actuator 1220 through the needle inlet catheter 1218 and into a second lumen 1312 of a flexible lumen catheter 1306, through which the sampling needle 1304 may extend into the body to collect samples. The needle inlet catheter 1218 may also be connected to the actuator 1212. Thus, when the needle inlet catheter 1218 and the imaging probe 1300 are secured to the actuator 1212, movement of the actuator 1212 along the housing 1214 can advance the elongated instrument 1204 and the imaging probe 1300 and sampling needle 1304 contained therein.

[0077] Figure 14 A schematic diagram of an exemplary computer-based clinical decision support system (CDSS) 1400 is shown, which is configured to control the endoscope system 100 based on input from any component of an endoscope system (e.g., endoscope system 100, sampling device 136, or medical device 300). Figure 1 ) or medical device 300 ( Figure 3One or more aspects of the endoscope system. In the example, CDSS 1400 may include an input interface 1404 through which patient-specific medical information (e.g., age, weight, gender) or process-specific information (e.g., location of the abnormality, planned path of the process, planned steps of the process, etc.) can be provided as input features to an artificial intelligence (AI) model 1406. A processor 1408 (e.g., image processing unit 202, ultrasound image processing unit 204, etc.) can perform an inference operation in which inputs from any component of the endoscope system, signals transmitted based on engagement with either the first or second engagement member, medical information, process-specific information, etc., are applied to the AI ​​model to generate a suggested medical procedure and a user interface (UI) through which the suggested medical procedure is communicated to a user (e.g., a clinician).

[0078] In some implementations, the input interface 1404 may be a direct data link between the CDSS 1400 and one or more medical devices (e.g., endoscope 104, sampling device 136, or medical device 300) that generate at least some of the input features. For example, the input interface 1404 may directly transmit inputs, medical information, process-specific information, etc., from any component of the endoscope system to the CDSS 1400 during treatment and / or diagnostic medical procedures. Alternatively or additionally, the input interface 1404 may be a classic user interface that facilitates interaction between the user and the CDSS 1400. For example, the input interface 1404 may provide a user interface through which the user manually inputs medical information, process-specific information, etc. Alternatively, the input interface 1404 may provide the CDSS 1400 with access to an electronic patient record from which one or more input features can be extracted. Such an electronic patient record may be stored on a database 1402. In any of these cases, the input interface 1404 may be configured to collect one or more of the following input characteristics associated with a specific patient at or before the time when the CDSS 1400 is used to evaluate the safest and most efficient process to complete the planned medical procedure.

[0079] Based on one or more of the aforementioned input features, processor 1408 uses AI model 1406 to perform inference operations to generate the safest and most efficient medical procedure for performing a medical task. For example, input interface 1404 may deliver any of the following to the input layer of AI model 1406: medical information, medical procedure information, output from any component of the endoscope system, or signals transmitted based on engagement with either the first or second engagement member. AI model 1406 then propagates these input features to the output layer. AI model 1406 can provide the computer system with the ability to perform tasks by inferring patterns discovered in the analysis of data, without being explicitly programmed. AI model 1406 explores the research and construction of algorithms (e.g., machine learning algorithms) that can learn from existing data and predict new data. Such algorithms build AI models from example training data to make data-driven predictions or decisions represented as outputs or evaluations.

[0080] Machine learning (ML) has two common paradigms: supervised ML and unsupervised ML. Supervised ML uses prior knowledge (e.g., examples that associate inputs with outputs or results) to learn the relationship between inputs and outputs. The goal of supervised ML is to learn a function that best approximates the relationship between training inputs and outputs, given some training data. The ML model can then replicate this relationship to generate the corresponding output given an input. Unsupervised ML, on the other hand, trains the ML algorithm using information that is neither classified nor labeled, enabling the algorithm to act on that information without guidance. Unsupervised ML is useful in exploratory analytics because it can automatically identify structures in the data.

[0081] Common tasks for supervised ML are classification and regression problems. Classification problems (also known as categorization problems) aim to classify an item into one of several category values ​​(e.g., is this object an apple or an orange?). Regression algorithms aim to quantify some items (e.g., by providing scores for some input values). Some examples of commonly used supervised ML algorithms are logistic regression (LR), Naive Bayes, random forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and support vector machines (SVM).

[0082] Some common tasks in unsupervised ML include clustering, representation learning, and density estimation. Examples of commonly used unsupervised ML algorithms include K-means clustering, principal component analysis, and autoencoders.

[0083] Another type of machine learning is federated learning (also known as collaborative learning), which trains algorithms on multiple distributed devices that store local data without exchanging data. This approach contrasts with traditional centralized machine learning techniques, where all local datasets are uploaded to a single server, and with more classic distributed methods, which typically assume that local data samples are uniformly distributed. Federated learning enables multiple participants to build general, robust machine learning models without sharing data, thus enabling them to address key issues such as data privacy, data security, data access permissions, and access to heterogeneous data.

[0084] In the example, the AI ​​model can be trained continuously or periodically by the processor 1408 before performing the inference operation. Then, during the inference operation, patient-specific input features provided to the AI ​​model can be propagated from the input layer through one or more hidden layers and ultimately to the output layer corresponding to the suggested medical procedure. For example, if the patient's age, body size, or any other medical information about the patient, as well as medical information indicating the sample such as the patient's location, may be difficult to obtain, the processor 1408 can suggest a smaller version of the endoscope, suggest different paths that can maximize imaging and sampling, or suggest the maximum energy for any cutting, ablation, or removal procedure.

[0085] During and / or after the inference operation, the output interface 1410 can send any of the safest and most effective medical procedures that can be communicated to the user via the user interface (UI) and / or automatically cause any component of the endoscope system to perform the desired action. For example, if the image quality is poor, the processor 1408 can send a signal to the light source control unit to change the brightness, color, saturation, or any other optical parameter of the transmitted light, send a control signal to the fluid source to change the fluid supplied to the pump, send a signal to the pump to change the speed or volume of the fluid supplied to the imaging site, and send a signal to the pump to increase or decrease the suction volume supplied to the imaging site. These are exemplary actions that the CDSS 1400 can take to assist in medical procedures. However, the inventors of this application have considered how the CDSS 1400 can assist in any aspect of a medical procedure, such as preoperative planning, intraoperative execution, or postoperative analysis.

[0086] Figure 15A block diagram of an example machine 1500 on which any or more of the techniques (e.g., methods) discussed herein may be performed. As described herein, the example may include, or may be operated by, logic or components or mechanisms in the machine 1500. A circuit (e.g., a processing circuit) is a collection of circuits implemented in a tangible entity of the machine 1500, including hardware (e.g., simple circuits, gates, logic, etc.). The relationships between circuit components may be flexible over time. A circuit includes components that can perform a specified operation individually or in combination during operation. In the example, the hardware of the circuit may be designed in an immutable manner to perform a specific operation (e.g., hardwired). In the example, the hardware of the circuit may include physical components (e.g., execution units, transistors, simple circuits, etc.) connected in a variable manner to encode instructions for a specific operation, said variablely connected physical components including machine-readable media that are physically modified (e.g., in the form of magnetic, electrical, movable placement of immutable aggregate particles, etc.). When connecting physical components, the underlying electrical properties of the hardware composition change, for example, from insulator to conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to create components of a circuit within the hardware via variable connections to perform specific operations during operation. Thus, in the example, a machine-readable medium element is part of the circuit, or communicatively coupled to other components of the circuit during device operation. In the example, any physical component can be used in more than one component of more than one circuit. For example, during operation, an execution unit can be used in a first circuit of a first circuit at one point in time, and reused at a different time by a second circuit in the first circuit or by a third circuit in the second circuit. Additional examples of these components of machine 1500 are as follows.

[0087] In alternative examples, machine 1500 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, machine 1500 may operate as a server machine, a client machine, or both in a server-client network environment. In the examples, machine 1500 may act as a peer-to-peer (P2P) (or other distributed) network environment. Machine 1500 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web device, network router, switch, or bridge, or any machine capable of (sequentially or otherwise) executing instructions specifying actions to be taken by that machine. Furthermore, although only a single machine is shown, the term "machine" should also be considered as any collection of machines that individually or jointly execute a set (or more) of instructions to perform any or more of the methods discussed herein, such as cloud computing, Software as a Service (SaaS), and other computer cluster configurations.

[0088] Machine 1500 may include a hardware processor 1502 (e.g., a central processing unit (CPU), graphics processing unit (GPU), hardware processor core, or any combination thereof), main memory 1504, static memory (e.g., memory or storage device for firmware, microcode, basic input / output (BIOS),) and mass storage device 1508 (e.g., hard disk drive, tape drive, flash memory, or other block device), some or all of which may communicate with each other via interconnect 1530 (e.g., a bus). Machine 1500 may also include a display unit 1510, an alphanumeric input device 1512 (e.g., a keyboard), and a user interface (UI) navigation device 1514 (e.g., a mouse). In this example, the display unit 1510, the input device 1512, and the UI navigation device 1514 may be a touchscreen display. Machine 1500 may also include a signal generation device 1518 (e.g., a speaker), a network interface device 1520, and one or more sensors 1516, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensors. Machine 1500 may include output controller 1528, for example, serial (e.g., Universal Serial Bus (USB), parallel or other wired or wireless (e.g., infrared (IR), near field communication (NFC) etc.)) connection, to communicate with or control one or more peripheral devices (e.g., printer, card reader, etc.).

[0089] The registers of processor 1502, main memory 1504, static memory 1506, or mass storage device 1508 may be or include machine-readable medium 1522 on which one or more sets of data structures or instructions 1524 (e.g., software) are stored, said set of one or more sets of data structures or instructions 1524 embodying or being utilized by any one or more of the techniques or functions described herein. During execution of instructions 1524 by machine 1500, instructions 1524 may also reside wholly or at least partially in any register of processor 1502, main memory 1504, static memory 1506, or mass storage device 1508. In this example, one or any combination of hardware processor 1502, main memory 1504, static memory 1506, or mass storage device 1508 may constitute machine-readable medium 1522. Although machine-readable medium 1522 is shown as a single medium, the term "machine-readable medium" can include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 1524.

[0090] The term "machine-readable medium" can include any medium capable of storing, encoding, or carrying instructions for use by machine 1500 and to cause machine 1500 to perform any one or more of the techniques disclosed herein, or capable of storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable media can include solid-state memory, optical media, magnetic media, and signals (e.g., radio frequency signals, other photon-based signals, sound signals, etc.). In examples, non-transitory machine-readable media includes machine-readable media having a plurality of particles with invariant (e.g., stationary) mass and thus being a composition of matter. Therefore, a non-transitory machine-readable medium is a machine-readable medium that does not include transiently propagating signals. Specific examples of non-transitory machine-readable media can include: non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0091] In the example, information stored on or otherwise provided on machine-readable medium 1522 may represent instructions 1524, such as instructions 1524 itself or a format from which instructions 1524 can be derived. Such a format from which instructions 1524 can be derived may include source code, encoded instructions (e.g., in compressed or encrypted form), packaged instructions (e.g., split into multiple packages), etc. The information representing instructions 1524 in machine-readable medium 1522 may be processed by processing circuitry into instructions to implement any of the operations discussed herein. For example, deriving instructions 1524 from information (e.g., processed by processing circuitry) may include: (e.g., from source code, object code, etc.) compiling, interpreting, loading, organizing (e.g., dynamic or static linking), encoding, decoding, encrypting, deencrypting, packaging, unpacking, or otherwise manipulating the information into instructions 1524.

[0092] In the example, deriving instruction 1524 may include (e.g., via processing circuitry) assembling, compiling, or decompiling information to create instruction 1524 according to some intermediate or preprocessed format provided by machine-readable medium 1522. Information provided in multiple parts may be combined, unpacked, and modified to create instruction 1524. For example, information may be in multiple compressed source code packages (or object code, or binary executable code, etc.) on one or more remote servers. The source code packages may be encrypted during network transmission and, if necessary, decrypted, decompressed, assembled (e.g., linked), and compiled or decompiled at the local machine (e.g., compiled or decompiled into libraries, standalone executables, etc.), and executed by the local machine.

[0093] Commands 1524 can also be sent or received via the communication network 1526 using a transmission medium via the network interface device 1520, utilizing any of several transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Example communication networks may include local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), LoRa / LoRaWAN or satellite communication networks, mobile phone networks (e.g., cellular networks, such as cellular networks compliant with 3G, 4G LTE / LTE-A, or 5G standards), conventional telephone (POTS) networks, and wireless data networks (e.g., the IEEE 802.11 family of standards known as Wi-Fi®, the IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, etc.). In the example, the network interface device 1520 may include one or more physical jacks (e.g., Ethernet, coaxial, or telephone jacks) or one or more antennas for connection to the communication network 1526. In the example, network interface device 1520 may include multiple antennas for wireless communication using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) technologies. The term "transmission medium" should be considered to include any intangible medium capable of storing, encoding, or carrying instructions executable by machine 1500, and includes digital or analog communication signals or other intangible media to facilitate communication of such software. The transmission medium is a machine-readable medium.

[0094] The following non-limiting examples illustrate certain aspects of this topic to address challenges and provide the benefits discussed herein.

[0095] Example 1 is a medical system including a medical device configured to be inserted into a patient during a medical procedure. The medical device includes: an elongated member including a distal portion; an instrument configured to extend through the elongated member and protrude from the side of the distal portion of the elongated member, the instrument being configured to interact with a target site during the medical procedure; a transducer mounted on the distal portion of the elongated member such that the instrument protruding from the elongated member is within the transducer's field of view, the transducer being configured to capture medical images within the field of view throughout the medical procedure; and a controller including processing circuitry coupled to a memory including instructions that, when executed by the processing circuitry, are configured to cause the processing circuitry to: determine the extension angle of the instrument from the elongated member via medical images from the transducer; and generate an alarm if the extension angle exceeds a threshold.

[0096] In Example 2, the subject matter of Example 1 may optionally include, wherein the elongated member comprises: a lumen extending through the elongated member, the lumen being configured to receive an instrument; and a side outlet oblique channel extending from the lumen and configured to guide the instrument from the side of the elongated member toward a target site of the medical procedure.

[0097] In Example 3, the subject of Example 2 may optionally include, wherein the threshold includes a set variance that is higher or lower than the expected extension angle defined by the side exit ramp.

[0098] In Example 4, the subject matter of any one or more of Examples 1 to 3 may optionally include, wherein the instructions cause the processing circuitry to: determine the curved shape of the instrument via a medical image from the transducer; determine, based on the curved shape including a concave shape, that the medical device is advanced as the instrument extends from the medical device; and generate a reverse translation alarm to indicate that the medical device should be withdrawn to remove the concave shape from the instrument.

[0099] In Example 5, the subject matter of Example 4 may optionally include instructions that cause the processing circuitry to: determine, based on the curved shape including the convex shape, that the medical device retracts when the instrument extends from the medical device; and generate a propulsion translation alarm to indicate that the medical device should be propped forward to remove the convex shape from the instrument.

[0100] In Example 6, the subject of any one or more of Examples 1 to 5 may optionally include, wherein the medical image includes a first medical image and a second medical image.

[0101] In Example 7, the subject matter of Example 6 may optionally include instructions that cause the processing circuitry to: determine a first position of a feature mark in a first medical image; determine a second position of the feature mark in a second medical image; and determine a movement of the feature mark based on the first and second positions of the feature mark.

[0102] In Example 8, the subject matter of Example 7 may optionally include instructions that cause the processing circuitry to generate a reverse translation alarm based on movement of a feature marker in a first direction, indicating that the medical device should be withdrawn from the patient.

[0103] In Example 9, the subject matter of Example 8 may optionally include instructions that cause the processing circuitry to generate a propulsion translation alarm based on the movement of a feature marker in a second direction, indicating that the medical device should be propped into the patient's body.

[0104] In Example 10, the subject matter of any one or more of Examples 1 to 9 may optionally include, wherein the medical image includes a first medical image and a second medical image, and wherein the instructions cause the processing circuitry to: determine a first extension angle of the instrument from the elongated member via the first medical image; determine a second extension angle of the instrument from the elongated member via the second medical image; and generate a warning indicating the change in extension angle based on the change between the first extension angle and the second extension angle exceeding a change threshold.

[0105] In Example 11, the subject matter of any one or more of Examples 1 to 10 may optionally include, wherein the medical device includes a sensor within the distal portion of the elongated member, the sensor being configured to detect longitudinal movement of the medical device.

[0106] In Example 12, the subject matter of Example 11 may optionally include instructions that cause the processing circuitry to: receive a translation signal associated with longitudinal movement of the medical device via a sensor; determine a first position of a feature marker in a first medical image; determine a second position of the feature marker in a second medical image; determine movement of the feature marker based on the first position of the feature marker, the second position of the feature marker, and the translation signal; and generate an unexpected movement alarm based on the disproportion between the movement of the feature marker and the longitudinal movement of the medical device detected by the sensor.

[0107] In Example 13, the subject matter of any one or more of Examples 1 to 12 may optionally include a clinical decision support system configured to assist healthcare professionals during a medical procedure, the clinical decision support system including a trained neural network to analyze information received from the processing circuitry of a controller and to send suggested clinical information based on the information received from the controller.

[0108] In Example 14, the subject matter of Example 13 may optionally include instructions that cause the processing circuitry to: send a sequence of ultrasound images of the target site of a medical procedure to a clinical decision support system; receive from the clinical decision support system an alarm signal indicating an abnormality of the instrument, the alarm signal including instructions for remedying the detected abnormality; and generate an alarm summary based on the alarm signal, the alarm summary including a description of the alarm and instructions for remedying the detected abnormality.

[0109] Example 15 is a method for automatically detecting device abnormalities in a medical device during a medical procedure, the method comprising: obtaining a sequence of ultrasound images of a target site of the medical procedure via a transducer of the medical device; detecting a device extending from the medical device and within the field of view of the transducer in the sequence of ultrasound images; determining characteristics of the device extending from the medical device for each ultrasound image in the sequence of ultrasound images including the device extending from the medical device; and generating an alarm indicating a potential abnormality of the device extending from the medical device based on the characteristics of the device extending from the medical device exceeding a threshold.

[0110] In Example 16, the subject matter of Example 15 may optionally include: determining the bending angle of the instrument by comparing the detected extension angle of the instrument with the expected extension angle; and generating a bending alarm in response to the bending angle exceeding a bending threshold.

[0111] In Example 17, the subject matter of Example 16 may optionally include: receiving a translation signal related to longitudinal movement of the medical device via a sensor mounted in a distal portion of the medical device; determining a first position of a feature marker in a first medical image; determining a second position of the feature marker in a second medical image; determining movement of the feature marker based on the first position of the feature marker, the second position of the feature marker, and the translation signal; and generating an unexpected movement alarm based on the disproportion between the movement of the feature marker and the longitudinal movement of the medical device detected by the sensor.

[0112] In Example 18, the subject matter of any one or more of Examples 16 to 17 may optionally include, wherein determining the characteristics of the device extending from the medical device for each ultrasound image in a sequence of ultrasound images including the device extending from the medical device includes: edge detection technology.

[0113] In Example 19, the subject matter of any one or more of Examples 15 to 18 may optionally include: sending a sequence of ultrasound images of a target site of a medical procedure to a clinical decision support system; receiving from the clinical decision support system an alarm signal indicating an abnormality of an instrument, the alarm signal including instructions for remediating the detected abnormality; and generating an alarm summary based on the alarm signal, the alarm summary including a description of the alarm and instructions for remediating the detected abnormality.

[0114] In Example 20, the subject of Example 19 may optionally include displaying an alert summary on a monitor visible to a clinician performing a medical procedure.

[0115] Example 21 includes a method, system, or apparatus that includes any element of any of Examples 1 to 20.

[0116] The above detailed description includes reference to the accompanying drawings, which form a part of the detailed description. The drawings illustrate specific examples that can be practiced by way of illustration. These embodiments are also referred to herein as "examples". Such examples may include elements other than those shown or described. However, the inventors also contemplate examples in which only those elements shown or described are provided. Furthermore, the inventors also contemplate examples (or one or more aspects thereof) using any combination or arrangement of those elements shown or described relative to a particular example (or one or more aspects thereof) shown or described herein, or relative to other examples (or one or more aspects thereof).

[0117] All publications, patents, and patent documents mentioned in this document are incorporated herein by reference in their entirety, as if they were incorporated individually by reference. In the event of any inconsistency between the usage in this document and those incorporated by reference, the usage in the incorporated reference shall be considered supplementary to the usage in this document; in the case of irreconcilable inconsistencies, the usage in this document shall prevail.

[0118] In this document, as is common in patent literature, the terms "a" or "an" are used to include one or more, regardless of any other instance or usage of "at least one" or "one or more". In this document, unless otherwise indicated, the term "or" is used to refer to a non-exclusive "or", such that "A or B" includes "A but not B", "B but not A", and "A and B". In the appended claims, the terms "comprising" and "in" are used as concise English equivalents to the corresponding terms "including" and "wherein". Furthermore, in the appended claims, the terms "comprising" and "including" are open-ended, meaning that a system, apparatus, article, or process that includes elements other than those listed in the claims following such terms is still considered to fall within the scope of the claim. Additionally, in the appended claims, the terms "first", "second", and "third", etc., are used merely as designations and are not intended to impose numerical requirements on their objects.

[0119] As used herein, the term “about” means approximately, within a range, roughly, or around. When the term “about” is used in conjunction with a numerical range, it modifies the range by extending the boundaries above and below the stated numerical value. Typically, the term “about” is used herein to modify numerical values ​​above and below the stated value by a change of 10%. In one aspect, the term “about” means plus or minus 10% of the numerical value of the number to which the term is used. Thus, about 50% means a range of 45% to 55%. The numerical ranges listed in this document by endpoints include all numbers and fractions contained within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, 4.24, and 5). Similarly, the numerical ranges listed by endpoints in this document include subranges contained within those ranges (e.g., 1 to 5 includes 1 to 1.5, 1.5 to 2, 2 to 2.75, 2.75 to 3, 3 to 3.90, 3.90 to 4, 4 to 4.24, 4.24 to 5, 2 to 5, 3 to 5, 1 to 4, and 2 to 4). It should also be understood that all numbers and their fractions are assumed to be modified by the term “about”.

[0120] The foregoing description is intended to be illustrative and not restrictive. For example, the examples described above (or one or more aspects thereof) may be used in combination with each other. Other examples may be used by one of ordinary skill in the art upon reading the foregoing description. The abstract is intended to enable the reader to quickly determine the nature of the technical disclosure and is provided with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, in the foregoing detailed description, various features may be combined to simplify the disclosure. This should not be construed as implying that any unclaimed disclosed feature is necessary for any claim. Rather, the subject matter of the invention may lie in fewer than all features of a particular disclosed embodiment. Therefore, the appended claims are hereby incorporated into the detailed description, wherein each claim exists independently as a separate embodiment. The scope of the examples should be determined by reference to the full scope of the appended claims together with the equivalents conferred by those claims.

[0121] The devices disclosed herein may be designed for single-use post-treatment or for multiple-use applications. However, in either case, the device may be repaired for reuse after at least one use. Repair may include a combination of the following steps: disassembly of the device, subsequent cleaning or replacement of specific parts, and subsequent reassembly. Specifically, the device may be disassembled, and any number of specific parts or components of the device may be selectively replaced or removed in any combination. After cleaning and / or replacement of specific parts, the device may be reassembled at a repair facility or immediately before a surgical procedure by a surgical team for subsequent use. Those skilled in the art will understand that the repair of the device can utilize a variety of different techniques for disassembly, cleaning / replacement, and reassembly. The use of such techniques and the resulting repair devices are within the scope of this application.

[0122] Preferably, the invention described herein is performed prior to surgical procedures. First, new or used instruments are obtained and, if necessary, cleaned. The instruments can then be sterilized. In one sterilization technique, the instruments are placed in a closed and sealed container, such as a plastic bag or a TYVEK® bag. The container and instruments are then placed in a radiation field, such as gamma radiation, X-rays, or high-energy electrons, that can penetrate the container. The radiation kills bacteria on the instruments and in the container. The sterile instruments can then be stored in a sterile container. The sealed container keeps the instruments sterile until they are opened in a medical facility. Any other techniques known in the art can also be used to sterilize the devices, including but not limited to beta or gamma radiation, ethylene oxide, or vapor.

Claims

1. A medical system comprising: A medical device configured to be inserted into a patient during a medical procedure, the medical device comprising: An elongated member, including a distal portion; An instrument configured to extend through the elongated member and protrude laterally from the distal portion of the elongated member, the instrument being configured to interact with a target site during the medical procedure; and A transducer, mounted on the distal portion of the elongated member, such that the instrument extending from the elongated member is within the transducer's field of view, the transducer being configured to capture medical images within the field of view throughout the medical procedure; and A controller includes processing circuitry coupled to a memory including instructions that, when executed by the processing circuitry, configure the processing circuitry to: The extension angle of the device from the elongated member is determined via the medical image from the transducer; and An alarm is generated if the extension angle is higher or lower than the threshold.

2. The medical system according to claim 1, wherein, The elongated component includes: A lumen extending through the elongated member, the lumen being configured to receive the instrument; and A side outlet oblique channel extends from the lumen and is configured to guide the instrument out from the side of the elongated member toward the target site of the medical procedure.

3. The medical system according to claim 2, wherein, The threshold includes a set variance that is higher or lower than the expected extension angle defined by the side exit ramp.

4. The medical system according to claim 1, wherein, The instruction causes the processing circuit to: The bending shape of the device is determined via the medical images from the transducer; The medical device is advanced based on the curvature, including the concave shape, as the instrument extends from the medical device; and A reverse translation alarm is generated to indicate that the medical device should be withdrawn to remove the concave shape from the instrument.

5. The medical system according to claim 4, wherein, The instruction causes the processing circuit to: The medical device is determined to retract when the instrument extends from the medical device based on the bending shape, including the convex shape; and Generate a translational advance alert to indicate that the medical device should be advanced to remove the convex shape from the instrument.

6. The medical system according to claim 1, wherein, The medical images include a first medical image and a second medical image.

7. The medical system according to claim 6, wherein, The instruction causes the processing circuit to: Determine the first location of the feature marker in the first medical image; Determine the second location of the feature marker in the second medical image; as well as The movement of the feature marker is determined based on the first and second positions of the feature marker.

8. The medical system according to claim 7, wherein, The instruction causes the processing circuit to: A reverse translation alarm is generated based on the movement of the feature marker in a first direction to indicate that the medical device should be withdrawn from the patient.

9. The medical system according to claim 8, wherein, The instruction causes the processing circuit to: An advance translation alarm is generated based on the movement of the feature marker in a second direction to indicate that the medical device should be advanced into the patient's body.

10. The medical system according to claim 1, wherein, The medical images include a first medical image and a second medical image, and wherein the instructions cause the processing circuit to: The first extension angle of the instrument from the elongated member is determined via the first medical image; The second medical image is used to determine the second extension angle of the instrument from the elongated member; and A warning is generated indicating the change in the extension angle if the change between the first extension angle and the second extension angle exceeds a change threshold.

11. The medical system according to claim 1, wherein, The medical device includes a sensor located in the distal portion of the elongated member, the sensor being configured to detect longitudinal movement of the medical device.

12. The medical system according to claim 11, wherein, The instruction causes the processing circuit to: The sensor receives translational signals related to the longitudinal movement of the medical device. Determine the first location of the feature marker in the first medical image; Determine the second location of the feature marker in the second medical image; The movement of the feature marker is determined based on the first position of the feature marker, the second position of the feature marker, and the translation signal; as well as An unexpected movement alert is generated based on the disproportion between the movement of the feature markers and the longitudinal movement of the medical device detected by the sensors.

13. The medical system according to claim 1, further comprising: A clinical decision support system configured to assist healthcare professionals during the medical procedure includes a trained neural network to analyze information received from the processing circuitry of the controller and to send suggested clinical information based on the information received from the controller.

14. The medical system according to claim 13, wherein, The instruction causes the processing circuit to: Send the ultrasound image sequence of the target site of the medical procedure to the clinical decision support system; The system receives an alarm signal indicating an abnormality of the device, the alarm signal including instructions for remediating the detected abnormality; and An alarm summary is generated based on the alarm signal. The alarm summary includes a description of the alarm and instructions for remediating the detected anomaly.

15. A method for automatically detecting device malfunctions of a medical device during a medical procedure, the method comprising: The ultrasound image sequence of the target area in the medical procedure is obtained via the transducer of the medical device; Detecting instruments extending from the medical device and within the field of view of the transducer in the ultrasound image sequence; The characteristics of the instrument extending from the medical device are determined for each ultrasound image in the ultrasound image sequence, including the instrument extending from the medical device; and An alarm is generated indicating a potential abnormality of the device extending from the medical device if the characteristics of the device exceed a threshold.

16. The method of claim 15, comprising: The bending angle of the device is determined by comparing the detected extension angle of the device with the expected extension angle. as well as A bending alarm is generated in response to the bending angle exceeding a bending threshold.

17. The method of claim 16, comprising: Translational signals related to the longitudinal movement of the medical device are received via sensors installed in the distal portion of the medical device. Determine the first location of the feature marker in the first medical image; Determine the second location of the feature marker in the second medical image; The movement of the feature marker is determined based on the first position of the feature marker, the second position of the feature marker, and the translation signal; as well as An unexpected movement alarm is generated based on the disproportion between the movement of the feature markers and the longitudinal movement of the medical device detected by the sensors.

18. The method according to claim 16, wherein, Determining the characteristics of the device extending from the medical device for each ultrasound image in the ultrasound image sequence, including the device extending from the medical device, includes edge detection technology.

19. The method of claim 15, comprising: Send the ultrasound image sequence of the target site of the medical procedure to the clinical decision support system; The system receives an alarm signal indicating an abnormality of the device, the alarm signal including instructions for remediating the detected abnormality; as well as An alarm summary is generated based on the alarm signal. The alarm summary includes a description of the alarm and instructions for remediating the detected anomaly.

20. The method of claim 19, comprising: The alert summary is displayed on a monitor visible to the clinician performing the medical procedure.

Images