Catheter and device identification for facilitating chest radiographic review

Abstract

The invention relates to a computer-implemented method of determining an obstruction of a medical device to a medical image. The method comprises: receiving a medical image of a region of interest of a subject, the region of interest comprising at least an anatomical structure and a medical device arranged at the subject; segmenting the medical device in the medical image; and classifying the segmented medical device as at least a medical device inside the subject or a medical device outside the subject. The method further includes segmenting an anatomical structure in the medical image and determining an overlap region of the medical device with the anatomical structure in the medical image. The method further includes classifying the determined overlap region as obstructing the medical image if the overlap region occludes at least a portion of the anatomical structure related to diagnosis of the subject based on the medical image and the data related to the subject, and classifying the determined overlap region as obstructing the medical image if the overlap region occludes at least a portion of the anatomical structure related to diagnosis of the subject based on the medical image and the data related to the subject; and providing, as an output of the method, an indication of the overlap region classified as obstructing the medical image.

Description

Technical Field

[0001] The present invention relates to a computer-implemented method, data processing apparatus, computer program, and computer-readable storage medium for determining interference of medical devices on medical images. Background Technology

[0002] Portable chest X-ray imaging is a common imaging modality used, for example, in intensive care units and emergency rooms. Typically, the imaging is performed using a portable system and provides anteroposterior views of the patient, resulting in poor image quality. Patients admitted to ICUs and emergency rooms are often very seriously ill and require monitoring and management through various devices and catheters attached to the body. The traces of these devices and catheters (which can be external or internal) appear on the X-ray images, making different tubes, cannulas, and leads visible in the resulting images. Because chest X-ray images are two-dimensional projections, they lack depth information. Consequently, external devices and catheters, such as ECG leads, can be difficult to distinguish from internal devices and catheters, such as chest drainage tubes or pacemaker leads.

[0003] These traces of medical equipment in X-ray images can cause problems during image review, especially if the reviewing physician fails to correctly identify these structures, if they are not directly in charge of the patient. This can also be problematic if the equipment obscures a critical area of ​​the organ or structure being imaged, leading to difficulties in review and reporting.

[0004] CN112150574B describes a method and system for automatically correcting image artifacts in reconstructed images.

[0005] Gambato Marco et al.'s "Chest X-ray Interpretation: Detecting Devices and Device-Related Complications", Diagnostics, Vol. 13, No. 4, February 6, 2023 (2023-02-06), p. 599, XP093142755, describes the detection of devices and device-related complications in chest X-ray images to identify the correct location of medical devices.

[0006] The inventors of this invention have thus discovered that it would be beneficial to have an improved method for identifying medical devices attached to a patient and potentially affecting the patient's image and diagnosis in X-ray images. Summary of the Invention

[0007] The purpose of this invention is to provide a method for determining the impairment of medical images by medical devices.

[0008] The object of the invention is achieved by the subject matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.

[0009] The described embodiments also relate to computer-implemented methods, data processing apparatuses, computer programs, and computer-readable storage media for determining computer-implemented impairment of medical images by medical devices. The further described embodiments can be combined in any possible manner. Although not described in detail, different combinations of embodiments may produce synergistic effects.

[0010] Furthermore, it should be noted that all embodiments of the method of the present invention can be performed in the order of the described steps; however, this is not necessarily the only or necessary order of the steps. The method presented herein can be performed in a different order of the disclosed steps without departing from the corresponding method, unless expressly stated otherwise below.

[0011] According to a first aspect of the invention, a computer-implemented method is provided for determining obstruction of a medical image by a medical device. The method includes the steps of: receiving a medical image of a region of interest of a subject, the region of interest including at least anatomical structures and a medical device disposed at the subject; segmenting the medical device in the medical image; and classifying the segmented medical device as either a medical device inside the subject or a medical device outside the subject. The method further includes the steps of: segmenting the anatomical structures in the medical image and determining an overlapping region between the medical device and the anatomical structures in the medical image. The method further includes the steps of: classifying the determined overlapping region as obstructing the medical image if the overlapping region occludes at least a portion of the anatomical structures relevant to a diagnosis of the subject based on the medical image and data related to the subject; and providing an indication of the overlapping region classified as obstructing the medical image as an output of the method.

[0012] Therefore, a method is proposed to identify devices and catheters in chest X-ray images and further characterize them as in vitro or in vivo, i.e., outside or inside the patient. Furthermore, identified devices and relevant overlapping areas between the devices and organs or structures of interest can be annotated in the images as needed, and traces of specific devices, multiple devices, or device categories can be switched to improve anatomical visualization. Additionally, feedback can be provided for external devices to better manage their placement.

[0013] Therefore, upon receiving a medical image, different devices and catheters, as well as different chest structures and organs, are segmented from the input chest X-ray image. The location of the devices is identified, and it is determined whether the devices obstruct the diagnosis of important parts of underlying structures or organs in the X-ray image, or important parts of another device of interest. Optionally, if an external device obstructs an important structure or discovery, the radiologist may be provided with instructions for manual correction of the device's position. Additionally, or alternatively, if traces of internal devices and catheters in the input image obscure chest structures or lesions, artifact suppression and / or image enhancement methods may be used to suppress these traces. Furthermore, the medical image is presented to the user through a user interface, along with instructions for identified devices, and optionally, the image is enriched by removing visualizations of detected devices from the image.

[0014] Therefore, a method is proposed to detect overlap between medical devices and organs or structures that may obscure regions of interest in an image. Furthermore, feedback can be provided to better manage device positioning, which will simplify image review. Optionally, image processing methods can be used to remove artifacts or traces of medical devices from the image to provide physicians with richer, undamaged X-ray images.

[0015] In an embodiment of the invention, the medical image is an X-ray image, and the region of interest is the chest of the object.

[0016] In embodiments of the invention, if the medical device is classified as a medical device located outside the object, the method includes an additional step of providing the user with instructions on how to reposition the medical device relative to the object. Thus, a second image can be acquired from the patient without affecting image quality due to the placement of the medical device.

[0017] In embodiments of the present invention, the method further includes the steps of: editing a medical image to remove the reproduction of a medical device from the medical image, and providing an edited medical image. Therefore, image processing methods can be used to remove traces of in vivo devices that are difficult to reposition from an image, thereby providing radiologists with high-quality images.

[0018] In an embodiment of the invention, the method further includes a step of receiving a selection of at least one medical device, the representation of which in a medical image will be removed from the medical image.

[0019] In embodiments of the present invention, the medical device is an external medical device including an ECG lead or an external catheter, or the medical device is an internal medical device including a central venous catheter, a pacemaker lead, or a chest drainage tube.

[0020] In embodiments of the present invention, the anatomical structure is an organ of the object or at least a part of an organ of the object.

[0021] In embodiments of the present invention, the data relating to the object corresponds to the object's condition, diagnosis, and / or treatment information.

[0022] In embodiments of the present invention, the data related to the object is derived from hospital information systems, clinical databases, and / or organ image databases.

[0023] In embodiments of the present invention, at least one of the steps of the method is performed by trained artificial intelligence.

[0024] In an embodiment of the present invention, the method further includes the following steps: training artificial intelligence to perform at least one of the following steps: segmenting medical devices in the medical image; classifying the segmented medical devices at least as medical devices inside the object or medical devices outside the object; and segmenting anatomical structures in the medical image.

[0025] In embodiments of the present invention, the method further includes the step of determining the overlapping region between the medical device and a second medical device of interest in the medical image. Therefore, it is also possible to identify overlaps between medical devices and other devices in a medical image and remove obstructions to the view of that device from the image.

[0026] According to another aspect of the present invention, a data processing apparatus is provided, the data processing apparatus comprising units for performing steps of the method according to any of the foregoing embodiments.

[0027] According to another aspect of the invention, a computer program including instructions is provided, which, when run by a computer, cause the computer to perform the steps of the method according to any of the foregoing embodiments.

[0028] According to another aspect of the invention, a computer-readable storage medium is provided that includes instructions, which, when executed by a computer, cause the computer to perform the steps of the method according to any of the foregoing embodiments.

[0029] In short, the present invention relates to a computer-implemented method for determining the obstruction of a medical device to a medical image. The method includes: receiving a medical image of a region of interest of a subject, the region of interest including at least anatomical structures and a medical device disposed at the subject; segmenting the medical device in the medical image; and classifying the segmented medical device at least as a medical device inside the subject or a medical device outside the subject. The method further includes: segmenting the anatomical structures in the medical image and determining an overlapping region between the medical device and the anatomical structures in the medical image. The method further includes: classifying the determined overlapping region as obstructing the medical image if the overlapping region occludes at least a portion of the anatomical structures relevant to a diagnosis of the subject based on the medical image and subject-related data; and providing an indication of the overlapping region classified as obstructing the medical image as an output of the method.

[0030] One advantage of embodiments of the present invention is the ability to identify devices and catheters that obscure important chest X-ray features and lesions. Another advantage may be the ability to resolve ambiguities in the identification of devices and catheters seen in chest X-rays. Yet another advantage may be the provision of a feedback system for better management of the placement of external and internal devices and the removal of image artifacts resulting from the placement of medical devices. The proposed method can be advantageously used for quality checks regarding the positioning of external and internal devices during digital X-ray imaging in intensive care units and emergency rooms.

[0031] These advantages are not limiting, and other advantages may be envisioned in the context of this application.

[0032] The above aspects and embodiments will become apparent and will be illustrated from the exemplary embodiments described below. Exemplary embodiments of the invention will now be described with reference to the following figures: Attached Figure Description

[0033] Figure 1 A block diagram of a computer-implemented method for determining interference with medical images by medical devices, according to an embodiment of the present invention, is shown.

[0034] Figure 2 The illustration shows the steps of segmenting medical devices in a medical image using trained artificial intelligence according to an embodiment of the present invention.

[0035] Figure 3 The illustration shows the steps of segmenting anatomical structures in a medical image using trained artificial intelligence according to an embodiment of the present invention.

[0036] Figure 4An example of a patient's chest X-ray image is shown, in which there is ground-glass opacity in the periphery of the lungs, and the external ECG lead obscures part of the periphery of the lungs. List of reference numerals 110 Medical Images 120 Medical Equipment 130 objects 140 Anatomical Structure 150 Artificial Intelligence Detailed Implementation

[0037] Figure 1 A block diagram of a computer-implemented method for determining obstruction of a medical image 110 by a medical device 120 according to an embodiment of the present invention is shown. The method includes: step S110, receiving a medical image 110 of a region of interest of an object 130, the region of interest including at least an anatomical structure 140 and a medical device 120 disposed on the object; step S120, segmenting the medical device 120 in the medical image 110; and step S130, classifying the segmented medical device 120 at least as a medical device inside the object 130 or a medical device outside the object 130. The method includes a further step S140, segmenting the anatomical structure 140 in the medical image 110, and step S150, determining the overlapping region of the medical device 120 and the anatomical structure 140 in the medical image 110. The method includes an additional step S160, classifying the identified overlapping region as obstructing the medical image 110 if the overlapping region occludes at least a portion of the anatomical structure 140 relevant to a diagnosis of the object 130 based on the medical image 110 and data relating to the object 130; and step S170, providing an indication of the overlapping region classified as obstructing the medical image 110 as an output of the method.

[0038] Figure 2 A diagram illustrating the steps of segmenting medical devices 120 in a medical image 110 using trained artificial intelligence 150 according to an embodiment of the present invention. Step S120 of the proposed method involves segmenting visible devices and catheters in the medical image 110 of object 130, and then classifying them as internal or external devices in step S130. To achieve this, a deep learning-based segmentation and classification model has been developed, the architecture of which is as follows... Figure 2The following is an exemplary illustration. For example, X-ray images containing at least one of various external and internal devices and catheters are labeled to train the aforementioned deep learning model for segmentation and classification tasks. The model acquires unseen images and segments the devices / catheters within the images, further classifying the segmented devices as external or internal devices. The model can also further subdivide the devices or catheters into more specific subcategories. Thus, preferably, in step S130, at least one medical device 120 detected and segmented in medical image 110 is not only classified as internal or external. Furthermore, if it is an internal device, it can be classified as a central venous catheter, nasogastric tube, or endotracheal tube; if it is an external device, it can be classified as an electrocardiogram lead or external tubing, etc.

[0039] Another approach to distinguishing and classifying external and internal devices is, if available, to use prospective image training models to segment and classify external devices from all devices detected and segmented in X-ray images. An alternative training method could be to acquire only X-ray images of the medical devices and overlay them onto X-ray images without any visible medical devices. This is accurate for external devices and can create strong enhancement probabilities because the device overlay can be moved, rotated, etc., on the medical image. In another embodiment, the segmentation results can be post-processed using tracking methods (e.g., vascular tracking methods) to separate cannulas, leads, and leads from different devices. This helps identify unique devices and makes it possible to propose methods for repositioning them, resulting in better image quality and less occlusion.

[0040] Figure 3 The illustration shows the steps of segmenting anatomical structures 140 in a medical image 110 using trained artificial intelligence 150 according to an embodiment of the present invention. Segmenting the lower anatomical structures 140 and organs in the X-ray image of object 130 to detect their overlap with equipment is important. A deep learning-based segmentation model is trained to detect and segment structures and organs that may overlap with different external or internal equipment and occlude regions of interest during image review.

[0041] To assess the position of the device and catheter relative to the underlying structure, in step S150, overlapping regions between the medical device 120 and the underlying anatomical structure 140 and / or organs are detected and segmented. External or internal devices, as well as thoracic organs and structures, are segmented using the methods / models discussed in previous sections. Based on the segmented regions, device-organ-structure overlaps are detected and quantified. Once a list of overlapping regions is identified, it is important to assess which overlapping regions are significant based on patient condition, diagnosis, and treatment information. Hospital information system (HIS) databases can be queried and analyzed to retrieve and analyze patient condition, diagnosis, and treatment-related data, and natural language processing (NLP) techniques can be applied to extract insights. Next, patient diagnosis and treatment information can be used to locate relevant device-organ overlaps that may obscure the region of interest. Clinical databases and / or organ mapping databases can be consulted to map the patient's diagnosis and treatment information to relevant thoracic organs or structures. This mapping may also include sub-regions of the organ of interest, such as the upper, middle, or lower lobe of the lung. The proposed method can also detect overlap between different devices and identify whether a part of one device obscures another device of interest; for example, an ECG lead may obscure part of a catheter.

[0042] Figure 4 An example of a patient's chest X-ray image is shown, in which there is ground-glass opacity in the periphery of the lung, and the external ECG lead obscures a portion of the lung's periphery. The proposed method detects and segments this overlapping area and provides feedback to the user. Once the relevant overlapping area is identified, it is important to indicate the findings to a radiologist or clinician and provide feedback for correction of the overlap.

[0043] If overlapping of internal devices is observed, the user is alerted as feedback. Repositioning of internal devices is often impossible. For example, if traces of internal devices and catheters in the input image obscure important areas of chest structures or lesions, artifact suppressor or image intensifier methods can be invoked to suppress traces of internal devices and catheters as needed.

[0044] If an external device obstructs a section of a structure, organ, or vital internal device, the present invention can alert the user and provide feedback on repositioning the external device at the patient or subject 130. If repositioning the device is not possible, artifact suppressors or image intensifiers can be invoked to suppress the device's traces. Figure 4 The example shown illustrates how radiographs were taken to understand a patient's disease progression, previous scans, and diagnoses, reporting that the patient had peripheral ground-glass opacities (GGO) in the lungs. Figure 4In the medical image 110, the ECG lead obscures the peripheral region of the left lung, which is precisely the region of interest for diagnosing disease progression. In this scenario, the proposed method can indicate positioning errors related to external devices and provide feedback to correct the positioning of the ECG lead.

[0045] Preferably, feedback can be provided to radiologists, reviewers, or other medical professionals in the following manner: A list of identified devices and catheters that may obscure the entire image or a specific diagnostically relevant region of interest (ROI), which can be user-defined or defined by analyzing a hospital information system. These devices and catheters can be colored with coded colors and labeled accordingly. Additionally, a list of repositionable external devices can be provided, along with optional toggle buttons that allow suppression of traces of all devices and catheters, or suppression of traces of specific categories of devices and catheters, or suppression of traces of specific catheters; these can also be suppressed by directly clicking on the identified structure.

[0046] Furthermore, a system is proposed configured to identify devices and catheters in X-ray images and further characterize them as external or internal devices. The system includes: a segmentation module for segmenting different devices and catheters from an input X-ray image; a segmentation module for segmenting different thoracic structures and organs from the X-ray image; and a module for identifying whether the location of a device obstructs important parts of underlying structures / organs or other devices of interest. The system also includes: a guidance module for instructing a radiologist to manually correct the location if an external device obstructs an important structure or discovery; and an artifact suppressor or image intensifier module for suppressing the presence of internal devices and catheters from the input image when these obscure thoracic structures or lesions. Additionally, a presentation module may be provided for presenting the enhanced image to a user via a user interface element. Therefore, devices and catheters can be prevented from obscuring important discoveries in X-ray images.

[0047] Although the invention has been illustrated and described in detail in the accompanying drawings and the foregoing description, such illustrations and descriptions should be considered illustrative or exemplary, not restrictive. The invention is not limited to the disclosed embodiments. Those skilled in the art, through studying the drawings, the disclosure, and the dependent claims, will understand and implement other variations of the disclosed embodiments in practicing the claimed invention.

[0048] In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. Although a particular measure is recited in different dependent claims, this does not indicate that combinations of these measures cannot be advantageously used. No reference numerals in the claims should be construed as limiting the scope.

Claims

1. A computer-implemented method for determining interference of a medical device (120) with a medical image (110), the method comprising the steps of: Receive (S110) a medical image (110) of a region of interest of an object (130), the region of interest including at least an anatomical structure (140) and a medical device (120) arranged on the object. Segment (S120) the medical device (120) in the medical image (110); The segmented medical device (120) is classified (S130) at least as a medical device inside the object (130) or a medical device outside the object; Segment (S140) the anatomical structures (140) in the medical image (110); Determine (S150) the overlapping area of ​​the medical device (120) and the anatomical structure (140) in the medical image (110); If the overlapping region obscures at least a portion of the anatomical structure (140) relevant to the diagnosis of the object (130) based on the medical image (110) and data relating to the object (130), then the determined overlapping region is classified (S160) as obstructing the medical image (110); and (S170) Provides (S170) an indication of the overlapping regions classified as obstructing the medical image (110) as the output of the method.

2. The method according to claim 1, wherein, The medical image (110) is an X-ray image, and the region of interest is the chest of the object (130).

3. The method according to any one of claims 1 or 2, wherein, If the medical device (120) is classified as a medical device outside the object, the method further includes the step of providing the user with guidance on how to reposition the medical device (120) relative to the object (130).

4. The method according to any one of the preceding claims, wherein, The method further includes the steps of: editing the medical image (110) to remove the reproduction of the medical device (120) in the medical image (110) and providing the edited medical image.

5. The method according to claim 4, wherein, The method further includes the step of receiving a selection of at least one medical device (120), the representation of which in the medical image (110) is to be removed from the medical image.

6. The method according to any one of the preceding claims, wherein, The medical device (120) is an external medical device, including an ECG lead or external tube, or wherein... The medical device (120) is an internal medical device, including a central venous catheter, pacemaker lead, or chest drainage tube.

7. The method according to any one of the preceding claims, wherein, The anatomical structure (140) is an organ of the object (130) or at least a part of an organ of the object.

8. The method according to any one of the preceding claims, wherein, The data relating to the object (130) corresponds to the condition, diagnosis and / or treatment information of the object (130).

9. The method according to any one of the preceding claims, wherein, The data relating to the object (130) is derived from hospital information systems, clinical databases and / or organ image databases.

10. The method according to any one of the preceding claims, wherein, At least one of the steps of the method is performed by a trained artificial intelligence (150).

11. The method according to claim 10, wherein, The method further includes the following steps: training artificial intelligence (150) to perform at least one of the following steps: segmenting the medical device (120) in the medical image (110); classifying the segmented medical device (120) at least as a medical device inside the object (130) or a medical device outside the object; and segmenting the anatomical structure (140) in the medical image (110).

12. The method according to any one of the preceding claims, wherein, The method further includes the following step: determining the overlapping area between the medical device (120) and the second medical device of interest in the medical image (110).

13. A data processing apparatus (100) comprising a unit for performing the steps of the method according to any one of claims 1 to 12.

14. A computer program comprising instructions which, when run by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 12.

Images