Deep learning-based methods and systems for automated subject anatomy and orientation recognition

Abstract

The present invention provides a system and method for automated patient anatomical structure and orientation recognition using a deep learning module based on artificial intelligence (AI). The method includes positioning a subject above a bed of a magnetic resonance imaging (MRI) system, and wrapping at least one radio frequency (RF) imaging coil around the subject. The method includes using a three-dimensional (3D) depth camera to obtain multiple depth images, color images, and infrared images of the subject, and using the image obtained by the 3D camera to identify the bed boundary of the MRI system. The method also includes identifying the position of the subject above the bed to determine whether the subject is positioned within the bed boundary of the MRI system, and identifying multiple key anatomical points or regions corresponding to multiple organs of the subject's body. The method also includes identifying all DICOM orientations of the subject above the bed of the MRI system, and identifying the coil of the MRI system wrapped around the subject's body, and determining the orientation of the subject relative to the coil, patient gown, and blanket of the MRI system. The method also includes identifying anatomical key points blocked by the coil, patient gown, and blanket of the MRI system to determine the accurate positioning of the coil of the MRI system above the anatomical structure of the subject, for automated landmarks of the anatomical structure and imaging.

Description

Technical Field

[0001] The present disclosure generally relates to improved medical imaging systems and methods, and more specifically to systems and methods for automated patient anatomy and orientation recognition using an artificial intelligence (AI)-based deep learning module during occlusion of a coil from a magnetic resonance imaging (MRI) system using a three-dimensional (3D) depth camera. Background Art

[0002] Various medical imaging systems and methods are used to obtain images of affected areas of a subject's anatomy for use in diagnosing medical conditions. Magnetic resonance imaging (MRI) is a known medical imaging technique for imaging different body parts of a subject, such as the head, chest, abdomen, and legs. MR imaging involves positioning a subject on an examination couch of a patient positioning system of an MRI apparatus, and moving the patient positioning system into a gantry of the MRI apparatus for imaging. An MRI system may include a variety of imaging radio frequency (RF) coils, such as a whole-body RF coil, which may be adapted to transmit waveforms toward a subject and configured to receive waves from the subject to acquire image data from the subject.

[0003] Although the images produced by these MR imaging techniques are of good quality, many images are adversely affected by the operating conditions of the MRI system and the movement of the subject. Movement of the subject, incorrect positioning of the subject on the patient positioning system, and incorrect positioning of the imaging coil above the subject can cause the MRI system to obtain erroneous images. Such images can be rejected by the radiologist, and in order to obtain a high-quality user-visible image, it is necessary to re-image the subject. Because such imaging errors cause the subject's health to deteriorate, valuable time for treating subjects with serious medical conditions such as trauma may be lost. Therefore, in order to avoid MR images being rejected, it is critical to accurately implement imaging techniques.

[0004] Accurate imaging of subjects using MRI equipment requires a skilled operator. Any error on the operator's side will result in images containing too much interference or noise. Therefore, the operator's training and experience in operating MR imaging equipment will affect the MR image quality. Due to the operator's experience in acquiring images of certain parts of the body, the operator can skillfully acquire high-quality images of that part of the body, but the operator may not be able to skillfully obtain high-quality images of the entire subject's body. This may be due to lack of imaging experience or lack of anatomical knowledge of the entire body. In addition, placing the MR imaging coil at the appropriate body part will affect the image quality. Physical parameters of the subject's body such as obesity, bone density, height, chest or abdominal size will bring challenges to the operator to correctly position the MR imaging coil to obtain a complete image of the region of interest. Any incorrect placement of the MR imaging coil will result in poor image quality and rejection of the image of the region of interest.

[0005] The main reasons for image rejection are listed in the following study: Little, Kevin J., et al., "Unified database for rejected images analysis across multiple vendors inradiography", Journal of the American college of Radiology 14.2 (2017); 208-2016. These rejection reasons can include incorrect positioning of the subject on the patient positioning system, incorrect use of imaging technology by the operator, movement of the subject, imaging artifacts, etc. The study also showed that among the rejected images of different body organs and systems, the rejection percentage of images of the chest, abdomen, pelvis, and spine regions was higher than the rejection percentage of images of other body organs.

[0006] The cause of most pre-scan errors in MRI radiology workflow is improper positioning of the subject, the operator of the MRI system follows incorrect imaging schemes for the anatomical structures to be scanned, and negligence of the operator / technician. The operator / technician responsible for scanning the subject may cause pre-scan errors, such as scanning subjects with incorrect subject orientations, scanning subjects with inappropriate postures, angles, and directions. In addition, when the subject registers with a prescription for the body part to be scanned and enters the scanner room, the operator checks the prescription and then prepares the patient for scanning. In this process, pre-scan errors may be caused due to, for example, the operator misinterpreting the prescription and instructing the subject to tend to inappropriate postures and orientations. The subject may misinterpret the instructions given by the operator, resulting in inappropriate postures and orientations. In another example, the operator may complete the scan without noticing the incorrect posture and orientation of the subject.

[0007] All of these pre-scan errors can result in scans being rejected and repeated. This causes discomfort to the subject, increases waiting time for the subject and cognitive stress for the operator and radiologist, and further reduces the throughput of the scanning instrument. In addition, the operator / radiologist must manually select the appropriate imaging protocol for scanning the subject based on positioning and orientation / posture. This is always error-prone because if the operator / technician is not alert enough to adopt the correct protocol, they will end up scanning the patient with the wrong protocol or incorrect posture.

[0008] Therefore, there is a need for systems and methods for providing automated guidance to the scanning personnel to properly position a subject on the examination table of an MRI system, set scanning parameters, use the appropriate imaging coils of the MRI system, and scan a region of the subject to generate high-quality MR images with minimal errors. Summary of the invention

[0009] This summary introduces concepts that are described in more detail in the detailed description. It should not be used to determine the essential features of the claimed subject matter, nor should it be used to limit the scope of the claimed subject matter. Its sole purpose is to present concepts in a simplified form as a prelude to a more detailed description that will be presented later.

[0010] According to one aspect of the present disclosure, a method for automatically identifying the anatomical structure and orientation of a subject using a deep learning module based on artificial intelligence (AI) is disclosed. The method includes positioning the subject above the examination bed of a magnetic resonance imaging (MRI) system, and wrapping at least one radio frequency (RF) imaging coil above the subject. The method also includes obtaining multiple depth images, color images, and infrared images of the subject using a three-dimensional (3D) depth camera. The method also includes using these images acquired by the 3D camera to identify the examination bed boundary of the MRI system. The method also includes identifying the position of the subject above the examination bed to determine whether the subject is positioned within the examination bed boundary of the MRI system, and identifying multiple key anatomical points corresponding to multiple organs of the subject's body. The method also includes identifying the orientation of the subject above the examination bed of the MRI system. The method also includes identifying the coils of the MRI system wrapped around the subject's body, and determining the orientation of the subject relative to these coils of the MRI system. The method also includes identifying the anatomical key points occluded by the coils of the MRI system to determine accurate positioning of the coils of the MRI system over the subject's anatomy for imaging.

[0011] According to one aspect of the present disclosure, a system for automating anatomical structure and orientation recognition of a subject using an artificial intelligence (AI)-based deep learning module during occlusion of an imaging coil from a magnetic resonance imaging (MRI) system is disclosed. The system includes a three-dimensional (3D) depth camera configured to capture multiple depth images, color images, and infrared images of a subject positioned on an examination bed of the magnetic resonance imaging (MRI) system. The system also includes a computer system connected to the 3D depth camera and configured to receive the multiple images from the 3D depth camera. The computer system includes: a processor; a memory connected to the processor and at least one artificial intelligence (AI)-based deep learning module deployed above the memory. The AI-based deep learning module can be configured to use the images obtained by the 3D camera to identify the examination bed boundary of the MRI system. The AI-based deep learning module can be further configured to identify the position of the subject above the examination bed to determine whether the subject is positioned within the examination bed boundary of the MRI system, and identify multiple key anatomical points corresponding to multiple organs of the subject's body. The AI-based deep learning module may be further configured to identify the orientation of the subject above the examination couch of the MRI system, and to identify the MRI coils wrapped around the subject's body, and to determine the orientation of the subject relative to the MRI coils. The AI-based deep learning module may be further configured to identify the anatomical key points occluded by the MRI coils to determine the exact position of the MRI coils above the subject's anatomical structure for imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG. 1( a ) is a graphical representation of various operating parameters that may be reasons for MR image rejection according to one aspect of the present disclosure.

[0013] FIG. 1( b ) shows a pie chart representing the organ-wise distribution of rejected MR images according to one aspect of the present disclosure.

[0014] Figure 2 An exemplary embodiment of a magnetic resonance imaging (MRI) system according to one aspect of the present disclosure is shown.

[0015] Figure 3 An exemplary three-dimensional (3D) depth camera mounted above an examination table of an MRI system is shown according to one aspect of the present disclosure.

[0016] Figure 4 A method of performing a health check of a 3D camera according to one aspect of the present disclosure is shown.

[0017] Figure 5A method of identifying couch boundaries to calculate four corners of a couch and a midpoint of the couch of an MR imaging system according to one aspect of the present disclosure is shown.

[0018] Figure 6 A method for using an artificial intelligence (AI) based deep learning module for MR couch recognition, subject orientation, identification of RF coils, and identification of anatomical structures according to one aspect of the present disclosure is shown.

[0019] Figure 7 A method for automated subject anatomy and orientation recognition using a three-dimensional (3D) depth camera using an artificial intelligence (AI) based deep learning module according to one aspect of the present disclosure is shown.

[0020] Figure 8 A method for automated patient anatomy and orientation recognition using a three-dimensional (3D) depth camera using an artificial intelligence (AI) based deep learning module according to another aspect of the present disclosure is shown.

[0021] FIG. 9( a ) illustrates a method for couch recognition of an MRI system using raw point cloud counters extracted from a depth frame according to one aspect of the present disclosure.

[0022] FIG9( b ) illustrates a method for couch recognition of an MRI system using a top-down view of a raw point cloud counter extracted from a depth camera according to one aspect of the present disclosure.

[0023] FIG9( c ) illustrates a method for performing couch recognition for an MRI system using point cloud data from a floor plane extracted from a depth camera according to one aspect of the present disclosure.

[0024] FIG9( d ) illustrates a method for performing couch recognition for an MRI system using point cloud data without a floor plane extracted from a depth camera according to one aspect of the present disclosure.

[0025] FIG9( e ) illustrates a method for performing couch recognition of an MRI system using a depth camera using point cloud data in a couch plane according to one aspect of the present disclosure.

[0026] FIG10( a ) illustrates an artificial intelligence (AI) based deep learning module for identifying patients according to one aspect of the present disclosure.

[0027] FIG10( b ) illustrates an artificial intelligence (AI) based deep learning module for identifying patients according to one aspect of the present disclosure.

[0028] Fig.11A method for identifying key anatomical points prior to coil placement according to one aspect of the present disclosure is shown.

[0029] Fig.12 AI-based orientation identification of a subject using shape and thickness according to one aspect of the present disclosure is shown.

[0030] Fig.13 AI-based identification of RF coils for an MRI system according to one aspect of the present disclosure is shown.

[0031] Fig.14 A class-imbalanced object recognition network based on focal loss according to one aspect of the present disclosure is shown.

[0032] Fig.15 AI-based identification of a subject using an RF coil according to one aspect of the present disclosure is shown.

[0033] Fig.16 A method for automated anatomical structure and orientation recognition of a subject using an artificial intelligence (AI) based deep learning module during occlusion of an imaging coil from a magnetic resonance imaging (MRI) system according to one aspect of the present disclosure is shown. DETAILED DESCRIPTION

[0034] In the following specification and claims, reference will be made to several terms which shall be defined as having the following meanings.

[0035] The singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.

[0036] As used herein, the term "non-transitory computer-readable medium" is intended to mean any tangible computer-based device implemented in any method or technology for short-term and long-term storage of information, such as computer-readable instructions, data structures, program modules and submodules, or other data in any device. Therefore, the methods described herein may be encoded as executable instructions embodied in a tangible non-transitory computer-readable medium including, but not limited to, a storage device and / or a memory device. When executed by a processor, such instructions cause the processor to perform at least a portion of the methods described herein. In addition, as used herein, the term "non-transitory computer-readable medium" includes all tangible computer-readable media, including, but not limited to, non-transitory computer storage devices, including, but not limited to, volatile and non-volatile media, and removable and non-removable media, such as firmware, physical and virtual storage, CD-ROMs, DVDs, and any other digital source (such as a network or the Internet), as well as digital devices yet to be developed, with the sole exception of transient propagation signals.

[0037] As used herein, the terms "software" and "firmware" are interchangeable and include any computer program stored in memory for execution by devices including, but not limited to, mobile devices, clusters, personal computers, workstations, clients, and servers.

[0038] As used herein, the term "computer" and related terms (e.g., "computing device," "computer system," "processor," "controller") are not limited to the integrated circuits known in the art as computers, but refer broadly to at least one microcontroller, microcomputer, programmable logic controller (PLC), application specific integrated circuit, and other programmable circuits, and these terms are used interchangeably herein.

[0039] As used herein throughout the specification and claims, approximating language may be applied to modify any quantitative representation that may allow variation without causing a change in the basic function to which it is associated. Therefore, a value modified by one or more terms such as "about" and "substantially" is not limited to the exact value specified. In at least some cases, approximating language may correspond to the precision of the instrument used to measure the value. Here and throughout the specification and claims, range limitations may be combined and / or interchanged, and such ranges may be identified and include all subranges contained therein unless the context or language indicates otherwise.

[0040] In one aspect of the present disclosure, a method for automating anatomical structure and orientation recognition of a subject using a deep learning module based on artificial intelligence (AI) is disclosed. The method includes positioning the subject above a bed of a magnetic resonance imaging (MRI) system and wrapping at least one radio frequency (RF) imaging coil above the subject. The method also includes obtaining multiple depth images, color images, and infrared images of the subject using a three-dimensional (3D) depth camera. The method also includes using these images acquired by the 3D camera to identify the bed boundary of the MRI system. The method also includes identifying the position of the subject above the bed to determine whether the subject is positioned within the bed boundary of the MRI system, and identifying multiple key anatomical points corresponding to multiple organs of the subject's body. The method also includes identifying the orientation of the subject above the bed of the MRI system. The method also includes identifying the coils of the MRI system wrapped around the subject's body, and determining the orientation of the subject relative to these coils of the MRI system. The method also includes identifying the anatomical key points occluded by the coils of the MRI system to determine accurate positioning of the coils of the MRI system over the subject's anatomy for imaging.

[0041] In another aspect of the present disclosure, a system for automating anatomical structure and orientation recognition of a subject using an artificial intelligence (AI)-based deep learning module during occlusion of an imaging coil from a magnetic resonance imaging (MRI) system is disclosed. The system includes a three-dimensional (3D) depth camera configured to capture multiple depth images, color images, and infrared images of a subject positioned on an examination bed of the magnetic resonance imaging (MRI) system. The system also includes a computer system connected to the 3D depth camera and configured to receive the multiple images from the 3D depth camera. The computer system includes: a processor; a memory connected to the processor and at least one artificial intelligence (AI)-based deep learning module deployed above the memory. The AI-based deep learning module can be configured to use the images obtained by the 3D camera to identify the examination bed boundary of the MRI system. The AI-based deep learning module can be further configured to identify the position of the subject above the examination bed to determine whether the subject is positioned within the examination bed boundary of the MRI system, and identify multiple key anatomical points corresponding to multiple organs of the subject's body. The AI-based deep learning module may be further configured to identify the orientation of the subject above the examination couch of the MRI system, and to identify the MRI coils wrapped around the subject's body, and to determine the orientation of the subject relative to the MRI coils. The AI-based deep learning module may be further configured to identify the anatomical key points occluded by the MRI coils to determine the exact position of the MRI coils above the subject's anatomical structure for imaging.

[0042] Embodiments of the present disclosure will now be described by way of example with reference to the drawings, wherein FIG. 1 shows a graphical representation of various operating parameters that are reasons for MR image rejection according to one aspect of the present disclosure. Magnetic resonance (MR) imaging involves positioning a subject on a couch of an MRI system and moving the couch into a gantry of the MRI system for imaging. The subject may be positioned and oriented in various imaging positions on the couch based on imaging requirements. Some examples of patient positions on the couch include head-first, feet-first, supine, prone, left lateral decubitus, and right lateral decubitus. The positioning and orientation of the subject on the couch of the MRI system may be affected by the region or organ of the subject to be imaged. In one example, the subject may be placed in a "head-first" and "supine" position on the couch for imaging the lungs within a chest portion of the subject, and the imaging coil of the MRI system may be positioned on the chest for acquiring image data from the subject's body. Such positioning of the imaging coil on different portions of the subject's body to be imaged is prone to several errors. FIG. 1( a) shows a classification of all such imaging errors that may cause an MR image to be rejected. It can be seen that inaccurate positioning of the subject above the examination couch is one of the most important reasons for generating inaccurate MR images and leading to image rejection. Other prominent causes of inaccurate MR images include the operator of the MR imaging system following an incorrect imaging protocol, motion of the patient or patient organs during imaging, for example, movement of the subject's arms, hands or legs. These rejections of MR images are highly influenced by the accuracy with which the operator follows the imaging protocol and are always subject to human error.

[0043] According to one aspect of the present disclosure, FIG. 1( b) shows a pie chart representing the organ-level distribution of rejected MR images. MR images are rejected more frequently for some organs than for other organs. For example, the rejection percentages of MR images of the chest, spine, and lower extremities are higher than the rejection percentages of MR images of other areas of the body (such as the head, abdomen, pelvis, and neck). Therefore, MR images of some parts of the subject's body are more prone to errors than MR images of other areas. Therefore, it is not only necessary to properly position the subject and follow the correct imaging protocol, but it is also important to provide the operator with automated guidance regarding the patient's anatomy and orientation.

[0044] According to one aspect of the present disclosure, Figure 2A schematic diagram of a magnetic resonance imaging (MRI) system (10) is shown. The operation of the system (10) can be controlled from an operator console (12), which includes an input device (13), a control panel (14), and a display screen (16). The input device (13) can be a mouse, a joystick, a keyboard, a trackball, a touch-activated screen, a light stick, a voice controller, and / or other input devices. The input device (13) can be used for interactive geometric shape specification. The console (12) communicates with a computer system (20) via a link (18), which enables an operator to control the generation and display of images on the display screen (16). The link (18) can be a wireless or wired connection. The computer system (20) can include modules that communicate with each other via a backplane (20a). The modules of the computer system (20) can include, for example, an image processor module (22), a central processing unit (CPU) module (24), and a memory module (26), which can include a frame buffer for storing image data arrays. The computer system (20) may be connected to an archival media device, permanent or backup storage, or a network for storing image data and programs, and communicates with an MRI system control (32) via a high-speed signal link (34). The MRI system control (32) may be separate from or integrated with the computer system (20). The computer system (20) and the MRI system control (32) together form an "MRI controller" (33) or "controller". An artificial intelligence (AI)-based deep learning module may be preloaded on the memory of the computer system (20). Several MRI images may be used to train the AI-based deep learning module to recognize different anatomical structures. The AI-based deep learning module may be further trained to recognize several different objects and generate reports, images, comparative analyses of objects for an operator of the MRI system. Further details of the structure and function of the AI-based deep learning module have been provided together with the accompanying drawings at appropriate locations in the specification.

[0045] In an exemplary embodiment, the MRI system control (32) includes modules that can be connected by a base plate (32a). These modules include a CPU module (36) and a pulse generator module (38). The CPU module (36) is connected to the operator console (12) via a data link (40). The MRI system control (32) receives commands from the operator via the data link (40) to indicate a scan sequence to be performed. The CPU module (36) operates the system components to perform the desired scan sequence and generates data indicating the timing, intensity and shape of the generated RF pulses and the timing and length of the data acquisition window. The CPU module (36) is connected to components operated by the MRI controller (32), including the pulse generator module (38) that controls the gradient amplifier (42), the physiological acquisition controller (PAC) (44) and the scan room interface circuit (46).

[0046] In one example, the CPU module (36) receives patient data from a physiological acquisition controller (44) that receives signals from sensors connected to the subject, such as ECG signals received from electrodes attached to the patient. The CPU module (36) receives signals from the sensors associated with the condition of the patient and the magnet system via a scan room interface circuit (46). The scan room interface circuit (46) also enables the MRI controller (33) to command a patient positioning system (48) to move the patient to a desired position for scanning.

[0047] The whole body RF coil (56) is used to transmit the waveform toward the subject's anatomy. The whole body RF coil (56) can be a body coil (as shown in FIG. 1 ). The RF coil can also be a local coil, which can be placed closer to the subject's anatomy than the body coil. The RF coil (56) can be a surface coil. A surface coil containing receive channels can be used to receive signals from the subject's anatomy. A typical surface coil will have eight receive channels; however, different numbers of channels are possible. It is known to use a combination of both a body coil (56) and a surface coil to provide better image quality.

[0048] The pulse generator module (38) can operate the gradient amplifier (42) to achieve the desired timing and shape of the gradient pulses generated during scanning. The gradient waveforms generated by the pulse generator module (38) can be applied to the gradient amplifier system (42) having Gx, Gy and Gz amplifiers. Each gradient amplifier excites a corresponding physical gradient coil in the gradient coil assembly (50) to generate a magnetic field gradient for spatially encoding the acquired signals. The gradient coil assembly (50) can form a part of the magnet assembly (52), which also includes a polarizing magnet (54) (in operation, the polarizing magnet provides a longitudinal magnetic field B0 throughout the target space (55) surrounded by the magnet assembly (52)) and a whole body RF coil (56) (in operation, the coil provides a transverse magnetic field B1, which is approximately perpendicular to B0 throughout the target space (55)). The transceiver module (58) in the MRI system control (32) generates pulses, which are amplified by the RF amplifier (60) and coupled to the RF coil (56) through the transmit / receive switch (62). The resulting signals emitted by the excited nuclei in the subject's anatomy can be sensed by a receive coil (not shown) and provided to a preamplifier (64) via a transmit / receive switch (62). The amplified MR signals are demodulated, filtered, and digitized in the receiver portion of the transceiver (58). The transmit / receive switch (62) is controlled by a signal from the pulse generator module (38) to electrically connect the RF amplifier (60) to the coil (56) during a transmit mode and to connect the preamplifier (64) to the receive coil during a receive mode.

[0049] The MR signals generated by the excitation of the target are digitized by the transceiver module (58). The digitized signals are then processed by the MR system control (32) by Fourier transform to generate k-space data, which is transmitted to the memory module (66) or other computer-readable medium via the MRI system control (32). "Computer-readable medium" may include, for example, a structure configured so that an electrical, optical, or magnetic state can be fixed in a perceptible and reproducible manner by a conventional computer (e.g., text or images printed on paper or displayed on a screen, a compact disk or other optical storage medium, "flash" memory, EEPROM, SDRAM or other electrical storage medium; a floppy disk or other magnetic disk, a tape or other magnetic storage medium).

[0050] The scan is complete when an array of raw k-space data is acquired in a computer readable medium (66). For each image to be reconstructed, the raw k-space data is rearranged into a separate k-space data array, and each of these k-space data arrays is input to an array processor (68) which operates to reconstruct the data into an array of image data using a reconstruction algorithm such as a Fourier transform. The image data is transmitted to the computer system (20) via a data link (34) and stored in a memory. In response to commands received from an operator console (12), the image data may be archived in a long-term storage device or may be further processed by an image processor (22) and transmitted to the operator console (12) and presented on a display (16).

[0051] According to one aspect of the present disclosure, Figure 3 A three-dimensional (3D) depth camera (310) is shown mounted above an examination bed (320) of an MRI system. The 3D depth camera (310) can be configured to detect the depth of a field of view (330) to accurately determine the boundaries of the examination bed (320) and the position of a subject (340) on the examination bed (320). In one aspect, the 3D depth camera (310) can include an infrared light emitting source and an infrared light detector for transmitting and detecting infrared light from the camera (310). The 3D depth camera can be mounted at a plurality of different locations relative to the MRI system (350) within a scanning room. Some non-limiting examples of the location of the 3D depth camera (310) include the ceiling of the room, a side wall, mounted on a stand, mounted on a scanner, or mounted laterally. When the 3D camera (310) is installed, the color model (red-green-blue or RGB), depth, and infrared flow of the 3D camera (310) can be tested. Appropriate firmware for processing image data can be installed on the 3D camera (310), which can be updated periodically.

[0052] According to one aspect of the present disclosure, Figure 4A method (400) for performing a health check of a 3D camera (310) is shown. The method (400) includes checking (410) whether camera settings such as color model (RGB), depth, infrared, and firmware updates are completed. If it is observed that no settings are completed, the method (400) includes checking (420) the 3D camera to check connectivity, field of view, USB cable, and firmware updates. Once the camera settings are completed, the method (400) includes calibrating (430) the different cameras in the room for imaging and obtaining the coordinates of the examination bed, the subject, and the midpoint of the examination bed.

[0053] According to one aspect of the present disclosure, Figure 5 A method (500) of identifying couch boundaries, calculating four corners and a midpoint of the couch is shown. The method (500) includes a subject registering with an imaging prescription and entering (510) an imaging room for scanning. The method (500) includes converting (520) camera coordinates to world coordinates for accurately positioning the subject on the imaging couch of the MRI system. Converting (520) the coordinates to world coordinates includes converting from a camera coordinate system to a subject coordinate system using a rotational transformation matrix and a homogeneous transformation matrix, principles, and a focus obtained during calibration (430) of a 3D camera. The method (500) also includes generating (530) patient coordinates relative to the MR imaging system. Generating (530) patient coordinates relative to the MR imaging system includes identifying a position and orientation of the subject relative to the MR imaging system. The method (500) includes activating (540) a 3D camera to obtain color, infrared, and depth imaging streams from the 3D camera. The method (500) also includes checking (550) the illumination of the color frame and the noise level of the depth frame obtained by the 3D imaging camera. The method (500) includes checking (560) the histogram values ​​of the color frame to view the illumination. If the illumination level in the scanning room is sufficient for imaging, further image processing (570) can be performed, including color depth alignment, decimation filtering, minimum and maximum distance thresholding, background subtraction, Gaussian smoothing of the depth frame and RGB frame.

[0054] The method (500) also includes identifying (580) the examination bed boundary based on image processing (570) to calculate the corners and midpoint of the examination bed. During the inspection (560), if insufficient lighting levels are observed, the method (500) includes switching the camera (561) to infrared mode and performing Gaussian smoothing on the infrared image frames obtained by the 3D camera. The method (500) also includes smoothing (562) the infrared frames to generate smoothed infrared frames and noise-removed depth frames. These smoothed infrared frames and noise-removed depth frames can be processed to identify (580) the examination bed boundary to calculate the corners and midpoint of the examination bed. Another image quality parameter of the image obtained by the 3D camera includes defining (563) a standard deviation of the depth values. If the standard deviation of the depth frame is less than ten (<10), further image processing (570) can be performed, including color depth alignment, extraction filtering, minimum and maximum distance thresholding, background subtraction, and Gaussian smoothing of the depth frame and RGB frame. However, if the standard deviation of the depth frame has a value greater than ten (>10), the method (500) includes waiting (564) for a subsequent depth frame from the 3D camera having a standard deviation value less than ten (<10) before performing further image processing (570) for identifying the couch boundary and the couch midpoint. Using the identified couch boundary and couch midpoint of the method (500), an artificial intelligence (AI) module may be trained to identify the couch boundary and couch midpoint.

[0055] According to one aspect of the present disclosure, Figure 6A method (600) for using an artificial intelligence (AI)-based deep learning module to perform MR examination table identification, subject orientation, identification of RF coils, and identification of anatomical structures is shown. In one aspect, the method (600) includes employing (610) an AI-based deep learning (DL) module to identify the boundaries of the examination table. The method (600) also includes (620) identifying the orientation of the subject through an AI-based deep neural network model. Examples of the subject's orientation include head first, feet first, supine, prone, left lateral decubitus, and right lateral decubitus. The method (600) also includes using the DL module to identify (630) an imaging coil wrapped around the subject's body. Multiple imaging coils can be wrapped around the subject's body at the same time for imaging, and the method (600) includes using the AI-based deep learning module to identify (630) different coils wrapped around the subject's body. Some non-limiting examples of imaging coils include a head coil, a chest coil, a leg coil, or a whole body coil. When the AI-based deep learning module identifies (630) different coils, method 600 also includes identifying (640) all anatomical structures corresponding to the coils. In one example, if the AI-based deep learning module identifies a particular RF coil as a head coil, the AI-based deep learning module will provide details of the orientation of the head. Method (600) also includes naming (650) the identified anatomical structure and displaying the coordinates f(x,y) of the midpoint of the anatomical structure on an operator console. An operator of the MR imaging system can manually select an imaging coil and position it over the subject's anatomical structure, and method (600) also includes comparing (660) the anatomical structure and orientation of the subject identified by the DL module with the anatomical structure and corresponding coil selected by the operator. The imaging scheme selected by the operator can be input into the computer system via the console for comparison with the imaging scheme preloaded on the AI-based deep learning module. If the imaging protocol selected by the operator does not match the protocol identified by the AI-based deep learning module, the method (600) includes alerting (670) the operator of the mismatch between the protocol and the protocol identified by the AI-based deep learning module, and not allowing the MR imaging system to scan until the anatomical structure location, the orientation of the subject, and the position of the imaging coil match the protocol identified by the AI-based deep learning module. The method (600) also includes scanning (680) the subject when the protocol identified by the AI-based deep learning module matches the protocol selected by the operator.

[0056] According to one aspect of the present disclosure, Figure 7 A method (700) for automated subject anatomy and orientation recognition using a three-dimensional (3D) depth camera using an artificial intelligence (AI) based deep learning module is shown. Figure 7 yes Figure 6A visual representation of the method (600) of FIG. 700 includes a subject registering with an imaging prescription and entering (710) a scanning room for imaging. The method (700) includes selecting (720) an imaging protocol by an operator via a console, and manually checking (730) by the operator whether an anatomical structure selected by the console and its orientation are present on the examination couch. The protocol selected by the operator is input into an AI-based deep learning module. The method (700) also includes capturing (740) the subject orientation and the position of the RF coil over the anatomical structure using a 3D camera. The method (700) also includes processing (750) images captured by the 3D camera for color model (RGB) inspection, depth frame inspection, and infrared frame inspection, and inputting these images into the AI-based deep learning module. The method (700) also includes comparing (760) the imaging protocol input by the operator with the orientation image and coil image received from the 3D camera by the AI-based deep learning module to determine whether the operator has accurately placed the correct type of coil on the anatomical structure specified by the prescription based on the imaging protocol. The AI ​​engine is preloaded with imaging protocols to determine the accuracy of the orientation of the subject and the placement of the coil over the anatomical structure. In one example, when the operator has correctly identified the anatomical structure, but incorrectly identified the orientation and incorrectly placed the coil, the AI ​​module will provide an output showing that the identification of the anatomical structure is correct but the orientation and placement of the coil are incorrect. In another example, if the operator correctly identifies the anatomical structure and orientation of the subject and accurately places the imaging coil, the AI ​​module will provide an output indicating that the identification of the anatomical structure is accurate, and the orientation and placement of the coil are accurate.

[0057] According to one aspect of the present disclosure, Figure 8A method (800) for automated patient anatomy and orientation recognition using a three-dimensional (3D) depth camera using an AI-based deep learning module is shown. The method (800) includes a subject registering and entering (810) an MRI scanning room and lying on an examination couch. The method (800) also includes capturing (820) an image of the subject using the 3D camera to obtain details of the patient anatomy, orientation, and coils used to image the subject. The method (800) also includes processing (830) the image captured by the 3D camera to obtain a color model (RGB), a depth frame, and an infrared frame to identify the couch boundary, patient anatomy, orientation, and RF coils, and inputting these details into the AI ​​module. The method (800) also includes processing (840) the images obtained from the 3D camera and the patient couch by the AI ​​module for AI-based recognition of the anatomy, orientation, and coils to generate a response. The method 800 may also include displaying (850) details of the anatomy, orientation, coils on a console and such other details may be required, such as the midpoint of the anatomy. The AI ​​module determines whether the identification of the anatomy, orientation, and coils matches the imaging plan and displays on the console any discrepancies between the imaging plan and the actual anatomy, subject's orientation, and coil placement.

[0058] A method for MRI examination bed identification will be described in detail. According to one aspect of the present disclosure, FIG. 9(a) shows a method for MRI examination bed identification using a raw point cloud counter extracted from a depth frame. The method (900) includes extracting (910) an examination bed plane from other planes within the depth data. This includes extracting point cloud data corresponding to a camera coordinate system from a depth frame of a 3D camera. FIG. 9(b) shows a method (920) for MRI examination bed identification using a top-down view of a raw point cloud counter extracted from a 3D depth camera. As can be seen from FIG. 9(b), the data points closest to the camera (e.g., a patient examination bed) and the data points farthest from the camera (e.g., the ground) can be identified as different structures and displayed in different colors.

[0059] According to one aspect of the present disclosure, FIG. 9( c ) illustrates a method ( 900 ) that may include MRI couch identification using point cloud data from a floor plane extracted from a depth camera. The method ( 900 ) includes extracting ( 930 ) point cloud data in a first plane (the floor) from the remainder of the point cloud data. Initially, the maximum point-to-plane distance may be set to the distance between a camera mounted on a ceiling and the floor. In other words, the furthest possible distance from a point to a plane (the point is still considered part of the plane) may be the distance between the camera and the couch. The normal vector of the desired plane may be [0,0,1] since the plane may be roughly in the xy plane. The maximum angular distance of the plane may be set to 30°. Using the function pcfitplane() , the point cloud data, the maximum point-to-plane distance, the reference vector, and the maximum angular distance, a floor plane may be created. The floor plane data point may be set to plane 1. pcfitplane() may refer to the [x,y,z] coordinates corresponding to the MRI couch point from which the best fit plane may be defined. The expected format is an array represented by an array ([[x1,y1,z1], ... [xn,yn,zn]]). In 3D Euclidean space, each plane can be represented as (1). Therefore, based on the distribution of points and the plane function, the best fit plane can be calculated by least squares fitting (LSF). LSF uses the concept of minimizing the sum of squares of normal distances from all 3d points to the best plane as (2) and (3) to determine the parameters of the best fit plane.

[0060] ax+by+cz+d=0, where a and b are the normals of the plane (1)

[0061] Where P i represents the normal distance from the i-th point on the plane,

[0062]

[0063] The point cloud plane fitting method is used to calculate the XYZ plane of the MRI bed point cloud, and the plane is fitted to the appropriate point cloud axis using LS. Point cloud plane fitting ensures that the bed points are grouped, the patient can be identified within the fitted plane, and the bed points are clearly delineated to the exact plane.

[0064] According to one aspect of the present disclosure, FIG9(d) illustrates a method (900) that may include MRI couch identification using point cloud data with a floor plane extracted from a depth camera. The method (900) includes displaying (940) the remaining point cloud data after the floor is removed. Due to noise and maximum point-to-plane distance values, some points that should be considered part of the floor plane are not shown. Furthermore, extracting the couch plane from the data may not require extracting the floor plane.

[0065] According to one aspect of the present disclosure, FIG. 9( e) illustrates a method (900) for MRI couch identification using point cloud data in a couch plane using a depth camera. The method (900) includes extracting (950) point cloud data in a second plane (the couch) from the remaining point cloud data. To extract the couch, a region of interest may be used to narrow the "z" position of the couch. The region of interest takes into account any point that may be within a distance of two meters or closer to the camera. The function pcfitplane() may again be used with the same maximum point-to-plane distance, but in combination with the remaining point cloud data and the index of the point within the region of interest. The couch plane data points are set to the plane 2 or normal plane of the point cloud. The second plane is used to calculate the four corners of the couch boundary and their midpoints. All subsequent intelligent processing, such as coil identification, patient identification, orientation, and anatomical structure identification based on an AI-based deep neural network, is performed within the four corners of the couch boundary. Whenever the depth is noisy and the point cloud is not accurately extracted, the AI-based shape detection neural network identifies the couch boundary and outputs the four corners of the couch boundary. Couch boundary extraction is applicable to both docked MRI patient couches and undocked MRI patient couches.

[0066] According to one aspect of the present disclosure, FIG. 10( a) shows an artificial intelligence (AI)-based deep learning module for identifying a patient. Before an imaging coil is placed on the subject's body, it is necessary to identify whether the subject lies within the boundary of the examination bed. The boundary of the examination bed and the four corner points of the examination bed can be determined according to the method (900) described with reference to FIG. 9( a) to FIG. 9( e). A 3D camera can be used to obtain an image of the subject, and an AI-based subject key point algorithm can be used to determine whether the patient lies within the four corner points of the bed boundary using point cloud computing. In addition, FIG. 10( a) shows the architecture of an AI-based deep learning module for region-based recognition of anatomical regions and coils. The input image from the camera, which is a combination of a color depth frame and an RGB color frame, is fed to a region proposal convolutional neural network (RP-CNN) (1010), which identifies the patient, the patient mask, the mask of the coil, and the anatomical region of the patient. Once the mask region is appropriately identified, features within the mask are extracted using region-based pooling and region of interest alignment methods. Multiple possible regions where objects of interest may exist may be queried, and bounding boxes may be filled into each of these possible regions. These possible regions are then passed to a convolutional neural network. The backbone of the convolutional neural network may be a residual network (ResNet) (1020), which is responsible for determining the categories of possible object regions and then highlighting the regions of each category with a mask of the boundary region of interest. Whenever the residual network fails to recognize certain complex features, another backbone called a feature pyramid network (FPN) (1030) is called to align the region of interest. After alignment, the region proposal network (1010) generates a mask region of possible regions where objects of interest may exist.

[0067] According to one aspect of the present disclosure, FIG. 10( b) shows an artificial intelligence (AI)-based deep learning module for identifying patients. FIG. 10( b) uses the entire AI module architecture of FIG. 10( a) as a processing block. The color depth input and the depth frame (1040) are aligned to extract the contours of the patient in the occluded state and the non-occluded state. According to one aspect of the present disclosure, as shown in FIG. 10( b), the AI-based deep learning module is used to identify whether the patient is present in the occluded coil. In order to perform patient identification, the network of FIG. 10( b) calculates possible anatomical region coordinates, and these coordinates can be further fed to the region proposal network (1050) in a fixed batch size. Each batch can consist of approximately fifty region-based proposal masks / patient shapes. Feature maps (1060) can be extracted from the depth information of possible patient masks and then fed to multiple cascaded multi-convolutional neural network blocks (1070) for identification. Once the region is confidently identified, the presence of a patient in the region may be verified via a feature descriptor based vector (1080) which calculates the average thickness of the possible patient region and gives a final confidence score for the presence of the patient.

[0068] The acquisition, processing, analysis, and storage of medical image data play an important role in the diagnosis and treatment of patients in a healthcare setting. Medical imaging workflows and the devices involved in the workflows can be configured, monitored, and updated throughout the operation of the medical imaging workflows and devices. Machine learning can be used to help configure, monitor, and update medical imaging workflows and devices.

[0069] For example, machine learning techniques (whether deep learning networks or other experience / observation learning systems) can be used to locate objects in images, understand speech and convert speech to text, and improve the relevance of search engine results. Deep learning is a subset of machine learning that uses a set of algorithms to model high-level abstractions in data using deep graphs with multiple processing layers (including linear and nonlinear transformations). While many machine learning systems start with initial features and / or network weights that are then modified through learning and updating of the machine learning network, deep learning networks train themselves to identify "good" features for analysis. When using a multi-layer architecture, machines using deep learning techniques can process raw data better than machines using conventional machine learning techniques. The use of different layers of evaluation or abstraction facilitates the examination of data for groups of highly relevant values ​​or distinctive themes.

[0070] In some examples, deep learning and / or other machine learning networks may be configured to determine an image acquisition prescription or otherwise form a data structure to instantiate parameters / settings for image acquisition based on a desired image quality (IQ). In some examples, multiple deep learning networks are used to generate an image acquisition prescription. For example, a first network (e.g., a regression network, etc.) calculates image acquisition prescription parameters based on a desired IQ and a selected (e.g., selected by a user, specified by a program, imaging protocol, and / or other system, etc.) scan time to form an image acquisition "prescription" or configuration data structure. For example, a regression network is lighter weighted and less computationally intensive than a deep learning network.

[0071] An image preview of an image having an IQ that is closest to an image acquisition prescription (e.g., calculated by a first network) may be generated. A second network (e.g., a deep learning network, etc.) periodically learns an IQ metric from the acquired site images and updates the first network to improve the generation of corresponding image acquisition parameters of the first network. The term "convolutional neural network" or "CNN" is a biologically inspired network of interconnected data used to detect, segment, and recognize related objects and regions in a data set in deep learning. CNN evaluates raw data in the form of multiple arrays, divides the data into a series of levels, and examines learned features in the data.

[0072] According to one aspect of the present disclosure, Fig.11 A method (1100) is shown for determining (1110) key points (1111) defining anatomical points of a subject prior to placement of an RF coil. Once the RF coil is mounted on the subject's body, the subject's anatomical structure may not be visible to the operator or camera. In such cases, the ability of the AI ​​module to automatically determine accurate placement of the coil over the appropriate anatomical structure may be hampered. This problem may be addressed by defining key anatomical points (1111) corresponding to various organs of the patient prior to placement of the RF coil. Once the key anatomical points (1111) are marked, grid lines (1112) may be used to connect these key points (1111) and generate an image (1113) of the subject's torso for anatomical structure identification. In determining the accurate positioning of the RF coil of an MRI system over the subject's anatomical structure for imaging, identifying key anatomical points and orientation of the patient's region may depend on areas obscured by blankets, hospital gowns, or any type of covering over the subject's body. Various types of RF imaging coils can be obscured by blankets, patient gowns, and covers, and identifying the RF coils is important for accurate positioning and automated landmarks during MRI examinations.

[0073] A torso image (1113) of a subject defines the anatomical shape, its dimensions, and identifies the location of organs to determine accurate placement of an RF coil over the anatomical structure. Generating the torso image (1113) includes identifying key points (1111) on the subject corresponding to various organs and anatomical regions, and connecting the key points (1111) to generate a torso image (1113) of the subject. When an imaging coil, such as an RF coil, is wrapped around the subject's body for imaging, the subject's organs may not be present in the field of view (FoV) of the camera used for imaging. In order to determine accurate placement of the imaging coil over the surface of the subject's body, the torso image (1113) can be used as a mimic of the subject's body, and the coordinates of the RF coil can be mapped to the location of the subject's organs on the torso image (1113).

[0074] According to one aspect of the present disclosure, Fig.12 AI-based orientation identification of a subject using shape and thickness is shown. The subject can be placed on the examination bed in various orientations based on the imaging requirements prescribed for the subject. Various subject placements on the examination bed include, for example, head-first supine (1210), left lateral decubitus (1220), right lateral decubitus (1230), and head-first prone (1240). Determining the accurate placement of the imaging coil over the anatomical region of the subject may require identification of the subject's orientation. Once the examination bed boundary identification and key point-based patient identification are performed, patient orientation identification can be performed. The AI ​​module can be configured to determine the orientation of the subject and generate an image of the subject on the console to identify the orientation of the subject. The AI ​​module can also be configured to identify a mismatch between the orientation prescribed by the subject's prescription and the actual orientation of the subject obtained by using a 3D imaging camera. When any mismatch between the orientation prescribed by the subject's prescription and the actual orientation is observed, the AI ​​module can be further configured to prescribe the necessary changes in the subject's orientation to the operator until the orientation prescribed by the subject's prescription is achieved.

[0075] According to one aspect of the present disclosure, Fig.13 AI-based identification of RF coils is shown. Figure 2 The RF coil can be used to image the subject, and imaging the subject using the RF coil (1310) includes wrapping the RF coil (1310) around the subject's body (1320) and imaging the subject. Some non-limiting examples of the imaging coil 1310 include a head coil (1311), a chest coil (1312), and a leg coil (1313). In one aspect of the present disclosure, a focus loss-based class-imbalanced object recognition network can be used to identify various RF coils wrapped around the subject's body. According to one aspect of the present disclosure, Fig.14A class-imbalanced object recognition network based on focal loss is shown that can be used to identify an RF coil wrapped around a subject's body. The focal loss function can be used as a discriminator to identify the RF coil. Fig.14 In the example function shown, the parameter pt is the symbol for the probability of the class with the value 1 as the label. If the variable y represents the reference truth value and is 0 or 1, then the probability of p∈[0,1] is the probability estimated by the model for the reference truth or actual class y=1. For symbolic simplicity, the variable pt is defined as:

[0076]

[0077] Focus loss can be calculated as follows: FL(pt) = -(1-pt) γ log(pt).

[0078] Adjustment factor (1-pt) γ It is therefore added to the original cross entropy formula. When the instance passed to the model is misclassified and pt is small, the adjustment factor is almost 1 and therefore does not affect the loss. As pt tends to 1, the adjustment factor tends to 0, so again the loss is not affected. Therefore, all well-classified examples have lower weights. This downward weighting of various examples can be smoothly controlled by the adjustment factor. The RF coil identified by the AI ​​can be presented to the operator for review along with its orientation.

[0079] According to another aspect of the present disclosure, Fig.14 RF coil identification using an AI-based deep learning network is shown, which can be a class-imbalanced object recognition network using a focal loss discriminator function. In one example, an input processed color and depth camera image of a subject or an input processed infrared image and depth from a depth camera can be input to the AI-based deep learning network. The AI ​​network can be configured for automated AI-based RF coil, anatomical structure and its corresponding orientation / view identification, which can calculate the spatial position, coordinates, midpoint of the identified anatomical region (2D), and 3D coordinates of the anatomical structure relative to the subject coordinates f(x, y, z), where Z is the thickness of the identified anatomical region.

[0080] Once the patient region has been identified and the presence of the patient has been confirmed by 10(b), the next step can be to identify the coils placed on or around the patient's body. In addition to the type, the depth map (1410) can also be used to calculate the spatial position of the coil and the depth pattern of the coil. In these depth maps (1410), the features of the coil are extracted, and the logical operations for identifying the patient are also applied here. RoI pooling, region proposal (1420) and depth-based feature extraction (1430) can be used to process the input depth map (1410). Additional processing that can be used is to combine the information from the feature descriptors (1440) of the coil with the information of the patient thickness and iteratively sum the maximum mean vector of the rectified linear unit. The iterative sum contains the coordinates of the identified coil. This information is fed to a discriminator (1450) based on focal loss to remove false and misidentified coil regions and retain only the true coil position. Once the true coordinates, spatial position and midpoint of the coil are calculated, the next step is to classify the type of coil and the orientation of the patient with the coil.

[0081] According to one aspect of the present disclosure, Fig.15 AI-based identification of a subject wrapped with an RF coil is shown. AI-based identification of a subject with a coil can be performed using a method similar to the method described with respect to Figures (9) and (10). An image of the subject obtained using a 3D camera together with the RF coil can be input into an AI-based deep learning module to identify the type of coil and the accurate positioning of the RF coil over the subject's anatomy. In addition, the coil type and the orientation of the patient with the coil can be classified. The classification categories refer to coil type categories such as head coil, neck coil, body coil, limb coil, and the orientation of the patient such as supine or prone.

[0082] This classification of coils can be performed by a vgg 19 neural network (1510). Vgg 19 is a convolutional neural network. The input to the cov1 layer is an RGB image of fixed size. The image is passed through a stack of convolutional (conv.) layers, where filters can be used with very small receptive fields (3×3) (this is the minimum size to capture the concept of left / right, up / down, center). In one of the configurations, the network also utilizes 1×1 convolutional filters, which can be viewed as a linear transformation of the input channels (followed by a nonlinearity). The convolution stride is fixed to 1 pixel; spatial padding of the convolution layer input allows the spatial resolution to be maintained after the convolution. Spatial pooling can be performed by five max pooling layers, which are after some of the conv. layers (not all conv. layers are after max pooling). Max pooling is performed on a 2×2 pixel window with a stride of 2. Three fully connected (FC) layers follow the stack of convolutional layers: the first two layers may have 4096 channels each, the third layer performs 1000-way ILSVRC classification and therefore contains 1000 channels (one channel for each class). The final layer is a softmax layer. The configuration of the fully connected layers is the same in all networks.

[0083] Classification of patient orientation may be performed by another convolutional neural network called inception v3 (1520). Inception v3 (1520) is a convolutional neural network that may be forty-eight layers deep. The network may contain a suitable algorithm that facilitates label smoothing, such as RMSProp. This type of regularization component is added to the loss formula, which prevents the network from becoming too confident about the patient's orientation and prevents overfitting. In addition, since patient orientation needs to be classified in addition to the coil, a factorized convolution may be performed to reduce the computational burden because it reduces the number of parameters involved in the network. It also monitors network efficiency.

[0084] According to one aspect of the present disclosure, Fig.16 AI-based identification of anatomical structures corresponding to RF coils is shown. Identifying anatomical structures corresponding to the RF coils includes identifying key anatomical points on the subject's body corresponding to various organs and anatomical regions of the subject's body, and connecting the key anatomical points using grid lines to generate a torso image of the subject's body. The key points indicate various organs of the subject's body. The 3D camera may be configured to capture the position of the RF coil above the subject's body, and the AI ​​module may be configured to identify the position of the RF coil above the key points identified by the AI ​​module on the subject's body. According to another aspect of the present disclosure, the AI ​​module may be configured to identify all key points located below the RF coil using a key point detection algorithm, and verify that the RF coil is appropriate for the anatomical region.

[0085] Based on the above system and method, according to another aspect of the present disclosure, Fig.16 A method (1600) for automated anatomical structure and orientation recognition of a subject using an artificial intelligence-based deep learning module during occlusion of an imaging coil from a magnetic resonance imaging (MRI) system is shown. The method (1600) includes obtaining (1610) multiple depth images, color images, and infrared images of the subject using a three-dimensional (3D) depth camera. The method (1600) also includes using the images acquired by the 3D camera to identify (1620) a couch boundary of the MRI system. The method (1600) also includes identifying (1630) the position of the subject above the couch to determine whether the subject is positioned within the couch boundary of the MRI system, and identifying multiple key anatomical points corresponding to multiple organs of the subject's body. The method (1600) also includes identifying (1640) the orientation of the subject above the couch of the MRI system. The method (1600) also includes identifying (1650) coils of the MRI system wrapped around the subject's body, and determining the orientation of the subject relative to the coils of the MRI system. The method (1600) also includes identifying (1660) the anatomical key points obscured by the coils of the MRI system to determine accurate positioning of the coils of the MRI system over the subject's anatomy for imaging. The method (1600) may include identifying all Digital Imaging and Communications in Medicine (DICOM) orientations and anatomical regions of the subject when the subject is obscured by a hospital gown or blanket. The method also includes identifying all types of RF coils wrapped around the subject when the subject is obscured by a hospital gown or blanket.

[0086] These and other features of the present disclosure provide automated AI-based detection of RF coils and corresponding orientation / view identification of RF coils based on color depth or infrared frames and depth frames using the shape and texture of the image. In addition, the systems and methods of the present disclosure provide automated AI-based RF coils, patient anatomical structures and their corresponding patient orientation / view identification, which will overlay the shapes on the color frame and infrared frame of the anatomical structure identification area, the patient video thumbnail on the console screen. The systems and methods of the present disclosure provide automated AI-based RF coils, anatomical structures and their corresponding orientation / view identification under various scanner room lighting conditions (such as bright, dark, semi-bright and semi-dark conditions). The systems and methods of the present disclosure provide automated AI-based RF coil, anatomical structure and their corresponding orientation / view identification, which overlays the orientation identification information as a string on the color frame and infrared frame of the patient video thumbnail on the console screen, overlaying all anatomical structures seen in the camera's field of view and can be used to calculate the spatial position, coordinates of the identified anatomical region (2D), the midpoint of the identified anatomical region (2D), and the 3D coordinates of the anatomical structure relative to the patient coordinates f(x,y,z) (where Z is the thickness of the identified anatomical region).

[0087] This written description uses examples to disclose the invention, including the best mode, and to enable those skilled in the art to practice the invention, including making and using any computing system or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method for automated anatomical structure and orientation recognition of a subject using an artificial intelligence (AI) based deep learning module, the method comprising: Positioning a subject above an examination couch of a magnetic resonance imaging (MRI) system and wrapping at least one radio frequency (RF) imaging coil above the subject; acquiring a plurality of depth images, color images, and infrared images of the subject using a three-dimensional (3D) depth camera while the subject's body is occluded by the at least one radio frequency (RF) imaging coil; identifying a couch boundary of the MRI system using the image acquired by the 3D depth camera; identifying a position of the subject in physical space above a couch of the MRI system to determine whether the subject is positioned within the couch boundaries of the MRI system, and identifying a plurality of key anatomical points and regions corresponding to a plurality of organs of the subject's body; identifying an orientation of the subject above the examination couch of the MRI system; identifying a type of the at least one RF imaging coil of the MRI system wrapped around the body of the subject, and determining an orientation of the subject relative to the at least one RF imaging coil of the MRI system; as well as The plurality of key anatomical points and regions obscured by the at least one RF imaging coil of the MRI system are identified to determine accurate positioning of the at least one RF imaging coil of the MRI system over the anatomical structure of the subject for imaging and performing automated landmarks of the anatomical structure of the subject.

2. The method according to claim 1, wherein obtaining the plurality of images by the 3D depth camera comprises: A health check is performed on the 3D depth camera for any color model, imaging depth, and infrared flow, and the 3D depth camera is calibrated for imaging.

3. The method of claim 1 , wherein identifying the couch boundary of the MRI system comprises: Change the 3D camera coordinate system to the patient coordinate system; generating a patient physical coordinate system corresponding to the MRI system; activating color, infrared, and depth streams of the 3D depth camera to generate a color frame, an infrared frame, and a depth frame of the subject; checking the illumination level of the color frame, the noise level of the depth frame, and performing color-depth alignment; identification of a blanket in the color frame, the infrared frame, and the depth frame; decimation filtering; distance thresholding; background subtraction and Gaussian smoothing when the color frame and the depth frame are illuminated; and switching to the infrared frame to perform Gaussian smoothing of the infrared frame when the illumination of the color frame and the depth frame is low, thereby identifying the examination bed boundary of the examination bed of the MRI system.

4. The method according to claim 3, further comprising: extracting point cloud data of the examination bed of the MRI system corresponding to the 3D depth camera coordinate system from the depth frame of the 3D camera; Obtaining an original point cloud of the examination bed of the MRI system; extracting the point cloud data in a first plane, a floor plane, and from the rest of the point cloud data; obtaining remaining point cloud data without the floor plane; extracting the point cloud data in a second plane of the MRI system, the couch plane, from the remaining point cloud data; as well as using the second plane to calculate four corners and a midpoint of the examination couch of the MRI system; Wherein if the depth image is noisy and the 3D depth camera cannot extract the point cloud data, the AI-based deep learning module identifies the examination bed boundary and uses a shape detection neural network to output the four corners of the examination bed boundary.

5. The method of claim 1 , wherein identifying the position of the subject above the couch of the MRI system to determine whether the subject is positioned within the couch boundaries of the MRI system comprises: The image obtained by the 3D depth camera is processed by the AI-based deep learning module to identify the MRI couch boundary and to identify whether the subject is located within the MRI couch boundary.

6. The method of claim 1 , wherein identifying the plurality of key anatomical points corresponding to the plurality of organs of the subject's body comprises: identifying a plurality of organs on the subject's body, marking the organs with points, and connecting the points by a plurality of grid lines to generate a torso image of the subject for automated landmarking, and identifying the plurality of key anatomical points and the orientation of the subject further comprises: identifying areas obscured by blankets, hospital gowns, and covers to determine accurate positioning of the at least one RF imaging coil of the MRI system over the subject's anatomical structure for imaging.

7. The method of claim 6, wherein identifying the plurality of key anatomical points corresponding to the plurality of organs of the subject's body is performed using the AI-based deep learning module before wrapping the at least one RF imaging coil around the subject's body.

8. The method of claim 1 , wherein identifying the orientation of the subject above the couch of the MRI system comprises: The image acquired by the 3D depth camera is processed by the AI-based deep learning module to identify the direction or the orientation in which the subject has been lying on the MRI examination bed.

9. The method of claim 1 , wherein identifying the type of the at least one RF imaging coil of the MRI system wrapped around the subject's body comprises: A focal loss based class imbalanced object recognition network is employed to recognize the type of the at least one RF imaging coil wrapped around the subject's body with and without a patient gown and blanket.

10. The method of claim 1, wherein identifying the key anatomical points and regions occluded by the at least one RF imaging coil of the MRI system comprises: Identifying the key anatomical points and regions obscured by the at least one RF imaging coil of the type of the MRI system, and using the AI-based deep learning module to identify whether the at least one RF imaging coil is accurately placed above a corresponding subject organ based on an imaging scheme preloaded on the AI-based deep learning module.

11. The method of claim 10, wherein identifying the accurate placement of the at least one RF imaging coil over the subject organ comprises: identifying the at least one RF imaging coil of the MRI system wrapped around a body of the subject; identifying the at least one RF imaging coil wrapped around the subject's body obscured by the blanket; identifying the anatomical structure selected by an operator to correspond to the at least one wound RF imaging coil of the MRI system; identifying the anatomical structure selected by an operator corresponding to the at least one wound RF imaging coil of the MRI system and obscured by the blanket; as well as The identified anatomical structure corresponding to the at least one RF imaging coil of the MRI system is compared with the anatomical structure selected by the operator using the AI-based deep learning module.

12. The method of claim 11, further comprising alerting the operator of the MRI system if the anatomical structure identified by the AI-based deep learning module does not match the anatomical structure selected by the operator.

13. The method of claim 11, further comprising performing MR imaging if the anatomical structure identified by the AI-based deep learning module matches the anatomical structure selected by the operator.

14. The method according to claim 1, further comprising: Inputting an operator-selected imaging protocol through an operator console and preloading the imaging protocol onto the AI-based deep learning module; obtaining the subject image using the 3D depth camera and inputting the subject image into the AI-based deep learning module; comparing the image obtained by the 3D depth camera with a protocol selected by the operator using the AI-based deep learning module to determine whether the subject position and the orientation match the imaging protocol; obtaining landmark locations relative to the identified anatomical structure and the at least one RF imaging coil location, and The operator is alerted to change the subject position and coil settings until the subject position and the orientation match the imaging protocol preloaded onto the AI-based deep learning module.

15. The method of claim 1, further comprising using the AI-based deep learning module to automatically identify an appropriate imaging protocol based on the identified at least one imaging RF imaging coil of the MRI system and the underlying anatomy of the subject.

16. The method of claim 1, further comprising automatically populating a list of the at least one RF imaging coil of the MRI system, anatomical structures, and corresponding anatomical structure midpoints for automated landmarks seen within the couch boundaries.

17. A system for automated anatomical structure and orientation recognition of a subject using an artificial intelligence (AI) based deep learning module during occlusion of at least one RF imaging coil from a magnetic resonance imaging (MRI) system, the system comprising: a three-dimensional (3D) depth camera configured to capture a plurality of depth images, color images, and infrared images of the subject positioned on an examination couch of the magnetic resonance imaging (MRI) system; a computer system connected to the 3D depth camera and configured to receive the image from the 3D depth camera, wherein the computer system comprises: processor; a memory connected to the processor; At least one artificial intelligence (AI) based deep learning module, wherein the at least one AI based deep learning module is deployed above the memory, wherein the AI ​​based deep learning module is configured to: identifying a couch boundary of the MRI system using the image acquired by the 3D depth camera; identifying a physical position of the subject above the couch to determine whether the subject is positioned within the couch boundaries of the MRI system, and identifying a plurality of key anatomical points corresponding to a plurality of organs of the subject's body; identifying a plurality of DICOM orientations of the subject on the examination couch of the MRI system; identifying the at least one RF imaging coil wrapped around the body of the subject and determining the orientation of the subject relative to the at least one RF imaging coil; and The key anatomical points obscured by the at least one RF coil are identified to determine an accurate position of the MRI coil over the subject's anatomy for imaging.

18. The system of claim 17, wherein the 3D depth camera is ceiling mountable, wall mountable, bracket mountable, scanner mountable, or landscape mountable.

19. The system of claim 17, wherein the orientation of the subject includes head-first, feet-first, supine, prone, left lateral decubitus, right lateral decubitus, and other DICOM positions.

20. The system of claim 17, wherein the at least one RF coil of the MRI system comprises a radio frequency (RF) head coil, a chest coil, a leg coil, a whole body coil, a limb coil, an abdominal coil, and an MRI volume coil.

21. The system of claim 17, wherein the computer system is configured to generate an alert if a mismatch is detected between an imaging plan recommended by the AI-based deep learning module and a plurality of imaging parameters set by an operator and identified through 3D depth camera frames.

22. The system of claim 21, wherein the imaging parameters include coil type, subject orientation, subject anatomy, and subject position above the couch of the MRI system.

23. The system of claim 22, wherein the location of the subject anatomy with the identified at least one RF imaging coil is used for automated landmarking.

Images