Systems and methods for deep learning based shoulder pathology measurement

Abstract

A method for shoulder pathology measurement includes obtaining three-dimensional (3D) medical imaging data of a shoulder of a subject. The method also includes localizing a glenohumeral joint in the 3D medical imaging data of the shoulder to a local view with a cylinder segmentation using a first trained neural network (180). The method also includes detecting a plane containing a circle surrounding a glenoid within the local view, predicting a ring segmentation mask from the local view with a circle / ring segmentation using a second trained neural network (184), and predicting a glenoid surface segmentation mask from the local view. The method even further includes calculating a lower joint diameter, a width of a bone pathology, and a glenoid track width based on the ring segmentation mask and the glenoid surface segmentation mask using a defect algorithm (270).

Description

Background Technology

[0001] The topics disclosed in this article relate to medical imaging, and more specifically to a system and method for measuring shoulder lesions based on deep learning.

[0002] Non-invasive imaging techniques allow for the acquisition of images of a patient's or subject's internal structures or features without the need for invasive procedures. Specifically, such non-invasive imaging techniques rely on various physical principles (such as differential transmission of X-rays through a target volume, sound wave reflection within the volume, paramagnetism of different tissues and materials within the volume, and the disintegration of the target radionuclide within the body) to acquire data and construct images or otherwise represent the observed internal features of a patient or subject.

[0003] During MRI, when material such as human tissue is subjected to a uniform magnetic field (polarization field B0), the individual magnetic moments of the spins within the tissue attempt to align with that polarization field, but precess around it in a random order at their characteristic Larmor frequencies. If the material or tissue is subjected to a magnetic field (excitation field B1) located in the xy-plane and close to the Larmor frequency, the net alignment torque, or "longitudinal magnetization," M... z It can be rotated or "tilted" into the xy plane to produce a net transverse magnetic moment M. t After the excitation signal B1 is terminated, a signal is emitted by the excitation spin, and this signal can be received and processed to form an image.

[0004] When these signals are used to generate images, the magnetic field gradient (G) is employed. x G y and G z Typically, the area to be imaged is scanned in a series of measurement cycles, during which these gradient fields vary depending on the specific localization method used. The resulting set of received nuclear magnetic resonance (NMR) signals is digitized and processed to reconstruct the image using one of many well-known reconstruction techniques.

[0005] The most common form of shoulder instability is anterior shoulder dislocation. Bone lesions anterior and inferior to the glenoid fossa (called Bankart lesions) and bone lesions posterolateral to the humeral head (called Hill-Sachs lesions) are closely associated with anterior shoulder dislocation, and their severity is closely related to recurrence. The severity of these lesions is used to prescribe the correct surgical approach for arthroscopic Bankart repair, Latarjet surgery, filler, and / or lateral humeral repair. These surgical procedures vary in their level of invasiveness and complexity. However, assessing Bankart and Hill-Sachs lesions can be time-consuming. Summary of the Invention

[0006] The following provides an overview of some of the embodiments disclosed herein. It should be understood that these aspects are provided merely to give the reader a brief overview of these specific embodiments, and are not intended to limit the scope of this disclosure. In fact, this disclosure may cover various aspects that may not be set forth below.

[0007] In one embodiment, a computer-implemented method for measuring shoulder lesions is provided. The computer-implemented method includes acquiring three-dimensional (3D) medical imaging data of a subject's shoulder via a processing system including one or more processors. The computer-implemented method further includes localizing the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view via cylindrical segmentation using a first trained neural network through the processing system. The computer-implemented method further includes detecting planes containing a circle surrounding the glenoid within the local view via a second trained neural network through the processing system, predicting a ring segmentation mask from the local view using circle / ring segmentation, and predicting a glenoid surface segmentation mask from the local view. The computer-implemented method further includes calculating the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width based on the ring segmentation mask and the glenoid surface segmentation mask via a defect algorithm through the processing system.

[0008] In another embodiment, a system for measuring shoulder lesions is provided. The system includes a memory encoding processor-executable routines. The system also includes a processing system comprising one or more processors configured to access the memory and execute the processor-executable routines, wherein the processor-executable routines cause the processing system to perform actions when executed by the processing system. These actions include acquiring three-dimensional (3D) medical imaging data of a subject's shoulder. The computer-implemented method further includes using a first trained neural network to localize the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view using cylindrical segmentation. The computer-implemented method also includes using a second trained neural network to detect planes containing a circle surrounding the glenoid within the local view, using circle / ring segmentation to predict a ring segmentation mask from the local view, and predicting a glenoid surface segmentation mask from the local view. The computer-implemented method even includes using a defect algorithm to calculate the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width based on the ring segmentation mask and the glenoid surface segmentation mask.

[0009] In another embodiment, a non-transitory computer-readable medium is provided, comprising processor-executable code that, when executed by a processing system comprising one or more processors, causes the processing system to perform actions. These actions include acquiring three-dimensional (3D) medical imaging data of a subject's shoulder. The computer-implemented method further includes using a first trained neural network to localize the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view using cylindrical segmentation. The computer-implemented method also includes using a second trained neural network to detect planes containing a circle enclosing the glenoid cavity within the local view, using circle / ring segmentation to predict a ring segmentation mask from the local view, and predicting a glenoid surface segmentation mask from the local view. The computer-implemented method even includes using a defect algorithm to calculate the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width based on the ring segmentation mask and the glenoid surface segmentation mask. Attached Figure Description

[0010] These and other features, aspects, and advantages of the invention will be better understood when reading the following detailed description with reference to the accompanying drawings, in which the same reference numerals denote the same parts throughout the drawings, wherein:

[0011] Figure 1 Magnetic resonance (MR) images of Bankart lesions according to various aspects of this disclosure and measures for measuring the severity of Bankart lesions are depicted;

[0012] Figure 2 Embodiments of magnetic resonance imaging (MRI) systems suitable for use with the disclosed techniques according to various aspects of this disclosure are illustrated;

[0013] Figure 3 The structure of a first trained neural network (e.g., an overlay network) utilizing the disclosed techniques according to various aspects of this disclosure is illustrated.

[0014] Figure 4 The structure of a second trained neural network (e.g., a scan plane network) utilizing the disclosed technology according to various aspects of this disclosure is illustrated.

[0015] Figure 5 Distance diagrams of annular masks according to various aspects of this disclosure are depicted;

[0016] Figure 6 A flowchart illustrating a method for measuring shoulder lesions according to various aspects of this disclosure is shown;

[0017] Figure 7 A schematic diagram illustrating a process for measuring shoulder lesions according to various aspects of this disclosure is provided.

[0018] Figure 8 A schematic diagram illustrating a process for exploiting a defective algorithm according to various aspects of this disclosure is provided.

[0019] Figure 9 A table depicts the quantitative results of the comparison between the disclosed technology and ground truth, based on a summary of various aspects of this disclosure;

[0020] Figure 10 A first qualitative example is described, comparing segmentation using the disclosed techniques with ground truth segmentation according to various aspects of this disclosure;

[0021] Figure 11 A frontal view image of the glenoid cavity, representing the output according to various aspects of this disclosure, is depicted, having a predicted loop derived from segmentation using the disclosed technique in a first qualitative example.

[0022] Figure 12 A second qualitative example is described, comparing segmentation using the disclosed techniques with ground truth segmentation according to various aspects of this disclosure;

[0023] Figure 13 A frontal view image of the glenoid cavity, depicting the output according to various aspects of this disclosure, has a predicted loop derived from segmentation using the disclosed technique in a second qualitative example.

[0024] Figure 14 A third qualitative example is described, comparing segmentation using the disclosed technique with ground truth segmentation according to aspects of this disclosure;

[0025] Figure 15 A frontal view image of the glenoid cavity, representing the output according to various aspects of this disclosure, is depicted, having a predicted loop derived from segmentation using the disclosed technique in a third qualitative example.

[0026] Figure 16 A frontal view image of the output glenoid cavity is depicted, which has a predicted loop derived from segmentation using the disclosed technique; and

[0027] Figure 17 Another output image of the glenoid cavity is depicted, which has a predicted loop derived from segmentation using the disclosed technique. Detailed Implementation

[0028] One or more specific implementations will be described below. To provide a concise description of these implementations, not all features of the actual implementation will be described in this specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, many implementation-specific decisions must be made to achieve the developer's specific objectives, such as complying with system-related and business-related constraints that may differ from implementation to implementation. Furthermore, it should be understood that such development efforts may be complex and time-consuming, but will in any case remain routine tasks of design, fabrication, and manufacturing for those skilled in the art who benefit from this disclosure.

[0029] When describing elements of various embodiments of the subject matter of this invention, the articles “a,” “an,” “the,” and “described” are intended to indicate the presence of one or more elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to the listed elements. Furthermore, any numerical examples in the following discussion are intended to be non-limiting, and therefore the additional values, ranges, and percentages are within the scope of the disclosed embodiments.

[0030] While the various aspects of the following discussion are presented in the context of medical imaging, it should be understood that the disclosed techniques are not limited to this medical context. In fact, the examples and explanations provided in this medical context are merely for the purpose of facilitating explanation by providing examples of real-world implementations and applications. However, the disclosed techniques can also be used in other contexts, such as image reconstruction for non-destructive inspection of manufactured parts or goods (i.e., quality control or quality inspection applications) and / or non-invasive inspection of packages, boxes, suitcases, etc. (i.e., security or screening applications). Generally, the disclosed techniques can be used in any imaging or screening context or in image processing or photography field, where a set or class of acquired data undergoes a reconstruction process to generate an image or volume.

[0031] The deep learning (DL) methods discussed in this paper can be based on artificial neural networks and therefore may encompass one or more of the following: deep neural networks, fully interconnected networks, convolutional neural networks (CNNs), unfolded neural networks, perceptrons, encoders / decoders, recurrent networks, wavelet filter banks, u-nets, generative adversarial networks (GANs), dense neural networks, or other neural network architectures. Neural networks may include shortcuts, activations, batch normalization layers, and / or other features. These techniques are referred to as DL techniques in this paper, although the term may also be used specifically with reference to the use of deep neural networks, which are neural networks with multiple layers.

[0032] As discussed in this paper, DL techniques (also known as deep machine learning, hierarchical learning, or deep structured learning) are a branch of machine learning techniques that employ mathematical representations of data and artificial neural networks used to learn and process such representations. For example, DL methods can be characterized as using one or more algorithms to extract or model a class of highly abstract concepts from data of interest. This can be accomplished using one or more processing layers, where each layer typically corresponds to a different level of abstraction and thus may take or utilize different aspects of the initial data or the output of the previous layer (i.e., the hierarchical or cascaded structure of the layers) as the target of the process or algorithm for a given layer. In the context of image processing or reconstruction, this can be characterized as different layers corresponding to different feature levels or resolutions in the data. Generally, the processing from one representation space to the next level of representation space can be viewed as a “stage” of a process. Each stage of the process can be performed by a single neural network or by different parts of a larger neural network.

[0033] In the following disclosure, techniques are discussed using three-dimensional (3D) MRI data as example image data. These techniques can also be used in conjunction with computed tomography imaging volumes. These techniques are also discussed for bone lesions anterior and inferior to the glenoid fossa. The techniques can also be used for bone lesions posterolateral to the humeral head. The techniques can also be used for other musculoskeletal joints.

[0034] This disclosure provides systems and methods for measuring shoulder lesions. Specifically, a deep learning-based pipeline is used to automate Bankart lesion measurements given a 3D medical imaging volume (MR imaging volume acquired using bone-specific sequences such as oZTEo), thereby improving the surgical diagnostic workflow for shoulder instability. The disclosed systems and methods utilize an artificial intelligence (AI)-based methodological model for glenoid defect measurement. The model is used to consistently identify geometric and anatomical features in the shoulder. A two-step approach for segmenting fine features is employed. In the first step, localization is performed using a first trained neural network (e.g., a coverage network called CoverageNet) to generate a cropped field of view. In the second step, segmentation with the cropped field of view is performed using a second trained neural network (e.g., a scanplanar network called ScanPlaneNet). A defect algorithm is then used to compute a metric for determining the severity of Bankart lesions based on the segmentation. For example, a commonly used metric for Bankart lesion severity is the glenoid trajectory width (GT), which is defined as:

[0035] (1)

[0036] Where D represents the subglenoid diameter, and d represents the width of the anterior glenoid bone loss (i.e., the width of the defect or lesion itself), such as Figure 1(MR image of the shoulder with Bankart's lesion) is shown. Figure 1 In the middle, the lower diameter of the glenoid cavity is determined by the optimal first circle located on the glenoid cavity ( Figure 1 The optimal diameter (D) of the first circle within the dashed circle (in the diagram) is represented. Figure 1 In the center, the posterior inferior edge is indicated by a solid line located to the right of the dashed circle. The width of the anterior glenoid bone loss is determined by... Figure 1 The width (d) in the figure indicates the extent of the glenoid cavity. In the absence of lesions, the entire glenoid cavity (e.g., within the optimal circle) will be entirely bone.

[0037] The disclosed implementation provides a method that handles acquisition variations more generally by providing geometric normalization. Furthermore, the method can be interpreted by replicating the final segmentation represented in the same way that clinicians would use in their practice. The disclosed implementation provides an automated diagnostic measurement process. The disclosed implementation reduces workflow time for measuring relevant clinical metrics used in surgical diagnoses.

[0038] Considering the above, Figure 2 The magnetic resonance imaging (MRI) system 100 is schematically illustrated as including a scanner 102, a scanner control circuitry system 104, and a system control circuitry system 106. According to the embodiments described herein, the MRI system 100 is generally configured to perform MR imaging.

[0039] System 100 also includes: a remote access and storage system or device, such as a Picture Archiving and Communication System (PACS) 108; or other devices, such as remote radiology equipment, enabling on-site or remote access to data acquired by system 100. In this way, MR data can be acquired and then processed and evaluated on-site or remotely. While MRI system 100 may include any suitable scanner or detector, in the illustrated embodiment, system 100 includes a whole-body scanner 102 having a housing 120 through which an aperture 122 is formed. An examination table 124 is movable into the aperture 122 to allow a patient 126 (e.g., a subject) to be positioned therein for imaging of selected anatomical structures within the patient's body.

[0040] Scanner 102 includes a series of associated coils for generating a controlled magnetic field used to excite gyromagnetic material within the anatomical structures of the patient being imaged. Specifically, a primary magnetic coil 128 is provided to generate a primary magnetic field B0 generally aligned with an aperture 122. A series of gradient coils 130, 132, and 134 allow the generation of a controlled gradient magnetic field during the examination sequence for positional encoding of certain gyromagnetic nuclei within the patient 126. A radio frequency (RF) coil 136 (e.g., an RF transmit coil) is configured to generate radio frequency pulses for exciting certain gyromagnetic nuclei within the patient. In addition to the coils that may be located locally within scanner 102, system 100 also includes a set of receiving coils or RF receiving coils 138 (e.g., an array of coils) configured to be placed proximal to (e.g., against) the patient 126. For example, receiving coils 138 may include cervical / thoracic / lumbar (CTL) coils, head coils, single-sided spinal coils, etc. Generally, the receiving coil 138 is placed near or above the patient 126 in order to receive weak RF signals generated by certain magnetic nuclei in the patient's body when the patient 126 returns to its relaxed state (weak in relation to the transmission pulses generated by the scanner coil).

[0041] The various coils of system 100 are controlled by an external circuit system to generate desired fields and pulses and to read out emissions from the gyromagnetic material in a controlled manner. In an illustrated embodiment, a main power supply 140 powers the primary field coil 128 to generate a primary magnetic field Bo. Power inputs (e.g., power from a utility or grid), a power distribution unit (PDU), a power supply (PS), and drive circuitry 150 may together provide power to cause gradient field coils 130, 132, and 134 to generate pulses. Drive circuitry 150 may include an amplification and control circuit system for supplying current to the coils according to a sequence of digitized pulses output by scanner control circuitry 104.

[0042] Another control circuit 152 is provided for regulating the operation of the RF coil 136. Circuit 152 includes a switching device for alternating between an active operating mode and a passive operating mode, wherein the RF coil 136 transmits a signal and does not transmit a signal, respectively. Circuit 152 also includes an amplifier configured to generate RF pulses. Similarly, a receiving coil 138 is connected to a switch 154 capable of switching the receiving coil 138 between a receiving mode and a non-receiving mode. Thus, in receiving mode, the receiving coils 138 resonate with the RF signal generated by the release of the magnetic nucleus within the patient 126, and in non-receiving mode, they do not resonate with the RF energy from the transmitting coil (i.e., coil 136) to prevent undesirable operation. Additionally, the receiving circuit 156 is configured to receive data detected by the receiving coil 138 and may include one or more multiplexing and / or amplification circuits.

[0043] It should be noted that although the scanner 102 and the control / amplifier circuit system described above are illustrated as being coupled by a single wire, in practice, many such wires may exist. For example, separate wires may be used for control, data communication, power transmission, etc. Furthermore, appropriate hardware may be provided along each type of wire for the proper handling of data and current / voltage. In practice, various filters, digitizers, and processors may be provided between the scanner and either or both of the scanner control circuit system 104 and the system control circuit system 106.

[0044] As shown in the figure, the scanner control circuit 104 includes an interface circuit 158 ​​that outputs signals for driving the gradient field coil and the RF coil, and for receiving data representing magnetic resonance signals generated in the examination sequence. The interface circuit 158 ​​is coupled to a control and analysis circuit 160. Based on a defined scheme selected via the system control circuit 106, the control and analysis circuit 160 executes commands for driving circuits 150 and 152.

[0045] The control and analysis circuit 160 is also used to receive magnetic resonance signals and perform subsequent processing before sending the data to the system control circuit 106. The scanner control circuit 104 also includes one or more memory circuits 162 that store configuration parameters, pulse sequence descriptions, inspection results, etc. during operation.

[0046] Interface circuitry 164 is coupled to control and analysis circuitry 160 for exchanging data between scanner control circuitry system 104 and system control circuitry system 106. In some embodiments, control and analysis circuitry 160, while exemplified as a single unit, may include one or more hardware devices. System control circuitry 106 includes interface circuitry 166 that receives data from scanner control circuitry system 104 and transmits data and commands back to scanner control circuitry system 104. Control and analysis circuitry 168 may include a CPU in a general-purpose or special-purpose computer or workstation. Control and analysis circuitry 168 is coupled to memory circuitry 170 to store programming code for operating MRI system 100, and to store processed image data for subsequent reconstruction, display, and transmission. The programming code may execute one or more algorithms configured to perform reconstruction of acquired data as described below when executed by a processor. For example, the algorithm may include a defect algorithm for calculating a metric for determining the severity of Bankart lesions based on segmentation generated by an AI-based method model (a neural network architecture with multiple trained neural networks), as discussed in more detail below. In some embodiments, memory circuitry 170 may store one or more neural networks. For example, the neural network may include a first trained neural network (e.g., a coverage network called CoverageNet) for localization and generation of cropped fields of view. The neural network may also include a second trained neural network (e.g., a scanplane network called ScanPlaneNet) for segmentation of the cropped field of view. In some embodiments, the techniques disclosed herein may occur on a separate computing device having processing circuitry and memory circuitry.

[0047] The processing unit (e.g., a microprocessor or processing circuitry) and memory (such as those residing in scanner control circuitry 104 and / or system control circuitry 106) of the magnetic resonance imaging system 100 can be used to execute stored software code, instructions, or routines for acquiring and processing MR data. As used herein, the term "code" or "software code" refers to any instruction or set of instructions that controls the magnetic resonance imaging system 100. The code or software code can exist in the following forms: a computer-executable form, such as machine code, which is a set of instructions and data directly executed by the processing unit of scanner control circuitry 104 and / or system control circuitry 106; a human-understandable form, such as source code, which can be compiled for execution by the processing unit of scanner control circuitry 104 and / or system control circuitry 106; or an intermediate form, such as object code, which is generated by a compiler. In some embodiments, the magnetic resonance imaging system 100 may include multiple controllers.

[0048] For example, the memory may store processor-executable software code or instructions (e.g., firmware or software) tangibly stored on a non-transitory computer-readable medium. Additionally or alternatively, the memory may store data. As an example, the memory may include volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, hard disk drive, or any other suitable optical, magnetic, or solid-state storage medium or combinations thereof). Furthermore, the processing unit may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more application-specific microprocessors, and / or one or more application-specific integrated circuits (ASICs) or some combination thereof. For example, the processing unit may include one or more Reduced Instruction Set Computing (RISC) or Complex Instruction Set Computing (CISC) processors. The processing unit may include multiple processors and / or the memory may include multiple memory devices.

[0049] In some implementations (e.g., for shoulder lesion measurement), a processing unit is configured to acquire three-dimensional (3D) medical imaging data of a subject's shoulder. The processing unit is configured to localize the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view using a first trained neural network with cylindrical segmentation. The processing unit is configured to detect planes containing a circle enclosing the glenoid cavity within the local view using a second trained neural network, predict a ring segmentation mask from the local view using circle / ring segmentation, and predict a glenoid surface segmentation mask from the local view. The processing unit is configured to compute a defect algorithm based on the ring segmentation mask and the glenoid surface segmentation mask to calculate the subglenoid diameter, the width of the bone lesion, and the width of the glenoid trajectory.

[0050] In some embodiments, the processing unit may be configured to output an image of the glenoid cavity from 3D medical imaging data of the shoulder to a display, wherein both a first segmented ring mask representing the subglenoid diameter and a second segmented ring mask representing the width of the bone lesion are overlaid on the glenoid cavity. In some embodiments, the processing unit may be configured to output the calculated subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width.

[0051] In some embodiments, the processing unit may be configured to predict a glenoid surface segmentation mask and a cylindrical segmentation mask having a cylindrical shape surrounding the glenoid surface segmentation mask when the glenohumeral joint is localized using a first trained neural network. In some embodiments, the processing unit may be configured to use the cylindrical segmentation mask to crop 3D medical imaging data of the shoulder to generate a local view. In some embodiments, the processing unit may be configured to normalize and enhance the 3D medical imaging data of the shoulder before cropping the 3D medical imaging data of the shoulder to generate a local view.

[0052] In some embodiments, the processing unit may be configured to normalize and enhance the 3D medical imaging data of the shoulder before localizing the glenohumeral joint in the 3D medical imaging data of the shoulder using a first trained neural network. In some embodiments, the processing unit may be configured to use a defect algorithm to calculate the subglenoid diameter, the width of the bone lesion, and the width of the glenoid trajectory by: fitting a plane derived from a ring segmentation mask; calculating the glenoid profile based on the projection of the glenoid surface segmentation mask onto the plane; parameterizing the ring mask to determine the ring center and diameter; and measuring the width of the bone lesion by finding the minimum distance between the glenoid profile and the ring center to use as the radius. In some embodiments, the 3D medical imaging data is magnetic resonance imaging data acquired using an oZTEo sequence. In some embodiments, the 3D medical imaging data is computed tomography (CT) imaging data.

[0053] Additional interface circuitry 172 may be provided for exchanging image data, configuration parameters, etc., with external system components, such as remote access and storage device 108. Finally, system control and analysis circuitry 168 may be communicatively coupled to various peripheral devices to facilitate the operator interface and the generation of hard copies of reconstructed images. In an illustrated embodiment, these peripheral devices include a printer 174, a monitor 176, and a user interface 178, which includes devices such as a keyboard, mouse, and touchscreen (e.g., integrated with monitor 176).

[0054] Figure 3 The structure of a first trained neural network 180 (e.g., an overlay network) utilized by the disclosed technology is illustrated. The first trained neural network 180 utilizes a U-Net architecture 182 based on a convolutional neural network (CNN). The input data to the first trained neural network 180 is 3D medical imaging data (e.g., 3D oZTEo image medical digital imaging and communication (DICOM) data). The 3D medical imaging data may be a 3D medical imaging volume or a stack of 2D medical images acquired from a 3D volume. Before being input into the first trained neural network 180, the input data may be normalized and / or augmented. Normalization may include resizing to 1.5 mm × 1.5 mm × 1.5 mm. 3 The conversion includes coarser pixel sizes, z-score normalization, and / or DICOM to the Neuroimaging Informatics Technology Initiative (NIfTI) format. Enhancements may include smoothing (e.g., planar [0.4mm, 1.7mm], 1.5mm slices), slice coverage, 3D rotations [-45, 45] simulating differences in patient anatomy and location, offset fields, noise, intensity, scaling, and / or orientation (e.g., coronal, sagittal).

[0055] The first trained neural network 180 is trained using a batch size of 16, a learning rate of 0.0001, and a loss function (e.g., smoothing Dice loss). The first trained neural network 180 is trained to identify the total imaging field of view (i.e., central field of view and extent) of relevant anatomical structures (e.g., the glenohumeral joint). Specifically, the first trained neural network 180 is trained to localize the glenohumeral joint in 3D medical imaging data of the shoulder to a local view (i.e., the local view is a cylinder-defined local view) using cylindrical segmentation. The first trained neural network 180 predicts and outputs a glenoid surface segmentation mask (e.g., a 3D glenoid surface segmentation mask). The first trained neural network 180 also predicts and outputs a cylindrical segmentation mask having a cylindrical shape (e.g., forming a local view) that encompasses the glenoid segmentation surface segmentation mask.

[0056] Figure 4 The structure of a second trained neural network 184 (e.g., a scanplane network) utilized by the disclosed technique is illustrated. The second trained neural network 184 utilizes a U-Net architecture 186 based on a convolutional neural network (CNN). The input data into the second trained neural network 184 is based on the input to... Figure 3 The input data (e.g., 3D oZTEo image DICOM data) is fed into a first trained neural network 180, along with a cylinder segmentation mask generated by the first trained neural network 180. Specifically, the cylinder segmentation mask is used to crop the 3D medical imaging data input to a local view defined by a cylinder to generate a cropped image. The input data (i.e., the cropped image) may be normalized and / or augmented before being input into a second trained neural network 184. Normalization may include resizing to 0.667 mm. 3 ×0.667mm 3 ×0.667mm 3 Smaller pixel size, reslicing to the patient axis orientation, z-score normalization, and / or DICOM to NIfTI format conversion. Enhancements may include smoothing (e.g., [0.4mm, 1.7mm], 1.0mm slices in a plane), 3D rotation [-45, 45], 3D translation, offset field, noise, intensity, left / right flip, and / or orientation (e.g., coronal, sagittal).

[0057] The second trained neural network 184 is trained using a batch size of 16, a learning rate of 0.0001, and a loss function (e.g., distance-weighted Dice loss), as explained in more detail below. The second trained neural network 184 is trained on 3D data to determine one or more image scan planes or image scan plane parameters. Specifically, the second trained neural network 184 is trained to detect planes containing circles enclosing the glenoid cavity within a local view, predicts a ring segmentation mask from the local view using circle / ring segmentation, and predicts a glenoid surface segmentation mask from the local view (i.e., finding the glenoid surface using the local view defined by the cylinder). The input size to the second trained neural network 184 is smaller than the input size to the first trained neural network 180. Furthermore, the second trained neural network 184 predicts more segmentations at its output compared to the first trained neural network 180.

[0058] The loss function of the second trained neural network 184 is a combination of the Dice coefficients and the boundary distance loss (i.e., a distance-weighted Dice loss), which helps in learning thin target loops. The distance and Dice losses are kept weighted (e.g., α1=0.5, α2=0.3 respectively) until the distance loss term becomes negative, at which point the weights are adjusted (e.g., α1=0.1, α2=0.3 respectively). In some implementations, the hyperparameters can be further refined along with enhancement / normalization strategies. Figure 5 An example depicting the distance map of a ring mask. Figure 5 Image 188 on the left is a side cross-sectional view of the distance map of the annular mask. Figure 5 Image 189 on the right is a front view of the distance map of the ring mask (i.e., the ring segmentation mask). The distance-weighted loss function L of the second trained neural network 184 is:

[0059] (2)

[0060] Where m represents the distance graph, and y pred This represents the prediction mask.

[0061] Figure 6 A flowchart illustrating method 190 for measuring shoulder lesions is provided. One or more steps of method 190 can be performed by... Figure 1 The processing circuitry of the magnetic resonance imaging system 100 or a remote computing device is used to execute the process.

[0062] Method 190 includes acquiring 3D medical imaging data of the subject's shoulder (box 192). In some embodiments, the 3D medical imaging data is magnetic resonance imaging data acquired using an oZTEo sequence. In some embodiments, the 3D medical imaging data is computed tomography imaging data. In some embodiments, method 190 includes normalizing and enhancing the 3D medical imaging data of the shoulder before inputting it into a first trained neural network (overlay network) (box 194). Method 190 also includes using the first trained neural network to localize the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view using cylindrical segmentation (box 196). Localizing the glenohumeral joint using the first trained neural network includes predicting a glenoid surface segmentation mask and predicting a cylindrical segmentation mask having a cylindrical shape surrounding the glenoid surface segmentation mask (which forms the local view defined by the cylinder).

[0063] In some embodiments, method 190 includes normalizing and enhancing the 3D medical imaging data of the shoulder (box 198) before cropping the 3D medical imaging data of the shoulder (using a cylindrical segmentation mask) to generate a local view. Method 190 includes cropping the 3D medical imaging data of the shoulder using a cylindrical segmentation mask via a processing system to generate a local view (box 200). Method 190 also includes using a second trained neural network (scan plane network) to detect planes containing a circle (e.g., a best-fit circle) surrounding the glenoid cavity within the local view, using circle / ring segmentation to predict a ring segmentation mask from the local view, and predicting a glenoid surface segmentation mask from the local view (box 202).

[0064] Method 190 further includes using a defective algorithm to calculate the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width based on a ring segmentation mask and a glenoid surface segmentation mask (box 204). Method 190 uses the defective algorithm to calculate the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width by: fitting a plane derived from the ring segmentation mask; calculating the glenoid profile based on the projection of the glenoid surface segmentation mask onto the plane; parameterizing the ring mask to determine the ring center and diameter; and measuring the width of the bone lesion by finding the minimum distance between the glenoid profile and the ring center to use as the radius. In some embodiments, method 190 includes outputting an image of the glenoid from 3D medical imaging data of the shoulder on a display, wherein both a first segmented ring mask representing the subglenoid diameter and a second segmented ring mask representing the width of the bone lesion are overlaid on the glenoid (box 206). In some embodiments, method 190 includes outputting (e.g., on a display) the calculated subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width (box 208). Clinicians may choose to employ these measurements and segmentations provided. Alternatively, clinicians may update, modify, or annotate the information provided.

[0065] Figure 7 A schematic diagram illustrating a procedure 210 for measuring shoulder lesions is provided. Procedure 210 includes acquiring 3D medical imaging data 212 of the subject's shoulder. As depicted, the 3D medical imaging data is magnetic resonance imaging data acquired using an oZTEo sequence. Procedure 210 includes normalizing and enhancing the 3D medical imaging data of the shoulder 212 (as shown in the reference). Figure 3 (As discussed), as shown by reference numeral 214 in the figures. After normalization and enhancement, 3D medical imaging data 212 is input into a coverage network 180 (referred to as CoverageNet). CoverageNet 180 outputs (and predicts) a glenoid surface segmentation mask 216 and a cylindrical segmentation mask 218. The cylindrical segmentation mask 218 covers the glenoid surface segmentation mask 216. Images 220, 222, 224, and 226 represent axial, coronal, sagittal, and oblique views with 3D rendering of the glenoid surface segmentation mask 216, respectively. Images 228, 230, and 232 represent axial, coronal, and sagittal views of the cylindrical segmentation mask 218, respectively.

[0066] Process 210 includes normalizing and enhancing 3D medical imaging data of the shoulder 212 (as referenced). Figure 4 As discussed, the data is shown in reference numeral 234. After normalization and enhancement, the 3D medical imaging data 212 is cropped using a cylindrical segmentation mask 218 (as shown in reference numeral 236) to generate a local view defined by the cylinder (i.e., the cropped image 238). The cropped image 238 is input into a scanplane network 184 (referred to as ScanPlaneNet). The scanplane network 184 outputs (and predicts) a circular segmentation mask 240, an annular segmentation mask 242, and a glenoid surface segmentation mask 244. Images 246, 248, 250, and 252 represent the sagittal, axial, coronal, and oblique views with 3D rendering of the circular segmentation mask 240, respectively. Images 254, 256, 258, and 260 represent the sagittal, axial, coronal, and oblique views with 3D rendering of the annular segmentation mask 242, respectively. Images 262, 264, 266, and 268 represent the sagittal, axial, coronal, and oblique views with 3D rendering of the glenoid surface segmentation mask 244, respectively. The annular segmentation mask 242 and the glenoid surface segmentation mask 244 are input into the defect algorithm 270, which calculates the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width.

[0067] Figure 8A schematic diagram illustrating process 284 for utilizing the defect algorithm is shown. An image 286 from a 3D medical imaging of the shoulder, a ring segmentation mask 242, and a glenoid surface segmentation mask 244 are fed into the defect algorithm. Process 284 includes fitting a plane 288 to the ring segmentation mask 242 using singular value decomposition (SVD), as shown by reference numeral 290. Then, process 284 includes reslicing the 3D medical imaging data (as shown by reference numeral 291) to obtain a resliced ​​image 292 that matches the plane 288. The resliced ​​image 292 is then oriented to a frontal view of the glenoid. Process 284 also includes reslicing the ring segmentation mask 242 (as shown by reference numeral 294) to match the plane 288, performing binary thresholding (as shown by reference numeral 296), and performing binary dilation (as shown by reference numeral 298) to obtain a resliced ​​ring segmentation mask 300. Process 284 also includes reslicing the glenoid surface segmentation mask 244 (as shown by reference numeral 302) and performing binary thresholding (as shown by reference numeral 304) to obtain the resliced ​​glenoid surface segmentation mask 306.

[0068] Process 284 utilizes a re-sliced ​​glenoid surface segmentation mask 306 to project the five intermediate slices onto a plane (as shown by reference numeral 308), fills the holes (e.g., via morphological closure) (as shown by reference numeral 310), and performs binary contouring (as shown by reference numeral 312) to obtain a glenoid profile 314 in the plane. Process 284 also utilizes a re-sliced ​​ring segmentation mask 300 to perform fitting of an n-sphere (as shown by reference numeral 316) to obtain a parametrically fitted “D” ring 318 of the original glenoid. Process 284 includes obtaining or determining the ring diameter 320 (i.e., the lower diameter of the glenoid or “D” metric) from the parametrically fitted “D” ring 318 (as shown by reference numeral 322).

[0069] Procedure 284 utilizes a parametrically fitted “D” ring 318 and glenoid contour 314 to find the minimum distance between the center of the glenoid contour 314 and the center of the parametrically fitted “D” ring 318 as a radius (as shown in reference numeral 324), which is used to obtain a parametrically fitted “d” ring 326 for the lesion. Procedure 284 includes determining the radius difference between the parametrically fitted “D” ring 318 and the parametrically fitted “d” ring 326 (as shown in reference numeral 328) to obtain the width of the bone lesion (anterior glenoid bone loss width) or “d” measure 330. Both the subglenoid diameter 320 and the width of the bone lesion 330 are input into the above formula (1) (and shown by reference numeral 332) to calculate the glenoid trajectory width (GT) 334. In addition to the output metric, process 284 also includes an automatically output image 336 (derived from a re-sliced ​​image 292 oriented to a frontal view of the glenoid fossa), wherein a parametrically fitted “D” loop 318 (e.g., a first segmentation mask loop) and a parametrically fitted “d” loop 326 (e.g., a second segmentation mask loop) are overlaid on the glenoid fossa.

[0070] Figure 9 Table 338 is described, which summarizes the disclosed techniques (i.e., as per the description of...). Figures 6 to 8 The quantitative results of comparing the described AI-based method model for shoulder lesion measurement with ground truth are presented. The image data used for comparison were 3D MRI data of the subject's shoulder obtained using the oZTEo sequence. Real data were derived from clinician annotations. Column 1, 338 of Table 336 represents the case number. Column 2, 340 of Table 338 represents the Dice score comparing the predicted segmentation ring mask with the ground truth segmentation ring mask. Column 3, 342 of Table 338 represents the Dice score comparing the predicted glenoid surface segmentation mask with the ground truth glenoid surface segmentation mask. Column 4, 344 of Table 338 represents the ground truth value of D (i.e., subglenoid diameter). Column 5, 346 of Table 338 represents the predicted value of D. Column 6, 348 of Table 338 represents the difference between the ground truth value and the predicted value of D. As depicted, the difference between the ground truth value and the predicted value of D is negligible. Column 7, 350 of Table 338 represents the ground truth value of d (i.e., the width of the bone lesion or preglenoid bone loss). Column 8, 352 of Table 338 represents the predicted value of d. Column 9, 354 of Table 338 represents the difference between the ground truth value and the predicted value of d. As depicted, the difference between the ground truth value and the predicted value of d is negligible.

[0071] Figure 10 The description describes the use of the disclosed technology (i.e., as about Figures 6 to 8The first example compares the segmentation of the described AI-based method model for shoulder lesion measurement with ground truth segmentation. Segmentation is used for ring targets and glenoid surface targets. The first example is from... Figure 9 The first (top) case in Table 338 is derived from this. Images 356, 358, 360, and 362 are combined images of ground truth segmentation (white) and predicted segmentation (gray) relative to overlapping ring targets. Images 356, 358, 360, and 362 are axial views, sagittal views, oblique views with 3D rendering, and coronal views, respectively. Images 364, 366, 368, and 370 are combined images of ground truth segmentation (white) and predicted segmentation (gray) relative to overlapping glenoid surface targets. Images 364, 366, 368, and 370 are axial views, sagittal views, oblique views with 3D rendering, and coronal views, respectively. Figure 11 An output frontal view image 372 of the glenoid cavity is depicted, having prediction rings 374 and 376 derived from the segmentation using the disclosed technique in the first example. The outer prediction ring 374 represents D (i.e., the subglenoid diameter). The inner prediction ring 376 represents d (i.e., the width of the bone lesion or the width of the anterior glenoid bone loss). As depicted in image 372, the glenoid cavity has been fractured.

[0072] Figure 12 The description describes the use of the disclosed technology (i.e., as about Figures 6 to 8 The second example compares the segmentation of the AI-based method model described for shoulder lesion measurement with ground truth segmentation. Segmentation is used for ring targets and glenoid surface targets. The second example is from... Figure 9 This is derived from the second case in Table 338. Images 378, 380, 382, ​​and 384 are combined images of ground truth segmentation (white) and predicted segmentation (gray) relative to overlapping ring targets. Images 378, 380, 382, ​​and 384 are axial views, sagittal views, oblique views with 3D rendering, and coronal views, respectively. Images 386, 388, 390, and 392 are combined images of ground truth segmentation (white) and predicted segmentation (gray) relative to overlapping glenoid surface targets. Images 386, 388, 390, and 392 are axial views, sagittal views, oblique views with 3D rendering, and coronal views, respectively. Figure 13 An output frontal view image 394 of the glenoid cavity is depicted, featuring prediction rings 396 and 398 derived from the segmentation using the disclosed technique in the second example. The outer prediction ring 396 represents D (i.e., the subglenoid diameter). The inner prediction ring 398 represents d (i.e., the width of the bone lesion or the width of the anterior glenoid bone loss). As depicted in image 394, the shape of the glenoid cavity (as shown by the nearly overlapping prediction rings 396 and 398) is... Figure 11The shape of the glenoid cavity in image 372 is much better.

[0073] Figure 14 The description describes the use of the disclosed technology (i.e., as about Figures 6 to 8 The third example compares the segmentation of the AI-based method model described for shoulder lesion measurement with ground truth segmentation. This is for segmentation of ring targets and glenoid surface targets. The third example is from... Figure 9 This is derived from the sixth (bottom) case in Table 338. Images 400, 402, 404, and 406 are combined images of ground truth segmentation (white) and predicted segmentation (gray) relative to overlapping ring targets. Images 400, 402, 404, and 406 are axial views, sagittal views, oblique views with 3D rendering, and coronal views, respectively. Images 408, 410, 412, and 414 are combined images of ground truth segmentation (white) and predicted segmentation (gray) relative to overlapping glenoid surface targets. Images 408, 410, 412, and 414 are axial views, sagittal views, oblique views with 3D rendering, and coronal views, respectively. Figure 15 An output frontal view image 416 of the glenoid cavity is depicted, having prediction rings 418 and 420 derived from the segmentation using the disclosed technique in the second example. The outer prediction ring 418 represents D (i.e., the subglenoid diameter). The inner prediction ring 420 represents d (i.e., the width of the bone lesion or the width of the anterior glenoid bone loss).

[0074] Figure 16 An output frontal view image 422 of the glenoid cavity is depicted, having predictive rings 424 and 426 derived from segmentation of 3D MRI data of the subject's shoulder obtained using the disclosed technique via an oZTEo sequence. The outer predictive ring 424 represents D (i.e., the subglenoid diameter). The inner predictive ring 426 represents d (i.e., the width of the bone lesion or the width of the anterior glenoid bone loss). Figure 17 An output frontal view image 428 of the glenoid cavity is depicted, having predictive rings 430 and 432 derived from segmentation of 3D MRI data of the shoulder of another subject obtained using the disclosed technique via an oZTEo sequence. The outer predictive ring 430 represents D (i.e., the subglenoid diameter). The inner predictive ring 432 represents d (i.e., the width of the bone lesion or the width of the anterior glenoid bone loss). Figure 17 The shape of the glenoid cavity in image 422 (as shown by the nearly overlapping prediction rings 424 and 426) is greater than Figure 17 The shape of the glenoid cavity in image 428 is much better.

[0075] The technical effects of the disclosed subject matter include providing systems and methods for measuring shoulder lesions. Specifically, a deep learning-based pipeline is used to automate Bankart lesion measurements given a 3D medical imaging volume (MR imaging volume acquired using bone-specific sequences such as oZTEo), thereby improving the surgical diagnostic workflow for shoulder instability. The technical effects of the disclosed subject matter include utilizing an artificial intelligence-based methodological model for measuring glenoid defects. The technical effects of the disclosed subject matter include providing a method that more generally handles acquisition variations by providing geometric normalization. Additionally, the technical effects of the disclosed subject matter include providing a method that can be interpreted by replicating the final segmentation represented in the same way that clinicians will use in their practice. The technical effects of the disclosed subject matter include providing an automated diagnostic measurement process. The technical effects of the disclosed subject matter include reducing workflow time for measuring relevant clinical metrics used in surgical diagnosis.

[0076] Referring to the technology presented herein and protected by the claims, and applying it to physical objects and concrete examples of practical nature, said practical nature explicitly improves the present art and is therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements designated as “component for [performing]…the function” or “step for [performing]…the function,” such elements are intended to be interpreted according to 35 USC 112(f). However, for any claim containing elements designated in any other manner, such elements are not intended to be interpreted according to 35 USC 112(f).

[0077] This written description uses examples to disclose the subject matter of the invention, including best practices, and also enables those skilled in the art to practice the subject matter, including making and using any device or system and performing any included methods. The patent scope of this subject matter is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. A computer-implemented method for measuring shoulder lesions, the method comprising: Three-dimensional (3D) medical imaging data of the subject's shoulder are obtained via a processing system including one or more processors; The processing system utilizes a first trained neural network (180) to localize the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view using cylindrical segmentation. The processing system uses a second trained neural network (184) to detect planes containing circles surrounding the glenoid cavity within the local view, uses circle / ring segmentation to predict ring segmentation masks from the local view, and predicts glenoid cavity surface segmentation masks from the local view. as well as The processing system uses a defect algorithm (270) to calculate the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width based on the ring segmentation mask and the glenoid surface segmentation mask.

2. The computer-implemented method of claim 1, further comprising outputting an image of the glenoid cavity from the 3D medical imaging data of the shoulder on a display via the processing system, wherein both a first segmented ring mask representing the lower diameter of the glenoid cavity and a second segmented ring mask representing the width of the bone lesion cover the glenoid cavity.

3. The computer-implemented method according to claim 1, further comprising outputting the calculated subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width via the processing system.

4. The computer-implemented method of claim 1, wherein using the first trained neural network (180) to localize the glenohumeral joint includes predicting a segmentation mask of the glenoid surface and predicting a cylindrical segmentation mask having a cylindrical shape surrounding the segmentation mask of the glenoid surface.

5. The computer-implemented method of claim 4, further comprising cropping the 3D medical imaging data of the shoulder using the cylindrical segmentation mask via the processing system to generate the partial view.

6. The computer-implemented method of claim 5, further comprising normalizing and enhancing the 3D medical imaging data of the shoulder via the processing system before cropping the 3D medical imaging data of the shoulder to generate the partial view.

7. The computer-implemented method of claim 1, further comprising normalizing and enhancing the 3D medical imaging data of the shoulder via the processing system before localizing the glenohumeral joint in the 3D medical imaging data of the shoulder using the first trained neural network (180).

8. The computer-implemented method according to claim 1, wherein calculating the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width using the defect algorithm (270) comprises: Fit the plane derived from the ring segmentation mask; The contour of the glenoid cavity is calculated based on the projection of the glenoid cavity surface segmentation mask onto the plane; Parametrically configure the ring mask to determine the ring center and diameter; as well as The width of the bone lesion is measured by finding the minimum distance between the glenoid contour and the center of the ring, which is used as the radius.

9. The computer-implemented method of claim 1, wherein the 3D medical imaging data includes magnetic resonance imaging data acquired using an oZTEo sequence.

10. A system for measuring shoulder lesions, the system comprising: A memory that encodes processor-executable routines; and A processing system, comprising one or more processors and configured to access the memory and execute processor-executable routines, wherein the processor-executable routines, when executed by the processing system, cause the processing system to: Obtain three-dimensional (3D) medical imaging data of the subject's shoulder; The first trained neural network (180) is used to localize the glenohumeral joint in the 3D medical imaging data of the shoulder to a local view using cylindrical segmentation; A second trained neural network (184) is used to detect planes containing circles surrounding the glenoid within the local view, and circle / ring segmentation is used to predict ring segmentation masks from the local view and glenoid surface segmentation masks from the local view. as well as The defect algorithm (270) is used to calculate the subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width based on the ring segmentation mask and the glenoid surface segmentation mask.

11. The system of claim 10, wherein the processor-executable routine, when executed by the processing system, further causes the processing system to output an image of the glenoid cavity from the 3D medical imaging data of the shoulder on a display, wherein both a first segmented ring mask representing the lower diameter of the glenoid cavity and a second segmented ring mask representing the width of the bone lesion cover the glenoid cavity.

12. The system of claim 10, wherein the processor-executable routine, when executed by the processing system, further causes the processing system to output the calculated subglenoid diameter, the width of the bone lesion, and the glenoid trajectory width.

13. The system of claim 10, wherein using the first trained neural network (180) to localize the glenohumeral joint includes predicting a segmentation mask of the glenoid surface and predicting a cylindrical segmentation mask having a cylindrical shape surrounding the glenoid surface segmentation mask.

14. The system of claim 13, wherein the processor-executable routine, when executed by the processing system, further causes the processing system to use the cylindrical segmentation mask to crop the 3D medical imaging data of the shoulder to generate the local view.

15. The system of claim 14, wherein the processor-executable routine, when executed by the processing system, further causes the processing system to normalize and enhance the 3D medical imaging data of the shoulder before cropping the 3D medical imaging data of the shoulder to generate the partial view.

Images