Determining properties of muscle structure using artificial neural networks

Abstract

Determining properties of muscle structures using artificial neural networks. Techniques for determining a quantification of at least one property of a muscle structure including at least one muscle and at least one tendon are disclosed. A quantification of at least one property of a rotator cuff can be determined by using at least one artificial neural network and based on one or more medical images depicting the muscle structure of a patient.

Description

[0001] This application claims priority to European Patent Application No. 21,177,272.8, filed on 1 June 2021, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] Various examples of the present invention relate to determining the characteristics of a patient's muscle structure, said muscle structure comprising at least one muscle and at least one tendon. Specifically, various examples of this disclosure relate to the quantification of at least one characteristic of a muscle structure based on one or more medical images depicting the patient's muscle structure, using one or more artificial neural networks. Background Technology

[0003] Muscle tears, especially tendon tears, are painful pathologies. For example, rotator cuff tears are one of the most common causes of shoulder pain, accounting for nearly 2 million physician visits in the United States in 2013. A meta-study estimated the overall prevalence of rotator cuff tears at 20-30% in asymptomatic individuals and as high as 64% in symptomatic individuals. Furthermore, the incidence of surgical treatment for rotator cuff tears continues to rise.

[0004] Magnetic resonance imaging (MRI) is considered the standard of care, assessment, and treatment planning for muscle and / or tendon tears, such as rotator cuff tears. The upper extremities, namely the shoulder and arm, account for 11% of all MRI procedures. Tears must be measured in two dimensions on different imaging planes. Furthermore, it has been shown that following rotator cuff injuries (such as tendon tears), the rotator cuff muscle structures are prone to fatty infiltration and atrophy, and severe fatty infiltration and atrophy are associated with poor functional outcomes from rotator cuff repair.

[0005] Consistent and accurate measurement of tears (especially rotator cuff tears) and assessment of rotator cuff muscle degeneration (e.g., fatty infiltration and atrophy) are necessary and critical for selecting the optimal treatment and surgical approach, as well as for influencing postoperative prognosis and tear recurrence. In routine clinical practice, radiologists scroll through numerous MR images to detect tears and then manually measure the tears or only estimate their size in a picture archiving and communication system (PACS), and / or classify fatty infiltration and describe the degree of muscle atrophy (volume loss) based on the Goutallier classification system.

[0006] However, such techniques face certain limitations and drawbacks. For example, the interpretation of MR imaging for suspected rotator cuff tears is complex and time-consuming, requiring the analysis of several image series acquired in different imaging planes. Furthermore, the study reported significant inter-reader and intra-reader variability, leading to moderate reproducibility in tear assessment, such as the quantification of tear size and the amount of fatty degeneration of the rotator cuff muscle structure. Summary of the Invention

[0007] Therefore, there is a need for advanced technologies to mitigate or overcome the aforementioned drawbacks or limitations. There is also a need for advanced technologies to assess muscle and / or tendon tears (such as rotator cuff tears).

[0008] This requirement is satisfied by the features of the embodiments described herein.

[0009] The following text discloses a technique for determining at least one characteristic of muscle structure.

[0010] As a general rule, a muscular structure may include at least one muscle. A muscular structure may also include at least one tendon. The at least one muscle can contract and expand. The at least one tendon connects the at least one muscle to a bone. Therefore, a muscular structure may be arranged around or adjacent to a bone.

[0011] Various characteristics can be determined, such as at least one of the following anatomical characteristics: the presence or absence of muscle and / or tendon tears, the size of the muscle tear and the degree of degeneration of the muscle structure, and in particular, the size of the muscle and / or tendon tear and the degree of muscle degeneration can be measured automatically and accurately.

[0012] As a general rule, it is likely that at least one characteristic is to classify the properties of muscle structure into multiple predefined classes. A classification algorithm can be used.

[0013] Alternatively or additionally, at least one characteristic may include the quantification of properties of muscle structure. Here, it is not necessary to rely on predefined classes. Regression algorithms can be used.

[0014] At least one characteristic of a muscle structure can be determined by using at least one artificial neural network and based on one or more medical images depicting the patient's muscle structure.

[0015] A computer-implemented method is provided. The method includes acquiring one or more medical images. The one or more medical images depict the muscle structure of a patient. The muscle structure includes at least one muscle and optionally at least one tendon. The method further includes using at least one artificial neural network to determine at least one characteristic of the muscle structure.

[0016] For example, the quantization of at least one characteristic can be determined.

[0017] For example, the rotator cuff muscle structure can be a separate muscle group. The rotator cuff muscle structure includes several muscles: the supraspinatus, infraspinatus, teres minor, and subscapularis.

[0018] A computer program, computer program product, or computer-readable storage medium comprising program code is provided. The program code can be loaded and executed by at least one processor. During the loading and execution of the program code, at least one processor executes a method. The method includes obtaining one or more medical images. The one or more medical images depict the muscle structures of a patient. The muscle structures include at least one muscle and optionally at least one tendon. The method further includes using at least one artificial neural network to determine at least one characteristic of the muscle.

[0019] For example, the quantization of at least one characteristic can be determined.

[0020] A system is provided comprising at least one processor and at least one memory. The at least one processor is configured to load program code from the at least one memory. When executing the program code, the at least one processor executes a method. The method includes acquiring one or more medical images. The one or more medical images depict the muscle structure of a patient. The muscle structure includes at least one muscle and optionally at least one tendon. The method further includes using at least one artificial neural network to determine at least one characteristic of the muscle structure.

[0021] For example, the quantization of at least one characteristic can be determined.

[0022] A medical imaging scanner comprising a system is provided. The system includes at least one processor and at least one memory. The at least one processor is configured to load program code from the at least one memory. When executing the program code, the at least one processor executes a method. The method includes acquiring one or more medical images. The one or more medical images depict the muscle structures of a patient. The muscles include at least one muscle and optionally at least one tendon. The method further includes using at least one artificial neural network to determine at least one characteristic of the muscle structure.

[0023] For example, the quantization of at least one characteristic can be determined.

[0024] It should be understood that, without departing from the scope of the invention, the above-described features and those to be explained below can be used not only in the indicated corresponding combinations, but also in other combinations or individually. Attached Figure Description

[0025] Figure 1 An exemplary MRI image acquired in the axial plane is schematically illustrated.

[0026] Figure 2 The illustrations schematically depict details of the system based on various examples.

[0027] Figure 3 The workflow is illustrated schematically from a clinical perspective.

[0028] Figure 4 This diagram illustrates another workflow from a clinical perspective.

[0029] Figure 5 It is a flowchart based on various examples of methods.

[0030] Figure 6 It is a block diagram of the system based on various examples. Detailed Implementation

[0031] Some examples of this disclosure typically provide multiple circuits or other electrical devices. All references to circuits and other electrical devices, and the functions provided by each device, are not intended to be limited to what is illustrated and described herein. While specific labels may be assigned to the various circuits or other electrical devices disclosed, such labels are not intended to limit the scope of the operation of the circuits and other electrical devices. Such circuits and other electrical devices may be combined and / or separated from each other in any way based on a particular type of desired electrical implementation. It should be appreciated that any circuit or other electrical device disclosed herein may include any number of microcontrollers, graphics processing units (GPUs), integrated circuits, memory devices (e.g., FLASH, random access memory (RAM), read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or other suitable variations thereof), and software that cooperates with each other to perform one or more of the operations disclosed herein. Furthermore, any one or more of the electrical devices may be configured to execute program code implemented in a non-transitory computer-readable medium programmed to perform any number of functions as disclosed.

[0032] In the following, embodiments of the invention will be described in detail with reference to the accompanying drawings. It should be understood that the following description of the embodiments should not be construed as limiting. The scope of the invention is not intended to be limited by the embodiments described below or by the accompanying drawings, which are to be understood only as illustrative.

[0033] The accompanying drawings are to be considered schematic representations, and the elements illustrated in the drawings are not necessarily shown to scale. Rather, the various elements are shown such that their function and general purpose will be apparent to those skilled in the art. Any connection or coupling between functional blocks, devices, components, or other physical or functional units shown in the drawings or described herein may also be achieved through indirect connection or coupling. Coupling between components may also be established via wireless connection. Functional blocks may be implemented using hardware, firmware, software, or a combination thereof.

[0034] The various techniques disclosed herein generally involve determining one or more characteristics of a patient's muscle structure. A muscle structure includes at least one muscle and typically includes at least one tendon. An example of a muscle structure would be the rotator cuff. For instance, one or more characteristics of a muscle structure can be determined, particularly those characterizing a tear in the muscle structure (e.g., a rotator cuff tear). It will be possible to automatically, accurately, and based on the same criteria determine the characteristics of muscle structures in different patients to facilitate inter-patient comparisons.

[0035] For example, it would be possible to determine the quantization of at least one characteristic. Quantization can involve numerical values ​​defined in a continuous result space.

[0036] In other examples, it would be possible to determine the classification of at least one characteristic. Here, a set of discrete predefined classes is available, and the result is a pointer to one of these predefined classes.

[0037] As a general rule, a muscular structure can include a group of muscles. Muscular structures can be located near any of the following joints in the human body: ball-and-socket joints, hinge joints, condylar joints, pivot joints, gliding joints, saddle joints, etc.

[0038] For example, at least one property can affect multiple muscles in muscle structure.

[0039] As a further general rule, a tear can include a tendon tear, a muscle fiber tear, or a strain of a tendon and / or muscle fiber.

[0040] According to various examples, at least one characteristic of muscle structure is determined based on data collected via non-invasive diagnostic methods, such as at least one of the following: projection radiography, computational tomography (CT), ultrasound imaging, MRI, or any other type of medical imaging modality. That is, the medical imaging data or images processed or analyzed in this disclosure may include at least one of the following: MRI image data, X-ray image data, computational tomography image data, ultrasound image data, or any other type of medical image data. In particular, imaging data acquired via these imaging modalities can be fed into at least one trained artificial neural network, and thus the trained artificial neural network accurately and automatically determines at least one characteristic.

[0041] As a general rule, the projection radiography imaging data and ultrasound imaging data used in this disclosure may each include images in the spatial domain. CT imaging data and MR imaging data may each include reconstructed images in the frequency domain, reconstructed images in the spatial domain, and so on. Medical imaging data may be 1-D data, 2-D reconstructed images, or 3-D reconstructed slices including multiple voxels, obtained directly from the corresponding scanner.

[0042] According to this disclosure, MR imaging, particularly MRI scanners with a 3.0 Tesla main magnetic field, is preferred for shoulder evaluation because the higher main magnetic field strength provides a greater signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) due to faster acquisition time and thinner slice selection. Standard conventional MR imaging of the shoulder is acquired in three orthogonal planes: axial, coronal oblique, and sagittal oblique. Various MR scanning protocols can be used to perform shoulder scans.

[0043] As explained above, the muscle structures described in this disclosure may include any one or more muscles and one or more tendons. Muscles may be human muscles or muscles of other animals. Hereinafter, various techniques of this disclosure will be described in detail based on the rotator cuff as an exemplary muscle structure. That is, the techniques disclosed in this disclosure can be readily applied to other muscles by simply replacing the term "rotator cuff" with the specific names of other muscle structures.

[0044] Figure 1 An exemplary MRI image 1000, acquired and depicting the shoulder in an axial plane, is schematically illustrated. The MRI image 1000 may include a region of interest (ROI) 1010, i.e., the area within a dashed rectangle. The ROI 1010 is associated with the rotator cuff. The MRI image 1000 may be cropped to obtain the ROI 1010, and then only the ROI 1010 may be fed into at least one trained artificial neural network to determine at least one characteristic of the rotator cuff. Similarly, the ROI may be defined on other images obtained via projection radiography, CT, ultrasound imaging, or MRI. Alternatively, the ROI may also be defined volumetrically (i.e., in 3D).

[0045] Based on various examples, the ROI 1010 can be determined manually by a clinician, for instance.

[0046] Based on various examples, a Region of Interest (ROI) 1010, such as a rotator cuff tear or rotator cuff tendon, can be determined based on at least one landmark in one or more medical images / slices. Such landmarks can be detected using a landmark detection algorithm configured to detect ROI 1010 in one or more medical images 1000. This landmark detection algorithm can be a machine learning algorithm.

[0047] According to this disclosure, an ROI may include at least one tendon, including rotator cuff muscles such as the tendon of the supraspinatus, infraspinatus, teres minor, or subscapularis. The following examples can be used in the context of a single tendon and / or a single muscle; that is, the techniques disclosed below can be applied separately to each individual tendon and / or muscle based on precise ROI detection. Therefore, multiple ROIs can be detected in a single medical image or slice.

[0048] As a further general rule, image preprocessing techniques may be applied to the image or imaging data before feeding it into at least one trained artificial neural network. These techniques include, for example, cropping the image or slice to obtain the region of interest (ROI), downsampling to reduce the resolution of the image or slice, and filtering out noise. Furthermore, when one or more medical images (or imaging data) depicting a patient's rotator cuff comprise multiple images or slices acquired from one or more imaging modalities, registration may be applied to one or more medical images (or imaging data).

[0049] According to this disclosure, at least one characteristic of the rotator cuff may include the presence or absence and / or location and / or type of tear in the tendons of the rotator cuff. The at least one characteristic may further include the classification and optional quantification of muscle atrophy in the rotator cuff. Additionally or alternatively, the at least one characteristic may include the classification and optional quantification of fatty infiltration in the rotator cuff. The at least one characteristic may also include the length, width, thickness, and tendon connection location of the rotator cuff tear.

[0050] According to this disclosure, the Goutallier classification (see, for example: Slabaugh, Mark A. et al., “Interobserver and intraobserver reliability of the Goutallier classification using magnetic resonance imaging: proposal of a simplified classification system to increase reliability,” The American journal of sports medicine 40.8 (2012): 1728-1734) can be used, particularly in the context of rotator cuff tendon tears, to quantify the amount of fatty degeneration in the rotator cuff muscles. Although initially described in shoulder CT, it is applicable and most commonly used in MRI. It is primarily based on the percentage of atrophy and fatty degeneration in the affected muscles. Increased severity grades and higher grades are associated with poorer functional outcomes after surgical repair of rotator cuff tears. The Goutallier classification system can include five grades, as shown in Table 1.

[0051] Figure 2 Details of a system 2000 according to various examples are schematically illustrated. System 2000 may include at least one artificial neural network that receives one or more medical images 1000 depicting a patient's rotator cuff as input and outputs values ​​indicating the corresponding characteristics for each of at least one characteristic. The values ​​indicating the corresponding characteristics may be written to a document 2010. System 2000 may also include only one artificial neural network comprising multiple subnetworks.

[0052] Level Index Classification criteria 0 normal muscles 1 Some fatty streaks 2 Less than 50% of fat and muscle atrophy 3 50% of fat and muscle atrophy 4 More than 50% of fat and muscle atrophy

[0053] Table 1: Various levels of Goutallier classification

[0054] According to various examples, system 2000 may include a convolutional neural network 2020, which may be configured to perform pre-segmentation (or coarse segmentation) on one or more medical images 1000 to determine a pixel probability map 2030 or directly determine a pixel mask 2040. The pixel mask 2040 is obtained by applying a threshold comparison to the probability values ​​of the pixel probability map 2030 and selecting the largest contiguous region.

[0055] As a general rule, a convolutional neural network can include one or more layers that perform convolutions. Here, a predefined kernel (whose weights are set during the training phase) is convolved with the input values ​​output by the previous layer.

[0056] According to this disclosure, system 2000 may further include a first 2050 and a second 2070 of at least one artificial neural network. The first 2050 of the at least one artificial neural network may be configured to determine a segmentation 2060 of a region of interest 1010 associated with the rotator cuff in one or more medical images 1000 based on pre-segmentation, such as based on a pixel probability map 2030 or a pixel mask 2040. The segmentation 2060 of the region of interest 1010 determined by the first 2050 of the at least one artificial neural network may further be based on one or more medical images 1000 or the ROI 1010. Additionally or alternatively, segmentation 2060 may include a bounding box, which may be determined based on the pixel mask 2040. The second 2070 of the at least one artificial neural network may be configured to determine a value indicating at least one characteristic based on the segmentation 2060 of the region of interest 1010 and one or more medical images 1000.

[0057] According to various examples, when the at least one characteristic includes multiple characteristics, the system 2000 may include multiple decoder branches configured to determine multiple characteristics separately based on a shared latent feature set determined based on one or more medical images 1000. For example, a second 2070 of at least one artificial neural network may include multiple decoder branches 2070a-2070d. Each of the multiple decoder branches 2070a-2070d may determine at least one different characteristic separately. For example, the multiple decoder branches 2070a-2070d may separately determine the size of the tear (e.g., including length, width, and thickness), the percentage of muscle atrophy, the grade of muscle fat infiltration, and the coordinates of the tendon connection location.

[0058] As a general rule, the pixel probability map 2030, pixel mask 2040, segmentation 2060, and bounding box can be 2-D or 3-D.

[0059] As a general rule, various kinds and types of artificial neural networks can be used as the first 2050 and second 2070 of at least one artificial neural network and benefit from the techniques described herein, such as convolutional neural networks, reinforcement neural networks, residual neural networks, recurrent neural networks, recurrent neural networks, long short-term memory (LSTM) neural networks, etc. For example, it would be possible to use deep neural networks, such as convolutional neural networks with one or more convolutional layers that perform convolutions between input data and kernels, to implement both the first 2050 and second 2070 of at least one artificial neural network. To give just a few examples, using support vector machines would also be possible. Preferably, the first 2050 of at least one artificial neural network may include an encoder and a decoder, see, for example, Ronneberger, Olaf, Philipp Fischer, and Thomas Brox, “U-net: Convolutional networks for biomedical image segmentation,” International Conference on Medical image computing and computer-assisted intervention, Springer, Cham, 2015, or Yang Dong et al., “Automatic liver segmentation using an adversarial image-to-image network,” International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, 2017. Preferably, the second 2070 of at least one artificial neural network may use deep reinforcement learning methods, see, for example, Xu Zhoubing et al., “Supervised action classifier: Approaching landmark detection as image partitioning,” International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Cham, 2017.Additionally or alternatively, muscle atrophy and / or muscle fat infiltration grading in ROI 1010 can be performed using a residual learning framework, see, for example, He Kaiming et al., “Deep residual learning for image recognition,” Proceedings of the IEEE conference on computer vision and pattern recognition, 2016.

[0060] Based on various examples, at least one artificial neural network can be implemented using deep reinforcement learning methods to determine the quantification of a tear, such as the tear location (i.e., coordinates of the start and end points, respectively), length, width, or thickness. Alternatively or optionally, at least one artificial neural network can be implemented using one or more different artificial neural networks to determine the classification and, optionally, the quantification of muscle atrophy or fatty infiltration.

[0061] According to this disclosure, the convolutional neural network 2020 can share the same network architecture as the first 2050 of at least one artificial neural network. However, the input of the convolutional neural network 2020 can have a lower resolution than the first 2050 of the at least one artificial neural network. A lower resolution may mean a smaller input vector size; therefore, fewer spatial points are sampled from the input vector.

[0062] According to various examples, the medical image 1000 or ROI 1010 fed to the first 2050 and / or the second 2060 of at least one artificial neural network also has a lower resolution than the original medical image (e.g., a medical image acquired by a medical imaging scanner) to reduce the computational burden.

[0063] According to this disclosure, various training methods for artificial neural networks can be applied to train at least one artificial neural network, such as supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc.

[0064] For example, at least one convolutional neural network 2020, a first 2050, and a second 2070 can be trained separately using different training datasets based on supervised learning techniques. Each training process may include determining a loss value based on a comparison between the predictions of the corresponding 2050 or 2070 of the at least one artificial neural network and the ground truth. The loss function can provide the loss value by performing the comparison. Based on the loss value, the weights of the artificial neural network can then be adjusted. Here, an optimization algorithm, such as gradient descent, can be used. Backpropagation can be an alternative.

[0065] Each training dataset typically includes one or more training medical images and a set of underlying facts indicating information relevant to the region of interest 1010. For example, the training dataset for the convolutional neural network 2020 may include annotated pixel probability maps or annotated pixel masks, the training dataset for the first 2050 of at least one artificial neural network may include annotated segmentations of the region of interest 1010, and the training dataset for the second 2070 of at least one artificial neural network may include clinical or ground truth values ​​quantifying the corresponding characteristics of the rotator cuff. These labels can be assigned by domain experts, such as clinicians. Alternatively, one or more training medical images for the convolutional neural network 2020, the first 2050, and the second 2070 of at least one artificial neural network may be identical.

[0066] On the other hand, at least one convolutional neural network 2020, the first 2050, and the second 2070 can be jointly trained; that is, the three neural networks can be treated as a whole, and the parameter values ​​of the three (sub)neural networks are updated together using, for example, backpropagation in a joint optimization process based on a common loss value. This corresponds to end-to-end training.

[0067] A robust approach to processing medical image data is provided by employing neural networks trained with appropriate training datasets. These neural networks can handle complex medical image data and deliver more accurate results than other existing computational techniques used for image processing.

[0068] Based on various examples, to eliminate the processing steps for obtaining training data for annotation (e.g., manual annotations from domain experts), unsupervised learning can be used to train convolutional neural networks 2020, at least one artificial neural network 2050, and a second 2070, such as using constraints disclosed in Celebi, M. Emre, and Kemal Aydin, eds. Unsupervised learning algorithms, Berlin: Springer International Publishing, 2016. Reinforcement learning can be an alternative.

[0069] Based on various examples, each of the convolutional neural networks 2020, and the first 2050 and second 2070 of at least one artificial neural network, can be trained using different training techniques. For example, the convolutional neural network 2020, and the first 2050 and second 2070 of at least one artificial neural network can be trained using supervised learning, unsupervised learning, and semi-supervised learning, respectively.

[0070] Next, we will combine Figure 3 and Figure 4This section explains aspects of the processing workflow for determining at least one characteristic of the rotator cuff. Specifically, the processing workflow can achieve the above combination. Figure 2 Discussion processing. System 2000 can combine the processing workflows of pipelines 5000 and 6000 to handle discussions.

[0071] Figure 3 This is an exemplary clinical processing pipeline 5000 for automatically determining the quantification of at least one characteristic of a rotator cuff tendon tear based on medical imaging data such as MR imaging data. The quantification of at least one characteristic may include the size of the rotator cuff tendon tear (e.g., length, width, thickness) and / or the location of the tendon connection.

[0072] As a general rule, while the technique will be described in conjunction with the quantification that identifies at least one characteristic, it will also be possible, as a general rule, to identify the classification of at least one characteristic. Techniques similar to those described below can be readily applied.

[0073] When loading program code, it can be done by Figure 2 The system 2000 executes the processing pipeline 5000. The details of the processing pipeline 5000 are described below.

[0074] At box 5010, the image series used for measurement (e.g., MRI series) is defined by the clinical protocol, such as measuring the mid-lateral dimension of a supraspinatus tendon tear on a coronal tilt fluid-sensitive series. To select an appropriate image series, system 2000 can use DICOM (Digital Imaging and Communication in Medicine) information. Image series can be loaded from a database. The imaging device can be controlled to acquire and deliver image series. Thus, one or more medical images are obtained.

[0075] At box 5020, an additional AI agent, such as those disclosed in patent application US 2020 / 0167911 A1, is used to determine the presence or absence and / or type of a tear in the rotator cuff tendon. When performing a medical imaging scan (e.g., an MR scan), the imaging data acquired during the scan (such as image slices) may include imaging data depicting both healthy and diseased tissue (e.g., a tendon tear). From a clinical perspective, imaging data depicting diseased tissue is particularly important. By using an additional AI agent, the imaging data acquired during the scan can be automatically and precisely assigned to at least a first group and a second group, respectively, corresponding to healthy and diseased tissue, based on the presence or absence of a tear in the tendon. Subsequent processing of the imaging data can focus on the second group (i.e., the imaging data corresponding to the diseased tissue), and thus reduce the consumption of computational resources. That is, pre-filtering can be performed. Furthermore, the additional AI agent can determine the type of tear in the tendon based on the second group of acquired imaging data. Alternatively, the presence or absence and / or type of a tear in the tendon can be determined by using a classification algorithm. The classification algorithm is configured to select a class from a list indicating: no tendon tear; presence of tendon tear; partial tear; low-grade partial tear; high-grade partial tear; or complete tear.

[0076] Therefore, box 5020 corresponds to preprocessing.

[0077] At box 5030, one or more medical images 1000 depicting the rotator cuff are selected based on the detection of the presence and / or type of tear. The one or more medical images 1000 can be processed based on a landmark detection algorithm configured to detect a region of interest 1010 within the one or more medical images 1000. Other image processing techniques, such as downsampling and / or normalization, can be applied before and / or after detecting the ROI 1010 using the landmark detection algorithm.

[0078] As a general rule, the preprocessing of boxes 5020 and 5030 can also be combined in a single preprocessing block.

[0079] At box 5040, a convolutional neural network, such as... Figure 2 The 2020 implementation of a rough tearing and splitting.

[0080] The convolutional neural network 2020 can provide a pixel probability map 2030 as output. To obtain a binary segmentation mask, such as a pixel mask 2040, a threshold, for example 0.5, is applied to the pixel probability map 2030. The largest connected component is selected as the final coarse segmentation mask 2040. The segmentation mask 2040 is then transferred to the original full-resolution image. Bounding boxes with additional padding in all dimensions can be generated, and the Regions of Interest (ROIs) are defined.

[0081] At box 5050, fine tear segmentation in ROI 1010 is performed by the first 2050 of at least one artificial neural network, thereby determining the segmentation 2060 of ROI 1010 associated with the rotator cuff in one or more medical images 1000.

[0082] At box 5060, by finding the farthest voxels in the segmentation mask and projecting them onto the slice at the centroid of the segmentation mask, the coordinates of the tear, such as the corresponding coordinates of the start and end points of the tear, can be extracted from the segmentation mask, such as segment 2060.

[0083] At box 5070, the coordinates of the tear are transferred to the target system, such as a PACS workstation, PACS storage device, and / or RIS (Radiation Information System).

[0084] The exemplary clinical processing pipeline 5000 facilitates the quantification of at least one characteristic of a rotator cuff tendon tear based on medical imaging data, such as the size of the rotator cuff tendon tear (e.g., length, width, thickness) and / or the location of the tendon connection, and thus significantly improves the treatment and / or surgical planning for patients with rotator cuff tears. The exemplary clinical processing pipeline 5000 utilizes AI-based technology and, compared to manual methods, can significantly reduce the time required to determine such characteristics and provide consistent and reproducible results, addressing both inter-reader and intra-reader variability in current clinical practice.

[0085] Figure 4 This is an exemplary clinical processing pipeline 6000 for automatically classifying and quantifying rotator cuff muscle mass based on medical imaging data such as MR imaging data. Rotator cuff muscle mass may include at least one of muscle atrophy or fatty infiltration.

[0086] When loading program code, pipeline 6000 can also be handled by... Figure 2 The system 2000 is running. The following describes the processing pipeline 6000 in detail.

[0087] At box 6010, for example, an image series (i.e., an MRI series) for classifying and quantifying the muscle mass of the rotator cuff is obtained from a database of rotator cuff MRI studies. It will be possible to load the image series from the imaging device.

[0088] Box 6010 typically corresponds to box 5010.

[0089] At box 6020, the scapular-Y structure of the rotator cuff is determined from the scapula, and the type of tear in each rotator cuff tendon is classified based on landmarks and using an AI agent. The AI ​​agent may be those disclosed in patent US 9 792 531 B2 or patent application US2020 / 0167911 A1.

[0090] Based on various examples, it can be determined that multiple muscles in a muscle structure exhibit muscle mass degeneration. For each individual muscle, the following techniques can be applied.

[0091] At box 6030, based on the detection of the presence and / or type of tear, one or more medical images 1000 depicting the rotator cuff are selected to perform muscle mass grading. One or more medical images 1000 can be processed based on a landmark detection algorithm configured to detect a region of interest 1010 in one or more medical images 1000. Before and / or after detecting the ROI 1010 using the landmark detection algorithm, other image processing techniques, such as downsampling and / or normalization and / or cropping, can be applied.

[0092] Alternatively, through a convolutional neural network, such as Figure 2 In the 2020 image, a coarse tearing segmentation is performed. The convolutional neural network 2020 can provide a pixel probability map 2030 as output. To obtain a binary segmentation mask, such as a pixel mask 2040, a threshold, such as 0.5, is applied to the pixel probability map 2030. The largest connected component is selected as the final coarse segmentation mask 2040. The segmentation mask 2040 is transferred to the original full-resolution image. Bounding boxes with extra padding in all dimensions can be generated, and the ROI is defined.

[0093] At box 6040, muscle structures in the ROI are segmented by, for example, the first 2050 of at least one artificial neural network and optionally based on the segmentation mask 2040, and thus the segmentation 2060 of the ROI 1010 associated with the rotator cuff in one or more medical images 1000 is determined.

[0094] At box 6050, muscle atrophy and fatty infiltration are classified based on both ROI 1010 segmentation 2060 and ROI 1010. Alternatively, the classification of muscle atrophy and fatty infiltration can be further based on tear classification.

[0095] At box 6060, based on both ROI 1010 segmentation 2060 and ROI 1010, muscle atrophy and fat infiltration are quantified.

[0096] At box 6070, both the classification and quantification of muscle atrophy and fatty infiltration are presented, and the examination report can be pre-filled optionally.

[0097] At box 6080, the classification and quantification of muscle atrophy and fatty infiltration are transferred to the target system, such as a PACS workstation, PACS storage, and / or RIS (Radiation Information System).

[0098] The exemplary clinical processing pipeline 6000 facilitates the automated classification and quantification of muscle mass, and thus improves postoperative assessment for rotator cuff tears.

[0099] Figure 3 and Figure 4 The processing pipelines 5000 and 6000 are modular. This means that it is not necessary to implement all the boxes. Furthermore, additional boxes can be implemented. For example, uploading results to RIS and / or PACS at boxes 5070 and 6080 respectively can be optional. Preprocessing at boxes 5020, 5030 and 6020, 6030 respectively can be optional.

[0100] Figure 5 This is a flowchart of method 3000 according to various examples. Method 3000 involves determining the quantification of at least one characteristic of a muscle structure including a tendon (such as a rotator cuff) based on one or more medical images 1000 depicting muscle structures including a patient's tendon by using at least one artificial neural network (such as at least one artificial neural network of system 2000).

[0101] The optional boxes are marked with dashed lines.

[0102] When loading program code, method 3000 can be executed by a computer including at least one processing unit, or by... Figure 2 The system 2000 is executed. The details of method 3000 are described below.

[0103] At box 3010, for example via Figure 2 The system 2000 acquires one or more medical images 1000. The one or more medical images 1000 depict muscular structures, including the patient's tendons.

[0104] Medical images 1000 can be loaded from a picture archiving system (PACS). Box 3010 may include control of the MRI unit to acquire images. Medical images 1000 can be loaded from memory. Alternatively, during scanning, medical images 1000 can be received directly from a medical imaging scanner to perform real-time determination of the quantification of at least one characteristic of the rotator cuff.

[0105] At box 3020, at least one artificial neural network is used to determine the quantification of at least one characteristic of the muscle structure. This at least one artificial neural network can be based on… Figure 2 The system was implemented in 2000.

[0106] Optionally, at box 3010, the acquisition of one or more medical images 1000 may include preprocessing the one or more medical images 1000 based on a landmark detection algorithm configured to detect a region of interest 1010 in the one or more medical images 1000. Such a landmark detection algorithm may be a machine learning algorithm, see, for example, US9 792 531 B2. The region of interest 1010 is associated with muscle structures. Additionally or alternatively, the above-described image preprocessing techniques may be applied before and / or after detecting the ROI 1010 by the landmark detection algorithm.

[0107] Optionally, at box 3030, for example via Figure 2 A convolutional neural network 2020 determines a pre-segmentation of the region of interest associated with muscle structures. The convolutional neural network 2020 can provide a pixel probability map 2030 as output. By applying a threshold comparison to the probability values ​​of the pixel probability map 2030 and selecting the largest neighboring region, the pre-segmentation can be obtained as a pixel mask 2040.

[0108] The pre-segmentation of ROI 1010 determined by the convolutional neural network 2020 can be based on a low-resolution version of ROI 1010 of one or more medical images 1000, for example, by applying downsampling to one or more medical images 1000, which can accelerate the determination of pre-segmentation.

[0109] Optionally, at box 3040, using the first 2050 of at least one artificial neural network, a segmentation 2060 of a region of interest 1010 associated with muscle structures in one or more medical images 1000 is determined. The segmentation 2060 may be determined based on pre-segmentation, i.e., a pixel mask 2040, or based on a pixel probability map 2030. The segmentation 2060 of the region of interest 1010 determined by the first 2050 of at least one artificial neural network may further be based on one or more medical images 1000 or ROI 1010.

[0110] Additionally or alternatively, segmentation 2060 may include bounding boxes that can be determined based on pixel mask 2040. As a general rule, the bounding box may be a rectangle (in two dimensions) or a cuboid (in three dimensions) that is the smallest volume enclosing all relevant features. Determining the bounding box may include determining the positional coordinates (e.g., x, y, and z coordinates), size parameters (e.g., height, width, and depth), and orientation parameters (e.g., angles θx, θy, and θz, which are angles about the x, y, and z axes defined relative to the medical image 1000) for each 2-D medical image or 3-D medical slice. The described coordinates and parameters may be determined such that the bounding box indicates ROI 1010.

[0111] Based on various examples, the full resolution or raw resolution of ROI 1010 can be selected based on pre-segmentation, i.e., pixel mask 2040. Then, the full resolution or raw resolution of ROI 1010 can be fed into the first 2050 of at least one artificial neural network to determine segmentation 2060.

[0112] Optionally or alternatively, method 3000 may further include at block 3020 using at least a second 2070 of at least one artificial neural network to determine a quantified value indicating at least one characteristic based on segmentation 2060 of region of interest 1010 and one or more medical images 1000.

[0113] According to various examples, when the at least one feature comprises multiple features, the at least one artificial neural network, such as system 2000, can be trained separately for each individual feature. That is, the quantization of each individual feature can be determined using at least one artificial neural network with different parameter values, such as several systems 2000 having the same network architecture but different parameter values.

[0114] Optionally or alternatively, when the at least one feature comprises multiple features, the quantization of the at least one feature can be jointly determined based on the same artificial neural network in the at least one artificial neural network, such as the same system 2000. For example, the at least one artificial neural network may include multiple decoder branches 2070a-2070d configured to determine the quantization of multiple features based on the same input, such as a combination of segmentation 2060 and medical image 1000, or specifically based on a shared latent feature set determined based on one or more medical images 1000. The multiple decoder branches 2070a-2070d may be part of a second 2070 of the at least one artificial neural network.

[0115] According to various examples, when one or more medical images 1000 are multi-slice images or include multi-slice images, method 3000 may further include, at box 3050, determining a reference point for a region of interest 1010 based on segmentation 2060, for example, using the location of detected landmarks as a reference point for ROI 1010; and at box 3060, selecting a given slice from multiple slices of one or more medical images 1000 based on the reference point. Then, the quantification of at least one characteristic may be determined based on the appearance of muscle structures in the given slice.

[0116] According to this disclosure, for example, tear coordinates, such as the corresponding coordinates of the start and end points of the tear, can be extracted from a segmentation mask (e.g., segmentation 2060) by finding the farthest voxel in the segmentation mask and projecting it onto a slice at the centroid of the segmentation mask. Alternatively, the tear coordinates can be determined by first determining the centroid of the segmentation mask, then selecting an image slice at the centroid, and measuring only the two farthest voxels of the segmentation mask in the selected image slice. Alternatively or alternatively, the tear coordinates can be determined based on the distance between the two farthest voxels of each slice of the segmentation mask, for example, by selecting the one with the longest length to determine the coordinates, or by using further machine learning algorithms to determine the optimal slice from which the coordinates are extracted.

[0117] Optionally or alternatively, the at least one artificial neural network may include a first artificial neural network configured to determine the presence or absence and / or type of tear in muscle structures. The at least one artificial neural network may further include a second artificial neural network configured to determine the classification and, optionally, quantification of muscle atrophy or fatty infiltration. The second artificial neural network may receive the output of the first artificial neural network as input.

[0118] According to various examples, after determining the quantification of at least one characteristic of a muscle structure, method 3000 may further include at least one of the following: presenting a visualization of the muscle structure including the value of at least one characteristic, pre-filling an inspection report including the value of at least one characteristic, or sending the value of at least one characteristic to a PACS.

[0119] According to this disclosure, method 3000 facilitates the automatic and precise quantification of at least one property of muscle structure by using at least one artificial neural network. Compared to manual methods, method 3000 can significantly reduce the time required to quantify such properties and provides consistent and reproducible results, addressing both inter- and intra-reader variability in current clinical practice. Consequently, treatment and surgical planning are greatly improved.

[0120] Figure 6 This is a block diagram of system 4000 based on various examples. System 4000 provides the functionality to quantify at least one characteristic of muscle structure based on method 3000.

[0121] System 4000 may include at least one processor 4020, at least one memory 4030, and at least one input / output interface 4010. At least one processor 4020 is configured to load program code from at least one memory 4030 and execute the program code. When executing the program code, at least one processor 4020 executes method 3000.

[0122] According to this disclosure, medical imaging scanners, such as CT scanners, MRI scanners, ultrasound scanners, or X-ray scanners, may include... Figure 6 The system 4000. Medical imaging scanners can determine the quantification of at least one characteristic of muscle structures while performing a scan of a patient's shoulder.

[0123] Alternatively, system 4000 may be embedded in or connected to a medical imaging scanner, and therefore the medical imaging scanner may also be configured to perform method 3000.

[0124] In summary, techniques have been described to facilitate the quantification and / or classification of at least one characteristic of muscle structures, including at least one muscle and at least one tendon, such as the rotator cuff, and thus significantly improve surgical planning for the treatment of tears in muscle structures. By using artificial intelligence (AI) techniques, such as at least one artificial neural network, at least one characteristic of a muscle structure can be determined automatically and accurately. Compared to manual methods, such AI-based techniques can significantly reduce the time required to determine such characteristics and provide consistent and reproducible results, addressing both inter- and intra-reader variability in current clinical practice.

[0125] Although this disclosure has been shown and described with respect to certain preferred embodiments, equivalents and modifications will occur to others skilled in the art upon reading and understanding this specification. This disclosure includes all such equivalents and modifications and is limited only by the scope of the appended claims.

Claims

1. A computer-implemented method, comprising: Obtain one or more medical images depicting the patient's muscle structures, wherein the muscle structures include at least one muscle and at least one tendon. Pre-segmentation of the region of interest associated with muscle structure; Using at least one of at least one artificial neural networks, segmentation of regions of interest associated with muscle structures in the one or more medical images is determined, wherein the segmentation is determined based on pre-segmentation, and Based on the segmentation of the region of interest and one or more medical images, the quantification of at least one property of the muscle structure is determined using a second of at least one artificial neural network.

2. The computer-implemented method according to claim 1, A convolutional neural network is used to determine the pre-segmentation. The convolutional neural network provides a pixel probability map as output. The pre-segmentation is obtained by applying a threshold to the probability values ​​of the pixel probability map, comparing them, and selecting the largest adjacent region.

3. The computer-implemented method according to claim 2, The segmentation includes a bounding box determined based on a pixel mask.

4. The computer-implemented method of claim 1, wherein determining the quantization of at least one characteristic of a muscle structure using a second of at least one artificial neural network based on the segmentation of the region of interest and one or more medical images comprises: Using at least a second of the at least one artificial neural network, a quantized value indicating the at least one characteristic is determined based on the segmentation of the region of interest and the one or more medical images.

5. The computer-implemented method according to claim 1, The one or more medical images mentioned above are multi-slice images. The method further includes: Reference points for the region of interest are determined based on segmentation. Selecting a given slice from multiple slices of one or more medical images based on a reference point. The quantification of at least one characteristic is determined based on the appearance of the muscle structure in a given slice.

6. The computer-implemented method according to claim 1, The acquisition of one or more medical images includes preprocessing one or more medical images based on a landmark detection algorithm configured to detect regions of interest in the one or more medical images, the regions of interest being associated with muscle structures.

7. The computer-implemented method according to claim 1, The at least one characteristic includes multiple characteristics. The second of the at least one artificial neural network includes multiple decoder branches configured to determine the quantization of the multiple features based on a shared latent feature set determined based on the one or more medical images.

8. The method according to claim 1, The quantification of the at least one characteristic includes at least one of the length, width, thickness of the tendon tear, or the location of the tendon connection.

9. The method according to claim 8, The second of the at least one artificial neural network includes a first artificial neural network configured to determine at least one of the following: 1) the presence or absence of a tendon tear in the at least one tendon, or 2) the type of tendon tear in the at least one tendon.

10. The method according to claim 9, The second of the at least one artificial neural network includes a second artificial neural network configured to determine at least one of the length, width, thickness, or location of tendon connection of a tendon tear. The second artificial neural network receives the output of the first artificial neural network as its input.

11. The method of claim 1, wherein the at least one characteristic includes muscle atrophy of at least one muscle of the muscle structure.

12. The method according to claim 11, The second of the at least one artificial neural network is configured to determine the quantification of muscle atrophy.

13. The method according to claim 1, The at least one of the characteristics mentioned includes fatty infiltration of at least one muscle in the muscle structure.

14. The method according to claim 13, The second of the at least one artificial neural network is configured to determine fat infiltration.

15. The method of claim 1, further comprising: The classification of at least one property or at least one additional property of muscle structure is determined using at least one artificial neural network.

16. A system comprising: At least one processor; as well as At least one memory, wherein when loading and executing program code from the at least one memory, the at least one processor is configured to: Obtain one or more medical images depicting the patient's muscle structures, wherein the muscle structures include at least one muscle and at least one tendon. Pre-segmentation of the region of interest associated with muscle structure; Using at least one of at least one artificial neural networks, segmentation of regions of interest associated with muscle structures in the one or more medical images is determined, wherein the segmentation is determined based on pre-segmentation, and Based on the segmentation of the region of interest and one or more medical images, the quantification of at least one property of the muscle structure is determined using a second of at least one artificial neural network.

17. A medical imaging scanner comprising the system of claim 16.

18. A non-transitory computer-readable medium storing computer program instructions, which, when executed by a processor, cause the processor to perform operations, comprising: Obtain one or more medical images depicting the patient's muscle structures, wherein the muscle structures include at least one muscle and at least one tendon. Pre-segmentation of the region of interest associated with muscle structure; Using at least one of at least one artificial neural networks, segmentation of regions of interest associated with muscle structures in the one or more medical images is determined, wherein the segmentation is determined based on pre-segmentation, and Based on the segmentation of the region of interest and one or more medical images, the quantification of at least one property of the muscle structure is determined using a second of at least one artificial neural network.

Images