Determine the rotation of the patient's chest in a medical image

Abstract

A computer-implemented method for determining a rotation of a patient's chest in a medical image is provided. The method comprises: receiving the medical image of the patient; processing the medical image to determine scapula space data associated with the patient's scapula; and using the scapula space data to determine a rotation of the patient's chest relative to at least one reference axis.

Description

Technical Field

[0001] The present invention relates to a method and a system for determining the rotation of a patient's chest in a medical image. Background Art

[0002] Ensuring non-rotational positioning of the patient in chest posteroanterior (PA) radiographs is an important quality aspect, as otherwise the position of relevant anatomical structures in the image may appear deviated or obscured from view. For example, the lung parenchyma to be analyzed may be shadowed by chest structures when not acquired at the correct angle. Previous solutions consisted of estimating the rotation angle by locating landmarks (particularly the clavicle tips and spinous processes) and measuring the deviation from symmetry. In some cases (e.g., scoliosis), this is not a good indicator of overall chest rotation. Summary of the Invention

[0003] To better address one or more of these issues, the present invention, in a first aspect, provides a computer-implemented method for determining a rotation of a patient's chest in a medical image. The method comprises: receiving a medical image of a patient; processing the medical image to determine scapula space data associated with the patient's scapula; and using the scapula space data to determine a rotation of the patient's chest relative to at least one reference axis.

[0004] Therefore, the method of patient rotation detection proposed in this article allows accurate reflection of the rotation of the patient's chest (i.e., thoracic cavity), especially the rotation of its internal organs (such as the heart and lungs), even in severe cases such as scoliosis.

[0005] The method may further include, before determining rotation, processing the scapula spatial data to confirm scapula symmetry of the patient, i.e., symmetry of the position of the scapula with respect to lateral scapular movement. This may serve as a form of "sanity check." Confirming scapula symmetry may include identifying the acromion and coracoid processes of the scapula and determining their lengths; calculating a ratio of the coracoid process length to the acromion process length for each of the left and right scapulae; and confirming scapula symmetry based on the calculated ratios. Confirming scapula symmetry based on the calculated ratios may include determining that the ratio of the left scapula corresponds to the ratio of the right scapula within an acceptable tolerance. In some examples, a similarity or distance metric may be used to compare the ratios to confirm symmetry or asymmetry. Additionally or alternatively, confirming scapula symmetry may be based on one or more other parameters selected from the group consisting of: the inferior angle of the scapula; the angle between the clavicle and the acromion; the angle between the lateral edge of the scapula and the humerus; and any combination thereof. Thus, broadly speaking, the method can include confirming scapular symmetry based on comparing the values of one or more scapular position parameters, particularly parameters related to the relative positions of bony structures within or surrounding the scapula, for the left and right scapulae. In other words, processing the scapular spatial data to confirm scapular symmetry can include confirming the absence of lateral scapular movement, or at least the absence of unequal lateral scapular movement, between the scapulae.

[0006] Processing a medical image to determine scapula spatial data may include segmenting the medical image to determine a contour (i.e., a boundary or edge) of the patient's scapula. The method may include performing image segmentation using a convolutional neural network trained using annotated sample images. The method may also include, before segmenting the medical image, processing the image to locate one or more landmarks to facilitate image segmentation.

[0007] Patient rotation can be determined geometrically. Thus, determining the rotation of the patient's chest can include using the scapula spatial data to calculate a scapula line connecting corresponding points on the patient's scapulae, and determining the rotation using a displacement between the scapula line and at least one reference axis. In one example, the scapula line connects two points on the contour of the corresponding scapula, where each point is a vertex on the inferior corner of the edge of the corresponding scapula, or in other words, the lowest relative to the patient. In other examples, other suitable anatomical reference points are selected for the scapula line. Determining the rotation can include determining the patient's rotation within the image plane and / or determining the patient's rotation outside the image plane. In the case of determining the in-plane patient rotation, the at least one reference axis includes a detector plane axis, and the displacement includes an angular displacement between the scapula line and the detector plane axis, and determining the rotation includes determining the in-plane rotation of the patient using the angular displacement. For example, the detector plane axis can be horizontal relative to the image or detector, but it will be appreciated that any orientation can be selected for the detector plane axis, including at least a vertical axis. In the case of determining out-of-plane patient rotation, at least one reference axis may include, for example, the mid-axis of the patient's body, while the displacement includes a linear displacement between the midpoint of the scapular line and the mid-axis, and the determination of the rotation includes using the said linear displacement to determine the patient's out-of-plane rotation.

[0008] In addition to or in lieu of geometric determination of patient rotation, patient rotation (and / or scapula symmetry) can be determined empirically, for example using machine learning. Thus, the method can include using a trained machine learning model to determine the rotation of the patient's chest relative to at least one reference axis using the scapula spatial data.

[0009] Thus, in a second aspect, a method for training the machine learning model of the first aspect is provided. The training data used to train the model may include medical images showing varying degrees of patient rotation, appropriately labeled by qualified radiographers. The training data may also include medical images of different age groups, gender groups, and from different regions to make the model more robust. The training data may also include artificially modified data of this type or synthetic data similar to the original patient data.

[0010] As used herein, the term "scapula spatial data" refers to data that can be used to determine the position and / or shape of the scapula. In particular, the scapula spatial data may include data indicating the position of the contour of the scapula, such as data generated by an image segmentation process.

[0011] The method of the first aspect and / or the second aspect may be computer-implemented.

[0012] According to a third aspect, there is provided a computing system configured to perform the method of the first and / or second aspect.

[0013] According to a fourth aspect, there is provided a computer program product comprising instructions which, when executed by a computing system, implement or cause the computing system to perform the method of the first and / or second aspect.

[0014] According to a fifth aspect, there is provided a computer-readable medium comprising instructions which, when executed by a computing system, implement or cause the computing system to perform the method of the first and / or second aspects.

[0015] The present invention may include one or more aspects, examples or features in isolation or combination, regardless of whether they are specifically disclosed in that combination or in isolation. Any optional feature or sub-aspect of one of the above aspects may apply to any other aspect as appropriate.

[0016] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] A detailed description will now be given, by way of example only, with reference to the accompanying drawings, in which:

[0018] Figure 1 illustrates image segmentation for locating anatomical structures including a patient's scapula;

[0019] Figure 2 The diagram shows how to determine the relative lengths of the acromion and coracoid processes of the scapula on both sides, which is used to confirm the symmetry of the scapular position;

[0020] Figure 3A and 3B illustrates the use of scapula space data to determine both in-plane and out-of-plane patient rotation; and

[0021] Figure 4 A computing system that can be used in accordance with the present disclosure is illustrated. DETAILED DESCRIPTION

[0022] According to the present invention, a rotation of a patient's chest in a medical image is determined by processing the medical image to determine scapula spatial data associated with the patient's scapula and using the scapula spatial data to determine a rotation of the patient's chest relative to at least one reference axis.

[0023] Figure 1The diagram illustrates the use of image segmentation to locate anatomical structures including the patient's scapula in a medical image 100, thereby obtaining scapula spatial data. As shown in the figure, the result of the image segmentation is the positioning of the scapula and its contour 102. In addition, the contours of the lungs 104 and the clavicle 106 can be located. The output of this image segmentation is spatial data, which at least includes scapula spatial data indicating the shape and position of the scapula. The image 100 can be segmented using a convolutional neural network trained using annotated sample images. In order to achieve better performance with a comparable annotation workload, landmarks may be located first before determining the contour, as described in, for example, "Robust chest x-ray quality assessment using convolutional neural networks and atlas regularization" (Medical Imaging 2020: Image Processing, International Society for Optics and Photonics. 2020. p. 113131L) by VON BERG, J. et al.

[0024] Using the scapula spatial data, the rotation of the patient's chest relative to at least one reference axis can then be determined. Using the shape and positioning of the scapula and optionally also using the lung fields to assess the patient's rotation provides a better reflection of the rotation of the entire chest. However, the scapula has increased mobility compared to the clavicle and spine, with greater degrees of freedom, capable of medial and lateral rotation, abduction, adduction, etc. Therefore, the present disclosure contemplates performing an optional integrity check to confirm that the scapula is symmetrical relative to the patient (i.e., there is no lateral movement of the scapula) before using the scapula spatial data to determine the patient's rotation.

[0025] Figure 2 Confirmation of symmetry of scapular position is shown. The acromion and coracoid processes of the scapula are identified bilaterally, their endpoints are identified by segmentation, and their lengths are determined. Figure 2 Indicated are the length of the left acromion 202L, the length of the left coracoid 204L, the length of the right acromion 202R, and the length of the right coracoid 204R. Calculating Ratios and In the absence of lateral scapular movement, the left ratio R L Corresponding to the right ratio R R It will be appreciated that other parameters may additionally or alternatively be used to assess scapular symmetry, i.e. the inferior scapular angle, the angle between the clavicle and the acromion, the angle between the lateral edge of the scapula and the humerus, etc.

[0026] Figure 3A and 3B The use of scapula space data to determine both in-plane and out-of-plane patient rotation is illustrated.

[0027] Out-of-plane patient rotations can be determined with reference to a suitable reference axis, which in this case is the medial axis of the patient's body, which is located at Figure 3A Indicated as line A in the figure. Such an axis of rotation can be robustly determined by registering a chest atlas to an image annotated with that axis, similar to the technique described in "Robust chest x-ray quality assessment using convolutional neural networks and atlas regularization" by VON BERG, J. et al. (Medical Imaging 2020: Image Processing, International Society for Optics and Photonics. 2020. p. 113131L). Calculate the lowest point of the lower edge of the scapula [P1, P2]. Figure 3A The scapula line connects points P1 and P2 as indicated by line B in the figure. Find the midpoint of line B, which is Figure 3A The intersection of line A and line B is found, which is indicated as point P4. The distance between P3 and P4 corresponds to the component of the out-of-plane patient rotation. Figure 3B This is also illustrated in the diagram. Figure 3B The phenomenon is shown with respect to the source 302 and detector 304 of the X-ray system 300. The more the patient is rotated, the greater the linear displacement between points P3 and P4.

[0028] The in-plane patient rotation can be determined with reference to the detector plane axis. For example, Figure 3A Line C is illustrated as the horizontal axis in the image or detector plane. The angular displacement between lines B and C corresponds to an in-plane rotation of the patient.

[0029] Such image information can be used to train a recurrent convolutional neural network to improve the accuracy of patient rotation determination. For example, a trained recurrent convolutional neural network can be used to estimate the rotation angle of the contour 102 represented by the scapula spatial data. This method can be followed et al. "CNN-based pose-estimation of musculoskeletal X-ray images" (Philips OcuPai conference.2021), according to which it was successfully applied to a similar task, namely, estimating the pose and flexion of the ankle joint from a single radiograph.

[0030] The present disclosure contemplates many variations of the above-described systems and methods. For example, the patient rotation determination methods disclosed herein can be used alone or in conjunction with existing methods using the clavicle and spine. In another example, the systems and methods disclosed herein use scapula spatial data generated from a process for locating the scapula and analyzing its shape and orientation for the purpose of a stand-alone "scapula checker." For this purpose, the lung fields may also be located. In yet another example, the ratio R L and R R , to generate a nomogram with an acceptable tolerance band. If the values calculated for the patient under investigation appear on the nomogram, it can be assumed that the scapula is symmetrical with respect to the patient. In yet another example, thoracic rotation is estimated based on redundant X-ray contours in addition to or instead of the scapula contour.

[0031] The systems and methods disclosed herein may be applied in the context of X-ray quality assessment.

[0032] Figure 4 An exemplary computing system 800 that can be used in accordance with the systems and methods disclosed herein is illustrated. Computing system 800 can form part of or include any desktop, laptop, server, or cloud-based computing system. Computing system 800 includes at least one processor 802 that executes instructions stored in memory 804. For example, the instructions can be instructions for implementing the functions described as being performed by one or more of the components discussed above, or instructions for implementing one or more of the methods described above. Processor 802 can access memory 804 via system bus 806. In addition to storing executable instructions, memory 804 can also store session input, scores assigned to session input, and the like.

[0033] The computing system 800 additionally includes a data store 808, which is accessible by the processor 802 via the system bus 806. The data store 808 may include executable instructions, log data, and the like. The computing system 800 also includes an input interface 810 that allows external devices to communicate with the computing system 800. For example, the input interface 810 may be used to receive instructions from an external computer device, from a user, and the like. The computing system 800 also includes an output interface 812 that interfaces the computing system 800 with one or more external devices. For example, the computing system 800 may display text, images, and the like via the output interface 812.

[0034] It is contemplated that external devices communicating with the computing system 800 via the input interface 810 and the output interface 812 may be included in an environment providing substantially any type of user interface with which a user can interact. Examples of user interface types include graphical user interfaces, natural user interfaces, and the like. For example, a graphical user interface may accept input from a user using (one or more) input devices such as a keyboard, mouse, remote control, and the like, and provide output on an output device such as a display. Furthermore, a natural user interface may enable a user to interact with the computing system 800 in a manner that is not constrained by input devices such as a keyboard, mouse, remote control, and the like. In contrast, a natural user interface may rely on speech recognition, touch and stylus recognition, gesture recognition on and near the screen, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, machine intelligence, and the like.

[0035] Additionally, although illustrated as a single system, it should be understood that the computing system 800 may be a distributed system. Thus, for example, several devices may communicate via a network connection and may jointly perform the tasks described as being performed by the computing system 800.

[0036] The various functions described herein can be implemented in hardware, software, or any combination thereof. If implemented in software, the functions can be stored on a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media. Computer-readable storage media can be any available storage medium that a computer can access. By way of example and not limitation, such computer-readable storage media can include FLASH storage media, RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer. Disks and optical disks as used herein include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks (BDs), wherein disks typically copy data magnetically, while optical disks typically copy data optically with lasers. In addition, propagated signals are not included within the scope of computer-readable storage media. Computer-readable media also include communication media, including any media that facilitates transferring a computer program from one place to another. For example, a connection can be a communication medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology such as infrared, radio, microwave, etc., then the definition of communications media includes coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology such as infrared, radio, microwave, etc. Combinations of the above should also be included within the scope of computer-readable media.

[0037] Alternatively or additionally, the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), program-specific integrated circuits (ASICs), program-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0038] It will be appreciated that the above-mentioned circuits may have other functions in addition to those mentioned, and that these functions may be performed by the same circuit.

[0039] Applicants hereby disclose each individual feature described herein in isolation and any combination of two or more such features, to the extent such feature or combination is capable of being performed based on the common general knowledge of a person skilled in the art based on the specification as a whole, regardless of whether such feature or combination of features solves any problem disclosed herein, and without limiting the scope of the claims. Applicants indicate that aspects of the present invention may consist of any such individual feature or combination of features.

[0040] It should be noted that embodiments of the present invention are described with reference to different categories. In particular, some examples are described with reference to methods, while others are described with reference to apparatuses. However, a person skilled in the art will appreciate from the description that, unless otherwise indicated, this application discloses any combination of features related to different categories, in addition to any combination of features belonging to one category. However, all features can be combined to provide synergistic effects that are greater than the simple sum of the features.

[0041] Although the present invention has been described and illustrated in detail in the drawings and the foregoing description, such illustration and description should be considered illustrative rather than restrictive. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments may be understood and effected by those skilled in the art from a study of the drawings, this disclosure, and the appended claims.

[0042] The word "comprising" does not exclude other elements or steps.

[0043] The word "a" or "an" does not exclude a plurality. In addition, the word "a" or "an" used herein should generally be interpreted as meaning "one or more", unless otherwise specified or clear from the context to indicate a singular form.

[0044] A single processor or other unit may fulfill the functions of several items recited in the claims.

[0045] The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0046] The computer program may be stored / distributed on a suitable medium, such as an optical storage medium or solid-state medium provided together with or as part of other hardware, but the computer program may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems.

[0047] Any reference signs in the claims should not be construed as limiting their scope.

[0048] Unless otherwise specified or clear from the context, the phrases "one or more of A, B, and C," "at least one of A, B, and C," and "A, B, and / or C" as used herein are intended to refer to all possible permutations of one or more of the listed items. That is, the phrase "X includes A and / or B" is satisfied by any of the following instances: X includes A; X includes B; or X includes both A and B.

Claims

1. A computer-implemented method for determining a rotation of a patient's chest in a medical image, the method comprising: receiving the medical image of the patient; processing the medical image to determine scapular spatial data associated with the patient's scapula; and determining a rotation of the patient's chest relative to at least one reference axis using the scapula spatial data, Wherein, before determining the rotation, a sanity check is performed by processing the scapula spatial data to confirm the symmetry of the patient's scapula.

2. The method according to claim 1, wherein Confirming the symmetry of the patient's scapula includes: identifying the acromion and coracoid process of the scapula and determining the lengths of the acromion and coracoid process; calculating a ratio of the coracoid process length to the acromion process length for each of the left and right scapulae; and The symmetry of the scapula is confirmed based on the calculated ratio.

3. The method according to claim 2, wherein: Confirming the symmetry of the scapulae based on the calculated ratios includes determining that the ratio for a left scapula corresponds to the ratio for the right scapula within an acceptable tolerance.

4. The method according to claim 2 or 3, wherein: Confirming the symmetry of the scapula is based on one or more parameters selected from the group consisting of: a lower angle of the scapula; an angle between the clavicle and the acromion; an angle between the lateral edge of the scapula and the humerus; and any combination thereof.

5. The method according to claim 4, wherein Confirming symmetry of the patient's scapula is based on comparing values of one or more of the parameters for the left scapula and the right scapula.

6. The method according to any one of claims 1 to 3, wherein: Processing the medical image to determine the scapula spatial data includes segmenting the medical image to determine a contour of the patient's scapula.

7. The method of claim 6, comprising performing the segmentation using a convolutional neural network trained using annotated sample images.

8. The method according to claim 6, further comprising: Prior to segmenting the medical image to determine the contour of the patient's scapula, the image is processed to locate one or more landmarks for facilitating the segmentation.

9. The method according to any one of claims 1 to 3, wherein: Determining the rotation of the patient's chest includes using the scapula space data to calculate a scapula line connecting corresponding points on the patient's scapulae, and determining the rotation using a displacement between the scapula line and the at least one reference axis.

10. The method according to claim 9, wherein: The scapular line connects two points on the contour of the corresponding scapula, each of the points being a vertex on the lower corner of the edge of the corresponding scapula.

11. The method according to claim 9, wherein The at least one reference axis comprises a detector plane axis, wherein the displacement comprises an angular displacement between the scapula line and the detector plane axis, and wherein determining the rotation comprises determining an in-plane rotation of the patient using the angular displacement.

12. The method according to claim 9, wherein The at least one reference axis comprises a medial axis of the patient's body, wherein the displacement comprises a linear displacement between a midpoint of the scapular line and the medial axis, and wherein determining the rotation comprises determining an out-of-plane rotation of the patient using the linear displacement.

13. The method of any one of claims 1-3, comprising using a trained machine learning model to determine the rotation of the patient's chest relative to at least one reference axis using the scapula space data.

14. A computing system for determining a rotation of a patient's chest in a medical image, the computing system comprising: means for receiving the medical image of the patient; a module for processing the medical image to determine scapula spatial data associated with the patient's scapula; as well as means for determining a rotation of the patient's chest relative to at least one reference axis using the scapula spatial data, Wherein, before determining the rotation, a sanity check is performed by processing the scapula spatial data to confirm the symmetry of the patient's scapula.

15. The computing system of claim 14, wherein: Confirming the symmetry of the patient's scapula includes: identifying the acromion and coracoid process of the scapula and determining the lengths of the acromion and coracoid process; calculating a ratio of the coracoid process length to the acromion process length for each of the left and right scapulae; and The symmetry of the scapula is confirmed based on the calculated ratio.

16. The computing system of claim 15, wherein: Confirming the symmetry of the scapulae based on the calculated ratios includes determining that the ratio for a left scapula corresponds to the ratio for the right scapula within an acceptable tolerance.

17. The computing system according to claim 15 or 16, wherein: Confirming the symmetry of the scapula is based on one or more parameters selected from the group consisting of: a lower angle of the scapula; an angle between the clavicle and the acromion; an angle between the lateral edge of the scapula and the humerus; and any combination thereof.

18. The computing system of claim 17, wherein: Confirming symmetry of the patient's scapula is based on comparing values of one or more of the parameters for the left scapula and the right scapula.

19. The computing system of any one of claims 14 to 16, wherein: The means for processing the medical image to determine the scapula spatial data includes means for segmenting the medical image to determine the contour of the patient's scapula.

20. The computing system of claim 19, comprising means for performing the segmentation using a convolutional neural network trained using annotated sample images.

21. The computing system of claim 19, further comprising: Means for, prior to segmenting the medical image to determine the contour of the patient's scapula, processing the image to locate one or more landmarks for facilitating the segmentation.

22. The computing system of any one of claims 14-16, wherein: The means for determining the rotation of the patient's chest includes means for calculating a scapula line connecting corresponding points on the patient's scapula using the scapula spatial data, and means for determining the rotation using a displacement between the scapula line and the at least one reference axis.

23. The computing system of claim 22, wherein: The scapular line connects two points on the contour of the corresponding scapula, each of the points being a vertex on the lower corner of the edge of the corresponding scapula.

24. The computing system of claim 22, wherein: The at least one reference axis comprises a detector plane axis, wherein the displacement comprises an angular displacement between the scapula line and the detector plane axis, and wherein determining the rotation comprises determining an in-plane rotation of the patient using the angular displacement.

25. The computing system of claim 22, wherein: The at least one reference axis comprises a medial axis of the patient's body, wherein the displacement comprises a linear displacement between a midpoint of the scapular line and the medial axis, and wherein determining the rotation comprises determining an out-of-plane rotation of the patient using the linear displacement.

26. The computing system of any of claims 14-16, comprising means for using a trained machine learning model to determine the rotation of the patient's chest relative to at least one reference axis using the scapula space data.

27. A computer program product comprising instructions which, when executed by a computing system, cause the computing system to perform the method according to any one of claims 1 to 13.

Images