Determining the rotation of a patient's chest in medical images.

Abstract

A computer-implemented method for determining rotation of a patient's thorax in a medical image is provided, the method comprising receiving a medical image of the patient, processing the medical image to determine scapular space data related to the patient's scapula, and determining a rotation of the patient's thorax relative to at least one reference axis using the scapular space data.

Description

Technical Field

[0001] The present invention relates to a method and system for determining the rotation of a patient's chest in a medical image.

Background Art

[0002] Ensuring the non-rotated positioning of the patient is an important quality aspect in posteroanterior (PA) chest X-ray imaging because otherwise the location of the relevant anatomical structures in the image may appear shifted or obscured in appearance.

Summary of the Invention

Problems to be Solved by the Invention

[0003] For example, the lung parenchyma to be analyzed may be shadowed by the thoracic cage structure if not acquired at the correct angle. Conventional solutions include locating landmarks, particularly the acromion and spinous processes, to estimate the rotation angle and measuring the deviation from a symmetric constellation. In some cases (e.g., scoliosis), this is not a good indicator of overall chest rotation.

Means for Solving the Problems

[0004] To better address one or more of these concerns, in a first aspect of the present invention, a computer-implemented method for determining the rotation of a patient's chest in a medical image is provided. The method includes receiving a medical image of a patient, processing the medical image to determine scapular spatial data regarding the patient's scapulae, and using the scapular spatial data to determine the rotation of the patient's chest relative to at least one reference axis.

[0005] Thus, the approach to detecting patient rotation proposed herein enables the accurate reflection of the rotation of the patient's chest (i.e., thoracic cage), particularly the rotation of internal organs such as the heart and lungs, even in severe cases such as scoliosis.

[0006] This method may further include processing scapular spatial data before determining rotation to confirm the symmetry of the patient's scapula, i.e., the symmetry of the scapular position with respect to external rotation of the scapula. This may be performed as a form of “health check.” Confirming the symmetry of the patient's scapula may include identifying the acromion and coracoid process of the scapula and determining their lengths, calculating the ratio of the length of the coracoid process to the length of the acromion for each of the left and right scapulae, and confirming the symmetry of the scapula based on the calculated ratio. Confirming the symmetry of the scapula based on the calculated ratio may include determining that the ratio of the left scapula matches the ratio of the right scapula within an acceptable range. In some cases, similarity or distance metrics may be used to compare the ratios and thereby confirm symmetry or other. Additionally or alternatively, confirming the symmetry of the scapula may be based on one or more other parameters selected from the group consisting of the inferior angle of the scapula, the angle between the clavicle and the acromion, the angle between the lateral edge of the scapula and the humerus, and any combination thereof. Therefore, broadly speaking, this method may include a step of verifying scapular symmetry based on comparing the values ​​of one or more scapular position parameters, particularly parameters relating to the relative position of bone structures within or around the scapula, for the left and right scapulae. In other words, processing scapular spatial data to verify the symmetry of a patient's scapula may include verifying that there is no scapular external rotation in either scapula, or that there is no unequal scapular external rotation.

[0007] Determining scapular spatial data by processing medical images may include segmenting the medical images to determine the contours (i.e., boundaryaries or borders) of the patient's scapula. This method may include using a convolutional neural network trained on annotated sample images to perform image segmentation. This method may further include processing the images before segmenting them to locate one or more landmarks to facilitate image segmentation.

[0008] Patient rotation can be determined geometrically. Therefore, determining the rotation of the patient's chest may involve using scapular spatial data to calculate a scapular line connecting corresponding points on the patient's scapulae, and determining the rotation using the displacement between the scapular line and at least one reference axis. In one example, the scapular line connects two points on the contour of each scapula, where each point is the lower angle of the boundary of each scapula, in other words, the lowest vertex relative to the patient. In other examples, other appropriate anatomical reference points are selected relative to the scapular line. Determining rotation may include determining patient rotation in the image plane and / or patient rotation outside the image plane. If in-plane patient rotation is determined, at least one reference axis includes the detector plane axis, the displacement includes the angular displacement between the scapular line and the detector plane axis, and the rotation determination involves using the angular displacement to determine the patient's in-plane rotation. The detector plane axis may be, for example, horizontal to the image or detector, but it will be understood that any orientation relative to the detector plane axis can be selected, including at least a vertical axis. When out-of-plane patient rotation is determined, at least one reference axis may be, for example, the medial axis of the patient's body, the displacement includes a linear displacement between the midpoint of the scapular line and the medial axis, and the determination of rotation includes determining the out-of-plane rotation of the patient using the linear displacement.

[0009] In addition to, or instead of, the geometric determination of the patient's rotation, the patient's rotation (and / or scapular symmetry) may be determined empirically, for example, using machine learning. Thus, the method may include using a trained machine learning model to determine the rotation of the patient's chest relative to at least one reference axis using scapular spatial data.

[0010] Accordingly, a method for training the machine learning model of the first embodiment is provided. The training data to be used when training the model may include medical images that show patient rotation to varying degrees and are appropriately labeled by a qualified radiographer. The training data may further include medical images from different age groups, sex groups, and different geographical locations to make the model more robust. The training data may also include artificially modified or synthetic data that looks similar to the original patient data.

[0011] As used herein, the term “scapular spatial data” refers to data that can be used to determine the position and / or shape of the scapula. In particular, scapular spatial data may include, for example, data indicating the position of the scapular contour resulting from the image segmentation process.

[0012] The methods of the first and / or second embodiments can be carried out by computer.

[0013] According to a third aspect, a computing system is provided that is configured to perform the methods of the first and / or second aspects.

[0014] According to a fourth aspect, a computer program product is provided which, when executed by a computing system, has instructions that cause the computing system to perform the methods of the first and / or second aspects.

[0015] According to a fifth aspect, a computer-readable medium is provided having instructions that cause a computing system to perform the methods of the first and / or aspects when executed by a computing system. The present invention may include one or more aspects, examples, or features, individually or in combination, whether or not they are specifically disclosed in combination or individually. Any feature or sub-aspect of one of the above aspects may be appropriately applied to any of the other aspects.

[0016] These and other aspects of the present invention will become apparent from and be explained with reference to the embodiments described below.

[0017] Here, a detailed explanation is given, merely as an example, with reference to the attached drawings. [Brief explanation of the drawing]

[0018] [Figure 1] A diagram showing image segmentation to locate anatomical structures, including the patient's scapula. [Figure 2] This diagram shows how to determine the relative lengths of the acromion and coracoid process of both scapulae to confirm the symmetry of scapular position. [Figure 3A] A diagram illustrating the determination of in-plane and out-of-plane rotation in a patient using scapular spatial data. [Figure 3B] A diagram illustrating the determination of in-plane and out-of-plane rotation in a patient using scapular spatial data. [Figure 4] A diagram illustrating a computing system that can be used in accordance with the present invention. [Modes for carrying out the invention]

[0019] According to this disclosure, rotation of the patient's chest in a medical image is determined by processing the medical image to determine scapular spatial data relating to the patient's scapula, and using the scapular spatial data to determine the rotation of the patient's chest with respect to at least one reference axis.

[0020] FIG. 1 shows the use of image segmentation to locate anatomical structures including the scapulae of a patient within a medical image 100, thereby obtaining scapular space data. As shown, the result of the image segmentation is to locate the scapulae and their contours 102. In addition, the contours of the lungs 104 and the clavicles 106 may be arranged. The output of the image segmentation is spatial data including at least scapular space data indicating the shape and position of the scapulae. Image 100 can be segmented using a convolutional neural network trained with annotated sample images. To achieve better performance with a comparable amount of annotation effort, landmarks can be first located, for example, as described in VON BERG, J. et al. Robust chest x-ray quality assessment using convolutional neural networks and atlas regularization. In: Medical Imaging 2020: Image Processing, International Society for Optics and Photonics. 2020. p.113131L, before determining the contours.

[0021] Using the scapular space data, the rotation of the patient's chest relative to at least one reference axis can be determined. Evaluating the patient's rotation using the shape and positioning of the scapulae and optionally the lung fields provides a better reflection of the rotation of the entire chest. However, the scapulae have a higher mobility compared to the clavicles and the spine and can be rotated internally, externally, abducted, adducted, etc. with more degrees of freedom. Thus, the present disclosure contemplates performing any sanity checks to confirm that the scapulae are symmetric with respect to the patient (i.e., there is no external rotation movement of the scapulae) before using the scapular space data to determine the patient's rotation.

[0022] Figure 2 shows the confirmation of the symmetry of the scapular position. The acromion and coracoid process of the scapula are identified on both sides, and their endpoints are identified by segmentation, and their lengths are determined. What is shown in Figure 2 are the length 202L of the left acromion, the length 204L of the left coracoid process, the length 202R of the right acromion, and the length 204R of the right coracoid process. The ratio, R L = length of left coracoid process / length of left acromion and R R = length of right coracoid process / length of right acromion is calculated. When there is no scapular movement in the external rotation direction, the left ratio R L coincides with the right ratio R R within a given accuracy. It will be understood that other parameters may be additionally or alternatively utilized to evaluate scapular symmetry, such as the inferior angle of the scapula, the angle between the clavicle and the acromion, and the angle between the lateral border of the scapula and the humerus.

[0023] Figures 3A and 3B show the determination of in-plane rotation and out-of-plane rotation of a patient using scapular spatial data.

[0024] The patient's out-of-plane rotation can be determined around a suitable reference axis (in this case, the central axis of the patient's body, shown as line A in Figure 3A). Such a rotation axis can be robustly determined by registering a chest atlas with annotated images, similar to the technique described in VON BERG, J. et al. Robust chest x-ray quality assessment using convolutional neural networks and atlas regularization. In: Medical Imaging 2020: Image Processing, International Society for Optics and Photonics. The lowest points [P1, P2] on the lower boundary of the scapula are calculated. The scapular line, shown as line B in Figure 3A, connects points P1 and P2. The midpoint of line B is found, which is shown as point P3 in Figure 3A. The intersection of line A and line B is found, which is shown as point P4. The distance between P3 and P4 corresponds to the component of the out-of-plane patient rotation. This is also shown in Figure 3B, which illustrates the phenomenon relating to the source 302 and detector 304 of the X-ray system 300. The greater the patient's rotation, the greater the linear displacement between points P3 and P4.

[0025] The patient's in-plane rotation can be determined relative to the detector plane axis. Figure 3A shows, for example, line C as the horizontal axis in the image or detector plane. The angular displacement between line B and line C corresponds to the patient's in-plane rotation.

[0026] Such image information can be used to train a regressive convolutional neural network to improve the accuracy of patient rotation determination. For example, the rotation angle can be estimated from contour 102 represented by scapular spatial data using a trained regressive convolutional neural network. This approach can follow the method described in KRONKE et al. CNN-based pose-estimation of musculoskeletal X-ray images. Philips ocupai conference. 2021, which has been successfully applied to a similar task of estimating ankle pose and flexion from a single X-ray image.

[0027] Numerous modifications of the systems and methods described herein are envisioned. For example, the patient rotation determination method disclosed herein can be used independently or in combination with existing methods using the clavicle and spine. In another example, scapular spatial data used by the systems and methods disclosed herein arise from a process used to locate both scapulae and analyze them in shape and positioning for the purpose of an independent “scapular checker.” The lung fields may also be locible for this purpose. In yet another example, to generate a nomogram with an acceptable bandwidth, the ratio R L and R R This may be plotted. If the calculated values ​​for the patient under examination appear on the nomogram, it can be assumed that the scapula is symmetrical with respect to the patient. In yet another example, thoracic rotation is estimated based on redundant X-ray contours in addition to, or instead of, the contours of the scapula.

[0028] The systems and methods disclosed herein can be applied in the context of X-ray quality assessment.

[0029] Figure 4 shows an exemplary computing system 800 that can be used according to the systems and methods disclosed herein. The computing system 800 may form part of or have part of any desktop, laptop, server, or cloud-based computing system. The computing system 800 has at least one processor 802 that executes instructions stored in memory 804. The instructions may be, for example, instructions for implementing the functions described above as being performed by one or more of the components described above, or instructions for implementing one or more of the methods described above. The processor 802 can access the memory via a system bus 806. In addition to storing executable instructions, memory 804 may also store conversation inputs, scores assigned to conversation inputs, and the like.

[0030] The computing system 800 further has a data store 808 accessible by the processor 802 via a system bus 806. The data store 808 may contain executable instructions, log data, etc. The computing system 800 also has an input interface 810 that allows external devices to communicate with the computing system 800. For example, the input interface 810 can be used to receive instructions from an external computer device, a user, etc. The computing system 800 also has an output interface 812 that interfaces the computing system 800 with one or more external devices. For example, the computing system 800 can display text, images, etc. via the output interface 812.

[0031] External devices communicating with the computing system 800 via the input interface 810 and output interface 812 are intended to be included in an environment that provides substantially any type of user interface with which the user can interact. Examples of user interface types include graphical user interfaces and intrinsic user interfaces. For example, a graphical user interface can accept input from a user using input devices such as a keyboard, mouse, or remote control, and provide output on an output device such as a display. Furthermore, an intrinsic user interface may enable the user to interact with the computing system 800 in a way that is not constrained by input devices such as a keyboard, mouse, or remote control. Rather, an intrinsic user interface may rely on speech recognition, touch and stylus recognition, on-screen and adjacent-screen gesture recognition, air gestures, head and eye tracking, voice and speech, vision, touch, gestures, machine intelligence, etc.

[0032] Furthermore, although computing system 800 is presented as a single system, it should be understood that it may be a distributed system. For example, several devices may be able to communicate via network connectivity and collectively perform tasks described as being performed by computing system 800.

[0033] The various functions described herein can be implemented in hardware, software, or any combination thereof. When implemented in software, the functions described above can be stored as one or more instructions or codes on a computer-readable medium or transmitted through a computer-readable medium. Computer-readable medium includes computer-readable storage media. Computer-readable storage media can be any available storage medium that can be accessed by a computer. Such computer-readable storage media, to the present but not limited to, include flash storage media, RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer. Disk and disc, as used herein, include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs (BDs), where a disk typically reproduces data magnetically, and a disc typically reproduces data optically using a laser. Furthermore, propagated signals are not included within the scope of computer-readable storage media. Computer-readable media also include communication media, which include any medium that facilitates the transfer of computer programs from one place to another. Connections can be, for example, communication media. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of communication media. The above combinations should also be included within the scope of computer-readable media.

[0034] Alternatively or additionally, the functions described herein can be performed, at least in part, by one or more hardware logic components. Exemplary types of hardware logic components that may be used include, but are not limited to, field-programmable gate arrays (FPGAs), program-specific integrated circuits (ASICs), program-specific standard products (ASSPs), system-on-chip systems (SOCs), and composite programmable logic devices (CPLDs).

[0035] It should be understood that the circuit described above may have other functions in addition to those described above, and that these functions may be performed by the same circuit.

[0036] The applicant hereby discloses each of the features described herein and any combination of two or more such features individually to the extent that such features or combinations can be implemented in whole according to this specification, in light of the common general knowledge of those skilled in the art, regardless of whether such features or combinations solve any problem disclosed herein, and without being limited to the claims. The applicant hereby shows that aspects of the present invention may consist of any individual or combination of such features.

[0037] It should be noted that embodiments of the present invention are described with reference to different categories. In particular, some examples are described with reference to methods, and others with reference to apparatus. However, those skilled in the art will understand from the description that, unless otherwise specified, any combination of features belonging to one category, as well as any combination of features relating to different categories, are disclosed by this application. However, all features combined can provide a greater synergistic effect than the simple sum of the features.

[0038] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions should be considered illustrative or descriptive and not limiting. The present invention is not limited to the disclosed embodiments. Other modifications to the disclosed embodiments can be understood and achieved by those skilled in the art from a consideration of the drawings, disclosure and appended claims.

[0039] The term "comprising" does not exclude other components or steps.

[0040] The indefinite articles "a" or "an" do not exclude plurality. In addition, as used herein, the articles "a" and "an" should generally be interpreted as meaning "one or more" unless otherwise specified or the context makes it clear that they refer to a singular form.

[0041] A single processor or other unit can perform the functions of several items listed in the claims.

[0042] The mere fact that certain means are described in different dependent claims does not indicate that a combination of these means cannot be used advantageously.

[0043] Computer programs can be stored / distributed on suitable media such as optical storage media or solid-state media supplied together with or as part of other hardware, but they can also be distributed in other forms, such as via the Internet or other wired or wireless notification systems.

[0044] No reference numeral in a claim should be construed as limiting its scope.

[0045] Unless otherwise specified or evident from the context, the phrases “one or more of A, B, and C,” “at least one of A, B, and C,” and “A, B, and / or C” as used herein are intended to mean all possible substitutions of one or more of the listed items. That is, the phrase “X has A and / or B” is satisfied by any of the following examples: X has A; X has B; or X has both A and B. The embodiments of the present invention are described below. (Note 1) A computer implementation method for determining the rotation of a patient's chest in a medical image, The steps include receiving the medical image of the patient, The steps include processing the aforementioned medical images to determine scapular spatial data relating to the patient's scapula, The steps include processing the scapular spatial data to confirm the symmetry of the patient's scapula, Using the scapular spatial data, the steps include determining the rotation of the patient's chest with respect to at least one reference axis, A method of having. (Note 2) The step of confirming the symmetry of the patient's scapula is, The steps include identifying the acromion and coracoid process of the left and right scapulae and determining their lengths, For each of the left and right scapulae, the step is to calculate the ratio of the length of the coracoid bone to the length of the acromion, A step of confirming the symmetry of the scapula based on the calculated ratio, The method described in Appendix 1, having the characteristics of the method described in Appendix 1. (Note 3) The method according to Appendix 2, wherein the step of verifying the symmetry of the scapula based on the calculated ratios includes determining that the ratio of the left scapula matches the ratio of the right scapula within an acceptable range. (Note 4) The method according to any one of Appendix 1 to 3, wherein the step of confirming the symmetry of the scapula is based on one or more parameters selected from the group having the inferior angle of the scapula, the angle between the clavicle and the acromion, the angle between the lateral boundary of the scapula and the humerus, and any combination thereof. (Note 5) The method according to Appendix 4, wherein the step of confirming the symmetry of the scapula is based on comparing one or more values ​​of the parameters for the left scapula and the right scapula. (Note 6) The method according to any one of Appendix 1 to 5, wherein the step of processing the medical image to determine the scapular spatial data includes segmenting the medical image to determine the contour of the patient's scapula. (Note 7) The method according to Appendix 6, further comprising using a convolutional neural network trained on annotated sample images to perform the aforementioned segmentation. (Note 8) The method according to Appendix 6 or 7, further comprising processing the medical image to locate one or more landmarks to facilitate the segmentation before segmenting the medical image to determine the contour of the patient's scapula. (Note 9) The step of determining the rotation of the patient's chest is: Using the scapular spatial data, the step of calculating scapular lines connecting corresponding points on the left and right scapulae of the patient; A step of determining the rotation using the displacement between the scapular line and the at least one reference axis, The method described in any one of the appendices 1 to 8, having the characteristics of: (Note 10) The scapular line is a line connecting two points on the contours of the left and right scapulae, and each of the two points is a vertex on the contour of the inferior angle of each scapula, as described in Appendix 9. (Note 11) The method according to Appendix 9 or 10, wherein the at least one reference axis includes a detector plane axis, the displacement includes an angular displacement between the scapular line and the detector plane axis, and the step of determining the rotation includes determining the in-plane rotation of the patient using the angular displacement. (Note 12) The method according to any one of the appendices 9 to 11, wherein the at least one reference axis has the central axis of the patient's body, the displacement has a linear displacement between the midpoint of the scapular line and the central axis, and the step of determining the rotation includes determining the out-of-plane rotation of the patient using the linear displacement. (Note 13) The method according to any one of the appendices 1 to 12, wherein the step of determining the rotation of the patient's chest with respect to the at least one reference axis using the scapular spatial data comprises determining the rotation using a trained machine learning model. (Note 14) A computing system configured to perform the actions described in any one of the items in Appendix 1 to 13. (Note 15) A computer program that, when executed by a computing system, includes instructions that cause the computing system to perform the method described in any one of the items 1 to 13.

Claims

1. A computer implementation method for determining the rotation of a patient's chest in a medical image, The steps include receiving the medical image of the patient, The steps include processing the aforementioned medical images to determine scapular spatial data relating to the patient's scapula, A step of processing the scapular spatial data to confirm the symmetry of the patient's scapula, which includes performing a check to ensure that the left and right scapulae of the patient are symmetrical. If the left and right scapulae of the patient are symmetrical, the step of determining the rotation of the patient's chest with respect to at least one reference axis using the scapular spatial data, A method having

2. The step of confirming the symmetry of the patient's scapula is, The steps include identifying the acromion and coracoid process of the left and right scapulae and determining their lengths, For each of the left and right scapulae, the step is to calculate the ratio of the length of the coracoid bone to the length of the acromion, A step of confirming the symmetry of the scapula based on the calculated ratio, The method according to claim 1, comprising:

3. The method of claim 2, wherein the step of verifying the symmetry of the scapula based on the calculated ratios includes determining that the ratio of the left scapula matches the ratio of the right scapula within an acceptable range.

4. The method according to claim 1, wherein the step of confirming the symmetry of the scapula is based on one or more parameters selected from the group having the inferior angle of the scapula, the angle between the clavicle and the acromion, the angle between the lateral boundary of the scapula and the humerus, and any combination thereof.

5. The method according to claim 4, wherein the step of confirming the symmetry of the scapula is based on comparing one or more values ​​of the parameter for the left scapula and the right scapula.

6. The method according to claim 1, wherein the step of processing the medical image to determine the scapular spatial data includes segmenting the medical image to determine the contour of the patient's scapula.

7. The method according to claim 6, further comprising using a convolutional neural network trained on annotated sample images to perform the segmentation.

8. The method according to claim 6, further comprising processing the medical image to locate one or more landmarks to facilitate the segmentation before segmenting the medical image to determine the contour of the patient's scapula.

9. The step of determining the rotation of the patient's chest is: Using the scapular spatial data, the step of calculating scapular lines connecting corresponding points on the left and right scapulae of the patient; A step of determining the rotation using the displacement between the scapular line and the at least one reference axis, The method according to claim 1, comprising:

10. The method according to claim 9, wherein the scapular line is a line connecting two points on the contours of the left and right scapulae, and each of the two points is a vertex on the contour of the inferior angle of each scapula.

11. The method according to claim 9, wherein the at least one reference axis includes a detector plane axis, the displacement includes an angular displacement between the scapular line and the detector plane axis, and the step of determining the rotation includes determining the in-plane rotation of the patient using the angular displacement.

12. The method according to claim 9, wherein the at least one reference axis has the central axis of the patient's body, the displacement has a linear displacement between the midpoint of the scapular line and the central axis, and the step of determining the rotation includes determining the out-of-plane rotation of the patient using the linear displacement.

13. The method according to claim 1, wherein the step of determining the rotation of the patient's chest with respect to the at least one reference axis using the scapular spatial data comprises determining the rotation using a trained machine learning model.

14. A computing system configured to perform the method described in any one of claims 1 to 13.

15. A computer program that, when executed by a computing system, includes instructions causing the computing system to perform the method described in any one of claims 1 to 13.

Images