Fine rotation for digital radiography

Abstract

A computer implemented method accesses a radiographic image of a human body part and determines an appropriate orientation of the imaged human body part for diagnostic purposes. The received radiographic image is rotated according to the determined appropriate orientation. Landmarks of the imaged human body part may be identified in order to determine an angular orientation of the initial unrotated radiographic image.

Description

FINE ROTATION FOR DIGITAL RADIOGRAPHYBACKGROUND OF THE INVENTION

[0001] The subject matter disclosed herein relates to digital radiography imaging. In particular, to a system and method that allows a user to correct an orientation of a radiographic image of patient anatomy as desired.

[0002] Generally, prior to clinical interpretation, it is recommended to perform fine adjustments to the orientation of anatomical structures in digital radiography (DR) images to ensure accurate diagnosis. Inappropriate orientation of the anatomy in DR images typically arises when the DR detector is out of the bucky or when the patient is unable to cooperate during the positioning process due to disability or injury. In these cases, tire radiologist may manually perform the fine rotational adjustment of a captured DR image to meet specific anatomical requirements. Making this fine adjustment is not always a one-click process, which means radiologists often need to first identify the anatomy, then click and drag the DR images multiple times to ensure that the specific anatomy has been correctly positioned and presented. As a result, the entire process can be time-consuming and labor-intensive, particularly as the daily average reading volume is gradually increasing for most radiologists.

[0003] The fine rotational adjustment of the orientation of the image of a patient anatomy is crucial for achieving a correct and accurate diagnosis. However, manually performing this adjustment can be time-consuming and tedious. Therefore, a fine rotation method that can automatically identify the anatomy presented in the DR image and adjust it to an appropriate orientation based on specific anatomical requirements can significantly increase the efficiency of a radiologist's daily work and simplify the diagnostic reading process.

[0004] One disadvantage of previous solutions is that the techniques are unable to perform precise rotations of the anatomy; rather, only rough adjustments of the orientation are possible. Further manual adjustment is often necessary.BRIEF DESCRIPTION OF THE INVENTION

[0005] A computer implemented method accesses a radiographic image of a human body part and determines a correct orientation of the imaged human body part for diagnostic purposes. The received radiographic image is rotated according to the determined correct orientation. Landmarks of the imaged human body part may be identified in order to determine an angular orientation of the initial unrotated radiographic image.

[0006] In one embodiment, a computer implemented method includes receiving a radiographic image of a human body part, determining a correct orientation of the image of the human body part, and rotating the received radiographic image according to the determined correct orientation.

[0007] In one embodiment, a computer processor may be programmed to access a radiographic image of a human body part, determine a correct orientation of the image, and to rotate the accessed radiographic image according to the correct orientation.

[0008] The present disclosure describes a system and method for automatically adjusting a current orientation of patient anatomy in an x-ray by identifying and locating one or more landmarks of the patient anatomy, then rotating the image according to an angle between the standard orientation and its current orientation. 'Fhis technique can solve most of the problems associated with previous solutions, which were limited in their ability to apply to only one or a few exam views and body parts, and to situations with limited orientation.]0009] This disclosure introduces a fine rotation method to overcome problems and disadvantages of previous solutions and methods, lire present method will first identify the patient anatomy in an x-ray image by using a pretrained classification module or network, then detect and locate one or more landmarks of the patient anatomy using another module or network. The next step involves applying a rule-based module to estimate the rotation angle required to align the expected orientation of the anatomy with its current orientation, based on the specificrequirements of the target exam view, lire estimated rotation angle is then applied to adjust the x-ray image to the appropriate orientation.

[0010] This disclosure describes a fine rotation system and method which consists of a body part and exam view classification module, a landmark detection module, a rule-based rotation angle estimation module and, alternatively, a beam shaping mask (BSM) adjustment module. Using the body part and exam view classification module, the body part and its exam view in the initial x-ray image are able to be identified. This information is used in the landmark identification detection module and in the rule-based rotation angle estimation module. With the landmark detection module, the anatomic landmarks of the anatomy are located in the x-ray image, according to the specific patient body part and exam view provided by the classification module.

[0011] With the rule-based rotation angle estimation module, the rotation angle can be estimated as between the expected orientation and the current orientation of the target anatomy by using the landmarks and the exam view information. Then the module will rotate the x-ray image according to this estimated angle. The BSM adjustment module provides the ability to adjust the BSM boundary further to make it parallel to the bottom boundary of the image, which eliminates the dark region introduced by image rotation and results in a consistent visual presentation across different exam views and body parts.

[0012] The summary descriptions above are not meant to describe individual separate embodiments whose elements are not interchangeable. In fact, many of the elements described as related to a particular embodiment can be used together with, and possibly interchanged with, elements of other described embodiments. Many changes and modifications may be made within the scope of the present invention without departing from the spirit thereof, and the invention includes all such modifications.

[0013] This brief description of the invention is intended only to provide a brief overview of subject matter disclosed herein according to one or more illustrativeembodiments, and does not serve as a guide to interpreting the claims or to define or limit the scope of the invention, which is defined only by the appended claims. This brief description is provided to introduce an illustrative selection of concepts in a simplified form that are further described below in the detailed description, Ihis brief description is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] So that the manner in which the features of the invention can be understood, a detailed description of the invention may be had by reference to certain embodiments, some of which are illustrated in the accompanying drawings. It is to be noted, however, that the drawings illustrate only certain embodiments of this invention and are therefore not to be considered limiting of its scope, for the scope of the invention encompasses other equally effective embodiments. The drawings below are intended to be drawn neither to any precise scale with respect to relative size, angular relationship, relative position, or timing relationship, nor to any combinational relationship with respect to interchangeability, substitution, or representation of a required implementation, emphasis generally being placed upon illustrating the features of certain embodiments of the invention. In the drawings, like numerals are used to indicate like parts throughout the various views. Thus, for further understanding of the invention, reference can be made to the following detailed description, read in connection with the drawings in which:

[0015] FIGS. 1A - I B and 2A - 2B are exemplary- radiographic images of body parts rotated to a selected orientation;

[0016] FIG. 3A illustrates exemplary method steps of the present invention;

[0017] FIG. 3B illustrates labeled landmarks on a human hand;

[0018] FIGS. 4 A - 4B are example images showing vectors used for image rotation; and

[0019] FIG. 5 is an exemplary system used for practicing embodiments of the present invention.DETAILED DESCRIPTION OF THE INVENTION

[0020] This application claims priority to U.S. Patent Application Serial No. 63 / 585,973, filed September 28, 2023, in the name of Duan et al., and entitled FINE ROTATION FOR DIGITAL RADIOGRAPHY, which is hereby incorporated by reference herein in its entirety.

[0021] With reference to FIG. 1 A, in one embodiment of the present invention, an unprocessed x-ray image of a human hand is accessed by a trained body part and exam view classification module, which may include an artificial intelligence module trained to recognize one of several body parts captured in an x-ray image.The module may be programmed to display a body part name and its exam view after recognition is performed. The exam view information may be used to indicate whether a portion or an entire body part is contained in the x-ray image. This x-ray image, along with its corresponding body part name and exam view, is passed to a landmark detection module, which may also include a pre-trained artificial intelligence module that is trained to identify and locate particular anatomic landmarks on the imaged body part based on the specified body part and exam view. In one embodiment, the body part name may include the text “hand” and the exam view may indicate that the entire hand is imaged. In one embodiment, the landmark detection module proceeds to identify and locate in the image of the human hand a number of joints and the tips of the fingers. The identified landmarks may each be indicated by a dot superimposed on the image by the landmark detection module, and each dot may be numbered or otherwise labeled according to a predetermined numbering or labeling scheme. In one embodiment, the landmark detection module may be trained to indicate, and number, directly on the x-ray image of the human hand, each location of the five joints where the metacarpal bones meet the carpal bones. The landmark detection module may be further trained to indicate, and number, each of five joints between the metacarpals and phalanges, as well as nine phalange joints. If the landmark detection module is further trained to indicate thetips of the phalanges, there may be altogether twenty four landmarks identified and numbered by the landmark detection module in the x-ray image of the human hand.

[0022] The identified landmarks on the x-ray image of the human hand, the body part and the exam view are then passed to a rule-based rotation angle estimation module, in which the current orientation of the x-ray image is indicated and the standard correct orientation of the anatomy is determined and indicated, based on the landmarks and the exam view. The arrow, or vector, in FIG. 1 A that is pointing generally upward is an electronically stored standard orientation vector, or desired orientation vector, associated with the identified body part and exam view. It is retrieved by the rotation angle estimation module which may be programmed to superimpose the arrow, or vector, on the x-ray image. At least two landmarks provided with the x-ray image are used to form an anatomical vector which indicates a current anatomical orientation of the imaged body part. In one embodiment, two landmarks for this body part may include the locations of two phalange joints identified for the middle finger. The arrow, or anatomical orientation vector, in FIG. 1 A which points generally upward and to the right is formed by the locations of two phalange joints of the middle finger of the imaged hand. This vector may be referred to herein as indicating the current anatomical orientation of the initial captured, unrotated x-ray image. In one embodiment, any two of four landmark locations indicated for the middle finger (i.e., metacarpal / phalange joint, two phalange joints, and tip of phalange), excluding the carpal / metacarpal joint, may be used to form the anatomical orientation vector which indicates the orientation of the captured image. A rotation angle is then estimated using the angle formed by the two vectors as shown in FIG. 1 A, and is implemented to rotate the x-ray image by the rotation angle to the desired orientation, as illustrated in FIG. IB. Finally, the BSM adjustment module may be used to finely adjust the orientation of the BSM boundary to be parallel with the image boundary.(0023] FIGS. 2 A and 2B illustrate a second example using the method as described above, whereby an x-ray image of a human body part, in the example of FIG. 2A — a foot and a portion of the lower leg, has superimposed thereon a standard orientation vector pointing generally upward and an anatomical orientation vectorpointing generally downward to the left. The anatomical orientation vector of FIG. 2A may be formed using any two preselected landmarks identified and labeled on the tibia or fibula as captured in the x-ray image. The angle formed by these two vectors is used to calculate the correction rotation angle for rotating the x-ray image to tire standard, or desired, orientation as shown in FIG. 2B. Although the methods described herein illustrate embodiments using superimposed visible arrows, or vectors, the system and method does not require that these vectors be visually displayed, The modules and / or computer system performing the methods described herein may execute the method by automatic internal computation to determine a rotation angle and to rotate the x-ray image to the standard orientation.

[0024] FIG. 3 A is a flow chart of the method as described herein above. An unprocessed x-ray image of a human hand 301 is accessed by a trained body part and exam view' classification module 302, which may include an artificial intelligence module trained to recognize one of several body parts captured in the unprocessed x- ray image 301. The module 302 may be trained to identify the body part and exam view and to display, or assign, a body part name and exam view on the image such as “hand_pa” 303. 'Phis x-ray image, along with its corresponding body part name and exam view, is passed to a landmark detection module 304, which may also include a pre-trained artificial intelligence module that is trained to identify and locate particular anatomic landmarks based on the specified patient body part and exam view. In this example, twenty-four landmarks are identified and labeled in the image of the hand 305. A similarly labeled image of a human hand is shown for exemplary purposes in FIG. 3B. The identified landmarks, body part and the exam view are then passed to a rule-based rotation angle estimation module 306, in which the current anatomical orientation vector of the x-ray image is indicated and the standard correct orientation vector of the anatomy is determined and indicated 307, based on the landmarks and the exam view. Ihe vector in 307 that is pointing generally upward is the electronically stored standard vector associated with the identified body part and exam view. It is retrieved by the rotation angle estimation module 306 which may be programmed to superimpose the vector on the x-ray image. The anatomical orientation vector pointing downward to the right in 307 is formed according to thelocations of two selected joints identified for the middle finger of the hand. In one embodiment, any two preselected landmark locations indicated for the middle finger may be used to form the vector indicating the anatomical orientation of the captured image. A rotation angle is then estimated by the rotation angle estimation module 308 using the angle formed by the vectors as shown in 307, and is implemented to rotate the x-ray image in 307 by the rotation angle to the desired orientation in 309.

[0025] As described herein, the trained body part and exam view classification module can identify the body part and exam view in the target x-ray image. This module is trained to support various body parts and exam views and can be configured based on specific requirements. In one embodiment, the body part and exam view classification module could include a known pretrained deep learning network, or a traditional machine learning model having engineered features. In one embodiment, the body part and exam view classification module can output either the body part name, exam view, or a combination of both. In one embodiment, the body part and exam view classification module can be replaced by user input. The body part and exam view can be manually input by the user or obtained from a preset configuration. As described herein, the landmark detection module may detect and locate the anatomic landmarks according to the specific body parts and exam views output by the body part and exam view classification module. The total amount and the type of the anatomic landmarks of a specific body part and its exam view can be predefined according to the specific requirement or determined through an unsupervised learning technique. In one embodiment, the landmark detection module can be a pretrained deep learning network, in which a group of anatomic landmarks are predefined before the training process is performed. In one embodiment, the landmark detection module can be an unsupervised deep learning network that learns to detect a group of anatomic landmarks and locates them automatically. Alternatively, the rule-based rotation angle estimation module can estimate the rotation angle between the standard desired orientation and the current orientation of the anatomy in the x-ray image.

[0026] In one embodiment, one or a few pairs of the landmarks will be used to construct one or more direction vectors to indicate the current orientation of theanatomy. These vectors may be selected to indicate the long / short axis of the anatomy, or the direction of the unique anatomic structure within the anatomy. In one embodiment, after the standard orientation of the anatomy is determined based on the specific requirements of the body part and exam view, the rotation angle may be calculated as either the average or the single angle between one or several direction vectors and the standard orientation vector. 'Hie x-ray image is then rotated according to the calculated rotation angle. Alternatively, in one embodiment, the strategy for constructing the direction vectors and the standard orientation vector, as well as calculating the rotation angle, is based on the total number and reliability of the detected landmarks. One or a few pairs of image direction vectors and standard vectors are ranked according to the reliability of the landmarks that make up those vectors. The rotation angle is then calculated as either the angle between the first- ranking pair of vectors or the average value of the first few pairs of ranked vectors.

[0027] In one embodiment, the BSM adjustment module calculates the rotation angle between one or a few BSM boundaries and their closest boundaries in the x-ray image. These rotation angles are then used to fine-tune the orientation of the x-ray image and eliminate dark areas that might have been induced by the previous rotation step. Installation and configuration of such a system may include configuring the support exam views, the functionality module and the trigger method of the fine rotation method. A target x-ray image can be input as well as the related information, including, but not limited to, the comer point of the BSM boundary. The BSM adjustment module may further adjust orientation of the the x-ray image if this module has been configured and enabled. I'he fine rotation method described herein may run automatically in the background, or it may be triggered case by case by the user according to its configuration.

[0028] With reference to FIGS. 4A - 4B, there are illustrated example images with single and multiple direction vectors. FIG. 4 A illustrates an example x-ray image, similar in certain respects to the example of FIG. 1 A, which uses one standard orientation vector 401 and one anatomical orientation vector 402. The anatomical orientation vector 402 is formed from two selected landmarks labeled 8 and 10 out of several identified and numbered landmarks in the x-ray image of FIG. 4 A. Asexplained above in relation to FIG. 1 A, the magnitude of the rotation angle to be used for rotating the image of FIG. 4 A will be the relative angle between the orientation vectors 401, 402.

[0029] FIG. 4B illustrates the use of several orientation vectors to calculate the rotation angle. The anatomical orientation vector 403 represents the average orientation vector of the two anatomical orientation vectors 402, 404. The anatomical orientation vector 402 is formed by using the landmarks indicated by the superimposed dots numbered 8 and 10. The anatomical orientation vector 404 is formed by using the landmarks indicated by the superimposed dots numbered 9 and 11. 'fhe rotation angle will be the relative angle between the average orientation vector 403 and the standard vector 401. In one embodiment, we can calculate the relative angle between the anatomical orientation vector 404 and the standard orientation vector 401, then calculate the second relative angle between the anatomical orientation vector 402 and the standard vector 401. The final rotation angle will then be the average angle between these two relative angles.

[0030] The schematic diagram of FIG. 5 shows a radiography system 100 for x-ray imaging of a human body. System 100 uses a DR detector 20 for image capture and provides computational logic for performing image processing on the captured image. In the radiography system 100 shown in FIG. 5, a patient 14 is positioned adjacent DR detector 20, which is mounted on a bucky or other support 22, and is exposed to radiation from an x-ray source 24, whereby detector 20 captures a DR image of a portion of patient 14, such as shown in FIGS. 4A and 4B, for example.The exposure is initiated by the technologist or other practitioner 26 using an operator control 42 that is in signal communication with a control logic processor 30 that can initiate exposure and the image acquisition described herein. Practitioner 26 controls the setup and operation from a workstation 28 having a display 90 or other control console. Control logic processor 30 is in signal communication with other components of apparatus 100, such as DR detector 20 and x-ray source 24, and provides the needed control signals for exposure, data acquisition, processing, storage, and transmission. A memory 32 or other storage apparatus, in signal communicationwith control logic processor 30, may include the modules described herein and other programmed methods described herein under operational control of processor 30.

[0031] As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method, computer program product, or module. Accordingly, aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “service,” “circuit,” “circuitry,” “module,” and / or “system.” Furthermore, aspects of the present invention may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.

[0032] Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc readonly memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0033] Program code and / or executable instructions embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects ofthe present invention may be written in any combination of one or more programming languages. The program code may execute entirely on the user's computer (device), partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer, for example, through the internet.

[0034] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0035] This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.

Claims

CLAIMS:

1. A computer implemented method comprising the steps of: accessing a radiographic image of a human body part; determining a standard orientation of the radiographic image of the human body part; and rotating the accessed radiographic image according to the determined standard orientation.

2. The method of claim 1, further comprising determining an anatomical orientation of the image of the human body part in the accessed radiographic image.

3. The method of claim 2, further comprising calculating an angular difference between the anatomical orientation of the human body part in the accessed radiographic image and the standard orientation.

4. The method of claim 3, further comprising rotating the radiographic image in the amount of the calculated angular difference.

5. The method of claim 2, further comprising identifying a plurality of landmarks in the human body part.

6. The method of claim 5, further comprising determining a directional vector using at least two of the identi fied l andmarks in the human body part.

7. The method of claim 6, further comprising rotating the image such that the determined directional vector is oriented substantially vertically.

8. An image processing system comprising a processor programmed to receive a radiographic image that includes a human body part, to determine a correct orientation of the human body part in the received radiographicimage, and to rotate the received radiographic image according to the determined correct orientation.

9. The system of claim 8, wherein the processor is further programmed to determine an orientation of the human body part in the received radiographic image.

10. The method of claim 9, wherein the processor is further programmed to calculate an angular difference between the orientation of the human body part in the received radiographic image and the correct orientation.

11. The method of claim 10, wherein the processor is further programmed to rotate the radiographic image in the amount of the calculated angular difference.

12. The method of claim 9, wherein the processor is further programmed to identifying landmarks in the human body part.

13. The method of claim 12, wherein the processor is further programmed to determine a directional vector using at least two of the identified landmarks in the human body part.

14. The method of claim 13, wherein the processor is further programmed to rotate the image such that the determined directional vector is oriented vertically.

Images