Method and system for anatomical structure / view classification in X-ray imaging

Abstract

The present invention provides various methods and systems for x-ray imaging. In one embodiment, a method includes: controlling an x-ray source and an x-ray detector to acquire an image of a subject; classifying an anatomical structure / view depicted in the image using a trained neural network; performing post-processing of the image based on the anatomical structure / view; and displaying the post-processed image. In this way, the user workflow for x-ray image acquisition and processing can be simplified, and the appearance consistency of radiographic images and the registration accuracy of image pasting inspection can be improved.

Description

Technical Field

[0001] Embodiments of the subject matter disclosed herein relate to x-ray imaging. Background Art

[0002] Imaging techniques such as x-ray imaging allow for the non-invasive acquisition of images of internal structures or features of a subject, such as a patient. Digital x-ray imaging systems produce digital data that can be reconstructed into radiographic images. In a digital x-ray imaging system, radiation from a source is directed toward the subject. A portion of the radiation passes through the subject and strikes a detector. The detector includes an array of discrete picture elements or detector pixels, and generates an output signal based on the amount or intensity of radiation striking each pixel area. The output signal is then processed to generate an image that can be displayed for viewing. These images are used to identify and / or examine internal structures and organs within the patient's body. Summary of the invention

[0003] In one embodiment, a method includes: controlling an x-ray source and an x-ray detector to acquire an image of a subject; classifying an anatomical structure / view depicted in the image using a trained neural network; performing post-processing of the image based on the anatomical structure / view; and displaying the post-processed image. In this way, the user workflow for x-ray image acquisition and processing can be simplified, and the appearance consistency of radiographic images and the registration accuracy of image pasting inspection can be improved.

[0004] It should be understood that the above brief description is provided to introduce in a simplified form selected concepts that are further described in the detailed description. It is not meant to identify key or essential features of the claimed subject matter, the scope of which is solely defined by the claims that follow the detailed description. Furthermore, the claimed subject matter is not limited to implementations that solve any disadvantages noted above or in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The present invention will be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0006] Figure 1 An exemplary x-ray imaging system is shown according to one embodiment;

[0007] Figure 2 A high-level flow chart illustrating an exemplary method for automatic anatomical structure / view classification is shown, according to an embodiment;

[0008] Figure 3 A high-level flow chart illustrating an exemplary method for post-processing acquired radiographic images based on anatomy / view classification, according to an embodiment, is shown;

[0009] Figure 4 The embodiment according to the embodiment is shown Figure 3 Image set of the method;

[0010] Figure 5 A high-level flow chart illustrating an exemplary method for image pasting of radiographic images based on anatomy / view classification, according to an embodiment, is shown;

[0011] Figure 6 A high-level flow chart illustrating an exemplary method for determining anatomy / view classification based on predictions of anatomy / view classifications for a plurality of radiographic images, in accordance with an embodiment, is shown;

[0012] Figure 7 The embodiment according to the embodiment is shown Figure 5 Image set of the method;

[0013] Figure 8 A high-level flow chart illustrating an exemplary method for anatomical structure / view classification for acquisition protocol selection is shown, according to an embodiment; and

[0014] Fig. 9 A set of images showing anatomical structure / view classification based on camera data is shown according to an embodiment. DETAILED DESCRIPTION

[0015] The following description relates to various embodiments of x-ray imaging. Specifically, systems and methods for automatic anatomical structure / view classification for x-ray imaging are provided. An x-ray imaging system (such as Figure 1 The x-ray imaging system described in (a) includes a trained classification network for automatically determining an anatomical structure / view of an imaged subject. The anatomical structure / view includes an orientation and position of the imaged subject relative to an x-ray source and an x-ray detector. A method for x-ray imaging (such as Figure 2 The method described in the embodiment of the present invention comprises: classifying anatomical structures / views depicted in one or more images; and post-processing one or more images based on the anatomical structures / views. For radiographic image processing, different anatomical structures / views require adaptive parameters related to the overall appearance, such as parameters related to edge details, contrast, noise level, etc. A method for radiographic image processing (such as Figure 3 and Figure 4The method described in ) includes automatically adjusting such parameters based on the anatomical structure / view classified by the classification network. In this way, even if the user of the imaging system selects an incorrect protocol via the imaging system user interface, the image is processed with the correct protocol. Since the user does not need to select the best view protocol for displaying the image, the workflow efficiency is improved while improving the consistency of the displayed image.

[0016] In addition, image pasting or the creation of composite images is often accomplished by having a system for acquiring images with a total field of view that is larger than the detector field of view (FOV). For applications such as full spine imaging or long leg imaging, the total coverage of the anatomy (e.g., 60cm-120cm) exceeds the coverage of most current detector and film screen systems. In one approach to overcoming the limitations of the detector FOV, multiple images are acquired with overlapping FOVs and stitched together during an image pasting examination. Historically, images were acquired with the detector FOV and then the various images were manually cut by the radiologist to avoid overlap and manually re-pasted to reconstruct an image with the entire FOV. Recently, automated techniques for digitally pasting consecutive images have improved the accuracy of image pasting examinations. Typically, multiple orientations or views of a subject or patient are acquired during an image pasting examination. In a typical workflow for acquiring multiple views of a subject during an image pasting examination, a user selects a different protocol for each view. If an incorrect view protocol is selected, the images of the view may be misaligned, thereby reducing the accuracy of the image pasting. A method for improving workflow efficiency and increasing the success rate of image pasting examinations (such as Figures 5 to 7 The method shown) includes: using a classification network to identify the anatomical structure / view in each image; evaluating the classification of the anatomical structure / view to ensure that the classification is correct; and then performing image pasting using an appropriate view protocol automatically selected based on the classification.

[0017] Additionally, in some examples, a camera may be provided in an x-ray imaging system to acquire radiographic images in the optical domain. Figure 8 and Fig. 9 The method shown in ) includes: automatically classifying anatomical structures / views in camera data acquired by the camera; and automatically selecting an acquisition protocol (e.g., selecting an x-ray tube current and / or an x-ray tube voltage) based on the anatomical structure / view of the subject. In this way, appropriate acquisition protocols and view protocols can be determined without user input, thereby improving workflow efficiency, improving accuracy and consistency of radiographic imaging, and enabling dose reduction.

[0018] Now turn to Figure 1, which shows a block diagram of an x-ray imaging system 100 according to one embodiment of the present disclosure. The x-ray imaging system 100 includes an x-ray source 111 that radiates x-rays, a stand 132 on which a subject 105 stands during examination, and an x-ray detector 134 for detecting x-rays radiated by the x-ray source 111 and attenuated by the subject 105. As non-limiting examples, the x-ray detector 134 may include a scintillator, one or more ion chambers, a photodetector array, an x-ray exposure monitor, an electrical substrate, etc. The x-ray detector 134 is mounted on a stand 138 and is configured to be vertically movable according to an imaging region of the subject.

[0019] The operation console 160 includes a processor 161, a memory 162, a user interface 163, a motor driver 145 for controlling one or more motors 143, an x-ray power unit 114, an x-ray controller 116, a camera data acquisition unit 121, an x-ray data acquisition unit 135, and an image processor 150. The x-ray image data transmitted from the x-ray detector 134 is received by the x-ray data acquisition unit 135. The collected x-ray image data is image-processed by the image processor 150. The display device 155 communicatively coupled to the operation console 160 displays the image-processed x-ray image thereon.

[0020] The x-ray source 111 is supported by a support column 141, which can be mounted to a ceiling (e.g., as depicted) or mounted on a movable stand for positioning within an imaging room. The x-ray source 111 can be vertically movable relative to the subject or patient 105. For example, one of the one or more motors 143 can be integrated into the support column 141 and can be configured to adjust the vertical position of the x-ray source 111, for example, by increasing or decreasing the distance of the x-ray source 111 from the ceiling or floor. To this end, a motor driver 145 of the operating console 160 can be communicatively coupled to the one or more motors 143 and configured to control the one or more motors 143.

[0021] The x-ray power unit 114 and the x-ray controller 116 provide power of appropriate voltage and current to the x-ray source 111. A collimator (not shown) may be fixed to the x-ray source 111 for specifying an irradiation field of the x-ray beam. The x-ray beam radiated from the x-ray source 111 is applied to the subject via the collimator.

[0022] The x-ray source 111 and the camera 120 can be pivoted or rotated relative to the support column 141 in an angular direction 129 to image different parts of the subject 105. For example, during an image pasting examination, multiple x-ray images of the subject 105 can be acquired at different angles and pasted or stitched together to form a single image. As depicted, the x-ray source 111 can be oriented with a first field of view 126 to acquire a first x-ray image. Then, the x-ray source 111 can be rotated in an angular direction 129 to a second field of view 127 to acquire a second x-ray image. Then, the x-ray source 111 can be rotated in an angular direction 129 to a third field of view 128 to acquire a third x-ray image. The three fields of view 126, 127, and 128 are depicted as partially overlapping. Therefore, the three acquired x-ray images can be stitched together as further described herein to form a single x-ray image.

[0023] Thus, in addition to performing typical mono-energetic x-ray image acquisitions, the x-ray imaging system 100 may also be used to perform image stitching inspections as described above.

[0024] As further described herein, such methods include controlling the processor 161 and / or the image processor 150 to provide the image to the classification network 170 to automatically identify the anatomical structures / views depicted in the image. The classification network 170 may include a neural network model (such as a convolutional neural network) that is trained on a set of radiological images using corresponding labels of the anatomical structures / views depicted in the image. For example, the classification network 170 may include an effective convolutional neural network, such as a MobileNet model. The classification network 170 may be initially trained or primed, for example, using a large-scale training data set of non-radiology images by loading pre-trained parameters obtained by training the neural network using a large-scale training data set of non-radiology images.

[0025] Memory 162 stores executable instructions 172 that, when executed, cause one or more of processor 161 and image processor 150 to perform one or more actions. Figure 2 , Figure 3 , Figure 5 , Figure 6 and Fig. 9 Exemplary methods that may be stored as executable instructions 172 are further described.

[0026] In some examples, the x-ray imaging system 100 may include a camera 120 positioned adjacent to the x-ray source 111 and co-calibrated or co-aligned with the x-ray source 111. The camera 120 may include an optical camera that detects electromagnetic radiation in an optical range. Additionally or alternatively, the camera 120 may include a depth camera or a range imaging camera. As an illustrative and non-limiting example, the camera 120 configured as a depth camera may include an optical camera, an infrared camera, and an infrared projector that projects infrared dots in the field of view of the camera 120. The infrared camera images these dots, which in turn can be used to measure the depth within the optical camera of the camera 120. As another illustrative and non-limiting example, the camera 120 may include a time-of-flight camera. The camera 120 is communicatively coupled to a camera data acquisition unit 121 of the operator console 160. Therefore, the camera data collected or generated by the camera 120 may be transmitted to the camera data acquisition unit 121, which in turn provides the collected camera image data to the image processor 150 for image processing. The field of view of the camera 120 may overlap with the field of view of the x-ray source 111. In this manner, the field of view of the x-ray source 111 may be automatically determined based on the camera data recorded by the camera 120. Figure 8 and Fig. 9 As further described, the image processor 150 may utilize the classification network 170 to process the acquired camera data to automatically determine the anatomical structure / view of the subject 105 relative to the x-ray source 111 and the x-ray detector 134. The image processor 150 may then determine acquisition parameters for controlling the x-ray source 111 and the x-ray detector 134 to acquire one or more radiographic images and view parameters for processing the acquired images based on the anatomical structure / view determined in the camera data.

[0027] Figure 2 A high-level flow chart illustrating an exemplary method 200 for automatic anatomical structure / view classification according to an embodiment is shown. Specifically, the method 200 involves automatically classifying anatomical structures / views depicted in one or more images using a classification network such as the classification network 170. Figure 1 The method 200 is described with reference to the systems and components of the present disclosure, but it should be understood that the method 200 can be implemented with other systems and components without departing from the scope of the present disclosure. The method 200 can be stored as executable instructions 172 in, for example, the non-transitory memory 162 of the operating console 160, and can be executed by one or more of the processor 161 and the image processor 150 to perform the actions described below in this document.

[0028] The method 200 starts at 205. At 205, the method 200 acquires one or more images. In some examples, the method 200 may control the x-ray source 111 to generate an x-ray beam toward the x-ray detector 134 and convert the x-rays detected by the x-ray detector 134 into a radiographic image. In other examples, the method 200 controls the x-ray source 111 and the x-ray detector 134 to acquire a series of radiographic images while adjusting the relative positions of the x-ray source 111 and the x-ray detector 134 so that the radiographic images depict different but adjacent regions of the subject 105. In such examples, the method 200 may control the x-ray source 111 and the x-ray detector 134 to acquire different views of a plurality of images of the subject 105. For example, the method 200 may acquire two or more of an image of the subject 105 (right lateral view), an image of the subject 105 (anteroposterior (AP) view), an image of the subject 105 (posteroanterior (PA) view), and an image of the subject 105 (left lateral view). As another example, method 200 may acquire multiple images in multiple views of subject 105 , where the multiple images depict different but adjacent regions of subject 105 .

[0029] After acquiring the image at 205, method 200 continues to 210. At 210, method 200 inputs the acquired image into a classification network, such as classification network 170. Classification network 170 is trained to classify anatomical structures / views depicted in the acquired image. For example, the anatomical structures may include the head, spine, abdomen, legs, etc., and the view may indicate the orientation of the subject relative to the x-ray detector, which is expressed in anatomical terms such as lateral, anteroposterior (AP), posteroanterior (PA), etc. Therefore, the anatomical structures / views classified by the classification network can be output as a combination of the anatomical structures depicted in the image and the views of the anatomical structures depicted in the image. For example, the output of classification network 170 may include anatomical structure / view classifications such as spine APPA, spine right side, spine left side, leg APPA, etc.

[0030] Continuing at 215, the method 200 receives from the classification network 170 a classification of the anatomical structures / views depicted in one or more acquired images. That is, for each acquired image input to the classification network 170, the method 200 receives a corresponding anatomical structure / view classification. For example, if a single radiographic image is acquired at 205, the method 200 receives a classification of the anatomical structures / views depicted in the single radiographic image. If a series of radiographic images is acquired at 205 for image pasting, the method 200 receives a classification of the anatomical structures / views depicted in each radiographic image in the series.

[0031] At 220, the method 200 performs post-processing on one or more acquired images based on the classification of the anatomical structure / view. For example, based on the classification of the anatomical structure / view in the image, the method 200 selects post-processing parameters for post-processing the image according to the classification, because different anatomical structures / views may use different post-processing parameters regarding edge details, contrast, noise level, etc. Therefore, a post-processing protocol including a set of post-processing parameters may be configured for each anatomical structure / view, and the method 200 may select an appropriate post-processing protocol based on the classification. The method 200 then processes the acquired image according to the post-processing protocol or according to the post-processing parameters selected based on the anatomical structure / view classification. In this way, the automatic classification of the anatomical structure / view can automatically select the correct parameters for post-processing of the image, thereby ensuring a consistent appearance for the user.

[0032] As another example, for multiple images acquired in a view, post-processing of the acquired images may include post-processing of contrast, edge details, noise levels, etc. for a single image as mentioned above, and may also include image pasting or image stitching. That is, in some examples, post-processing of the acquired images may include stitching the multiple images together into a single image having a larger field of view than the field of view of each individual image. In order to ensure that the correct images are stitched together according to the correct protocol, method 200 automatically selects a protocol for stitching images based on the anatomical structure / view classification of each image. In this way, the image pasting registration success rate is improved while eliminating the need for the user to manually select a protocol for each view.

[0033] At 225, the method 200 outputs the post-processed image. The method 200 may, for example, output the post-processed image to the display device 155 for display to a user. In addition, the method 200 may output the post-processed image to the memory 162 for later viewing, and / or to a picture archiving and communication system (PACS). After outputting the post-processed image, the method 200 then returns.

[0034] Figure 3 A high-level flow chart illustrating an exemplary method 300 for post-processing acquired radiographic images based on anatomical structure / view classification according to an embodiment is shown. Specifically, the method 300 involves automatically selecting parameters for post-processing acquired images based on automatically classified anatomical structures / views depicted in the acquired images. Figure 1The method 300 is described with reference to the systems and components of the present disclosure, but it should be understood that the method 300 can be implemented with other systems and components without departing from the scope of the present disclosure. The method 300 can be stored as executable instructions 172 in the non-volatile memory 162 of the operating console 160, for example, and can be executed by one or more of the processor 161 and the image processor 150 to perform the actions described below in this document.

[0035] The method 300 begins at 305. At 305, the method 300 receives a selection of a protocol. Specifically, the method 300 receives a selection of an acquisition protocol that specifies one or more parameters, such as tube voltage, tube current, etc., for acquiring an image. The method 300 may receive the selection of a protocol, for example, via the user interface 163. Continuing at 310, the method 300 controls an x-ray source, such as the x-ray source 111, to acquire an image according to the selected protocol. The method 300 may also control an x-ray detector, such as the x-ray detector 134, to acquire an image according to the protocol.

[0036] After acquiring the image, method 300 continues to 315. At 315, method 300 inputs the acquired image into a classification network, such as classification network 170. As discussed above, classification network 170 is trained to classify anatomical structures / views depicted in the acquired image. For example, the anatomical structures may include the head, spine, abdomen, legs, etc., and the view may indicate the orientation of the subject relative to the x-ray detector, which is expressed in anatomical terms such as lateral, anteroposterior (AP), posteroanterior (PA), etc. Therefore, the anatomical structures / views classified by the classification network can be output as a combination of the anatomical structures depicted in the image and the views of the anatomical structures depicted in the image. For example, the output of classification network 170 may include anatomical structure / view classifications, such as spine APPA, spine right side, spine left side, leg APPA, etc.

[0037] Thus, continuing at 320, the method 300 receives a classification of the anatomical structure / view depicted in the acquired image from the classification network 170. At 325, the method 300 performs post-processing on the acquired image based on the anatomical structure / view classification. For example, based on the classification of the anatomical structure / view in the image, the method 300 selects post-processing parameters for post-processing the image according to the classification, because different anatomical structures / views can use different post-processing parameters with respect to edge details, contrast, noise level, etc. Therefore, a post-processing protocol including a set of post-processing parameters can be configured for each anatomical structure / view, and the method 300 can select an appropriate post-processing protocol based on the classification. The method 300 then processes the acquired image according to the post-processing protocol or according to the post-processing parameters selected based on the anatomical structure / view classification. In this way, the automatic classification of the anatomical structure / view can automatically select the correct parameters for post-processing of the image, thereby ensuring a consistent appearance for the user.

[0038] At 330, the method 300 outputs the post-processed image. For example, the method 300 may output the post-processed image to the display device 155 for display to the user. In addition, the method 300 may output the post-processed image to the memory 162 for storage, and / or to a PACS for remote storage and / or viewing. The method 300 then returns.

[0039] As an illustrative example, Figure 4An image set 400 showing automatic classification of acquired images is shown. The image set 400 includes an exemplary acquired image 405. The image set 400 also includes an exemplary output 410 of the classification network, which includes the acquired image 405 and an anatomy / view classification 412 for the acquired image 405. As depicted, the anatomy / view classification 412 indicates that the view of the acquired image 405 is a "left lateral" view. In some examples, the anatomy / view classification 412 can be superimposed on the acquired image 405 and can be displayed to the user, for example, via the display device 155, for confirmation. In other examples, the anatomy / view classification 412 may not be superimposed on the acquired image 405 or displayed to the user. In addition, while the anatomy / view classification 412 does not explicitly indicate the anatomy as depicted, in some examples, the anatomy / view classification 412 can further explicitly indicate the anatomy, for example, by indicating the "neck" anatomy in addition to indicating the "left lateral" view in the anatomy / view classification 412. As discussed above, the acquired image 405 is post-processed according to the anatomical structure / view classification 412. Specifically, one or more post-processing parameters, including but not limited to field of view, contrast, edge detail, noise reduction, etc., may be determined based on the anatomical structure / view classification 412, and the acquired image 405 may be processed according to the one or more post-processing parameters to generate a post-processed image 415. In this way, the workload of manually selecting post-processing or view protocols may be reduced while maintaining high-quality post-processing performance.

[0040] Figure 5 A high-level flow chart illustrating an exemplary method 500 for image pasting of radiographic images based on anatomy / view classification according to an embodiment is shown. Specifically, the method 500 involves acquiring at least one view of a plurality of images and automatically post-processing the plurality of images based on automatic anatomy / view classification to obtain at least one stitched image. Figure 1 The method 500 is described with reference to the systems and components of the present disclosure, but it should be understood that the method 500 can be implemented with other systems and components without departing from the scope of the present disclosure. The method 500 can be stored as executable instructions 172 in, for example, the non-transitory memory 162 of the operating console 160, and can be executed by one or more of the processor 161 and the image processor 150 to perform the actions described below in this document.

[0041] The method 500 begins at 505. At 505, the method 500 receives a selection of a protocol. Specifically, the method 500 receives a selection of an acquisition protocol that specifies one or more parameters for acquiring images, such as tube voltage, tube current, etc. The acquisition protocol may include an image pasting acquisition protocol and therefore may include a definition of patient coverage. The acquisition protocol may correspond to a specific view. For example, the acquisition protocol may include an APPA acquisition protocol or a lateral acquisition protocol. In addition, the acquisition protocol may specify the anatomical structure to be imaged so that appropriate acquisition parameters may be selected. The method 500 may receive the selection of a protocol, for example, via the user interface 163.

[0042] At 510, the method 500 acquires at least one view of the plurality of images. To this end, the method 500 controls the x-ray source 111 and the x-ray detector 134 to acquire a series of radiographic images, for example, according to the protocol selected at 505, while adjusting the relative positions of the x-ray source 111 and the x-ray detector 134 so that the radiographic images depict different but adjacent regions of the subject 105. In such an example, the method 500 may control the x-ray source 111 and the x-ray detector 134 to acquire a first view (e.g., APPA, lateral, etc.) of the plurality of images of the subject 105 and a second view different from the first view.

[0043] After acquiring the plurality of images, the method 500 continues to 515. At 515, the method 500 inputs the plurality of images into a classification network, such as the classification network 170. As discussed above, the classification network 170 is trained to classify the anatomical structures / views depicted in the acquired images. For example, the anatomical structures may include the head, spine, abdomen, legs, etc., and the views may indicate the orientation of the subject relative to the x-ray detector, which is expressed in anatomical terms, such as lateral, anteroposterior (AP), posteroanterior (PA), etc. Therefore, the anatomical structures / views classified by the classification network may be output as a combination of the anatomical structures depicted in the image and the views of the anatomical structures depicted in the image. For example, the output of the classification network 170 may include anatomical structure / view classifications, such as spine APPA, spine right side, spine left side, leg APPA, etc. Therefore, at 520, the method 500 receives a prediction of the anatomical structure / view for each of the plurality of images. That is, the classification network 170 outputs a classification of the anatomy / view for each image, as well as a prediction or probability as to the likelihood that the anatomy / view classification for each image is correct.

[0044] Continuing at 525, method 500 determines the anatomical structure / view of each image based on the prediction. Figure 6As further described, a method for determining the anatomy / view of each image based on the prediction of the anatomy / view classification output by the classification network 170 includes evaluating each prediction according to one or more conditions to ensure that the image is accurately classified. Therefore, in some examples, the method 500 can determine that the anatomy / view classification of each image predicted by the classification network 170 is correct. At the same time, in other examples, for example, if the second image is predicted to be the same as the anatomy / view classification of the first image while still having a high probability of belonging to the anatomy / view classification, and if no image is classified as a second view of the corresponding anatomy by the classification network 170, the method 500 can adjust the anatomy / view classification of the first image from the first view to the second view. That is, even in the case of redundant anatomy / view classifications, the method 500 can automatically and accurately derive the appropriate anatomy / view classification.

[0045] After determining the anatomical structure / view classification of each of the multiple images, method 500 continues to 530. At 530, method 500 performs image pasting on the multiple images based on the determined anatomical structure / view to generate a stitched image for each view. In some examples, method 500 can process each image according to the corresponding anatomical structure / view classification using post-processing parameters selected according to the anatomical structure / view classification. In addition, method 500 selects an image pasting protocol for each view to apply to multiple images determined to correspond to the view, and then performs image pasting on the multiple images according to the image pasting protocol. Therefore, images in the lateral view can be automatically stitched according to the lateral image pasting protocol, while images in the anteroposterior view can be automatically stitched according to the anteroposterior image pasting protocol. By using the correct image pasting protocol, the alignment of images within the view can be improved.

[0046] At 535, the method 500 outputs the stitched image. For example, the method 500 may output the stitched image of each view to the display device 155 for display, and may additionally output the stitched image to the memory 162 and / or the PACS. The method 500 then returns.

[0047] Figure 6 A high-level flow chart illustrating an exemplary method 600 for determining an anatomical structure / view classification based on predictions of the anatomical structure / view classification for a plurality of radiographic images is shown in accordance with an embodiment. Specifically, the method 600 may include a subroutine of the method 500, for example by including the action 525 described above. Figure 1 systems and components and Figure 5Method 600 is described with reference to method 500, but it should be understood that method 600 may be implemented with other systems, components, and methods without departing from the scope of the present disclosure. Method 600 may be stored as executable instructions 172 in, for example, non-transitory memory 162 of operating console 160, and may be executed by one or more of processor 161 and image processor 150 to perform the actions described herein below.

[0048] The method 600 starts at 605. At 605, the method 600 evaluates predictions for anatomical structures / views for each of the plurality of images, the predictions including predicted labels and probabilities for leg predictions. At 610, the method 600 determines whether a first condition is satisfied. Specifically, the method 600 determines whether the probability P of at least one image being classified as a leg is greater than a probability threshold P t If the first condition is met ("yes"), the method 600 continues to 612, where the method 600 classifies the anatomical structure / view as a leg. The method 600 then returns.

[0049] However, if the first condition is not met at 610 ("No"), the method 600 continues to 615. At 615, the method 600 determines whether the second condition is met. Specifically, the method 600 determines whether all images are predicted as the same class. If the second condition is met ("Yes"), the method 600 continues to 617, where the method 600 determines that the predictions are the prediction results of one image. Then, the method 600 returns.

[0050] However, if the second condition is not met at 615 ("No"), method 600 continues to 620. At 620, method 600 determines whether the third condition is met. Specifically, method 600 determines whether one label has a higher occurrence rate than other labels in the predictions. If the third condition is met ("Yes"), method 600 continues to 622. At 622, method 600 determines that most images correspond to labels that have a higher occurrence rate than other labels. Then, method 600 returns.

[0051] However, if the third condition is not met at 620 ("No"), the method 600 continues to 625. At 625, the method 600 deletes the prediction result for the first image. The method 600 then returns to 605 to continue evaluating the prediction. Thus, the method 600 evaluates multiple images until the method 600 terminates under one of the conditions. By evaluating the predictions for the anatomy / view classification according to the method 600, conflicting anatomy / view classifications can be automatically resolved.

[0052] To illustrate the image pasting method provided in this article, Figure 7 The embodiment according to the embodiment is shown Figure 5An exemplary image set 700 of the method 500 of the present invention is shown. The exemplary image set 700 includes a plurality of acquired images 705. The acquired images 705 include a first image set 710 and a second image set 715, wherein the first set 710 includes a first image 711, a second image 712, and a third image 713, and the second set 715 includes a fourth image 715, a fifth image 716, and a sixth image 717. As depicted, the first image set 710 is acquired for a first view (specifically, a PA view), and the second image set 715 is acquired for a second view (specifically, a lateral view) different from the first view. The plurality of acquired images 705 are input to a classification network, such as the classification network 170, to automatically determine an anatomical structure / view classification for each of the plurality of acquired images 705.

[0053] As depicted, the classification network 170 outputs a set of predictions 725 for the acquired image 705, including a first set of predictions 730 and a second set of predictions 735. The first set of predictions 730 includes a first prediction 731 for a first image 711 in the first set of images 710, a second prediction 732 for a second image 712, and a third prediction 733 for a third image 713. The second set of predictions 735 includes a fourth prediction 736 for a fourth image 716 in the second set 715, a fifth prediction 737 for a fifth image 717, and a sixth prediction 738 for a sixth image 718.

[0054] Specifically, the first prediction 731 indicates that the first image 711 is a PA view, the second prediction 732 indicates that the second image 712 is a PA view, and the third prediction 733 indicates that the third image 713 is a side view. Meanwhile, the fourth prediction 736 indicates that the fourth image 716 is a side view, the fifth prediction 737 indicates that the fifth image 717 is a side view, and the sixth prediction 738 indicates that the sixth image 718 is a side view.

[0055] Since both the third prediction 733 and the sixth prediction 738 indicate that the third image 713 and the sixth image 738 correspond to the lateral view having the same anatomical structure, more images are classified as the lateral view than the PA view, and thus as described above with reference to Figure 5 and Figure 6 The prediction set 725 is evaluated as described to determine the appropriate anatomy / view classification for each image. Since the probability of the sixth prediction 738 is higher than the probability of the third prediction 733, the third prediction 733 is reclassified as a PA view.

[0056] Therefore, in the application Figure 6After the decision algorithm of , a classification set 745 is obtained, in which the first classification set 750 is clearly classified as a PA view, and the second classification set 755 is clearly classified as a lateral view. Specifically, the first classification set 750 includes a first image 751, a second image 752, and a third image 753 that are determined to include a first view (specifically, a PA view). The first image 751, the second image 752, and the third image 753 correspond to the first image 711, the second image 712, and the third image 713 of the first set 710, respectively.

[0057] The second classification set 755 includes a fourth image 756, a fifth image 757, and a sixth image 758 that are determined to include a second view (specifically, a side view). The fourth image 756, the fifth image 757, and the sixth image 758 correspond to the fourth image 716, the fifth image 717, and the sixth image 718 of the second image set 715, respectively.

[0058] Then, the classified images 745 are image pasted or stitched to obtain a pasted image 765. Specifically, the first classified image set 750 is stitched according to the PA image pasting protocol to obtain a first pasted image 770, and the second classified image set 755 is stitched according to the lateral image pasting protocol to obtain a second pasted image 775. By using an appropriate image pasting protocol to stitch the classified images 745, the accuracy of the registration of the classified images 745 is improved, and thus the classified images 745 are accurately stitched to generate the pasted image 765.

[0059] Thus, methods and systems are provided for improving post-processing of one or more acquired images, including automatic classification of anatomical structures / views depicted in the one or more acquired images.

[0060] In some examples, an x-ray imaging system, such as x-ray imaging system 100, may include a camera, such as camera 120, co-aligned with x-ray source 111 to capture camera data of subject 105. In such examples, a classification network, such as classification network 170, may be trained to automatically classify anatomical structures / views depicted in such camera data. Thus, an appropriate acquisition protocol may be automatically selected prior to acquiring an x-ray image without manual input from a user of x-ray imaging system 100.

[0061] As an illustrative example, Figure 8 A high-level flow chart illustrating an exemplary method 800 for anatomical structure / view classification for acquisition protocol selection according to an embodiment is shown. Specifically, the method 800 involves classifying anatomical structures / views based on camera data and then acquiring x-ray images based on the classified anatomical structures / views. Figure 1The method 800 is described with reference to the systems and components of the present disclosure, but it should be understood that the method 800 can be implemented with other systems and components without departing from the scope of the present disclosure. The method 800 can be stored as executable instructions 172 in, for example, the non-transitory memory 162 of the operating console 160, and can be executed by one or more of the processor 161 and the image processor 150 to perform the actions described below in this document.

[0062] The method 800 begins at 805. At 805, the method 800 receives camera data from a camera, such as the camera 120, which is co-aligned with an x-ray source, such as the x-ray source 111. For example, the camera data includes an optical or non-radiographic image of the subject 105. As the camera 120 is co-aligned with the x-ray source 111, the captured camera data depicts the orientation of the subject 105 relative to the x-ray source 111 and the x-ray detector 134. Continuing at 810, the method 800 inputs the camera data into a classification network, such as the classification network 170. As discussed above, the classification network 170 is trained to classify the anatomical structures / views depicted in the acquired images. In addition to classifying the anatomical structures / views depicted in the radiographic images as described above, the classification network 170 can also be trained to classify the anatomical structures / views depicted in the camera data. For example, the anatomical structures may include the head, spine, abdomen, legs, etc., and the view may indicate the orientation of the subject relative to the x-ray detector, which is expressed in anatomical terms such as lateral, anteroposterior (AP), posteroanterior (PA), etc. Thus, the anatomical structures / views classified by the classification network can be output as a combination of the anatomical structures depicted in the camera data and the views of the anatomical structures depicted in the camera data. For example, the output of the classification network 170 may include anatomical structure / view classifications such as spine APPA, spine right side view, spine left side view, leg APPA, etc.

[0063] At 815, the method 800 receives a classification of the anatomical structure / view depicted in the camera data from the classification network. Continuing at 820, the method 800 selects a protocol for the x-ray source 111 based on the anatomical structure / view classification. For example, the method 800 automatically selects one or more acquisition parameters based on the anatomical structure / view classification. Then, at 825, the method 800 controls the x-ray source 111 using the selected protocol to acquire an x-ray or radiographic image. In addition, since the anatomical structure / view has been classified based on the camera data, it is not necessary to classify the anatomical structure / view in the acquired x-ray image using the classification network 170. Therefore, continuing at 830, the method 800 performs post-processing on the acquired image based on the anatomical structure / view classification as described above. At 835, the method 800 outputs, for example, the post-processed image to the display device 155 for display to the user. The method 800 may also output the post-processed image to the memory 162 for storage and / or output to the PACS. Then, the method 800 returns.

[0064] Fig. 9 An image set 900 showing anatomical structure / view classification based on camera data is shown according to an embodiment. Image set 900 includes camera images 910 acquired via a camera, such as camera 120, which is co-aligned with an x-ray source, such as x-ray source 111. Camera image 910 depicts an imaged subject 917 positioned in front of a detector 915, which may include Figure 1 The detector 134 of the camera image 910 (also referred to herein as camera data) is input to the classification network 170 to automatically classify the anatomical structure / view of the camera image 910. The output 930 of the classification network 170 includes the camera image 910 and a label or classification 932 of the anatomical structure / view determined by the classification network 170. As depicted, the classification 932 output by the classification network 170 indicates that the view corresponds to an APPA view. It should be understood that although in Fig. 9 , but the classification 932 may also indicate an anatomical structure depicted in the camera data 910, such as a spine or a torso. As described above, the classification 932 may be used to automatically select an acquisition protocol for controlling the x-ray source 111 and the x-ray detector 134 to acquire one or more radiographic images. In addition, the classification 932 may also be used to determine a view or post-processing parameter to post-process the acquired radiographic images. Thus, by including the camera 120 in the x-ray imaging system 100 and including the classification network 170, the user may be relieved from selecting an acquisition protocol and / or a view or post-processing protocol.

[0065] Technical effects of the present disclosure include classification of anatomical structures and views depicted in images. Another technical effect of the present disclosure includes automatic selection of acquisition protocols and / or post-processing protocols based on anatomical structures and views. Yet another technical effect of the present disclosure includes improved registration between images in image pasting inspection. Another technical effect of the present disclosure includes accurate processing of acquired radiological images based on automatically classified anatomical structures and views depicted in the acquired radiological images.

[0066] In one embodiment, a method includes: controlling an x-ray source and an x-ray detector to acquire an image of a subject; classifying an anatomical structure / view depicted in the image using a trained neural network; performing post-processing of the image based on the anatomical structure / view; and displaying the post-processed image.

[0067] In a first example of the method, the method further includes: determining parameters for adjusting one or more of edge details, contrast, and noise levels in the image according to the anatomical structure / view; and performing post-processing of the image according to the determined parameters. In a second example of the method, which optionally includes the first example, the method further includes controlling an x-ray source and an x-ray detector to acquire a second image of the subject, wherein the first image and the second image depict adjacent and overlapping areas of the subject. In a third example of the method, which optionally includes one or more of the first example and the second example, the method further includes classifying the second anatomical structure / view depicted in the second image using a trained neural network. In a fourth example of the method, which optionally includes one or more of the first to third examples, performing post-processing of the image based on the anatomical structure / view includes: registering the image to the second image according to the anatomical structure / view and the second anatomical structure / view; and stitching the image to the second image to generate a stitched image, and displaying the post-processed image includes displaying the stitched image. In a fifth example of the method, which optionally includes one or more of the first to fourth examples, the method further includes adjusting the contrast of the image and the second image and performing noise reduction according to the anatomical structure / view and the second anatomical structure / view, respectively, before registering the image to the second image. In a sixth example of the method, which optionally includes one or more of the first to fifth examples, the method further includes controlling an x-ray source and an x-ray detector to acquire a third image and a fourth image of the subject, wherein the first image and the second image depict a first view of adjacent and overlapping regions of the subject, and wherein the third image and the fourth image depict a second view of adjacent and overlapping regions of the subject that is different from the first view. In a seventh example of the method, which optionally includes one or more of the first to sixth examples, the method further includes classifying a third anatomical structure / view depicted in the third image and a fourth anatomical structure / view depicted in the fourth image with a trained neural network. In an eighth example of the method, which optionally includes one or more of the first to seventh examples, the method further includes: registering the third image to the fourth image according to the third anatomical structure / view and the fourth anatomical structure / view; stitching the third image to the fourth image to generate a second view of a second stitched image; and displaying the second stitched image. In a ninth example of the method, which optionally includes one or more of the first to eighth examples, the method also includes receiving probabilities of corresponding anatomical structures / views classified for the image, the second image, the third image, and the fourth image from the trained neural network and determining whether the corresponding anatomical structure / view is correct based on the probabilities before the second probability, respectively.

[0068] In another embodiment, a method includes: acquiring one or more images of a subject; determining, using a classification network, an orientation and position of the subject relative to an x-ray source and an x-ray detector from the one or more images of the subject; processing one or more x-ray images of the subject acquired using the x-ray source and the x-ray detector based on the orientation and position of the subject; and displaying one or more processed x-ray images.

[0069] In a first example of the method, acquiring one or more images of a subject includes acquiring one or more images using an optical camera co-aligned with an x-ray source and also includes selecting one or more acquisition parameters based on orientation and position and controlling an x-ray source and an x-ray detector according to the one or more acquisition parameters to acquire one or more x-ray images. In a second example of the method, it optionally includes the first example, the one or more images include one or more x-ray images, and processing the one or more x-ray images includes adjusting the appearance of the one or more x-ray images according to a view protocol selected based on the orientation and position of the subject. In a third example of the method, it optionally includes one or more of the first example and the second example, the one or more x-ray images include a first plurality of x-ray images of the subject in a first orientation and a second plurality of x-ray images of the subject in a second orientation, wherein determining the orientation using a classification network includes determining a first orientation of the first plurality of x-ray images and a second orientation of the second plurality of x-ray images. In a fourth example of the method, which optionally includes one or more of the first to third examples, processing one or more x-ray images based on the orientation and position of the subject includes: aligning and stitching a first plurality of x-ray images based on a first orientation into a first stitched image having a first field of view larger than each of the first plurality of x-ray images; and aligning and stitching a second plurality of x-ray images based on a second orientation into a second stitched image having a second field of view larger than each of the second plurality of x-ray images.

[0070] In yet another embodiment, an x-ray imaging system includes an x-ray source for generating x-rays; an x-ray detector configured to detect x-rays; a display device; and a processor configured to have instructions in a non-volatile memory that, when executed, cause the processor to: control the x-ray source and the x-ray detector to acquire an image of a subject; classify anatomical structures / views depicted in the image using a trained neural network; perform post-processing of the image based on the anatomical structures / views; and display the post-processed image via the display device.

[0071] In a first example of the system, the processor is further configured to have instructions in a non-transitory memory that, when executed, cause the processor to determine parameters for adjusting one or more of edge detail, contrast, and noise level in the image according to the anatomical structure / view and perform post-processing of the image according to the determined parameters. In a second example of the system, which optionally includes the first example, controlling an x-ray source and an x-ray detector to acquire an image includes controlling an x-ray source and an x-ray detector to acquire a plurality of images of a subject, and classifying the anatomical structure / view depicted in the image includes classifying the anatomical structure / view depicted in each image of the plurality of images using a trained neural network. In a third example of the system, which optionally includes one or more of the first and second examples, the processor is further configured to have instructions in a non-transitory memory that, when executed, cause the processor to: determine a first plurality of images corresponding to a first view in a plurality of images; determine a second plurality of images corresponding to a second view in a plurality of images; register and stitch the first plurality of images into a first stitched image according to the first view; and register and stitch the second plurality of images into a second stitched image according to the second view. In a fourth example of the system, which optionally includes one or more of the first to third examples, the system also includes an optical camera positioned adjacent to and co-aligned with an x-ray source, wherein the processor is further configured to have instructions in a non-volatile memory that, when executed, cause the processor to: acquire camera data via the optical camera; classify the anatomical structure / view of the subject depicted in the camera data using the trained neural network; select one or more acquisition parameters based on the anatomical structure / view of the subject depicted in the camera data; and control the x-ray source and the x-ray detector using the one or more acquisition parameters to acquire an image of the subject.

[0072] As used herein, the elements or steps listed in the singular and beginning with the word "one" or "a kind of" should be understood as not excluding a plurality of the elements or steps, unless such exclusion is explicitly stated. In addition, the reference to "an embodiment" of the present invention is not intended to be interpreted as excluding the existence of additional embodiments that also include the cited features. In addition, unless explicitly stated to the contrary, the embodiment of "including", "comprising" or "having" an element or multiple elements with a specific characteristic may include additional such elements without the characteristic. The terms "including" and "in..." are used as the concise language equivalents of the corresponding terms "including" and "wherein". In addition, the terms "first", "second" and "third" etc. are only used as marks, and are not intended to impose numerical requirements or specific positional order on their objects.

[0073] This written description uses examples to disclose the invention, including the best mode, and also to enable a person of ordinary skill in the relevant art to practice the invention, including making and using any devices or systems and performing any included methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to a person of ordinary skill in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insignificant differences from the literal language of the claims.

Claims

1. A method comprising: controlling an x-ray source and an x-ray detector to acquire a plurality of images of a subject; utilizing the trained neural network to classify a corresponding anatomical structure / view depicted in each of the plurality of images, wherein each corresponding anatomical structure / view includes a predicted anatomical structure and a predicted view depicted in that image; performing post-processing of each image based on the corresponding anatomical structure / view, including selecting two or more images classified as having the same predicted view from the plurality of images and stitching the two or more images to form a stitched image; and The stitched image is displayed.

2. The method according to claim 1 also includes determining parameters for adjusting one or more of edge detail, contrast and noise level in each image according to the corresponding anatomical structure / view and performing the post-processing of the images according to the determined parameters.

3. The method of claim 1, wherein the predicted view is a first predicted view, and the method further comprises: determining, based on a corresponding anatomical structure / view depicted in each of the plurality of images, that the plurality of images further include two or more additional images classified as having a second predicted view different from the first predicted view; stitching the two or more additional images together to form a second stitched image; as well as The second stitched image is displayed.

4. The method of claim 1 , wherein the plurality of images comprises a first image and a second image depicting adjacent and overlapping regions of the subject, and wherein classifying the corresponding anatomical structures / views depicted in each of the plurality of images using the trained neural network comprises classifying at least a first anatomical structure / view depicted in the first image and a second anatomical structure / view depicted in the second image.

5. The method of claim 4, wherein performing the post-processing comprises registering the plurality of images according to the anatomical structures / views depicted in the plurality of images, comprising registering the first image to the second image according to the first anatomical structure / view and the second anatomical structure / view, and stitching the registered first and second images to generate the stitched image. 6 . The method of claim 5 , wherein stitching the first image and the second image comprises stitching the first image and the second image according to an image pasting protocol configured for the predicted views of the first image and the second image.

7. The method according to claim 6 also includes controlling the x-ray source and the x-ray detector to acquire a second plurality of images of the subject, the second plurality of images including a third image and a fourth image, wherein the plurality of images including the first image and the second image depict adjacent and overlapping areas of the subject in a first view, and wherein the second plurality of images depict adjacent and overlapping areas of the subject in a second view different from the first view.

8. The method of claim 7, further comprising classifying, using the trained neural network, corresponding anatomical structures / views depicted in each of the second plurality of images, the corresponding anatomical structures / views depicted in each of the second plurality of images comprising a third anatomical structure / view depicted in the third image and a fourth anatomical structure / view depicted in the fourth image.

9. The method according to claim 8, further comprising: determining that the corresponding anatomical structure / view depicted in each image of the second plurality of images includes the second view; registering the second plurality of images according to an image pasting protocol for the second view; stitching the registered second plurality of images to generate a second stitched image having the second view; and displaying the second stitched image.

10. The method of claim 9, further comprising receiving probabilities of corresponding anatomical structures / views classified for the plurality of images and the second plurality of images from the trained neural network, determining whether the corresponding anatomical structures / views are correct based on occurrence rates of the anatomical structures / views and the probabilities, and adjusting the classification of the anatomical structure / view for a given image if one of the occurrence rates of the anatomical structure / views and the probabilities indicates that the classification of the anatomical structure / view for the given image is incorrect.

11. An x-ray imaging system comprising: An x-ray source, the x-ray source is used to generate x-rays; an x-ray detector configured to detect the x-rays; Display devices; as well as A processor configured with instructions in a non-transitory memory, the instructions, when executed, causing the processor to: controlling the x-ray source and the x-ray detector to acquire a plurality of images of a subject; classifying, using the trained neural network, a corresponding anatomical structure / view depicted in each of the plurality of images; receiving from the trained neural network a respective probability for each anatomical structure / view classified for the plurality of images, determining whether each anatomical structure / view is correct based on an occurrence rate of the anatomical structure / view and each probability, and adjusting the classification of the anatomical structure / view for a given image if one of the occurrence rate of the anatomical structure / view and the probability indicates that the classification of the anatomical structure / view for the given image is incorrect; performing post-processing of each image on a per-anatomical structure / view basis; as well as The post-processed image is displayed via the display device.

12. A system according to claim 11, wherein the processor is further configured to have instructions in the non-volatile memory, which, when executed, cause the processor to determine parameters for adjusting one or more of edge detail, contrast, and noise level in each image based on the corresponding anatomical structure / view and to perform the post-processing of each image based on the determined parameters.

13. The system of claim 11, wherein the processor is further configured to have instructions in the non-transitory memory that, when executed, cause the processor to: determine a first plurality of images of the plurality of images corresponding to a first view; determining a second plurality of images in the plurality of images corresponding to a second view; registering and stitching the first plurality of images into a first stitched image according to the first view; and registering and stitching the second plurality of images into a second stitched image according to the second view.

14. The system of claim 11, further comprising an optical camera positioned adjacent to and co-aligned with the x-ray source, wherein the processor is further configured to have instructions in the non-transitory memory that, when executed, cause the processor to: acquiring camera data via the optical camera; utilizing the trained neural network to classify anatomical structures / views of the subject depicted in the camera data; selecting one or more acquisition parameters based on the anatomy / view of the subject depicted in the camera data; as well as The x-ray source and the x-ray detector are controlled using the one or more acquisition parameters to acquire the plurality of images of the subject.

Images