Providing normalized medical images
By warping 2D medical images from intensive care settings to images from a library, the problem of low image quality is solved, and the shape and intensity of the region of interest are normalized, thereby improving the reliability of image comparison and the accuracy of object condition assessment.
Patent Information
- Application Number
- CN202380027601.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-03-14
- Filing Date
- 2023-03-01
- Publication Date
- 2024-11-05
AI Technical Summary
The low quality of medical images acquired in intensive care settings makes it challenging to assess longitudinal changes in the condition of individuals, and existing technologies struggle to effectively identify and assess temporal changes between images.
By receiving time-series 2D medical images and library images, techniques such as affine transformation are used to warp the medical images to the library images to provide normalized medical images and output the variation amplitude of the region of interest, reducing shape and intensity differences and improving the reliability of image comparison.
It achieves efficient normalization of the shape and intensity of the region of interest, improves reliable comparison between images, and allows for more accurate assessment of longitudinal variations in object condition.
Smart Images

Figure CN118901082A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to providing a normalized medical image representing a region of interest in a subject. A computer-implemented method, computer program product, and system are disclosed. Background Art
[0002] Clinical research often involves acquiring medical images of a subject. The images may be acquired at different points in time, i.e., as a time series of images. The images may be used to identify temporal changes in a region of interest, and thereby assess the progression of a medical condition in the subject. However, differences in the manner in which images are acquired may hamper clinical research. For example, differences in the subject's posture, differences in the viewing angle of the medical imaging system, and differences in the amount of ionizing radiation dose used to acquire the images exacerbate the challenge of identifying temporal changes in the region of interest itself. Such differences therefore reduce the diagnostic value of the images and may lead to misdiagnosis of the subject's condition.
[0003] Some clinical environments allow for positioning of the subject in an optimal arrangement relative to the imaging system. In such environments, there may be a relatively small amount of image-to-image variability caused by differences in the way the images are acquired. However, in other clinical environments, positioning the subject in an optimal arrangement may be impractical or even impossible. For example, in a critical care environment, it may be impractical to acquire images of the subject using a desired posture or with the subject in a desired position relative to the medical imaging system.
[0004] As an example, acquiring diagnostic chest X-ray images in a critical care environment in order to assess the progression of a lung condition can be a challenging task for a radiologist. The immobility of the subject (who is typically bedridden), the limited space to maneuver a mobile X-ray imaging system, and the bulky nature of such an imaging system are just some of the confounding factors that present challenges to the radiologist. The resulting X-ray images typically suffer from degraded image quality due to suboptimal fields of view, suboptimal viewing angles, limitations on X-ray dose, and limitations on subject posture (such as the subject's inhalation state, its arm position, and its semi-upright position). Therefore, X-ray images acquired in a critical care environment may have significantly lower image quality than images acquired during a standard X-ray chest examination. This makes it challenging to interpret X-ray images. In particular, the fact that these confounding factors are not constant over time, and they differ between each image, makes it challenging to assess longitudinal changes in the subject's condition. Therefore, radiologists interpreting such images may instinctively compensate for such differences, which may lead to misdiagnosis of lung conditions.
[0005] Therefore, there is a need for improvements that facilitate identifying temporal changes in medical images.
[0006] Document US2007 / 003117 A1 discloses a technique for comparative image analysis and / or change detection using a computer-aided detection and / or diagnosis "CAD" algorithm. The technique includes registering two or more images, comparing the images to each other to generate a change map, and detecting abnormalities in the images based on the change map. Summary of the invention
[0007] According to one aspect of the present disclosure, there is provided a computer-implemented method for providing a normalized medical image representing a region of interest in an object. The method comprises:
[0008] receiving image data comprising a time series of 2D medical images including a region of interest in the object;
[0009] receiving a gallery image representing the region of interest; and
[0010] For each medical image in the time series:
[0011] warping the medical image to the gallery image to provide a normalized medical image with a warped region of interest for comparison with the region of interest in the gallery image; and
[0012] The method further comprises outputting the normalized medical image, and / or outputting a magnitude of a change in the warped region of interest between the normalized medical images.
[0013] In the above method, since the medical images in the time series are warped to the gallery images, the shapes of the regions of interest are more similar in each normalized medical image. This facilitates more reliable comparisons between regions of interest in the images, thereby allowing a more reliable assessment of longitudinal changes in the subject's condition.
[0014] Other aspects, features and advantages of the present disclosure will become apparent from the following description of examples with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is a flow chart illustrating an example of a method of providing a normalized medical image representing a region of interest in a subject according to some aspects of the present disclosure.
[0016] Figure 2 is a schematic diagram illustrating an example of a system 200 for providing a normalized medical image representing a region of interest in a subject, according to some aspects of the present disclosure.
[0017] Figure 3 An example of a lung field gallery image 130 is illustrated in accordance with some aspects of the present disclosure.
[0018] Figure 4 An example of a thorax gallery image 130 is illustrated in accordance with some aspects of the present disclosure.
[0019] Figure 5 Illustrated is an example of warping two 2D medical images in a time series 1101 , 1102 to a gallery image to provide corresponding normalized medical images 110 ′ 1 , 110 ′ 2 according to some aspects of the present disclosure.
[0020] Figure 6 An example of the results of applying a feature detector to a medical image 1101 to identify a plurality of landmarks in the medical image according to some aspects of the present disclosure is illustrated.
[0021] Figure 7 An example of warping two 2D medical images in a time series 1101 , 1102 to a gallery image and adjusting the intensity of the medical images according to some aspects of the present disclosure is illustrated.
[0022] Figure 8 Illustrated is an example of warping two 2D medical images 1101 , 1102 in a time series to a gallery image, adjusting the intensity of the medical images, and suppressing image features outside the lung region of interest, according to some aspects of the present disclosure.
[0023] Fig. 9 2D medical image 110 according to some aspects of the present disclosure is illustrated. 1..n Time series (above), 2D medical images 110 1..n The time series have been warped to the gallery image and their intensities have been adjusted to provide adjusted intensity values, and the corresponding normalized image 110' 1..n Examples of a time series of (middle) and a time series of corresponding normalized images where image features outside the lung region of interest have been suppressed (bottom). DETAILED DESCRIPTION
[0024] Examples of the present disclosure are provided with reference to the following description and accompanying drawings. In the description, many specific details of certain examples are set forth for the purpose of explanation. References in the specification to "example," "implementation," or similar language mean that the features, structures, or characteristics described in conjunction with the example are included in at least one of the examples. It should also be understood that features described with respect to one example may also be used in another example, and for the sake of brevity, all features do not have to be repeated in each example. For example, features described with respect to a computer-implemented method may be implemented in a corresponding manner in a computer program product and system.
[0025] In the following description, reference is made to a computer-implemented method comprising providing a normalized medical image representing a region of interest in a subject. In some examples, reference is made to a region of interest that is a lung of a subject. However, it should be understood that this region of interest is used as an example only, and the methods disclosed herein may alternatively be used to provide a normalized medical image representing other regions of interest in a subject.
[0026] Note that the computer-implemented method disclosed herein may be provided as a non-transient computer-readable storage medium including computer-readable instructions stored thereon, which cause the at least one processor to perform the method when executed by at least one processor. In other words, the computer-implemented method may be implemented as a computer program product. The computer program product may be provided by dedicated hardware or hardware capable of running software in association with appropriate software. When provided by a processor, the function of the method feature may be provided by a single dedicated processor, by a single shared processor, or by a plurality of separate processors that may be shared by some of them. The function of one or more method features may be provided, for example, by a processor shared within a networked processing architecture (such as a client / server architecture, a peer-to-peer architecture, the Internet, or a cloud).
[0027] The explicit use of the term "processor" or "controller" should not be interpreted as exclusively referring to hardware capable of running software, and may implicitly include, but is not limited to, digital signal processor "DSP" hardware, read-only memory "ROM" for storing software, random access memory "RAM", non-volatile storage devices, etc. In addition, examples of the present disclosure may take the form of a computer program product accessible from a computer-usable storage medium or a computer-readable storage medium, which provides program code for use by or in conjunction with a computer or any instruction execution system. For the purposes of this description, a computer-usable storage medium or a computer-readable storage medium may be any device that can contain, store, communicate, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The medium may be an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system or device or a propagation medium. Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer disk, random access memory "RAM", read-only memory "ROM", hard disk and optical disk. Current examples of optical disks include compact disk - read only memory "CD-ROM", compact disk - read / write "CD-R / W", Blu-ray Disc TM and DVD.
[0028] As noted above, a need exists for improvements that facilitate identification of temporal changes in medical images.
[0029] Figure 1is a flow chart illustrating an example of a method of providing a normalized medical image representing a region of interest in a subject according to some aspects of the present disclosure. Figure 2 is a schematic diagram illustrating an example of a system 200 for providing a normalized medical image representing a region of interest in a subject according to some aspects of the present disclosure. Figure 1 The operations described in the method shown can also be performed in Figure 2 The system 200 shown in FIG. 1 is implemented in FIG. 1 , and vice versa. Figure 1 , providing a normalized medical image 110' representing a region of interest 120 in a subject 1..n The computer-implemented method comprises:
[0030] Receiving S110 includes 2D medical image 110 1..n The time series image data, the 2D medical image 110 1..n including a region of interest 120 in the object;
[0031] receiving S120 a gallery image 130 representing the region of interest 120; and
[0032] For each medical image 110 in the time series 1..n :
[0033] The medical image is warped S130 to a gallery image 130 to provide a warped region of interest 120′ 1..n Normalized medical image 110' 1..n , for comparison with the region of interest 120 in the gallery image 130; and
[0034] The method further comprises outputting S140a a normalized medical image 110' 1..n , and / or output S140b normalized medical image 110' 1..n The warped region of interest 120' between 1..n the magnitude of the change.
[0035] Since in the above method, the medical images in the time series are warped to the gallery images, the shapes of the regions of interest are more similar in each normalized medical image. This facilitates more reliable comparisons between regions of interest in the images, thereby allowing more reliable assessments of longitudinal changes in the subject's condition.
[0036] refer to Figure 1 In operation S110, image data is received. The image data includes a 2D medical image 110 1..n Time series, 2D medical images 110 1..nThe received 2D medical images form a time series in the sense that the images are acquired at different points in time. The images may be acquired periodically (i.e., at regular intervals) or intermittently (i.e., at irregular intervals). The time series of 2D medical images may be acquired over a period of time (such as over a period of minutes, hours, days, weeks, or longer). The time series of 2D medical images may represent a so-called longitudinal study of the object. In general, the region of interest represented in the image may be any anatomical region. For example, the region of interest may be a lung, a heart, a liver, a kidney, etc. As an example, the 2D medical image 110 received in operation S110 1..n The time series of 2D medical images may represent a chest of a subject. The time series of 2D medical images may, for example, represent daily images of the chest acquired as part of a longitudinal study in a subject to assess the progression of COVID 19 in the lungs of the subject.
[0037] The 2D medical image 110 received in operation S110 1..n The time series may generally be generated by any 2D medical imaging system. The 2D medical image may include, for example, an X-ray image or an ultrasound image. The 2D X-ray image may generally be generated by any projection X-ray imaging system. The 2D medical image 110 may be generated 1..n Examples of current X-ray projection imaging systems include the MobileDiagnost M50 (which is a mobile X-ray imaging system), the DigitalDiagnost C90 (which is a ceiling mounted X-ray imaging system), and the Azurion 7 (which includes a source-detector arrangement mounted to a C-arm), all of which are sold by Philips Healthcare, Best, the Netherlands. 2D ultrasound images can be generated by any 2D ultrasound imaging system.
[0038] As an example, the 2D medical image 110 received in operation S110 1..n Can include Figure 2 An X-ray image generated by a projection X-ray imaging system 220 is shown. Figure 2 The projection X-ray imaging system shown in is a so-called mobile X-ray imaging system which is typically used in intensive care environments in order to obtain images of objects with restricted mobility. Figure 2 The projection X-ray imaging system 220 shown in FIG. 1 includes an X-ray source 230 and a wirelessly coupled X-ray detector 240. The versatility of this type of X-ray imaging system is helpful in complex environments where the subject has limited mobility, such as Figure 2However, as discussed above, assessing the subject's condition from images acquired in such an environment can be challenging given the limited mobility of the subject, who may not be able to position themselves in an optimal or even repeatable posture for each imaging session. Figure 5 The left side provides Figure 2 Illustration of the posture variability and quality variability of images acquired in a bed environment as shown in . Figure 5 Illustrated is an example of warping two 2D medical images in a time series 1101 , 1102 to a gallery image to provide corresponding normalized medical images 110 ′ 1 , 110 ′ 2 according to some aspects of the present disclosure. Figure 5 The two 2D X-ray images 1101 and 1102 on the left side of show the chest of a subject at two different points in time. As can be appreciated, the identification of changes in the state of the subject's lungs in these images is confounded by differences in the subject's posture and differences in the viewing angle of the X-ray imaging system relative to the subject. It should be noted that Figure 2 The environment shown in is intended only as an example, and it should be understood that the principles disclosed herein are not limited to this type of imaging system or this type of environment.
[0039] Return to Figure 1 Typically, the time series 110 of the 2D medical images received in operation S110 1..n The medical images may be received via any form of data communication, including, for example, wired, optical, and wireless communications. As some examples, when wired or optical communications are used, the communications may be via signals transmitted over electrical or optical cables, and when wireless communications are used, the communications may be, for example, via RF or optical signals. 1..n can be performed by one or more processors (such as Figure 2 The one or more processors 210 shown in FIG. 2 may receive the one or more processors 210 from a 2D medical imaging system such as Figure 2 ), or from a 2D ultrasound imaging system, or from another source such as a computer readable storage medium, the Internet, or the cloud. 1..n .
[0040] Continue to refer Figure 1 In operation S120 , a gallery image 130 representing the region of interest 120 is received. Figure 3 An example of a lung field gallery image 130 is illustrated in accordance with some aspects of the present disclosure. Figure 4 An example of a thorax gallery image 130 is illustrated in accordance with some aspects of the present disclosure. Figure 3 and Figure 4The gallery image 130 shown in FIG. 1 indicates the probability of structures belonging to the lung boundary and the ribs, respectively, by their intensity values. For example, the gallery image 130 may be composed of Figure 2 Received by one or more processors 210 shown in . Gallery images can be received from a database of gallery images. Gallery images can be selected from the database based on the similarity between the object and the reference object represented in the gallery image. For example, the gallery image can be selected based on the similarity between the age, gender and size of the object and the reference object. The gallery image is used as a reference image of the region of interest. The gallery image can represent the preferred viewing angle of the region of interest in the reference object. The gallery image can be collected when the reference object maintains a preferred posture. As an example, if the region of interest is the lung, the gallery image for the lung can be a so-called posterior-anterior "PA" chest view representing the lung, the bony thorax, the mediastinum and the great vessels. Such a gallery image can be collected with the object upright and facing the upright X-ray detector, in a specified inhalation state, at a specified distance above the shoulder joint in the upper aspect of the X-ray detector, with the chin lifted to leave the image field, with the shoulder rotated forward to allow the scapula to move laterally away from the lung field, and at the specified operating settings (e.g., X-ray kVp energy and exposure time) of the X-ray imaging system. More generally, the gallery images 130 may represent 2D medical images 110 used to evaluate a time series 1..n The preferred viewing angle of a region of interest for a medical condition in a 2D medical image. Using such a gallery image facilitates reliable and also clinically meaningful comparisons between regions of interest in 2D medical images. For example, it can be used to determine clinically meaningful changes in a subject's lung volume, since the changes are observed from a clinically recognized perspective.
[0041] In one example, the gallery images 130 represent 2D medical images 110 used to evaluate a time series 1..n The preferred viewing angle of the region of interest of the medical condition in the Figure 1 The described method includes the operation of selecting a gallery image 130 from a database of gallery images based on a medical condition. In this example, the gallery image 130 may be selected from the database of gallery images further based on similarity between the object and a reference object represented in the gallery image, as described in the examples above.
[0042] In another example, the gallery image 130 represents a 2D medical image 110 used to evaluate a time series 1..n The preferred viewing angle of the region of interest of the medical condition in the Figure 1The described method includes an operation of selecting a gallery image 130 from a database of gallery images based on the type of imaging examination performed on the region of interest. In this example, for example, if the region of interest is the lungs, the gallery image for the lungs can be a so-called posterior-anterior "PA" chest view representing the lungs, bony thorax, mediastinum, and great vessels. Such gallery images can be referred to as a "PA" chest view type of imaging examination. The database can store multiple gallery images, each gallery image representing a different preferred viewing angle for a different type of imaging examination. In this example, the gallery image is selected from the gallery images in the database based on the corresponding type of imaging examination. In this example, the gallery image 130 can be further selected from the gallery image database based on the similarity between the object and the reference object represented in the gallery image, as described in the example above.
[0043] return Figure 1 In operation S130, the medical images 110 in the time series received in operation S110 are then 1..n Warping to the gallery image. The warping operation may be performed using various known transformations. One example of a suitable transformation is an affine transformation. Other examples of suitable transformations include B-splines, thin plate splines, and radial basis functions. Warping S130 may be based on a plurality of corresponding landmarks 140 represented in both the medical image and the gallery image 130. 1..k The warping operation S130 may be performed using a neural network. If the warping is performed by a neural network, the neural network may explicitly use such a flag or may not explicitly use such a flag. Flag 140 1..k It can be provided by anatomical features or by fiducial markers. As some examples, anatomical features such as bones (e.g., ribs, scapula, etc.) or organ contours (e.g., contours of the lungs, diaphragm, heart shadow, etc.) can be used as anatomical landmarks. Such features are identifiable in 2D medical images by virtue of their X-ray attenuation. Fiducial markers formed of X-ray attenuating materials are also identifiable in medical images, and these fiducial markers can also be used as landmarks. Fiducial markers can be located on the surface or inside the body and at known reference positions. In the latter case, the fiducial markers may have been implanted for use, for example, as surgical guides. Implanted devices such as pacemakers can also be used as fiducial markers. Fiducial markers can also be provided by interventional devices inserted into the body.
[0044] Various techniques may be used to identify 2D medical images 110 1..n The time series of signs 140 1..k and corresponding landmarks in the gallery images. In one example, a feature detector is used to identify a 2D medical image 110 1..nThe identified landmarks are then mapped to corresponding anatomical landmarks labeled in the gallery image. In this example, the gallery image 130 includes a plurality of labeled anatomical landmarks; and reference Figure 1 The described methods also include:
[0045] Applying feature detectors to each medical image in the time series110 1..n , to identify a plurality of landmarks in the medical image corresponding to the labeled anatomical landmarks; and
[0046] Therein, the mapping is determined using the identified corresponding flags.
[0047] In this example, for example, a feature detector is applied to medical images 110 in a time series. 1..n The operation can be performed using an edge detector, or model-based segmentation, or a neural network. As an example, Figure 6 An example of the results of applying a feature detector to a medical image 1101 to identify a plurality of landmarks in the medical image according to some aspects of the present disclosure is illustrated. Figure 6 The 2D medical image 1101 shown in FIG. 1 represents a chest and includes bone regions such as ribs and a spine, as well as a heart shadow and contours of the lungs. In this example, the region of interest is the lungs, and the feature detector has identified contours representing the right lung, the right side of the heart shadow, the right side of the diaphragm, the left lung, the left side of the heart shadow, and the left side of the diaphragm in the 2D medical image 1101 as landmarks 140, respectively. 1..6 These icons correspond to gallery images such as Figure 3 In operation S130, the landmarks 140 identified in the 2D medical image 1101 are 1..6 Then they are mapped to their corresponding landmarks in the library image in order to warp the medical image 1101 to the library image. Figure 6 As shown in , one or more landmarks identified and used in the mapping operation may represent a region of interest, which in this example is the lungs.
[0048] The result of the warping operation S130 is that for each medical image 110 in the time series 1..n Providing a region of interest 120' with warping 1..n Normalized medical image 110' 1..n For comparison with the region of interest 120 in the library image 130. Normalized medical image 110' 1..n Region of interest 120' with warping 1..n For each normalized medical image 110' 1..n The warped region of interest 120' 1..nThe warping operation S130 is compared with the region of interest 120 in the gallery image 130 in the sense that the shape difference between the region of interest 120 in the gallery image 130 has been reduced. Since the warped regions of interest 120'1 and 120'2 are suitable for comparison with the region of interest 120 in the gallery image 130, they can be compared with each other. The warping operation S130 thus reduces the difference between the shapes of the regions of interest 120'1 and 120'2, which in turn allows for more reliable identification of temporal changes in the regions of interest caused by changes in the condition of the subject. In other words, the warping operation S130 provides a normalized medical image 110'1. 1..n The warped region of interest 120' 1..n have a similar shape. Figure 5 The normalized medical image 110′ provided in this manner is illustrated in the image on the right side of FIG. 1..n As can be appreciated, the lung regions of interest 120'1 and 120'2 in these images have more similar shapes in the normalized medical images 110'1 and 110'2, and this allows for more reliable identification of temporal changes in the regions of interest due to progression of the lung condition.
[0049] Return to Figure 1 In operations S140a and S140b, the method performs output S140a of the normalized medical image 110'. 1..n Operation, and / or output S140b normalized medical image 110' 1..n Warping region of interest between 120' 1..n The normalization of the medical image 110′ in operation S140a may be performed in various ways. 1..n Output, including display, printing and storage of normalized medical images 110' 1..n The image may be displayed on a display device such as Figure 5 As an example, the normalized medical image 110′ 1..n Can be Figure 5 The images may be displayed side by side in the manner shown in , or displayed as a time series, or displayed as a superimposed image, or displayed in another manner.
[0050] The output of the change magnitude of the warped region of interest in operation S140b may also be performed in various ways, including displaying, printing, and storing the change magnitude. Typically, the change may represent the normalized medical image 110′ 1..nThe change in intensity or shape of the region of interest between any two of the images in the normalized medical image 110' may be determined, for example, between successive images. In some examples, a statistical analysis may be applied to the warped region of interest in order to determine the magnitude of the change. As some examples, if the region of interest is the lung, the magnitude of the change may quantify the volume of a pneumothorax (i.e., the volume of a "collapsed lung"), or the amount of pleural effusion (i.e., the amount of "fluid surrounding the lung"), or the amount of pulmonary edema (i.e., the amount of "fluid in the lung"), or the stage of pneumonia. The normalized medical image 110' may be determined by 1..n Such changes can be calculated by comparing the image intensity with healthy reference values in the lungs in the gallery images. In one example, the normalized medical image 110′ can be obtained by, for example, 1..n The lungs are contoured, pixels within the lungs are assigned to air or water based on their intensity values, and the normalized medical image 110′ is 1..n The changes are determined by estimating the volumes of the corresponding air and water regions for each normalized medical image.
[0051] As some examples, outputting the magnitude of the change in the warped region of interest in operation S140b may include outputting a numerical value of the magnitude (e.g., as a percentage), or outputting the magnitude of the change graphically. In one example, the magnitude of the change may be graphically represented as a percentage of the normalized medical image 110′ by representing an increase in lung volume between successive images in green and a decrease in lung volume between successive images in red. 1..n The superposition of one or more of them.
[0052] By outputting a normalized medical image in operation S140a and / or outputting the magnitude of the change in the warped region of interest in operation S140b, the method helps the reviewing clinician make a more reliable comparison between regions of interest in the images. This in turn allows the clinician to make a more accurate assessment of the longitudinal changes in the subject's condition.
[0053] You can also refer to the above reference Figure 1 Methods are described to perform various additional operations.
[0054] In some examples, reference Figure 1 The described methods also include:
[0055] Based on the intensity at one or more locations in the gallery image 130, or based on the medical images 110 in the time series 1..n The intensity of one or more positions in the time series is adjusted based on the intensity of one or more positions in the reference image, or based on the intensity of one or more positions in the reference image. 1..n The intensity of the normalized medical image 110 is provided. 1..nThe adjusted intensity value in ; and / or
[0056] Use image style transfer algorithm to adjust each medical image in the time series110 1..n The intensity of the normalized medical image 110 is provided. 1..n The adjusted intensity value in .
[0057] A normalized medical image 110′ may then be calculated based on the adjusted intensity values in the normalized medical image and also output 1..n Warping region of interest between 120' 1..n the magnitude of the change.
[0058] The inventors have observed that not only is the assessment of temporal changes in medical images hampered by differences in the shape of the region of interest between the images, but also that the assessment of such changes may also be hampered by differences in the intensity scale of the images. More specifically, differences in the intensity scale caused by differences in the way in which the 2D medical images in the time series are acquired may hinder their comparison. As an example, the intensity at any point in a 2D X-ray image depends on the value of the X-ray energy "kVp", the exposure time, and the sensitivity of the X-ray detector used to acquire the image. Since the images in the time series are acquired over a period of time, the images in the time series may be generated by X-ray imaging systems with different settings, or indeed by different X-ray imaging systems. Unlike computed tomography images, and where a Hounsfield Unit value may be assigned to each voxel, in 2D X-ray images there is no corresponding absolute attenuation scale for pixel intensity values. By comparing the intensity at one or more locations in the gallery image 130, or based on the intensity at one or more locations in the medical images 110 in the time series, the image may be compared to the image in question. 1..n or adjusting each medical image in the time series based on the intensity at one or more positions in the reference image 110. 1..n The intensity of the normalized medical image 110 is provided 1..n The adjusted intensity values in the image further reduce the image-to-image variability caused by differences in the way the 2D medical images are acquired. Each medical image 110 in the time series is adjusted based on the intensity at one or more locations in the gallery image 130 1..n The intensity operation can provide a normalized medical image 110' 1..n The adjusted intensity values at the corresponding one or more positions in the image gallery 130 are the same as the intensity values at the one or more positions in the image gallery 130. Similarly, based on the medical images 110 in the time series 1..n The intensity of one or more positions in each medical image in the time series is adjusted 110 1..n The intensity operation can provide a normalized medical image 110'1..n The adjusted intensity values at the corresponding one or more positions in the medical image 110 in the time series are adjusted. 1..n Similarly, each medical image 110 in the time series is adjusted based on the intensity at one or more positions in the reference image. 1..n The intensity operation can provide a normalized medical image 110' 1..n The adjusted intensity values at the corresponding one or more positions in the reference image are the same as the intensity values at the one or more positions in the reference image.
[0059] According to this example, the intensity values may be normalized using techniques such as windowing and histogram normalization. The locations in the gallery image 130 may be defined manually or automatically. As an example, if the region of interest is the lung, the location may be defined, for example, within a lung field or within the mediastinum. The location may be predefined in the gallery image or defined based on user input provided via a user input device in conjunction with a displayed image. Also refer to Figure 7 The example is described, Figure 7 An example of warping two 2D medical images in a time series 1101 , 1102 to a gallery image and adjusting the intensity of the medical images according to some aspects of the present disclosure is illustrated. Figure 7 The intensities in the image on the right have been adjusted in the manner described above and facilitate more reliable identification of changes in the region of interest between images.
[0060] Alternatively or additionally, an image style transfer algorithm may be used to provide a normalized medical image 110 ′. 1..n The adjusted intensity values in the image are used to transfer the image style. Various image style transfer transforms for this purpose are known. These can be based on, for example, a Laplacian pyramid or a (convolutional) neural network. An example of an image style transfer algorithm is disclosed in document WO 2013 / 042018 Al. The document discloses a technique for transforming a slave image, which involves generating a color or grayscale transform based on a master image and a slave image. The transform is used to optically adapt the slave image to the master image. The technique can be used to adjust the medical image 110 in a time series 1..n The intensity of the normalized medical image 110' 1..n The adjusted intensity values in the image have a similar appearance or style. 1..n As a slave image and using a gallery image or a medical image in a time series 110 1..n A medical image in or another reference image is used as a main image to apply the technology disclosed in this document.
[0061] In a related example, each medical image 110 in the time series is adjusted based on the intensity at one or more locations in the gallery image 130. 1..n Intensity manipulations may include:
[0062] calculating an average intensity value within a portion of gallery image 130; and
[0063] Use the average intensity value to compare each medical image in the time series 110 1..n The intensity values in are normalized.
[0064] In this example, the average intensity value is used to compare each medical image 110 in the time series. 1..n The normalization operation of the intensity value in the time series can be performed by dividing each medical image 110 1..n The intensity values in the corresponding part of the medical image 110 are mapped to the average intensity value in the part of the library image 130. In this example, the library image 130 is used to map the intensity values in the corresponding part of the medical image 110 to the average intensity value in the part of the library image 130. 1..n The portion in which the intensity values in the 2D medical image 110 are normalized may be selected as a portion of the gallery image, where the expected image intensity is 1..n For example, if the region of interest is the lung, the selected portion of the gallery image may correspond to a portion of the spine, since the attenuation of the spine is not expected to vary over time. Alternatively, if the region of interest is the lung, the selected portion may correspond to a portion of the lung in the gallery image over the duration of the time series of 2D medical images 110. 1..n For example, if the condition of the subject under study is pulmonary edema, the accumulation of fluid over time may be expected to result in temporal changes in the image intensity in the lower portion of the lung, while the upper portion of the lung may be expected to remain free of disease and therefore have a stable image intensity. In this case, the upper portion of the lung in the gallery images 130 may be used as the gallery image 130 for each medical image 110. 1..n The part in which the intensity values are normalized.
[0065] As an example where the region of interest is a lung, these operations may be performed by identifying a portion of a lung in a gallery image. The portion of the lung may be identified automatically or based on user input received via a user input device in conjunction with a display device. The portion of the gallery image may represent a combination of air in the lung and ribs surrounding the lung. The average intensity of the pixel values in that portion of the gallery image may then be calculated. This average may then be used to analyze the medical images 110 in the time series. 1..n The intensity values are normalized. Figure 5This may, for example, include identifying a non-disease corresponding portion of the lung, e.g., toward the top of the lung in the medical image 1101, calculating an average of intensity values in the corresponding non-disease portion, calculating a scaling factor as a ratio of the average intensity value in the non-disease region of the medical image 1101 to the average intensity value in the portion of the lung in the gallery image, and scaling the image intensity values in the medical image 1101 using the scaling factor to provide adjusted intensity values in the normalized medical image 110'1. The normalized medical image 110'1 is then scaled for the other medical images 1101 in the time series. 1..n These operations are repeated. Since the lung portion in the gallery image is used to identify each medical image 110 in the time series. 1..n Normalization is performed so the medical image is scaled to a common reference intensity, which facilitates more reliable comparisons between image intensities in other parts of the image.
[0066] Additional portions of the gallery image may also be used to perform the above-described intensity normalization. For example, a second region of interest may be identified within the mediastinum or within a bone region of the gallery image and also used to perform intensity normalization. Using two regions of interest with different attenuation values to normalize the medical images 110 in the time series 1..n Normalizing the intensities in the image can facilitate more accurate mapping of intensity values and therefore more reliable identification of changes in regions of interest in the image.
[0067] In this example, the normalized medical image 110′ can also be 1..n The value of the quality metric is calculated and output based on the adjusted intensity values in the image. The quality metric can be calculated based on, for example, a portion of a gallery image or a medical image 110' 1..n The value of the quality metric is calculated based on the statistics of the image intensity values in the corresponding parts of the image, or based on the quality of the mapping between the landmarks in operation S130. If the variance of the image intensity values in the part of the gallery image is high, the value of the quality metric may be relatively lower, and vice versa. The value of the output quality metric is re-provided to the reviewing clinician to ensure the accuracy of the method.
[0068] In one example, when adjusting the medical image 110 1..n Before performing the operation of adjusting the intensity of the medical image 110, the medical image is warped S130 to the gallery image 130. In this example, the adjustment of the intensity of the medical image can be performed according to the example described above. Therefore, the intensity of the medical image can be adjusted based on the intensity at one or more locations in the gallery image 130, or based on the intensity of the medical image 110 in the time series. 1..n The adjustment is performed based on the intensity at one or more positions in the reference image, or based on the intensity at one or more positions in the reference image, or using an image style transfer algorithm. Figure 5As described, warping has the effect of making the warped regions of interest have a more similar shape. Thus, by warping each medical image to the gallery image before adjusting its intensity, a more reliable basis for adjusting the intensity values is provided. Thus, it improves the normalized medical image 110' 1..n In contrast, if the medical image is normalized prior to warping, the adjusted intensity values may not take into account adjustments to pixel intensity values that may be provided by warping. For example, if normalization is to be provided based on lung regions in a gallery image, normalization should be performed in the warped image because the warped lungs have a comparable shape and thus the image intensity values within the lungs more accurately represent the lungs with the warped shape.
[0069] The inventors have also observed that after the warping operation S130, the normalized medical image 110′ 1..n Some areas outside the region of interest may be warped to unnatural shapes. This may be confusing to the reviewing physician. Figure 1 The described method may also include:
[0070] Suppressed Normalization of Medical Images 110' 1..n Region of interest 110' 1..n One or more image features other than
[0071] By suppressing the region of interest 120' 1..n One or more image features other than the MRI scan are present, thus avoiding the presentation of confounding information to the reviewing clinician. Figure 8 Describe the example, Figure 8 An example of warping two 2D medical images in a time series 1101, 1102 to a gallery image, adjusting the intensity of the medical images, and suppressing image features outside the lung region of interest according to some aspects of the present disclosure is illustrated. Figure 8 In the image on the right, the normalized medical image 110' 1..n The portion of outside the lung region of interest has been suppressed. Figure 8 In the example of , the image has suppressed features outside the lung region of interest by mapping their pixel values to black level pixel intensity values. However, these features may alternatively be suppressed in a different manner. For example, their pixel values may be mapped to pixel intensity values different from the black level pixel intensity value, or their pixel values may be gradually adjusted to a predetermined value in order to "feather-out" the region of interest. This operation of suppressing image features outside the region of interest may be performed using a gallery image. For example, in this example, reference Figure 1 The described method may also include:
[0072] identifying a region of interest 120 in a gallery image 130;
[0073] Mapping the contour of the region of interest 120 from the gallery image 130 to the normalized medical image 110 ′ 1..n for each normalized medical image in ; and
[0074] Adjusting the normalized medical image outside the mapping contour 110' 1..n The image intensity value in order to suppress the region of interest 120' 1..n One or more image features other than
[0075] In this example, various example regions of interest may be predefined in the gallery image and used to suppress the region of interest 120 ′. 1..n Alternatively, the region of interest may be defined by segmenting the gallery image. Since in this example the contours from the gallery image are used to suppress the region of interest 120' 1..n The contour is reliable because the normalized medical image 110′ has been transformed into the image using the landmarks in the gallery image in operation S130. 1..n The image is warped to the gallery image, and thus the shape of the region of interest in the normalized image is similar to the shape of the region of interest in the gallery image.
[0076] As another example, Fig. 9 2D medical image 110 according to some aspects of the present disclosure is illustrated. 1..n Time series (above), 2D medical images 110 1..n The time series have been warped to the gallery image and their intensities have been adjusted to provide adjusted intensity values corresponding to the normalized image 110' 1..n Examples of a time series of (middle) and a time series of corresponding normalized images where image features outside the lung region of interest have been suppressed (bottom). Fig. 9 The diagram shows the Figure 5 , Figure 7 and Figure 8 The foregoing operation of the two images illustrated in the figure further illustrates the operation of the two images, and is based on a longitudinal study of the subject in the 2D medical image 110 1..n The amount of variation observed in and the normalized medical image 110' provided by the method described herein 1..n Further illustration of the advantages of.
[0077] In another example, reference Figure 1 The described method may also include:
[0078] Performing normalized medical image 110' 1..nRegion of interest 110' 1..n Statistical analysis of image intensity values within; and
[0079] Among them, output S140b normalized medical image 110' 1..n Warping region of interest between 120' 1..n The magnitude of the changes include:
[0080] Output for normalized medical image 110' 1..n The results of statistical analysis.
[0081] Various statistical analysis methods can be applied to the normalized medical image 110' 1..n Region of interest 120' 1..n An example implementation of this statistical analysis is based on the normalized medical image 110′ 1..n In this example, the normalized medical image 110′ may be analyzed based on the local image block. 1..n Statistics are generated for each image block in the normalized medical image 110 ′. This may include determining the mean and standard deviation of the pixel values within the image block, assuming, for example, a Gaussian distribution. When comparing, for example, the normalized medical image 110 ′ 1..n When the corresponding image patches of two images A and B are viewed in a computer program, the Gaussian statistics of the image patch of image A can be used to estimate the likelihood of the measured intensity value in the image patch of image B. The likelihood can be thresholded to provide a prediction of the change within the image patch. The likelihood can also be visualized as an overlay of images A and B to allow the reviewing clinician to analyze the detected changes in the images over time. Thus, this visualization can allow the reviewing clinician to assess whether the detected changes are statistically relevant, or, for example, whether they simply result from an imperfect normalization process.
[0082] In a related example, the output is for the normalized medical image 110 ′ 1..n The operation of the statistical analysis result may include displaying the spatial map of the statistical analysis as a normalized medical image 110′ 1..n Providing the results of the statistical analysis in this manner can assist clinicians in providing an effective analysis of the progression of a medical condition in a subject.
[0083] In another example, a computer program product is provided. The computer program product includes instructions that, when executed by one or more processors, cause the one or more processors to perform a method of providing a normalized medical image 110' representing a region of interest 120 in a subject. 1..n The method comprises:
[0084] Receiving S110 includes 2D medical image 110 1..n Time series image data, 2D medical images 110 1..n including a region of interest 120 in the object;
[0085] receiving S120 a gallery image 130 representing the region of interest 120; and
[0086] For each medical image in the time series:
[0087] Warping S130 the medical image to the gallery image 130 to provide a warped region of interest 120' 1..n Normalized medical image 110' 1..n , for comparison with the region of interest 120 in the gallery image 130; and
[0088] The method further comprises outputting S140a the normalized medical image 110′ 1..n , and / or output S140b normalized medical image 110' 1..n Warping region of interest between 120' 1..n the magnitude of the change.
[0089] In another example, a method for providing a normalized medical image 110 ′ representing a region of interest 120 in a subject is provided. 1..n The system 200 includes one or more processors 210, and the one or more processors are configured to:
[0090] Receiving S110 includes 2D medical image 110 1..n Time series image data, 2D medical images 110 1..n including a region of interest 120 in the object;
[0091] receiving S120 a gallery image 130 representing the region of interest 120; and
[0092] For each medical image in the time series:
[0093] Warping S130 the medical image to a gallery image 130 to provide a warped region of interest 120' 1..n Normalized medical image 110' 1..n , for comparison with the region of interest 120 in the gallery image 130; and
[0094] The one or more processors are further configured to output S140a a normalized medical image 110′. 1..n , and / or output S140b normalized medical image 110'1..n Warping region of interest between 120' 1..n the magnitude of the change.
[0095] exist Figure 2 An example of the system 200 is illustrated in FIG. It should be noted that the system 200 may also include one or more of the following: a 2D medical imaging system for generating the image data received in operation S110, such as, for example Figure 2 The projection X-ray imaging system 220 shown in FIG. 2 is used to display the normalized medical image 110 ′ outputted from the display. 1..n And / or output a normalized medical image 110' 1..n Warping region of interest between 120' 1..n a display device such as a monitor 250, a tablet computer, etc. for displaying the magnitude of changes in the image intensity values, the results of statistical analysis of the image intensity values, etc.; a bed 260; and a user input device ( Figure 2 (not shown in the figure), such as keyboard, mouse, touch screen, etc.
[0096] The above examples should be understood as explanations of the present disclosure rather than limitations. Additional examples are also contemplated. For example, examples of computer-implemented descriptions may also be provided in a corresponding manner by a computer program product, or by a computer-readable storage medium, or by a system 200. It should be understood that the features described with respect to any one example may be used alone, or in combination with other described features, and may be used in combination with one or more features of another example or a combination of other examples. In addition, equivalents and modifications not described above may also be adopted without departing from the scope of the present invention as defined in the appended claims. In the claims, the word "include" does not exclude other elements or operations, and the word "one" or "an" does not exclude multiple. Although specific features are recorded in mutually different dependent claims, this does not indicate that the combination of these features cannot be used advantageously. Any figure mark in the claims should not be interpreted as limiting its scope.
Claims
1. A method for providing a normalized medical image (110') representing a region of interest (120) in a subject 1..n ), the method comprising: Receive (S110) a 2D medical image (110 1..n ), the 2D medical image comprising a region of interest (120) in the object; receiving (S120) a 2D gallery image (130) representing the region of interest (120); and For each medical image (110 1..n ): The medical image is warped (S130) to the gallery image (130) to provide a warped region of interest (120') 1..n ) of the normalized medical image (110' 1..n ) for comparison with the region of interest (120) in the gallery image (130); and The method further comprises outputting (S140a) the normalized medical image (110' 1..n ), and / or output (S140b) the normalized medical image (110' 1..n ) between the warped region of interest (120' 1..n ) in the range of change.
2. The computer-implemented method of claim 1 , wherein: The 2D gallery image representation is used to evaluate the 2D medical image (110 1..n )'s preferred viewing angle of the region of interest for the medical condition in a time series.
3. A computer-implemented method according to claim 1 or claim 2, wherein: The warping (S130) is based on a plurality of corresponding landmarks (140) represented in both the medical image and the gallery image (130). 1..k ) between them.
4. The computer-implemented method according to any one of claims 1 to 3, wherein: The gallery image (130) includes a plurality of labeled anatomical landmarks; and Wherein, the method further comprises: A feature detector is applied to each medical image in the time series (110 1..n ) to identify a plurality of landmarks in the medical image corresponding to the labeled anatomical landmarks; and Therein, the mapping is determined using the identified corresponding markers.
5. The computer-implemented method of claim 4, wherein: Application feature detectors are performed using edge detectors, or model-based segmentation, or neural networks.
6. The computer-implemented method of claim 1, wherein: The method further comprises: Based on the intensity at one or more locations in the gallery image (130), or based on the medical image (110) in the time series 1..n ), or based on the intensity at one or more positions in the reference image, to adjust each medical image in the time series (110 1..n ) to provide the normalized medical image (110 1..n ) in the adjusted intensity value; and / or Use an image style transfer algorithm to adjust each medical image in the time series (110 1..n ) to provide the normalized medical image (110 1..n ) in the adjusted intensity value.
7. The computer-implemented method of claim 6, wherein: Warping (S130) the medical image to the gallery image (130) is to adjust the medical image (110 1..n ) was performed before the strength of the 8. A computer-implemented method according to claim 6 or claim 7, wherein: The method comprises: Calculating the normalized medical image based on the adjusted intensity values in the normalized medical image (110') 1..n ) between the warped region of interest (120' 1..n ); and Wherein, the output (S140b) the normalized medical image (110' 1..n ) between the warped region of interest (120' 1..n ) includes outputting a calculated magnitude of the change.
9. The computer-implemented method of any one of claims 6 to 8, wherein: Adjusting each medical image 110 in the time series based on the intensity at one or more locations in the gallery image 130 1..n The intensity provides the normalized medical image 110' 1..n , the adjusted intensity values being the same as the intensity values at the one or more locations in the gallery image 130; and Wherein, based on the medical image 110 in the time series 1..n The intensity of one or more positions in each medical image in the time series is adjusted 110 1..n The intensity provides the normalized medical image 110' 1..n The adjusted intensity values at the corresponding one or more positions in the time series are the same as the adjusted intensity values of the medical image 110 in the time series. 1..n The intensity values at the one or more positions in are the same; and Each medical image in the time series is adjusted based on the intensity at one or more locations in the reference image 110 1..n The intensity provides the normalized medical image 110' 1..n , wherein the adjusted intensity values are the same as the intensity values at the one or more positions in the reference image.
10. The computer-implemented method of claim 6, wherein: Adjusting each medical image (110) in the time series based on the intensity at one or more locations in the gallery image (130) 1..n ) include: calculating an average intensity value within the portion of the gallery image (130); and Using the average intensity value to analyze each medical image in the time series (110 1..n ) are normalized.
11. The computer-implemented method of claim 6, wherein: Each medical image in the time series is analyzed using the average intensity value (110 1..n ) in the time series comprises normalizing each medical image (110 1..n ) is mapped to the average intensity value within the portion of the gallery image (130).
12. The computer-implemented method of claim 1, wherein: The method further comprises: Suppressing the normalized medical image (110' 1..n ) in the region of interest (110') 1..n ) other than one or more image features.
13. The computer-implemented method of claim 12, wherein: The method further comprises: identifying the region of interest (120) in the gallery image (130); Mapping the contour of the region of interest (120) from the gallery image (130) to the normalized medical image (110') 1..n ) for each normalized medical image in; and Adjusting the normalized medical image outside the mapped contour (110' 1..n ) in order to suppress the image intensity value in the region of interest (120' 1..n ) other than the one or more image features.
14. A computer-implemented method according to any preceding claim, wherein: The method further comprises: Performing the normalized medical image (110' 1..n ) in the region of interest (110') 1..n ) within a statistical analysis of the image intensity values; and Wherein, output (S140b) the normalized medical image (110' 1..n ) between the warped region of interest (120' 1..n ) include: Output for the normalized medical image (110' 1..n ) of the statistical analysis.
15. The computer-implemented method of claim 14, wherein: Output for the normalized medical image (110' 1..n ) includes: The spatial graph of the statistical analysis is displayed as a normalized medical image (110' 1..n ) on one or more of them.
16. A computer-implemented method according to any preceding claim, wherein: The 2D medical image (110 1..n ) time series includes X-ray images or ultrasound images.
17. A computer-implemented method according to any preceding claim, wherein: The 2D medical image (110 1..n ) time series includes X-ray images generated by a mobile X-ray imaging system.
18. A computer-implemented method according to any preceding claim, wherein: The 2D medical image (110 1..n ) time series represents a longitudinal study of the subjects.
19. A computer program product comprising instructions which, when executed by one or more processors, cause the one or more processors to perform a method of providing a normalized medical image (110′) representing a region of interest (120) in a subject. 1..n ), the method comprising: Receive (S110) a 2D medical image (110 1..n ) of time series image data, the 2D medical image (110 1..n ) includes a region of interest (120) in the object; receiving (S120) a 2D gallery image (130) representing the region of interest (120); and For each medical image in the time series: The medical image is warped (S130) to the gallery image (130) to provide a warped region of interest (120') 1..n ) of the normalized medical image (110' 1..n ) for comparison with the region of interest (120) in the gallery image (130); and The method further comprises outputting (S140a) the normalized medical image (110' 1..n ), and / or output (S140b) the normalized medical image (110' 1..n ) between the warped region of interest (120' 1..n ) in the range of change.
20. A method for providing a normalized medical image (110') representing a region of interest (120) in a subject 1..n ) system (200), the system comprising one or more processors (210), the one or more processors being configured to: Receive (S110) a 2D medical image (110 1..n ) of time series image data, the 2D medical image (110 1..n ) includes a region of interest (120) in the object; receiving (S120) a 2D gallery image (130) representing the region of interest (120); and For each medical image in the time series: The medical image is warped (S130) to the gallery image (130) to provide a warped region of interest (120') 1..n ) of the normalized medical image (110' 1..n ) for comparison with the region of interest (120) in the gallery image (130); and in, The one or more processors are further configured to output (S140a) the normalized medical image (110' 1..n ), and / or output (S140b) the normalized medical image (110' 1..n ) between the warped region of interest (120' 1..n ) in the range of change.
