Automated system for rapid detection and indexing of critical areas in non-contrast head CT scans.

Abstract

This invention relates to a processing system and corresponding method for processing image data and indexing regions of interest (ROIs) within objects of interest. The system includes: a registration unit for registering the image data; a sub-volume generator for generating sub-volumes based on the image data; and a composite image generator for generating multiple sets of composite images based on each sub-volume, each set of composite images representing a different projection. The system further includes: an indexing unit configured to generate an indexed ROI in each sub-volume by evaluating each composite image in the set of composite images in parallel; and a combining unit for combining each set of composite images for each sub-volume into a scan-level priority sort.

Description

Technical Field

[0001] The present invention generally relates to a processing system for processing image data of an object of interest and indexing regions of interest in the object of interest, and a method for processing image data of an object of interest and indexing regions of interest in the object of interest. Background Technology

[0002] Non-contrast head CT (NCHCT) imaging is a non-invasive and frequent radiographic scan of a patient's head (the object of interest) used for screening in the emergency department to examine neurological, pathological, and traumatic conditions in both emergency and non-emergency situations. They are widely available and have relatively short acquisition times, making them the imaging modality of choice for patients accessing any emergency services and potentially the sole investigation guiding management in patients with symptoms of head trauma or stroke.

[0003] The diagnosis obtained from CT images determines the subsequent treatment path. In emergency situations, such as trauma, the timing of treatment is crucial, and physicians must determine the next course of action as early as possible.

[0004] With the increased accessibility of CT scans (imaging), there are growing problems in areas where trained radiologists may not be available for interpretation. Furthermore, during "off-hours" periods, senior emergency room staff are only correct in interpretations two-thirds of the time, thus still relying on radiologists for the final decision.

[0005] Given the volume of patients requiring CT scans entering the emergency department, prompt initial radiographic reporting to trigger an appropriate level of care is crucial in the emergency department. There is an urgent need for a preliminary triage (or indexing) system to detect life-threatening pathologies for prioritization and to pre-identify affected areas (regions of interest), thereby accelerating the diagnostic and treatment process and improving patient care.

[0006] Currently, deep convolutional neural networks (DCNNs) have become the dominant machine learning method for medical image classification tasks. The data-driven nature of DCNNs benefits from the increasing amount of publicly accessible medical imaging datasets. Existing literature utilizes existing DCNN techniques, such as the Dense Convolutional Network (DenseNet) model and the Residual Network (ResNet) model. Both of these models have been transferred and learned for medical image classification.

[0007] Current solutions for automated interpretation of non-contrast head CT scans focus on three-dimensional (3D) full-scan analysis. The 3D whole-scan model considers scene information while making its predictions. However, when evaluating large scans, small hemorrhagic areas may be easily missed using sequential pooling in techniques such as DCNNs.

[0008] Some anomalies also depend on inter-slice contextual information (such as mass effects indicated by vertical thrust of the ventricles), and models built at the slice level will not be able to detect these unless the model is also provided with contextual information. While slice-level analysis is good at identifying small anomalies (e.g., tiny hemorrhage areas), it can be computationally and time-intensive. For experts, labeling each slice required to train these models is also an extremely time-consuming and expensive process.

[0009] There is a need for an improved index for regions of interest within objects of interest that is faster, less complex, takes into account context information, and addresses the issue of missing small-sized anomalies. Summary of the Invention

[0010] The purpose of this invention is to provide a system and method for processing image data of objects of interest, which solves the above-mentioned problems or at least provides alternative solutions.

[0011] In particular, the purpose of this invention is to provide a system and method that takes into account scene information in image data and small-sized anomalies in objects of interest to provide faster indexing or anomaly priority sorting.

[0012] In a first aspect of the invention, a processing system is provided for processing image data of an object of interest and for indexing regions of interest within the object of interest.

[0013] Image data is typically generated by imaging systems, such as CT imaging systems, particularly non-contrast head CT imaging systems. In the case of non-contrast head CT imaging systems, the object of interest is the patient's head.

[0014] The processing system includes a registration unit configured to register the image data from the imaging system.

[0015] Image data is generated by a CT imaging system and received and registered by a processing system. The image data can be 3D volumetric image data representing an object of interest (e.g., a patient's head). In an embodiment, the image data is received by a segmentation or extraction unit that extracts a region of interest, such as brain and skull regions.

[0016] The processing system further includes a sub-volume generator configured to generate sub-volumes based on the image data, each sub-volume having a slice thickness. Each slice thickness comprises several slices of the original scan. Preferably, the sub-volumes are created based on an axial view of the original image data or scan data. Alternatively, the sub-volumes are created based on a coronal view of the original image data or scan data. In the most preferred embodiment, the sub-volumes are created based on both an axial view and a coronal view of the original image data or scan data.

[0017] The processing system is configured to generate sub-volumes from received and registered image data, typically 3D volumetric image data. By creating sub-volumes, the 3D volumetric image data is divided into multiple sub-volumes, each with a flask thickness. Preferably, the registered image data or 3D volumetric image data is divided such that the sub-volumes have optimal flask thicknesses, thereby ensuring coverage of small anomalies.

[0018] The processing system also includes a composite image generator configured to generate multiple sets of composite images based on each sub-volume, each set of composite images representing a different projection. Preferably, each composite image is a 2D composite image. In other words, multiple 2D composite images of sub-volumes are created based on the original scan.

[0019] By generating multiple sets of composite images based on each sub-volume, the risk of losing critical information in the image data is reduced, making it more robust. It allows for better visibility of various anomalies. Therefore, the composite image generator creates multiple composite images for a single sub-volume. Preferably, there are three different types of composite images: maximum intensity projection (MIP), minimum intensity projection (MinIP), and projection based on a threshold selected from the grayscale histogram (OtsuIP).

[0020] The projection method based on threshold selection from gray-level histograms is a projection method published by Nobuyuki Otsu under the title "AThresholdSelection Method from Gray-Level Histograms" (IEEE Transactions on Systems, Man, and Cybernetics, vol. smc-9. No. 1, January 1979), which is incorporated herein by reference.

[0021] Collections of MIP-based composite images have the advantage of preserving high-density areas (such as hemorrhage).

[0022] The collection of composite images based on MinIP has the advantage of preserving low-density regions, ventricles, and subdural abnormalities.

[0023] The use of OtsuIP-based composite images further reduces the risk of suppressing relevant information because it takes into account the maximum variance point across composite slices.

[0024] The processing system includes an indexing unit configured to generate an indexed region of interest in each subvolume by evaluating each composite image in the set of composite images in parallel.

[0025] Preferably, sub-volumes are created based on axial and coronal views. This yields axial view sub-volumes and coronal view sub-volumes. This allows for the evaluation of coronal slices to detect vertical pull / push of brain structures. Existing systems used to detect abnormalities in the head using CT focus solely on the evaluation of axial scans or axial views. This is done slice-by-slice and lacks context across several surrounding slices. Considering coronal view sub-volumes allows for the detection of more complex clinical conditions, such as mass effects. For this purpose, evaluating coronal views is beneficial because such conditions may manifest as vertical movement of structures. Evaluating coronal views also overcomes the limitations of slice-by-slice evaluation by considering context information in the vertical direction, which is important in detecting vertical push / push of structures that might otherwise appear normal when only axial views are considered.

[0026] The indexing unit is preferably a classifier configured to classify anomalies in the object of interest and generate anomaly probability scores for each indexed region, wherein the classifier is preferably a deep convolutional neural network (DCNN).

[0027] The processing system further includes a combination unit configured to combine each set of composite images of each sub-volume into a scan-level priority sort.

[0028] Preferably, the combining unit is configured to cascade each deep convolutional neural network with a recurrent neural network for each set of composite images of each sub-volume (all sub-volumes) to provide the scan level priority sorting.

[0029] The advantage of cascading multiple DCNNs with recurrent neural networks (RNNs) is that it allows scene information across subvolumes to be considered.

[0030] As previously mentioned, evaluating scene information across slices is important for detecting complex clinical conditions such as mass effects. Evaluating scene information to obtain clear boundaries (e.g., the boundaries of calcifications across vertical slices) is also crucial in cases of questionable conditions. Automation of this process can be improved by cascading subvolume DCNN modules with recursive-based networks.

[0031] Priority sorting and pre-indexing can allow for more efficient diagnostics.

[0032] Critical findings detected in subvolumes can be easily combined using RNNs. This creates patient-level scores or scan-level priority rankings, which can be used for alerting and prioritizing for immediate action.

[0033] To further improve the time to diagnosis and treatment, the present invention also provides pre-indexing of sub-volumes of a scan, wherein critical pathologies are detected for immediate access when a scan is proposed for interpretation.

[0034] In other words, the present invention involves creating sub-volumes (of sufficient thickness) based on axial and coronal views of the original scan (image data from the imaging system). Multiple composite images are generated for each sub-volume using maximum intensity projection (MIP), minimum intensity projection (MinIP), and projection generated from the Otsu threshold point (OtsuIP). The three composite images of each sub-volume are then evaluated in parallel to detect critical pathology in the patient and are then combined to obtain scan-level priority ranking. Upon detection of a critical finding, automatic selection of the affected sub-volume is performed, allowing experts to immediately view the critical area, thereby improving the time to diagnosis and treatment.

[0035] The present invention also relates to a method for processing image data of an object of interest and for indexing regions of interest within the object of interest. The method includes the following steps:

[0036] The image data from the imaging system is registered using a registration unit.

[0037] A sub-volume generator is used to generate sub-volumes based on the image data, each sub-volume having a slice thickness;

[0038] Multiple sets of composite images are generated from each sub-volume using a composite image generator, each set of composite images representing a different projection;

[0039] The region of interest in each sub-volume is indexed by means of an indexing unit, which evaluates each composite image in the set of composite images in parallel; and

[0040] The indexed regions of interest of each set of composite images are combined into a scan-level priority order by means of a combination unit.

[0041] In an embodiment, the method further includes a brain / bone region extraction step, as explained below. Preferably, this step is performed between image data registration and subvolume generation.

[0042] Preferably, the step of generating a sub-volume includes the steps of creating a sub-volume based on an axial view and creating a sub-volume based on a coronal view.

[0043] Preferably, the step of generating multiple sets of composite images further includes the following steps: generating a set of composite images generated by maximum intensity projection, generating a set of composite images generated by minimum intensity projection, and generating a set of composite images generated by grayscale thresholding.

[0044] In an embodiment of the method according to the invention, the indexing step further includes: classifying anomalies in the object of interest, and generating anomaly probability scores for each indexed region by means of a classifier, preferably a deep convolutional neural network (DCNN).

[0045] In another embodiment of the method according to the invention, the combining step further includes: cascading each depth convolutional neural network of each set of composite images of each subvolume with a recurrent neural network to provide the scan level priority ordering.

[0046] In an embodiment of the method according to the invention, the combined step further includes pre-indexing the sub-volumes according to the scan level priority sorting.

[0047] In other words, the method includes one or more of the following steps:

[0048] Divide each view (coronal and axial) into sub-volumes of the desired thickness;

[0049] Create composite images (MIP, MinIP, OtsuIP) based on each sub-volume;

[0050] Evaluate each set of composite images for a single subvolume in parallel to detect critical findings;

[0051] Subvolume prediction is used for pre-indexing and on-demand access during viewing;

[0052] View level predictions are created using scene information across subvolumes via RNN;

[0053] Combine view level scores to create patient-level predictions;

[0054] Use patient-level predictions to update the work list and for prioritization.

[0055] In other words, time and computational complexity can be reduced because evaluation per scan subvolume (by creating a composite image from each scan subvolume) is more efficient in terms of time and computational complexity than evaluation per scan.

[0056] In critical situations, such as trauma, the timing of intervention is crucial, and physicians must determine subsequent treatment as early as possible. An accurate initial evaluation of the pathology is necessary because this determines the case handler and treatment pathway (i.e., surgical or medical intervention), and any error causing a delay in NCHCT is sufficient to warrant intervention. Attached Figure Description

[0057] The invention will now be described by way of example with reference to the accompanying drawings, in which:

[0058] Figure 1 The processing system, imaging system, and imaging viewing system according to the present invention are shown;

[0059] Figure 2 A schematic flowchart of the method according to the present invention is shown;

[0060] Figure 3 A schematic flowchart of a preferred method according to the present invention is shown;

[0061] Figure 4a , 4b Figures 4c and 4c show an example of three successive slices of an axial scan / view, and Figure 4d The resulting composite image of the successive slices is shown, and Figure 4e A full-scan composite image is shown;

[0062] Figure 5a and 5b Composite images generated by maximum and minimum intensity projections of axial scans / views containing hemorrhage and hydrocephalus are shown respectively;

[0063] Figure 6a and 6b Composite images generated by the maximum and minimum intensity projections of an axial scan / view containing hemorrhages in a low-density region are shown respectively.

[0064] Figure 7a and 7b Composite images generated by the maximum and minimum intensity projections of the axial scan / view containing subdural hemorrhage are shown respectively.

[0065] Figure 8a , 8b Figures 8c show composite images generated by maximum intensity projection, minimum intensity projection, and Otsu intensity projection, respectively, through an axial scan / view containing hemorrhage with midline shift;

[0066] Figure 9a and 9b Axial scan / view and coronal scan / view of the object of interest containing hemorrhage with mass effect are shown respectively;

[0067] Figure 10 A schematic flowchart illustrating an embodiment of the method according to the present invention is shown;

[0068] Figure 11 A sample user interface of the image viewing system according to the present invention is shown;

[0069] Figure 12 This shows brain ROI extraction and slices of the most prominent brain regions. maxbr An overview of the identification of );

[0070] Figure 13 This shows that when slice i = slice maxbr Fine-grained extraction of brain regions; and

[0071] Figure 14 It shows when slice i <slice maxbr Fine extraction of brain regions during the brain. Detailed Implementation

[0072] The processing system is an automated system used to quickly detect and index critical areas to improve the time required for diagnosis and treatment. Figure 1 This illustrates how the present invention can be adapted to a hospital workflow. Digital images S1, S2, ..., Sn or image data A are captured or scanned from various scanners or image acquisition units 110. The quality of image data A is first checked and standardized by a quality and standardization unit 120.

[0073] The verified digital image B is then stored in image storage server 131 and also transferred to processing system 140 for evaluation and allocation. Processing unit 140 includes scanning and viewing unit 141, which includes a sub-volume generator, a composite image generator, an indexing unit, and a combination unit.

[0074] The scanning and viewing unit 141 may also include a registration unit 130. Alternatively, the registration unit is an image storage server 131.

[0075] Once the verification image data B is received at the scanning viewing unit 141, it is processed, and the severity score C and the indexed slice of interest D are found from each sub-volume and stored back in the viewing image storage server 132, so that the viewing image storage server 132 includes an archive of the viewed images.

[0076] In parallel, the severity score C of the viewed image is passed to the workload allocation unit 142, where the work list E can be updated by referencing the severity score C and stored back to the work list storage server 133. Finally, at the viewing and reporting unit 150, the work list E is retrieved from the configuration storage server 134 for each individual.

[0077] When a user at the intelligent viewing station 151 queries an image in the work list C, the system also retrieves a pre-indexed slice G of interest identified by the processing system 140 or the scanning viewing unit 141 and displays the slice on the display 152. The user can then create a report H and diagnose the situation, which is ultimately returned to the report storage server 135.

[0078] The scanning and viewing unit 141 or the image viewing unit includes a sub-volume generator, a composite image generator, an indexing unit, and a combination unit.

[0079] The sub-volume generator generates sub-volumes based on the verification image data B, where each sub-volume has a slice thickness.

[0080] The composite image generates multiple sets of composite images from each sub-volume, where each set of composite images represents a different projection.

[0081] The indexing unit generates the indexed region of interest in each subvolume by evaluating each composite image in the set of composite images in parallel.

[0082] The combining unit combines the indexed regions of interest from each set of the composite image into a scan-level priority sort.

[0083] Figure 2 A schematic flowchart of the image / scan viewing unit 141 is shown.

[0084] In the first step 205, the input scanned or verified image data B is retrieved from the verification image storage server 131.

[0085] In the separation and enhancement step 210, after applying brain and bone windowing to enhance contrast, the scan is separated / segmented / extracted into brain and skull regions.

[0086] This flowchart only illustrates the analysis of brain regions; a similar approach will be applied to the skull subsequently.

[0087] In this example, an unsupervised method is used to extract intracranial regions from the head. This involves two main steps: first, identifying the Region of Interest (ROI) and slices containing the most prominent brain regions; and second, fine-tuning the extraction of brain regions.

[0088] The following sub-steps are used to identify the ROI and the slices containing the most brain regions:

[0089] Apply brain windowing to the original scan;

[0090] The brain was divided into three regions using the intensity value of the S-wavelength brain scan: bone, air, and soft tissue.

[0091] Identify the largest bone island for each slice;

[0092] Fit a convex hull on the largest bone island (representing the region of interest of the slice);

[0093] Use a shell for each slice identification to create an ROI mask Rmask and a brain scan mask R;

[0094] - By targeting R for each slice i Find the largest soft tissue island C in R. Ri To identify in parallel slices containing the largest brain region, the index of the slice containing the largest area of ​​that soft tissue contour is given by the following formula.

[0095]

[0096] Based on ROI scans and the largest brain volume slice (slice) maxbr Brain region B was extracted using the following sub-steps:

[0097] For slice R i Where i = slice maxbr Brain regions are identified by the largest soft tissue contour in the slices (Bmask). i );

[0098] For each slice R i Where i>slice maxbr First, slice the material using the mask Bmask of the previous slice. i-1 Multiply to obtain the filtered slice R Fi ;

[0099] Use the largest of the three largest contours identified above to create a mask of the brain region;

[0100] Contours smaller than a threshold T are removed, an aperture filter is applied for further noise removal, and simple region growing is applied to create a slice-level brain region mask (Bmask). i ;

[0101] For each slice R i , where i <slice maxbr First, the mask Bmask of the slice and the next slice are used. i+1Multiply to obtain the filtered slice R Fi Follow steps 3-4;

[0102] Finally, scan the ROI with R and Bmask. i Multiply to obtain brain region extraction scan B.

[0103] exist Figure 12 , 13 In sections 1 and 14, the overview discloses brain ROI extraction and the identification of the largest brain slice, when slice i = slice maxbr Fine extraction of brain regions during the process, and when slice i <slice maxbr The steps involved in the fine extraction of brain regions.

[0104] Based on brain region scans, coronal and axial views are created from the original images by appropriately reordering pixel values. In other words, 215 axial and coronal views are created.

[0105] In the next step 220, each of the generated views is then divided into n sub-volumes (e.g., 16) to give a total of 2n sub-volumes.

[0106] In the next step 225, for each sub-volume i, three composite images using MIP, MinIP, and OtsuIP are generated.

[0107] Once three composite images for a given subvolume i are obtained, they are fed into the RGB channels of the DCNN for parallel processing in parallel processing step 230 for classification.

[0108] Preferably, DenseNet-121 is used as a classifier for detecting anomalies at the sub-volume level. Such a DenseNet-121 is disclosed in the publication entitled “DenseNet: Implementing efficient convnet descriptor pyramids” (arXiv preprint arXiv: 1404.1869 (2014)) by Iandola, F., Miskewicz, M., Karayev, S., Girshick, R., Darrell, T., and KEutzer K, which is incorporated herein by reference.

[0109] In the next step 240, each composite image set is passed through the DCNN to obtain probability scores.

[0110] The probability fraction is defined by the following equation:

[0111]

[0112] Where i is the input composite image passed as a single RGB image with a given subvolume.

[0113] Let represent the probability score of sub-volume image i, where view v belongs to critical category a.

[0114] In step 245, the probability score is checked to see if it exceeds a threshold T in order to determine whether the subvolume contains critical pathology and thus index it.

[0115] A threshold T can be configured. If the score is greater than T, the index, view type (i.e., coronal or axial), and probability score of the first and last slices of the sub-volume relative to the full scan, along with the scan ID, are recorded and stored back in the image viewing archive. In other words, it includes step 250 of obtaining the indexes of the first and last slices in the sub-volume relative to the full scan and step 255 of saving the index, view type, and probability score to the image viewing archive.

[0116] In order to incorporate sub-volume fractions into scan-level predictions and also take into account scene information across sub-volumes, the method includes step 238 of cascading a DCNN with a recurrent neural network (RNN).

[0117] Therefore, in step 237, the output of the last global average pooling layer of the DCNN for each sub-volume I of a given view v is extracted to obtain a set of features f.

[0118] Then, also Figure 2 In step 238 shown, the feature set f of each of the n sub-volumes of a given view v is fed into the RNN.

[0119] In this embodiment, the RNN consists of a single gated recurrent unit (GRU) layer with input / output sizes of 1024 and 2048, respectively. Following the GRU is a fully connected layer with an output size of 1.

[0120] In step 260, the output of the last time step of the RNN is extracted and normalized using a sigmoid function in order to calculate the anomaly probability score at the scan level. Similar to Formula 2. The scan level anomaly probability score is the scan level priority ranking, view level priority ranking, or view urgency score.

[0121] Once all v views have been processed and scores have been obtained, the maximum score for each view is taken (i.e., To obtain the overall patient-level severity score, as shown in step 262.

[0122] In storage step 265, the score is then stored back in the image viewing archive and will also be sent to the workload allocation unit for priority sorting.

[0123] Figure 3 An overview of the method according to the invention is shown. In the registration step 300, image data or input view v or input scan is acquired from the imaging system and registered by the registration unit.

[0124] In a preferred embodiment, the next windowing and extraction step 301 is performed.

[0125] In the next step 305, sub-volumes are generated from the image data using a sub-volume generator, wherein each sub-volume has a slice thickness. Preferably, the image data is divided into n sub-volumes (i1, i2, ..., i...). n ).

[0126] In subsequent steps 310a, 310b, and 310n, a plurality of sets of composite images are generated from each sub-volume n using a composite image generator. Each set of composite images represents a different projection.

[0127] In step 310a, the set of composite images is generated for the first sub-volume. In step 310b, the set of composite images is generated for the second sub-volume. And in step 310n, the set of composite images is generated for the nth sub-volume.

[0128] Following each of steps 310a, 310b, and 310n (and thus in parallel within each sub-volume), indexing of the region of interest is performed via an indexing unit. In other words, each set of composite images within each sub-volume is evaluated in parallel. Preferably, the evaluation is performed using a deep convolutional neural network (DCNN) that preferably includes a density net model (DN).

[0129] Preferably, once three composite images for a given subvolume are obtained, they are fed into the RGB channels of the DCNN for parallel processing for classification. Preferably, DenseNet-121 is used as the classifier for detecting anomalies at the subvolume level.

[0130] There are two main parts: extracting features using a feature extractor and classifying using a classifier.

[0131] The steps for extracting features using a feature extractor include four dense blocks, as defined in the publication entitled “Densenet: Implementing efficient convnet descriptor pyramids” (arXiv preprint arXiv: 1404.1869 (2014)) by Iandola, F., Miskewicz, M., Karayev, S., Girshick, R., Darrell T. and KEutzer K., which is incorporated herein by reference.

[0132] After four dense blocks comes a global average pooling layer. Then the output of the global average pooling layer is flattened into a matrix by DF. i The 1-D vector is represented. Then the feature DF is... i The feed into the classifier includes a fully connected FC layer and a sigmoid activation for classification.

[0133] The normalized output vector p of the final sigma activation layer is added to the FC layer using the following formula. d (a|i),

[0134]

[0135] Where i is the input sub-volume, Let represent the probability score of subvolume i belonging to ICH category a. The parameters W of the global feature extractor are optimized by minimizing the binary cross-entropy (BCE) loss. d :

[0136]

[0137] The regions of interest for each set of the composite image of each sub-volume are then combined by a combining unit. Preferably, the combining unit is a recurrent neural network (RNN) whose output is, for example, a scan level priority ranking I indicating the probability of the presence of an anomaly.

[0138] Therefore, in a preferred embodiment, to combine sub-volume fractions into scan level-specific predictions and also consider scene information across sub-volumes, DenseNet is cascaded with a recurrent neural network (RNN). DF for each sub-volume is extracted. i The output of the final global average pooling layer of DenseNet. Then, for a given scan I, It is fed into the RNN.

[0139] In this embodiment, the RNN comprises a single gated recurrent unit (GRU) layer with input / output sizes of 1024 and 2048, respectively. Following the GRU is a fully connected layer with an output size of 1. The output of the final sequence step FC is considered for prediction of the scan level ICH.

[0140] Similar to Equation 3, the final sigmoid activation layer is added to the normalized output vector p of the FC layer. r (a|i). Then, the weights W of the RNN are optimized using BCE loss. r .

[0141] For each sub-volume i of shape (J,K,L), where i={1,2…2n}, j={1…J}, k={1…K}, l={1…L}, three composite images using MIP, MinIP, and OtsuIP are generated as follows:

[0142] MIP(i jk )=max(i(J=j,K=k)) (5)

[0143] MinIP(i jk )=min(i(J=j,K=k)) (6)

[0144] OtsuIP(i jk )=otsu(i(J=j,K=k)) (7)

[0145] Here, MIP is the maximum intensity projection, and MinIP is the minimum intensity projection.

[0146] Furthermore, Otsu determined the intensity value representing the point of maximum variance in the reduction from a grayscale image to a binary image. It was found that using this threshold point allowed for capturing the most significant differentiators across the slice. MIP images help identify high-density areas, particularly hemorrhages. MIP images are also beneficial for distinguishing between calcification and areas of intense hemorrhage. Preferably, in this case, the number of sub-volumes n = 16, as this appears sufficient during labeling.

[0147] like Figure 4a , 4b As shown in 4c and 4d, MIP is effective in capturing even extremely small areas of bleeding. Here, Figure 4a , 4b Images 4c show three successive slices of an axial scan or axial view of a patient's head with hemorrhage. In other words, the object of interest is the head, and the region of interest is the hemorrhage within the head. Figure 4d MIP composite images from a collection of axial slices are shown, including (but not limited to) images from the same axial scan. Figure 4a , 4bAnd 4d axial slices.

[0148] Figure 4e Full-scan MIP images of the same axis are shown, where hemorrhage is not visible.

[0149] MinIP images allow for the highlighting of abnormalities in the ventricles, low-density areas, and subdural regions.

[0150] like Figure 5a As shown, MIP composite images help identify bleeding, and as... Figure 5b As shown, the MinIP composite image reveals clear hydrocephalus.

[0151] Typically, MinIP improves the highlighting of low-density areas, making it easier to identify low-intensity bleeding (such as...). Figure 6a and 6b (as shown) and subdural hemorrhage (such as...) Figure 7a and 7b (As shown).

[0152] Figure 6a and 6b These represent MIP and MinIP, respectively, axial scans containing hemorrhage within low-density regions.

[0153] Figure 7a and 7b These represent the MIP and MinIP, respectively, for axial scans containing subdural hemorrhage.

[0154] Each as Figure 8a and 8b As shown, MIP highlights the hemorrhage area (region of interest), and MinIP highlights the impact on the ventricles (another region of interest). OtsuIP captures both high-density and low-density areas very well, such as... Figure 8c As seen in the text.

[0155] Figure 9a and 9b The image shows the head of a patient with a hemorrhage that has caused vertical displacement of the ventricles. This is in Figure 9b This is clearly visible in the coronal view / scan, and can be easily detected using its composite image. Therefore, not only should we consider... Figure 9a The axial scan / view shown is shown, and it is also beneficial to consider the coronal view, as it provides inter-slice scene information for the axial slices.

[0156] The following is presented as follows: Figure 3 The overall training process of the disclosed method.

[0157] By applying the brain window, brain regions are extracted to obtain MIP, MinIP, and OtsuIP for each subvolume of the scan.

[0158] A DCNN (DenseNet) model is trained on subvolume composite images using a stochastic gradient descent optimizer (SGD) and a binary cross-entropy loss (BCE) function to obtain the network's W, which represents the normalized output probability of the output individual subvolumes. d Weights.

[0159] Load the weights W of the DenseNet model d And for fine-tuning (W) r or W d and W d A further stage involves training an RNN using an SGD optimizer on the BCE loss to obtain the network's W, which outputs normalized output probabilities for each scan. r (and W) d Weight.

[0160] The following is a more detailed description of embodiments of the present invention including a workload distribution unit.

[0161] Once the score (i.e., the severity of the scan / view / image) is generated by the (image) processing unit 140, it is passed to the workload allocation unit 142.

[0162] The scan can then be categorized into three regions based on a pre-configured threshold and the score: red (immediate attention), yellow (potential danger discovery), and green (no immediate attention).

[0163] If a scan is in the red zone, an alert can be automatically sent to the relevant specialists for immediate action. Scans in the yellow zone are moved to the front of the work list after scans in the red zone for prioritization. Scans in the green zone are unlikely to contain critical findings but will still be evaluated according to standard hospital workflows. The severity classification score can be further used to assign priority scores to case handlers (i.e., neurologists or neurosurgeons) for immediate evaluation.

[0164] like Figure 10 and 11 As shown, the invention also includes embodiments of viewing pre-indexed slides (which may have critical pathology) during radiologist / expert reports and diagnoses.

[0165] Figure 10 A schematic flowchart of an embodiment of a method according to the present invention, including the step of pre-displaying index slices, is shown.

[0166] At the smart viewing station (or monitor), the item at the top of the 405 work list is retrieved first. In other words, step 405 is equivalent to obtaining the input scan / view.

[0167] View the image from archive 415 to obtain 410 pre-indexed slices, along with associated scores and view types.

[0168] Then check the configuration in the storage server to retrieve the pre-display flag 420, which determines whether the viewer is preset to a specific slice of interest.

[0169] If the pre-display flag is set to false, the default axial and coronal views of the 435 scan are displayed. If the pre-display flag is set to true, then the index associated with the maximum sub-volume severity score at 440 is identified. The viewer is then configured such that the first slice of the identified sub-volume is in view 445.

[0170] Figure 11 An example user interface for the intelligent viewing station (or monitor) is shown. Regardless of the pre-displayed flags, the user can view the critically identified area at any time by the image viewing unit. By providing this, the user can evaluate the area of ​​interest and make quick decisions about the course of action.

[0171] In summary, the present invention provides a rapid and effective system and method for identifying critical findings in non-contrast head CT scans, making it highly useful in emergency situations requiring swift action.

[0172] This system and method can also be used when radiologists are unable to act as a second opinion for physicians.

[0173] In the validation study, the publicly available CQ500 dataset was used. This dataset contains various hemorrhage subtypes of different sizes. The study utilized majority voting from three radiologist labels to determine the true extent. Additionally, subvolume level labels were created by individuals (without clinical background or experience) to train the subvolume density 121 network.

[0174] Rotation enhancement was applied to increase the dataset size by 12 times. For all our experiments without patient overlap, the training, test, and validation decompositions were 60, 20, and 20, respectively. Each image was represented by a 1D vector L = [l a ] mark, where l a ∈0,1, where 1 indicates the presence of ICH and 0 indicates the absence of ICH.

[0175] The results of the study show improvements achieved by using subvolumes and cascading DCNN with RNN.

[0176] To illustrate the values ​​of the subvolumes, a DenseNet model (similar to those previously discussed) was trained on a single MIP image obtained from a full scan, and this was used as a benchmark for our model (full scan MIP).

[0177] In addition, another benchmark is created by combining sub-volume probabilities using a simple maximum operation (sub-volume maximum fraction) to obtain scan level predictions.

[0178] This allows for an increase in the value of the RNN (subvolume RNN) used in this invention, to not only learn the realities of the scanned horizontal ground but also to handle background aspects better.

[0179] Table 1 below shows a comparison of the methods described above. As can be seen, there is a significant improvement from full-scan analysis to sub-volume analysis because small anomalies are better visible. There is also an improvement by combining sub-volume features with RNNs instead of using simple maximum fractions. Table 2 shows a comparison of the present invention with previous work.

[0180] Table 1: Comparison of results from various methods on the CQ500 test set. Specificity at a given high-sensitivity control point is given.

[0181]

[0182] Table 2: Comparison of the proposed method with previous work. The metrics reported below by Chilamkyurthy et al. are on a subset of our CQ500 tests, where TPR / TNR (sensitivity / specificity) were obtained at the points of least difference.

[0183]

[0184] Arbabshirani, MR, Fornwalt, BK, Mongelluzzo, GJ, Suever, JD, Geise, BD, Patel, AA, & Moore, GJ (2018). Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. npj Digital Medicine, 1(1), 9.

[0185] Chang,PD,Kuoy,E.,Grinband,J.,Weinberg,BD,Thompson,M.,Homo,R.,Chen,J.,Abcede,H.,Sha_e,M.,Sugrue,L.and Filippi,CG,2018.Hybrid3D / 2Dconvolutional neural network for hemorrhage evaluation on head CT.AmericanJournal of Neuroradiology,39(9),pp.1609-1616.

[0186] Chilamkurthy,S.,Ghosh,R.,Tanamala,S.,Biviji,M.,Campeau,NG,Venugopal,VK,...&Warier,P.(2018).Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study.The Lancet,392(10162),2388-2396.

[0187] RNNs are generalized for any sequential data. The basic idea is that an RNN can store some parameters from previous states to compute the value of the current state. In this invention, "state" refers to the current sub-volume being processed. The RNN is also used to weight the parameters of the current sub-volume based on information from previous sub-volumes, meaning it includes the context of previous sub-volumes used for decision-making at the scan / view level. This improves the results.

[0188] This invention considers both coronal and axial views in order to evaluate the vertical pull / push of a structure for mass effects. While coronal views appear to be the optimal approach for evaluating such vertical pull / push of a structure for mass effects, any combination of views with sagittal views can also be advantageous.

[0189] In summary, this invention is an advantageous method that combines the benefits of both slice-level and global scan analysis. Creating small subvolumes ensures that it does not miss minute anomalies and also allows for increased dataset size and improved learning. The use of RNNs to process scene information across slices further improves the results.

[0190] By creating multiple composite images to ensure no important information is lost, the model is generalized to any abnormality in the brain region.

Claims

1. A processing system for processing medical volumetric image data of scanned objects of interest and for indexing regions of interest in said objects of interest regarding whether they contain abnormalities, said processing system comprising: A registration unit configured to register the image data from the imaging system; A sub-volume generator is configured to generate sub-volumes based on the volume image data, each sub-volume having a slice thickness, each slice thickness comprising several slices of the original scan of the volume image data; A composite image generator is configured to generate multiple sets of composite images based on each sub-volume, each set of composite images representing a different projection of its corresponding sub-volume image data; An indexing unit is configured to generate an indexed region of interest in each subvolume by evaluating each composite image in the set of composite images in parallel and generating a score for each region of interest regarding whether the region of interest contains anomalies. as well as A combining unit is configured to combine sub-volume fractions obtained from each set of composite images of each sub-volume into a scan level priority order of image scans in a working list, the working list defining a priority order for human evaluation.

2. The processing system according to claim 1, wherein, The sub-volume generator is configured to create sub-volumes based on an axial view and also based on a coronal view.

3. The processing system according to claim 1 or 2, wherein, The plurality of sets of the composite images include at least: A collection of composite images generated by maximum intensity projection; A collection of composite images generated by means of minimum intensity projection; and A collection of composite images generated by gray-level thresholding using gray-level histograms.

4. The processing system according to claim 1 or 2, wherein, The indexing unit includes a classifier configured to classify anomalies in the object of interest and generate anomaly probability scores for each indexed region.

5. The processing system according to claim 4, wherein, The classifier is a deep convolutional neural network (DCNN).

6. The processing system according to claim 3, wherein, The combining unit is configured to cascade each depth convolutional neural network with a recurrent neural network for each set of composite images of each subvolume, thereby providing the scan level priority sorting.

7. The processing system according to claim 1 or 2, wherein, The combining unit is also configured to pre-index the sub-volumes according to the scan level priority sorting.

8. An imaging system, comprising: An image acquisition unit is used to acquire image data of the object of interest. as well as A processing system for processing medical volumetric image data of a scanned object of interest and for indexing regions of interest in the object of interest regarding whether they contain abnormalities, according to any one of claims 1-7.

9. The imaging system according to claim 8, wherein, The imaging system is a CT imaging system.

10. An image viewing system, comprising: The processing system according to any one of claims 1-7; as well as monitor.

11. A method for processing medical volumetric image data of a scanned object of interest and for indexing regions of interest in the object of interest regarding whether they contain abnormalities, comprising the steps of: The image data from the imaging system is registered using a registration unit. A sub-volume generator is used to generate sub-volumes based on the image data. Each sub-volume has a slice thickness, and each slice thickness includes several slices of the original scan of the volume image data. A composite image generator is used to generate multiple sets of composite images based on each sub-volume, each set of composite images representing a different projection of its corresponding sub-volume image data; The region of interest in each sub-volume is indexed by means of an indexing unit, which evaluates each composite image in the set of composite images in parallel and generates a score for each region of interest regarding whether the region of interest contains anomalies. as well as The sub-volume fractions obtained from each set of composite images of each sub-volume are combined into a scan level priority order of image scans in a working list by means of a combination unit, the working list defining the priority order for human evaluation.

12. The method according to claim 11, wherein, The steps for creating a subvolume also include: Create sub-volumes based on the axial view; and Create sub-volumes based on the coronal view.

13. The method according to any one of claims 11-12, wherein, The steps for generating multiple sets of composite images also include: Generate a set of composite images generated by maximum intensity projection; Generate a set of composite images generated by means of minimum intensity projection; and Generate a set of composite images generated by grayscale thresholding.

14. The method according to any one of claims 11-12, wherein, The indexing process also includes: Anomalies in the object of interest are classified using a classifier, and anomaly probability scores are generated for each indexed region.

15. The method according to claim 14, wherein, The classifier is a deep convolutional neural network (DCNN).

16. The method of claim 14, wherein, The combination process also includes: Each depth convolutional neural network for each set of composite images for each sub-volume is cascaded with a recurrent neural network to provide the scan level priority ordering.

17. The method according to any one of claims 11-12, wherein, The combination step also includes pre-indexing the sub-volumes according to the priority order of the scan levels.

18. A computer program comprising program code units, wherein when the computer program is executed on a computer, the program code units are configured to cause the computer to perform the steps of the method according to claim 11.

Images