Magnetic resonance histopathology and neural network classification for cancer

Abstract

Diagnosis of prostate cancer often involves invasive measures, such as biopsies, in which tissue is removed and graded by a pathologist. Non-invasive measures, such as multi-parametric MRI, can be used to examine lesions and grade them using the PI-RADS scoring scale; however, achievable image resolution is limited by motion. A new paradigm, magnetic resonance histopathology (MRH), is disclosed, which aims to evaluate tissue texture at sub-millimeter resolution through a fast clinical acquisition process. The ability of MRH to identify cancerous tissue in the prostate through computer simulation analysis is demonstrated, reproducing the MRH measurement process on a set of high-resolution, histology slides annotated by pathologists (2, 4, 6, 8, 10, 12, 14). A dataset of spectral intensities (20) at sub-millimeter wavelengths is created, and a deep learning model (22) trained to classify the spectral data is based on normal or tumor tissue. A set of spatial frequencies is identified, which is used to optimize diagnostic capability under the constraint of limited acquisition time. In addition to single-region classification, the disclosed method and architecture integrate spatial context and local information, the inclusion of which improves model performance and denoises inference results. In addition to algorithms for estimating the physical length scale of lesions identified by the model, the trained model is demonstrated to be applied to unlabeled high-resolution 3D MRI data.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63590428, filed October 14, 2023, and U.S. Provisional Application No. 63675506, filed July 25, 2024, both of which are entitled “Magnetic Resonance Histopathology and Neural Network Classification for Prostate Cancer”, and both of which have a common assignee with this application, the disclosures of which are incorporated herein by reference. Technical Field

[0002] The disclosed methods relate to the assessment of cancer using magnetic resonance histopathology (MRH), with the use for prostate cancer described as an example, and more specifically to a method for training a neural network using MRH inputs to provide a probability vector for predicting the probability of tumor tissue. Background Technology

[0003] Prostate cancer (PCa) affects millions of people worldwide, with over one million new cases diagnosed each year. When the cancer is detected at an early stage (approximately 80% of diagnoses), the five-year relative survival rate exceeds 90%; however, if the cancer has metastasized outside the prostate, the five-year relative survival rate can drop below 40%. PCa is typically diagnosed via biopsy after an abnormal result from a digital rectal exam and / or a blood test for prostate-specific antigen (PSA). Diagnostic uncertainty, leading to overtreatment and undertreatment, remains a major challenge in PCa diagnosis. Systematic transrectal ultrasound-guided biopsy is the standard approach, but it can miss tumors located in the anterior region and carries a risk of infection. More precise non-invasive screening methods, such as multiparametric magnetic resonance imaging (mpMRI), can help avoid unnecessary biopsies through better alignment; compared to ultrasound, mpMRI achieves better contrast between benign and cancerous tissue, making serious lesions often clearly visible. While mpMRI-guided biopsies are more sensitive in locating clinically significant tumors, they may miss lower-grade cancers. On the other hand, systematic biopsies are more likely to identify low-grade malignancies that do not require treatment, but may miss more serious tumors located in certain areas. The combined approach of mpMRI-guided and systematic biopsies reduces diagnostic uncertainty, but this process requires an increased amount of core tissue, which equates to an overall increase in time, cost, and risk of infection.

[0004] Biopsy is a highly invasive procedure, but the resulting histopathological analysis remains the gold standard for diagnosing PCa. Analysis of hematoxylin and eosin (H&E) stained histological sections from biopsy samples is performed to obtain Gleason pattern characteristics of the cancer, and Gleason scores are assigned based on the severity of the pattern. In recent years, AI-based methods have supplemented pathologists' findings and enabled new scientific discoveries. Summary of the Invention

[0005] In a wide range of exemplary implementations, this disclosure provides a method for tuning a neural network to analyze MRH data, wherein multiple two-dimensional regions of interest (ROIs) are selected in each of multiple histological or electronic images, each image annotated with normal region indicators and tumor region indicators. Each of the multiple ROIs is then collapsed along a lateral direction associated with its lateral width to generate a one-dimensional signal along an analysis direction associated with the analysis width. The one-dimensional signal in each ROI is convolved with a set of complex sine functions to compute a simulated MRH spectrum as a function of wavelength. Simulated spectra are then computed from each of the multiple ROIs at multiple k-values, generating multiple datasets corresponding to each of multiple histological sections. One or more neighbors for each ROI are then selected from a set of 4 nearest neighbors, 8 nearest neighbors, 24 nearest neighbors, or 16 non-adjacent neighbors. The datasets are then shuffled and divided into two parts. In the neural network, simulated spectra are received from each Region of Interest (ROI) and its selected neighbors in the first part of these components to generate a two-element probability vector. This vector indicates the probability that the input spectrum of the ROI originates from normal tissue and the probability that the input spectrum of the ROI originates from a tumor; the sum of these values ​​is 1. The network is trained by comparing the resulting probability vector with the annotation of the ROI. In the neural network, simulated spectra are received from each ROI and its selected neighbors in the second part of these components to generate a two-element probability vector. This vector indicates the probability that the input spectrum of the ROI originates from normal tissue and the probability that the input spectrum of the ROI originates from a tumor; the sum of these values ​​is 1. The training of the neural network in the first part is validated by comparing the resulting probability vector with the annotation of the ROI. Then, 3D MRH data is obtained from the tissue region of interest (in vivo or from a tissue phantom), which generates actual spectra on multiple volumes of interest (VOIs) in the 3D prostate, and the actual spectra from each VOI and its selected neighbors are received in the neural network to generate a two-element probability vector indicating the probability that the spectrum of the VOI comes from normal tissue and the probability that the spectrum of the VOI comes from tumor, with the sum of these values ​​being 1.

[0006] In a detailed exemplary implementation, the described method provides MRH measurements by training a neural network classifier using annotated histological sections. A pseudo-complete organ digitized H&E-stained histological section is created, with annotations indicating tumors, prostatic intraepithelial neoplasia (PIN), and atypical adenomas. The tumor annotations are converted into an image mask used to label regions of interest as normal or tumor. The complete organ section is segmented into peripheral zones. The peripheral zones are further divided into multiple regions of interest (ROIs). Exemplary normal ROIs and ROIs located within the tumor are identified. Each of the multiple ROIs is averaged in one dimension to generate a 1-D location signal, which is then Fourier transformed to produce a simulated MRH spectrum. The spectra from the normal and tumor ROIs are compared, revealing significant differences in texture. Since only a limited number of wavelengths can be encoded in clinical practice, in the exemplary implementation, 10 wavelengths selected and uniformly sampled spectrally are identified to enable differentiation between normal and tumor tissue. The intensity at a specific wavelength extracted from the spectrum is used as input. The classifier outputs a probability vector for the corresponding sample, indicating whether it is normal or has a tumor.

[0007] This method also employs adjacent ROI inputs to incorporate the influence of spatial information into the model. In addition to each individual ROI among multiple ROIs, it may include a choice of one of the following: ROIs with one nearest neighbor on each side, giving an input size of (3,1); ROIs with two neighbors on each side, giving an input size of (5,1); ROIs with three neighbors on each side, giving an input size of (7,1); or a square input, for which: one ROI has one layer of neighbors along all directions, forming an input of size (3,3); the ROI has two layers of neighbors, forming an input size of (5,5); the ROI has three layers of neighbors, forming an input size of (7,7). Attached Figure Description

[0008] The patent or application documents contain at least one color-printed drawing. Upon request and payment of the necessary fees, the organization will provide a copy of the published text of this patent or patent application with color-printed drawings.

[0009] Figure 1A A 2-D histological image is shown for generating annotation data for computer simulation analysis; Figure 1B An example is shown in 3D MRH data acquisition where histological images are divided into regions of interest (ROIs) corresponding to volumes of interest (VOIs); Figure 2This is a schematic diagram of a neural network classifier, where the intensity at a specific wavelength extracted from the spectrum is used as input, and the classifier outputs a probability vector indicating whether the corresponding sample is normal or tumor. Figure 3 It is a graph of the receiver operating characteristic (ROC) output by the neural network based on the intensity at different wavelengths as input; Figure 4 A to Figure 4 E is a description of neighboring VOIs, using neighboring VOIs to incorporate the influence of spatial information in the neural network, where, Figure 4 A shows an isolated VOI (center red rectangle); Figure 4 B shows VOI and its 8 next nearest neighbors (light blue); Figure 4 C shows the VOI and its 24 nearest neighbors (green); Figure 4 D shows VOI and its 48 nearest neighbors (yellow); and Figure 4 E is the VOI and its 16 non-adjacent neighbors (blue); Figure 5 Is it used as Figure 4 A to Figure 4 The receiver operating characteristic (ROC) curve of the neural network with a variable number of neighboring spectra described in E shows that the best performance is achieved when the spectrum of each VOI is accompanied by the spectra of 24 or 48 neighbors when it is input into the network, as shown by the maximum area under the curve. Figure 6 The predicted map for each of the three test images is shown. The predicted map was generated using the 0-neighbor model, the 8-neighbor model and the 24-neighbor model. It is placed side by side with the ground truth annotation. The color scale indicates the probability of high tumors in red, the probability of 50% tumors in white, and the probability of low tumors in blue. Figure 7 The receiver operating characteristic (ROC) curves of a neural network with VOI and 0 neighbors as input are shown, comparing the VOI sizes of 2mm×2mm, 2.5mm×2mm and 3mm×2mm.

[0010] Figure 8 The receiver operating characteristic (ROC) curves of a neural network with VOI and 24 neighbors as input are shown, comparing the cases with VOI sizes of 2mm×2mm, 2.5mm×2mm and 3mm×2mm. Figure 9A The image shows a 12mm × 7.8mm × 2.7mm 3D MRI sample collapsing along the z-axis (2.7mm). Figure 9BA predicted map of a 3D MRI sample is shown, in which each VOI is colored with the predicted probability of containing tumor tissue, and the input is the spectrum from each VOI (excluding its neighbors); Figure 9C is the prediction graph for the case of 8 neighbors; Figure 9D is the prediction diagram in the case of 24 neighbors; Figure 10 A to Figure 10 F shows a flowchart in which MRH measurements are simulated from annotated histological slices and used to train a neural network classifier. Figure 11 This is a representation of a 10-frequency input neural network; Figure 12 This is a graph showing the attribution magnitude of the input features, averaged across all test sets of the best-performing fully connected neural network using all 39 wavelengths as input features. Longer and shorter wavelengths tend to have a greater impact on the network output. Figure 13 A shows an exemplary ground-based annotated histological section with a large cancer (red, tumor; blue, normal). Figure 13 B shows a comparison between the exemplary model inference results and the ground truth (red, tumor; blue, normal) ground truth label map of the peripheral zone; Figure 13 C shows a comparison between the exemplary model inference results and a ground-based (red, tumor; blue, normal) single-region inference heatmap using a trained neural network; Figure 13 D illustrates a comparison between an exemplary model inference result and a ground truth (red, tumor; blue, normal) inference heatmap from the model, where the model incorporates data from neighboring regions to denoise the model predictions; Figure 14 A to Figure 14 H is a graphical representation of adjacent ROI inputs, using adjacent VOI inputs to incorporate the effects of spatial information into the model.

[0011] Figure 15 The inference results using an increased number of neighbors in the rod structure are shown compared to the ground truth labels. Colors represent the output predicted probability that a sample contains tumor tissue, where blue (0) indicates a 100% confidence prediction that the sample is normal, white (0.5) indicates no confidence that the sample is normal or tumor, and red (1) indicates a 100% confidence prediction that the sample is tumor. Gray represents areas with no data.

[0012] Figure 16The inference results of the stacked random forest (RF) model at the optimal classification threshold t are shown; blue represents normal data, red represents anomalies, and gray represents no data. As the number of neighbors increases from 2 to 6, the predictions become less noisy, and the model becomes more confident. However, the sensitivity to smaller anomaly areas appears to decrease. The “no data” gray area grows with increasing input size, where it cannot adapt to larger inputs near the edges of organs.

[0013] Figure 17 This is a graphical comparison of different models that incorporate neighbor information. The test AUC-ROC plot for cross-validation is drawn relative to the number of neighboring ROIs included in the input. CNN performance is divided by evaluation when predicting all 2n+1 labels (red) of the input pole, relative to predicting only the central ROI (green), which is the sole objective of the stacked RF model. Performance begins to degrade with larger input sizes greater than approximately 1 cm in organs.

[0014] Figure 18 The isolated tumor subarray and the inference heatmap generated by the stacked RF model are shown, where n=2; this corresponds to 5 ROIs as input, with the goal of predicting the center label; Figure 19 This is a flowchart of the size estimation algorithm; Figure 20 The tumor size density function is shown, obtained from the inference results using a stacked RF model and a size estimation algorithm. The 95% confidence intervals are shaded in red. The ground truth tumor width of the subarray is plotted as a histogram in blue. Figure 21 The first panel shows a 12mm × 15mm × 2mm 3D MRI sample collapsed along the z-axis. Panels two through seven: predicted maps of the 3D MRI samples output by models trained on input dimensions containing 0 to 5 neighbors. Each VOI is colored with the predicted probability of containing tumor tissue.

[0015] Figure 22 This is a graphical representation of the receiver operating characteristic (ROC) curves of a neural network using the intensity at different wavelengths as input. The network trained on 10 wavelengths evenly spaced between 0.05 mm and 0.95 mm exhibits the best performance and the lowest probability of overfitting. Figure 23 It is a graphical representation of the receiver operating characteristic (ROC) curve of a neural network using a variable number of adjacent spectra aligned in a rod-like structure; Figure 24This is a graphical representation of the distribution of tumor size (within connected regions of interest) across the entire peripheral zone (PZ) segmentation dataset. Regions of interest with a threshold of 500 connections (corresponding to a linear scale of ~7 mm; shown as black dashed lines) were selected to distinguish between the categories "small" and "large". Figure 25 This is a graphical representation of the receiver operating characteristic (ROC) curve of the stacked RF model on the complete dataset (black) and datasets divided by tumor size (red, large tumors; blue, small tumors). Performance is good on large tumors, while performance degrades on small tumors due to the presence of many neighbors. The AUC metric peaks around n≈16. Figure 26 illustrates the effect of modulating the classification threshold t on the inference graph. Four thresholds are shown, obtained by perturbation around the optimal Fl threshold t*=0.3. On the left, the tumor subarray is delineated on the ground truth label map. Inference is performed using a stacked RF model with 6 adjacent VOIs. Figure 27 shows the reasoning results using an increased number of neighbors in the square construction; Figure 28A presents a graphical representation of the precision and recall of the stacked RF model relative to the number of neighbors n included in the total input size of 2n+1 VOIs of the complete dataset; and Figure 28B presents the dataset divided by tumor size, where small tumors are defined as having fewer than 500 connected VOIs. Detailed Implementation

[0016] The typical diagnostic process for prostate cancer involves invasive measurements, such as a biopsy, in which tissue is removed and graded by a pathologist. Non-invasive measurements, such as multiparametric MRI, can be used to examine and grade lesions using the PI-RADS scoring scale; however, the achievable image resolution is limited by motion. Therefore, it is desirable to use MRH to assess tissue texture at sub-millimeter resolution through a rapid clinical acquisition process trained with a deep neural network classifier using computer simulation analysis, which employs simulated MRH measurements to evaluate actual MRH data.

[0017] 1.0 Basic Methods This document discloses a method called magnetic resonance histopathology (MRH), which non-invasively probes tissue texture at microscopic resolution exceeding that of conventional image-based magnetic resonance imaging (MRI). Pulse-sequence imaging and manipulation methods for magnetic resonance microtexture (MRμT) scanning, which form the basis for MRH, are disclosed in U.S. Patents Nos. 10061003, 10215827, and 10955503, and pending application No. 17577098, all of which have common assignee with this application, and the disclosures of all of these patents and applications are incorporated herein by reference. MRI provides in vivo imaging, which is highly sensitive to both active and involuntary movements of the patient, such as breathing and bowel movements. The current PI-RADS resolution standard for T2-weighted prostate imaging is a pixel size of ≤0.4 mm (frequency) and ≤0.7 mm (phase) (see Turkbey, Baris et al., “Prostate Imaging Reporting and Data System Version 2.1: An Update to Version 2 of Prostate Imaging Reporting and Data System 2, 2019”, European Urology, 76(3):340-351, 2019 (Turkbey) and Barrett, T. et al., “PI-RADS Version 2: What You Need to Know”, Clinical Radiology, 70(11):1165-1176, November 2015, (Barrett)), while glandular structures can vary at a scale of 0.1 mm or smaller. However, by balancing image acquisition with spectral acquisition designed to measure texture, MRH can achieve a resolution approximately an order of magnitude higher (0.05 mm) than conventional MRI.

[0018] MRH uses a one-dimensional signal containing spatial frequency information of the underlying tissue, rather than imaging the organ in two dimensions. In one exemplary implementation, MRH is unaffected by patient movement, as it measures a single spatial frequency (k-value) for each excitation, or a series of k-space codes by modulating the encoding of each echo refocused in a multi-echo sequence; the encoded spins move with the tissue, thus ensuring precisely localized measurements provided the excited area remains within the receiver. Multiple k-values ​​are also measured using repeated excitations (see, for example, U.S. Patent No. 10,215,827). Although patient movement occurs during this process, spatial coherence is not required because the goal of MRH is to capture texture rather than a complete image. In a second exemplary implementation, MRH may use a selective excitation sequence of a first slice within the sample of interest and a selective refocusing sequence of a second slice to define a rod, followed by the application of a gradient along the analysis direction that sweeps through a small range of k values, wherein the receiver bandwidth is narrowly set to depict the length of the volume of interest (VOI), and post-processing is performed at various bandwidths to define additional VOIs in the rod along the analysis direction (e.g., see 10955503).

[0019] While measurements using MRH can be considered independent, they collectively provide a picture of spatial variation within the underlying tissue. Because it is assumed that the tissue has a quantifiable texture—that is, a structure that repeats in a somewhat regular manner (in the prostate, glands scattered throughout the stroma)—the tissue will possess identifiable and distinct spatial frequency characteristics (spectral signatures) corresponding to the prominent patterns present within the tissue. Since the time required for data acquisition is directly related to the number of k-values ​​measured, identifying a small set of information-rich k-values ​​is a primary objective. This paper discloses a calibration technique for finding the optimal selection of wavelengths that maximizes diagnostic capability while minimizing clinical acquisition time.

[0020] MRI is a widely used technique, and the MRH procedure can be performed on any MRI scanner. For example, the MRH procedure can be added to an mpMRI scan, and the two together can produce preliminary diagnostic results and enhance the targeting of biopsy sites to minimize unnecessary invasive procedures.

[0021] Examples of pulse sequence imaging and manipulation methods for MRμT disclosed in pending application No. 17577098, and validation of the method on phantom and in vitro samples, can be found in Sonn, Geoffrey A. et al., “MR methods for measuring the texture of soft tissue microstructures,” MRμT in Medicine, 86(1):308-319, 2021. Preprint: https: / / onlinelibrary.wiley.com / doi / pdf / 10.1002 / mrm.28731 (Sonn). Sonn demonstrated a significant difference in signal intensity from normal tissue compared to tumor. Using a 114-slice dataset, he systematically showed that deep learning methods can be used to distinguish MRμT spectra from normal tissue and tumor, enabling non-invasive diagnosis and tumor localization.

[0022] MRH measurements can be simulated from “ground truth,” which can be annotated high-resolution two-dimensional (2D) histological images, a three-dimensional (3D) model created from multiple stacked annotated 2D histological images, or annotated image slices from 3D MRI image data. In the first exemplary implementation described herein, seven complete organ histological slices are used: five slices 2, 4, 6, 8, and 10 imaged at 8 μm resolution, and two slices 12 and 14 imaged at 16 μm resolution, as shown below. Figure 1A As shown. Each image has been annotated by a pathologist using normal region 16 and tumor region 17, as shown in the image. Figure 6 The instructions are given as shown. After training the model in the neural network on these 2D images, the model is used to classify the 3D dataset, as described below. The 3D dataset consists of MRI images of the prostate, which is an exemplary organ, imaged at a resolution of 60 μm × 60 μm × 90 μm.

[0023] Simulated MRH data like Figure 1B As shown, from Figure 1ATwo-dimensional regions of interest (ROIs) are selected from high-resolution images of multiple histological slices shown.18 The “signal” associated with the image on each slice is determined by the pixel color spectral intensity, or by converting the associated pixel signal to grayscale. Each ROI is then collapsed into a 1D signal by summing the signal in one of the two dimensions of the 2D image, or, in the case of 3D data discussed below, by collapsing along two orthogonal dimensions, incrementally along the analysis direction each time. The simulated MRH spectrum is then calculated by convolving the 1D signal with a complex sine function, so that the simulated MRH spectrum is calculated as a function of wavelength. The dimension along which the ROI is collapsed is called the “lateral dimension”,20 and the remaining uncollapsed dimensions are called the “analysis dimensions”,21. The width of the ROI along the lateral dimension is called the “lateral width,” and the width along the analysis dimension is called the “analysis width.” The lateral width and analysis width are parameters to be optimized during computer simulation analysis, as described below.

[0024] Actual MRH pulse sequence (timing diagram) Figure 1C As shown in the diagram. As indicated by trace 100, RF pulse 101 is a 90° section-selective RF excitation of a first tissue block (bar width) when a gradient pulse 102 is applied in a first magnetic field gradient defined as the Y-axis. A negative gradient pulse 104 rephases the excitation within a defined thickness of the section or tissue block. Gradient pulse 106 is applied together with a 180° RF refocusing pulse 108. On the Z-axis, a tissue block orthogonal to the first tissue block is defined, generating a first spin echo (SE) 110. A master gradient pulse 111 and a negative coding gradient pulse 112 establish the desired k-space coding (k-space coding & rod height) along an analysis direction orthogonal to the second tissue block. A readout gradient 114 is applied simultaneously with SE 110 on the Z-axis. RF pulses 116 and gradient pulses 118 provide 180° slice-selective refocusing to generate multiple echoes 120 using the associated readout gradient 122 (only the first echo is shown). In addition to the identified gradient pulses, a crushing gradient 124 is present around all slice-selective 180° refocusing pulses.

[0025] The structure generated from the obtained data Figure 1D As shown in the diagram. Its axis runs along... A selectively excited rod 150, having k-space coding along the Z-axis, is defined by selective excitation of a 90° slice of a first tissue block to define the rod width, and by selective refocusing of a 180° slice to define the rod height, and generates k-space coding along the Z-axis by applying a gradient pulse. During spin echo recording... Applying a readout gradient to the axis generates a frequency gradient along the rod, providing a way to assign signals to individual pixels 151, each representing the VOI along the rod (shown as a 20-pixel bandwidth 152). This process is then repeated interleaved, such that adjacent and coplanar rod acquisitions are temporally separated to confine the data to the Y-axis. A texture image plane 154 with 20×20 voxels (pixels) in the plane and parallel to the Y-axis in the Z direction. Multiple texture image planes of a plane (for example, 10 adjacent texture image planes are shown 156). The voxel matrix and the number of image planes can vary.

[0026] The selection of ROIs in the simulated MRH mimics the MRH data collection process: First, fixed-width "bars" (two-dimensional columns with an applied width in the third dimension) are selected from the ground-based imagery to simulate the VOIs in the actual MRH data, and these bars can be considered simulated VOIs. Therefore, the terms VOI and ROI are used interchangeably in this paper. The width of these bars corresponds to the analysis width. The bars are then laterally divided into ROIs. The ROIs are collapsed along the lateral dimension and then convolved with a complex sine function to compute the simulated MRH spectrum as a function of wavelength.

[0027] Neural Network Classification The neural network is trained on spectra extracted from 2-D histological sections, which are ROIs (Regions of Interest) labeled as corresponding to pathologist annotations of normal tissue or tumors, examples of which will be described later. In an exemplary implementation, each ROI has an analysis length of 2 mm and a lateral width of 2 mm. To fine-sample each image and generate a broad training dataset, ROIs are extracted using a sliding window with a step size of 20 pixels (approximately 0.16 mm) across each entire image, and only ROIs completely contained within the organ section are used. Therefore, ROIs will overlap spatially but correspond to different spectra. For the exemplary implementation described herein, in Figure 1A A total of 208,201 spectra were collected from 7 image sets.

[0028] Fit the data for each ROI (one example of which is spectrum 20) to... Figure 2 The diagram schematically illustrates a three-layer artificial neural network 22, where, as an example, three frequency inputs are selected from a spectrum to perform a binary classification of whether the spectrum is normal or tumor. As will be described later, the number of frequency inputs (k values) selected to enter the model can vary (e.g., see [link to relevant documentation]). Figure 11(Displaying 10 input representations). In an exemplary implementation, the first hidden layer is a 128-node fully connected layer activated by ReLU; the second hidden layer is a 64-node fully connected layer activated by ReLU; the third hidden layer is a 64-node fully connected layer activated by ReLU; followed by an output layer activated by softmax for binary classification. The Adam optimizer is used with a learning rate of 0.001. The network output is a two-element probability vector 24, which contains the probability that the input spectrum originates from normal tissue and the probability that the input spectrum originates from a tumor, the sum of which is 1.

[0029] Feature selection In computer simulation analysis, spectra can be obtained from each ROI sampled at a greater number of wavelengths (k-values) compared to the number of wavelengths typically achievable in clinical k-space coding procedures. While simulated spectra are calculated at approximately 40 wavelengths from 0.05 mm to 1 mm (in increments of 0.025 mm), clinical MRH procedures can involve approximately 10 k-space codings to limit scan length and acquire more samples from a wider tissue region in the same amount of time. Therefore, a feature selection routine is performed to determine which wavelengths are most informative in binary classification. The performance of the neural network classifier was compared to that of each classifier trained on five different sets of inputs, each containing 10 features: a) spectral intensity at short wavelengths (between 0.05 mm and 0.275 mm, in 0.025 mm increments); b) spectral intensity at short to medium wavelengths (between 0.3 mm and 0.525 mm, in 0.025 mm increments); c) spectral intensity at medium to long wavelengths (between 0.55 mm and 0.775 mm, in 0.025 mm increments); d) spectral intensity at long wavelengths (between 0.775 mm and 1 mm, in 0.025 mm increments); and e) spectral intensity across the entire wavelength range (between 0.05 mm and 0.95 mm, in 0.01 mm increments). These results were also compared to a classifier trained on all 40 wavelengths, which is not feasible in clinical practice but demonstrates the potential upper limit of performance.

[0030] For each case, perform repeated 2-fold cross-validation on the dataset, where the data is shuffled and divided into two equal parts. Train and validate the network on one half of the data and test it on the other half. Next, train the network on the previously used test half and test it on the other half. Then shuffle the data again, and repeat this process for a total of 10 iterations.

[0031] Because the training data has a large class imbalance, with far more samples labeled as normal than as tumor, unequal weights are applied to each sample to correct this imbalance. ROIs containing tumor samples are assigned higher class weights compared to normal samples; the class weights are defined as the inverse of the prevalence of tumor samples in the training set.

[0032] like Figure 3 As shown, receiver operating characteristic (ROC) curves for each case were compared to determine the network's performance for each selected input feature. The network trained on 10 wavelengths evenly spaced between 0.05 mm and 0.95 mm exhibited the best performance and the lowest probability of overfitting. The network performed worst when only short or medium-short wavelengths were used, as characterized by the smaller area under the ROC curve. The network performed much better at longer wavelengths and slightly better at 10 wavelengths sampled across the entire wavelength span. Only a small improvement was achieved when all 40 wavelengths were used. Therefore, there may be diminishing returns when using more wavelengths as network inputs, making it possible to identify a scheme that satisfies the data-time tradeoff, i.e., the "optimal point," where a small amount of k-space encoding can be performed to achieve high diagnostic capability in a shorter clinical time.

[0033] Impact of spatial information Since regions of normal or cancerous tissue are continuous and can be correlated with feature length scales, information from neighboring VOIs can be considered as additional input to the neural network for each ROI in the dataset, as simulated by computer simulations of the spectra of neighboring ROIs of each ROI derived from the histological samples described above. Comparisons can be made as follows: Figure 4 A to Figure 4 The input shown in E is a combination of the following: intensities from 10 wavelengths located between 0.05 mm and 0.95 mm (as described in the previous section), which are derived from a) each (2 mm × 2 mm) ROI 30 as an independent sample; or compared with multiple selected neighbors. Figure 4 B shows each ROI 30 and its 8 next nearest neighbors 32, which are 6mm × 6mm regions; Figure 4 C shows each ROI 30 and its 24 nearest neighbors 34, which are 10mm × 10mm regions; Figure 4 D shows each ROI 30 and its 48 nearest neighbors 36, which are 14mm × 14nm regions; and Figure 4 E shows each ROI 30 and 16 non-adjacent neighbors 38, which also spans a 14mm × 14mm area.

[0034] The neural network is trained on data obtained from a first set of images (four images in the example shown), and on data obtained from a second set of images (the remaining three images in the example shown, such as...). Figure 1A The network was tested on the data obtained (as shown). For each case listed above, the same 4 (3) images were used for training (testing). Images 2, 4, 12, and 14 used for training were... Figure 1A The top row shows the results, while images 6, 8, and 10 used for testing are shown in the bottom row. For each case, the network performance is evaluated by comparing the ROC curves. Figure 5 The network performs poorly when data from each VOI is used as an independent input; performance improves significantly when eight second-nearest neighbors are also included. Using 16 non-adjacent neighbors produces very similar performance.

[0035] Using 24 neighbors achieves further improvements. Using 48 neighbors yields similar performance. When the false positive threshold is stricter (i.e., more sensitive at stricter thresholds), using 24 neighbors achieves a higher true positive rate because the curve rises more rapidly compared to the 48-neighbor model. However, the 24-neighbor curve remains stable at slightly smaller values ​​compared to the 48-neighbor model. Ultimately, the 24-neighbor model has a slightly higher area under the curve (0.917 vs. 0.905) compared to the 48-neighbor model. Larger regions spanned by 48 or 16 non-adjacent neighbors may contain too much variation in the underlying tissue structure and may wash away, rather than amplify, the relevant information contained in the central VOI. In short, 24 neighbors spanning a 10mm × 10mm square produce the best classification performance.

[0036] To visualize the network's output probabilities and how spatial information can influence them, the output probabilities are presented as heatmaps for each test image, such as... Figure 6 As shown in the left column, tumor 17 (right column) and normal 16 are annotated by real pathologists on the ground. Each pixel corresponds to an output probability indicating whether the underlying VOI is tumor (red) or normal (blue), where the middle (white) value indicates a lack of confidence. As the number of neighbors increases, the prediction becomes more confident (fewer white or light pixels) and also more accurate.

[0037] The performance of different sizes of basic VOI was also compared, such as Figure 7 As shown. The analysis width of each VOI remained fixed at 2mm, but datasets with horizontal widths of 2.5mm and 3mm were also simulated. The results were similar, but the VOI with a horizontal width of 3mm performed slightly better than the other sizes (here, each VOI used was isolated and had no neighbors).

[0038] When comparing 24-neighbor models from three different VOI sizes (it should be noted that the total area used as input will vary depending on the VOI size), the performance is almost identical across all VOI sizes, as shown in [example data]. Figure 8 What we see in the video.

[0039] This method can also be applied in 3D, where multiple sequentially arranged histological sample slices α, β, γ, δ, and ε correspond to, for example, Figure 1D The plane on the Z-axis shown, and based on Figure 1B The specific ROI shown 16. Y11 (specified element 19) is used to define the center "VOI", applicable to the sample γ selected in the middle of the slice and its neighbors at the same matrix position in the preceding and following samples. For example, the VOI of interest is the ROI (γ, 16, Y11); for a 2-neighbor rod, the neighbor is (β, 16, Y11) and (δ, 16, Y11); and for 4 neighbor rods, the neighbors are (α, 16, Y11) and (β, 16, Y11), and (δ, 16, Y11) and (ε, 16, Y11).

[0040] Optimal parameters In this example, the optimal parameters for clinical MRH data acquisition to achieve the best classification performance are as follows: Approximately 10 wavelengths were sampled uniformly between [0.05, 1] ​​mm.

[0041] Spatial information from 24 neighbors covering an area of ​​up to approximately 10mm x 10mm was used.

[0042] 2mm analysis width and 2-3mm lateral width.

[0043] Inference of 3D data 3D MRH data were created using MRH simulations of high-resolution 3D MRI images from tissue samples. Figure 9AThe tissue sample was observed to have a continuous region 50 measuring 12mm × 7.8mm × 2.7mm at the center of the actual organ 52. This region was further subdivided into 3D VOIs (corresponding to the ROI defined in the aforementioned histological sample image) with dimensions of 2mm × 2mm × 2.7mm. The VOI collapsed along the last two dimensions, producing a 2mm long 1-D signal from which a spectrum with wavelengths between 0.15mm and 0.95mm (in increments of 0.1mm) was generated. This matches the wavelengths (k-values) used to train the neural network using the aforementioned histological sample image, except for 0.05mm (0.05mm is not feasible due to the 0.06mm resolution of the 3D MRH data along the analysis dimension). The neural network was retrained on these nine wavelengths extracted from the spectra generated from all seven 2D histological images. The network was then tested on the spectrum of the 3D MRI image. Since no 3D MRH data was annotated in the example shown, there are no ground truth markers; however, in Figure 9B The diagram shows a prediction map of the probability that each VOI contains tumor tissue. Prediction maps were generated repeatedly for cases with 8 neighboring VOIs as input (third panel) and cases with 24 neighboring VOIs as input (rightmost panel). (Note that due to the finite size of the 3D tissue region, overlapping neighboring VOIs were selected to generate enough data to form a prediction map. The size of the 8-neighbor region is 2.6mm × 2.6mm, and the size of the 24-neighbor region is 3.2mm × 3.2mm.) As the number of neighbors increases, the basic pattern of the prediction map remains similar, but the predicted probability values ​​change dramatically. When using 0 neighbors, the probability is a relatively weak value close to 0.5, but as the number of neighbors increases, the prediction becomes more confident, growing closer and closer to 0 or 1. Predicted tumor regions located near the center of the bottom of the sample remain consistent, but become more pronounced as the number of neighbors increases.

[0044] In an example of using MRI image data as ground truth images to train a neural network, a pathologist annotates multiple MRI image slices derived from an MRI dataset as described above, using indications of tumor or normal tissue, to create a ground truth labeled dataset. Then, 1D data derived from multiple image slices in the 3D MRI data is used as input to train the neural network in a manner comparable to that described for multiple histological images, using cross-validation of the datasets. Training, validation, and testing are performed by shuffling and dividing the datasets into multiple parts, as described for the histological slice ground truth data above. The neural network can be trained using annotated 2D or 3D annotations of the MRI acquired image data; for example, such 2D or 3D annotations are radiologist interpretations or PIRAD reports, pathologist interpretations of subsequent optical histology of removed tissue, or determinations from analytical models containing conventional MRI image data (e.g., DWI, T1, T2, STIR, etc.).

[0045] Then, a neural network trained on any of the annotated image types described above can be used to analyze the MRH data obtained in vivo using the previously described MRH pulse format, where the extracted data corresponds to a plane on the Z-axis with the selected ROI and its neighbors.

[0046] Detailed experimental examples This paper discloses simulated MRH measurements, which involve obtaining spectral intensity in the frequency space at a selected k-value. Because MRH achieves higher resolution than MRI and is closer to the resolution achievable only by histological imaging performed in vitro, simulated MRH measurements were performed on histological sections involved in radical prostatectomy. Deep learning was used to identify the most information-rich k-values ​​in tumor classification, and a model architecture was developed to classify regions into abnormal or normal categories. An overview of the data generation process is provided in [the following section]. Figure 10 A to Figure 10 As shown in E, step a) provides 100 digitized H&E-stained histological sections of a pseudo-complete organ, such as... Figure 10 As shown in Figure A, the pseudo-intact organ digitized H&E stained histological section 100 has annotations 102 indicating tumors (red), PIN 104 (blue), and atypical adenomas 106 (green). In step b), the tumor annotations are converted into an image mask 108, which is then displayed in... Figure 10 As seen in B, the region of interest is marked as normal or tumor. In step c), the complete organ slice is segmented into the peripheral zone 110 (the region with outlines and shading), as shown... Figure 10 As shown in C. In step d), the outer periphery 110 is divided into 2mm × 2mm regions of interest (ROI) 118, as shown in... Figure 10 As seen in D. Exemplary normal ROI 118a is highlighted in blue, while ROI 118b located within the tumor is highlighted in orange. In step e), each ROI is averaged in one dimension to generate a 1-D location signal, which is then subjected to a Fourier transform to generate a simulated MRH spectrum 120, as shown. Figure 10 E is shown. will come from Figure 10 A comparison of the spectra of normal ROI 118a and tumor ROI 118b, highlighted in D, reveals significant differences in texture. Since only a limited number of wavelengths can be encoded in clinical practice, identifying 10 wavelengths uniformly sampled across the spectrum allows for differentiation between normal and tumor tissue. In step f), spectral data 120 is provided to the neural network classifier 122, such as... Figure 10 As shown in F (again, using three frequency inputs selected from the spectrum as an example; the representation of a neural network with 10 frequency inputs is shown in...), Figure 11 (As shown in the diagram). The intensity at a specific wavelength extracted from the spectrum is used as input. The classifier outputs a probability vector (124) corresponding to whether the sample is normal or tumor.

[0047] As disclosed in this paper, the best-performing model achieves over 85% classification accuracy. As previously described, integrating data from neighboring regions demonstrates improved inference results and enhanced predictions. Local information can be integrated in various ways, and different architectures are suggested for this purpose. These methods for tumor identification demonstrate good performance on computer-simulated datasets, and the effectiveness of methods for estimating tumor length scales from simulated MRH measurements is confirmed. Furthermore, a case study example of applying the disclosed neural network model trained on simulated MRH data to an unseen 3D prostate dataset is disclosed.

[0048] Results Overview The deep learning model was trained on MRH measurements simulated from “ground-real” histological images obtained from a dataset described by Singanamalli et al. (see “Identifying in vivo DCE MRI markers associated with microvascular architecture and Gleason grade in prostate cancer”, Singanamalli, Asha et al., 2016, Magnetic Resonance Imaging Journal, 43(1): 149-158, 2016 (Singanamalli)). Images were annotated by pathologists using regions labeled as carcinoma, prostatic intraepithelial neoplasia (PIN), or atypical adenoma; only carcinoma annotations were used to label the data used for training. Furthermore, the dataset was limited to the peripheral zone (PZ) of each prostate slice because the dense stroma, which is more prevalent outside the PZ, is easily confused with carcinoma. Dense stroma is typically located in areas that can be excluded during clinical MRH acquisition.

[0049] As previously described, the first step in simulating MRH spectral data is to divide each image slice into a 2D region of interest (ROI), the size of which is comparable to the area to be excited within the MRI scanner. Figure 10 D, step d). Each ROI is then collapsed along one dimension to generate a 1D signal from which a measurement of texture is derived. For clinical MRH measurements, a specific k-value or spatial frequency / wavelength is encoded after a volume of tissue is excited. The signal read from the MR scanner represents a measurement of the degree to which the tissue in the excited region has a texture matching the encoded k-value. This process is extended using repeated k-spatial encoding to obtain a spectrum consisting of intensities across a spatial frequency range. To simulate this spectrum, a discrete Fourier transform is applied to the collapsed 1D ROI at a specific k-value (or wavelength) to generate a frequency-spatial spectrum representative of the clinical measurement.

[0050] Then, each spectrum is labeled according to the tumor annotation. Figure 10 B, step b). A threshold is applied such that if at least 50% of the area corresponding to the ROI is located within a region labeled as cancer, the spectrum is assigned a positive label (1); otherwise, the spectrum is assigned a negative label (0) corresponding to the class weights previously described for the training data. This threshold can be varied to train a model that is more or less sensitive to spectral changes caused by the presence of a tumor; however, setting the threshold too low increases the false positive rate, while setting it too high increases the false negative rate. From the resulting dataset, 515,547 individual spectra were obtained, of which 80,475 spectra were labeled positive (15.6%). This dataset was then used to train a deep neural network model to classify spectra as normal or tumor (…). Figure 10E, step e). First, a feature selection process is performed to determine the wavelength or k-value that is most informative in distinguishing between normal tissue and tumor.

[0051] Feature selection Encoding more k-values ​​increases the length of MRH scans and can introduce motion sensitivity. Therefore, it is necessary to select an optimal number of k-values ​​for determining clinical parameters and for training the predictive model on simulated data. The number of k-values ​​should be large enough to span a range of textures, enabling differentiation between healthy and cancerous tissues, but also as small as possible to minimize scan time. Generally, no more than 10 wavelengths are considered ideal. Precisely determining which wavelengths to encode is crucial for maximizing diagnostic capability. Therefore, feature selection experiments are performed to determine which wavelengths input to the model produce the best classification performance.

[0052] A comparison of neural network classifiers was published, each trained on five sets of features: a) spectral intensity at short wavelengths (between 0.05 mm and 0.275 mm, in 0.025 mm increments); b) spectral intensity at short to medium wavelengths (between 0.3 mm and 0.525 mm, in 0.025 mm increments); c) spectral intensity at medium to long wavelengths (between 0.55 mm and 0.775 mm, in 0.025 mm increments); d) spectral intensity at long wavelengths (between 0.775 mm and 1 mm, in 0.025 mm increments); and e) spectral intensity at wavelengths spanning the entire range (between 0.05 mm and 0.95 mm, in 0.01 mm increments). These results were also compared to a classifier trained on 39 wavelengths (between 0.05 mm and 1 mm, in 0.025 mm increments), which is not feasible in clinical practice but demonstrates the potential upper limit of performance.

[0053] A joint parameter-hyperparameter scan was performed to determine the network architecture and overall optimal configuration (architecture and features) for each wavelength set. For each case, the receiver operating characteristic (ROC) curve and the corresponding area under the curve (AUC-ROC) were compared to determine the network performance for each selected input feature. Figure 22The results observed at short, medium-short, or medium-long wavelengths yielded the worst model performance, as quantified by AUC-ROC. The network performed significantly better at long wavelengths, and still performed slightly better at 10 wavelengths sampled uniformly across the entire measurement span. Optimal performance was achieved when using all 39 wavelengths, but this was only a very small improvement compared to using 10 uniformly sampled wavelengths. Therefore, there may be diminishing returns when using more wavelengths as network input, allowing us to identify schemes that satisfy the data-time tradeoff, i.e., the “optimal point,” where a small amount of k-space encoding can be performed to achieve high diagnostic capability in a shorter clinical time. From a deep learning perspective, restricting input features can also combat model overfitting. The best-performing architecture from the model trained on 10 uniformly sampled wavelengths was used as the baseline fully connected model, and the analysis was extended from the baseline fully connected model to include a broader spatial context, as described in the following subsections.

[0054] Similar performance levels can be achieved using fewer than 10 wavelengths, while further reducing measurement time. We address this issue by performing a full parameter scan of the model, where the model is trained on all possible combinations of 6 and 8 wavelengths from a set of 10 uniformly sampled wavelengths identified during the feature selection process. Results show that even using 6 wavelengths does not significantly degrade performance. The best-performing subset with 6 wavelengths achieves an AUC-ROC of 0.773, while the best-performing subset with 8 wavelengths achieves an AUC-ROC of 0.776. In contrast, the best-performing subset with 10 wavelengths achieves an AUC-ROC of 0.780. An overview of the best-performing subsets is listed in Tables S1 and S2.

[0055] Table S1: The top 5 best performing subsets of the 6 wavelengths and their corresponding AUC-ROC. Wavelength (mm) AUC-ROC {0.25, 0.55, 0.65, 0.75, 0.85, 0.95} 0.773 {0.15, 0.55, 0.65, 0.75, 0.85, 0.95} 0.771 {0.35, 0.55, 0.65, 0.75, 0.85, 0.95} 0.771 {0.45, 0.55, 0.65, 0.75, 0.85, 0.95} 0.770 {0.25, 0.45, 0.65, 0.75, 0.85, 0.95} 0.770 Attribution analysis To investigate and validate the empirical feature selection process, attribution analysis was performed using an integral gradient method implemented through the Captum package (see Narine Kokhlikyan et al., “Captum: A Unified and General Model Interpretability Library for PyTorch,” arXiv preprint platform, arXiv: 2009.07896, September 2020 (Kikhlikyan)), implemented in arXiv (see Mukund Sundararajan et al., “Axiomatic Attribution of Deep Networks,” 2017 (Sundararajan)). The gradient was integrated over the network, assigning an attribution score to each input feature. The attribution score is a signed value representing the influence of the input feature on the log-odds of the output of the classification layer constituting the network. The geometric mean of the cross-validation attribution magnitudes on the test set of each partitioned dataset is considered a measure of the influence of the network on a given input feature (i.e., k-space encoding). Figure 12 The geometric mean of the attribution amplitudes for all 39 wavelengths considered in feature selection, calculated using the integral gradient of a fully connected network, is shown. The longest wavelengths exhibit the largest amplitude effect; this is likely due to the stronger intensity of large wavelengths in organ regions lacking dense stroma, where abnormal growth is unlikely. This is also consistent with empirical feature selection results, where longer wavelengths perform relatively better. Shorter wavelengths show a larger amplitude effect compared to the center wavelength.

[0056] Feature selection using attribution results was considered in two ways: selecting the top 10 wavelengths by attribution and selecting the top 10 non-adjacent wavelengths by attribution as input features. Each of these selections resulted in poor performance from the experimentally identified, evenly spaced set of wavelengths (in AUC-ROC). This validates the selection of input features.

[0057] To investigate and validate our empirical feature selection process, attribution analysis was performed using the integral gradient method implemented through the Captum package (see “Captum: A unified and general model interpretability library for PyTorch”, arXiv preprint platform, arXiv: 2009.07896, September 2020, by NarineKokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan and OrionReblitz-Richardson et al., September 2020, arXiv preprint platform, arXiv: 2009.07896, September 2020) (see “Axiomatic Attribution of Deep Networks”, 2017, by Mukund Sundararajan, Ankur Taly and Qiqi Yan, 2017). The gradient is integrated over the network, assigning an attribution score to each input feature. The attribution score is a signed value representing the influence of the input feature on the output log odds of the classification layer that makes up the network. The geometric mean of the cross-validation of the attribution magnitudes on the test set of each partitioned data point is considered a measure of the influence of the network on a given input feature (i.e., wavelength). The gradient is calculated relative to the output tumor probability after the softmax operation.

[0058] Figure 12 The geometric mean of the attribution amplitudes for all 39 wavelengths considered in feature selection, calculated using the integral gradient of a fully connected network, is shown. The longest wavelengths exhibit the largest amplitude effect; this is likely due to the stronger intensity of large wavelengths in organ regions lacking dense stroma, where abnormal growth is unlikely. This is also consistent with empirical feature selection results, where longer wavelengths perform relatively better. Shorter wavelengths show a larger amplitude effect compared to the center wavelength.

[0059] Table S2: The top 5 best performing subsets of the 8 wavelengths and their corresponding AUC-ROC. Wavelength (mm) AUC-ROC {0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95} 0.776 {0.05, 0.15, 0.25, 0.55, 0.65, 0.75, 0.85, 0.95} 0.776 {0.15, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95} 0.775 {0.05, 0.25, 0.35, 0.55, 0.65, 0.75, 0.85, 0.95} 0.775 {0.01, 0.25, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95} 0.775 Exemplary model inference results compared to ground reality Figure 13 A to Figure 13 As shown in D. Figure 13 A shows an annotated histological section with a large cancerous area (tumor area 102 in red; normal area 104 in blue). Figure 13 B shows the above information regarding Figure 10 B and Figure 10 The ground truth label map of the peripheral zone 110 discussed in C; the data acquisition process effectively smoothed the edges of the cancerous area. Figure 13 C shows a single-region inference heatmap using a trained neural network. Figure 13 D shows the inference heatmap from the model, where the model incorporates data from neighboring regions, as discussed below, to denoise the model predictions.

[0060] Impact of spatial information Since cancer typically manifests as contiguous blocks with dense texture larger than individual ROIs, it is expected that training the model on inputs representing larger regions will improve classification performance, thereby providing greater spatial context to each ROI. Instead of expanding the ROI size, which would result in the removal of important texture signals due to increased variation, we use the spectrum from each ROI as described previously, plus the spectra of adjacent ROIs, as input.

[0061] The auxiliary benefit of considering the spatially expanded set of neighboring ROIs is the ability to estimate the length scale of the identified tumor by predicting for each ROI in the expanded input. This adds another layer to the diagnostic capability of the method, thereby identifying tumors and placing them within size categories. Algorithms for tumor scale estimation are developed and described as an application of the reasoning capabilities of neural networks, as discussed later.

[0062] As previously described, to incorporate spatial context, spectra from adjacent regions can be employed. Regions located on either side of a given ROI, input along the same axis in the direction of analysis, can be used to form the input “bars.” This mimics a clinical procedure where spectra are measured from multiple VOIs aligned with the volume along the excited bar. Figure 14A shows the isolated ROI 30 (center red rectangle); Figure 14 B shows an ROI 30 with a nearest neighbor 40 (light blue) on each side, given an input size of (3,1); Figure 14 C shows an ROI 30 with two neighbors 42 (green) on each side, given an input size of (5, 1); Figure 14 D shows an ROI 30 with three neighbors 44 (yellow) on each side, with an input size of (7,1).

[0063] Alternatively, for a square input, a 2D array of neighbors along the lateral direction can be used. Figure 14 E shows the isolated ROI 30; Figure 14 F shows an ROI 30 with one layer of neighbors 46 along all directions, forming an input of size (3,3); Figure 14 G shows an ROI 30 with two layers of neighbors 48, forming an input dimension of (5, 5); Figure 14 H shows an ROI 30 with three neighboring layers 50, forming an input dimension of (7,7).

[0064] As previously described, it is also possible to apply rod neighbors to the neighbors in the 2D array to form 3D arrays of 3, 3, 3, 5, 5, 5 and 7, 7, 7.

[0065] In practice, the simulated data is arranged in rod-shaped or square inputs to reduce the overall size of the dataset, because larger input sizes can extend beyond the edges of the organ. Instead of padding the data, it is assumed that the data is entirely contained within the organ.

[0066] When ROIs are input in the form of rods or squares, the classification objective is used as the label for the central ROI. For rod-shaped inputs, if n neighbor ROIs are included, the model receives 2n+1 spectra as input, each with a spatial encoding intensity of 10k. The training objective is to identify the label corresponding to only one of these spectra. Throughout, the term "n neighbors" is used to refer to an input including 2n+1 ROI spectra in the rod case or (2n+1)×2 ROI spectra in the square case.

[0067] Changing the number of neighbors requires retraining a new neural network to account for the change in the dimension of the input layer. In other words, different networks will need to be retrained to predict cases such as ROI with n=1 more neighbors, ROI with n=2 more neighbors, ROI with n=3 more neighbors, etc.

[0068] Typically, when retraining a network to account for different input sizes, the same hidden architecture is retained as the network that performed best in the previous feature selection parameter scan, with the only change being the increase in the dimension of the input layer.

[0069] Modal containing neighbor information In clinical applications, data acquisition from a group of neighboring ROIs in a rod shape is more efficient than square inputs; MRH measurements naturally trigger rods, while measuring the entire square ROI requires multiple separate acquisitions. When optimizing models that incorporate neighbor information, rod-based inputs are the primary consideration.

[0070] The simplest way to incorporate neighbor information is to use all spectral intensities as input features to the model in an unstructured manner, i.e., without any explicit location encoding. The desired outcome is that the model learns implicit concepts of locality or the relationships between spectral intensities from neighboring ROIs. Figure 15 and Figure 23 This demonstrates that even when unstructured neighbor information is used as input to a fully connected network, the model can still effectively integrate this information to improve inference performance on the central ROI. In the model without neighbors, the network predicts with relatively low confidence whether the underlying sample is normal or tumorous, as shown by the lighter blue and red areas and the white spots scattered throughout the inference graph. Figure 15 ).

[0071] As the number of neighbors increases, the prediction becomes more reliable, as evidenced by the sharpening of colors and the reduction of noisy white areas. The prediction also becomes more accurate, as... Figure 23 As shown at the dataset level, AUC-ROC increases with the number of neighbors, but there is diminishing return. In fact, if the input size is too large, the base region will contain varying tissue textures. Performance peaks when the input size consists of approximately 12 neighbors on each side, and degrades beyond that size. Figure 17 This process requires training a new network for each input size 2n+1, which can be expensive. Furthermore, the presence of many unstructured spectral measurements raises concerns about overfitting for deep learners.

[0072] Stacked Random Forest To determine whether a model receiving input from unstructured neighbors is actually learning complex relationships between neighboring spectra, a new class of models is constructed, configured to not have access to all spectral data from neighboring ROIs. Instead, this new class is only given the output of the fully connected network that performs best for each of the 2n+1 neighboring ROIs. The null hypothesis of the experiment is essentially that the model receiving all spectral data learns a “voting” process among the probabilities of tumors in neighboring ROIs, rather than learning more complex relationships between neighboring spectra. This hypothesis is supported if the model configured to only “vote” performs comparably to the model receiving all available spectral data.

[0073] A stacked architecture is developed where, firstly, a trained fully connected network makes individual predictions for the spectra of each of the 2n+1 input ROIs; this generates 2n+1 target probabilities. These probabilities are used as input to a random forest (RF) classifier, which is significantly regularized and aims to classify the centered ROI. Models with the same configuration are trained for each input size, and the input data is augmented by spatial inversion along the poles to support parity invariance in the predictions.

[0074] Findings: The stacked Random Forest (RF) model outperforms at least as well as a fully connected network that receives unstructured spectral intensities from neighboring ROIs, and its computational cost during training is significantly lower. The RF model acts as an ensemble voting process on the output of the benchmark neural network, and inference results show that, given neighboring inputs, the stacked RF model reduces noise and increases model confidence in the same way as a fully connected network; exemplary inference results are shown in... Figure 16 As shown in the figure. These performance gains, measured by AUC-ROC, can be... Figure 17 I saw it in the middle.

[0075] The feature importance of the RF model was also calculated to determine which of the 2n+1 inputs should be prioritized. The learned, cross-validated feature importances indicate that the RF model does indeed learn the concept of locality: the central ROI has the highest relative importance, which decreases with distance from the center. An additional increase in feature importance was observed at 8 ROIs away from the center, which are the points where the data acquisition window does not overlap with the predicted ROI from the center.

[0076] In the stacked RF model, the RF components do not have access to the spectral data due to their configuration. The fact that the stacked RF model performs as well as a fully connected network that receives all the spectral data suggests that the fully connected architecture does not characterize neighboring spectra, or is at least hindered by overfitting. The stacked RF model is the best-performing architecture that incorporates neighbor information and is used for the tumor size inference problem addressed later in this work.

[0077] Convolutional Neighbor Model Given any spatially structured data, such as a rod-shaped or square input, convolution is the natural choice for a translationally equivariant feature layer. An alternative convolutional neural network (CNN) model is developed, which predicts each label in its 2n+1 ROI inputs. Findings: Evaluated by AUC-ROC across all outputs, the CNN model outperforms a stacked RF model on different input shapes. Figure 17 As shown. Comparing a CNN that predicts all labels of the input with an RF model that predicts only the label of the central ROI is not necessarily meaningful. When comparing CNN and RF models based solely on their predictions of the central ROI, their performance is very similar, and for large input sizes n>16, CNN and RF models are equivalent. However, for smaller input sizes, CNN's prediction performance remains weaker. Despite the performance gap, CNN models have a significant advantage over stacked models. In principle, CNN models predict the labels of all ROIs in the stack, and this can be done for any input n≥3; the lower bound is the kernel size. Because the model can accept inputs of different shapes, the same trained model can be used for inference on the same input shape, whereas a different RF model must be trained for each value of n.

[0078] Tumor size inference - determining tumor size Using the classification model developed in this work, not only can certain ROIs be identified as cancerous, but estimates of tumor size can also be obtained, which has significant clinical implications. If data is collected in the bars of adjacent ROIs, in principle, an estimate of linear tumor size can be extracted from the classification results.

[0079] Reliable estimation of tumor size across all size categories should involve collaboration between models with different input sizes; when analyzing model performance on data separated by tumor size, models with spatially expanded inputs often perform poorly in detecting small tumors. Using an integrated set of models that accept varying amounts of spatial context can balance performance across tumor size categories. Each model with a specific input size will also have an associated optimal classification threshold that balances precision and recall. In size inference algorithms, a small range of thresholds around optimality is used to obtain predictions that emphasize precision and recall. Considering these points, an algorithm is sought that, given multiple adjacent bar inputs, provides an estimate of tumor size and a confidence bound around that estimate.

[0080] Experiments were conducted using an aggregated stacked RF model and a benchmark fully connected neural network. Considering input shapes of 2n+1 ROIs, where 0 ≤ n ≤ 16, an upper bound was chosen near the performance saturation point as measured by AUC-ROC. For a given data partition, the model and associated test set containing three or four organs were loaded. Within a given organ slice, tumors were identified, and local subarrays containing tumors were selected. A subarray can be viewed as a set of adjacent rod-like inputs, on which inference can be performed to generate a heatmap of model predictions. An example of an isolated subarray containing tumors and associated inference heatmaps is shown in [example of example]. Figure 18 As shown in the image.

[0081] Developing tumor size estimation algorithms; an overview of tumor size estimation algorithms is provided in [link to article]. Figure 19 The algorithm is shown below. The technical details of the algorithm are then described. The method shown in steps 1-6 obtains a set of size estimates, which are aggregated across different models and different classification thresholds. Kernel density estimation (KDE) is performed on the set of estimates to obtain the final result. Figure 18 A size estimation algorithm is performed on the subarray shown, yielding a density function of tumor size. Confidence intervals can be calculated from this density function using the log-likelihood method or Fisher information correlation metrics. This density is compared to the distribution of label widths in the ground-based real tumor subarray. The comparison between size inference and the labeled tumor widths is shown in... Figure 20 As shown in the image.

[0082] The density function is consistent with the actual tumor width, and the confidence interval correctly includes the data from the tumor. Figure 18 The actual tumor width of the labeled subarray. Besides the KDE process, the dispersion introduced in the estimation using multiple thresholds and multiple input shapes gives a sense of diffusion in the size distribution, thus providing the confidence of the model. Clinically, only a few rod-shaped inputs are available for analysis. In this case, the dispersion formed in size prediction through model ensemble and threshold range is even more important for giving accurate estimates and confidence in size estimation.

[0083] Inference from high-resolution MRI data This demonstrates the application of the model to a limited test dataset that more closely represents data acquired in clinical practice. The test dataset consists of in vitro prostate segments imaged using high-field (7T) MRI at a resolution of 60 μm × 60 μm × 90 μm, selected from data previously analyzed in "Magnetic Resonance Microscopy Enables Differentiation Between Normal Histological Features and Prostate Cancer in Resected Prostate," BJU International, 119(3): 414-423, March 2017 (Durand), published by Durand, Matthieu et al., March 2017 in BJU International. That is, each image slice is spaced 90 μm apart, an order of magnitude smaller than the approximately 3000 μm spacing of the histological slices used in the training dataset.

[0084] A continuous 12mm × 15mm × 2mm region located at the center of the organ was extracted from 3D MRI data. This region was further subdivided into 3D VOIs of size 2mm × 2mm × 2mm. The VOIs collapsed along the last two dimensions, producing a 2mm long 1-D signal from which a spectrum with wavelengths between 0.15mm and 0.95mm (in 0.1mm increments) was generated. This matches the wavelengths used to train the aforementioned neural network, except for 0.05mm (0.05mm is not feasible due to the 0.06mm resolution of the 3D MRI data along the analysis dimension). The neural network was retrained on these nine wavelengths extracted from the spectrum generated from the 114-slice training dataset. The network was then tested on the spectrum of the 3D MRI images. Since the 3D MRI data was not annotated, ground truth labels were unavailable; however, a predicted map of the probability that each VOI contains tumor tissue is shown here. Figure 21 For models trained on 1 to 5 neighbors present on either side of the VOI in the pole, prediction maps are repeatedly generated. In the absence of neighbors, the model predicts a high prevalence of cancer in the sample. As the number of neighbors increases, the region predicted as a tumor shrinks, but always remains in the right half of the sample. This is similar to what is observed when neighbor information is included in the original dataset, such as... Figure 15 As shown, as the number of neighbors increases, the probability value of normal organization sharpens, making the predicted probability more reliable.

[0085] discuss As disclosed in this paper, a submillimeter-resolution texture data acquisition method called MRH (Magnetic Resonance Histopathology) is introduced, and its potential in identifying prostate cancer is validated through computer simulations to estimate tumor size and severity. MRH balances image acquisition with the ability to achieve higher resolution texture measurements that more closely approximate changes in glandular structure. Distinctive features of tissue texture were observed to change with disease progression (Barrett). Here, a deep learning classifier is trained on simulated MRH data to demonstrate how texture measurements can be used to distinguish between healthy and cancerous tissue.

[0086] The MRH procedure can be added in conjunction with a standard multiparametric MRI prostate scan to obtain additional dimensions of information to determine cancer likelihood and severity. This enables more precise targeted biopsies, reducing patient discomfort and risk. During MRH acquisition, a single spatial frequency is measured in each excitation, and multiple spatial frequencies are measured in repeated excitations. While precise spatial coherence is not required for measuring texture, it remains important to minimize patient movement between acquisitions and to limit acquisition time. Therefore, the key challenge is to select the optimal number of spatial frequencies to measure in order to obtain sufficient diagnostic information in the shortest possible time. Here, a feature selection process and subsequent attribution analysis are performed, demonstrating that 10 wavelengths uniformly sampled across a short length scale (0.05 mm) and a long length scale (1 mm) optimally capture texture variations.

[0087] While the model performs well when trained on spatially isolated and unrelated samples, a method incorporating spatial context is presented, which improves performance and denoising results because tumors are largely contiguous. Models are created using samples and their n nearest neighbors for prediction. Findings show that increasing the number of neighbors leads to more accurate and confident predictions, up to a maximal prediction level where the input is based on too much spatial variation. These models are also able to estimate the distribution of predicted tumor size. In practice, the resolution of this distribution is set by the density and width of tissue rods excited during a series of MR measurements. Given the good agreement between the predicted tumor size distribution and the actual tumor size distribution (…),… Figure 24 Even with sparse sampling, meaningful estimates of tumor size are expected. When combined with MR imaging of the lesion, MRH can validate and enhance tumor characteristics and set guidelines for further biopsies. In this way, several targeted MRH acquisitions used to generate estimates of tumor size and severity can serve as a non-invasive analogy for biopsies.

[0088] The higher resolution of MRH also enables more detailed grading of lesions, similar to the Gleason scoring method used in histopathology. Currently, the PI-RADS scoring system for MRI involves scoring each lesion on a 1-5 scale (indicating the clinically significant likelihood of cancer (Turkbey)). MRH trained on histological data graded by pathologists can be used not only to determine the likelihood of cancer but also to grade the severity of cancer. Future work will allow for training not only on data containing tumors labeled with a series of levels but also on other non-cancerous but atypical cases (e.g., prostatic intraepithelial neoplasia).

[0089] The study focused on simulated MRH measurements in the peripheral zone, where approximately 70% of tumors originate. However, over 20% of tumors originate in the transition zone, making it crucial to extend the analysis beyond the PZ. Identifying tumors in the transition zone using mpMRI remains challenging because benign hyperplasia and stroma in this region can be confused with cancer, leading to false positives (see “Benign Lesions Mimicking Prostate Cancer: MR Imaging Features with Histopathological Relevance”, Kitzing, Yu Xuan et al., January 2016, Radiology, 36(1): 162-175, Publisher: Radiological Society of North America (Kitzing)). Improved MRH resolution allows for further training on tissues located outside the PZ and potentially helps mpMRI resolve the confusion between benign and cancerous tissues. In the published experiments, it was found that model accuracy and overall performance deteriorated when trained on whole-organ data: large areas of dense stroma outside the PZ were incorrectly labeled as cancer. Most likely, different models should be used for the PZ and non-PZ portions of an organ to avoid this type of confounding. A separate auxiliary classifier could be developed to determine whether a given MRH measurement is within or outside the PZ, although this information may already be known in advance in a clinical setting. Since the focus is on developing MRH analyses solely for tumor detection, classification issues outside the PZ are considered beyond the scope of this work.

[0090] method Computer simulation analysis of prostate images MRH measurements were simulated from “ground-real” histological images, which offer a much higher resolution than that achievable with in vivo 2D MRI (and in some cases, in vitro 2D MRI). These sections were obtained from a dataset described by Singanamalli et al., consisting of 114 digitized H&E-stained sections obtained from 16 resected prostates. Each organ was divided into 3–4 mm sections, and each section was then subdivided into four quadrants. Each quadrant was imaged at a resolution of 0.5 μm, and these quadrants were then stitched together using HistoStitcher software to form a complete section image (see Chappelow, Jonathan et al., “HistoStitcher”, 2011, in *Computerized Medical Imaging and Graphics*). : An interactive program for the precise and rapid reconstruction of digitally complete histological segments from tissue fragments, Computerized Medical Imaging and Graphics, 35(7-8): 557-567, 2011 (Chappelow). The slides were annotated by a pathologist, where areas were labeled CAP (prostate cancer), PIN (prostate intraepithelial neoplasia), or atypical ( Figure 10 A). In the analysis below, only cancer annotations are used to determine positive labels ( Figure 10 B).

[0091] To improve model performance, the dataset was limited to the peripheral zone (PZ) of each prostate. Most tumors originate in the PZ, with an incidence of approximately 70% within it (see McNeal, John E. et al., “Zonal distribution of prostate adenomatous carcinoma: correlation with histological pattern and direction of spread,” American Journal of Surgical Pathology, 12(12):897, December 1988 (McNeal)), and most tumors in the dataset are contained within the PZ. Furthermore, excluding other regions allowed us to avoid dense stroma, which can be easily confused with cancer; dense stroma is typically located in known areas that can be excluded from clinical MRH data collection, thus avoiding confusion in the diagnostic process. The PZ of each image slice was manually segmented (see Fine, Samson W. and Reuter, Victor E. “Revisiting the Anatomy of the Prostate: Implications of Prostate Biopsy and the Band-like Origin of Prostate Cancer”, Histopathology, 60(1): 142-152, 2012 (Fine)), and only this segmented region was used to train the model. Figure 10 C).

[0092] Simulated MRH MRH acquires high-resolution 1D signals; however, 3D volumes must be excited to measure this signal because it is impossible to excite 1D lines. First, a slice-selective gradient and a 90-degree pulse are applied to excite a tissue block. Then, a gradient along an orthogonal direction and a 180-degree refocusing pulse are applied to excite a tissue block orthogonal to the first tissue block. This defines a rod formed by the intersection of the two tissue blocks. The volume of interest (VOI) along this excited rod can then be spatially encoded in the same TR. In each VOI, the k-space encoding direction (called the analysis direction) is orthogonal to the long axis of the rod. The result is that the 3D volume is “smeared” in two dimensions orthogonal to the analysis direction to effectively provide a 1D signal. This process of exciting the rod and k-space encoding the VOI is repeated multiple times to obtain a signal from a large amount of tissue. In practice, the VOI will be approximately 4 mm long along the analysis direction and 2.5 mm × 2.5 mm along the other two dimensions, with the analysis direction aligned with the head-to-foot axis (parallel to the urethra).

[0093] To simulate MRH spectral data, 2D regions of interest (ROIs) are selected from ground-based, real-world histological images of the organ. However, the dataset consists of high-resolution images of histological sections; in this exemplary case, the high-resolution images of the histological sections are images of the prostate taken perpendicular to the urethra. When simulating MRH data, any analytical orientation chosen for these ROIs must lie in the plane of the image and therefore be orthogonal to the clinical analytical orientation. It is assumed that the prostate texture is substantially isotropic and that the simulated data can be analyzed in a manner similar to clinical data.

[0094] Each image is converted to grayscale and divided into a 2mm × 2mm region of interest (ROI) corresponding to 1000 × 1000 pixels. Figure 10 D). Each ROI is collapsed along a direction labeled "lateral direction" to give a 1D signal in location space along the analysis direction. In the disclosure of this paper, the lateral direction is chosen to be parallel to the sagittal direction, and the analysis direction is chosen to be placed along the coronal direction.

[0095] Since each complete image of a prostate slice is approximately (2.104) × (2.104) pixels, a grid of approximately 20 × 20 pixels can be provided, resulting in a total of 400 ROIs. However, several of these ROIs will be located outside the prostate zone (PZ) or outside the boundaries of the prostate, and the amount of usable data will be relatively small. Furthermore, the model's ability to classify a given sample can depend on the precise localization of that sample within the tumor. To expand the dataset, the images were divided into a grid of overlapping ROIs with a stride of 0.25 mm. It should also be noted that in clinical practice, spatial encoding of stimulated rods can result in encoded VOIs that overlap with each other in space.

[0096] During the MRH procedure, an MR scanner signal measures the similarity between the texture of the excited tissue and an encoded k-value or wavelength. Repeated k-space encoded measurements then construct a spectrum with intensity across a wavelength range. This spectrum is simulated on histological sections by performing a (discrete) Fourier transform on a 1D-averaged region of interest (ROI) at a specific set of k-values. The frequency-spatial spectrum generated by this process represents the clinical measurements performed during the MRH procedure and is used to train a deep learning model.

[0097] According to the Nyquist theorem, simulated MRH spectra can be calculated at any wavelength between 0.004 mm and 1 mm. However, in practice, MRH can only achieve wavelengths as small as approximately 0.05 mm. Therefore, spectra are first calculated at 39 wavelengths between 0.05 mm and 1 mm (in increments of 0.025 mm)—a resolution far exceeding clinically achievable resolution. A subsequent section describes the feature selection process, which determines which subset of the 10 wavelengths is most informative for classification, and whether using fewer wavelengths significantly hinders model performance compared to more detailed spectra, thus informing the parameters of the MRH k-space encoding.

[0098] Neural Network Classification Deep neural networks are trained on simulated MRH spectra. This problem is one of the binary classification problems given the intensity of the input wavelength of a single ROI. In the experiments, a wide variety of models and architectures are considered. Each model is tested on the same dataset using the same cross-validation split, and performance is compared based on AUC-ROC scores. Even after hyperparameter optimization, the performance of classic ensemble learners such as random forests and gradient boosting trees is still inferior to that of deep learning models. Transformer (attention-based) models are considered, adapting the architecture published in "Swin Transformer: A Hierarchical Visual Transformer Using Offset Windows" by Liu, Ze et al., 2021 (Liu) to a multi-head self-attention module; these models are comparable to simpler, shallower fully connected neural networks, but with slightly lower performance. One-dimensional feature-encoding convolutions along the spectrum are also considered, the idea being to effectively feature local correlations between similar wavelengths for prediction, but these models also perform poorly. Fully connected networks are the focus, classifying individual ROIs by encoding the intensity of the input k-space.

[0099] Feature selection In computer simulation analysis, spectra can be obtained from each ROI sampled at a greater number of wavelengths (k-values) compared to the number of wavelengths typically achievable in clinical k-space encoding procedures. While simulated spectra are calculated at approximately 40 wavelengths from 0.05 mm to 1 mm (in increments of 0.025 mm), clinical MRH procedures can involve approximately 10 k-space encodings to limit scan length and acquire more samples from a wider tissue region in the same amount of time. The goal is to find a relatively small subset of k-values ​​that maintains diagnostic capability while reducing acquisition time. To investigate this trade-off, a feature selection routine is performed to determine which wavelengths are most informative for binary classification.

[0100] This provides a performance comparison of neural network classifiers, each trained on five different sets of inputs. Each set of inputs contains ten features: a) spectral intensity at short wavelengths (between 0.05 mm and 0.275 mm, in 0.025 mm increments); b) spectral intensity at medium-short wavelengths (between 0.3 mm and 0.525 mm, in 0.025 mm increments); c) spectral intensity at medium-long wavelengths (between 0.55 mm and 0.775 mm, in 0.025 mm increments); d) spectral intensity at long wavelengths (between 0.775 mm and 1 mm, in 0.025 mm increments); and e) spectral intensity across the entire wavelength range (between 0.05 mm and 0.95 mm, in 0.01 mm increments). In each case, the spectral intensities in the training and test datasets are globally normalized to a zero mean and unit standard deviation, suitable for a zero-centered activation function in the neural network. These results, trained on a subset of the input features, will also be compared with a classifier trained on all 40 wavelengths as a (clinically impractical) reference and upper limit for predictive performance.

[0101] As a feature selection experiment, a joint parameter-hyperparameter scan was performed to determine the optimal architecture and input features. For each wavelength set, multiple fully connected network architectures were tested. Each architecture consisted of an input layer, at least one hidden fully connected layer, and an output layer. The hidden fully connected layer had GeLU activation, followed by batch normalization and discarding, and the output layer had two output nodes that, after a softmax operation, gave the probability that the sample was normal or tumor-like. If multiple hidden layers existed, the last hidden layer contained half the number of nodes of the previous layer. For each feature set, architectures generated by all combinations of the following parameters were tested: {1, 2, 3} hidden layers, dimensions {16, 32, 64}, and discard probabilities {0, 0.1, 0.2, 0.5}. Details of the model configuration and training process are included in Appendix A.

[0102] For each configuration (feature set and network architecture), a 5-fold cross-validation process is performed. First, the data is categorized by the organ from which it originated (a total of 16 organs). All data from 3 to 4 organs—approximately 20% of the total dataset—is reserved for testing, and the model is trained on the remaining data; this organ-based categorization is crucial to avoid leakage between the training and test sets. This process is repeated a total of 5 folds to ensure all data is seen in the training set.

[0103] Using 10 evenly spaced wavelengths as input features, the best-performing architecture for single-spectral classification is a fully connected network where layers (64, 64, 64, 32) represent two output log odds, followed by batch normalization and discarding (p=0.05) after each hidden layer. This architecture achieves an AUC-ROC of 0.780 with cross-validation. At the optimal classification threshold determined by the Fl score, the (cross-validated) model achieves 82% precision, 44% accuracy, and 52% recall on the test set. This fully connected model, trained on 10 wavelengths, forms the backbone of most inference analytics.

[0104] Includes neighbor information The MRH measurement process naturally involves exciting tissue rods, from which k-space encoded measurements can be performed in localized regions of the organ, rather than strictly within a single VOI. Cancer is localized and often a continuous object; theoretically, cancer should be easier to identify if its local context within the organ is known. This provides a basis for including multiple adjacent spectra in the input of the predictive model.

[0105] An input with n neighbors contains (2n+1) spectra. Due to the use of overlapping windows for simulated MRH measurements, the input with n neighbors covers a true spatial extent of 2 + 0.5n mm within the organ. Neighbors are symmetrically included around the central ROI, and classification labels are extracted from the central ROI.

[0106] Consider three architectures that incorporate spectral information from neighboring ROIs: fully connected networks, stacked random forest models, and convolutional neighbor models. Details of the configuration and training process for these models are then provided. Regardless of modality, as evaluated by testing AUC-ROC, recall, and precision, incorporating neighbor information cleanses predictions and improves model performance. This supports the assumption that local context is important for the accurate identification of tumors. Figure 17 The diagram illustrates how the AUC of each model increases, then saturates, and subsequently decreases with the number of neighbors of the input. Above, in Figure 23 The data includes the ROC curves of neural networks trained using neighbor inputs, where the performance improvement is more significant.

[0107] However, some reasoning suggests that models with large inputs containing many regions of interest (ROIs) may become less sensitive to smaller tumors. The reason is simple: when the input size expands beyond the spatial extent of the tumor, the “normal” spectra far from the central ROI can negatively influence the model and produce negative predictions. Without any location encoding, how the model determines which spectra correspond to the labels depends on the training process and can lead to incorrectly prioritizing spectra relative to the center of the region under consideration. While this isn't a problem for large, extended tumors and might even improve model recall, the lack of explicit location encoding or the concept of hard-wired localities can cause models to make confusing predictions for large inputs containing small, highly localized cancer signals.

[0108] Performance related to tumor size category In the histological dataset (segmented by peripheral zones) used in this work, there are multiple annotated tumor sizes, ranging from just a few ROIs to tumors covering macroscopic portions of the organ sections considered. Each tumor is clustered, and the size of each tumor is calculated, defined as the number of connected neighboring ROIs constituting the tumor. Figure 24 As shown, the size distribution is severely skewed, with many small tumors with less than 100 ROIs and a long tail of large tumors with 1000 or more ROIs. Essentially, although large tumors are a minority in the size distribution, they constitute a large portion of the dataset.

[0109] The dataset was split between small tumors (including ROIs with <500 connections) and large tumors. Non-tumor data was split evenly and randomly, resulting in two datasets of approximately equal size for each tumor category. To examine the impact of tumor size on inference results, the stacked RF models trained on each of the split datasets were re-evaluated and the metrics were compared. Figure 25 The results show that including many neighboring ROIs improves performance on large tumors but degrades performance on small tumors. This suggests that in practice, while neighbor information can often clear predictions, a balance must be struck between overall accuracy and sensitivity to small tumors.

[0110] Tumor size inference Using data collected within the ROI poles, it is theoretically possible to predict the spatial extent of an organ. While most models incorporating neighbor information aim to predict the central VOI with higher confidence, by combining these models, an estimate of tumor size can be determined by predicting each ROI along the pole. Here, tumor size means an estimate of the linear tumor scale along the pole to which the measurement is taken. If multiple ROI poles are excited, these linear size predictions can be combined to give an estimate of the overall area of ​​the cancer region. This process is largely independent of the details of the underlying prediction model.

[0111] The simplest way to estimate tumor size is to choose the input shape, perform inference, and threshold the target probability. The estimated size in each row is then simply the number of positive predictions in that row. However, this process does not introduce any dispersion into the estimation and does not include the strength of model variations given different input shapes. Size predictions are integrated in two ways: by combining estimates from models with different input shapes, and by modulating the classification threshold.

[0112] If possible, for each row of the subarray, construct an input shape of size 2n+1, where n≤16, or construct the input shape as large as the subarray allows. Aggregate each submodel in the stacked RF model using an input of appropriate size to generate an inference heatmap. Then, threshold the inference heatmap for a classification threshold range, designing the classification threshold to generate confidence intervals for size predictions.

[0113] While various methods exist for determining the optimal classification threshold, this paper focuses on maximizing the harmonic mean of Fl score, precision, and recall. The Fl(t) score as a function of the classification threshold t is considered at some optimal threshold t. * Maximize at ∈6[0,1]. Due to the classic bias-variance tradeoff, the curve Fl(t) is usually convex in the neighborhood of this point.

[0114] By considering the second derivative of Fl(t), it is possible to naturally find the optimal t. * The surrounding generation threshold range; this is limited to t. * The natural scale of the curvature of the surrounding Fl curve. This scale will be smaller for models highly sensitive to the classification threshold, and larger for models less sensitive to the threshold. Using this scale, the threshold interval is defined around optimality as [t]. * -L Fl / 2,t * +L Fl / 2]. These perturbation thresholds will emphasize inference recall when it is below the optimal threshold, and inference precision when it is above the optimal threshold. By doing so, each aspect of the precision-recall tradeoff can be utilized when aggregating inference results.

[0115] Size estimation algorithm Therefore, the algorithm used for size inference can be summarized as follows: Isolate the tumor subarrays and divide each row into rod-shaped inputs of shape 2n+1, where 0≤n≤16. Perform inference for each input shape using a suitable module of the stacked RF model, generating an inference heatmap for each value of n. For each input size, within the interval [t] defined by the Fl curve... * -L Fl / 2,t * +L Fl A fixed number of thresholds Nt are generated in [ / 2]. For each index i, 1≤i≤Nt, a predicted (thresholded) label array is generated across all rows and the input shape, and averaged over the input shape. In each row, the size estimate S is... i Calculated as the sum of the values ​​of the (averaged) predicted tag array. This size estimate is converted from ROI to physical millimeters.

[0116] This process is performed on all Nt thresholds, yielding an estimate of the Nt size for each row of the input array. A Gaussian kernel density estimate is then performed on this estimate to obtain a density function representing the physical tumor size in millimeters. This density function can be used to compute statistics on the predicted tumor size in the subarray.

[0117] Model Configuration Fully connected neural networks The fully connected neural network architecture consists of a variable number of hidden layers and an encoded output block with two output channels. Probabilities are generated by applying softmax to the model output. Each hidden layer is followed by batch normalization (see Ioffe, Sergey and Szegedy, Christian, “Batch Normalization: Accelerating Deep Network Training by Reducing Intrinsic Covariate Shifts,” arXiv Preprint Platform, arXiv: 1502.03167, February 2015 (Ioffe)) and discard blocks (as described in Srivastava, Nitish et al., “Discard: A Simple Way to Prevent Overfitting in Neural Networks,” Journal of Machine Learning Research, 15(56): 1929–1958, 2014 (Srivastava)), which has been found to significantly improve training stability. The Adam optimizer (as described in "Adam: A Stochastic Approach to Optimization" by Kingma, Diederik P., and Ba, Jimmy, 2014, arXiv preprint platform, arXiv: 1412.6980, 2014 (Kingma)) is used with a learning rate of 0.0001 (with cosine annealing), weight decay, and class-weighted binary cross-entropy loss. No improvement was observed when focus or dice loss was used during training. Experiments using Gaussian white noise data augmentation were also conducted, but this generally degrades model performance.

[0118] When neighbor information is included in a fully connected network, data is first generated in a structured array of 2n+1 10-wavelength spectra for each input with n neighbors to the central ROI. The prediction target is the central VOI label. The data is flattened to form 10·(2n+1) input features. No structural or location encoding is given for these inputs.

[0119] Stacked Random Forest Model The stacked RF model was trained on the same training / test split as the best-performing fully connected network used for label prediction from the spectrum of a single ROI. On each data split, the trained benchmark neural network performed inference on the data and generated a set of (2n+1) probabilities for each pole of the ROI input and its associated center label. Each pole was augmented by its spatial inversion, which did indeed improve model performance.

[0120] The random forest classifiers were trained using 200 estimators, a light α regularization at 1e-4, a maximum depth of 10, and a square root decreasing number of features considered at each node. These classifiers were evaluated in the same manner as neural network models. Random forests tend to have lower classification threshold sensitivity compared to fully connected networks that accept neighbor inputs.

[0121] Convolutional Neighbor Model The convolutional model architecture consists of three one-dimensional convolutional layers with residual connections, followed by multiple self-attention modules, and finally two fully connected classification modules. A large kernel size with 7 ROIs was found to perform optimally. The model's objective is set to predict all labels for 2n+1 ROI inputs, where n is the number of neighboring ROIs. This objective differs from stacked RF models that only predict the center ROI. After hyperparameter search, the CNN was trained on an input with 16 neighbors (33 ROIs) using three convolutional layers (32, 64, 64), three (single-head) attention modules, and a final fully connected output block (16, 8, 2); this is based on a 1cm real-world spatial extent within an organ as input. ReLU activation was used, followed by batch normalization after convolution, and discarding at p=0.3. The training process is essentially the same as that used for fully connected networks.

[0122] Although this paper does not disclose the model used for size inference, it is included for the sake of completeness and to demonstrate the traditional image segmentation architecture (there is no obstacle in principle to using any architecture for the size inference process). Because datasets have many channels and are often only available in small subarrays compared to traditional image data, techniques that perform well in binary or semantic segmentation of images may not be suitable for the current task, where large-scale features are often inaccessible.

[0123] Various implementations have now been described in detail in accordance with the requirements of patent law, and those skilled in the art will recognize modifications and substitutions to the specific embodiments disclosed herein. These modifications fall within the scope and intent of the invention as defined in the appended claims.

Claims

1. A method for training a neural network to analyze MRH data, the method comprising: Multiple two-dimensional regions of interest (ROIs) are selected in each of the multiple histological images (2, 4, 6, 8, 10, 12, 14) or electronic images of the cancerous organ (18), and each image is annotated using normal region indicators and tumor region indicators (16, 17); Each of the plurality of ROIs is collapsed along a lateral direction associated with the lateral width of the ROI to generate a one-dimensional signal along an analysis direction associated with the analysis width. The one-dimensional signal in each ROI is convolved with a set of complex sine functions so that the simulated MRH spectrum is used as a function of wavelength for calculation. Simulated spectra are calculated from each of the plurality of ROIs at multiple k values ​​to generate multiple training datasets corresponding to each of the plurality of images; Choose one or more neighbors of each ROI (30) from the selected multiple neighboring ROIs (32, 34, 36, 38) and add the spectra of those neighbors to the training dataset of that ROI; In the neural network (22), simulated spectra are received from each ROI to generate a two-element probability vector (24) indicating the probability that the input spectrum of the ROI is from normal tissue and the probability that the input spectrum of the ROI is from tumor, the sum of which is 1. The network is trained by comparing the resulting probability vector with the normal region indication and tumor region indication annotated for the ROI.

2. The method according to claim 1, wherein, The selected multiple neighbors include a set of 4 nearest neighbors (32), 8 nearest neighbors (34), 24 nearest neighbors (36), or 16 non-adjacent neighbors (38).

3. The method according to claim 1, wherein, The step of selecting multiple two-dimensional ROIs includes selecting bars (150) of fixed width in a ground-based image as a simulation of the volume of interest (VOI) in the actual MRH data, the bars comprising two-dimensional columns having an applied width in a third dimension, wherein the width of the selected bars corresponds to the analysis width, and the bars are laterally divided into ROIs.

4. The method according to claim 1, wherein, The multiple images include complete organ histological sections (2, 4, 6, 8, 10, 12, 14) that have been imaged and annotated by pathologists using normal and tumor areas (16, 17).

5. The method according to claim 1, wherein, The plurality of images include pseudo-complete organ digitized H&E stained histological sections with annotations indicating tumors, prostatic intraepithelial neoplasia (PIN), and atypical adenomas. The tumor annotations are converted into image masks (108) used to label the regions of interest (ROIs) as normal (118a) or tumor (118b), and the complete organ sections are segmented into peripheral zones (110), which are defined as the ROIs.

6. The method according to claim 1, wherein, The plurality of images include a plurality of MRI image slices (52) which are derived from an MRI dataset and annotated by a pathologist using indications of tumor or normal tissue to create a dataset of ground-based true labels.

7. The method according to claim 1, wherein, Compared to normal samples, ROIs containing tumor samples are assigned higher class weights, whereby the class weights are defined as the reciprocal of the prevalence of tumor samples in the training dataset.

8. The method according to claim 1, further comprising: Shuffle the training dataset and divide the shuffled dataset into two parts; In the neural network (22), simulated spectra are received from each ROI in the first part of the part and the selected neighbors of the ROI to generate a two-element probability vector (24) indicating the probability that the input spectrum of the ROI is from normal tissue and the probability that the input spectrum of the ROI is from tumor, the sum of these values ​​being 1. The network is trained by comparing the resulting probability vector with the normal region indication and tumor region indication annotated for the ROI. In the neural network (22), simulated spectra are received from each ROI in the second part of the portion and the selected neighbors of the ROI to generate a two-element probability vector indicating, according to the first element of the two elements, that the input spectrum of the ROI comes from normal tissue, and according to the second element of the two elements, that the input spectrum of the ROI comes from tumor, the sum of these values ​​being 1, to verify the training of the neural network on the first part by comparing the resulting probability vector with the normal region indication and tumor region indication annotated for the ROI.

9. A method for training a neural network to analyze MRH data and using the neural network to analyze the MRH data, the method comprising: Multiple two-dimensional regions of interest (ROIs) are selected in each of the multiple histological images (2, 4, 6, 8, 10, 12, 14) or electronic images of the cancerous organ, and each image is annotated using normal region indicators and tumor region indicators (16, 17). Each of the plurality of ROIs is collapsed along a lateral direction associated with the lateral width of the ROI to generate a one-dimensional signal along an analysis direction associated with the analysis width. The one-dimensional signal in each ROI is convolved with a set of complex sine functions so that the simulated MRH spectrum (20) is calculated as a function of wavelength; Simulated spectra are calculated from each of the plurality of ROIs at multiple k values, generating multiple datasets corresponding to each of the plurality of images; Choose one or more neighbors for each ROI (30) from the set of 4 nearest neighbors (32), 8 nearest neighbors (34), 24 nearest neighbors (36), or 16 non-adjacent neighbors (38); In the neural network (22), simulated spectra are received from each ROI to generate a two-element probability vector indicating the probability that the input spectrum of the ROI is from normal tissue and the probability that the input spectrum of the ROI is from tumor, the sum of which is 1. The network is trained by comparing the resulting probability vector with the normal region indication and tumor region indication annotated for the ROI. Obtain MRH data that produce actual spectra on multiple VOIs in 3D organs; The VOI is collapsed along two dimensions to generate a 1-D signal, the spectrum at the wavelength of the 1-D signal being matched with the k-value of the simulated spectrum, and In the neural network, 1-D signals are received from the actual spectra of each VOI and its selected neighbors to generate a two-element probability vector indicating the probability that the spectrum of the VOI is from normal tissue and the probability that the spectrum of the VOI is from a tumor, the sum of which is 1.

10. A method for performing magnetic resonance histopathology (MRH) measurements by training a neural network classifier using annotated histological sections, the method comprising: Create pseudo-complete organ digital H&E stained histological sections, which have annotations indicating tumors, prostatic intraepithelial neoplasia (PIN), and atypical adenomas; The tumor annotations are converted into image masks, which are used to label regions of interest (ROIs) as normal or tumor (104, 102). The complete organ slice is divided into peripheral zones (110). The outer peripheral zone is divided into multiple ROIs; Identify exemplary normal ROIs (118a) and ROIs located within tumors (118b); Each of the plurality of ROIs is collapsed in one dimension to generate a 1-D position signal; Fourier transform is performed on the 1D position signals of the multiple ROIs to generate simulated MRH spectra; as well as Comparison of spectra from normal ROI and tumor ROI reveals significant differences in texture; The simulated MRH spectrum is provided to the neural network classifier (122), the simulated MRH spectrum having intensity at a plurality of selected wavelengths; The neural network classifier outputs a vector, which indicates the probability that the corresponding sample is normal or has a tumor. Obtain MRH data that generate actual MRH spectra at multiple VOIs in 3D organs; and In the neural network, the actual spectrum is received from each VOI to generate a two-element probability vector, which indicates the probability that the spectrum of the VOI comes from normal tissue and the probability that the spectrum of the VOI comes from a tumor, and these values ​​sum to 1.

11. The method according to claim 10, wherein, The selected multiple wavelengths are sampled uniformly across the simulated MRH spectrum, enabling differentiation between normal and tumor tissues. The intensity at selected wavelengths extracted from the simulated MRH spectrum is used as input to the classifier.

12. The method of claim 10, further comprising: In the step of collapsing each ROI, adjacent ROI inputs are used to incorporate the effects of spatial information, wherein the adjacent ROI inputs are selected from either a bar input or a square input. The bar inputs include: ROIs with one nearest neighbor (40) on each side, giving an input size of (3,1); ROIs with two neighbors (42) on each side, giving an input size of (5,1); ROIs with three neighbors (44) on each side, giving an input size of (7,1); or for the square inputs: one of the ROIs has one layer of neighbors (46) along all directions, forming an input of size (3,3); ROIs have two layers of neighbors (48), forming an input size of (5,5); ROIs have three layers of neighbors (50), forming an input size of (7,7).

Images