Methods and systems for evaluation and treatment of cancer based on wrinkling of the nuclear lamina
An AI-driven deep learning model analyzes nuclear wrinkling patterns in lamin-stained images to enhance cancer detection and treatment by quantifying extreme laminar wrinkling, addressing the limitations of conventional methods and improving diagnostic accuracy.
Patent Information
- Application Number
- PCT/US2025/040800
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-08-05
- Filing Date
- 2025-08-05
- Publication Date
- 2026-02-12
AI Technical Summary
Existing methods fail to effectively utilize nuclear wrinkling as a diagnostic marker for cancer detection and treatment, particularly in the context of patient tissues, despite its potential significance in nuclear morphological alterations.
An artificial intelligence-driven approach using deep learning models, such as ResNet50, analyzes lamin-stained images to classify nuclear wrinkling patterns, distinguishing between cancerous and non-cancerous cells based on the degree of nuclear lamina wrinkling, employing data augmentation techniques to enhance model robustness and address data imbalance.
The method provides accurate and prompt cancer detection and prognosis by quantifying extreme laminar wrinkling as a morphological marker, enhancing clinical decision-making through improved image analysis beyond conventional staining methods.
Smart Images

Figure US2025040800_12022026_PF_FP_ABST
Abstract
Description
METHODS AND SYSTEMS FOR EVALUATION AND TREATMENT OF CANCER BASED ON WRINKLING OF THE NUCLEAR LAMINAGovernment Support
[0001] This invention was made with government support under grant number U01 CA225566 awarded by National Institutes of Health and Established Investigator Award RR200 from Cancer Prevention & Research Institute of Texas. The government has certain rights in the invention.Technical Field
[0002] The disclosure relates to ar tificial intelligence-driven methods and systems for detection and treatment of cancer based on extreme nuclear wrinkling.Background
[0003] Nuclear morphological alterations, or nuclear atypia, have been associated with human cancers for over 150 years. Nuclear atypia is critical in the diagnosis of breast carcinoma, breast ductal carcinoma in situ (DCIS), thyroid carcinoma, skin carcinoma, ovarian carcinoma, and precancerous dysplasia of the larynx. Nuclear- atypia includes features such as larger nuclear size, altered heterochromatin, more prominent nucleoli, and nuclear shape alterations. Variations in nuclear shapes may result from changes in the mechanical properties of the nucleus in cancer. Indeed, cancer nuclei tend to be softer than normal nuclei, due to alterations in nuclear lamin levels, particularly a decrease in lamin A / C levels. Lamin A / C is a protein found in the nuclear lamina, a stiff two-dimensional sheath that underlies the nuclear envelope. The resistance of the nuclear lamina to extension and the resistance of chromatin to flow underlie the stability of the nucleus to changes in shape. Thus, the loss of lamin A / C eliminates the resistance to areal extension, resulting in irregular nuclear shapes. However, the nuclear lamina resists extensional deformation only in extreme nuclear- shapes where the lamina is smooth, such as the flattened nuclei in tissue culture. Nuclear lamina in non-extreme shapes, such as those in 3D culture or in elongated or rounded nuclei, tend to form folds and wrinkles. However, the roles of these wrinkles have not been evaluated in the context of patient tissues.Summary
[0004] Provided here are systems and methods to address these shortcomings of the art and provide other additional or alternative advantages. The disclosure herein provides artificial intelligence-driven methods and systems for detection and treatment of cancer based on nuclear wrinkling. Embodiments include a deep learning-based method for detecting a cancer or providing prognosis or evaluating treatment for a cancer as shown and described herein. Embodiments include systems configured for detecting a cancer or providing prognosis or evaluating treatment for a cancer as shown and described herein. Extreme laminar wrinkling is a quantifiable, diagnostically valid morphological marker of certain human cancers. In certain embodiments, the human cancers are specifically carcinomas in adult patients.
[0005] An embodiment of the computer-implemented method includes the steps of providing a lamin-stained image of cells in a sample from a subject to a computer-implemented deep learning model; analyzing degreeof wrinkling of nuclear laminar of the cells with the deep learning model to classify the cells as cancerous or non-cancerous; and communicating the classification to a user interface. A greater proportion of cells in the sample with extreme wrinkling of the nuclear lamina results in classification of the sample as cancerous. The extreme wrinkling includes high-frequency wrinkles, inner wrinkles, or both of the nuclear lamina. In certain embodiments, the degree of wrinkling nuclear laminar is analyzed as proportion of the cells with nuclei in each of the following classes: out-of-focus / wrongly-cropped nuclei, smooth nuclei, nuclei with low-frequency contour waviness, nuclei with high-frequency contour waviness, and nuclei with internal wrinkles. The cancerous sample can be of any origin. In certain embodiments, the cancer is any of head and neck cancer, skin cancer, breast cancer, and thyroid cancer. Embodiments can include fluorescence microscopy or immunohistochemistry.
[0006] Another embodiment of the disclosure is directed to a system configured for detecting a cancer. The system may include a processor and a non-transitory machine readable storage to store instructions. The instructions, when executed by the processor, cause the processor to, in response to reception of a one or more lamin-stained cell images of cells of a subject, analyze the one or more lamin-stained cell images via a trained model to generate a classification to indicate a type of nuclear wrinkle corresponding to the cells of the subject. The instructions, when executed by the processor, cause the processor to, based on the classification and on subject information, determine if the cells of the subject are cancerous and a subsequent treatment.
[0007] In an embodiment for the nuclear morphological analysis, the deep learning model includes the following. Images of nuclei identified by the Cellpose algorithm that passed quality filters are cropped as individual images of the nuclei. The cropped images are segregated into one of the five classes and serve as inputs to train a multi-class classifier. A transfer learning approach is implemented using a pre-trained ResNet50 model. This model features 50 layers, including residual blocks with skip connections to preserve gradient flow and bottleneck layers that reduce computational burden as they maintain processing depth. ResNet50 uses global average pooling to reduce overfitting and decrease the total number of parameters, enhancing its efficiency.
[0008] In certain embodiments, the training pipeline includes a series of data augmentation techniques to enhance model robustness and address data imbalance due to varied representations of nuclear types across samples. Transformations can include one or more of resizing to 224 x 224 pixels, random horizontal and vertical flips, rotations up to 15 degrees, color jittering for brightness, contrast, and saturation adjustments, random affine transformations, and center cropping to maintain focus on the nucleus. Normalization is performed with specific mean and standard deviation values typical for pre-trained networks on ImagcNct. In certain embodiments, to counter data imbalance, these augmentation techniques are combined with resampling methods to equalize class presence in the training set. Results are quantified using a corrected normalized count, considering individual class accuracies, providing a robust assessment of nuclear morphological variations.Brief Description of the Drawings
[0009] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
[0010] Embodiments will be readily understood by the following detailed description in conjunction with the accompanying drawings. Embodiments are illustrated by way of example and not by way of limitation in the accompanying drawings.
[0011] FIGs. 1A - IF depict nuclei in diverse control and cancer tissues with a wrinkled lamina. FIG. 1A presents formalin-fixed paraffin-embedded (FFPE) Head and Neck (HN) tissue stained for lamin Bl (yellow or gray) and pan-cytokeratin (magenta), imaged at 20x (left) and 60x (middle). The right columns show zoomed regions from the 60x image, tissue. FIG. IB presents the same of Skin (SK) tissue and cancer adjacent tissue (CAT). FIG. 1C presents the same of Ovarian (OV) tissue. FIG. ID presents the same of Breast (BR) tissue. FIG. IE presents the same of Colon (CO) tissue. FIG. IF presents the same of Thyroid (TH) tissue. Scale bars are 100, 10, and 5 pm for 20x, 60x inset, and 60x zoom, respectively.
[0012] FIGs. 2A - 2E demonstrate that nuclear wrinkles are not an artifact of tissue processing and allow nuclei to assume diverse shapes. FIG. 2A is a graphical representation of the percentage of excess perimeter distribution for pooled DCIS patient nuclei and pooled breast cancer nuclei. Mean values were calculated from DCIS (n = 1882) and invasive cancer (n = 4575) nuclei. Error bars present SD. FIG. 2B is a collage of nuclei observed in FFPE DCIS tissue sample (left) and invasive cancer samples (right) stained for lamin B 1. FIG. 2C presents frozen normal and cancer breast tissue fixed with acetone and stained for lamin B 1. Scale bar is 5 pm. FIG. 2D (top) presents FFPE tissue section exhibiting shrinkage in the form of empty spaces (visible in brightfield) stained for lamin B 1 (yellow) and pan-cytokeratin (magenta). Scale bar is 20 pm for 60x and 5 pm for zoomed. FIG. 2D (bottom) presents zoomed regions from the lamin B1 image above (gray). FIG. 2E presents FFPE tissue sections featuring adjacent smooth and wrinkled nuclei. Scale bar is 5 pm. All images were taken at 60x.
[0013] FIGs. 3A - 3P demonstrate that extreme nuclear wrinkling is a morphological feature of diverse cancers. FIG. 3A presents examples of nuclei sorted into classes 0-5 for deep learning model training. 0 = invalid nuclei, 1 = smooth, 2 = low-frequency contour waviness, 3 = high-frequency contour waviness, and 4 = internal wrinkles. FIG. 3B presents the ResNet50 architecture of the model. An input image is processed through a 7x7 convolutional layer followed by max pooling. The network includes multiple residual learning blocks increasing in depth and complexity, each containing convolutional layers of varying filter sizes, with skip connections to ensure efficient training and feature extraction by mitigating the vanishing gradient problem. After feature extraction, an average pooling layer and a fully connected layer with a softmax function was used to classify the images into the five categories. FIGs. 3C-3H present bar plots showing the normalized corrected count of nuclei in each class for different types of cancer. FIG. 3C presents head and neck; n = 1851,2605, 1619, 1943, 1737 nuclei for adjacent, grades 1-3, respectively. FIG. 3D presents Skin; n = 848, 4180, 1637, 1487, 890 nuclei for adjacent, basal cell carcinoma (BCC), grades 1-3. FIG. 3E presents Ovary; n = 139, 1952, 3355, 3104 nuclei for adjacent, grades 1-3. FIG. 3F presents Breast; n = 3223, 2095, 859, 4296, 1054 nuclei for adjacent, ductal carcinoma in situ (DOS), grades 1-3. FIG. 3G presents Colon; n = 1458, 2147, 7011, 2521 nuclei for adjacent, grades 1-3. FIG. 3H presents Thyroid; n = 275, 4765 nuclei for normal and cancer, respectively. Error bars present a 95% confidence interval of the mean. *False discovery rate adjusted p < 0.05 by the Benjamini-Hochberg procedure.
[0014] FIGs. 3I-3M present bar plots showing the corrected normalized count of nuclei in each class for skin, cervix, liver, lung, and pancreatic normal tissues and cancers, grades 1-3, respectively. There are generally more extremely wrinkled nuclei present in each of these cancers compared to control tissue.
[0015] FIGs. 3N-3O are photographic images of the tissue adjacent to the cancer and cancer tissue of Grade 4 tumor. FIG. 3P is a graphical representation of the corrected normalized count of nuclei in Class 3 (extreme wrinkling) for melanoma for different tumor grades and cancer adjacent tissue. More nuclear wrinkling is visible in the melanoma sample compared to the control epithelial tissue.
[0016] FIGs. 3Q - 3T present nuclear wrinkling in the context of lymph node involvement. Bar plots show the normalized corrected count of nuclei in each wrinkling class. FIG. 3Q presents Ovary; n = 134, 16944, 2414 nuclei for control, TNM grade NO, and Nl, respectively. FIG. 3R presents Breast; n = 2541, 5124, 2489, 510 nuclei for control, N0-N2. FIG. 3S presents Colon; n = 1458, 7479, 2514, 1686 nuclei for control, N0- N2. FIG. 3T presents Thyroid; n = 303, 8395, 1601 nuclei for control, NO, and Nl. Error bars present a 95% confidence interval of the mean. *False discovery rate adjusted p < 0.05 by the Benjamini-Hochberg procedure. Patient information is in Table 3.
[0017] FIGs. 4A - 4F present the Elliptical Fourier analysis reveals higher contour irregularity in cancer. Kernel density plots show the distribution of Elliptical Fourier Coefficient (EFC) ratios and nuclear- areas for control and cancer tissues and for different tumor grades. The estimated kernel densities from the R = 100 random subsets were combined by averaging (see Methods). FIG. 4A presents Head and neck tissue; n = 1508, 1951, 1336, 250, 1603, 5140 nuclei for adjacent, grades 1-3, and pooled tumor grades, respectively. FIG. 4B presents Skin tissue; n = 762, 3724, 1658, 1509, 907, 4074 nuclei for adjacent, basal cell carcinoma (BCC), grades 1-3, and pooled grades. FIG. 4C presents Ovarian tissue; n = 58, 1530, 2027, 7085, 1382, 2074, 3251, 6707 nuclei for control, mucinous, low grade serous, high grade serous, grades 1-3 endometrioid adenocarcinoma, and pooled grades. FIG. 4D presents Breast tissue; n = 2455, 1918, 819, 3859, 1054, 5732 nuclei for adjacent, ductal carcinoma in situ (DCIS), grades 1-3, and pooled grades. FIG. 4E presents Colon tissue; n = 904, 1416, 5987, 2110, 9513 nuclei for adjacent, grades 1-3, and pooled grades. FIG. 4F presents Thyroid tissue; n = 69, 6030 nuclei for adjacent and cancer, p values for equality of means and homogeneity of scales obtained from the Kruskal- Wallis test and the Fligner test and adjusted using Benjamini-Hochberg false discovery rate corrections are labeled. Key for the FIGs. 4A - 4F is presented adjacent to FIG. 4F.
[0018] FIG. 5A presents MDCK cells on fibronectin-coated glass dishes. FIG. 5B presents MDCK acinus in Matrigel. Cells were stained for lamin A / C (gray). X-y views shown above x-z reconstructions. Scale bar is 10 pm.
[0019] FIG. 6A presents a method developed for segmenting nuclear contours from lamin-stained tissue optimized on a per nucleus basis. This method delineates more precise nuclear contours by tracing the intensity maxima (green) on each normal line (red) around the Cellpose mask. Scale bar is 3 pm. FIG. 6B presents a comparison between EFC ratio and solidity metric for a subset of nuclei. Scale bar is 3 pm. FIG. 6C presents a heat map example showing the EFC ratios determined for squamous cell carcinoma of the cheek nuclei (raw image above) using the segmentation method applied equally to all nuclei. Nuclear masks were colorized based on their EFC ratio values: red indicates irregular contours with low EFC ratios, while blue indicates regular contours with high EFC ratios. Scale bar is 20 pm.
[0020] FIG. 7 A presents the posterior probability plots of EFC ratio, nuclear area, and nuclear aspect ratio of head and neck tissue; n = 1508, 1951, 1336, 250, 1603, 5140 nuclei for adjacent, grades 1-3, and pooled tumor grades, respectively. FIG. 7B presents the posterior probability plots of EFC ratio, nuclear- area, and nuclear aspect ratio of skin tissue; n = 762, 3724, 1658, 1509, 907, 4074 nuclei for adjacent, BCC, grades 1-3, and pooled grades. FIG. 7C presents the posterior probability plots of EFC ratio, nuclear area, and nuclear aspect ratio of ovarian tissue; n = 58, 1830, 2966, 3336, 8132 nuclei for adjacent, grades 1-3, and pooled grades. FIG. 7D presents the posterior probability plots of EFC ratio, nuclear area, and nuclear aspect ratio of breast tissue; n = 2455, 1918, 819, 3859, 1054, 5732 nuclei for adjacent, DCIS, grades 1-3, and pooled grades. FIG. 7E presents the posterior probability plots of EFC ratio, nuclear area, and nuclear aspect ratio of colon tissue; n = 904, 1416, 5987, 2110, 9513 nuclei for adjacent, grades 1-3, and pooled grades. FIG. 7F presents the posterior probability plots of EFC ratio, nuclear area, and nuclear aspect ratio of thyroid tissue; n = 69, 6030 nuclei for adjacent and cancer. Key for the FIGs. 7A - 7F is presented above FIG. 7A.
[0021] FIG. 8 A presents Grade 3 colon carcinoma sample stained for DAPI (cyan), lamin Bl (gray), lamin A / C (yellow), and pan-cytokcratin (magenta) and imaged at 60x (N.A. = 1.30). Scale bar- is 10 pm. FIG. 8B presents the same of Grade 2 skin carcinoma sample. Red arrows mark laminar contour irregularities in the same location as indentations in the grouping of condensed chromosomes. Scale bar is 10 pm.
[0022] FIG. 9A presents on the left - 5 micron-apart adjacent sections of FFPE cancer adjacent tongue tissue, imaged with a 60x objective (N.A. = 1.50). One section is stained with H&E and imaged with the color camera. The other section is immunostained for lamin Bl (yellow) and pan-cytokeratin (magenta). Scale bar is 20 pm. FIG. 9A presents on the right: Zoomed individual nuclei from the images on the left, including the lamin Bl channel alone (gr ay). Scale bar 5 pm. FIG. 9B presents on the left: One tissue sample of breast invasive ductal carcinoma immunostained for lamin Bl (yellow) and counterstained with DAPI (cyan). Scale bar is 20 pm. FIG. 9B presents on the left right: Zoomed nuclei from images on the left including merge. Scale bar is 5 pm.
[0023] FIG. 10A presents FFPE skin (SK) tissue stained for lamin Bl (LmnBl; gray), lamin A / C (LmnA / C; yellow), and pan-cytokeratin (Panek; magenta) imaged at 60x (N.A. — 1.30); n = 862, 4719, 2907, 1670, 1021, 10317 nuclei for adjacent, BCC, grades 1-3, and pooled, respectively. FIG. 10B presents Breast (BR); n = 2342, 1886, 505, 1834, 1167, 5392 nuclei for adjacent, DOS, grades 1-3, and pooled. FIG. 10C presents Head and Neck (HN); n = 2291, 2430, 2620, 471, 1259, 6780 nuclei for control, grades 1-3, and pooled. FIG. 10D presents Colon (CO); n = 1816, 2933, 11993, 3175, 18101 nuclei for normal, grades 1-3, and pooled. Scale bar is 20 pm. Patient information for bar graphs of lamin A / C:B 1 ratios calculated for control tissue and cancer grades is in Supplementary Table 4. Kruskal-Wallis tests and Mann-Whitney tests were performed. **** p < 0.0001, *** p < 0.001, ** p < 0.01, * p < 0.05. Error bars present SEM. FIG. 10E presents HN cell line transfected with siSCRM (left) and siLMNA (right), fixed, and stained for lamin A / C and lamin Bl. Scale bar is 50 pm. FIG. 10F presents Bar graphs showing lamin A / C:B1 ratio and individual lamin intensity values for HN transfected with siSCRM (white) and siLMNA (gray); n = 121 and 66 nuclei for siSCRM and siLMNA, respectively. Error bars present SEM. FIG. 10G presents HN wild type cells were fixed and stained for lamin Bl alone (left) or both lamin A / C and lamin Bl (right). Scale bar is 100 pm.
[0024] FIG. 11 presents the Frechet distance between the measured nuclear border and Fourier reconstructed contour plotted against mode number. 15 modes (H = 15) produced a minimum for smooth nuclei, whereas it plateaued at 30 modes (H = 30) for wavy nuclei.
[0025] FIG. 12 presents jigsaw packing which occurs when nuclei deform around each other. The example illustrated in FIG. 12 shows what the typical jigsaw packed nuclei appear in thyroid follicular carcinoma cells as when imaged (FIG. 12, right panel) as opposed to lamin-stained cells when imaged (FIG. 12, left panel).
[0026] FIGs. 13A - 13H present the photographic and the graphical representation of the nuclear wrinkling present in pediatric brain and bone cancers. FIG. 13A is a photographic representation of the immunofluorescent staining for lamin Bl in pediatric cerebrum, meningioma, and medulloblastoma. FIG. 13B is a photographic representation of the immunofluorescent staining for lamin Bl in pediatric cancer adjacent tissue and osteosarcoma. FIG. 13C presents examples of the mcdullobalstoma and meningioma nuclei sorted into each wrinkling class by the methods disclosed herein. FIGs. 13D - 13H are graphical representations of the corrected normalized count of nuclei in each wrinkling class. Error bars present 95% confidence intervals.
[0027] FIGs. 14A- 14C present data regarding the nuclear wrinkling in primary and metastatic osteosarcoma. FIG. 14A is a photographic representation of the immunofluorescent staining for lamin Bl in primary osteosarcoma. FIG. 14B is a photographic representation of the immunofluorescent staining for lamin Bl in metastatic osteosarcoma in the lung. Scale bars is 20 pm. FIG. 14C is a graphical representation of the corrected normalized count of nuclei in each wrinkling class compared with t-tests. **** p<0.0001, **p<0.01, *p<0.05. Error bars represent 95% confidence intervals.
[0028] FIGs. 15A-15G present a comparison of nuclear wrinkling in pediatric and adult brain cancers. FIG. 15A is a photographical representation of the immunofluorescent staining for lamin Bl in anaplasticoligodendroglioma, astrocytoma, and glioblastoma. FIG. 15B is a photographical representation of the adult cerebrum stained for lamin B. FIG. 15C is a graphical representation of the corrected normalized count of nuclei in each wrinkling class in pediatric and adult cerebrum, compared with t-tests. **** p<0.0001, ***p<0.001, **p<0.01, *p<0.05. Error bars represent 95% confidence intervals. FIGs. 15D-15G are graphical representations of corrected normalized count of nuclei in each wrinkling class for each of normal adult cerebrum, astrocytoma, glioblastoma, and anaplastic oligodendroglioma, respectively. There are more nuclei in class 1 in the normal adult cerebrum. There are more nuclei in class 2, class 3, and class 4 in the astrocytoma, glioblastoma, and anaplastic oligodendroglioma samples.
[0029] FIGs. 16A - 16C present the use of epifluorescence microscopy and immunohistochemistry for detecting wrinkles in nuclei rather than immunofluorescence staining and confocal fluorescent microscopy.
[0030] FIG. 17 is a block diagram of a system to train a nuclear morphological analysis model, according to an embodiment of the present disclosure.
[0031] FIG. 18 is a block diagram of a system to utilize a nuclear morphological analysis model, according to an embodiment of the present disclosure.
[0032] FIG. 19 is a flowchart of a method to utilize a nuclear morphological analysis mode to determine whether a cancer prognosis, according to an embodiment of the present disclosure.
[0033] FIG. 20 is a set of images demonstrating that lamin immunostaining reveals nuclear contours with superior sensitivity than H&E staining alone. FIG. 20, left panel is an image of colon carcinoma stained with H&E and imaged at 60x. FIG. 20, middle panel is an image of same location on an adjacent slide stained for pan-cytokeratin (magenta, epithelial marker) and lamin Bl (yellow) and imaged at 60x. FIG. 20, right panel is an image of lamin Bl channel alone. Scale bar is 20 pm.
[0034] FIG. 21 is an image of WGA-stained (left), lamin Bl stained (middle), and merged (right) cells in patient breast cancer tissue.
[0035] FIG. 22 is a set of confocal image of F-actin (magenta) in fibrosarcoma (HT-1080) and head and neck cancer cells (HN) expressing GFP-LMNA (green) cultured on a 30 (top) or 50 pm (bottom) fibronectin circular micropattern.Detailed Description
[0036] Some embodiments of the present disclosure will now be described more fully hereinafter with reference to the accompanying figures, in which some, but not all, embodiments of the disclosures are shown. Indeed, these disclosures may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments arc provided so that this disclosure will satisfy applicable legal requirements.
[0037] The term “computing device” is used herein to refer to any one or all of a controller, programmable logic controllers (PLCs), programmable automation controllers (PACs), industrial computers, desktop computers, personal data assistants (PDAs), laptop computers, tablet computers, smart books, palm-topcomputers, personal computers, smartphones, wearable devices (such as headsets, smartwatches, or the like), and similar electronic devices equipped with at least a processor and any other physical components necessarily to perform the various operations described herein. Devices such as smartphones, laptop computers, tablet computers, and wearable devices are generally collectively referred to as mobile devices.
[0038] The term “server” or “server device” is used to refer to any computing device capable of functioning as a server, such as a master exchange server, web server, mail server, document server, or any other type of server. A server may be a dedicated computing device or a server module (e.g., an application) hosted by a computing device that causes the computing device to operate as a server. A server module (e.g., server application) may be a full function server module, or a light or secondary server module (e.g., light or secondary server application) that is configured to provide synchronization services among the dynamic databases on computing devices. A light server or secondary server may be a slimmed-down version of server type functionality that can be implemented on a computing device, such as a smart phone, thereby enabling it to function as an Internet server (e.g., an enterprise e-mail server) only to the extent necessary to provide the functionality described herein.
[0039] As used herein, a “non-transitory machine-readable storage medium” or “memory” may be any electronic, magnetic, optical, or other physical storage apparatus to contain or store information such as executable instructions, data, and the like. For example, any machine-readable storage medium described herein may be any of random access memory (RAM), volatile memory, non-volatile memory, flash memory, a storage drive (e.g., hard drive), a solid state drive, any type of storage disc, and the like, or a combination thereof. The memory may store or include instructions executable by the processor.
[0040] As used herein, a “processor” or “processing circuitry” may include, for example one processor or multiple processors included in a single device or distributed across multiple computing devices. The processor may be at least one of a central processing unit (CPU), a semiconductor-based microprocessor, a graphics processing unit (GPU), a field-programmable gate array (FPGA) to retrieve and execute instructions, a real time processor (RTP), other electronic circuitry suitable for the retrieval and execution instructions stored on a machine-readable storage medium, or a combination thereof.
[0041] As used herein, “signal communication” refers to electric communication such as hard wiring two components together or wireless communication, as understood by those skilled in the art. For example, wireless communication may be Wi-Fi®, Bluetooth®, ZigBee, cellular networks, or forms of near or far field communications. In addition, signal communication may include one or more intermediate controllers or relays disposed between elements that arc in signal communication with one another.
[0042] As used herein, to “treat” means decrease, suppress, attenuate, diminish, arrest, or stabilize the development or progression of a cancer or tumor, and includes reducing the frequency with which symptoms of a disease (i.e., tumor growth and / or metastasis, or other effect mediated by the numbers and / or activity of immune cells, and the like) are experienced by a patient. Treatment may be prophylactic (to prevent or delaythe onset of the disease, or to prevent the manifestation of clinical or subclinical symptoms thereof) or therapeutic suppression or alleviation of symptoms after the manifestation of the disease. The term “treat” includes (a) physical intervention, such as surgery, phototherapy, or laser therapy, or (b) biochemical intervention, such as the administration of chemotherapeutic compounds or biological agents, to (i) prevent or delay the onset of the symptoms, complications, or biochemical indicia of, (ii) alleviate the symptoms of, and / or (iii) inhibit or arrest the further development of the tumor or cancer.
[0043] Nuclear folds, which were quantified as excess surface area relative to a sphere of the same volume as the nucleus, imply that the nuclear lamina does not resist extensional strain so long as these folds are present. The presence of excess area in the lamina provided support for a recent model of nuclear shaping in which the nuclei offer little mechanical resistance to deformation as long as there is sufficient excess surface area in the lamina . The excess area in the lamina is visible as laminar folds / wrinkles in flattened nuclei. The artificial intelligence-driven model analyzes images to evaluate the extent of nuclear wrinkling in the cells in the sample. Nuclei will be soft or compliant in human tissue cells and become stiff only when the lamina is smooth, consistent with recent measurements. Given that a smooth lamina is present only in extreme nuclear shapes, such as flattened or elongated nuclei, nuclei with a smooth lamina should be rare in vivo, as cells and nuclei in a 3D environment are less likely to be flattened.
[0044] Embodiments of the artificial intelligence-driven systems and methods herein are based on the nuclear shaping in human tissues. Data here demonstrate that: a) nuclear laminar folds and wrinkles are present in epithelial cells in diverse tissues, b) nuclei from epithelial tissues with a smooth lamina are relatively rare, and c) nuclei assume a wide range of shapes when wrinkles / folds are present. Extreme laminar wrinkling is a quantifiable, diagnostically valid morphological marker of human carcinomas. Additionally, the use of artificial intelligence-driven systems and methods enable improved, accurate, and / or prompt image analysis that aid in and / or automatically make relevant clinical decisions / recommendations for therapy or treatment regimens. Such decisions include determining a new therapy for a patient and determining efficacy of and alteration of a current or prior therapy. Thus, these artificial intclligcncc-drivcn systems and methods provide the information and analysis to support highly informed clinical decisions as compared to information from conventional hematoxylin and eosin (H&E) staining of the tissues.
[0045] Nuclei in diverse control and cancer tissues have a wrinkled nuclear lamina.
[0046] The artificial intelligence-driven systems evaluate the extent to which the nucleus is wrinkled in the cells in the sample. FIGs. 5 A - 5B demonstrate that MDCK cells in 2D culture are both flatter and less wrinkled than those in an acinus, consistent with nuclei geometrically develop folds in their rounded shapes. In contrast to cultured cells, epithelial nuclei in human patient tissues are unlikely to be flattened, because cells in vivo do not adhere to flat and stiff smooth surfaces but rather adhere to a three-dimensional extracellular matrix. Thus, there should be folds and wrinkles in the nuclear lamina in normal as well as cancer tissues, unless cancer causes nuclear flattening or reduces the area of the nuclear lamina at a constant nuclear volume. Therefore, thenuclear lamina was imaged in diverse patient tissues, including grade 1-3 cancer tissues and cancer adjacent or normal tissues. Formalin-fixed paraffin-embedded (FFPE) human tissue microarrays were immunostained for lamin B 1 and pan-cytokeratin as a marker of normal epithelial and carcinoma cells and performed high- resolution confocal microscopy using head and neck, skin, ovarian, breast, colon, and thyroid tissues (FIGs. 1A - IF). Most nuclei in cytokeratin-stained cells had some degree of folding / wrinkling in the nuclear lamina, in both control and cancer tissues. The only tissue that had predominantly nuclei with a smooth lamina was pure breast ductal carcinoma in situ (DCIS). Patients with DOS who also had invasive carcinoma elsewhere had a visually higher percentage of nuclear wrinkling than those with pure DCIS. These data confirm that nuclei in diverse normal and cancer tissues in vivo contain folds / wrinkles in the nuclear lamina, and nuclei with a smooth lamina are rare in vivo. Visual inspection as supported by deep learning methods confirm, as shown in FIGs. 3A-3P, an increased prevalence of laminar wrinkling in all cancer types relative to controls.
[0047] Nuclei with a wrinkled lamina should assume diverse shapes, because there is little mechanical resistance to shape deformation. In contrast, nuclei with smooth lamina should be “stiff’ to deformation, resulting in a narrower range of shapes. Shapes in DCIS were confirmed, where the lamina was smoother as compared to the invasive breast carcinoma (nuclei were wrinkled). The perimeter of the shapes were computed to identify samples whose perimeters were greater than that of a circle of the same area. The standard deviation of the excess perimeter was significantly higher in invasive cancer compared to DCIS, confirming that nuclear shapes are highly variable when there are more folds / wrinkles in the nuclear lamina (FIG. 2A). The diversity of nuclear shapes in wrinkled nuclei in invasive carcinoma as opposed to smooth nuclei in DCIS was evaluated (FIG. 2B).
[0048] Nuclear wrinkles are not an artifact of tissue processing.
[0049] Although FFPE samples are the workhorse of pathological diagnosis, and nuclear abnormalities observed in these samples inform routine diagnosis, FFPE sample preparation may contribute to nuclear wrinkling. Therefore, frozen, acetone-fixed normal, and breast cancer tissue were stained for lamin Bl. Unlike FFPE tissue, which is fixed in formalin for 24 to 48 h, frozen tissue here was snap-frozen in the gas phase of liquid nitrogen after embedding in an optimum cutting temperature compound and was then fixed for only 15 min in acetone. Here, frozen tissue did not undergo the FFPE heat- and pH-based antigen retrieval process to reduce excessive crosslinking from lengthy formalin fixation. Unlike FFPE tissue, which is washed with xylene and ethanol, frozen tissue is air-dried before blocking. Despite these major processing differences, nuclear wrinkling was also observed in frozen cancer and control tissues (FIG. 2C). As shrinkage due to formalin fixation or other factors may cause nuclear wrinkling, some rare FFPE tissue samples were examined with minor shrinkage, indicated by blank spaces, and found cases with smooth nuclei in these regions, demonstrating that tissue shrinkage does not necessarily result in wrinkled nuclei (FIG. 2D). Further, many examples of adjacent smooth and wrinkled nuclei in the same FFPE tissue sample were found for diverse tissue types (FIG. 2E). Finally, as processing artifacts cannot explain the systematic and reproducible differences innuclear laminar wrinkling across tissue types (FIGs. 1 A - IF), wrinkling differences are not sample preparation artifacts.
[0050] Extreme nuclear wrinkling is a morphological feature of diverse cancers.
[0051] Nuclear lamina in both control and cancer tissues contains folds / wrinkles, but nuclear laminar irregularities appeared to be more common in cancer compared to control tissue (FIGs. 1A - IF). To quantify wrinkling, deep learning models were developed to classify nuclear laminar irregularities. Based on visual inspection of thousands of nuclei, five classification categories were chosen — (1) out-of-focus / wrongly- cropped nuclei, (2) smooth nuclei, (3) nuclei with low-frequency contour waviness, (4) nuclei with high- frequency contour waviness, and (5) nuclei with internal wrinkles (FIG. 3A). Nuclei tended to have inner wrinkles or high frequency contour waviness, which represented distinct types of extreme nuclear wrinkling. Internal wrinkles appeared due to deep invaginations in the lamina, while surface wrinkling without deep invaginations appeared as high frequency contour waviness. These correspond closely with the “star-like” or “garland-like” nuclei reported in emerin-stained thyroid carcinoma samples.
[0052] To train the deep learning algorithm, the Cellpose anatomical segmentation algorithm was used to segment and crop nuclei from confocal microscopy images of control and breast cancer tissue, which were then sorted manually into the five nuclear shape categories. Only nuclei in pan-cytokeratin-expressing cells that passed a focus test were included in the analysis, thereby excluding about 45% of nuclei. These cropped images were used to train a multi-class classifier (FIG. 3B). A transfer learning approach was implemented using a pre-trained ResNet50 model, which was fine-tuned to the specific dataset (as discussed infra). The deep learning model was trained on several thousand segmented nuclei, with a high overall classification accuracy of 90.26%, confirming the robustness of the model. The model achieved a high overall Fl-score of 0.9014 and an area under the receiver operating characteristic curve (AUC) value of 0.9809, indicating excellent precision, recall, and discriminative ability across classes. Class-specific performance, expressed in the confusion matrix, was 88.21% for smooth nuclei, 90.65% for low-frequency contour waviness, 94.29% for high-frequency contour waviness, 90.63% for interior wrinkles, and 86.02% for less typical nuclei (out of focus). This model was used to classify nuclear wrinkling types across diverse cancer cell types, whose nuclei were cropped the same as the training data. The model sorted thousands of nuclei from the control and cancer grades 1-3 for each tissue type, with 50-80 patients per tissue type. Patient information is included in Table 1.
[0053] Table 1. Case information for deep learning and elliptical Fourier analysis.
[0054] Tumor grade was determined as the majority decision based on the grades of two external pathologists and the grade provided by the tissue supplier’s pathologists. In the few cases where all three evaluations differed, the grade provided by the tissue supplier company was assigned. Results were quantified using a corrected, normalized count considering individual class accuracies.
[0055] Low-frequency contour waviness in the nuclear lamina was the most prevalent wrinkling type in every cancer and control tissue (FIGs. 3C - 3P and Table 2), consistent with FIGs. 1A - IF, in which even control tissues exhibited wrinkling. (data not shown here). Breast, skin, and thyroid cancers exhibited a similar pattern, with smoother nuclei more prevalent in adjacent tissue and reduced in cancerous tissues. Conversely, high- frequency wrinkles were consistently more frequent in cancer tissue compared to control tissue in these three cancer types. Notably, breast DCIS tissue had a high fraction of smooth nuclei, consistent with the visualobservations (FIGs. 1 A - IF). Head and neck cancer tissues had fewer smooth nuclei in cancer tissue compared to control. Nuclei with inner wrinkles were more prevalent in this cancer type compared to high-frequency contours. Ovarian and colon cancer tissues had a low frequency of smooth nuclei in adjacent tissue, consistent with our visual observations (FIGs. 1 A - IF). Although the high-frequency contours were much more prevalent in tissue from ovarian cancer vs. the control, tissue from colon cancer showed more subtle differences in extreme wrinkling compared to control tissue. Collectively, these findings show that extreme wrinkling of the nuclear lamina, comprising high-frequency wrinkles and inner wrinkles, is a morphological hallmark of diverse cancers.
[0056] Table 2. Normalized corrected nuclei counts for each wrinkling category. Probabilities and 95% confidence intervals are presented as percentages (%).
[0057] Extreme nuclear wrinkling was more prevalent in ovarian, breast, colon, and thyroid cancers that had lymph node involvement (Table 3). FIGs. 3Q - 3T present nuclear wrinkling in the context of lymph node involvement. In cancer staging, NO, Nl, and N2 describe the extent of lymph node involvement. NO means no cancer cells are found in nearby lymph nodes. Nl indicates cancer spread to nearby lymph nodes, while N2 signifies more extensive spread to lymph nodes further away. Bar plots show the normalized corrected count of nuclei in each wrinkling class. FIG. 3Q presents Ovary; n = 134, 16944, 2414 nuclei for control, TNM grade NO, and Nl, respectively. FIG. 3R presents Breast; n = 2541, 5124, 2489, 510 nuclei for control, N0-N2. FIG. 3S presents Colon; n = 1458, 7479, 2514, 1686 nuclei for control, N0-N2. FIG. 3T presents Thyroid; n = 303, 8395, 1601 nuclei for control, NO, and Nl. Error bars present a 95% confidence interval of the mean. *False discovery rate adjusted p < 0.05 by the Benjamini-Hochberg procedure. Patient information is in Table 3. The TNM classification was used, where an N value greater than or equal to1 represents lymph node involvement. This result, consistent with the trend observed in tumor grades (FIGs. 3C - 3T), indicates that extreme nuclear wrinkling may be associated with cancer progression and metastatic potential.
[0058] Table 3 : Case information for lymph node involvement analysis. Sex
[0059] Elliptical Fourier analysis reveals that laminar contour irregularity is higher in cancer tissues.
[0060] To confirm that the high-frequency laminar contour waviness revealed by the deep lear ning model was more common in cancers, elliptical Fourier analysis was used, which specifically quantifies contour irregularities. For the Fourier analysis, accurate segmentation of nuclear contours was needed; however, segmentation of the nucleus using Cellpose algorithm was unsuccessful as it tended to smooth out micron and sub-micron variations in the nuclear contour. Given that imaging nuclear lamins produces a clearly delineated thin boundary, a previously developed method was adapted and used to segment microtubules to segment the nuclear shape (FIG. 6A). Briefly, the method approximated nuclear- segmentation using the deep learning program Cellpose, then normal lines were computationally drawn around the Cellpose mask, and the point on each line with maximum intensity was taken as the most likely location of the nuclear lamina. Limited smoothing connected the points around the nuclear contour. This method resulted in clear and visually accurate segmentation of the nuclear contour (FIG. 6A). Elliptical Fourier analysis was performed on the segmented contours and the elliptical Fourier coefficient (EFC) ratio was calculated for each nucleus. The EFC ratio varied inversely with the visible level of nuclear irregularity and was a much more sensitive metric than the commonly used solidity parameter (FIG. 6B). These segmentation and EFC ratio quantification methods applied to a standard field of view are shown in a heat map in FIG. 6C.[0061 J Using the same approach to restrict the analysis to only in-focus nuclei from cytokeratin-stained cells described above, the distributions of EFC ratios were compared visually via kernel density estimate plots(FIGS. 4A - 4F) and formally via robust statistical tests for means and scales (standard deviations) between control and pooled cancer grades and between each cancer type. As a positive control, nuclear area was also computed, which is higher in cancer, and their distributions were compared. EFC ratios varied between control and cancer tissue depending on cancer type, with a lower EFC ratio for head and neck, skin, and breast cancer compared to control tissues and a higher EFC ratio for colon cancer. In ovarian and thyroid cancer, there was a trend toward lower EFC ratios compared to control tissues, but differences in the mean values lacked statistical significance. This lack of significance was likely due to the difficulty in collecting sufficient epithelial cell images from normal ovarian (n — 58) and thyroid tissue (n — 69), where epithelial cells form a monolayer in the germinal epithelium or follicles, respectively. In contrast, nuclear area was higher in cancer tissue compared to control tissue in all cancer types. These results suggest that nuclear contour irregularity is a marker for a subset of cancers. Given that deep learning detects differences in extreme wrinkling across all cancers, the small differences in EFC ratios in some cancers may reflect the inability of elliptical Fourier analysis to fully capture nuclear wrinkling, especially folds inside the nuclear body. Also, the small differences in some cases may be due to low-frequency contour waviness, which is similar across cancer and control tissues and may be as frequent or more frequent than high-frequency waviness. This might mask the contribution of low EFC ratios, which represent the high-frequency waviness, in the comparisons.
[0062] To address this complication, a linear discriminant-based posterior probability analysis was performed to compare the probabilities of tumor grade or control for a given EFC ratio magnitude. The posterior probabilities were calculated, defining discrimination boundaries of cancer adjacent tissue vs. tumor grades under a uniform / flat prior assumption for tumor grade proportions, using a linear discriminant analysis of log- transformed nuclear EFC ratios, nuclear areas, and nuclear aspect ratios (FIGS. 7A-7F). EFC ratio values associated with control vs. cancer nuclei were significantly different with a clear separation between control tissue and cancer grades. Consistent with the results obtained using the deep learning analysis, low EFC ratios, corresponding to high-frequency contour waviness, were especially predictive of cancer grades. These results confirm the results of the deep learning analysis that cancer is characterized by extreme nuclear wrinkling.
[0063] Nuclear atypia is a common char acteristic of human cancers. A wrinkled nucleus can assume a range of shapes with little mechanical resistance. When the nucleus reaches a limiting shape, as in the flattened nuclei in culture, the lamina is smooth and the nucleus is predicted to become “stiff’ to deformation. The nuclear lamina in diverse control and cancer tissues had folds / wrinkles accompanied by a wide variety of nuclear shapes. In a minority of tissues, the nuclear- lamina was smooth, as it is in in vitro cultures, which is likely because nuclei in tissues rarely take on extremely flattened shapes as they do in culture. Predominantly wrinkled nuclei in patient tissues was expected, as the lamina is assembled in a wrinkled state dur ing mitosis because it forms around the non-smooth surface presented by postmitotic chromosomes. This characteristic is directly visible in patient tissues (FIGS. 8A-8B). Wrinkled nuclei are compliant, yielding diverse nuclear shapes in tissues.
[0064] Nuclear wrinkling was common across diverse tissues; however, extreme wrinkling was more prevalent in cancer tissues than in control tissues in all cancer types studied. Extreme wrinkling was quantified using both deep learning and elliptical Fourier analysis. Both methods demonstrated that extreme wrinkling, marked by increased waviness of the nuclear contour, was more frequent in cancer tissue.
[0065] The gold standard for cancer diagnosis uses hematoxylin and eosin (H&E) images of tissue samples. Although nuclear shape irregularities have been quantified from H&E images, it is unlikely that the high- frequency wrinkling observed in lamin stains is visible in H&E images because hematoxylin yields a solidly stained nucleus, precluding sensitive visualization of nuclear contour s. When a tissue section using H&E and an adjacent section for lamin Bl were stained and imaged, lamin Bl stained nuclei showed the greater spatial detail that is necessary for visualizing wrinkling compared with the H&E images. Images of nuclei from the same field of a tissue sample immunostained for lamin Bl and counterstained with DAPI (4’,6-diamidino-2- phenylindole), which also stains DNA, confirmed this observation (FIGS. 9A-9B). The inability of DNA stains to produce detailed visualizations of nuclear contours is supported by studies using lamin or emerin stains, which have diagnostic or prognostic utility. In emerin-stained breast tissue in which nuclei were classified manually as having either low or high nuclear envelope pleomorphism (NEP), there was a correlation between NEP and lymph node metastasis. Folds were identified in the nuclear contours of breast cancer cells but not in control tissues, perhaps because of the few control samples. In contrast, nuclear folds were observed in many of the control samples. Other studies used emerin stains in thyroid tissue samples to assist in the diagnosis of papillary thyroid carcinoma in borderline cases by providing a clearer image of the nuclear shape. In these studies, a pathologist classified emerin-stained nuclei into different shape categories, including nuclei with garlands that would have been considered to have high-frequency contour waviness, and nuclei with grooves that appeared to be a type of inner nuclear wrinkling.
[0066] Others have used lamin or emerin stains to quantify nuclear shape. One group used deep learning on images of lamin-stained ovarian tissue and datasets of nuclear shape factors from lamin-stained images to successfully sort nuclei into healthy or cancer groups. However, they did not consider nuclear wrinkling and did not restrict the analysis to any cell types in their classification. Another study of wrinkling in lung cancer measured the difference in perimeter between hematoxylin-stained and emerin-stained nuclei to estimate the occurrence of nuclear grooves or cytoplasmic inclusions. Although this study indirectly quantified nuclear wrinkling, it required two nuclear stains to calculate the less precise excess perimeter value. In contrast, embodiments of the systems and methods here identified wrinkled nuclei in both cancer and control tissue and used deep learning and quantitative techniques to assess the degree of wrinkling. Segregating nuclei into the correct classes revealed extreme wrinkling as a hallmark of cancer. These aspects were supported by the quantitative Fourier analysis of nuclear contours.
[0067] Using lamin-stained images as input for Al / deep learning in digital pathology to determine the extent of nuclear wrinkling improved cancer diagnosis and prognosis for multiple tissue types, especially head andneck, skin, breast, and thyroid cancers. Lamin staining is a relatively inexpensive addition to standard pathology workflows. Embodiments can include fluorescence microscopy or immunohistochemistry. Imaging the folds / wrinkles will likely require at least 40x magnification with a high numerical aperture (~1). Deconvolution of images obtained with a camera can result in images that approach the image quality from a confocal microscope.
[0068] Wrinkling may be induced geometrically in nuclei by rounding up flattened cells. The rounding of a flattened nucleus with a smooth lamina at a constant volume induces wrinkles geometrically because a sphere provides the minimum surface area for a given volume. Likewise, cell rounding may occur in some cancers, due perhaps to crowding of proliferating cells. Further, the area of the nuclear lamina may be higher for a given nuclear volume in cancer compared to control tissue. Alternatively, irregular shapes may result from the depletion of lamin A / C, which occurs in diverse cancers and is associated with a worse prognosis. In vitro, migrating lamin A / C null mouse embryonic fibroblasts deform without an apparent limit on areal expansion when they are indented by external, slender obstacles. Consistent with these prior experiments, a significantly lower lamin A / C:B1 ratio in breast cancer tissue was observed compared to control tissue (FIGS. 10A - 10G), but no differences between breast cancer tissues of different grades. Among different cancer types, there were only minor differences in lamin A / C:B1 ratios between control and cancer tissue (Table 4). Thus, a decrease in the lamin A / C:B 1 ratio does not explain the systematic differences in nuclear wrinkling between control and cancer tissue. FIG. 10A presents FFPE skin (SK) tissue stained for lamin Bl (LmnBl; gray), lamin A / C (LmnA / C; yellow), and pan-cytokeratin (Panek; magenta) imaged at 60x (N.A. = 1.30); n = 862, 4719, 2907, 1670, 1021, 10317 nuclei for adjacent, BCC, grades 1-3, and pooled, respectively. FIG. 10B presents Breast (BR); n = 2342, 1886, 505, 1834, 1167, 5392 nuclei for adjacent, DOS, grades 1-3, and pooled. FIG. 10C presents Head and Neck (HN); n = 2291, 2430, 2620, 471, 1259, 6780 nuclei for control, grades 1-3, and pooled. FIG. 10D presents Colon (CO); n = 1816, 2933, 1 1993, 3175, 18101 nuclei for normal, grades 1-3, and pooled. Scale bar is 20 pm. Patient information for bar graphs of lamin A / C:B 1 ratios calculated for control tissue and cancer grades is in Supplementary Table 4. Kruskal-Wallis tests and Mann-Whitney tests were performed. **** p < 0.0001, p < 0.001, ** p < 0.01, * p < 0.05. Error bars present SEM. FIG. 10E presents HN cell line transfected with siSCRM (left) and siLMNA (right), fixed, and stained for lamin A / C and lamin Bl. Scale bar is 50 pm. FIG. 10F presents Bar graphs showing lamin A / C:B1 ratio and individual lamin intensity values for HN transfected with siSCRM (white) and siLMNA (gray); n = 121 and 66 nuclei for siSCRM and siLMNA, respectively. Error bars present SEM. FIG. 10G presents HN wild type cells were fixed and stained for lamin Bl alone (left) or both lamin A / C and lamin Bl (right). Scale bar is 100 pm.
[0069] Overall, the nuclear shapes observed in diverse tissues are consistent with a model in which nuclei resist deformation only when the nuclear lamina is smooth and that a difference in the extent and type of nuclear laminar wrinkling is a characteristic morphological feature of diverse human cancers. The difference in nuclear wrinkling was identified by deep learning algorithms and quantitative analysis of segmented nuclearcontours. Thus, altered nuclear wrinkling is a morphological cancer biomarker that can be identified by digital pathology.
[0070] Table 4. Case information for lamin A / C:B1 ratio analysis.
[0071] Wrinkling of the nuclear lamina is a marker of cancer in pediatric cancers.
[0072] Embodiments of the methods include obtaining high-resolution immunofluorescence imaging of nuclear lamin B 1 and applying Al-based classification of the nuclear lamina to detect the presence of wrinkling in pediatric brain and bone cancers. Unlike nuclei in other tissues, normal pediatric cerebellar nuclei tended to be remarkably smooth. Extreme nuclear laminar wrinkling occurred at higher frequency in both pediatric brain and bone cancers compared to normal or cancer adjacent tissue, indicating that extreme wrinkling is a morphological marker of pediatric brain and bone cancers. Nuclear wrinkling in the brain is not an age-related phenomenon but instead develops during tumor formation.
[0073] There is a need for methods of detecting, subtyping, and developing a robust prognosis of brain or bone cancers in pediatric subjects. These methods would assist healthcare professionals to avoid over- andunder-treatment of patients through targeted therapies, and thereby reduce morbidity and mortality from these cancers. Prognostic information can include features like tumor location and nuclear morphology. Nuclear morphology has been identified as a possible prognostic indicator in osteosarcoma, medulloblastoma, and perhaps ependymoma. Additionally, nuclear morphology has also been used to subtype benign meningiomas7and medulloblastoma. However, such studies, and indeed clinical practice, rely on hematoxylin and eosin (H&E) staining to assess nuclear morphology. H&E staining, which has been in use since the 1800s, provides a solid nuclear stain by staining the nucleic acids, which obscures significant nuclear shape features. Staining for proteins on or near- the nuclear envelope like lamins or emerin provides a much clearer image of the nuclear contour, especially when imaged at high resolution. Presence of nuclear laminar wrinkling was assessed in pediatric normal, cancer adjacent, and cancer tissues.
[0074] FIGs. 13 A - 13H present the photographic and the graphical representation of the nuclear wrinkling present in pediatric brain and bone cancers. Table 5 presents the patient information for each tissue type analyzed. FIG. 13A is a photographic representation of the immunofluorescent staining for lamin Bl in pediatric cerebrum, meningioma, and medulloblastoma.
[0075] Table 5. Patient information for each tissue type analyzed.
[0076] FIG. 13B is a photographic representation of the immunofluorescent staining for lamin Bl in pediatric cancer adjacent tissue and osteosarcoma. FIG. 13C presents examples of the medullobalstoma and meningioma nuclei sorted into each wrinkling class by the methods disclosed herein. These are examples of nuclei from pediatric tissues sorted into the four wrinkling classes by the deep learning model. FIGs. 13D - 13H are graphical representations of the corrected normalized count of nuclei in each wrinkling class. Error bars present 95% confidence intervals. In pediatric cerebrum tissue, class 1 nuclei are most common, followed by class 2 nuclei. In contrast, in medulloblastoma and meningioma, class 2 nuclei are most common. There are also nuclei in class 3 and 4 which were absent in cerebrum tissue. Normal bone has mostly class 1 and class 2 nuclei, while osteosarcoma has less class 1 and slightly more class 3 nuclei.
[0077] Regions of interest were annotated by a pathologist viewing a digitalized H&E-stained slide of a tissue slice. Then, immunofluorescent staining for lamin B 1 was performed on a second tissue slice approximately5-10 |im away, and the regions of interest were imaged for each tissue type. Visually, nuclei in normal cerebrum and cancer adjacent bone tended to be quite smooth (FIGs. 13A - 13B). Nuclei in brain and bone cancer tissues tended to have more nuclear wrinkling.
[0078] Jigsaw packing
[0079] Jigsaw packing occurs when nuclei in a sample deform around each other, which results in mirrored contours, similar to that of puzzle pieces. Jigsaw packing is not visible in H&E stained tissues, even when imaged at high resolution. However, by utilizing nuclear lamina staining and imaging at high resolution, jigsaw packing becomes visible. FIG. 12 presents jigsaw packing which occurs when nuclei deform ar ound each other. Both panels both show lamin staining (yellow). FIG. 12, right panel shows the pan-cytokeratin (epithelial stain, magenta) in addition to the lamin stain. The pan-cytokeratin stain indicates which nuclei in the image are epithelial cells. The example illustrated in FIG. 12 shows what extremely jigsaw packed nuclei appear in thyroid follicular carcinoma cells as when imaged. Further, jigsaw packing occurs when cells deform around each other (nuclear wrinkling indicates that nuclei are not tensed to deformation). Embodiment include methods of detecting and treating cancers by assessing images of cell samples for jigsaw packing.
[0080] Extreme wrinkling is detected in brain and bone pediatric cancers.
[0081] To quantify the extent of nuclear wrinkling in these tissues, nuclei were roughly segmented with the deep learning program, Cellpose, and sorted into nuclear wrinkling classes by the deep learning methods presented herein. The nuclear wrinkling classes include smooth nuclei (class 1), low frequency contour waviness (class 2), high frequency contour waviness (class 3), and inner nuclear wrinkles (class 4). Classes 3 and 4 are both considered extremely wrinkled nuclei (FIG. 13C). The proportion of nuclei in each wrinkling category, corrected for the accuracy, was calculated for each tissue (FIGs. 13D - 13H). In normal pediatric cerebrum, some nuclear wrinkling was observed, but most nuclei were smooth (class 1), with less than one third of normal nuclei observed to have some low frequency contour variation (class 2), and no extremely wrinkled nuclei (classes 3 and 4), in contrast to the findings in adult epithelial tissues. Most brain cancer nuclei were class 2, with far less class 1 nuclei present. Brain cancer tissue also had a significant number of extremely wrinkled nuclei (classes 3 and 4).
[0082] In cancer adjacent bone tissue, smooth nuclei (class 1) were the most common, followed by low frequency contour nuclei (class 2). A few extremely wrinkled nuclei were present. In osteosarcoma tissues, class 1 nuclei were less common and class 3 wrinkled nuclei were more common.
[0083] Metastatic tissue has more extreme wrinkling than primary tumors in osteosarcoma.
[0084] Nuclear wrinkling between the primary and metastatic osteosarcoma (to the lung) was evaluated (FIG. 14A - 14C). FIGs. 14A- 14C present data regarding the nuclear wrinkling in primary and metastatic osteosarcoma. FIG. 14A is a photographic representation of the immunofluorescent staining for lamin Bl in primary osteosarcoma. FIG. 14B is a photographic representation of the immunofluorescent staining for lamin Bl in metastatic osteosarcoma in the lung. Scale bars is 20 pm. FIG. 14C is a graphical representation of thecorrected normalized count of nuclei in each wrinkling class compared with t-tests. **** p<0.0001, **p<0.01, *p<0.05. Error bars represent 95% confidence intervals. There were less class 1, less class 2, and more class 3 nuclei when comparing metastatic osteosarcoma to primary tumors.
[0085] Comparison of wrinkling in adult and pediatric brain tissues.
[0086] FIGs. 15A-15G present a comparison of nuclear wrinkling in pediatric and adult brain cancers. FIG. 15 A is a photographical representation of the immunofluorescent staining for lamin Bl in anaplastic oligodendroglioma, astrocytoma, and glioblastoma. FIG. 15B is a photographical representation of the adult cerebrum (bottom) stained for lamin B. FIG. 15C is a graphical representation of the collected normalized count of nuclei in each wrinkling class in pediatric and adult cerebrum, compared with t-tests. **** p<0.0001, ***p<0.001, **p<0.01, *p<0.05. Error bars represent 95% confidence intervals.
[0087] To test whether patient age influenced nuclear wrinkling, laminar wrinkling was assessed in normal adult brain cerebrum and brain cancer tissues. Similarly to pediatric brain cancer, there were less class 1 nuclei, and more class 2, 3, and 4 nuclei in adult brain cancer compared to adult brain cerebrum. Notably, when comparing pediatric cerebrum to adult cerebrum, there were no significant differences in any nuclear- wrinkling class (FIGs. 15D - 15G).
[0088] Quantification of nuclear contours alone misses internal nuclear wrinkles and does not fully capture extreme nuclear wrinkling, indicating the superiority of deep learning to assess complete nuclear shape. Assessing for extreme nuclear wrinkling through lamin B 1 immunostaining has relevance in pediatric bone cancers and in pediatric (and adult) brain cancers. Notably, metastatic osteosarcoma had more extremely wrinkled nuclei than primary osteosarcoma, suggesting a relationship between osteosarcoma aggression and nuclear laminar wrinkling. Further, nuclear wrinkling did not vary between normal pediatric and adult brain tissues argues that nuclear wrinkling is not a symptom of aging but is instead a result of cancer development.
[0089] Use of immunohistochemistry (IHC) to evaluate laminar wrinkling
[0090] The gold standard for assessing nuclear atypia is qualitative, visual observation of hematoxylin and cosin (H&E) stained tissues by a pathologist. However, the inherent subjectivity of this method introduces inter- and intra-pathologist variability to the diagnostic and prognostic process, which in turn results in under- and over-treatment of cancer patients.
[0091] Embodiments disclosed herein take advantage of extreme wrinkling of the nuclear lamina as a quantifiable morphological marker of cancer. In an embodiment, the method includes conducting immunofluorescence (IF) staining and confocal fluorescent microscopy. In certain embodiments, the confocal fluorescent microscopy measurements arc obtained at 60X with at least a 1.3 NA.
[0092] In other embodiments, the method includes conducting epifluorescence microscopy to detect extreme wrinkling instead of confocal imaging and capitalizes on the ability of immunohistochemistry (IHC) rather than IF to detect laminar wrinkling. Embodiments of methods include reducing diagnostic variability by quantifying nuclear shape, either mathematically or using machine learning. However, such attempts arelimited by the use of H&E stained tissues in this analysis. As hematoxylin stains nucleic acids, using this stain to detect nuclear shape results in a filled-in nuclear- shape. Staining for proteins on or near the nuclear envelope (including the nuclear lamina or emeriti) results in a clearer image of the nuclear shape, especially the nuclear contour, than staining for hematoxylin alone. Staining for the nuclear lamina reveals nuclear wrinkling, a phenotype, which is quantifiable with deep learning, to facilitate robust detection of cancer, assessment of efficacy of treatment.
[0093] Nuclear wrinkling is visible with epifluorescence imaging as well as confocal imaging (FIG. 16A). These are 40X images with 1.15 NA. Different types of nuclear- wrinkling are visible with both epifluorescence and confocal imaging, although the images with confocal imaging are more crisp.
[0094] Extreme nuclear wrinkling can be detected with IHC staining.
[0095] Tissues were immunostained for the nuclear lamina as before, but a peroxidase-based detection system was used rather than a secondary antibody tagged with a fluorophore to visualize the nuclear lamina. The different classes of extreme wrinkling presented in our last paper were still distinguishable with this stain when imaged at 60X (FIGs. 16B and 16C). Nuclei in different wrinkling classes are visible with the IHC stain.
[0096] Computer-implemented Methods and Systems
[0097] An embodiment of the computer-implemented method includes the steps of providing images of cells in a sample from a subject to a computer-implemented deep learning model; analyzing degree of wrinkling of nuclear laminar of the cells with the deep learning model to classify the cells as cancerous or non-cancerous; and communicating the classification to a user interface. These images may be obtained from subjecting the sample to one or more of epifluorescence microscopy, immunohistochemical treatment, immunofluorescence staining, and confocal fluorescent microscopy. A greater proportion of cells in the sample with extreme wrinkling of the nuclear lamina results in classification of the sample as cancerous. The extreme wrinkling includes high-frequency wrinkles, inner wrinkles, or both of the nuclear lamina. In certain embodiments, the degree of wrinkling nuclear laminar is analyzed as proportion of the cells with nuclei in each of the following classes: out-of-focus / wrongly-croppcd nuclei, smooth nuclei, nuclei with low-frequency contour waviness, nuclei with high-frequency contour waviness, and nuclei with internal wrinkles. The cancerous sample can be of any origin. In certain embodiments, the cancer is any of head and neck cancer, skin cancer, breast cancer, and thyroid cancer. Embodiments can include fluorescence microscopy or immunohistochemistry.
[0098] Another embodiment of the disclosure is directed to a system configured for detecting a cancer. The system may include a processor and a non-transitory machine readable storage to store instructions. The instructions, when executed by the processor, cause the processor to, in response to reception of a one or more lamin-stained cell images of cells of a subject, analyze the one or more lamin-stained cell images via a trained model to generate a classification to indicate a type of nuclear wrinkle corresponding to the cells of the subject. The instructions, when executed by the processor, cause the processor to, based on the classification and on subject information, determine if the cells of the subject are cancerous and a subsequent treatment.
[0099] Deep Learning Model. Turning to FIG. 17, which illustrates a system 100 to train a nuclear morphological analysis model, for the nuclear morphological analysis, a deep learning approach was implemented to classify the types of nuclear wrinkling (out of focus: invalid nuclei, no wrinkling, low frequency contour waviness, high frequency contour waviness, and interior wrinkles). Nuclei identified by the Cellpose algorithm that passed quality filters were cropped as individual images of the nuclei. A custom MATLAB code was built to annotate the five classes of the cropped nuclei from breast cancer tissue samples. These annotated images 102 served as inputs to train a multi-class classifier. A transfer learning approach was implemented using a pre-trained ResNet50 model, which was fine-tuned to the current dataset. Transfer learning allowed the utilization of the comprehensive feature-detection capabilities of ResNet50, which had been initially trained on ImageNet dataset, thereby accelerating the training process and enhancing the model accuracy with limited data. ResNet50, a convolutional neural network architecture known for its deep residual learning framework, is particularly effective in handling vanishing gradients, allowing the training of much deeper networks. This model features 50 layers, including residual blocks with skip connections to preserve gradient flow and bottleneck layers that reduce computational burden as they maintain processing depth. ResNetSO uses global average pooling to reduce overfitting and decrease the total number of parameters, enhancing its efficiency.
[0100] The training pipeline included a series of data augmentation techniques 104 to enhance model robustness and address data imbalance due to varied representations of nuclear types across samples. Transformations included resizing to 224 x 224 pixels, random horizontal and vertical flips, rotations up to 15 degrees, color jittering for brightness, contrast, and saturation adjustments, random affine transformations, and center cropping to maintain focus on the nucleus. Normalization was performed with specific mean and standard deviation values typical for pre-trained networks on ImageNet. To counter data imbalance, these augmentation techniques were combined with resampling methods to equalize class presence in the training data set 106. Building on this validated model, its applications were expanded to evaluate nuclear wrinkling types in other cancer types that were not in the initial dataset. Results were quantified using a corrected normalized count, considering individual class accuracies, providing a robust assessment of nuclear morphological variations. Training (see 108) such a model may produce a machine learning model 110, a classifier, or Al model that classifies the cells based on the type of nuclear wrinkle of the cell.
[0101] Nuclear Morphometric Analysis. Nuclear irregularity was quantified using an elliptical Fourier analysis, as reported previously. This approach approximates nuclear shapes by decomposing the shape into a scries of harmonic ellipses. The precise segmented contour was fitted using a scries of elliptic harmonics, defined by Fourier series coefficients that were calculated from the x and y coordinates of the nuclear outline. 15 harmonic ellipses were employed in this study to effectively capture the complexity of irregular nuclei while avoiding overfitting smooth nuclei, which was verified by Frechet distance calculations (FIG. 11). Each single elliptic harmonic at different frequencies can be geometrically visualized as a pair of orthogonal semiaxes. Thefirst-frequency Fourier coefficients describe a rough ellipsoidal shape, and the Fourier coefficients at higher frequencies approximate more convoluted outlines. To quantify the shape irregularity, EFC ratio is defined as the ratio of the length sum of the major and minor semiaxes at the first frequency to the sum of semiaxes lengths for the subsequent 14 harmonics at higher frequencies. A regular nuclear contour, where the first- frequency elliptic harmonic captures most of the contour with small axis lengths at higher frequencies, has a larger EFC ratio. Conversely, an irregular nuclear contour requires larger axe lengths at higher frequencies, resulting in a lower EFC ratio. Other parameters such as nuclear cross-section area A, perimeter P, solidity, and aspect ratio were quantified using the MATLAB Image Processing Toolbox. Excess perimeter was calculated byExcess perimeter = P / PCircie ~ 1 (1) where Pcircieis the perimeter of the circle and is geometrically related to A by
[0102] To quantify the level of lamin based on staining brightness, the mean pixel intensity was calculated within a peripheral area approximately 1 pm wide surrounding the nuclear contour for each nucleus.
[0103] Further Methods and Systems
[0104] FIG. 18 illustrates a system 200 to utilize one or more models to detect or diagnose cancer, monitor treatment, and / or determine subsequent treatment for a subject based on one or more received images of lamin- stained cells. The system device 202 may receive such images from one or more of an image sensor 214, a database 216 or other type of storage or machine readable storage medium, or a user interface 218. The image sensor 214 may capture images from a microscope and may, in an embodiment, transmit that image directly to system device 202. In another embodiment, an apparatus or device may include a microscope and image sensor 214 and may be in communication with the system device 202. When a sample is added to the microscope, the image sensor 214 may automatically capture an image of the cells and subsequently transmit that image to the system device 202.
[0105] The system device 202 may comprise a computing device. The system device 202 may include one or more processors 204 configured to execute instructions stored in memory 206. The memory 206 may store one or more models 208 trained to classify the type of wrinkling in the lamin-stained cells. The models 208, in an example, may determine such a classification based on a probability generated via application of the lamin- stain cell images thereto. In another embodiment, the one or more models 208 may further be configured to generate a probability or other indicator that indicates the likelihood that the lamin-stained cells are cancerous. Such models 208 may include a trained deep learning model, as described above. For example, the system device 202 or other device may utilize transfer learning to re-train a pre-existing model for the purpose of classifying the lamin-stained cell images.
[0106] The memory 206 may further include analysis instructions 210 that, when executed by the processors 204, cause the processor to determine whether the output of such models 208 indicate cancerous cells. Inembodiments, the analysis instructions 210 may compare the results from application of multiple images to the models 208 and, based on the classification or probabilities provided, may determine whether the imaged cells are cancerous. In another example, the imaged cells may originate from a subject currently diagnosed with cancer and undergoing treatment. In such embodiments, such information or indication of current treatment may be obtained from the database 216 or from a user interface 218 (for example, a user may provide such indication to the system device). The analysis instructions may further determine the efficacy of such treatments and / or the progress, in comparison to previous analysis, of treatment. In yet another example, the analysis instructions 210 may further determine subsequent treatment. Such a determination may be based on the classification of the imaged cells, as well as how many images provided such a classification. In a further embodiment, the analysis instructions 210 may use characteristics of or information related to (for example, age, sex, weight, prior illnesses, and / or other information relevant to treatment) a subject to determine treatment. For example, the system device 202 may store such data or may obtain such data from the database 216 or user interface 218. In an embodiment, treatment may include surgery, chemotherapy, radiation therapy, targeted therapy and / or immunotherapy. Further, rather than or in addition to determining treatment, the analysis instructions may determine that further testing may be performed to determine the extent and / or stage of cancer.
[0107] FIG. 19 illustrates a flowchart of a method to utilize a nuclear morphological analysis mode to determine whether a cancer prognosis, according to an embodiment of the present disclosure. The method is detailed with reference to the system device 202. Unless otherwise specified, the actions of method 500 may be completed within the system device 202 or other similar devices or systems. Specifically, method 300 may be included in one or more programs, protocols, or instructions loaded into the memory 206 of the system device 202 and executed on the processor or one or more processors 204 of the system device 202. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks may be combined in any order and / or in parallel to implement the methods.
[0108] At block 302, a user may obtain tissue or a tissue sample from a subject or patient. At block 304, the user may stain the cells of the tissue or tissue sample, as described herein. In an embodiment, the tissue or tissue sample may be treated for staining or labeling of nuclear lamina and / or cell membrane. At block 306, an image sensor may capture images of the cells. These images can be obtained from one or more of epifluorescence microscopy, immunohistochemical treatment, immunofluorescence staining, and confocal fluorescent microscopy. In another embodiment, cell images may be obtained from a database or other readable storage medium connected to a system device 202.
[0109] Once cell images are obtained, the system device 202 may apply the images to one or more models. In an embodiment, the model may be trained to classify the type of wrinkling or lack thereof of the nuclear lamina. In an embodiment, the type of model utilized may be based on the type of cell analyzed and / or the type of staining or labeling treatment applied to the cells. For example, cells from a specific organ may be analyzedby a model trained with images from those specific organs. For example, a first model may be trained in relation to lung cells and, thus, lung cell samples may be applied thereto. In another example one model may analyze all types of cells. At block 308, when the images are applied to the one or more models, each model or a model may begin by extracting features via nodes, ‘neurons’, or other modules in the trained learning model. Feature extraction may include transforming the image into a set of feature vectors.
[0110] At block 310, the feature vectors may be reduced via a max pooling layer. The max pooling layer may downsample or reduce the vectors. At block 312, feature extraction may continue via additional blocks within the model. For example, as illustrated in FIG. 3B, the model may include multiple convolutional layers, each continuing to extract features. At block 314, the features may be reduced further via an average pooling layer which groups features and averages the values for each group. Finally, a fully connected layer and softmax function may, based on the final feature set, generate a probability or probabilities. The probability or probabilities may be utilized by the system device 202 to determine a classification for the imaged cells. For example, as illustrated in FIG. 3B, the output of the model may include one of five different classifications. Based on previously determined probability thresholds, the system device 202 may set the classification as one of those five different classifications. In another embodiment, rather than a probability, the fully connected layer and the softmax function may produce the classification, which, in embodiments, may be represented by a number and / or by text.
[0111] At block 318, the system device 202 may determine whether the tissue or tissue sample has been or is currently subject to any therapy or treatment (in other words, whether the subject is currently undergoing therapy for cancer). Therapy or treatment as used herein can include physical intervention, such as surgery, phototherapy, or laser therapy, and / or biochemical intervention, such as the administration of chemotherapeutic compounds or biological agents. Such a determination may be based on input from a healthcare provider, feedback from the subject, from the clinical records database, and / or based on some indicator included in or with the image. At block 320, the system device 202 may determine an efficacy of the therapy or treatment; or may provide an assessment of the efficacy of the therapy or treatment. Such a determination may be based on the current results of the model and results from previous application of older samples from the same subject to the model. Further, other subject data may be utilized to determine the efficacy of the previous or current therapy or treatment.
[0112] At block 322, the system device 202 may determine a prognosis. Using the classification, the system device 202 may determine whether the sample is cancerous or non-cancerous or what grade of cancer or nodal involvement or a combination of the foregoing. At block 324, the system device 202 may provide recommendations for therapy or treatment based on the prognosis or classification of the tissue sample. At block 324, the system device 202 may provide recommendations for a therapy or a subsequent therapy or treatment or adjustments / alternatives to the same based on the determination of the efficacy of the therapy or treatment; or guidelines following the assessment of the efficacy of the therapy or treatment. The system device202 may utilize the prognosis, any data related to previous or current therapy or treatments, subject characteristics or information, and / or the classification of the sample, among other information relevant to the subject. For example, if the tissue has undergone a selected therapy and that therapy is determined to be treating cancer, then the determination may include continuing therapy based on effectiveness, while, in other examples, the therapy may be discontinued or a new, subsequent therapy may be initiated.
[0113] The systems, methods, and other embodiments described herein enable pathological analysis for making clinical decisions based on improved, accurate, and prompt image analysis. The use of artificial intelligence or machine learning models trained with the data described herein, enables precise decisions made based on accurate and prompt predictions and analysis that would otherwise not be possible. Further still, the systems and methods described herein increase reliability in diagnosis and therapy selection.
[0114] Integration of nuclear lamina imaging and cell membrane labeling.
[0115] Accurate grading is essential because it determines treatment strategies and predicts prognosis. However, reproducibility remains limited, even among experienced pathologists, largely due to the limitations of conventional hematoxylin and eosin (H&E) staining. H&E highlights chromatin and cytoplasmic proteins but does not delineate membrane structures, leaving nuclear boundaries and cellular contours poorly defined. As a result, critical features such as nuclear contour irregularity, nuclear-to-cytoplasmic ratio, and the relationship between nuclear shape and cell shape are difficult to assess consistently, contributing to variability and uncertainty, particularly in tumors with ambiguous histologic patterns. Cytology preparations, such as fine- needle aspiration smears commonly used in salivary gland tumors, improve visualization of nuclear and cytoplasmic details, including the nuclear-to-cytoplasmic ratio. However, they lack the architectural context needed to assess invasion, growth patterns, and perineural extension, which are essential for accurate grading and treatment planning. No existing method combines the structural clarity of cytology with the spatial integrity of histology to support reproducible diagnosis and grading in salivary gland tumors.
[0116] Embodiments include methods that combine evaluation of nuclear morphology in the context of cell shape. One such method includes integrating nuclear lamina imaging with cell membrane labeling to extract composite features, such as nuclear-to-cytoplasmic ratio, that are not accessible in H&E images. While nuclear shape is widely used as a diagnostic marker, it may be context-sensitive, particularly in inflamed or crowded tissues, where reactive nuclei can mimic malignancy. This ambiguity limits the reliability of shape-based criteria. In contrast, as shown herein, nuclear wrinkling is a morphological feature reflecting cancer-specific structural changes. As extreme wrinkling reflects an intrinsic property of malignant nuclei, it can offer greater specificity as a diagnostic criterion. Wrinkling may arise from intrinsic factors such as lamin composition or nuclear geometry, or from extrinsic influences such as cell crowding. This integrated approach of imaging nuclear and cellular contours as a combination enables a more comprehensive and interpretable morphological framework for cancer diagnosis. The approach bridges cytology and histology by preserving tissue architecture while enabling high-resolution morphological quantification and Al-based analysis. FIG. 20 is a set of imagesdemonstrating that lamin immunostaining reveals nuclear contours with superior sensitivity than H&E staining alone. FIG. 20, left panel is an image of colon car cinoma stained with H&E and imaged at 60x. FIG. 20, middle panel is an image of same location on an adjacent slide stained for pan-cytokeratin (magenta, epithelial marker) and lamin Bl (yellow) and imaged at 60x. FIG. 20, right panel is an image of lamin Bl channel alone. Scale bar is 20 pm.
[0117] Embodiments include clinical decision support software that derives information from the methods disclosed herein. Using a combination of computational feature extraction with deep learning classification, subtle morphological patterns not apparent to conventional assessment were detected, linked to tumor grade, and presented to an interface for informed pathologic decision-making. These methods can include sub-micron resolution imaging of nuclear and cellular contours. For example, by combining lamin immunostaining with optimized membrane labeling, high-resolution images of nuclear and cellular boundaries of patient tissues are obtained. Analysis of these tissues provides more robustness to the cancer diagnosis and prognosis, as this information is not accessible with H&E imaging alone. The novel segmentation algorithm disclosed herein traces intensity maxima along the surface normals to the annular staining of lamins and membrane markers, preserving fine contour details. This method enables sensitive quantification of boundary irregularities that are otherwise lost in typical segmentation algorithms. In certain embodiments, these models can be trained to categorize nuclear wrinkling and cell shape patterns, creating novel morphological descriptors of a tumor.
[0118] In certain embodiments, both lectin-based and antibody-based membrane labeling methods can be used. For example, fluorescently labeled WGA (e.g., WGA-Alexa Fluor 488) is used to label cell membranes in salivary gland tumors, given its broad glycan binding. These methods can also include a photobleaching step to reduce auto-fluorescence. FIG. 21 is an image of WGA-stained (left), lamin Bl stained (middle), and merged (right ) cells in patient breast cancer tissue. The tissue was photobleached for 48 h with LED exposure before staining to remove auto-fluorescence. WGA-stained cellular outlines and lamin Bl staining can be clearly seen. Other methods can also include concanavalin A (mannose / glucose binding) and peanut agglutinin (Gal[31-3GalNAc binding) as alternatives. Other methods can also include antibody-based agents, such as pan- cadherin, P-catenin, Na+ / K+-ATPase, and caveolin-1. Preferred staining methods are those that preserve strong and consistent lamin A / C and lamin Bl signals, instead of weakening or disrupting the lamin staining. Immunostained images can be evaluated using fluorescence microscopy at 40x and 60x magnification. These methods include one or more of quantitative evaluation steps, such as line-scan intensity profiles, cross- sectional signal continuity, and minimal non-specific fluorescence in mucinous regions.
[0119] Methods will include an automated pipeline to segment nuclear and cellular contours from the stained images collected as discussed herein. This segmentation algorithm uses a surface-normal approach originally developed for cytoskeletal filaments. At each edge point, perpendicular vectors are projected outward, and local intensity maxima along these vectors define the boundary. This approach avoids thresholding artifacts that can obscure micron-scale undulations. Segmented contours are subjected toquantitative feature extraction. For nuclear contours, metrics include area, perimeter, aspect ratio, circularity, boundary tortuosity, local curvature variation, and elliptical Fourier descriptors capturing low- and high- frequency waviness. Cellular features include area, perimeter, aspect ratio, orientation, and elliptical Fourier descriptors. The nuclear- to-cytoplasmic area ratio is derived by linking each segmented nucleus to its corresponding segmented cell. In addition to individual feature distributions, joint feature spaces are constructed to evaluate correlations between nuclear and cellular morphology. All measurements are aggregated across tumor grades and statistically compared using ANOVA with Tukey post-hoc correction. Effect sizes and confidence intervals are calculated to assess the strength of associations. Principal component analysis is used to reduce dimensionality and UMAP to visualize structure in feature space. Clustering algorithms such as k-means and Gaussian mixture models are applied to identify discrete morphological subgroups without prior grade labels. In parallel, deep learning classifiers using ResNet50 models are trained for classifying nuclear wrinkling. The classifier categorizes each nucleus into pre-defined wrinkling patterns (smooth, low-frequency waviness, high-frequency waviness, internal folds). A parallel classifier categorizes cell shapes into rounded, elongated, polygonal, or irregular classes. To assess classifier performance, cross- validation accuracy, precision, recall, and Fl scores are reported. Distributions of wrinkling categories are compared across grades using chi-square tests and logistic regression. Multinomial models evaluate the predictive value of combined nuclear and cell shape categories. The quantitative and machine learning-based framework rigorously defines and classifies nuclear and cellular morphology for evaluation of the cancer tissues and providing clinical decision support to the pathologist or other healthcare provider.
[0120] Embodiments include a method of evaluation of cancer based on cell shape, nuclear geometry (including wrinkling), and lamin composition. In certain embodiments of the methods, for each segmented cell-nucleus pair, cellular geometry parameters are measured, including aspect ratio, circularity, area, and perimeter, as well as nuclear parameters such as area, perimeter, circularity, and wrinkling category (determined using the deep learning classifier described herein). Spatial relationship metrics, such as nuclear centering within the cell and alignment of major axes, arc also computed. Multivariate regression analyses arc performed to identify cell shape features predictive of nuclear wrinkling severity, while controlling for lamin composition and excess surface area. These datasets are combined to develop a multivariate model quantifying the relative contributions of lamin composition, nuclear geometry, and cell shape to nuclear wrinkling. Using linear mixed-effects modeling, specimen-level clustering and variability across cases are also accounted.
[0121] Cell shape can control nuclear laminar wrinkling (FIG. 22), with rounded cells exhibiting increased laminar wrinkling compared to flattened cells which have smooth nuclei. As cancer cells spread more on stiff gels vs. soft gels, laminar' unfolding can be controlled by micropatterning cell shape. In an experiment, cancer cells were micropatterned on 30-micron and 50-micron circular fibronectin islands, and the cell body and nucleus were visualized by F-actin staining and GFP-lamin A, respectively. Two types of cancer cells cultured on 30-micron circular islands had wrinkled lamina vs. cells cultured on 50-micron islands, which was evidentin the x-y and y-z planes. FIG. 22 is a set of confocal image of F-actin (magenta) in fibrosarcoma (HT-1080) and head and neck cancer cells (HN) expressing GFP-LMNA (green) cultured on a 30 (top) or 50 pm (bottom) fibronectin circular micropattern. Scale bar is 10 pm. Consistent with the nuclear morphometric comparisons between soft and stiff gels, the EFC ratio was lower, and nuclei were taller on smaller islands vs. larger islands, whereas nuclear volume and surface area were similar between 30-micron and 50-micron islands. Nuclear deformation occurs at constant nuclear volume and surface area of the lamina.Examples
[0122] Various examples are describe to illustrate selected aspects of the various methods used in developing the embodiments.Methods
[0123] FFPE tissue immunostaining and imaging. Microarrays of 5-pm thick FFPE tissues (TissueArray) were deparaffinized with xylene, rinsed in ethanol, and rehydrated with a gradient of ethanol to deionized water. Heat-induced antigen retrieval was performed using a lx universal antigen retrieval solution (Abeam) in an instant pot on the high setting for 20 minutes. The tissue was rinsed and blocked with blocking buffer (3% w / v bovine serum albumin, 1% v / v goat serum, and 0.1% Triton X-100 in phosphate-buffered saline (PBS)). Samples were incubated overnight at 4°C with the primary antibodies in blocking buffer, including rabbit anti-lamin Bl (Abeam, Ab229025, diluted 1:2000), mouse anti-lamin A / C (Santa Cruz Biotech, sc- 376248, diluted 1:100), and guinea pig anti-pan-cytokeratin antibody (LS-Bio, LS-B16812, diluted 1:50). Tissue was then incubated with secondary antibodies in PBS, including goat anti-rabbit Alexa Fluor 405 (Invitrogen, A48264, diluted 1:500), goat anti-rabbit Alexa Fluor 488 (Invitrogen, Al 1034, diluted 1:500), goat anti-guinea pig Alexa Fluor 594 (Invitrogen, Al 1076, diluted 1:500), and goat anti-mouse Alexa Fluor 647 (Invitrogen, A21235, diluted 1 :500) for 1 hour at room temperature (RT). DNA was counterstained with DAPI (Thermo Fisher Scientific, diluted to 1 pg / ml) for 5 minutes. Tissues were rinsed and mounted with Diamond Antifadc mountant (Thermo Fisher Scientific) and imaged on an Olympus confocal microscope FV3000 with 20x (N.A. = 0.80) and 60x (N.A. = 1.50) objectives. Images for the lamin A / C:B1 ratio experiments were collected with a 60x objective (N.A. = 1.30) at constant laser intensity, gain, and offset settings so that imaging settings would not artificially change staining intensity between imaging fields. Brightness and contrast enhancements were applied to some images shown in the figures but not applied to those used for calculations.
[0124] FFPE tissue H&E Staining and Imaging. Tissue was deparaffinized as described above, incubated with hematoxylin for 3 minutes, rinsed, exposed to a differentiator (0.3% v / v HC1 in 70% ethanol) for 2-3 seconds, rinsed, and treated for bluing with Scott’s tap water substitute. The tissue was then treated with 80% ethanol before incubation with eosin for 10 seconds. The tissue was rinsed and treated with xylene, mountedwith DPX mountant (Sigma-Aldrich), and imaged on an Olympus confocal microscope with a 60x objective (N.A. = 1.50) and color camera (Olympus DP23).
[0125] Frozen Tissue Immunostaining. Unstained frozen tissue (OriGene Technologies) was stored at -80°C. After a brief thawing, the tissue was fixed in ice-cold acetone for 10 minutes, which was removed prior to air-drying the samples for 20 minutes. The tissue samples were rinsed, blocked, and immunostained as described above for FFPE tissue immunostaining.
[0126] Cell Culture, Staining, and Imaging. Human head and neck cancer cell line HN (Deutsche Sammlung von Mikroorganismen und Zellkulturen GmbH) was grown in a humidified incubator at 37°C and with 5% CO2 and cultured in Dulbecco’s Modified Eagle’s Medium with 4.5 g / 1 glucose (Corning), supplemented with 10% v / v donor bovine serum (Gibco) and 1% v / v penicillin / streptomycin (Corning). HN cells were rinsed with PBS before fixing with 4% paraformaldehyde (Alfa Aesar) for 15 minutes at RT. After treatment with permeabilization buffer (0.1% Triton X-100 (Thermo Fisher Scientific) and 1 mg / ml bovine serum albumin (Thermo Fisher Scientific) in PBS) for 1 hour and blocking in superblock (Thermo Fisher Scientific) for 30 minutes, HN cells were exposed to primary antibodies in superblock overnight at 4°C, including rabbit anti-lamin Bl (Abeam, Ab229025, diluted 1:500) and mouse anti-lamin A / C (Santa Cruz Biotechnology, sc-376248, diluted 1:200). Secondary staining was done at RT in PBS for 1 hour and 40 minutes with goat anti-mouse Alexa Fluor 488 (Invitrogen, A32723, diluted 1 :200) and goat anti-rabbit Alexa Fluor 647 (Invitrogen, A21244, diluted 1:200) antibodies. HN cells were imaged at 20x (N.A. = 0.80) with constant laser power settings so that imaging settings would not artificially change staining intensity between imaging fields.
[0127] MDCK cells were cultured in high glucose DMEM (Thermo Fisher Scientific), supplemented with 10% v / v Plenty (Omeat / Plenty Bio) and 1% v / v penicillin-streptomycin mix. For imaging of 2D MDCK cells, 35 mm glass bottom dishes were first coated with 1 pg / ml fibronectin (Corning) for 1 hour, washed thrice with PBS, and seeded with cells. The cells were allowed to spread and fixed for imaging. For 3D acinar cultures, Nunc Lab Tck II 8-wcll chamber slides (Thermo Fisher Scientific, #155409) were coated with 15 pl of growth- factor-reduced (GFR) Matrigel (Corning) in each well and allowed to polymerize for at least 1 hour at 37°C. Then, the MDCK cells were trypsinized from tissue culture plates and suspended in the growth medium supplemented with 2% v / v Matrigel at a final concentration of 5 cells / l. 400 pl aliquot of cells was added to each well of the chamber slide, and the cells were allowed to form acini for 7-12 days before fixation. The growth medium was changed every 3-5 days. 2D and 3D acinar MDCK samples were fixed with warm 2% paraformaldehyde for 10-15 minutes at 37°C and washed thrice with PBS for 5 minutes each. The cells were permeabilized using 0.5% Triton X-100 in PBS for 30 minutes at RT, followed by 1-hour incubation with an immunofluorescence buffer (130 mM NaCl; 7 mM Na2HPC>4; 3.5 mM NaH2POr; 7.7 mM NaNu 0.1% BSA; 0.2% Triton X-100; 0.05% Tween-20) supplemented with 10% goat serum at RT. The samples were incubated with rabbit anti-lamin A / C primary antibody (Abeam, diluted 1:1000) overnight at 4°C, washed thrice withPBS, and incubated with goat anti-rabbit Alexa Fluor 594 secondary antibody (Abeam, diluted 1:000) for 2 hours at RT. Fixed-cell fluorescence imaging of 2D MDCK cells was performed using an Olympus FV3000 confocal microscope at 60x (N.A. = 1.5), whereas acini were imaged with an ImageXpress Ht.ai spinning disk confocal microscope (Molecular Devices) at 40x (N.A. = 1.15) with a 0.1 pm step size between z-slices.
[0128] Transfection with siRNAs. Depletion of lamin A / C level was performed using siRNA transfection according to the manufacturer’s protocol (Invitrogen). HN cells were cultured in a 12-well plate in antibiotic- free media at transfection with 0.5% lipofectamine RNAiMAX transfection reagent (Invitrogen) and 0.5% siRNA (Dharmacon, siGENOME Non-Targeting siRNA Pool #2, D-001206- 14-05, target sequences: UAAGGCUAUGAAGAGAUAC, AUGUAUUGGCC UGUAUUAG, AUGAACGUGAAUUGCUCAA, UGGUUUACAU GUCGACUAA; LMNA siGENOME SMARTpool siRNA, D-004978-01, target sequence: GAAGGAGGGUGAC CUGAUA) in the reduced serum Opti-MEM medium (Gibco). After 96 hours, transfected cells were passed onto fibronectin-coated dishes and allowed to spread overnight before fixation the next day.
[0129] Segmentation of nuclear contour in tissue array images. Raw confocal images of tissue arrays were segmented using Cellpose for the initial identification of nuclei, except for control ovarian tissue, where high background and few epithelial nuclei necessitated manual generation of Cellpose masks by tracing around the nuclear contour. The generated nuclear masks and the raw images were imported into MATLAB, where a customized MATLAB code was developed for precise segmentation and subsequent nuclear morphometric analysis. Nuclei touching the image border or below an empirically determined area threshold were eliminated to remove small debris or imaging artifacts. Since the bulk masks generated from Cellpose did not capture details like folds and wrinkles on nuclear contours, a more precise segmentation approach was employed that the intensity maxima were traced on each normal line along the bulk nuclear periphery, achieving sub-pixel resolution for delineating precise nuclear contours. Following precise segmentation, additional filters were applied to refine the selection of nuclei of interest. A contrast filter calculated the lamin intensity ratio between the maximum pixels and the pixels covered by normal lines for each nucleus. Nuclei with a contrast ratio below 1.5, indicating a blurry contour, were excluded. Similarly, a pan-cytokeratin filter calculated the pan- cytokeratin intensity ratio between the ring area outside the nucleus vs. inside. Nuclei with a pan-cytokeratin ratio below 1. indicative of non-epithelial cells, were also excluded.
[0130] Comparing corrected normalized counts (proportions) of nuclear wrinkling categories across cancer grades and cancer sites. To compar e the proportions of nuclei belonging to four different wrinkling categories (smooth, low-frequency, high-frequency, and inner wrinkles as determined by the deep learning classifier) across different cancer grades and sites, a series of multi-class multinomial logistic regression models was conducted. A separate model was developed for each cancer site. These models used the expected / corrected counts (raw counts determined by the deep learning classifier multiplied by the probability of correct classification) of nuclei in the four wrinkling categories as multivariate responses, with cell types(cancer grades and adjacent cells) serving as explanatory variables. The models were fitted using the method of maximum likelihood. From each fitted model, (a) the probabilities (proportions) of each nuclear wrinkling category was estimated for adjacent and various cancer grades, along with their 95% confidence intervals, and (b) the differences between the probabilities of each wrinkling category was formally tested across different cancer grades using marginal probability contrasts. To account for the multiplicity of hypothesis tests, the Benjamini-Hochberg false discovery rate adjustment was applied. The results are presented in FIG. 3, which displays the estimated probabilities of each wrinkling category (normalized corrected counts) with their 95% confidence intervals represented by vertical bars with error whiskers. The corresponding pairwise test results are visualized through horizontal lines.
[0131] Comparing the distributions of nuclear EFC ratios and areas for different cell types.
[0132] To compare the distributions of nuclear EFC ratios and areas for different cell types, kernel density estimates were obtained using default Gaussian kernels as implemented in the density function in R for the nuclear EFC ratios and areas, separately for each cell type — control, BCC (skin), DCIS (breast), serous and mucinous (ovary), and cancer — with individual grades in one set of analysis and all grades combined in another. To address potential imbalances in the number of nuclei per sample and their impact on the final KDE plots, as well as on the means and scales comparisons, random data subsampling was applied to ensure homogeneity in the number of nuclei per sample in each random subset. Specifically, Nnuciei distinct nuclei were randomly selected from the set of all imaged nuclei per sample, separately for each tissue type. A total of R = 100 random data subsets were generated for each tissue type and the common number Nnuciei of nuclei in each subset was set to the minimum number of nuclei imaged across all samples within that tissue type. KDE analysis was performed for the EFC ratios and nuclear areas separately for the adjacent, pooled cancer, and grade- specific cancer cells, and separately in each random data subset. Comparisons of the means and scales for the nuclear measurements were conducted using the Kruskal-Wallis test and the Fligner test, respectively, for both adjacent vs. pooled cancer cells (two-sample) comparisons and adjacent vs. grade- specific cancer cells (multi-sample) comparisons. The estimated kernel densities from the R = 100 random subsets were combined by averaging (i.e., mixing the densities with equal weights). The computed test statistics obtained across the random replicates were also combined via averaging (meta-analysis), and the resulting approximate chi-squared p-values were derived from the averaged test statistics. This analysis was carried out separately for each metric (EFC ratio; nuclear area), measure (mean; scale), comparison (adjacent vs. pooled cancer; adjacent vs. grade- specific cancers), and tissue combination. Multiplicities of the hypotheses tested were acknowledged via the Benjamini- Hochberg false discovery rate adjustment procedure, and the adjusted p-values are reported in FIG. 4.
[0133] Posterior probability analysis for cancer grades given nuclear measurements.
[0134] To assess cancer grade-specific discriminative information embedded in the nuclear wrinkle measurements, a posterior probability analysis was performed based on linear discriminants separately for each cancer site and nuclear morphometric measurements (EFC ratio, area, and aspect ratio). With a log-normal probability distribution assumption — effectuated by a normality assumption on the log-transformed measurements — made separately for each of the three nuclear morphometric measurements at each cancer site, a linear discriminant analysis was performed to obtain the discrimination boundaries for adjacent, BCC (skin), DCIS (breast), individual cancer grade cells and all cancer cells combined (grouping all individual grades). In the absence of reliable population-level (prior) estimates for the prevalence / proportions of the different cancer grades, a flat / uniform probability distribution allocating equal weights to the different cancer grades was assumed. The flat prior provides a way to assess the discriminative information solely in the nuclear morphometry measurements by invoking a theoretical population where the different cancer grades and adjacent cells are equally prevalent. From the computed linear discriminants, the resulting posterior probabilities for a nucleus to be one of the different cell types — adjacent or one of the different cancer grades — were obtained for a range of values of the underlying nuclear- morphometric measurement. The resulting posterior probabilities were plotted on the y-axis as a function of the nuclear morphometric measurement on the x-axis, yielding curves that were color-coded by the cell type (FIGs. 7A - 7F).
[0135] Other objects, features and advantages of the disclosure will become apparent from the foregoing figures, detailed description, and examples. It should be understood, however, that the figures, detailed description, and examples, while indicating specific embodiments of the disclosure, are given by way of illustration only and are not meant to be limiting. Additionally, it is contemplated that changes and modifications within the spirit and scope of the disclosure will become apparent to those skilled in the art from the detailed description. In further embodiments, features from specific embodiments may be combined with features from other embodiments. For example, features from one embodiment may be combined with features from any of the other embodiments. In further embodiments, additional features may be added to the specific embodiments described herein.
Claims
ClaimsWhat is claimed is:
1. A computer-implemented method, the method comprising: providing a lamin-stained image of cells in a sample from a subject to a computer-implemented deep learning model; analyzing a degree of wrinkling of nuclear laminar of the cells with the deep learning model to generate a classification of the cells as cancerous or non-cancerous; and communicating the classification to a user interface.
2. The method of claim 1 , wherein a greater proportion of cells in the sample with extreme wrinkling of the nuclear lamina results in classification of the cells as cancerous.
3. The method of claim 2, wherein extreme wrinkling of the nuclear lamina comprises high-frequency wrinkles, inner wrinkles, or both of the nuclear lamina.
4. The method of claim 1 , wherein the degree of wrinkling nuclear laminar is analyzed as proportion of the cells with nucleic in one or more of the following classes: out-of-focus / wrongly-cropped nuclei, smooth nuclei, nuclei with low-frequency contour waviness, nuclei with high-frequency contour waviness, and nuclei internal wrinkles.
5. The method of claim 1 , wherein a cancerous cell is one of head and neck, skin, breast, and thyroid cancer.
6. The method of claim 1, where the cells are further classified into a particular grade of cancer.
7. The method of claim 1, further comprising: determining a treatment based on the classification and based on a subject's characteristics, wherein the treatment comprises one or more of surgery, chemotherapy, radiation therapy, targeted therapy and / or immunotherapy.
8. The method of claim 1 , further comprising: providing an indication of one or more types of cancer treatment being applied to the cells, and determining treatment efficacy based on the classification and based on a plurality of previously classified images of cells.
9. A system configured for detecting a cancer, the system comprising: a processor, and a non-transitory machine -readable storage to store instructions that, when executed by the processor, cause the processor to: in response to reception of a one or more lamin-stained cell images of cells of a subject, analyze the one or more lamin-stained cell images via a trained model to generate a classification to indicate a type of nuclear wrinkle corresponding to the cells of the subject, and based on the classification and on subject information, determine if the cells of the subject are cancerous and a subsequent treatment.
10. The system of claim 9, wherein the instructions, when executed, further cause the processor to, in response to reception of information indicative of subject treatment, determine efficacy of the treatment.
11. The system of claim 9, wherein the trained model comprises a deep learning model trained with annotated and augmented images of cells with varying types of nuclear wrinkles.
12. The system of claim 11, wherein the deep learning model, prior to training, comprises a pre-trained model trained with another data set.
13. The system of claim 11, wherein augmentation includes resizing pixels, random horizontal and vertical flips, rotations up to 15 degrees, color jittering adjustments, contrast adjustments, saturation adjustments, random affine transformations, and center cropping.
14. The system of claim 9, wherein the type of nuclear wrinkle comprises one of out-of-focus / wrongly- cropped nuclei, smooth nuclei, nuclei with low-frequency contour waviness, nuclei with high-frequency contour waviness, or nuclei internal wrinkles.