Pathological image analysis method and system

Abstract

The invention provides a pathological image analysis method and system, and the method comprises the steps: collecting a clinical tissue sample, carrying out the dispersed preparation of the clinical tissue sample, and obtaining a to-be-processed sample; performing hyperspectral imaging on the cell sample and extracting a region of interest to obtain a region of interest image; on the basis of an electromagnetic theory and a morphological theory, feature extraction is carried out on the region-of-interest image, and the extracted features are fused to obtain fused features; and based on deep learning, predicting a cell sample through the fusion features to obtain an analysis result. According to the method, rapid pathological classification of the fresh tissue cells and evaluation and prediction of the treatment effect can be realized by applying an electromagnetic-morphology dual-mode intelligent analysis technology of hyperspectral imaging, and the detection efficiency and precision can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of pathological analysis and diagnosis, and in particular to a pathological image analysis method and system. Background Art

[0002] In medical and biological research, rapid diagnosis of pathological samples and long-term online observation and analysis of fresh tissue cell samples in situ are crucial. However, existing technologies still face certain challenges in achieving these requirements.

[0003] Rapid intraoperative pathological diagnosis refers to the rapid acquisition of diagnostic results of pathological sections during surgery to assist surgeons in making real-time surgical plan decisions. This is crucial for clinical decisions such as determining tumor boundaries and evaluating whether the resection is thorough. At present, rapid intraoperative pathological diagnosis mainly relies on frozen section technology and rapid staining technology. However, these methods have certain limitations: frozen sections must be prepared with the help of expensive large instruments and equipment, and their production and staining processes need to be completed in a short time. The operation is complicated and prone to human errors; the preparation of frozen sections is prone to ice crystal artifacts, which affects image quality and diagnostic accuracy; pathological diagnosis is highly dependent on the experience and expertise of pathologists, and is subjective and inconsistent.

[0004] In the field of cell biology and drug development, long-term observation and analysis of the dynamic behavior and drug effects of living cells, tissues, and organoids is an important research method. Traditional long-term live cell imaging mainly relies on fluorescence microscopy, which has many problems: first, phototoxicity and photobleaching. Long-term fluorescence excitation can cause phototoxicity of cells and attenuation of fluorescence signals, affecting the physiological state of cells and the continuity of imaging; second, labeling has limitations. Fluorescent labeling may change the physiological state of cells, and the labeling process is complicated and costly; finally, multi-color imaging is also challenging. Multi-color fluorescence imaging technology has the problem of spectral overlap, which makes it difficult to accurately distinguish and quantitatively analyze multiple markers. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the related art. To this end, the present invention provides a pathological image analysis method and system.

[0006] The present invention provides a pathological image analysis method, comprising:

[0007] S1: collecting clinical tissue samples, and dispersing and slicing the clinical tissue samples to obtain samples to be processed;

[0008] S2: performing hyperspectral imaging on the sample to be processed and extracting the region of interest to obtain an image of the region of interest;

[0009] S3: Based on electromagnetic theory and morphological theory, feature extraction is performed on the image of the region of interest respectively and the extracted features are fused to obtain fused features;

[0010] S4: Based on deep learning, the samples to be processed are predicted by using the fusion features to obtain analysis results.

[0011] According to a pathological image analysis method provided by the present invention, the sample to be processed in step S1 includes a thin layer cell sample, a fresh cell sample, and an organoid sample, and the step of obtaining the thin layer cell sample further includes:

[0012] S11: collecting clinical tissue samples, and physically fragmenting the clinical tissue samples to obtain fragmented samples;

[0013] S12: retaining single cells in the fragmented sample through cell screening to obtain a single cell sample;

[0014] S13: performing high-speed centrifugal sedimentation on the single cell sample to obtain a thin layer cell sample of single cell dispersion.

[0015] According to a pathological image analysis method provided by the present invention, in step S2, after performing hyperspectral imaging on the sample to be processed, the method further comprises:

[0016] S21: preprocessing the hyperspectral image obtained by hyperspectral imaging, wherein the preprocessing includes data cleaning, data alignment, data standard normal variable exchange, data normalization, data smoothing, data feature enhancement and data splicing.

[0017] According to a pathological image analysis method provided by the present invention, in step S2, the region of interest extraction of the hyperspectral image of the hyperspectral imaging is achieved by an image segmentation model.

[0018] According to a pathological image analysis method provided by the present invention, in step S3, feature extraction of the image of the region of interest based on electromagnetic theory is achieved through Mie scattering theory, and the electromagnetic features extracted based on electromagnetic theory are particle features contained in the spectral data in the hyperspectral image, and the particle features include particle size features and particle density distribution features.

[0019] According to a pathological image analysis method provided by the present invention, in step S3, the morphological features of the region of interest image extracted based on morphological theory include color features, shape features, texture features, edge features and depth morphological features of the region of interest.

[0020] According to a pathological image analysis method provided by the present invention, the expression of the fusion feature in step S3 is:

[0021]

[0022] Among them, Z contact is the fusion feature, B is the total number of electromagnetic features, b is the electromagnetic feature index value, X b is the bth electromagnetic feature, K b is the weight value corresponding to the bth electromagnetic feature, C is the total number of morphological features, c is the morphological feature index value, and Y c is the cth morphological feature, K c is the weight value corresponding to the cth morphological feature.

[0023] According to a pathological image analysis method provided by the present invention, step S4 further comprises:

[0024] S41: Preset sample type and disease category;

[0025] S42: Pre-training the sample type and the disease category to obtain a classifier;

[0026] S43: Activate and optimize the classifier by adjusting parameters, and input the fusion features into the activated and optimized classifier to obtain analysis results to complete pathological analysis.

[0027] The present invention also provides a pathological image analysis system, comprising:

[0028] A sample preparation device, a hyperspectral imaging device and a pathological image analysis device, wherein the hyperspectral imaging device is communicatively connected with the pathological image analysis device;

[0029] The sample preparation device is used to prepare a sample to be processed of a clinical tissue sample, and the hyperspectral imaging device is used to perform hyperspectral imaging on the sample to be processed to generate hyperspectral image data of the sample to be processed and transmit the hyperspectral image data to the pathological image analysis device;

[0030] The pathological image analysis device comprises a memory, a processor and a communication interface. The memory stores a computer program that can be run on the processor. The pathological image analysis device is configured to receive hyperspectral image data transmitted from the hyperspectral imaging device through the communication interface and send it to the memory for storage. When the processor executes the computer program, a pathological image analysis method as described in any one of the above items is implemented.

[0031] According to a pathological image analysis system provided by the present invention, the sample preparation device is configured as a sample acquisition module, which is used to collect clinical tissue samples, and use a single cell dispersion preparation unit to disperse and prepare the clinical tissue samples to obtain thin layer cell samples, and is also used to collect clinical tissue samples, and use a tissue cell organoid chip and a multifunctional micro-culture unit to perform primary cell culture or organoid culture on the clinical tissue samples to obtain fresh cell samples or organoid samples; the single cell dispersion preparation unit is used to make thin layer tissue cell slides, and the tissue cell organoid chip and the multifunctional micro-culture unit are used to maintain the activity of fresh cells and organoids, so as to perform organoid cell chip detection and analysis;

[0032] The hyperspectral imaging device is configured as an image acquisition and processing module, which is used to perform hyperspectral imaging on the sample to be processed and extract the region of interest to obtain an image of the region of interest;

[0033] The hyperspectral imaging device comprises a light source, an aspheric condenser, a beam splitter, an objective lens and a focusing control module, a tube lens, a narrow slit diaphragm and a one-dimensional motion control module, a collimator, a beam splitter, an imaging lens, a CCD detector, a two-dimensional motion platform, and a microprocessor controller, which constitute an incident illumination light path and an imaging detection light path, wherein the beam splitter is configured as a grating, a prism, a spatial light modulator or a color filter switch;

[0034] The pathological image analysis device further comprises:

[0035] The feature extraction module is used to extract features from the hyperspectral image based on electromagnetic theory and morphological theory and fuse the extracted features to obtain fused features;

[0036] Analysis module: used to perform pathological analysis on the sample to be processed based on deep learning and through the fusion features to obtain analysis results.

[0037] A pathological image analysis method and system provided by the present invention adds a spectral modality to the collected data through a hyperspectral imaging method, contains a larger amount of information, and can provide richer sample information. Compared with using a spectrometer to collect average spectral information in the field of view, it increases the morphological information of the sample, and the morphology of each pixel corresponds to the spectral information one by one, which can perform better analysis of the sample at the tissue, single cell and even subcellular levels.

[0038] In addition, the present invention uses a hyperspectral imaging control system that includes hardware control, algorithm call and human-computer interaction, so the user only needs to set necessary information such as the sample type, which greatly reduces the demand for the operator's professional and technical level; in terms of applicable fields, the present invention is suitable for a variety of biomedical samples such as fixed tissue sections, frozen tissue sections, cytological smears, live adherent cells, live suspended cells, three-dimensional cultured cell balls, etc., which expands the use scenarios while reducing the application cost; secondly, the present invention can be applied to various unlabeled samples, reducing the reagent cost and time cost required for staining or fluorescent labeling, greatly improving the detection efficiency, reducing the difficulty of technical operation, and avoiding the impact of staining and fluorescent labeling on the sample as well as phototoxicity and photobleaching. The present invention can also be applied to stained or fluorescently labeled samples. Since hyperspectral imaging can obtain continuous spectral information of each pixel point after sample imaging, there is no problem of spectral passband overlap faced by traditional technical solutions such as color filters, which can greatly expand the number of colors used in staining or fluorescent labeling, and obtain more target structure and function detection information more efficiently.

[0039] The present invention adopts electromagnetic-morphological dual-mode intelligent analysis technology and uses electromagnetic theory based on the propagation and scattering characteristics of light to calculate and analyze hyperspectral data. Compared with the current black-box hyperspectral data processing solutions, it has higher reliability and interpretability, and has broader application prospects in the fields of life science and medical cutting-edge scientific research, clinical medicine, health and epidemic prevention, food safety, etc.

[0040] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a schematic diagram of a pathological image analysis method provided by an embodiment of the present invention;

[0043] Figure 2 is a schematic diagram of the structure of a pathological image analysis system provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of the structure of a tissue cell organoid chip in a pathological image analysis system provided by an embodiment of the present invention;

[0045] Figure 4 It is a schematic diagram of the optical path structure of a hyperspectral imaging device in a pathological image analysis system provided by an embodiment of the present invention.

[0046] Reference numerals:

[0047] 100. Sample acquisition module; 110. Single cell dispersion preparation unit; 120. Tissue cell organoid chip; 130. Multifunctional micro-culture unit; 200. Image acquisition and processing module; 300. Feature extraction module; 400. Analysis module. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. The following embodiments are used to illustrate the present invention, but cannot be used to limit the scope of the present invention.

[0049] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0050] In order to better understand the present invention, the terms appearing in the embodiments of the present invention are first explained below.

[0051] Hyperspectral imaging technology can provide richer spectral data than traditional imaging technology by acquiring spectral information of each pixel. This makes hyperspectral imaging show unique advantages in rapid pathological diagnosis and long-term observation and analysis of living cells: first, hyperspectral imaging does not require the use of fluorescent labels, and can directly obtain the spectral information of the sample, avoiding the phototoxicity and photobleaching problems in the labeling process; second, hyperspectral imaging can provide high-resolution and high-sensitivity data in both space and spectrum, which helps to accurately distinguish and analyze subtle changes in cells and tissues; in addition, by analyzing spectral information in different bands, hyperspectral imaging can achieve simultaneous detection and analysis of multiple parameters, providing more comprehensive biological information.

[0052] Combine the following Figures 1 to 4 Embodiments of the present invention are described.

[0053] like Figure 1 As shown, the present invention provides a pathological image analysis method, comprising:

[0054] S1: Collect clinical tissue samples, and disperse and slice the clinical tissue samples to obtain samples to be processed.

[0055] Wherein, the sample to be processed in step S1 includes a thin layer cell sample, a fresh cell sample, and an organoid sample, and the step of obtaining the thin layer cell sample further includes:

[0056] S11: collecting clinical tissue samples, and physically fragmenting the clinical tissue samples to obtain fragmented samples;

[0057] S12: retaining single cells in the fragmented sample through cell screening to obtain a single cell sample;

[0058] S13: performing high-speed centrifugal sedimentation on the single cell sample to obtain a thin layer cell sample of single cell dispersion.

[0059] Furthermore, the specific steps are to first use single-cell dispersion preparation technology, that is, use any one or more physical fragmentation methods such as cutting, grinding, vibration, ultrasound, etc. to convert the minimally invasive sampled tissue into a fragmented suspension at room temperature or low temperature, remove cell fragments and cell clusters through a cell sieve, retain intact single cells, and then use high-speed centrifugal sedimentation to make a single-cell dispersed thin layer tissue cell slide or organoid chip.

[0060] S2: Performing hyperspectral imaging on the sample to be processed and extracting the region of interest to obtain an image of the region of interest.

[0061] Wherein, in step S2, after performing hyperspectral imaging on the sample to be processed, the method further includes:

[0062] S21: preprocessing the hyperspectral image obtained by hyperspectral imaging, wherein the preprocessing includes data cleaning, data alignment, data standard normal variable exchange, data normalization, data smoothing, data feature enhancement and data splicing.

[0063] Furthermore, after the image is collected, the hyperspectral imaging data is first subjected to one or more of the following data preprocessing: (1) data cleaning, screening the collected hyperspectral data to exclude data of poor quality due to overexposure, low signal-to-noise ratio, or uncontrollable external interference; (2) data alignment, using the data of the sample area and the spectral data of the non-sample area to perform a difference or ratio transformation to avoid interference of the collected hyperspectral data caused by differences in the optical properties of sample carrier materials such as different chips, culture dishes, and well plates; (3) data standard normal variable exchange, Gaussian standardization of the spectrum of each point to eliminate the offset effect between spectra; (4) data normalization , scaling the spectral data to a specific range so that different data can be compared or analyzed at the same scale; (5) data smoothing, using moving average smoothing, Savitzky-Golay convolution smoothing and other methods to reduce data noise and interference while retaining the overall trend characteristics of the spectral data; (6) data feature enhancement, using first-order derivative and other methods to enhance the peak and valley information in the spectral data, making the local spectral features more obvious; (7) data stitching, when the target field of view cannot be completed through one imaging, the images obtained by multiple imaging are stitched together through corner point matching, feature point association and other methods to complete large-field tissue cell scanning detection.

[0064] In a specific embodiment, when preprocessing the hyperspectral imaging data, data cleaning is first performed to exclude data of poor quality due to overexposure, low signal-to-noise ratio, and uncontrollable external interference; then, data alignment is performed on the hyperspectral imaging data with data cleaning, and the data of the sample area is compared with the spectral data of the sample-free area to avoid interference of the collected hyperspectral data due to differences in the optical properties of sample carrier materials such as different glass slides and culture dishes; secondly, data splicing is performed on the hyperspectral data after data alignment, and the Harris corner detection algorithm is first used for corner matching, and the autocorrelation matrix of the midpoint (x, y) of the collected image is calculated, and then each pixel is Gaussian filtered to obtain the update matrix M, and then the corner metric of the image point (x, y) is calculated, and then the local maximum value of the image is selected for comparison with the set threshold. If the local maximum value is higher than the threshold, it is a corner point, and then the mutual information method is used to associate the corner points with feature points, thereby realizing data splicing of images obtained by multiple imaging.

[0065] Wherein, in step S2, the region of interest extraction of the hyperspectral image of the hyperspectral imaging is achieved by an image segmentation model based on deep learning.

[0066] Furthermore, before feature extraction, it is necessary to segment the hyperspectral imaging data and extract the regions of interest using methods based on threshold, regional method, edge recognition, graph theory, energy functional, wavelet transform, machine learning, deep learning, etc.

[0067] In a specific embodiment, after data preprocessing, the hyperspectral imaging data is segmented and the region of interest is extracted using the SegmentAnything Model (SAM) method based on deep learning. SAM is a comprehensive image segmentation model that combines the advantages of interactive segmentation and automatic segmentation and can handle various complex segmentation tasks. The model achieves strong zero-sample generalization capabilities through pre-training and prompt engineering, that is, it can effectively segment object categories that have not been seen in the training phase.

[0068] S3: Based on electromagnetic theory and morphological theory, feature extraction is performed on the image of the region of interest respectively and the extracted features are fused to obtain fused features.

[0069] Among them, in step S3, the feature extraction of the image of the region of interest based on electromagnetic theory is realized through Mie scattering theory, and the electromagnetic features of the hyperspectral image extracted based on electromagnetic theory are particle features contained in the spectral data in the hyperspectral image, and the particle features include particle size features and particle density distribution features.

[0070] When extracting electromagnetic features, the present invention uses electromagnetic theories of light propagation, interference, and diffraction, or one or more electromagnetic derivation theories including first-order Born approximation, multi-layer Born scattering, dielectric film theory, electromagnetic wave scattering optical cross-section theorem, scattering potential reconstruction diffraction tomography, optical coherence tomography, scattering matrix, Mie scattering, Rayleigh scattering, Raman scattering, etc., to analyze the data of the region of interest of hyperspectral imaging, so as to obtain the particle size, density, distribution, dry mass and other characteristics of thin cells, tissues, and organoids at the subcellular level of single cells or cell nuclei, organelles, vesicles, etc.

[0071] In a specific embodiment, the present invention obtains the particle size and density distribution characteristics contained in the sample by inverting spectral data based on Mie scattering theory. Mie scattering is an optical scattering phenomenon, which mainly occurs when the diameter a of the scattering particles is near the wavelength λ of the incident light (0.1λ<a<50λ). For visible light with a wavelength of 400nm-700nm, the scattering of particles with a diameter of 0.04-35 microns all follows the Mie scattering effect. Therefore, by collecting the scattering intensity of the sample for incident light of different wavelengths, that is, the scattering spectrum of the sample, the internal structure and composition information of the sample can be estimated by numerical methods to achieve electromagnetic feature extraction.

[0072] Wherein, in step S3, the morphological features of the interest region image extracted based on morphological theory include color features, shape features, texture features, edge features and depth morphological features of the interest region.

[0073] As for the feature extraction of morphological theory, the present invention uses direct measurement and calculation methods to obtain the color features, shape features, texture features, and edge features of the hyperspectral imaging area of ​​interest, or uses machine learning and deep learning methods to obtain other morphological features of the hyperspectral imaging area of ​​interest.

[0074] The expression of the fusion feature in step S3 is:

[0075]

[0076] Among them, Z contact is the fusion feature, B is the total number of electromagnetic features, b is the electromagnetic feature index value, X b is the bth electromagnetic feature, K b is the weight value corresponding to the bth electromagnetic feature, C is the total number of morphological features, c is the morphological feature index value, and Y c is the cth morphological feature, K c is the weight value corresponding to the cth morphological feature.

[0077] S4: Based on deep learning, pathological analysis is performed on the sample to be processed through the fusion features to obtain analysis results.

[0078] In this stage, the fused features are used by machine learning or deep learning methods, preferably deep learning methods, to achieve pathological classification of clinical tissue samples in hyperspectral images, assessment of benign and malignant tumors, and prediction of drug treatment effects.

[0079] The above-mentioned machine learning includes but is not limited to one or more traditional machine learning methods including support vector machine, decision tree, random forest, logistic regression, principal component analysis, K nearest neighbor, and Bayesian method; deep learning methods include but are not limited to one or more deep learning methods including convolutional neural network, recurrent neural network, natural language processing, long short-term memory network, autoencoder, generative adversarial network, clustering, self-attention mechanism, reinforcement learning, and transfer learning.

[0080] Wherein, step S4 further comprises:

[0081] S41: Preset sample type and disease category;

[0082] S42: Pre-training the sample type and the disease category to obtain a classifier;

[0083] S43: Activate and optimize the classifier by adjusting parameters, and input the fusion features into the activated and optimized classifier to obtain analysis results to complete pathological analysis.

[0084] Furthermore, in this stage, a classifier pre-trained with a BP neural network is used to predict the category to which the sample belongs. The BP neural network is a multi-layer feedforward neural network trained according to the error back propagation algorithm. When the fusion features obtained by the sample type selected by the user have z features and the selected disease has p categories, z input nodes and p output nodes are set during the setting and operation of the BP neural network classifier. After parameter adjustment and optimization, the classifier can be predicted until the effect is optimal. Finally, a report is issued based on the prediction results of the classifier.

[0085] The present invention also provides a pathological image analysis system, comprising:

[0086] A sample preparation device, a hyperspectral imaging device and a pathological image analysis device, wherein the hyperspectral imaging device is communicatively connected with the pathological image analysis device;

[0087] The sample preparation device is used to prepare a sample to be processed of a clinical tissue sample, and the hyperspectral imaging device is used to perform hyperspectral imaging on the sample to be processed to generate hyperspectral image data of the sample to be processed and transmit the hyperspectral image data to the pathological image analysis device;

[0088] The pathological image analysis device comprises a memory, a processor and a communication interface, the memory stores a computer program that can be run on the processor, the pathological image analysis device is configured to receive the hyperspectral image data transmitted from the hyperspectral imaging device through the communication interface and send it to the memory for storage, and the processor implements a pathological image analysis method as described in any one of the above when executing the computer program. Figure 2 As shown, according to a pathological image analysis system provided by the present invention, the cell sample preparation device is configured as a sample acquisition module 100, which is used to collect clinical tissue samples and also used to disperse and slice the clinical tissue samples to obtain thin-layer cell samples; the sample acquisition module 100 includes a single-cell dispersion slice making unit 110, a tissue cell organoid chip 120, and a multifunctional micro-culture unit 130, the single-cell dispersion slice making unit 110 is used to make a thin-layer tissue cell slide, the tissue cell organoid chip 120 and the multifunctional micro-culture unit 130 are used to maintain the activity of tissue cells and organoids, so as to perform organoid cell chip detection and analysis;

[0089] Furthermore, the above-mentioned single-cell dispersion preparation unit includes a fragmentation module, a suspension collection cavity cartridge, a cell sieve, a motor and a controller, etc.; the fragmentation module adopts physical fragmentation methods such as cutting, grinding, vibration, and ultrasound to fragment the minimally invasive sampled tissue at room temperature or low temperature to achieve a single-cell particle size level; the cell sieve is used to remove cell fragments and cell clusters in the sample after the fragmentation treatment to obtain a single-cell sample; the suspension collection cavity cartridge is loaded with a certain tissue cell centrifugal fixative according to specific usage requirements, which is used to load single-cell samples and keep the tissue cells evenly dispersed by shaking; the bottom side of the suspension collection cavity cartridge is the insertion interface of the thin-layer tissue cell slide or organoid chip, and the surrounding contains absorbent material; the motor and controller are used to centrifuge the single cells evenly dispersed in the fixative in the suspension collection cavity cartridge at high speed onto the thin-layer tissue cell slide or organoid chip, and centrifuge the excess culture preservation buffer to the absorbent material to be absorbed.

[0090] like Figure 3 As shown, a tissue cell organoid chip provided in this embodiment uses PDMS as a substrate, with a rectangular shallow pit in the middle, and a plurality of deeper micro chambers in the shallow pit. A tissue sample suspension inlet, a 5% CO2 gas inlet, a culture medium inlet, a drug solution inlet, a gas outlet, a liquid outlet and supporting micro channels are distributed around the shallow pit. The diameter, depth and surface treatment of the supporting micro channels of different interfaces are different. A waterproof and breathable film is attached to the gas outlet, and a polycarbonate cover sheet with holes punched at each gas path interface and flow path interface is placed on the chip.

[0091] Furthermore, in the clinical tissue sample rapid pathology online detection system, the multifunctional micro-culture module includes a loading platform, a humidification chamber, a heating module, a connecting pipe, a flow path interface, a temperature and humidity sensor, a CO2 gas sensor, etc.;

[0092] The loading platform is in the center, and has a light hole in the center. The platform can load and fix the tissue cell organoid chip or install culture dishes and well plates of different sizes, such as Φ35mm-100mm culture dishes or 6-well-384-well plates, etc.; and is compatible with the detection of live adherent cells, suspended cells, three-dimensional cultured cell sphere samples, tissue cell organoids, microorganisms, pathogens, model organisms, etc. in these carriers;

[0093] The humidification cavity, temperature and humidity sensor, and CO2 gas sensor are distributed around the loading platform;

[0094] The heating modules are distributed on the upper and lower surfaces of the loading platform, and the upper surface is heated by transparent ITO glass, which plays a role of packaging and light transmission, and the lower surface heating can be ITO glass, resistance wire, semiconductor refrigerator, etc.

[0095] The flow path interface is composed of multiple microvalves, which are distributed outside the humidification cavity. One end is connected to the flow path interface of the tissue cell organoid chip through a connecting pipe, and the other end is connected to the culture medium bottle and the CO2 gas cylinder respectively to realize the in-and-out exchange of culture medium, CO2 gas, etc., and the temperature and humidity and CO2 gas concentration changes inside the multifunctional micro-culture module are controlled through the feedback of the temperature and humidity sensor and the CO2 gas sensor, so as to keep the tissue cells or organoids in the microcavity, culture dish or well plate on the tissue cell organoid chip in a normal growth state.

[0096] Furthermore, in the clinical tissue sample rapid pathology online detection system, the hyperspectral imaging detection module includes a light source, an aspheric condenser, a beam splitter, an objective lens and a focusing control module, a tube lens, a narrow slit diaphragm and a one-dimensional motion control module, a collimator, a spectroscopic element, an imaging lens, a CCD detector, a two-dimensional motion platform, a microprocessor controller, etc., which constitute an incident illumination light path and an imaging detection light path. The spectroscopic element can be a grating or a prism or a spatial light modulator or a color filter switch, etc.;

[0097] The thin layer tissue cell slide is mounted on the two-dimensional motion platform, and scanning detection of a single dispersed tissue cell sample on the thin layer tissue cell slide is achieved through the scanning motion of the two-dimensional motion platform;

[0098] The multifunctional micro-culture module is mounted on the two-dimensional motion platform, and the tissue cell organoid chip is fixed inside the multifunctional micro-culture module, and scanning spectral detection of all tissue cells in the microcavity on the tissue cell organoid chip is achieved through the scanning movement of the two-dimensional motion platform;

[0099] In the incident illumination light path, the light emitted by the light source is converged by an aspheric condenser, reflected or transmitted by a beam splitter, and then received by an objective lens and focused on the thin layer tissue cell slide or the tissue cells in the tissue cell organoid chip microcavity; in the imaging detection light path, the reflected or transmitted and scattered light emitted from the tissue cell sample is received by the objective lens, then transmitted or reflected by a beam splitter, received by a tube lens and imaged on a slit diaphragm, and then transmitted by a slit diaphragm and a collimator to form parallel light to illuminate a spectroscopic element, and then split by the spectroscopic element to form a spectral image, which is imaged by an imaging lens onto a CCD detector, and finally the CCD detector transmits the spectral image to a microprocessor controller.

[0100] The objective lens is combined with a focus control module to realize the focus control of the objective lens, ensuring that the objective lens is accurately focused on tissue cells in a small area inside the microcavity of the tissue cell organoid chip inside the multifunctional micro-culture module on the two-dimensional motion platform;

[0101] The slit diaphragm is combined with a one-dimensional motion control module to control the slit diaphragm to perform one-dimensional scanning motion in the vertical direction of the slit, so as to realize scanning spectrum detection of the image formed by tissue cells in a small area of ​​the thin layer tissue cell slide or the tissue cell organoid chip microcavity on the focal plane of the tube microscope; and then realize scanning spectrum detection of all tissue cells in the thin layer tissue cell slide or the tissue cell organoid chip microcavity through motion control of the two-dimensional motion platform;

[0102] The two-dimensional motion platform is adapted to a variety of sample racks, and can be installed with a multifunctional micro-culture module, as well as the thin layer tissue cell slide, culture dish, well plate and fixed tissue slice, frozen tissue slice, cytological smear or other transparent or translucent samples loaded on a standard slide.

[0103] The hyperspectral imaging device is configured as an image acquisition and processing module 200, which is used to perform hyperspectral imaging on the sample to be processed and extract the region of interest to obtain an image of the region of interest;

[0104] like Figure 4 As shown, the optical path of the image acquisition processing module provided by the present embodiment, that is, the hyperspectral imaging detection module, includes a collimated light source, a sample stage, an objective lens, a reflector, a field lens, a beam splitter, a preview CCD, a slit diaphragm, a collimating lens, a transmission type beam splitter grating, an off-axis parabolic mirror, and a spectral imaging CCD; wherein the sample stage can be driven by a two-dimensional motion platform to move in the XY direction, the objective lens can be driven by a focusing module to move in the Z direction, and the slit diaphragm, the collimating lens, the transmission type grating, the off-axis parabolic mirror, and the spectral imaging CCD can be driven by a one-dimensional motion control module to move in the Z direction together. In the imaging detection optical path described in this example, the collimated light emitted by the light source is irradiated on the tissue cell sample, and the transmitted and scattered light emitted from the tissue cell sample is received by the objective lens, and then passed through the reflector, received by the tube lens and divided into two paths by the beam splitter. One path is imaged on the target surface of the preview CCD for focusing preview, and the other path is imaged on the plane where the slit diaphragm is located. After transmission through the slit diaphragm and the collimator, parallel light is formed to illuminate the transmission grating spectroscopic element, forming a spectral image that is imaged by the off-axis parabolic mirror to the target surface of the imaging CCD, and finally the imaging CCD transmits the spectral image to the microprocessor controller.

[0105] Further, the collimated light source described in this example is a full-spectrum white light LED light source, and in other examples of the present invention, an LED, a halogen tungsten lamp, a xenon lamp or other white light, infrared light, or ultraviolet light source may be used;

[0106] Furthermore, the spectroscopic grating described in this example is a transmissive blazed grating, and in other examples of the present invention, various types of gratings suitable for various wavelength bands such as reflective gratings, holographic gratings, blazed gratings, prisms, spatial light modulators, and color filters may be used;

[0107] Furthermore, the off-axis parabolic mirror described in this example is a silver-coated parabolic reflector with an off-axis angle of 45°. In other examples of this example, off-axis parabolic mirrors with other model parameters may be used, and doublets, achromatic lens pairs or other lenses or lens groups may also be used;

[0108] Furthermore, in this example, a beam splitter is used to split the sample image into two beams, one for preview imaging and one for hyperspectral imaging. In other examples of the present invention, other optical path design schemes may be adopted, for example, instead of using a beam splitter, a preview CCD is mechanically moved, and the preview CCD is moved into the optical path when preview is required, and the preview CCD is moved out of the optical path when hyperspectral imaging is required.

[0109] Furthermore, all optical devices described in this example can be replaced in other examples of the present invention as long as the autofocus function, preview function and hyperspectral imaging function can be realized, and there is no special limitation on the specific brand, model and parameters.

[0110] The pathological image analysis device further comprises:

[0111] The feature extraction module 300 is used to extract features from the hyperspectral image based on electromagnetic theory and morphological theory and fuse the extracted features to obtain fused features;

[0112] Analysis module 400: used to perform pathological analysis on cell samples based on deep learning and through the fusion features to obtain analysis results.

[0113] In addition, in some embodiments, a human-computer interaction interface can also be set, including four sub-interfaces: workbench, visual analysis, settings, and personal center. Through the human-computer interaction interface, the user can select the type of sample; the user can also select the purpose of the test. When selecting intraoperative pathology detection, the user can select the type of disease to be studied; the user can also enter the basic information of the sample; the user can preview the image of the sample for manual or automatic focusing; the user can collect hyperspectral imaging data for the current field of view or a specified field of view; the user can perform automatic visual analysis of hyperspectral data as needed; the user can view the analysis report; the user can set parameters such as the data storage address, default scanning area, and default selected algorithm through the "Settings" sub-interface; the user can maintain personal information.

[0114] In other examples of the present invention, the sub-interfaces and specific functions included in the human-computer interaction interface may be different from those in the examples, and should not be excluded from the protection scope of the present invention due to differences in the design of the human-computer interaction interface.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pathological image analysis method, characterized in that: include: S1: collecting clinical tissue samples, and dispersing and slicing the clinical tissue samples to obtain samples to be processed; S2: performing hyperspectral imaging on the sample to be processed and extracting the region of interest to obtain an image of the region of interest; S3: Based on electromagnetic theory and morphological theory, feature extraction is performed on the image of the region of interest respectively and the extracted features are fused to obtain fused features; S4: Based on deep learning, the samples to be processed are predicted by using the fusion features to obtain analysis results.

2. A pathological image analysis method according to claim 1, characterized in that: The samples to be processed in step S1 include thin layer cell samples, fresh cell samples, and organoid samples. The step of obtaining the thin layer cell samples further includes: S11: collecting clinical tissue samples, and physically fragmenting the clinical tissue samples to obtain fragmented samples; S12: retaining single cells in the fragmented sample through cell screening to obtain a single cell sample; S13: performing high-speed centrifugal sedimentation on the single cell sample to obtain a thin layer cell sample of single cell dispersion.

3. A pathological image analysis method according to claim 1, characterized in that: In step S2, after performing hyperspectral imaging on the sample to be processed, the following steps are further performed: S21: preprocessing the hyperspectral image obtained by hyperspectral imaging, wherein the preprocessing includes data cleaning, data alignment, data standard normal variable exchange, data normalization, data smoothing, data feature enhancement and data splicing.

4. A pathological image analysis method according to claim 1, characterized in that: In step S2, the region of interest extraction of the hyperspectral image of the hyperspectral imaging is achieved through an image segmentation model.

5. A pathological image analysis method according to claim 1, characterized in that: In step S3, feature extraction of the image of the region of interest based on electromagnetic theory is achieved through Mie scattering theory. The electromagnetic features extracted based on electromagnetic theory are particle features contained in the spectral data in the hyperspectral image, and the particle features include particle size features and particle density distribution features.

6. A pathological image analysis method according to claim 1, characterized in that: In step S3, the morphological features of the interest region image extracted based on morphological theory include color features, shape features, texture features, edge features and depth morphological features of the interest region.

7. A pathological image analysis method according to claim 1, characterized in that: The expression of the fusion feature in step S3 is: Among them, Z contact is the fusion feature, B is the total number of electromagnetic features, b is the electromagnetic feature index value, X b is the bth electromagnetic feature, K b is the weight value corresponding to the bth electromagnetic feature, C is the total number of morphological features, c is the morphological feature index value, and Y c is the cth morphological feature, K c is the weight value corresponding to the cth morphological feature.

8. A pathological image analysis method according to claim 1, characterized in that: Step S4 further comprises: S41: Preset sample type and disease category; S42: Pre-training the sample type and the disease category to obtain a classifier; S43: Activate and optimize the classifier by adjusting parameters, and input the fusion features into the activated and optimized classifier to obtain analysis results to complete pathological analysis.

9. A pathological image analysis system, characterized in that: include: A sample preparation device, a hyperspectral imaging device and a pathological image analysis device, wherein the hyperspectral imaging device is communicatively connected with the pathological image analysis device; The sample preparation device is used to prepare a sample to be processed of a clinical tissue sample, and the hyperspectral imaging device is used to perform hyperspectral imaging on the sample to be processed to generate hyperspectral image data of the sample to be processed and transmit the hyperspectral image data to the pathological image analysis device; The pathological image analysis device includes a memory, a processor and a communication interface. The memory stores a computer program that can be run on the processor. The pathological image analysis device is configured to receive hyperspectral image data transmitted from the hyperspectral imaging device through the communication interface and send it to the memory for storage. When the processor executes the computer program, a pathological image analysis method as described in any one of claims 1 to 8 is implemented.

10. A pathological image analysis system according to claim 9, characterized in that: The sample preparation device is configured as a sample acquisition module, which is used to collect clinical tissue samples, and use a single-cell dispersion and preparation unit to disperse and prepare the clinical tissue samples to obtain thin-layer cell samples. It is also used to collect clinical tissue samples, and use a tissue cell organoid chip and a multifunctional micro-culture unit to perform primary cell culture or organoid culture on the clinical tissue samples to obtain fresh cell samples or organoid samples; the single-cell dispersion and preparation unit is used to make thin-layer tissue cell slides, and the tissue cell organoid chip and the multifunctional micro-culture unit are used to maintain the activity of fresh cells and organoids, so as to perform organoid cell chip detection and analysis; The hyperspectral imaging device is configured as an image acquisition and processing module, which is used to perform hyperspectral imaging on the sample to be processed and extract the region of interest to obtain an image of the region of interest; The hyperspectral imaging device comprises a light source, an aspheric condenser, a beam splitter, an objective lens and a focusing control module, a tube lens, a narrow slit diaphragm and a one-dimensional motion control module, a collimator, a beam splitter, an imaging lens, a CCD detector, a two-dimensional motion platform, and a microprocessor controller, which constitute an incident illumination light path and an imaging detection light path, wherein the beam splitter is configured as a grating, a prism, a spatial light modulator or a color filter switch; The pathological image analysis device further comprises: The feature extraction module is used to extract features from the hyperspectral image based on electromagnetic theory and morphological theory and fuse the extracted features to obtain fused features; Analysis module: used to perform pathological analysis on the sample to be processed based on deep learning and through the fusion features to obtain analysis results.

Images