Virtual dyeing method and device based on wide-spectrum autofluorescence and storage medium

Abstract

The invention discloses a virtual dyeing method and device based on wide-spectrum autofluorescence and a storage medium, and the method comprises the following steps: configuring a plurality of excitation wavelengths and a wide-spectrum autofluorescence channel through a fluorescence microscopic imaging system, firstly obtaining wide-spectrum autofluorescence signals of a plurality of endogenous fluorophores in a sample under different excitation wavelengths; then the wide-spectrum autofluorescence image is sequentially distributed to three color channels of blue, green and red, a virtual RGB image is synthesized, through further image contrast enhancement processing, the morphological contrast of different tissue structures and lesion areas can be improved, and unmarked acquisition of tissue pathological information is achieved. Compared with narrow-spectrum autofluorescence imaging, the wide-spectrum autofluorescence imaging method has the advantages that autofluorescence of different wavelengths radiated after various endogenous fluorophores are excited is coupled into a wide-spectrum autofluorescence signal, so that the information amount and the signal-to-noise ratio of a spectrum channel under single photographing are enhanced, and chemical dyeing is not needed.

Description

Technical Field

[0001] This invention relates to the field of pathological fluorescence staining technology, specifically to a virtual staining method, apparatus, and storage medium based on broadband autofluorescence. Background Technology

[0002] Cancer has become one of the most serious challenges to human health, and rapid and accurate pathological diagnosis is crucial for improving the prognosis of cancer patients. Currently, hematoxylin-eosin (H&E) staining, as the gold standard for histopathological diagnosis, has several drawbacks. Firstly, the preparation process for H&E-stained sections is cumbersome, requiring specialized and experienced pathologists and consuming a significant amount of time. This severely limits diagnostic efficiency, especially in time-sensitive intraoperative histopathological diagnosis scenarios, negatively impacting patient prognosis. Secondly, H&E staining or labeling can have irreversible chemical effects on tissue characteristics, hindering the reuse of tissue in subsequent diagnostic procedures (such as molecular testing), representing a significant waste of valuable tissue samples. How to reduce or even eliminate histochemical staining has become an urgent clinical problem to solve.

[0003] In recent years, label-free imaging characterization techniques have developed rapidly to address the aforementioned issues. Among them, autofluorescence-based imaging techniques utilize the autofluorescence properties of endogenous fluorophores in biological tissues to generate image contrast, enabling the acquisition of tissue function and structural information, and showing great potential in the field of label-free tissue imaging characterization. However, existing autofluorescence-based label-free tissue imaging characterization workflows mostly use narrow-spectrum autofluorescence signals from tissues. A single acquisition of narrow-spectrum autofluorescence signals can only utilize the autofluorescence of a small number of endogenous fluorophores, resulting in limited information and signal-to-noise ratio obtained from a single spectral imaging channel. Deep learning methods are needed to enhance image visualization through virtual staining, but the deep learning approach is complex, the model's generalization ability is limited, and it may produce erroneous virtual staining results.

[0004] Therefore, there is an urgent need for a virtual staining method, device, and storage medium based on broadband autofluorescence to solve the above problems. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a virtual staining method, apparatus, and storage medium based on broadband autofluorescence.

[0006] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides a virtual staining method based on broadband autofluorescence, comprising the following steps: (1) Place the tissue sample to be tested on the stage of the fluorescence microscopy system, close the excitation source channel by controlling the shutter in the illumination optical path, start the camera and set the gain and exposure value; (2) Switch the fluorescence cube in the fluorescence microscopy system and configure the required excitation wavelength channel and broadband autofluorescence channel; (3) Turn on the excitation source of the fluorescence microscopy system and adjust its power. Switch the imaging channel to the eyepiece end, observe the tissue sample to be tested through the eyepiece, and adjust the stage position to focus so as to move the target area to the center of the field of view. (4) Switch the imaging channel to the camera end, observe the camera display interface, and fine-tune the focal length of the stage and the power of the excitation light source to the optimal image contrast, and save the broadband autofluorescence image at this time; by switching the fluorescence cube, obtain the broadband autofluorescence images of the tissue sample under test at at least three different excitation wavelengths. (5) Map the broadband autofluorescence image to different color channels of the color space respectively, and synthesize the broadband autofluorescence image of the color channel to form a virtual RGB image to realize virtual staining.

[0007] Furthermore, the tissue sample to be tested is a tissue section or thick tissue.

[0008] Furthermore, in step (1), the fluorescence microscopy system is an upright fluorescence microscope, an inverted fluorescence microscope, or a fluorescence microscopy system with any other optical path layout. The camera gain is 0-20dB, and the camera exposure value is 1ms-1s.

[0009] Furthermore, the fluorescence cube includes a narrow-spectrum excitation filter, a long-pass dichroic mirror, and a long-pass fluorescence filter, which are used to couple the different wavelengths of spontaneous fluorescence emitted by various endogenous fluorophores after excitation into a broadband spontaneous fluorescence signal.

[0010] Furthermore, step (2) specifically includes: Switch the fluorescence cube in the fluorescence microscopy system. First, configure the corresponding narrow-spectrum excitation filter based on the required excitation wavelength. Then, configure the cutoff wavelength of the long-pass dichroic mirror to be greater than the selected excitation wavelength. Next, configure the cutoff wavelength of the long-pass fluorescence filter to be slightly greater than the cutoff wavelength of the long-pass dichroic mirror, thus completing the configuration of the required excitation wavelength channel and the broadband autofluorescence channel.

[0011] Furthermore, the excitation light source is an LED or laser illumination system with a wavelength range covering deep ultraviolet to infrared, which can be used in conjunction with a fluorescent cube to achieve excitation at multiple wavelengths; The camera is an sCMOS, CMOS, CCD scientific research camera or industrial camera. The at least three different excitation wavelengths include the lowest excitation wavelength, the intermediate excitation wavelength, and the highest excitation wavelength.

[0012] Furthermore, the step of mapping the broadband autofluorescence image to different color channels of the color space specifically includes: According to the order of excitation wavelength from low to high, the broadband autofluorescence image corresponding to the lowest excitation wavelength is mapped to the blue channel, the broadband autofluorescence image corresponding to the middle excitation wavelength is mapped to the green channel, and the broadband autofluorescence image corresponding to the highest excitation wavelength is mapped to the red channel.

[0013] Furthermore, it also includes: Image contrast enhancement processing is performed on virtual RGB images. The methods for image contrast enhancement processing include high-pass filtering, gray-level stretching, nonlinear gray-level transformation, and contrast-limited adaptive histogram equalization.

[0014] A second aspect of the present invention provides a virtual staining apparatus based on broadband autofluorescence, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described virtual staining method based on broadband autofluorescence.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-described virtual staining method based on broadband autofluorescence.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention utilizes the autofluorescence of different wavelengths emitted by various endogenous fluorophores after they are excited, and couples them into a broadband autofluorescence signal, thereby enhancing the morphological information content and signal-to-noise ratio of the spectral channels in a single photograph; (2) Based on the autofluorescence images of different wide-spectrum channels obtained under multiple excitation wavelengths, the present invention can synthesize virtual RGB images, realize rapid virtual staining, enhance the morphological contrast of different tissue structures and lesion areas without relying on deep learning, simplify the image data processing process, and improve efficiency.

[0017] (3) The present invention can realize label-free imaging characterization of tissues, avoid the use of chemical reagents, and obtain tissue morphology recognition effect similar to H&E staining without chemical staining, thus shortening the process of obtaining tissue morphology information.

[0018] (4) This invention confirms that the broadband autofluorescence imaging method has the potential for clinical application in rapid pathological diagnosis, which helps to improve examination efficiency, promote the clinical transformation of label-free pathological analysis technology based on autofluorescence broadband fluorescence, and assist in pathological diagnosis, providing diagnostic reference for pathologists. Attached Figure Description

[0019] Figure 1 This is a flowchart of the virtual staining method based on broadband autofluorescence of the present invention; Figure 2 This is a comparison of the imaging effects and differences between a broadband autofluorescence image and a narrowband autofluorescence image of a colon sample in one embodiment of the present invention. Figure 3 This is a comparison of the tissue morphology information of virtual RGB images and gold standard images of frozen sections and dewaxed sections in one embodiment of the present invention; Figure 4 This is a time-varying curve of the structural similarity index of broadband autofluorescence images of frozen sections and dewaxed sections under different excitation wavelengths in one embodiment of the present invention; wherein, Figure 4 (a) in the figure is the time-varying curve of the structural similarity index of broadband autofluorescence images of frozen sections under different excitation wavelengths; Figure 4 (b) in the figure is the time-varying curve of the structural similarity index of the broadband autofluorescence images of the dewaxed sections under different excitation wavelengths; Figure 5 This is a comparison of enhanced virtual RGB images and gold standard images of dewaxed sections of tumor-bearing liver tissue in different tissue regions according to an embodiment of the present invention. Figure 6 This is another comparison result of enhanced virtual RGB images and gold standard images of dewaxed sections of liver tissue with tumorous lesions in different tissue regions in one embodiment of the present invention; Figure 7 This is a comparison of virtual RGB images, virtual RGB-to-grayscale images, and gold standard images of frozen sections of liver tissue with tumorous lesions in one embodiment of the present invention. Figure 8 This is a comparison between a virtual RGB-to-grayscale image and a gold standard image of a frozen section of a tumorous liver tissue in one embodiment of the present invention. Figure 9 This is a schematic diagram of a virtual staining device based on broadband autofluorescence according to the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. It is obvious that the drawings used in the following description are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0023] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0024] See Figure 1 The virtual staining method based on broadband autofluorescence of the present invention specifically includes the following steps: (1) Place the tissue sample to be tested on the stage of the fluorescence microscopy system, close the excitation light source channel by controlling the shutter in the illumination optical path, start the camera and set the gain and exposure value.

[0025] Furthermore, the tissue sample to be tested is a tissue section or thick tissue, which can be represented as follows: Sample ∈{tissue sections, thick tissue}, where SampleThis represents the tissue sample to be tested. When the tissue sample is a tissue section, it can be prepared from the same tissue section or from a pair of adjacent tissue sections. When preparing a tissue sample from the same tissue section, first obtain a broad-spectrum autofluorescence image of the unstained section, then stain the section, and finally obtain a color image of the stained section. When preparing a tissue sample from a pair of adjacent tissue sections, directly select adjacent unstained and stained sections, and obtain a broad-spectrum autofluorescence image of the unstained section and a color image of the stained section, respectively. When the tissue sample is a thick tissue, first obtain a broad-spectrum autofluorescence image of the unstained tissue block, then stain it, and finally obtain a color image of the stained tissue block.

[0026] The results of the staining process are used to provide a gold standard; that is, the color image of the stained tissue sample can provide a gold standard for the unstained tissue sample.

[0027] Furthermore, staining treatments include conventional staining methods such as hematoxylin-eosin (H&E), immunohistochemistry, and immunofluorescence staining.

[0028] Furthermore, the fluorescence microscopy imaging system is a fluorescence microscopy imaging system that is upright or inverted, or has any other optical path layout (such as a cage system built with optical and mechanical components).

[0029] Specifically, the tissue sample to be tested is placed on the stage of the fluorescence microscopy system. This stage can be manual or motorized. A motorized stage is preferred because it allows for autofocus and scanning imaging. The autofocus and scanning imaging functions are implemented using a host computer control program written in LabVIEW or C++. The fluorescence microscopy system has an illumination path; therefore, by controlling the shutter in the illumination path to close the excitation source channel, photobleaching of the sample caused by accidental strong light excitation can be effectively prevented. The camera is then started, and appropriate gain and exposure values ​​are set. The camera gain is set to 0-20 dB, and the exposure value to 1 ms-1 s.

[0030] (2) Switch the fluorescence cube in the fluorescence microscopy system and configure the required excitation wavelength channel and broadband autofluorescence channel.

[0031] Furthermore, the fluorescence cube includes a narrow-spectrum excitation filter, a long-pass dichroic mirror, and a long-pass fluorescence filter, which are used to couple the different wavelengths of spontaneous fluorescence emitted by various endogenous fluorophores after excitation into a broadband spontaneous fluorescence signal.

[0032] Specifically, by switching the fluorescence cube in the fluorescence microscopy system, firstly, a narrow-spectrum excitation filter is configured based on the required excitation wavelength; then, the cutoff wavelength of the long-pass dichroic mirror is configured to be greater than the selected excitation wavelength; then, the cutoff wavelength of the long-pass fluorescence filter is configured to be slightly greater than the cutoff wavelength of the long-pass dichroic mirror. Since the narrow-spectrum excitation filter, long-pass dichroic mirror, and long-pass fluorescence filter contained in the fluorescence cube can be used to couple the different wavelengths of spontaneous fluorescence emitted by various endogenous fluorophores after excitation into a broadband spontaneous fluorescence signal, the configuration of the required excitation wavelength channel and broadband spontaneous fluorescence channel can be completed.

[0033] In this embodiment, a Nikon ECLIPSE Ci-L plus microscope is used to construct a fluorescence microscopy imaging system with a spectral range of 315-900 nm. The microscope can be equipped with multiple fluorescence cubes; in this embodiment, three are used. Each fluorescence cube contains a narrow-spectrum excitation filter, a long-pass dichroic mirror, and a long-pass fluorescence filter. Fluorescence cube 1 contains a narrow-spectrum excitation filter of 355±25 nm, a long-pass dichroic mirror of 400-700 nm, and a long-pass fluorescence filter of 410-700 nm. Fluorescence cube 2 contains a narrow-spectrum excitation filter of 470±20 nm, a long-pass dichroic mirror of 505-700 nm, and a long-pass fluorescence filter of 510-700 nm. Fluorescence cube 3 contains a narrow-spectrum excitation filter of 535±25 nm, a long-pass dichroic mirror of 575-700 nm, and a long-pass fluorescence filter of 580-700 nm.

[0034] (3) Turn on the excitation source of the fluorescence microscopy system and adjust its power. Switch the imaging channel to the eyepiece end, observe the tissue sample to be tested through the eyepiece, and adjust the stage position to focus so as to move the target area to the center of the field of view.

[0035] Furthermore, the excitation source is a high-power LED or laser illumination system with a wavelength range covering deep ultraviolet to infrared. Combined with a suitable fluorescent cube, excitation at multiple wavelengths can be achieved. Correspondingly, the wavelength range of the broadband fluorescence imaging channel in this invention also covers deep ultraviolet to infrared.

[0036] Specifically, the excitation source of the fluorescence microscopy imaging system is turned on and its power is adjusted to a low level (i.e., low power). A high-power LED or laser illumination system is used as the excitation source to effectively excite the tissue sample. In this embodiment, Nikon's D-LEDI fluorescence LED illumination system is used, which can emit excitation light of multiple wavelengths, covering excitation wavelengths of 385nm, 475nm, and 550nm, corresponding to fluorescence cubes 1, 2, and 3 respectively. Then, the imaging channel is switched to the eyepiece end, and the tissue sample is observed through the eyepiece. The stage position is adjusted for focusing, specifically by adjusting the focusing knob, until the texture of the tissue sample is clearly visible through the eyepiece. Finally, the stage is moved to center the target area of ​​the tissue sample in the field of view.

[0037] (4) Switch the imaging channel to the camera end. While observing the camera display interface, fine-tune the focal length of the stage and the power of the excitation source to achieve optimal image contrast, and save the broadband autofluorescence image at this time. By switching the fluorescence cube, obtain broadband autofluorescence images of the tissue sample under test at at least three different excitation wavelengths. The broadband autofluorescence images at different excitation wavelengths can be represented as follows: ,in Indicates the excitation wavelength is Broadband autofluorescence image below, This indicates the acquisition of a single-channel broadband autofluorescence image. Indicates the excitation wavelength is The corresponding fluorescent cube, Indicates the excitation wavelength is The power of the excitation light source, , and These represent the minimum and maximum power of the excitation source, respectively.

[0038] Furthermore, the camera can be an sCMOS, CMOS, or CCD scientific research camera or an industrial camera, and a black-and-white or color camera can be flexibly selected according to the observation needs.

[0039] Furthermore, at least three different excitation wavelengths are included, including the lowest excitation wavelength, the intermediate excitation wavelength, and the highest excitation wavelength.

[0040] Specifically, the imaging channel is switched to the camera end. While observing the camera display interface, the focus knob is finely adjusted, thereby fine-tuning the focal length of the stage until the image is clear. The power of the excitation source is adjusted from low to high to a suitable value (10%~100%) until the contrast of the tissue sample features observed by the camera is optimal. The broadband autofluorescence image at this point is saved. By switching the fluorescence cube and fine-tuning the power of the excitation source again, broadband autofluorescence images of the tissue sample under test at different excitation wavelengths are obtained. Specifically, when tissue samples were irradiated with 385nm excitation light through a 355±25nm narrow-spectrum excitation filter, the camera obtained a broad-spectrum autofluorescence image in the 410-700nm spectral range after passing through a long-pass dichroic mirror and a long-pass fluorescence filter; when tissue samples were irradiated with 475nm excitation light through a 470±20nm narrow-spectrum excitation filter, the camera obtained a broad-spectrum autofluorescence image in the 510-700nm spectral range after passing through a long-pass dichroic mirror and a long-pass fluorescence filter; and when tissue samples were irradiated with 550nm excitation light through a 535±25nm narrow-spectrum excitation filter, the camera obtained a broad-spectrum autofluorescence image in the 580-700nm spectral range after passing through a long-pass dichroic mirror and a long-pass fluorescence filter.

[0041] (5) Map the broadband autofluorescence image to different color channels of the color space respectively, and synthesize the broadband autofluorescence image of the color channel to form a virtual RGB image to realize virtual staining.

[0042] Furthermore, the broadband autofluorescence images are mapped to different color channels in the color space, specifically including: mapping the broadband autofluorescence image corresponding to the lowest excitation wavelength to the blue channel, mapping the broadband autofluorescence image corresponding to the middle excitation wavelength to the green channel, and mapping the broadband autofluorescence image corresponding to the highest excitation wavelength to the red channel, in order of excitation wavelength from low to high.

[0043] Specifically, the broadband autofluorescence image obtained in configuration 1 (excitation light center wavelength 385nm, fluorescence cube 1) is mapped to the Blue channel; the broadband autofluorescence image obtained in configuration 2 (excitation light center wavelength 475nm, fluorescence cube 2) is mapped to the Green channel; and the broadband autofluorescence image obtained in configuration 3 (excitation light center wavelength 550nm, fluorescence cube 3) is mapped to the Red channel. The broadband autofluorescence images in the Blue, Green, and Red channels are synthesized to form a virtual RGB image, achieving virtual staining. The virtual RGB image can be represented as... , This indicates the generation of a virtual RGB image. Indicates the minimum excitation wavelength The corresponding broadband autofluorescence image is mapped to the Blue channel. Indicates the intermediate excitation wavelength The corresponding broadband autofluorescence image is mapped to the Green channel. Indicates the highest excitation wavelength The corresponding broadband autofluorescence image is mapped to the Red channel.

[0044] It should be noted that when assigning broadband autofluorescence images to the Blue, Green, and Red channels in sequence, the allocation order of broadband autofluorescence images in the Blue, Green, and Red channels can be flexibly configured according to the actual display effect, rather than being limited to implementing it in the order of excitation wavelength from low to high.

[0045] In some other embodiments, after step (5), the method further includes: (6) performing image contrast enhancement processing on the virtual RGB image, which can be expressed by the following formula:

[0046] In the formula, This represents the enhanced virtual RGB image. This refers to image contrast enhancement processing. Methods for image contrast enhancement include, but are not limited to: high-pass filtering, grayscale stretching, nonlinear grayscale transformation, and contrast-limited adaptive histogram equalization.

[0047] It's important to note that image contrast enhancement processing on virtual RGB images aims to improve overall image contrast. Image contrast refers to the degree of difference in grayscale values ​​(or colors) between different regions (or pixels) in an image. This difference directly reflects the clarity of the image's brightness, detail, and tonal range. Therefore, to enhance image contrast, methods such as contrast-limited adaptive histogram equalization can be employed. For example, contrast-limited adaptive histogram equalization divides the image into sub-blocks and performs contrast-limited histogram equalization on each block individually. This improves local image contrast while avoiding overexposure and loss of detail.

[0048] It should be understood that by performing image contrast enhancement processing on virtual RGB images, the contrast of virtual RGB images can be enhanced, overexposure and loss of detail can be avoided, lesion areas of tissues can be highlighted, morphological features of tissues can be obtained, and the display effect of tissue structure and morphological features of lesion areas can be improved.

[0049] In summary, this invention eliminates the need for chemical staining. By utilizing the autofluorescence of different wavelengths emitted after the excitation of various endogenous fluorophores, a broadband autofluorescence signal is coupled, thereby enhancing the morphological information and signal-to-noise ratio of the spectral channels in a single photograph. Based on the autofluorescence images of different broadband channels obtained under multiple excitation wavelengths, virtual RGB images can be synthesized to achieve rapid virtual staining, thereby enhancing the morphological contrast of different tissue structures and lesion areas. This enables label-free tissue imaging characterization, providing diagnostic references for pathologists.

[0050] For example, such as Figure 2 As shown, for the same colon sample, the center wavelength of the excitation source was first set to 385 nm, paired with fluorescence cube No. 1, and broadband autofluorescence RGB images were acquired using an sCMOS color scientific camera. Then, the long-pass fluorescence filter in fluorescence cube No. 1 was replaced with a narrow-band fluorescence filter (wavelength 447±60 nm), and narrow-band autofluorescence RGB images were acquired using the same sCMOS color scientific camera. Based on the broadband or narrow-band autofluorescence RGB images, grayscale images of the Red, Green, and Blue channels were extracted, and the information level of the three channels was evaluated using information entropy. Specifically, the results are as follows: RGB images of broad-spectrum autofluorescence acquired in the 410-700nm range provide better color and tissue structure information than RGB images of narrow-spectrum autofluorescence acquired in the 447±60nm range; in the broad-spectrum autofluorescence images acquired in the 410-700nm range, the information entropy of the Red, Green, and Blue channels are 3.7587, 7.2042, and 7.5959, respectively, and all exhibit tissue morphology features; in the narrow-spectrum autofluorescence images acquired in the 447±60nm range, the information entropy of the Red, Green, and Blue channels are 3.4575, 3.1077, and 6.7174, respectively, and the Red and Green channels do not show tissue morphology features; for all three channels, the information content of the broad-spectrum autofluorescence images acquired in the 410-700nm range is superior to that of the narrow-spectrum autofluorescence images acquired in the 447±60nm range. Therefore, according to the method described in this invention, broadband autofluorescence images under three different excitation lights are obtained and virtual RGB images are synthesized, which can realize virtual staining and enhance the display effect of tissue morphology features.

[0051] For example, Figure 3 This demonstrates the results of broadband autofluorescence imaging in identifying morphological information of tissue structures, such as... Figure 3As shown, normal mouse kidneys and livers represent solid organs, and normal mouse colons represent hollow organs. Broadband autofluorescence images of three color channels were acquired from frozen and dewaxed sections of the samples, along with color images of adjacent H&E-stained sections. Virtual RGB images were then synthesized from the three-channel broadband autofluorescence images to achieve virtual staining. The morphological information of the kidneys, liver, and colon was identified and analyzed using the virtual RGB images, and the effectiveness was evaluated based on adjacent H&E-stained sections. The color images of the H&E-stained sections were used as the gold standard for evaluating the virtual staining results. Overall, the virtual RGB images of both frozen and dewaxed sections of mouse kidneys, liver, and colon could identify the morphological features of the tissue structures. Specific results are shown in Table 1.

[0052] Table 1: Morphological characteristics of tissue structure

[0053] For example, frozen and dewaxed sections of normal mouse livers were used as samples. The samples were excited for a prolonged period (60 seconds), with broadband autofluorescence images acquired every 1 second. Using the images at the beginning of each color channel as a baseline, the structural similarity index (SSIM) between the images at other times and the beginning time was calculated to obtain the temporal stability results of the broadband autofluorescence image stability, such as... Figure 4 As shown in the figure. The curves showing the structural similarity index of broadband autofluorescence images of frozen sections at different excitation wavelengths over time are shown in the figure. Figure 4 As shown in (a) above, the curves showing the change in structural similarity index of broadband autofluorescence images of dewaxed sections at different excitation wavelengths over time are as follows: Figure 4 As shown in (b) of the figure, it can be seen that for both frozen and dewaxed sections, the SSIM values ​​of the broadband autofluorescence images are all above 0.7, indicating that the broadband autofluorescence images have good temporal stability and the effect of photobleaching can be ignored.

[0054] For example, the method described in this invention is used to obtain a broadband autofluorescence image of a dewaxed section of mouse liver tissue with tumorous lesions (for postoperative pathological examination), and a virtual RGB image is synthesized to achieve virtual staining. The virtual RGB image is then subjected to image contrast enhancement processing to obtain an enhanced virtual RGB image, such as... Figure 5 and Figure 6 As shown, morphological information was then compared using color images of adjacent H&E stained sections as the gold standard images. Figure 5 and Figure 6 Broadband autofluorescence imaging results of dewaxed sections of liver tissue with neoplastic lesions are presented. Figure 5The white arrows in the diagram indicate stained blood cells, and the black arrows indicate cell nuclei. The gold blood cells indicated by the white arrow are dummy stains, and the stained blood cells indicated by the other white arrow are H&E stains. Figure 5 It can be seen that the virtual RGB image of dewaxed sections can effectively distinguish poorly differentiated tumors, well-differentiated tumors, necrotic tissues, and normal tissue regions, and the boundary contrast of different tissue regions is better than that of H&E stained images. The identification results of other tumor regions are as follows... Figure 6 As shown.

[0055] For example, such as Figure 7 and Figure 8 As shown, the method described in this invention is used to obtain broadband autofluorescence images of frozen sections of mouse tumor-bearing liver tissue (for intraoperative pathological examination), and to synthesize virtual RGB images to achieve virtual staining. The color images of adjacent H&E stained sections are used as the gold standard images for morphological information comparison. Figure 7 and Figure 8 Broadband autofluorescence imaging results of frozen sections of liver tissue with neoplastic lesions are presented. Figure 7 The virtual RGB image, virtual RGB-to-grayscale image, and gold standard image can all identify tumor, normal, and necrotic tissue regions. In the virtual RGB image, there is a clear banded texture between normal and diseased tissue regions, exhibiting strong contrast, while this feature is less pronounced in the gold standard image. Figure 8 As shown, in both the virtual RGB-to-grayscale image and the gold standard image, the boundaries between immune cells and tumor cells in the tumor tissue region are visible. In particular, abnormal mitotic figures are also present in the tumor tissue region in both images. Furthermore, in both the virtual RGB-to-grayscale image and the gold standard image, aggregation of immune cells near blood vessels is observed in the normal tissue region.

[0056] It should be noted that histopathological diagnosis primarily focuses on the morphological structural characteristics of tissues, including the tissue's structural arrangement, the size and shape characteristics of cells (especially cell nuclei), and objective morphological information such as the deposition of specific substances and interstitial reactions. This allows for the differentiation of different endogenous fluorophore components. Therefore, by using the method described in this invention to virtually stain tissues and perform image contrast enhancement processing, the lesion areas of the tissue can be highlighted, the morphological characteristics of the tissue can be obtained, and the display effect of tissue structure and lesion area morphological characteristics can be improved. Subsequently, the enhanced virtual RGB image is compared with its corresponding stained slide image, which serves as the gold standard. That is, by referring to the stained slide image as the gold standard, the correlation between the virtual RGB image and the pathology is constructed, thereby achieving pathological diagnosis.

[0057] As can be seen from the results of the above examples, the method of the present invention can achieve tissue morphology recognition results comparable to those of H&E staining, thus confirming the clinical application potential of the method of the present invention in rapid pathological diagnosis, establishing a link between intraoperative decision-making (applicable to frozen sections) and postoperative verification (applicable to paraffin sections), and promoting the clinical translation of label-free histopathology technology based on broadband autofluorescence.

[0058] Corresponding to the aforementioned embodiments of the virtual staining method based on broadband autofluorescence, the present invention also provides embodiments of a virtual staining apparatus based on broadband autofluorescence.

[0059] See Figure 9 The present invention provides a virtual staining device based on broadband autofluorescence, comprising one or more processors and a memory, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the virtual staining method based on broadband autofluorescence in the above embodiments.

[0060] The embodiments of the virtual staining device based on broadband autofluorescence of this invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 9 The diagram shown is a hardware structure diagram of any device with data processing capabilities, including the virtual staining device based on broadband autofluorescence of this invention. (Except for...) Figure 9 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0061] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0062] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0063] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the virtual staining method based on broadband autofluorescence described in the above embodiments.

[0064] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0065] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual staining method based on broadband autofluorescence, characterized in that, Includes the following steps: (1) Place the tissue sample to be tested on the stage of the fluorescence microscopy system, close the excitation source channel by controlling the shutter in the illumination optical path, start the camera and set the gain and exposure value; (2) Switch the fluorescence cube in the fluorescence microscopy system and configure the required excitation wavelength channel and broadband autofluorescence channel; (3) Turn on the excitation source of the fluorescence microscopy system and adjust its power. Switch the imaging channel to the eyepiece end, observe the tissue sample to be tested through the eyepiece, and adjust the stage position to focus so as to move the target area to the center of the field of view. (4) Switch the imaging channel to the camera end, observe the camera display interface, and fine-tune the focal length of the stage and the power of the excitation light source to the optimal image contrast, and save the broadband autofluorescence image at this time; by switching the fluorescence cube, obtain the broadband autofluorescence images of the tissue sample under test at at least three different excitation wavelengths. (5) Map the broadband autofluorescence image to different color channels of the color space respectively, and synthesize the broadband autofluorescence image of the color channel to form a virtual RGB image to realize virtual staining.

2. The virtual staining method based on broadband autofluorescence according to claim 1, characterized in that, The tissue sample to be tested is a tissue section or thick tissue.

3. The virtual staining method based on broadband autofluorescence according to claim 1, characterized in that, In step (1), the fluorescence microscopy system is an upright fluorescence microscope, an inverted fluorescence microscope, or a fluorescence microscopy system with any other optical path layout. The camera gain is 0-20dB, and the camera exposure value is 1ms-1s.

4. The virtual staining method based on broadband autofluorescence according to claim 1, characterized in that, The fluorescence cube includes a narrow-spectrum excitation filter, a long-pass dichroic mirror, and a long-pass fluorescence filter, which are used to couple the different wavelengths of spontaneous fluorescence emitted by various endogenous fluorophores after excitation into a broadband spontaneous fluorescence signal.

5. The virtual staining method based on broadband autofluorescence according to claim 4, characterized in that, Step (2) specifically includes: Switch the fluorescence cube in the fluorescence microscopy system. First, configure the corresponding narrow-spectrum excitation filter based on the required excitation wavelength. Then, configure the cutoff wavelength of the long-pass dichroic mirror to be greater than the selected excitation wavelength. Next, configure the cutoff wavelength of the long-pass fluorescence filter to be slightly greater than the cutoff wavelength of the long-pass dichroic mirror, thus completing the configuration of the required excitation wavelength channel and the broadband autofluorescence channel.

6. The virtual staining method based on broadband autofluorescence according to claim 1, characterized in that, The excitation light source is an LED or laser illumination system with a wavelength range covering deep ultraviolet to infrared. It is used in conjunction with a fluorescent cube to achieve excitation at multiple wavelengths. The camera is an sCMOS, CMOS, CCD scientific research camera or industrial camera. The at least three different excitation wavelengths include the lowest excitation wavelength, the intermediate excitation wavelength, and the highest excitation wavelength.

7. The virtual staining method based on broadband autofluorescence according to claim 6, characterized in that, The process of mapping broadband autofluorescence images to different color channels in a color space specifically includes: According to the order of excitation wavelength from low to high, the broadband autofluorescence image corresponding to the lowest excitation wavelength is mapped to the blue channel, the broadband autofluorescence image corresponding to the middle excitation wavelength is mapped to the green channel, and the broadband autofluorescence image corresponding to the highest excitation wavelength is mapped to the red channel.

8. The virtual staining method based on broadband autofluorescence according to claim 1, characterized in that, Also includes: Image contrast enhancement processing is performed on virtual RGB images. The methods for image contrast enhancement processing include high-pass filtering, gray-level stretching, nonlinear gray-level transformation, and contrast-limited adaptive histogram equalization.

9. A virtual staining device based on broadband autofluorescence, comprising one or more processors and a memory, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the virtual staining method based on broadband autofluorescence as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, is used to implement the virtual staining method based on broadband autofluorescence as described in any one of claims 1-8.

Images