Spectral imaging chip and spectral pathological diagnosis method

Abstract

The present application provides a spectral imaging chip and a spectral pathological diagnosis method. The chip comprises an optical system, a filtering structure, an image sensor, and a data processing unit. The optical system is used for adjusting propagation and distribution of input light acquired from a location to be diagnosed to obtain first input light; the filtering structure is used for modulating the first input light to obtain light under test; the image sensor is used for acquiring a spectral response corresponding to the light under test; and the data processing unit is used for performing pathological classification on the basis of the spectral response to obtain a classification result. In the present application, the optical system, the filtering structure, and the image sensor are used to quickly, conveniently, and cost-effectively introduce material information at a spectral dimension to pathological diagnosis, offering good feasibility; in addition, the data processing unit is used to efficiently and accurately analyze lesion features of pathological tissue, so as to implement intelligent pathological classification and determination, thereby improving the accuracy and efficiency of diagnosis.

Description

Spectral imaging chips and spectral pathology diagnostic methods

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese patent application No. 2024114475724, filed on October 16, 2024, entitled "Spectral Imaging Chip and Spectral Pathological Diagnosis Method", which is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to the field of image processing technology, and in particular to a spectral imaging chip and a spectral pathology diagnosis method. Background Technology

[0004] Pathological diagnosis is the "gold standard" for diagnosing most diseases, especially tumors. Current pathological diagnoses mainly rely on the appearance and morphological characteristics of tissue cells, requiring the use of microscopic imaging to observe stained tissue cells and make a diagnosis.

[0005] The existing problems with microscopic morphological pathological diagnosis are as follows: First, diagnosis can only be made through the morphological characteristics of tissues and cells, which relies heavily on the doctor's experience, is highly subjective, and has limited accuracy and precision. It is also unable to clearly diagnose early diseases with inconspicuous morphological changes and is difficult to distinguish between diseases with similar morphologies. Second, it requires processes such as staining and slide preparation and microscopic imaging, which are cumbersome and time-consuming to process and examine patient tissues, and cannot be applied to real-time detection in in vivo tissues or intraoperative scenarios.

[0006] Optical information possesses numerous dimensions, including spatial and spectral dimensions. Beyond spatial morphology, the spectral dimension contains rich material information, serving as a fingerprint of matter. Introducing spectral information into pathological diagnostic systems can add a new material dimension to pathological diagnosis, potentially improving accuracy and expanding its application range. Existing mature spectral imaging technologies employ a spatial optical path-based scanning scheme. This scheme acquires the spectral image data cube of the target scene through spatial spectral dispersion combined with mechanical scanning. While this scheme achieves high spectral resolution, it comes at the cost of a large spatial optical path, long imaging time, and the inability to achieve real-time imaging. Furthermore, it requires the fabrication of complex precision gratings and the precise assembly of various mechanical components to form the scanning structure. Therefore, this scheme suffers from problems such as large equipment size, slow imaging, and high cost, making its application in pathological diagnostic systems challenging.

[0007] In summary, existing technologies suffer from high costs, low efficiency, and poor feasibility. Summary of the Invention

[0008] This application provides a spectral imaging chip and a spectral pathology diagnosis method to overcome the shortcomings of high cost, low efficiency and poor feasibility in the prior art, and realize a spectral imaging chip with low cost, high efficiency and good feasibility, so as to be used for spectral pathology diagnosis.

[0009] This application provides a spectral imaging chip, including an optical system, a filter structure, an image sensor, and a data processing unit;

[0010] The optical system is used to adjust the propagation and distribution of the input light at the acquired location to be diagnosed, thereby obtaining a first input light;

[0011] The filter structure is used to modulate the first input light to obtain the light to be measured;

[0012] The image sensor is used to acquire the spectral response corresponding to the light under test;

[0013] The data processing unit is used to perform pathological classification based on the spectral response to obtain classification results.

[0014] According to the spectral imaging chip provided in this application, the filter structure includes a preset number of structural units, and the structural units are micro-nano modulation structures designed and prepared using a preset algorithm to distinguish the spectral characteristics of different pathological tissues.

[0015] The pathological tissue spectral features are obtained by extracting features from pre-collected spectral data samples of pathological tissues.

[0016] According to a spectral imaging chip provided in this application, pathological classification is performed based on the spectral response to obtain classification results, specifically including:

[0017] The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor;

[0018] The spectral reconstruction results are obtained by calculating the light response matrices for different wavelengths and the spectral response in advance.

[0019] Based on the spectral reconstruction results, feature extraction is performed to obtain spectral feature data;

[0020] The spectral feature data is input into a pre-trained first spectral pathological feature model to obtain a classification result; wherein, the first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples.

[0021] According to the spectral imaging chip provided in this application, the first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples, specifically including:

[0022] Collect spectral response data samples from pathological tissues;

[0023] Pathological features of tissues with different pathological properties are extracted from the spectral response data of the pathological tissue to obtain spectral pathological feature samples;

[0024] The spectral pathological feature samples are categorized;

[0025] A first basic spectral feature model is constructed based on a neural network;

[0026] The first basic spectral feature model is trained using the labeled spectral pathological feature samples until a preset termination condition is met, thus obtaining the first spectral pathological feature model.

[0027] According to a spectral imaging chip provided in this application, pathological classification is performed based on the spectral response to obtain classification results, specifically including:

[0028] The spectral response is input into a pre-trained second spectral pathological feature model to obtain the classification result;

[0029] The second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples.

[0030] According to the spectral imaging chip provided in this application, a second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples, specifically including:

[0031] Collect spectral response data samples from pathological tissues;

[0032] The spectral response data samples are categorized.

[0033] A second fundamental spectral feature model is constructed based on a neural network;

[0034] The second basic spectral feature model is trained using the labeled spectral response data samples until a preset termination condition is met, thus obtaining the second spectral pathological feature model.

[0035] According to the spectral imaging chip provided in this application, the chip further includes a lens interface; the lens interface is used to connect an external detection lens or microscope.

[0036] According to a spectral imaging chip provided in this application, the optical system includes a lens assembly and / or a light homogenizing assembly.

[0037] This application also provides a spectral pathology diagnostic method, applied to any of the aforementioned spectral imaging chips, comprising:

[0038] The first input light is obtained by adjusting the propagation and distribution of the input light at the location to be diagnosed based on the optical system.

[0039] The first input light is modulated based on the filter structure to obtain the light to be measured.

[0040] The spectral response corresponding to the light under test is obtained based on the image sensor;

[0041] The data processing unit performs pathological classification based on the spectral response to obtain the classification result.

[0042] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the above-described spectral pathology diagnostic methods.

[0043] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spectral pathology diagnostic method as described above.

[0044] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the above-described spectral pathology diagnostic methods.

[0045] This application provides a spectral imaging chip and a spectral pathological diagnosis method. The chip includes an optical system, a filter structure, an image sensor, and a data processing unit. The optical system is used to adjust the propagation and distribution of input light at the location to be diagnosed, obtaining a first input light. The filter structure is used to modulate the first input light, obtaining a test light. The image sensor is used to acquire the spectral response corresponding to the test light. The data processing unit is used to perform pathological classification based on the spectral response, obtaining a classification result. This application utilizes an optical system, filter structure, and image sensor to quickly, conveniently, and cost-effectively introduce spectral material information into pathological diagnosis, demonstrating good feasibility. Simultaneously, the data processing unit efficiently and accurately analyzes the lesion characteristics of pathological tissues, intelligently classifying and determining pathological conditions, thereby improving the accuracy and efficiency of diagnosis. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 is a schematic diagram of the structure of the spectral imaging chip provided in an embodiment of this application.

[0048] Figure 2 is a schematic diagram of the classification results of the spectral imaging chip provided in the embodiment of this application.

[0049] Figure 3 is a flowchart of the training and application of the second spectral pathological feature model of the spectral imaging chip provided in the embodiments of this application.

[0050] Figure 4 is a schematic flowchart of the spectral pathological diagnosis method provided in the embodiments of this application.

[0051] Figure 5 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0053] The spectral imaging chip of this application is described below with reference to Figures 1-3. Figure 1 is a schematic diagram of the structure of the spectral imaging chip provided in the embodiment of this application. As shown in Figure 1, the chip includes an optical system 110, a filter structure 120, an image sensor 130, and a data processing unit 140.

[0054] The optical system is used to adjust the propagation and distribution of the input light at the acquired location to be diagnosed, thereby obtaining a first input light;

[0055] The filter structure is used to modulate the first input light to obtain the light to be measured;

[0056] The image sensor is used to acquire the spectral response corresponding to the light under test;

[0057] The data processing unit is used to perform pathological classification based on the spectral response to obtain classification results.

[0058] It should be clarified that the location to be diagnosed is the location of the object being tested that requires pathological diagnosis, and this application does not limit this. The object to be tested refers to the object that needs to be detected, including human beings, animals, and other objects, and this application does not limit this either.

[0059] The input light is the reflected light from a light source at the location to be diagnosed, and this application does not limit the method of acquiring the input light. In some embodiments, a detection device is used to penetrate deep into the object under test to illuminate it and acquire the reflected light as the input light.

[0060] The optical system 110 is an optical device for adjusting the propagation and distribution of input light, and the optical system 110 is located on the photosensitive path of the image sensor 130.

[0061] It should be noted that, in some embodiments, the optical system 110 is an optical system such as a lens assembly or a light-diffusing assembly. In one specific embodiment, the optical system includes a lens assembly and / or a light-diffusing assembly.

[0062] The filter structure 120 is a broadband filter structure in the frequency domain or wavelength domain, and is also located on the photosensitive path of the image sensor 130. It should be noted that the optical system 110 and the filter structure 120 are sequentially arranged on the photosensitive path of the image sensor 130 along the incident direction of the input light.

[0063] Furthermore, the transmittance of the filter structure at various spatial locations of the spectral imaging chip is not entirely the same for different wavelengths.

[0064] In practical applications, the input light is first adjusted by the optical system 110, then modulated by the filter structure 120, and finally received by the image sensor 130 to obtain the spectral response.

[0065] Furthermore, after obtaining the spectral response, the data processing unit 140 is responsible for exporting it and further processing it outside the device.

[0066] It should be further explained that the filter structure 120 can be a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties, and this application does not limit this. For example, in a specific embodiment, the filter structure 120 can be the light modulation layer in Chinese Patent CN201921223201.2.

[0067] The image sensor 130 can be a CMOS image sensor (CIS), CCD, array photodetector, etc., and this application does not limit it.

[0068] The data processing unit 140 can be a processing unit such as an MCU, CPU, GPU, FPGA, NPU, or ASIC, and this application does not limit it.

[0069] The spectral imaging chip provided in this application can utilize more miniaturized and low-cost spectral imaging equipment to quickly and conveniently introduce spectral material information into pathological diagnosis. It can also efficiently and accurately analyze the pathological tissue's lesion characteristics using artificial neural networks or other data processing units, intelligently classifying and judging pathological conditions. This avoids the subjectivity of manual diagnosis, improves diagnostic accuracy, and can solve the problem of diagnosing pathological conditions with no obvious morphological lesions or similar morphologies. Compared with existing microscopic pathological imaging technologies, this application eliminates the need for a microscopic system and corresponding slide preparation process, simplifying diagnostic steps and directly targeting gross applications. Compared with existing spectral imaging technologies, the solution in this application can reduce equipment size, accelerate spectral imaging time, reduce equipment cost, and better realize real-time pathological diagnosis in important medical scenarios such as in vivo and intraoperative procedures.

[0070] The filter structure 120 is further described below. In some embodiments, the filter structure includes a predetermined number of structural units, which are micro / nano modulation structures designed and fabricated using a predetermined algorithm to distinguish the spectral characteristics of different pathological tissues;

[0071] The pathological tissue spectral features are obtained by extracting features from pre-collected spectral data samples of pathological tissues.

[0072] Specifically, the filter structure 120 has m sets of structural units, each set of structural units has a different transmission spectrum. The design method of the structural units includes the following steps: collecting spectral data of pathological tissues, extracting the spectral features of different tissue parts such as cancer, necrosis, and muscle through principal component analysis, and then designing micro-nano modulation structures that have a significant distinguishing effect on the spectral features of different pathological tissue parts through artificial intelligence optimization algorithms.

[0073] It should be noted that in the design process of micro-nano modulation structures, some embodiments, based on the spectral characteristics of existing pathological sample tissue parts, comprehensively consider the similarity between metasurface structures, the correlation between transmission spectra, and the resolution capability of metasurfaces for characteristic spectra. Based on artificial intelligence optimization algorithms such as genetic algorithms, metasurfaces with different structures and periods are designed for cancer pathology diagnosis application scenarios.

[0074] Subsequently, each structural unit is prepared based on the design results, thereby obtaining the filter structure.

[0075] The data processing unit 140 is further described below. In some embodiments, pathological classification is performed based on the spectral response to obtain classification results, specifically including:

[0076] The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor;

[0077] The spectral reconstruction results are obtained by calculating the light response matrices for different wavelengths and the spectral response in advance.

[0078] Based on the spectral reconstruction results, feature extraction is performed to obtain spectral feature data;

[0079] The spectral feature data is input into a pre-trained first spectral pathological feature model to obtain the classification result;

[0080] The first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples.

[0081] Specifically, the spatial spectral data acquired by the spectral imaging chip is processed by combining spectral reconstruction algorithms and artificial neural network algorithms to achieve disease diagnosis.

[0082] In practical applications, after the input light is adjusted by the optical system 110 and modulated by the filter structure 120, the spectral response can be measured by the image sensor 130, and then the data processing unit 140 first performs spectral recovery calculation.

[0083] Specifically, the process includes the following steps: The intensity signals of the incident light at different wavelengths λ are denoted as x(λ). The filter structure 120 has m sets of structural units, each with a different transmission spectrum. Therefore, the transmission spectra of the filter structure can be denoted as Ti(λ) (i = 1, 2, 3, ..., m). Below each set of structural units, there is a corresponding physical pixel that detects the light intensity bi modulated by that filter structural unit. In practice, one physical pixel can correspond to one structural unit, or multiple physical pixels can be grouped together to correspond to one structural unit.

[0084] For example, in one specific embodiment, multiple sets of structural units and the corresponding image sensor 130 together constitute a spectral pixel. This application can use at least one spectral pixel to reconstruct the spectrum of an image. That is, the target spectral pixel can be all pixels or a selected subset of pixels; this application does not impose any limitations on this.

[0085] It is important to note that the number of effective transmission spectra (i.e., the transmission spectrum used for spectral reconstruction) Ti(λ) of the filter structure 120 may not be the same as the number of structural units; it may be less or more than the number of structural units. This can be manually set, tested, or calculated according to certain rules based on the needs of identification or reconstruction. A particular transmission spectrum curve is not necessarily determined by only one structural unit.

[0086] After the incident light passes through the optical system 110 and the filter structure 120, the relationship between the measured value obtained by the image sensor 130 and the spectral distribution of the incident light can be expressed by the following formula. i =∫x(λ)T i (λ)R(λ)dλ.

[0087] Where R(λ) is the response of the image sensor. Discretizing it, we get b. i =∑(x(λ)T i (λ)R(λ)).

[0088] Note: A i (λ)=T i (λ)R(λ).

[0089] The above formula can then be extended into matrix form.

[0090] Where bi (i = 1, 2, 3, ..., m) is the response of the image sensor 130 after the light under test passes through the filter structure 120, corresponding to the light intensity measurement values ​​of the image sensor corresponding to each of the m structural units. When one physical pixel corresponds to one structural unit, b can be understood as the light intensity measurement values ​​corresponding to m "physical pixels", which is a vector of length m. A is the system's response matrix for different wavelengths of light, with m rows and n columns. Each row vector of the matrix corresponds to a set of structural units' responses to incident light of different wavelengths, and the response value is determined by the transmittance of the filter structure 120 and the quantum efficiency of the image sensor 130. The incident light is sampled discretely and uniformly, with a total of n sampling points, and the number of columns in A is the same as the number of sampling points n of the incident light. x(λ) is the incident light spectrum to be measured, corresponding to the light intensity of the incident light at n different wavelength sampling points, which is a vector of length n. Since the system characteristic matrix A is known, after obtaining the corresponding b from the image sensor, x can be solved inversely, thus completing the reconstruction of the incident light spectrum and obtaining the spectral reconstruction result. Furthermore, in some embodiments, after obtaining the reconstructed spectral reconstruction result, it can be processed by the data processing unit 140 to obtain high color gamut, high bit depth color data, achieving high-fidelity transmission of the color data. This color data can be used for computer vision applications, encoded, stored, or directly displayed through the display unit of a terminal device.

[0091] Subsequently, spectral feature data is obtained based on the spectral reconstruction results. This application does not restrict the method of extracting spectral feature data; feature extraction can be performed through methods such as principal component analysis.

[0092] The extracted spectral feature data is input into a pre-trained first spectral pathological feature model for pathological diagnosis, yielding classification results. Figure 2 shows the classification results of one embodiment. As shown in Figure 2, the classification results represent the pathological classification of different regions in the tissue at the location to be diagnosed.

[0093] It should be noted that the first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples.

[0094] In some embodiments, the first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples, specifically including:

[0095] Collect spectral response data samples from pathological tissues;

[0096] Pathological features of tissues with different pathological properties are extracted from the spectral response data of the pathological tissue to obtain spectral pathological feature samples;

[0097] The spectral pathological feature samples are categorized;

[0098] A first basic spectral feature model is constructed based on a neural network;

[0099] The first basic spectral feature model is trained using the labeled spectral pathological feature samples until a preset termination condition is met, thus obtaining the first spectral pathological feature model.

[0100] Specifically, the first step is to obtain spectral response data samples of pathological tissues. It should be noted that this application does not limit the source of the spectral response data samples of pathological tissues; they can be existing tissue spectral samples or spectral response data samples of pathological tissues acquired using the spectral imaging chip provided in this application.

[0101] Subsequently, pathological features of tissue portions with different pathological properties are extracted from the spectral response data of the pathological tissue to obtain spectral pathological feature samples. Feature extraction methods include, but are not limited to, using prior experience to extract features from the samples. Specifically, based on the doctor's annotation of existing tissue samples and spectral data with pathological feature markers pre-acquired for a specific disease, the pathological features of the area to be diagnosed are obtained through comparative analysis.

[0102] Next, the spectral pathological feature samples are categorized. That is, the samples are categorized using pathological features. The spectral pathological diagnostic neural network corresponding to the disease is trained based on the categorized samples until the preset termination condition is met, and the first spectral pathological feature model is obtained.

[0103] It should be noted that the preset termination conditions include preset training rounds or accuracy verification conditions.

[0104] The data processing unit 140 is further described below. In some embodiments, pathological classification is performed based on the spectral response to obtain classification results, specifically including:

[0105] The spectral response is input into a pre-trained second spectral pathological feature model to obtain the classification result;

[0106] The second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples.

[0107] Specifically, in terms of data processing in this embodiment, since the output response of the image sensor corresponding to each filter structure group is equivalent to the projection of the incident light spectral vector onto the filter structure measurement base, the response itself carries spectral information. When addressing pathological diagnostic results without needing to analyze the spectral features corresponding to the pathological state, the spectral reconstruction step can be omitted. The measured values ​​of the image sensor (i.e., the spectral response) can be directly used as the input to the artificial neural network. Data processing and analysis, as well as the training of the corresponding spectral pathological model, are performed based on the second spectral pathological feature model, ultimately achieving accurate classification of different pathological sites. Compared to schemes requiring spectral reconstruction, this pathological diagnostic scheme saves more computational power and time, avoids errors introduced by the spectral reconstruction process, and improves detection speed and stability.

[0108] Furthermore, the second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples, specifically including:

[0109] Collect spectral response data samples from pathological tissues;

[0110] The spectral response data samples are categorized.

[0111] A second fundamental spectral feature model is constructed based on a neural network;

[0112] The second basic spectral feature model is trained using the labeled spectral response data samples until a preset termination condition is met, thus obtaining the second spectral pathological feature model.

[0113] Specifically, the training process of the second spectral pathological feature model is similar to that of the first spectral pathological feature model. First, spectral response data samples of pathological tissues are acquired. It should be noted that this application does not restrict the source of the spectral response data samples of pathological tissues; they can be existing tissue spectral samples or spectral response data samples of pathological tissues acquired using the spectral imaging chip provided in this application.

[0114] Next, the spectral response data samples are categorized. The categorization method is not limited to using prior experience to categorize the samples. Specifically, based on the annotations of existing tissue samples by doctors and the spectral data with pathological feature labels pre-acquired for specific diseases, a comparative analysis is performed to determine the category of the region to be diagnosed in the spectral response data samples.

[0115] Then, the spectral pathological diagnostic neural network corresponding to the disease is trained based on the labeled samples until the preset termination condition is met, thus obtaining the second spectral pathological feature model.

[0116] It should be noted that the preset termination conditions include preset training rounds or accuracy verification conditions.

[0117] The following is an example of training and applying a second-spectrum pathological feature model, as shown in Figure 3.

[0118] By installing a spectral imaging chip, the spectral information of each pixel in the patient's tissue slice sample is collected to obtain spectral response data samples.

[0119] Based on the doctor's experience in labeling samples, the category label of each pixel in the patient's tissue slice is obtained;

[0120] The labeled spectral response data samples are processed to obtain the training set and the test set;

[0121] A convolutional neural network is built as the second basic spectral feature model. Based on the dataset, an automatic pathological diagnosis model (i.e., the second spectral pathological feature model) is trained to obtain the model.

[0122] The model automatically outputs pathological diagnosis and classification results by inputting real-time spectral images of actual patient tissues.

[0123] This application leverages the differences in spectral characteristics between cancerous and normal tissues, employing a computational spectral scheme based on filter structures to apply snapshot-style spectral imaging to medical pathology diagnosis, enabling efficient and accurate analysis of the pathological state of patient tissues. Building upon this, an artificial neural network algorithm is used to train a spectral pathology analysis model, developing a spectral imaging intelligent pathology diagnostic module to achieve intelligent and automated spectral pathology diagnosis. Furthermore, for scenarios such as in vivo tissue examination or intraoperative examination, based on this spectral imaging scheme and equipped with corresponding spectral pathology analysis technology, a portable spectral pathology detection device is developed, ultimately achieving real-time and automated pathological diagnosis of patient tissues in vivo or during surgery.

[0124] Furthermore, in some embodiments, the chip also includes a lens interface for connecting an external probe lens or microscope.

[0125] Specifically, the chip also includes a lens interface for configuring a lens to receive input light at the location to be diagnosed.

[0126] Furthermore, the probe lens can also be a microscope. When acquiring spectral images of pathological samples based on this spectral imaging device, the spectral imaging device can be connected to a microscope via a microscope interface to maintain the stability of the spectral imaging.

[0127] The lens interface provided in this application is universal for different imaging scenarios. On the one hand, it can be embedded into existing microscopic pathological imaging systems to supplement existing pathological diagnosis with spectral material information. On the other hand, by configuring the lens, it is possible to perform spectral imaging and acquire and analyze spectral pathological feature information on gross samples in scenarios such as surgery without the need for a microscopic imaging system, and train a spectral pathological diagnosis neural network model for gross samples to achieve real-time pathological diagnosis for in vivo or surgical applications.

[0128] To further illustrate the spectral imaging chip provided in this application, the following specific embodiments are given.

[0129] In one embodiment, it is understood that gastric cancer is a malignant tumor originating from the gastric mucosal epithelium, and its final diagnosis often requires the results of biopsy or cytological examination. A patient's gastric slice can be divided into areas such as cancerous, necrotic, fibrotic, and muscular regions, each with different spectral characteristics. Therefore, this embodiment can classify different regions using the spectral imaging chip provided in this application. By mounting a spectral imaging chip on a camera, the spectral information of the slice can be obtained. Using the spectral information of each pixel and existing labels, a convolutional neural network model can be trained. Finally, the trained model is used for pathological diagnosis. Inputting the spectral image of the patient's tissue obtained using the spectral imaging chip yields the pathological classification of different regions in the tissue. A sample of the test results is shown in Figure 2. The classification accuracy of lesions and necrotic regions on the test set reached 94.4%, improving the accuracy and speed of pathological diagnosis.

[0130] In another embodiment, it is understood that intraoperative pathological diagnosis involves rapidly slicing, staining, and microscopically examining tissue samples submitted during surgery to determine the nature of the tumor, its spread, and whether there is residual tumor at the surgical margins, thus helping clinicians choose the correct surgical treatment. This technology requires speed and accuracy, but existing technologies can only shorten the diagnostic time to about 30 minutes. This embodiment utilizes the spectral imaging chip provided in this application to extract, analyze, and train models from existing gross specimens, enabling rapid division of cancerous and healthy areas. This allows for real-time monitoring of patient tissue pathology during surgery, providing guidance for the surgeon's procedures.

[0131] The spectral imaging chip provided in this application includes an optical system, a filter structure, an image sensor, and a data processing unit. The optical system adjusts the propagation and distribution of input light at the location to be diagnosed to obtain a first input light. The filter structure modulates the first input light to obtain a test light. The image sensor acquires the spectral response corresponding to the test light. The data processing unit performs pathological classification based on the spectral response to obtain a classification result. This application utilizes an optical system, filter structure, and image sensor to quickly, conveniently, and cost-effectively introduce spectral material information into pathological diagnosis, demonstrating good feasibility. Simultaneously, the data processing unit efficiently and accurately analyzes the pathological characteristics of tissues, enabling intelligent classification and judgment of pathological findings, thereby improving the accuracy and efficiency of diagnosis.

[0132] The spectral pathology diagnosis method provided in this application is described below. The spectral pathology diagnosis method described below can be referred to in correspondence with the spectral imaging chip described above. Figure 4 is a schematic flowchart of the spectral pathology diagnosis method provided in an embodiment of this application. As shown in Figure 4, the method includes:

[0133] Step 410: Adjust the propagation and distribution of the input light at the location to be diagnosed based on the optical system to obtain the first input light;

[0134] Step 420: Modulate the first input light based on the filter structure to obtain the light to be measured;

[0135] Step 430: Obtain the spectral response corresponding to the light to be measured based on the image sensor;

[0136] Step 440: The data processing unit performs pathological classification based on the spectral response to obtain the classification result.

[0137] Specifically, by adding the spectral imaging chip provided in the aforementioned embodiments, this application provides a snapshot-type, real-time spectral pathological imaging diagnostic solution to improve the accuracy of pathological diagnosis; and it is expected to be applied in in vivo, surgical and other application scenarios to expand the application scope of pathological diagnosis.

[0138] According to the spectral pathological diagnosis method provided in this application, the filter structure includes a preset number of structural units, and the structural units are micro-nano modulation structures designed and prepared using a preset algorithm to distinguish the spectral characteristics of different pathological tissues.

[0139] The pathological tissue spectral features are obtained by extracting features from pre-collected spectral data samples of pathological tissues.

[0140] According to the spectral pathological diagnosis method provided in this application, pathological classification is performed based on the spectral response to obtain a classification result, specifically including:

[0141] The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor;

[0142] The spectral reconstruction results are obtained by calculating the light response matrices for different wavelengths and the spectral response in advance.

[0143] Based on the spectral reconstruction results, feature extraction is performed to obtain spectral feature data;

[0144] The spectral feature data is input into a pre-trained first spectral pathological feature model to obtain the classification result;

[0145] The first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples.

[0146] According to the spectral pathology diagnosis method provided in this application, a first spectral pathology feature model is obtained by training a neural network using pre-labeled spectral pathology feature samples, specifically including:

[0147] Collect spectral response data samples from pathological tissues;

[0148] Pathological features of tissues with different pathological properties are extracted from the spectral response data of the pathological tissue to obtain spectral pathological feature samples;

[0149] The spectral pathological feature samples are categorized;

[0150] A first basic spectral feature model is constructed based on a neural network;

[0151] The first basic spectral feature model is trained using the labeled spectral pathological feature samples until a preset termination condition is met, thus obtaining the first spectral pathological feature model.

[0152] According to the spectral pathological diagnosis method provided in this application, pathological classification is performed based on the spectral response to obtain a classification result, specifically including:

[0153] The spectral response is input into a pre-trained second spectral pathological feature model to obtain the classification result;

[0154] The second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples.

[0155] According to the spectral pathology diagnosis method provided in this application, a second spectral pathology feature model is obtained by training a neural network using pre-labeled spectral response samples, specifically including:

[0156] Collect spectral response data samples from pathological tissues;

[0157] The spectral response data samples are categorized.

[0158] A second fundamental spectral feature model is constructed based on a neural network;

[0159] The second basic spectral feature model is trained using the labeled spectral response data samples until a preset termination condition is met, thus obtaining the second spectral pathological feature model.

[0160] According to a spectral pathological diagnosis method provided in this application, the chip further includes a lens interface; the lens interface is used to connect an external detection lens or microscope.

[0161] According to a spectral pathological diagnosis method provided in this application, the optical system includes a lens assembly and / or a light homogenizing assembly.

[0162] The spectral pathological diagnosis method provided in this application involves adjusting the propagation and distribution of input light at the location to be diagnosed using an optical system to obtain a first input light; modulating the first input light using a filter structure to obtain a test light; acquiring the spectral response corresponding to the test light using an image sensor; and performing pathological classification based on the spectral response using a data processing unit to obtain a classification result. This application utilizes an optical system, filter structure, and image sensor to quickly, conveniently, and cost-effectively introduce spectral material information into pathological diagnosis, demonstrating good feasibility. Simultaneously, the data processing unit efficiently and accurately analyzes the lesion characteristics of pathological tissues, intelligently classifying and determining the pathology, thereby improving the accuracy and efficiency of diagnosis.

[0163] Figure 5 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 5, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. The processor 510, communication interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call data and instructions from the memory 530 to execute a spectral pathological diagnosis method. This method includes: adjusting the propagation and distribution of input light at the location to be diagnosed based on an optical system to obtain a first input light; modulating the first input light based on a filter structure to obtain a test light; acquiring the spectral response corresponding to the test light based on an image sensor; and performing pathological classification based on the spectral response using a data processing unit to obtain a classification result.

[0164] Furthermore, when the data and instructions in the aforementioned memory 530 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the spectral pathological diagnosis method provided by the above methods. The method includes: adjusting the propagation and distribution of input light at the location to be diagnosed based on an optical system to obtain a first input light; modulating the first input light based on a filter structure to obtain a test light; acquiring the spectral response corresponding to the test light based on an image sensor; and performing pathological classification based on the spectral response by a data processing unit to obtain a classification result.

[0166] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the spectral pathological diagnosis method provided by the methods described above. The method includes: adjusting the propagation and distribution of input light at the location to be diagnosed based on an optical system to obtain a first input light; modulating the first input light based on a filter structure to obtain a test light; acquiring the spectral response corresponding to the test light based on an image sensor; and performing pathological classification based on the spectral response using a data processing unit to obtain a classification result.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A spectral imaging chip, comprising an optical system, a filter structure, an image sensor, and a data processing unit; The optical system is used to adjust the propagation and distribution of the input light at the acquired location to be diagnosed, thereby obtaining a first input light; The filter structure is used to modulate the first input light to obtain the light to be measured; The image sensor is used to acquire the spectral response corresponding to the light under test; The data processing unit is used to perform pathological classification based on the spectral response to obtain classification results.

2. The spectral imaging chip according to claim 1, wherein, The filter structure includes a predetermined number of structural units, which are micro-nano modulation structures designed and fabricated using a predetermined algorithm to distinguish the spectral characteristics of different pathological tissues. The pathological tissue spectral features are obtained by extracting features from pre-collected spectral data samples of pathological tissues.

3. The spectral imaging chip according to claim 1, wherein, Pathological classification is performed based on the spectral response to obtain the classification results, specifically including: The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a structural unit and a fixed number of physical pixels of the image sensor; The spectral reconstruction results are obtained by calculating the light response matrices for different wavelengths and the spectral response in advance. Based on the spectral reconstruction results, feature extraction is performed to obtain spectral feature data; The spectral feature data is input into a pre-trained first spectral pathological feature model to obtain the classification result; The first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples.

4. The spectral imaging chip according to claim 3, wherein, The first spectral pathological feature model is obtained by training a neural network using pre-labeled spectral pathological feature samples, specifically including: Collect spectral response data samples from pathological tissues; Pathological features of tissues with different pathological properties are extracted from the spectral response data of the pathological tissue to obtain spectral pathological feature samples; The spectral pathological feature samples are categorized; A first basic spectral feature model is constructed based on a neural network; The first basic spectral feature model is trained using the labeled spectral pathological feature samples until a preset termination condition is met, thus obtaining the first spectral pathological feature model.

5. The spectral imaging chip according to claim 1, wherein, Pathological classification is performed based on the spectral response to obtain the classification results, specifically including: The spectral response is input into a pre-trained second spectral pathological feature model to obtain the classification result; The second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples.

6. The spectral imaging chip according to claim 5, wherein, The second spectral pathological feature model is obtained by training a neural network using pre-labeled spectral response samples, specifically including: Collect spectral response data samples from pathological tissues; The spectral response data samples are categorized. A second fundamental spectral feature model is constructed based on a neural network; The second basic spectral feature model is trained using the labeled spectral response data samples until a preset termination condition is met, thus obtaining the second spectral pathological feature model.

7. The spectral imaging chip according to claim 1, wherein, The chip also includes a lens interface; the lens interface is used to connect an external detection lens or microscope.

8. The spectral imaging chip according to claim 1, wherein, The optical system includes a lens assembly and / or a light-diffusing assembly.

9. A spectral pathology diagnostic method, applied to the spectral imaging chip as described in claim 1, comprising: The first input light is obtained by adjusting the propagation and distribution of the input light at the location to be diagnosed based on the optical system. The first input light is modulated based on the filter structure to obtain the light to be measured. The spectral response corresponding to the light under test is obtained based on the image sensor; The data processing unit performs pathological classification based on the spectral response to obtain the classification result.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the spectral pathology diagnostic method as described in claim 9.

Images