Endoscopic system capable of spectral imaging and detection method

Abstract

Provided in the present application are an endoscopic system capable of spectral imaging and a detection method. The system comprises: a detection device, configured for entering the interior of a target and illuminating a position of interest in the target, and acquiring reflected light from the position of interest in the target as input light; a spectral imaging device, configured for adjusting and modulating the received input light to give a spectral response; a data processing unit, configured for performing data processing according to the spectral response to give a processing result, wherein the data processing comprises spectral reconstruction; and a display unit, configured for displaying the processing result. On the basis of an existing endoscope, the present application quickly and easily introduces spectral information into the imaging process of the endoscope by means of the spectral imaging device and the data processing unit, thereby expanding material information for endoscopic imaging with cost-efficiency and high feasibility.

Description

Spectral imaging endoscope systems and detection methods

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 2024114475762, filed on October 16, 2024, entitled "Spectral Imaging Endoscopic System and Detection Method", which is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to the field of endoscopy technology, and more particularly to an endoscopy system and detection method capable of spectral imaging. Background Technology

[0004] An endoscope consists of a light source, lens, flexible tube, image sensor, eyepiece or display, etc. Its working principle is as follows: the light source penetrates into the object being probed through the flexible tube and illuminates the interior of the object being probed. The reflected light signal inside the object is received by the image sensor and converted into an image signal. The detected image signal is transmitted to the display device for display.

[0005] Based on the method of receiving light signals inside the object and the location of the detector, endoscopes can be divided into two types: fiber optic endoscopes and electronic endoscopes. In fiber optic endoscopes, the reflected light signal to be detected is transmitted to the outside of the object through tens of thousands of optical fibers, then received by an external image sensor and transmitted to a display device for display. Electronic endoscopes, on the other hand, integrate the image sensor (such as a charge-coupled device (CCD)) at the tip of a flexible tube, directly acquiring image signals inside the object, and then transmitting the electrical signals through a signal transmission line inside the tube for display. The flexible tube of an electronic endoscope houses the electrical signal transmission line, avoiding the drawbacks of fiber optic image bundles such as large size and susceptibility to breakage. It is more compact and portable, therefore, electronic endoscopes are generally preferred in practical applications.

[0006] Endoscopes have significant applications in industrial exploration, clinical medicine, and other fields. Depending on the specific application, endoscopes can be categorized into medical endoscopes and industrial endoscopes. Different applications require tailored designs. For example, medical endoscopes generally have upper limits on the size of the image sensor integrated at the tip of the flexible tube; for instance, fiberoptic upper gastrointestinal endoscopes typically require the outer diameter of the rigid tip and the main flexible tube to be no greater than 11 mm. Medical endoscopes can enter the human body through natural orifices or tiny incisions, helping doctors observe morphological lesions in internal tissues and organs, take samples, and perform intraoperative procedures. Specifically: On the one hand, during the examination phase, it helps doctors observe the lesions inside the body through natural orifices such as the mouth, nose, and anus, obtaining information on lesions such as the location of ulcers and tumors, and formulating the best treatment plan accordingly; on the other hand, it serves as the "eyes" of surgeons during surgery, providing them with a surgical field of vision for parts of the body such as the abdominal cavity, enabling surgeons to perform surgery with only tiny incisions, and patients only need to bear the cost of minimal invasiveness, thus minimizing the harm of surgical wounds to patients.

[0007] Optical information possesses numerous dimensions, including spatial and spectral dimensions. Beyond the spatial dimension information acquired through existing endoscopic imaging, the spectral dimension contains rich material information, serving as a fingerprint of matter. Since light travels at the speed of light without a medium, spectral information can be used for real-time, non-invasive detection of material composition and content. Spectral detection and analysis have significant application value and promising development prospects in medical and chemical fields such as pathological diagnosis, water quality monitoring, and precision agriculture. Currently, mature spectral detection schemes employ spatial optical path-based spectral splitting technology, using gratings or similar methods to separate light of different wavelengths at different spatial locations, allowing sensors at different locations to acquire intensity information at different wavelengths. Furthermore, a cubic spectral image of the target scene is acquired through point-by-point or line-by-line mechanical scanning.

[0008] Current endoscopic technology can only acquire the light intensity distribution and RGB three-channel color information inside an object, but cannot acquire high-dimensional spectral information. In applications involving the detection of material composition and content, sampling and subsequent chemical or physical analysis are required, thus real-time detection is not possible, and the object under test may be damaged. Especially in the pathological diagnosis of clinical medicine, the accuracy of visual observation through endoscopy alone is low. Generally, sampling, slide preparation, and observation of the slides are required, which is cumbersome, time-consuming, and heavily reliant on the doctor's subjective experience. The accuracy of early diagnosis is low, and the real-time issues of sampling operations and intraoperative scenarios can easily cause secondary harm to the patient.

[0009] Existing mature spectral imaging technologies are all based on spatial spectral dispersion. Because spatial spectral dispersion schemes can only detect one line in two-dimensional space at a time using an array detector, complete spectral imaging requires precise mechanical scanning line by line. This results in large spectral imaging devices, high stability requirements for the imaging system, and time-consuming spectral imaging processes, making real-time imaging impossible. Therefore, direct application to endoscopy presents a high technical barrier. For example, regarding medical endoscopes: if an electronic endoscope is used, the required stability during scanning makes long-distance operation with a flexible tube extremely difficult. Furthermore, considering insertion into the human body, the integrated unit at the tip of the tube must be sufficiently small. Integrating a complete spectral imaging device (with a volume on the order of 10 cm or more) including spatial spectral dispersion and scanning components into the tip of the tube is extremely challenging. If a fiber optic endoscope is used, the number of optical fibers corresponds to the number of image pixels that can be acquired, often around 10. 6 At this scale, a large number of spectral detection or spectral imaging devices need to be connected after the optical fiber, which is extremely costly and has very low feasibility.

[0010] In summary, existing technologies suffer from high costs and low feasibility. Summary of the Invention

[0011] This application provides an endoscope system and detection method capable of spectral imaging, which addresses the shortcomings of high cost and low feasibility in the prior art, and achieves low cost and high feasibility.

[0012] This application provides an endoscope system capable of spectral imaging, comprising the following modules:

[0013] A detection device is used to penetrate deep into the interior of a target and illuminate the target location to be measured, and to acquire the reflected light from the target location as input light;

[0014] A spectral imaging device is used to adjust and modulate the received input light to obtain a spectral response;

[0015] A data processing unit is used to process data based on the spectral response to obtain a processing result; wherein, the data processing includes spectral reconstruction;

[0016] The display unit is used to display the processing results.

[0017] According to the present application, an endoscope system capable of spectral imaging includes an optical system, a filter structure, and an image sensor; wherein the optical system and the filter structure are sequentially arranged along the incident direction of the input light on the photosensitive path of the image sensor.

[0018] The optical system is used to adjust the propagation and distribution of the input light to obtain the first input light;

[0019] The filter structure is used to modulate the first input light to obtain the light to be measured;

[0020] The image sensor is used to generate a spectral response based on the physical pixels corresponding to the light to be measured.

[0021] According to the spectral imaging endoscope system provided in this application, the filter structure includes a preset number of structural units, and the transmission spectra of the preset number of structural units are different from each other.

[0022] According to the spectral imaging endoscope system provided in this application, spectral reconstruction is performed based on the spectral response to obtain the processing result, specifically including:

[0023] The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor;

[0024] The processing result is obtained by calculating the light response matrix for different wavelengths and the spectral response in advance.

[0025] According to the spectral imaging endoscope system provided in this application, the detection device includes a light source, a lens, and a flexible tube;

[0026] Accordingly, when the endoscope is an electronic endoscope, the flexible tube contains an electrical signal transmission line;

[0027] When the endoscope is a fiber optic endoscope, the flexible tube contains an image-bundle optical fiber.

[0028] According to the present application, an endoscope system capable of spectral imaging includes a display unit comprising an eyepiece or a display.

[0029] This application also provides a detection method applied to any of the above-described spectrally imaging endoscope systems, comprising the following steps:

[0030] The detection device penetrates deep into the target and illuminates the target location to be measured, and the reflected light from the target location to be measured is obtained as the input light;

[0031] The spectral response is obtained by adjusting and modulating the received input light using a spectral imaging device.

[0032] The data processing unit performs data processing based on the spectral response to obtain the processing result; wherein, the data processing includes spectral reconstruction;

[0033] The processing results are displayed on the display unit.

[0034] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement any of the detection methods described above.

[0035] This application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the detection method as described above.

[0036] This application also provides a computer program product, including a computer program that, when executed by a processor, implements any of the detection methods described above.

[0037] This application provides a spectral imaging endoscope system and detection method. The system includes a detection device for penetrating deep into the target and illuminating the target location to be measured, acquiring the reflected light from the target location as input light; a spectral imaging device for adjusting and modulating the received input light to obtain a spectral response; a data processing unit for processing data based on the spectral response to obtain a processing result; wherein the data processing includes spectral reconstruction; and a display unit for displaying the processing result. Based on existing endoscopes, this application rapidly and conveniently introduces spectral dimension information into endoscopic imaging through a spectral imaging device and a data processing unit, expanding endoscopic imaging with material dimension information, offering low cost and high feasibility. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 is one of the structural schematic diagrams of the spectral imaging endoscope system provided in the embodiments of this application.

[0040] Figure 2 is a second schematic diagram of the spectral imaging endoscope system provided in the embodiments of this application.

[0041] Figure 3 is a third schematic diagram of the spectral imaging endoscope system provided in the embodiments of this application.

[0042] Figure 4 is an implementation flow of one embodiment of the spectral imaging endoscope system provided in this application.

[0043] Figure 5 is a flowchart illustrating the detection method provided in an embodiment of this application.

[0044] Figure 6 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0046] The spectral imaging endoscope system of this application is described below with reference to Figures 1-4. Figure 1 is one of the structural schematic diagrams of the spectral imaging endoscope system provided by this application. As shown in Figure 1, the system includes a detection device 110, a spectral imaging device 120, a data processing unit 130, and a display unit 140.

[0047] The detection device 110 is used to penetrate deep into the target and illuminate the target location to be measured, and to obtain the reflected light from the target location to be measured as the input light.

[0048] It should be explained that the detection device 110 includes means for illuminating and acquiring reflected light inside the object being measured. In some embodiments, the detection device 110 includes a light source 111, a lens 112, and a flexible tube 113.

[0049] The light source 111 provides sufficient illumination to allow clear visualization of the internal structure of the object being examined during an examination or procedure. In some embodiments, the light source 111 is a cold light source, which does not generate significant heat, thereby reducing potential damage to the tissue.

[0050] Lens 112 is used to collect reflected light from tissues inside the body and transmit it outside the device. In some embodiments, lens 112 typically consists of a set of lenses, which may be spherical or aspherical, and this application is not limited thereto. It may also include multiple lenses to optimize image quality.

[0051] The flexible tube 113 is a flexible conduit that allows doctors to explore and observe different parts of an object without causing significant trauma by manipulating it. In some embodiments, the tip of the tube can be bent, typically operated via a lever or steering mechanism.

[0052] Furthermore, the target location to be measured is the location of the object to be measured that needs to be probed using an endoscope, and this application does not limit this. The object to be measured refers to the object that needs to be probed, including human bodies, animals, and other objects, and this application does not limit this either.

[0053] Furthermore, this application does not restrict the manner in which the detection device 110 enters the object to be tested; it can enter through a surgical incision or through a natural cavity.

[0054] Based on the above embodiments, in actual application, after the doctor manipulates the flexible tube 113 to allow the detection device 110 to enter the human body and reach the target location to be tested, the light emitted by the reflected light source 111 inside the target location to be tested is captured by the lens 112 to obtain the input light.

[0055] The spectral imaging device 120 is used to adjust and modulate the received input light to obtain a spectral response.

[0056] Furthermore, Figure 3 shows a schematic diagram of a spectral imaging device. As shown in Figure 3, the spectral imaging device includes an optical system, a filter structure, and an image sensor; wherein, the optical system and the filter structure are sequentially arranged on the photosensitive path of the image sensor along the incident direction of the input light.

[0057] The optical system is used to adjust the propagation and distribution of the input light to obtain the first input light;

[0058] The filter structure is used to modulate the first input light to obtain the light to be measured;

[0059] The image sensor is used to generate a spectral response based on the physical pixels corresponding to the light to be measured.

[0060] Specifically, the spectral imaging device 120 includes a filter structure 122, an optical system 121, an image sensor 123, etc.

[0061] The filter structure 122 is a broadband filter structure in the frequency domain or wavelength domain, located on the photosensitive path of the image sensor 123. The transmittance of the filter structure at different spatial locations varies with different wavelengths. Furthermore, the optical system 121 is also located on the photosensitive path of the image sensor 123. It is important to emphasize that the optical system 121 and the filter structure 122 are sequentially arranged on the photosensitive path of the image sensor 123 along the direction of input light incidence.

[0062] In practical applications, the input light acquired by the detection device 110 is first adjusted for propagation and distribution by the optical system 121, then modulated by the filter structure 122 to obtain the light to be measured, and finally received by the image sensor 123 to acquire the spectral response. The obtained spectral response can be further derived and processed outside the object being measured.

[0063] Furthermore, in some embodiments, the filter structure 122 can be a metasurface, photonic crystal, nanopillar, multilayer film, dye, quantum dot, MEMS (microelectromechanical systems), FP etalon, cavity layer, waveguide layer, diffraction element, or other structures or materials with filtering properties. For example, in one specific embodiment, the filter structure can be the light modulation layer in Chinese Patent CN201921223201.2.

[0064] In some embodiments, the optical system 121 includes an optical system such as a lens assembly and a light-diffusing assembly.

[0065] In some embodiments, the image sensor 123 may be a CMOS image sensor (CIS), a CCD, an array photodetector, etc.

[0066] Based on the above embodiments, in some embodiments, the filter structure includes a preset number of structural units, and the transmission spectra of the preset number of structural units are different from each other. Specifically, after the input light is modulated by the filter structure, the spectral response can be measured by the image sensor, and then the spectral reconstruction calculation can be performed by the data processing unit. The process is specifically described as follows: the intensity signal of the incident light at different wavelengths λ is denoted as x(λ). The filter structure 122 has m sets of structural units, and the transmission spectrum of each set of structural units is different from each other. Therefore, the transmission spectrum of the filter structure can be denoted as Ti(λ) (i=1,2,3,…,m). Below each set of structural units, there is a corresponding physical pixel that detects the light intensity bi after being modulated by the filter structure unit.

[0067] It should be noted that the embodiments of this application do not limit the correspondence between structural units and physical pixels. One physical pixel can correspond to one structural unit, or multiple physical pixels can be grouped together to correspond to one structural unit, with the corresponding physical pixel-structural unit pair forming a spectral pixel. For example, in one specific embodiment, multiple groups of structural units and their corresponding image sensors 123 together constitute a spectral pixel.

[0068] It is important to note that the number of effective transmission spectra (i.e., the transmission spectrum used for spectral reconstruction) Ti(λ) of the filter structure 122 may not be the same as the number of structural units; it may be less or more than the number of structural units. This can be manually set, tested, or calculated according to certain rules based on the needs of recognition or reconstruction. A particular transmission spectrum curve is not necessarily determined by only one structural unit. Ultimately, the spectrum of the image can be reconstructed using at least one spectral pixel.

[0069] As an endoscopic device capable of spectral imaging, the spectral imaging device 120 in this application is more miniaturized, simpler to integrate, and lower in cost compared to existing spectral imaging technologies, making it more suitable for integration with endoscopes for spectral imaging. Furthermore, the spectral imaging time of this application is shorter, enabling real-time, non-invasive detection of substance composition and content using spectral information, which aligns with the application goals of endoscopes. It can directly target fields such as chemical reaction monitoring and clinical medical diagnosis, broadening the application scope and scenarios of endoscopes.

[0070] The data processing unit 130 is used to perform data processing based on the spectral response to obtain a processing result; wherein the data processing includes spectral reconstruction.

[0071] Specifically, the algorithm for spectral reconstruction is performed by the data processing unit 130. The data processing unit 130 can be a processing unit such as an MCU, CPU, GPU, FPGA, NPU, or ASIC, and this application does not impose any restrictions on it. The data processing unit 130 can receive the response generated by the aforementioned spectral imaging device and perform spectral reconstruction and other algorithmic operations on it.

[0072] Furthermore, data processing can also include intelligent detection. Specifically, in addition to the data after spectral reconstruction, the corresponding data from the spectral imaging device 120 can be transmitted to the data processing unit 130 for data analysis and intelligent detection algorithms, depending on the application scenario. It should be noted that this application does not limit the specific algorithm for intelligent detection; the algorithm can be selected according to the actual situation.

[0073] The following is an example of intelligent detection. In this example, based on the spectral characteristics of different substances, algorithms such as artificial neural networks are used to train a substance spectral analysis model, which is then integrated into the endoscope system to perform intelligent spectral analysis on the object under test, thereby achieving automatic and real-time endoscopic spectral and substance detection.

[0074] Furthermore, it should be noted that for endoscopic spectral imaging, if real-time requirements are strict and computing power is insufficient, the following optimizations can be made to the data processing in the embodiments of this application. On the one hand, in scenarios such as pathological diagnosis and chemical composition analysis, where only the detection results are considered and spectral features do not need to be analyzed, the spectral reconstruction step can be omitted. The spectral response of the endoscope can be directly used as input to the artificial neural network to train the corresponding spectral pathological model, ultimately achieving accurate classification of different components. On the other hand, in applications such as intraoperative pathological examination or uniform chemical reaction monitoring, only the spectral situation of local areas or a subset of pixels needs to be considered. Therefore, key points can be selectively chosen for spectral reconstruction. These optimization schemes save computing power and time, avoid errors introduced by the full-image spectral reconstruction process, and improve detection speed and stability.

[0075] Display unit 140 is used to display the processing result.

[0076] After obtaining the data processing results, the data processing unit 130 directly displays the processing results in the display unit 140. Based on the above embodiment, the processing results include spectral images or detection results.

[0077] In some embodiments, the display unit includes an eyepiece or a display.

[0078] To address the bottleneck of existing endoscopic technologies' inability to perform spectral imaging, this application introduces material-dimensional information into the endoscope, based on the unique spectral characteristics of each material. By applying a computational spectral scheme based on filter structures to the endoscope, a spectral imaging-enabled endoscope system is proposed. This system boasts high spatial and spectral resolution, can acquire real-time spectral image data, and is equipped with data analysis algorithms appropriate for potential application scenarios. Consequently, it can non-invasively and endoscopically acquire real-time spectral images of the interior of the object under test in scenarios such as industrial flaw detection, chemical reaction monitoring, tissue lesion detection, early cancer diagnosis, and intraoperative cancer diagnosis.

[0079] Furthermore, Figure 2 shows a schematic diagram of the structure of an spectrally imaging endoscope system according to one embodiment. In this embodiment, when the endoscope is an electronic endoscope, the flexible tube contains an electrical signal transmission line; when the endoscope is a fiber optic endoscope, the flexible tube contains an image bundle optical fiber.

[0080] Specifically, for fiber optic endoscopes, a spectral imaging device can be integrated into the outer end of the image bundle fiber for spectral imaging; while for electronic endoscopes, a spectral imaging device can be integrated into the lens at the inner end of the flexible tube, and its spectral response can be directly transmitted to the outside of the object being measured in the form of an electrical signal.

[0081] Furthermore, for electronic endoscopes, it is also possible to integrate an image sensor with a filter structure and an image sensor without a filter structure at the tip of the flexible tube. The former is used for spectral imaging, and the latter is used to directly display color or black and white images of the existing endoscopic field of view, so as to select key pixels in the spectral image or perform comparative analysis, etc.

[0082] The following further describes the specific details of the spectral reconstruction performed by the data processing unit 130. In some embodiments, spectral reconstruction is performed based on the spectral response to obtain the processing result, specifically including:

[0083] The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor;

[0084] The processing result is obtained by calculating the light response matrix for different wavelengths and the spectral response in advance.

[0085] Specifically, after the incident light passes through the optical system 121 and the filter structure 122, the relationship between the measured value obtained by the image sensor 123 and the spectral distribution of the incident light can be expressed by the following formula. i =∫x(λ)T i (λ)R(λ)dλ.

[0086] Where R(λ) is the response of the image sensor. Discretizing it, we get b. i =∑(x(λ)T i (λ)R(λ)).

[0087] Note: A i (λ)=T i (λ)R(λ).

[0088] The above formula can then be extended into matrix form.

[0089] After the light under test passes through the filter structure 122, it generates a response bi (i = 1, 2, 3, ..., m) in the image sensor 123, which corresponds to the light intensity measurement values ​​of the image sensor corresponding to the m structural units. When one physical pixel corresponds to one structural unit, b can be understood as the light intensity measurement values ​​corresponding to m "physical pixels", which is a vector of length m. A is the system's response matrix for different wavelengths of light, with m rows and n columns. Each row vector of the matrix corresponds to a set of structural units' responses to incident light of different wavelengths, and the response value is determined by the transmittance of the filter structure and the quantum efficiency of the image sensor. The incident light is sampled discretely and uniformly, with a total of n sampling points. The number of columns in A is the same as the number of sampling points n of the incident light. x(λ) is the spectrum of the incident light to be measured, corresponding to the light intensity of the incident light at the n different wavelength sampling points, which is a vector of length n. Since the system characteristic matrix A is known, after obtaining the corresponding b of image sensor 123, x can be solved by the data processing unit to complete the reconstruction of the incident light spectrum.

[0090] Furthermore, the target pixel referred to in the embodiments of this application can be all spectral pixels or a selected portion of spectral pixels.

[0091] It should be explained that, for spectral reconstruction operations, depending on the specific application scenario, selective reconstruction of some pixels or the entire spectral image can be performed. When the data processing unit 130 has limited computing power and only focuses on information from a portion of the spatial locations, a selective spectral reconstruction method can be adopted, focusing only on the spectral features of a subset of pixels. After obtaining the reconstructed spectral information, it can be further processed by the data processing unit to obtain high color gamut, high bit depth color data, achieving high-fidelity transmission of the color data. This color data can be used for computer vision applications, encoded, stored, or directly displayed through the display unit 140 of the terminal device.

[0092] Furthermore, in industrial fields such as mechanical flaw detection, corrosion detection, and reaction monitoring, as well as in medical fields such as clinical examination, early cancer diagnosis, and intraoperative pathological diagnosis, the spectral imaging endoscope based on this application can further process the results obtained by the data processing unit 130. By using algorithms such as artificial neural networks, a spectral feature model of the characteristic substances in the test object can be established, and the spectral features of the substances can be analyzed efficiently and accurately. This enables the identification, analysis, and precise positioning of characteristic substances inside the test object, making real-time, endoscopic object and content detection and analysis possible.

[0093] Furthermore, to further illustrate the spectral imaging endoscope system provided in this application, the following specific embodiments are provided.

[0094] In one embodiment, in the chemical and pharmaceutical industries, it is often necessary to monitor the actual situation of reactants in a reaction vessel. Traditional detection techniques are mostly offline tests; for example, chromatography requires removing samples from the reaction vessel for analysis, which is slow and requires preventing product decomposition, making it difficult to perform under real-world conditions. Other online analysis techniques that provide molecular information also have limitations; for example, nuclear magnetic resonance spectroscopy requires a special reaction environment and cannot monitor chemical reactions within the reaction vessel. For instance, in the synthesis of antibiotics, acyl chlorides react with organic carboxylates to generate important intermediate products, acid anhydrides, but techniques such as chromatography cannot detect the formation of acid anhydrides. Endoscopes, however, can enter the reaction vessel. By installing the spectral imaging endoscope provided in this application within the reaction vessel, different chemical substances can be identified through spectral information without the need for sampling. Furthermore, algorithms such as artificial neural networks can be used to quickly achieve intelligent detection and classification of products, and it is expected to enable non-invasive, real-time tracking of changes in reactants, intermediates, and products.

[0095] In another embodiment, in clinical examinations of diseases of the respiratory and digestive tracts, endoscopic observation of color and morphology alone is sometimes insufficient for a definitive diagnosis. Endoscopic biopsy needles or forceps are needed to extract tissue samples from the patient under the guidance of the lens for further examination. However, this method has limitations and risks: the diagnostic procedure is cumbersome and time-consuming; the sampling results are limited by the quantity and quality of the samples; and complications such as bleeding and infection may occur during the sampling process, potentially leading to misdiagnosis or harm to the patient's health. Furthermore, existing endoscopic imaging technologies and endoscopic sampling-based pathological diagnostic techniques for early-stage cancers suffer from low accuracy. Using the endoscopic device provided in this application that enables spectral imaging, doctors can acquire spectral images of patient tissues in real time using non-invasive or minimally invasive methods, thereby analyzing their material information. Pathological examinations of multiple tissues can be performed without sampling, reducing harm to the patient during the sampling process. Simultaneously, accurate pathological diagnoses, including for early-stage cancers, can be achieved based on spectral material information.

[0096] In another embodiment, during resection surgery for cancers such as gastric cancer and laryngeal cancer, doctors need to use endoscopes to locate lesions and determine treatment boundaries in order to formulate surgical plans and guide surgical procedures. However, due to the indistinct morphology and color characteristics of some lesions, relying solely on traditional endoscopic identification may lead to missed detections and incomplete lesion removal. While using other rapid pathological diagnostic methods such as tissue sampling and intraoperative frozen section can achieve higher diagnostic accuracy, these methods are cumbersome, taking at least 30 minutes, and involve significant technical complexity and medical risks, potentially harming patients. Using the spectral imaging endoscope provided in this application, doctors can acquire spectral information of lesions in real time at a non-invasive or minimally invasive cost, enabling intraoperative pathological examination through spectral images. Furthermore, algorithms such as artificial neural networks can be used to process the acquired spectral information and train models, thereby quickly dividing cancerous and healthy areas, achieving intelligent intraoperative pathological diagnosis, and providing timely reference for the doctor's surgical procedures. The implementation process of this technical solution is shown in the figure, including the following steps:

[0097] The spectral information of each pixel in the patient's tissue is obtained through the endoscope with the integrated spectral imaging device 120.

[0098] The data processing unit 130 processes the data using algorithms such as spectral reconstruction, and displays spectral images of the patient's tissues on the display unit 140.

[0099] Based on the doctor's experience in labeling the samples, the category label of each pixel in the patient's tissue was obtained, and the training set and test set were obtained after data processing.

[0100] Convolutional neural networks are built and trained using a dataset to obtain an automatic pathological diagnosis model;

[0101] The model automatically outputs pathological diagnosis and classification results by inputting real-time spectral images of actual patient tissues.

[0102] This application provides a spectral imaging endoscope system, comprising: a detection device for penetrating deep into a target and illuminating the target location to be measured, acquiring the reflected light from the target location as input light; a spectral imaging device for adjusting and modulating the received input light to obtain a spectral response; a data processing unit for processing data based on the spectral response to obtain a processing result; wherein the data processing includes spectral reconstruction; and a display unit for displaying the processing result. Based on existing endoscopes, this application rapidly and conveniently introduces spectral dimension information into endoscopic imaging through a spectral imaging device and a data processing unit, expanding endoscopic imaging with material dimension information, offering low cost and high feasibility.

[0103] The detection method provided in this application is described below. The detection method described below can be referred to in correspondence with the spectral imaging endoscope system described above. Figure 5 is a schematic flowchart of the detection method provided in an embodiment of this application. As shown in Figure 5, the method includes the following steps:

[0104] Step 510: Based on the detection device penetrating deep into the target and illuminating the target location to be measured, the reflected light from the target location to be measured is obtained as the input light;

[0105] Step 520: Adjust and modulate the received input light based on the spectral imaging device to obtain the spectral response;

[0106] Step 530: The data processing unit performs data processing based on the spectral response to obtain the processing result; wherein, the data processing includes spectral reconstruction;

[0107] Step 540: Display the processing result based on the display unit.

[0108] According to a detection method provided in this application, the spectral imaging device includes an optical system, a filter structure, and an image sensor; wherein the optical system and the filter structure are sequentially arranged along the incident direction of the input light on the photosensitive path of the image sensor;

[0109] The optical system is used to adjust the propagation and distribution of the input light to obtain the first input light;

[0110] The filter structure is used to modulate the first input light to obtain the light to be measured;

[0111] The image sensor is used to generate a spectral response based on the physical pixels corresponding to the light to be measured.

[0112] According to a detection method provided in this application, the filter structure includes a preset number of structural units, and the transmission spectra of the preset number of structural units are different from each other.

[0113] According to a detection method provided in this application, spectral reconstruction is performed based on the spectral response to obtain a processing result, specifically including:

[0114] The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor;

[0115] The processing result is obtained by calculating the light response matrix for different wavelengths and the spectral response in advance.

[0116] According to a detection method provided in this application, the detection device includes a light source, a lens, and a flexible tube;

[0117] Accordingly, when the endoscope is an electronic endoscope, the flexible tube contains an electrical signal transmission line;

[0118] When the endoscope is a fiber optic endoscope, the flexible tube contains an image-bundle optical fiber.

[0119] According to a detection method provided in this application, the display unit includes an eyepiece or a display.

[0120] The detection method provided in this application involves: a detection device penetrating deep into the target and illuminating the target location to be measured, acquiring the reflected light from the target location as input light; adjusting and modulating the received input light using a spectral imaging device to obtain a spectral response; performing data processing based on the spectral response using a data processing unit to obtain a processing result; wherein, the data processing includes spectral reconstruction; and displaying the processing result using a display unit. This application, based on existing endoscopes, rapidly and conveniently introduces spectral dimension information into endoscopic imaging through a spectral imaging device and a data processing unit, expanding endoscopic imaging with material dimension information, and is low-cost and highly feasible.

[0121] Figure 6 illustrates a schematic diagram of the physical structure of an electronic device. As shown in Figure 6, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640. The processor 610, communication interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call data and instructions from the memory 630 to execute a detection method. This method includes: based on the detection device penetrating deep into the target and illuminating the target's test location, acquiring the reflected light from the test location as input light; based on the spectral imaging device, adjusting and modulating the received input light to obtain a spectral response; based on the data processing unit, performing data processing according to the spectral response to obtain a processing result; wherein, the data processing includes spectral reconstruction; and based on the display unit, displaying the processing result.

[0122] Furthermore, the data and instructions in the aforementioned memory 630 can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0123] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the detection methods provided by the above methods. The method includes: based on the detection device penetrating into the interior of the target and illuminating the target's test location, acquiring the reflected light from the target's test location as input light; based on the spectral imaging device, adjusting and modulating the received input light to obtain a spectral response; based on the data processing unit, performing data processing according to the spectral response to obtain a processing result; wherein, the data processing includes spectral reconstruction; and based on the display unit, displaying the processing result.

[0124] In another aspect, this application also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the detection methods provided by the above methods. The method includes: based on a detection device penetrating into the interior of a target and illuminating the target's test location, acquiring the reflected light from the target's test location as input light; based on a spectral imaging device, adjusting and modulating the received input light to obtain a spectral response; based on a data processing unit, performing data processing according to the spectral response to obtain a processing result; wherein the data processing includes spectral reconstruction; and based on a display unit, displaying the processing result.

[0125] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An endoscope system capable of spectral imaging, comprising: A detection device is used to penetrate deep into the interior of a target and illuminate the target location to be measured, and to acquire the reflected light from the target location as input light; A spectral imaging device is used to adjust and modulate the received input light to obtain a spectral response; A data processing unit is used to process data based on the spectral response to obtain a processing result; wherein, the data processing includes spectral reconstruction; The display unit is used to display the processing results.

2. The spectrographically imageable endoscope system of claim 1, wherein, The spectral imaging device includes an optical system, a filter structure, and an image sensor; wherein the optical system and the filter structure are sequentially arranged along the incident direction of the input light on the photosensitive path of the image sensor; The optical system is used to adjust the propagation and distribution of the input light to obtain the first input light; The filter structure is used to modulate the first input light to obtain the light to be measured; The image sensor is used to generate a spectral response based on the physical pixels corresponding to the light to be measured.

3. The spectrographically imageable endoscope system of claim 2, wherein, The filter structure includes a predetermined number of structural units, and the transmission spectra of the predetermined number of structural units are different from each other.

4. The spectrographically imageable endoscope system of claim 3, wherein, Based on the spectral response, spectral reconstruction is performed to obtain the processing results, which specifically include: The spectral response of the target spectral pixel is acquired; wherein the spectral pixel is composed of a fixed number of physical pixels of the structural unit and the image sensor; The processing result is obtained by calculating the light response matrix for different wavelengths and the spectral response in advance.

5. The spectrographically imageable endoscope system of claim 1, wherein, The detection device includes a light source, a lens, and a flexible tube; Accordingly, when the endoscope is an electronic endoscope, the flexible tube contains an electrical signal transmission line; When the endoscope is a fiber optic endoscope, the flexible tube contains an image-bundle optical fiber.

6. The spectrographically imageable endoscope system of Claim 1, wherein, The display unit includes an eyepiece or a display.

7. A detection method applied to a spectrally imaging endoscopic system as described in any one of claims 1-6, comprising: The detection device penetrates deep into the target and illuminates the target location to be measured, and the reflected light from the target location to be measured is obtained as the input light; The spectral response is obtained by adjusting and modulating the received input light using a spectral imaging device. The data processing unit performs data processing based on the spectral response to obtain the processing result; wherein, the data processing includes spectral reconstruction; The processing results are displayed on the display unit.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the detection method as described in claim 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, the computer program implementing the detection method as described in claim 7 when executed by a processor.

10. A computer program product comprising a computer program that, when executed by a processor, implements the detection method as described in claim 7.

Images