Endoscopic imaging device, method, system and electronic equipment

Abstract

An endoscopic imaging device, method, system, and electronic device thereof are suitable for acquiring images of a target object. The endoscopic imaging device includes: an optical lens for receiving and focusing illumination light reflected from the target object; a spectral chip assembly, wherein the spectral chip assembly is correspondingly arranged at the focal plane of the optical lens and is configured to receive the illumination light focused by the optical lens to obtain raw signal data; and a data processor, wherein the data processor is communicatively connected to the spectral chip assembly and is configured to perform data processing on the raw signal data obtained by the spectral chip assembly to obtain spectral image data corresponding to the target object.

Description

Technical Field

[0001] The present invention relates to the technical field of endoscopes, and in particular to an endoscope imaging device and method, system and electronic equipment thereof. Background Art

[0002] An endoscope is a device that includes an image sensor, optical lens, lighting structure, and mechanical device. It can penetrate deep into the digestive tract or other tubes to observe the internal environment and transmit images to enable doctors or professionals to make diagnoses. Medical endoscopes require very high image quality because clear images enable doctors and professionals to make more accurate judgments and operate on patients. Furthermore, due to the complex environment within the digestive tract and other tubes, if medical endoscope images can clearly distinguish between mucosa, tissue, blood vessels, etc., it will undoubtedly be of great help to doctors and professionals.

[0003] Traditional electronic endoscopes typically use RGB sensors, which cover the visible light spectrum range of 400nm to 800nm, which is similar to ordinary lighting. Although the image can be realistic and clear, it cannot improve the contrast of capillaries and subcutaneous microvessels.

[0004] Currently, to improve contrast, one existing endoscopic imaging technology is narrowband imaging (NBI). This technique incorporates narrowband filters into the illumination system to produce illumination with central wavelengths of 415nm and 540nm, respectively, and a bandwidth of 30nm. Because the 415nm and 540nm wavelengths of light have different penetration depths within mucosal tissue, the 415nm wavelength primarily reveals superficial blood vessels, which appear brown, while the 540nm wavelength displays blood vessels within the submucosal layer as blue-green. This makes it easy to identify the distribution of blood vessels within the mucosa. However, this narrowband imaging technology not only suffers from insufficient brightness due to the use of narrowband filters, but also, because the filters are fixed, it cannot be combined to create images with other wavelengths.

[0005] Another existing endoscopic imaging technology is Intelligent Spectrocolorimetry (FICE). This technology does not utilize narrowband filters. Instead, it decomposes the RGB image acquired by a conventional electronic endoscope into a single-wavelength spectral image. It then generates a spectrum image of each wavelength by selecting a combination of red, green, and blue light at any wavelength. This spectrum image is then reconstructed using a spectral estimation algorithm to create the FICE image. Although this intelligent spectrocolorimetry technology can select any wavelength between 400nm and 600nm in 5nm intervals, allowing for up to 50 wavelength combinations to achieve electronic staining, FICE suffers from image clarity issues due to its use of RGB images for spectral estimation to obtain images at specific wavelengths. This makes it inferior to narrowband imaging in demonstrating microvascular morphology. Summary of the Invention

[0006] An advantage of the present invention is that it provides an endoscopic imaging device and its method, system and electronic equipment, which can obtain spectral images of any wavelength band, helping to achieve better electronic staining effects.

[0007] Another advantage of the present invention is that it provides an endoscopic imaging device and its method, system and electronic device. In one embodiment of the present invention, the endoscopic imaging device can use a spectral chip to directly obtain multi-spectral hyperspectral images from the physical level, so as to achieve higher electronic staining quality in combination with a spectral recovery algorithm.

[0008] Another advantage of the present invention is that it provides an endoscopic imaging device and its method, system and electronic equipment. In one embodiment of the present invention, the endoscopic imaging device can obtain images with higher brightness and obtain richer and more accurate spectral information, thereby helping to achieve better electronic staining effects.

[0009] Another advantage of the present invention is to provide an endoscopic imaging device and its method, system and electronic equipment. In one embodiment of the present invention, the endoscopic imaging device can provide doctors and professionals with better quality endoscopic images to enable more accurate judgment and operation on patients.

[0010] Another advantage of the present invention is that it provides an endoscopic imaging device, method, system, and electronic device thereof, wherein, in order to achieve the above-mentioned objectives, the present invention does not require the use of expensive materials or complex structures. Therefore, the present invention successfully and effectively provides a solution that not only provides a simple endoscopic imaging device, method, system, and electronic device thereof, but also increases the practicality and reliability of the endoscopic imaging device, method, system, and electronic device thereof.

[0011] In order to achieve at least one of the above advantages or other advantages and purposes, the present invention provides an endoscopic imaging device suitable for being disposed at a front end of an endoscope to acquire an image of a target object, wherein the endoscopic imaging device comprises:

[0012] an optical lens for receiving and focusing the illumination light reflected back from the target object;

[0013] a spectral chip assembly, wherein the spectral chip assembly is correspondingly arranged at the focal plane of the optical lens, and is used to receive the illumination light focused by the optical lens to obtain raw signal data; and

[0014] A data processor, wherein the data processor is communicatively connected to the spectral chip assembly and is used to perform data processing on the raw signal data obtained via the spectral chip assembly to obtain spectral image data corresponding to the target object.

[0015] According to one embodiment of the present invention, the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly arranged on the pixels of the image sensor, and the spectral filter is located between the optical lens and the image sensor, so that the illumination light focused by the optical lens first passes through the spectral filter and is then received by the image sensor.

[0016] According to an embodiment of the present invention, the spectral filter is a narrow spectral band filter.

[0017] According to an embodiment of the present invention, the narrow spectral band filter includes a plurality of filter arrays having the same specifications, and the plurality of filter arrays are respectively attached to a plurality of pixel arrays of the image sensor correspondingly.

[0018] According to one embodiment of the present invention, the data processor includes a direct reading module and a reconstruction module that are communicatively connected to each other, wherein the direct reading module is used to directly read the original signal data obtained via the image sensor to obtain a series of original spectral data of specific wavelengths, and the reconstruction module is used to reconstruct the electronic dyeing effect by superimposing multiple original spectral data of selected wavelengths and restoring them into multispectral image data.

[0019] According to an embodiment of the present invention, the spectral filter is a quantum dot thin film filter or a micro-nanostructure filter having a specific spectral response curve.

[0020] According to one embodiment of the present invention, the data processor includes a spectral reconstruction module and a reconstruction module that are communicatively connected to each other, wherein the spectral reconstruction module is used to perform spectral reconstruction processing on the original signal data obtained via the image sensor according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of specific wavelengths, wherein the reconstruction module is used to reconstruct the electronic dyeing effect by superimposing multiple reconstructed spectral data of selected wavelengths and restoring them into multispectral image data.

[0021] According to one embodiment of the present invention, the spectrum reconstruction algorithm model is L=S*P, where L is the light response input of a pixel; S is the system matrix representing the characteristics of the entire endoscope system; and P is the reconstructed spectrum of the pixel.

[0022] According to an embodiment of the present invention, the endoscopic imaging device further comprises a light source assembly, wherein the light source assembly is correspondingly disposed near the optical lens for emitting the illumination light to the target object.

[0023] According to an embodiment of the present invention, the light emitting direction of the light source assembly is substantially parallel to the optical axis direction of the optical lens.

[0024] According to another aspect of the present invention, the present invention further provides an electronic device, adapted to acquire an image of a target object, comprising:

[0025] At least one endoscopic imaging device, wherein the endoscopic imaging device comprises:

[0026] an optical lens for receiving and focusing the illumination light reflected back from the target object;

[0027] a spectral chip assembly, wherein the spectral chip assembly is correspondingly arranged at the focal plane of the optical lens, and is used to receive the illumination light focused by the optical lens to obtain raw signal data; and

[0028] a data processor, wherein the data processor is communicatively connected to the spectral chip assembly and is configured to perform data processing on the raw signal data obtained via the spectral chip assembly to obtain spectral image data corresponding to the target object; and

[0029] An electronic device body, wherein the electronic device body comprises:

[0030] a monitor component for displaying a corresponding image based on the spectral image data; and

[0031] An endoscope tube, wherein the endoscope tube communicatively connects the endoscopic imaging device and the monitor, and the endoscopic imaging device is arranged at the front end of the endoscope tube, wherein the endoscope tube is used to transmit the spectral image data obtained by the at least one endoscopic imaging device to the monitor.

[0032] According to another aspect of the present invention, the present invention further provides an endoscopic imaging method, comprising the steps of:

[0033] Acquiring raw signal data obtained by imaging illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor and is located between the optical lens and the image sensor;

[0034] The original signal data is processed to obtain spectral image data corresponding to the target object.

[0035] According to an embodiment of the present invention, the step of processing the raw signal data to obtain spectral image data corresponding to the target object includes the following steps:

[0036] directly reading the acquired raw signal data to obtain a series of raw spectral data of specific wavelengths, wherein the spectral filter is a narrow spectral band filter; and

[0037] The electronic staining effect is reconstructed by superimposing multiple original spectral data of selected wavelengths to restore them into multispectral image data.

[0038] According to an embodiment of the present invention, the step of processing the raw signal data to obtain spectral image data corresponding to the target object includes the following steps:

[0039] performing spectrum reconstruction processing on the acquired original signal data according to a spectrum reconstruction algorithm model to obtain a series of reconstructed spectrum data of specific wavelengths, wherein the spectrum filter is a wide spectral band filter; and

[0040] The electronic staining effect is reconstructed by superimposing a plurality of reconstructed spectral data of a selected wavelength and restoring them into multispectral image data.

[0041] According to another aspect of the present invention, the present invention further provides an endoscopic imaging system for acquiring an image of a target object, wherein the endoscopic imaging system comprises:

[0042] an acquisition module for acquiring raw signal data obtained by imaging illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor and is located between the optical lens and the image sensor;

[0043] A data processing module is used to process the original signal data to obtain spectral image data corresponding to the target object.

[0044] According to one embodiment of the present invention, the data processing module includes a direct reading module and a reconstruction module that are communicatively connected to each other, wherein the direct reading module is used to directly read the acquired original signal data to obtain a series of original spectral data of specific wavelengths, wherein the spectral filter is a narrow spectral band filter; wherein the reconstruction module is used to reconstruct the electronic dyeing effect by superimposing multiple original spectral data of selected wavelengths and restoring them into multispectral image data.

[0045] According to one embodiment of the present invention, the data processing module includes a spectral reconstruction module and a reconstruction module that are communicatively connected to each other, wherein the spectral reconstruction module is used to perform spectral reconstruction processing on the acquired original signal data according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of specific wavelengths, wherein the spectral filter is a wide spectral band filter; wherein the reconstruction module is used to reconstruct the electronic staining effect by superimposing multiple reconstructed spectral data of selected wavelengths and restoring them into multispectral image data.

[0046] According to another aspect of the present invention, the present invention further provides an electronic device, comprising:

[0047] at least one processor for executing instructions; and

[0048] a memory communicatively connected to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executed by the at least one processor to cause the at least one processor to perform some or all steps of an endoscopic imaging method, wherein the endoscopic imaging method includes the steps of:

[0049] Acquiring raw signal data obtained by imaging illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor and is located between the optical lens and the image sensor;

[0050] The original signal data is processed to obtain spectral image data corresponding to the target object.

[0051] Further objects and advantages of the present invention will be fully apparent from an understanding of the following description and accompanying drawings.

[0052] These and other objects, features and advantages of the present invention will be more fully understood from the following detailed description, accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1FIG. 1 is a block diagram of an endoscopic imaging device according to an embodiment of the present invention.

[0054] Figure 2 A schematic structural diagram of a spectral chip assembly of the endoscopic imaging device according to the above embodiment of the present invention is shown.

[0055] Figure 3 A schematic structural diagram of the spectral filter of the spectral chip assembly according to the above embodiment of the present invention is shown.

[0056] Figure 4A An example of a narrow spectral band filter of the spectral chip assembly according to the above embodiment of the present invention is shown.

[0057] Figure 4B An example of the micro-nanostructure filter of the spectral chip assembly according to the above embodiment of the present invention is shown.

[0058] Figure 5A An example of the data processor of the spectral chip assembly according to the above embodiment of the present invention is shown.

[0059] Figure 5B A modified example of the data processor according to the above embodiment of the present invention is shown.

[0060] Figure 6 A schematic diagram comparing the outputs of the endoscopic imaging device according to the above embodiment of the present invention and a traditional RGB chip is shown.

[0061] Figure 7 FIG. 4 is a flow chart of an endoscopic imaging method according to an embodiment of the present invention.

[0062] Figure 8 A schematic flow chart of one of the steps in the endoscopic imaging method according to the above embodiment of the present invention is shown.

[0063] Figure 9 is an example of an endoscopic imaging system according to an embodiment of the present invention.

[0064] Figure 10 is another example of the endoscopic imaging system according to the above-mentioned embodiment of the present invention.

[0065] Figure 11 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

[0066] Figure 12 FIG. 1 is a schematic structural diagram of another electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are for illustrative purposes only, and those skilled in the art will readily appreciate other obvious variations. The basic principles of the present invention defined in the following description may be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not depart from the spirit and scope of the present invention.

[0068] In the present invention, the term "a" or "an" in the claims and the specification should be understood as "one or more." That is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple. Unless the disclosure of the present invention clearly indicates that the number of the element is only one, the term "a" or "an" should not be understood as a unique or singular element, and the term "a" or "an" should not be understood as a limitation on the quantity.

[0069] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance. In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through a medium. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0070] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0071] Conventional electronic endoscopes use RGB sensors. While these sensors cover the visible light spectrum from 400nm to 800nm, similar to ordinary lighting, resulting in realistic and clear images, they do not improve the contrast of capillaries and subcutaneous microvessels. Therefore, the present invention provides an endoscopic imaging device, method, system, and electronic device that utilize a spectral chip rather than a traditional RGB sensor. This utilizes special materials on the pixels, combined with computational relationships between pixels, to reconstruct spectral images of any wavelength band using a spectral estimation algorithm, thereby achieving superior electronic staining effects.

[0072] Reference to the accompanying drawings Figures 1 to 6 As shown, an endoscopic imaging device according to an embodiment of the present invention is illustrated, which is suitable for being arranged at the front end of an endoscope to obtain an image of a target object. Specifically, as Figure 1 and Figure 2 As shown, the endoscopic imaging device 10 may include an optical lens 11 and a spectral chip assembly 12, wherein the optical lens 11 is used to receive and focus the illumination light reflected back by the target object, and the spectral chip assembly 12 is correspondingly arranged at the focal plane of the optical lens 11, and is used to receive the illumination light focused by the optical lens 11 to obtain original signal data, thereby obtaining spectral image data corresponding to the target object.

[0073] It is worth noting that the endoscopic imaging device 10 of the present application uses the spectral chip component 12 to replace the traditional RGB sensor, so that the endoscopic imaging device 10 can obtain spectral information of more spectral bands from a physical level, thereby making the information at a specific wavelength estimated by the spectrum more accurate and the image clearer.

[0074] More specifically, if Figures 1 to 3 As shown, the spectral chip assembly 12 of the endoscopic imaging device 10 may include a spectral filter 121 and an image sensor 122, wherein the spectral filter 121 is correspondingly arranged on the pixels of the image sensor 122, and the spectral filter 121 is located between the optical lens 11 and the image sensor 122, so that the illumination light focused by the optical lens 11 first passes through the spectral filter 121 and is then received by the image sensor 122 to obtain original signal data.

[0075] For example, in one example of the present invention, Figure 3As shown, the spectral filter 121 of the spectral chip assembly 12 can be, but is not limited to, implemented as a narrow spectral band filter 1211, wherein the narrow spectral band filter 1211 includes a plurality of filter arrays with the same specifications, and the plurality of filter arrays are respectively attached to the plurality of pixel arrays of the image sensor 122. For example, Figure 4A As shown, M*K pixels are selected from the image sensor 122 to form a pixel array. Then, a filter of fixed specifications is attached to each pixel in the pixel array to form a corresponding M*K filter array. In particular, the filter arrays attached to all the pixel arrays are completely consistent.

[0076] In other examples of the present invention, Figure 3 As shown, the spectral filter 121 of the spectral chip assembly 12 may also be implemented as, but not limited to, a wide spectral band filter with a specific spectral response curve, such as a quantum dot thin film filter 1212 or a micro-nano structure filter 1213. Figure 4B As shown, when the spectral filter 121 is implemented as the micro-nano structure filter 1213, the micro-nano structure filter 1213 is generally composed of photosensitive units (i.e., micro-nano structure units) made of special materials such as photonic crystals or plasma, and the specifications of the photosensitive units are completely consistent, and their spatial dimensions are generally at the micron level, but the photosensitive units and the pixels of the image sensor do not form a one-to-one correspondence, that is, one photosensitive unit can correspond to multiple pixels.

[0077] It is worth noting that, since an endoscope usually needs to penetrate into the digestive tract or other pipes to observe the internal environment (i.e., the target object is usually located in a dark environment), it is required to be equipped with a light source to provide illumination. Figure 1 As shown, the endoscopic imaging device 10 may further include a light source assembly 13, wherein the light source assembly 13 is correspondingly arranged near the optical lens 11 for emitting the illumination light to the target object, so that the illumination light reflected back by the target object can be received by the optical lens 11 for focusing.

[0078] Preferably, the light emitting direction of the light source assembly 13 is basically parallel to the optical axis direction of the optical lens 11, so that the illumination area of ​​the light source assembly 13 and the field of view area of ​​the optical lens 11 have a large overlapping range, so as to ensure that the target object illuminated by the light source assembly 13 can be photographed by the endoscopic imaging device 10 to obtain corresponding spectral image data.

[0079] For example, the light source assembly 13 may be, but is not limited to, implemented as a white light source, such as an LED, a halogen lamp, or a laser light source, or a combination thereof, as long as it meets the lighting requirements of the endoscope application scenario. In addition, the optical lens 11 of the present application may also be, but is not limited to, implemented as a traditional lens, as long as it meets the optical requirements of the endoscope application scenario, and this application will not elaborate on this.

[0080] It is worth mentioning that since the data acquired by the image sensor 122 in the endoscopic imaging device 10 of the present application is raw signal data, the spectral image data can be obtained only after data processing of the raw signal data, so that different target objects such as diseased organs or lesions can be accurately observed according to the spectral image data. Therefore, Figure 1 and Figure 2 As shown, the endoscopic imaging device 10 of the present application may further include a data processor 14, wherein the data processor 14 is communicatively connected to the spectral chip assembly 12 and is configured to perform data processing on the raw signal data obtained by the spectral chip assembly 12 to obtain the spectral image data. It is understood that the data processor 14 can be integrated with the image sensor 122 of the spectral chip assembly 12, or can be arranged separately; in other words, the data processor 14 of the present invention can be designed to be integrated with the spectral chip assembly 12, or can be designed to be relatively independent of the spectral chip assembly 12.

[0081] It is worth noting that the type of raw signal data acquired by the image sensor 122 of the spectral chip assembly 12 varies depending on the type of spectral filter 121 in the spectral chip assembly 12. For example, when the spectral filter 121 of the spectral chip assembly 12 is implemented as the narrow spectral band filter 1211, since the illumination light directly generates a narrowband transmission spectrum after passing through the narrow spectral band filter 1211, the corresponding narrowband multispectral data is suitable for direct pixel readout methods and generally does not require spectral reconstruction. However, when the spectral filter 121 of the spectral chip assembly 12 is implemented as the wide spectral band filter (such as the quantum dot thin film filter 1212 or the micro-nanostructure filter 1213), since each quantum dot glue or micro-nanostructure unit forms a broadband projected spectral filter, it is necessary to reconstruct the sample's spectral data based on the measured light response and the known transmission spectrum of each filter.

[0082] For example, in one example of the present invention, Figure 5AAs shown, the spectral filter 121 of the spectral chip assembly 12 is implemented as the narrow spectral band filter 1211. At this time, the data processor 14 of the endoscopic imaging device 10 may include a direct reading module 141 and a reconstruction module 142 that are communicatively connected to each other, wherein the direct reading module 141 is communicatively connected to the image sensor 122 for directly reading the original signal data obtained by the image sensor 122 to obtain a series of original spectral data of specific wavelengths, wherein the reconstruction module 142 is used to reconstruct the electronic staining effect by restoring the multiple original spectral data of selected wavelengths into multi-spectral image data by superimposing them.

[0083] In another example of the present invention, Figure 5B As shown, the spectral filter 121 of the spectral chip assembly 12 is implemented as the wide spectral band filter. At this time, the data processor 14 of the endoscopic imaging device 10 may include a spectral reconstruction module 41' and a reconstruction module 142' that are communicatively connected to each other, wherein the spectral reconstruction module 141' is communicatively connected to the image sensor 122, and is used to perform spectral reconstruction processing on the original signal data obtained through the image sensor 122 according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of specific wavelengths, wherein the reconstruction module 142' is used to reconstruct the electronic staining effect by superimposing a plurality of the reconstructed spectral data of the selected wavelengths to restore them into multispectral image data.

[0084] Preferably, the spectral reconstruction algorithm model can be, but is not limited to, implemented as: L=S*P, where L is the light response input of a pixel; S is the system matrix to represent the characteristics of the entire endoscope system; and P is the reconstructed spectrum of the pixel.

[0085] For example, the light response input L can be specifically expressed as Wherein, M represents the input light response dimension of each pixel point, wherein the dimension is equal to the number of quantum dots of the quantum dot thin film filter 1212 or the number of micro-nano structure units of the micro-nano structure filter 1213 .

[0086] The system matrix S is specifically expressed as Where E represents the light source radiation distribution matrix, C represents the CMOS spectral response matrix, F represents the filter's inherent spectral transmission rate matrix, and N is the latitude of the reconstructed spectral data.

[0087] The specific form of the reconstructed spectrum P is expressed as

[0088] It is worth noting that the spectral reconstruction algorithm model used to solve the reconstructed spectrum P generally employs nonlinear optimization methods, including simulated annealing, non-negative least squares, gradient descent, and convex optimization. It is understood that the spectral restoration effect of subsequent reconstruction algorithms generally improves as the number of filters in the filter increases.

[0089] In addition, the selected wavelength of the present invention is preferably selected according to the different diseased organs and lesions to be observed, so that the electronic staining effect (such as Figure 6 The device (as shown) can clearly display the diseased organs and lesions that need to be observed, which helps doctors or professionals make more accurate judgments and operations on patients.

[0090] Schematic method

[0091] Reference to the accompanying drawings Figure 7 and Figure 8 As shown, an endoscopic imaging method according to an embodiment of the present invention is illustrated. Specifically, as Figure 7 As shown, the endoscopic imaging method may include the steps of:

[0092] S100: Acquire raw signal data obtained by imaging illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor, and the spectral filter is located between the optical lens and the image sensor; and

[0093] S200: Processing the original signal data to obtain spectral image data corresponding to the target object.

[0094] More specifically, in one example of the present invention, Figure 8 As shown, the step S200 of the endoscopic imaging method may include the following steps:

[0095] S210: directly reading the acquired original signal data to obtain a series of original spectral data of specific wavelengths, wherein the spectral filter is a narrow spectral band filter; and

[0096] S220: Reconstructing the electronic dyeing effect by superimposing the plurality of original spectral data of the selected wavelengths and restoring them into multispectral image data.

[0097] In another example of the present invention, Figure 8 As shown, the step S200 of the endoscopic imaging method may further include the following steps:

[0098] S210′: performing spectrum reconstruction processing on the acquired original signal data according to a spectrum reconstruction algorithm model to obtain a series of reconstructed spectrum data of specific wavelengths, wherein the spectrum filter is a wide spectrum band filter; and

[0099] S220 ′: reconstructing the electronic staining effect by superimposing the plurality of reconstructed spectral data of the selected wavelength and restoring them into multispectral image data.

[0100] It is worth noting that the spectral reconstruction algorithm model can be, but is not limited to, implemented as: L=S*P, where L is the light response input of a pixel; S is the system matrix to represent the characteristics of the entire endoscope system; and P is the reconstructed spectrum of the pixel.

[0101] It is worth mentioning that, according to the above embodiment of the present invention, the endoscopic imaging method may further include the following steps before step S100:

[0102] A light source component is controlled to be turned on to emit the illumination light to the target object.

[0103] Schematic system

[0104] Reference to the accompanying drawings Figure 9 and Figure 10 As shown in FIG. 1 , an endoscopic imaging system according to an embodiment of the present invention is illustrated. Specifically, as Figure 9 and Figure 10 As shown, the endoscopic imaging system 300 may include:

[0105] an acquisition module 310 for acquiring raw signal data of illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly to form an image, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor and is located between the optical lens and the image sensor; and

[0106] A data processing module 320 is used to process the original signal data to obtain spectral image data corresponding to the target object.

[0107] More specifically, in one example of the present invention, Figure 9As shown, the data processing module 320 of the endoscopic imaging system 300 may include a direct reading module 321 and a reconstruction module 322 that are communicatively connected to each other, wherein the direct reading module 321 is used to directly read the acquired original signal data to obtain a series of original spectral data of specific wavelengths, wherein the spectral filter is a narrow spectral band filter; wherein the reconstruction module 322 is used to reconstruct the electronic staining effect by superimposing multiple original spectral data of selected wavelengths and restoring them into multispectral image data.

[0108] In another example of the present invention, Figure 10 As shown, the data processing module 320 of the endoscopic imaging system 300 may include a spectral reconstruction module 321' and a reconstruction module 322', wherein the spectral reconstruction module 321' is used to perform spectral reconstruction processing on the acquired original signal data according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data of specific wavelengths, wherein the spectral filter is a wide spectral band filter; wherein the reconstruction module 322' is used to reconstruct the electronic staining effect by superimposing a plurality of the reconstructed spectral data of the selected wavelengths and restoring them into multispectral image data.

[0109] It is worth noting that according to the above embodiments of the present invention, Figure 9 and Figure 10 As shown, the endoscopic imaging system 300 further includes a control module 330 , wherein the control module 330 is used to control the start of a light source assembly to emit the illumination light to the target object.

[0110] Schematic electronic equipment

[0111] Below, reference Figure 11 An electronic device according to an embodiment of the present invention is described below. Figure 11 As shown, the electronic device 90 includes one or more processors 91 and a memory 92 .

[0112] The processor 91 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions. In other words, the processor 91 includes one or more physical devices configured to execute instructions. For example, the processor 91 may be configured to execute instructions as a part of the following: one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical structures. Such instructions may be implemented to perform tasks, implement data types, convert the state of one or more components, implement technical effects, or otherwise obtain desired results.

[0113] The processor 91 may include one or more processors configured to execute software instructions. In addition or in lieu thereof, the processor 91 may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. The processors of the processor 91 may be single-core or multi-core, and the instructions executed thereon may be configured for serial, parallel and / or distributed processing. The various components of the processor 91 may optionally be distributed across two or more separate devices, which may be remotely located and / or configured to perform collaborative processing. Various aspects of the processor 91 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration.

[0114] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute the program instructions to implement some or all of the steps in the above-described illustrative method of the present invention, and / or other desired functions.

[0115] In other words, the memory 92 includes one or more physical devices configured to store machine-readable instructions that can be executed by the processor 91 to implement the methods and processes described herein. When implementing these methods and processes, the state of the memory 92 can be changed (e.g., to store different data). The memory 92 can include removable and / or internal devices. The memory 92 can include optical storage (e.g., CD, DVD, HD-DVD, Blu-ray Disc, etc.), semiconductor storage (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic storage (e.g., hard drive, floppy disk drive, tape drive, MRAM, etc.), etc. The memory 92 can include volatile, non-volatile, dynamic, static, read / write, read-only, random access, sequential access, location addressable, file addressable, and / or content addressable devices.

[0116] It will be appreciated that the memory 92 comprises one or more physical devices. However, various aspects of the instructions described herein may alternatively be transmitted via a communication medium (e.g., electromagnetic signals, optical signals, etc.) that is not held by a physical device for a limited period of time. Various aspects of the processor 91 and the memory 92 may be integrated together into one or more hardware logic components. These hardware logic components may include, for example, field programmable gate arrays (FPGAs), program and application specific integrated circuits (PASIC / ASICs), program and application specific standard products (PSSP / ASSPs), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0117] In one example, if Figure 11 As shown, the electronic device 90 may also include an input device 93 and an output device 94, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). For example, the input device 93 may be, for example, a camera module for collecting image data or video data, etc. As another example, the input device 93 may include or interface with one or more user input devices such as a keyboard, a mouse, a touch screen, or a game controller. In some embodiments, the input device 93 may include or interface with selected natural user input (NUI) components. Such component parts may be integrated or peripheral, and the transduction and / or processing of input actions may be handled on-board or off-board. Example NUI components may include a microphone for language and / or speech recognition; infrared, color, stereoscopic display, and / or depth cameras for machine vision and / or gesture recognition; a head tracker, eye tracker, accelerometer, and / or gyroscope for motion detection and / or intent recognition; and an electric field sensing component for evaluating brain activity and / or body movement; and / or any other suitable sensor.

[0118] The output device 94 can output various information to the outside, including classification results, etc. The output device 94 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0119] Of course, the electronic device 90 may further include the communication device, wherein the communication device may be configured to communicatively couple the electronic device 90 to one or more other computer devices. The communication device may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem may be configured to communicate via a wireless telephone network or a wired or wireless local area network or wide area network. In some embodiments, the communication device may allow the electronic device 90 to send messages to other devices and / or receive messages from other devices via a network such as the Internet.

[0120] It will be understood that the configurations and / or methods described herein are exemplary in nature, and that these specific embodiments or examples should not be considered restrictive, as many variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Thus, the various actions shown and / or described may be performed in the order shown and / or described, in other orders, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.

[0121] Of course, to simplify, Figure 11 Only some of the components related to the present invention in the electronic device 90 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 90 may further include any other appropriate components according to specific application scenarios.

[0122] According to another aspect of the present invention, an embodiment of the present invention further provides another electronic device. Figure 12 As shown, the electronic device includes an electronic device body 400 and at least one endoscopic imaging device 10 described above. The electronic device body 400 may include an endoscope tube 410 and a monitor assembly 420. The endoscopic imaging device 10 is communicatively configured at the front end of the endoscope tube 410 of the electronic device body 400, and the endoscopic imaging device 10 may include an optical lens 11 and a spectral chip assembly 12, wherein the optical lens 11 is used to receive and focus illumination light reflected back from a target object, and the spectral chip assembly 12 is correspondingly arranged at the focal plane of the optical lens 11, and is used to receive the illumination light focused by the optical lens 11 to form a light-sensitive image, thereby obtaining spectral image data corresponding to the target object, wherein the endoscope tube 410 transmits the spectral image data obtained by the endoscopic imaging device 10 to the monitor assembly 420, and the monitor assembly 420 is used to display a corresponding image based on the spectral image data.

[0123] More specifically, if Figure 12 As shown, the endoscopic imaging device 10 may further include a light source assembly 13, wherein the light source assembly 13 is correspondingly arranged near the optical lens 11 for emitting the illumination light to the target object, so that the illumination light reflected back by the target object can be received by the optical lens 11 for focusing.

[0124] Preferably, the optical lens 11 and the light source assembly 13 are arranged side by side at the front end of the endoscope tube 410 of the electronic device body 400 so as to observe internal target objects in the digestive tract or other pipes.

[0125] In addition, the monitor component 420 of the electronic device body 400 of the present application may include an endoscope controller 421, an image processor 422 and a display 423 that are communicatively connected to each other to realize multiple functions of the endoscope.

[0126] It is worth noting that Figure 12 As shown, the electronic device body 400 of the present application can be, but is not limited to, implemented as a medical endoscope such as a gastroscope, a colonoscope, or a bronchoscope. Of course, in other examples of the present application, the electronic device body 400 can also be implemented as an industrial endoscope.

[0127] It should also be noted that in the apparatus, device and method of the present invention, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present invention.

[0128] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0129] Those skilled in the art will appreciate that the embodiments of the present invention described above and shown in the accompanying drawings are intended to be illustrative only and are not intended to limit the present invention. The objectives of the present invention have been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and illustrated in the embodiments. Any variations or modifications may be made to the embodiments of the present invention without departing from the principles described.

Claims

1. An endoscopic imaging device, adapted to be disposed at the front end of an endoscope to acquire an image of a target object, characterized in that: Wherein the endoscopic imaging device comprises: an optical lens for receiving and focusing the illumination light reflected back from the target object; a spectral chip assembly, wherein the spectral chip assembly is correspondingly arranged at the focal plane of the optical lens, and is used to receive the illumination light focused by the optical lens to obtain raw signal data; and a data processor, wherein the data processor is communicatively connected to the spectral chip assembly and is configured to perform data processing on the raw signal data obtained via the spectral chip assembly to obtain spectral image data corresponding to the target object; The spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly arranged on the pixels of the image sensor, and the spectral filter is located between the optical lens and the image sensor, so that the illumination light focused by the optical lens first passes through the spectral filter and is then received by the image sensor; The data processor includes a spectral reconstruction module and a reconstruction module that are communicatively connected to each other, wherein the spectral reconstruction module is used to perform spectral reconstruction processing on the original signal data obtained by the image sensor according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data at specific wavelengths, and wherein the reconstruction module is used to reconstruct the electronic staining effect by superimposing a plurality of reconstructed spectral data at selected wavelengths to restore them into multispectral image data; The spectrum reconstruction algorithm model is L=S*P, where L is the light response input of a pixel; S is the system matrix to represent the characteristics of the entire endoscope system; and P is the reconstructed spectrum of the pixel. The system matrix S is expressed in the form of S=E*C*F, wherein E represents the light source radiation distribution matrix, C represents the CMOS spectral response matrix, and F represents the filter's inherent spectral transmission rate matrix.

2. The endoscopic imaging device according to claim 1, wherein The spectral filter is a quantum dot thin film filter or a micro-nano structure filter with a specific spectral response curve.

3. The endoscopic imaging device according to claim 1 or 2, further comprising a light source assembly, wherein the light source assembly is correspondingly arranged near the optical lens for emitting the illumination light to the target object.

4. The endoscopic imaging device according to claim 3, wherein: The light emitting direction of the light source assembly is substantially parallel to the optical axis direction of the optical lens.

5. An electronic device, adapted to acquire an image of a target object, characterized in that: include: The endoscopic imaging device according to any one of claims 1 to 4; and An electronic device body, wherein the electronic device body comprises: a monitor component for displaying a corresponding image based on the spectral image data; and An endoscope tube, wherein the endoscope tube communicatively connects the endoscopic imaging device and the monitor, and the endoscopic imaging device is arranged at the front end of the endoscope tube, wherein the endoscope tube is used to transmit the spectral image data obtained by the endoscopic imaging device to the monitor.

6. An endoscopic imaging method, characterized in that Including steps: Acquiring raw signal data obtained by imaging illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor and is located between the optical lens and the image sensor; performing data processing on the raw signal data to obtain spectral image data corresponding to the target object; The step of processing the original signal data to obtain spectral image data corresponding to the target object includes the following steps: performing spectrum reconstruction processing on the acquired original signal data according to a spectrum reconstruction algorithm model to obtain a series of reconstructed spectrum data of specific wavelengths, wherein the spectrum filter is a wide spectral band filter; and The electronic staining effect is reconstructed by superimposing a plurality of reconstructed spectral data of a selected wavelength to restore the multispectral image data; The spectrum reconstruction algorithm model is L=S*P, where L is the light response input of a pixel; S is the system matrix to represent the characteristics of the entire endoscope system; and P is the reconstructed spectrum of the pixel. The system matrix S is expressed in the form of S=E*C*F, wherein E represents the light source radiation distribution matrix, C represents the CMOS spectral response matrix, and F represents the filter's inherent spectral transmission rate matrix.

7. An endoscopic imaging system for acquiring an image of a target object, characterized in that The endoscopic imaging system includes: an acquisition module for acquiring raw signal data obtained by imaging illumination light reflected from a target object and focused by an optical lens, received by a spectral chip assembly, wherein the spectral chip assembly includes a spectral filter and an image sensor, wherein the spectral filter is correspondingly disposed on pixels of the image sensor and is located between the optical lens and the image sensor; a data processing module, configured to process the raw signal data to obtain spectral image data corresponding to the target object; The data processing module includes a spectral reconstruction module and a reconstruction module that are communicatively connected to each other, wherein the spectral reconstruction module is used to perform spectral reconstruction processing on the acquired original signal data according to a spectral reconstruction algorithm model to obtain a series of reconstructed spectral data at specific wavelengths, wherein the spectral filter is a wide spectral band filter; wherein the reconstruction module is used to reconstruct the electronic staining effect by superimposing a plurality of reconstructed spectral data at selected wavelengths to restore them into multispectral image data; The spectrum reconstruction algorithm model is L=S*P, where L is the light response input of a pixel; S is the system matrix to represent the characteristics of the entire endoscope system; and P is the reconstructed spectrum of the pixel. The system matrix S is expressed in the form of S=E*C*F, wherein E represents the light source radiation distribution matrix, C represents the CMOS spectral response matrix, and F represents the filter's inherent spectral transmission rate matrix.

8. An electronic device, characterized in that include: at least one processor for executing instructions; and A memory communicatively connected to the at least one processor, wherein the memory has at least one instruction, wherein the instruction is executed by the at least one processor to enable the at least one processor to perform all steps in the endoscopic imaging method according to claim 6.

Images