Endoscope imaging method and system, imaging method and device and electronic equipment

Abstract

The invention provides an endoscope imaging method and system, an imaging method and device and electronic equipment. The system comprises a white light illumination module used for obtaining standard white light and illuminating target tissue of an endoscope; the beam splitting module is used for splitting reflected light of the white light into a first light beam and a second light beam; the white light imaging processing module is used for acquiring a standard white light image by using the first light beam; meanwhile, the hyperspectral imaging processing module is used for collecting a hyperspectral image by using the second path of light beam, and selecting more than two narrow-band images of which the central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and performing fusion of a promotion and suppression mechanism and color channel combination based on at least any two of the more than two narrowband images to generate a pseudo-color image; and the image display module is used for displaying the standard white light image, the hyperspectral image and the pseudo-color image, and is used for realizing real-time inspection of the endoscope.

Description

Technical Field

[0001] The present invention relates to the field of imaging technology, and in particular to an endoscopic imaging method and system, an imaging method and device, and electronic equipment. Background Art

[0002] Image imaging in the related art requires the use of multiple light sources with different spectral characteristics to generate different images corresponding to the multiple light sources. For example, the light sources of different wavelengths include a light source of a first wavelength and a light source of a second wavelength. First, the user manually switches to the light source of the first wavelength, and then uses the light source of the first wavelength to generate an image corresponding to the light source of the first wavelength. Then, the user manually switches to the light source of the second wavelength, and uses the light source of the second wavelength to generate an image corresponding to the light source of the second wavelength.

[0003] In this way, the user needs to switch the light source multiple times, which complicates the user operation process and affects the imaging efficiency and real-time performance. Summary of the invention

[0004] The present application provides an improved endoscopic imaging method and system, an imaging method and device, and an electronic device.

[0005] The present application provides an endoscopic imaging system, comprising:

[0006] A white light illumination module, used to obtain standard white light to illuminate the target tissue of the endoscope;

[0007] A beam splitting module, used for splitting the reflected light of the white light into a first light beam and a second light beam;

[0008] A white light imaging processing module is used to obtain a standard white light image using the first light beam; at the same time,

[0009] A hyperspectral imaging processing module is used to collect a hyperspectral image using the second light beam, and select two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generate a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images;

[0010] The image display module is used to display the standard white light image, the hyperspectral image and the pseudo-color image, so as to realize real-time inspection of the endoscope.

[0011] Further, the more than two narrow-band images include any two narrow-band images; each time the mapping is performed, the promotion and inhibition mechanisms are fused based on the same two narrow-band images to generate a monochrome image; the hyperspectral imaging processing module includes: a mapping unit, and a promotion and inhibition fusion unit; the promotion and inhibition fusion unit is used to fuse the promotion and inhibition mechanisms of biomimetic vision based on the same two narrow-band images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on the same two narrow-band images to the first color channel of the RGB channel, the second color channel mapped to the RGB channel, and the third color channel mapped to the RGB channel; the images mapped by the first color channel, the second color channel, and the third color channel are combined by color channels to obtain a pseudo-color image;

[0012] or,

[0013] The two or more narrow-band images include any two or more narrow-band images; each time mapping is performed, a fusion of promotion and inhibition mechanisms is performed based on at least every two narrow-band images to generate a monochrome image accordingly; the hyperspectral imaging processing module includes: a mapping unit, and a fusion unit of promotion and inhibition; the fusion unit of promotion and inhibition is used to fuse the promotion and inhibition mechanisms of biomimetic vision based on the different two narrow-band images to generate the monochrome image accordingly; the mapping unit is used to map the monochrome image generated accordingly based on the different two narrow-band images to the first color channel of the RGB channel, the second color channel mapped to the RGB channel, and the third color channel mapped to the RGB channel; the images mapped by the first color channel, the second color channel and the third color channel are combined by color channels to obtain a pseudo-color image.

[0014] Furthermore, the image display module includes one or two displays; the endoscope imaging system further includes: a control module, which is used to receive switching instructions for the standard white light image, the hyperspectral image and the pseudo-color image, and switch to the image to be displayed;

[0015] or,

[0016] The image display module includes three displays; the three displays are used to display the standard white light image, the hyperspectral image and the pseudo-color image respectively.

[0017] Furthermore, the hyperspectral imaging processing module includes an HSI image acquisition unit:

[0018] The HSI image acquisition unit is used to obtain a hyperspectral image using a hyperspectral camera; the hyperspectral image includes a plurality of narrow-band light images generated by the hyperspectral camera acquiring reflected light within the entire spectral range at one time;

[0019] and / or,

[0020] The HSI image acquisition unit is used to obtain a hyperspectral image by combining an image sensor with a filter color wheel; the filter color wheel includes filters of different wavelengths, and the multiple narrow-band light images are obtained by rotating the filter color wheel to filter light of different wavelengths sequentially onto a monochromatic sensor;

[0021] and / or,

[0022] The HSI image acquisition unit is used to obtain a hyperspectral image using a grating spectrometer; the hyperspectral image is obtained by the grating spectrometer decomposing the incoming white light into a plurality of spectra of different wavelengths through a grating and recording the spectral information of each wavelength respectively.

[0023] The present application provides an endoscopic imaging method, comprising:

[0024] The white light illumination module obtains standard white light to illuminate the target tissue of the endoscope;

[0025] The beam splitting module splits the reflected light of the white light into a first light beam and a second light beam;

[0026] The white light imaging processing module uses the first light beam to obtain a standard white light image; at the same time,

[0027] The hyperspectral imaging processing module uses the second light beam to collect a hyperspectral image, and selects two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generates a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images;

[0028] The image display module displays the standard white light image, the hyperspectral image and the pseudo-color image, so as to realize real-time inspection of the endoscope.

[0029] The present application provides an imaging method, comprising:

[0030] Obtain standard white light to illuminate the object to be inspected;

[0031] Splitting the reflected light of the white light into a first light beam and a second light beam;

[0032] Using the first light beam, a standard white light image is obtained; at the same time,

[0033] Using the second light beam, a hyperspectral image is collected, and two or more narrow-band images whose central wavelength difference reaches a preset difference are selected from a plurality of narrow-band images of the hyperspectral image; and based on at least any two narrow-band images of the two or more narrow-band images, a fusion of promotion and inhibition mechanisms and a combination of color channels are performed to generate a pseudo-color image;

[0034] The standard white light image, the hyperspectral image, and the pseudo color image are displayed.

[0035] Further, the two or more narrow-band images include any two narrow-band images; each time any color channel of the RGB channel is mapped, the monochrome image is generated based on the same two narrow-band images;

[0036] The step of fusing the promotion and suppression mechanisms and combining the color channels based on at least any two of the two or more narrow-band images to generate a pseudo-color image includes:

[0037] The promotion and inhibition mechanisms of biomimetic vision are integrated based on the two narrow-band images to generate a monochromatic image;

[0038] Mapping the monochrome image generated based on the two narrow-band images to a first color channel of an RGB channel, to a second color channel of the RGB channel, and to a third color channel of the RGB channel;

[0039] Color channel combination is performed on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image.

[0040] Further, the two or more narrow-band images include any two or more narrow-band images; each time any color channel of the RGB channel is mapped, the monochrome image is generated based on each of the two narrow-band images;

[0041] The step of fusing the promotion and suppression mechanisms and combining the color channels based on at least any two of the two or more narrow-band images to generate a pseudo-color image includes:

[0042] The promotion and inhibition mechanisms of biomimetic vision are integrated based on at least the two narrow-band images to generate a monochromatic image accordingly;

[0043] Mapping the monochrome images correspondingly generated based on each of the two narrow-band images to a first color channel of an RGB channel, to a second color channel of the RGB channel, and to a third color channel of the RGB channel;

[0044] Color channel combination is performed on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image.

[0045] Furthermore, the acquisition of the hyperspectral image includes: using a hyperspectral camera to obtain a hyperspectral image; the hyperspectral image includes a plurality of narrow-band light images generated by the hyperspectral camera acquiring reflected light within the entire spectral range at one time;

[0046] and / or,

[0047] The method for collecting hyperspectral images comprises: using an image sensor and a color filter wheel to obtain a hyperspectral image; the color filter wheel comprises filters of different wavelengths, and the multiple narrow-band light images are obtained by rotating the color filter wheel to filter light of different wavelengths sequentially onto a monochromatic sensor;

[0048] and / or,

[0049] The collecting of the hyperspectral image comprises: using a grating spectrometer to obtain the hyperspectral image; the hyperspectral image is obtained by the grating spectrometer decomposing the incoming white light into a plurality of spectra of different wavelengths through a grating, and recording the spectral information of each wavelength respectively.

[0050] Further, the displaying of the standard white light image, the hyperspectral image and the pseudo-color image comprises: displaying the standard white light image, the hyperspectral image and the pseudo-color image on one display; the imaging method further comprises: receiving a switching instruction for the displayed standard white light image, the hyperspectral image and the pseudo-color image, and, in response to the switching instruction, switching to the image to be displayed;

[0051] or,

[0052] The displaying of the standard white light image, the hyperspectral image and the pseudo color image comprises: displaying the standard white light image, the hyperspectral image and the pseudo color image on three displays respectively.

[0053] The present application provides an imaging device for implementing the imaging method as described above, the imaging device comprising:

[0054] An illumination module, used to obtain standard white light to illuminate the object under inspection;

[0055] A beam splitting module, used for splitting the reflected light of the white light into a first light beam and a second light beam;

[0056] A white light imaging processing module is used to obtain a standard white light image using the first light beam; at the same time,

[0057] A hyperspectral imaging processing module is used to collect a hyperspectral image using the second light beam, and select two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generate a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images;

[0058] An image display module is used to display the standard white light image, the hyperspectral image and the pseudo-color image.

[0059] The present application provides an electronic device, comprising one or more processors, for implementing the imaging method as described in any one of the above items.

[0060] The present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the imaging method as described in any one of the above items is implemented.

[0061] The present application provides a computer program product, comprising a computer program / instruction, which implements the imaging method as described in any one of the above items when executed by a processor.

[0062] In some embodiments, the imaging method of the present application automatically generates standard white light images and pseudo color images, and displays hyperspectral images, standard white light images and pseudo color images without user participation, thereby improving imaging efficiency and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Shown is a schematic flow chart of an imaging method according to an embodiment of the present application;

[0064] Figure 2 Shown Figure 1 A schematic diagram of a specific process of the imaging method shown;

[0065] Figure 3 The figure is a schematic diagram of the structure of an imaging device provided in an embodiment of the present application;

[0066] Figure 4 Shown is a schematic structural diagram of an endoscopic imaging system provided in an embodiment of the present application;

[0067] Figure 5 The figure is a schematic diagram of the specific structure of the endoscope imaging system provided in the embodiment of the present application;

[0068] Figure 6 Shown is a schematic flow chart of an endoscopic imaging method according to an example of the present application;

[0069] Figure 7 Shown is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of the present specification as detailed in the appended claims.

[0071] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0072] In the related art, users manually switch light sources with different spectral characteristics, which requires users to switch light sources multiple times, complicating the user operation process and affecting imaging efficiency and real-time performance.

[0073] In order to solve the above-mentioned technical problems that users need to switch light sources multiple times, the user operation process is complicated, and the imaging efficiency and real-time performance are affected, the embodiment of the present application provides an imaging method, which divides the reflected light of white light into two paths of light, and automatically uses the two paths of light sources to simultaneously perform real-time acquisition of white light images and real-time acquisition of hyperspectral images, so as to generate standard white light images and pseudo-color images in real time accordingly; and display hyperspectral images, standard white light images, and pseudo-color images in real time. In this way, standard white light images and pseudo-color images are automatically generated without user participation, and hyperspectral images, standard white light images, and pseudo-color images are displayed, thereby improving imaging efficiency and real-time performance.

[0074] The imaging method of the embodiment of the present application can be applied to the field of endoscopic tissue detection, the field of endoscopic functional imaging such as blood oxygen detection, and the field of remote sensing. Detailed description is as follows.

[0075] Figure 1 Shown is a schematic flow chart of the imaging method according to an embodiment of the present application.

[0076] like Figure 1 As shown, the imaging method may include but is not limited to the following steps 110 to 140:

[0077] Step 110: Obtain standard white light to illuminate the object under inspection.

[0078] When taking photos, white light can be used to understand the situation of the illuminated object.

[0079] Step 120 , dividing the reflected light of the white light into a first light beam and a second light beam.

[0080] The "first" in the "first beam" and the "second" in the "second beam" are used to distinguish the two white lights. In this way, the two white lights can be used at the same time to automatically collect the standard white light image and the hyperspectral image at the same time, thereby avoiding multiple operations to switch the light source and reducing user operations.

[0081] Step 130, using the first light beam to obtain a standard white light image; at the same time, using the second light beam to collect a hyperspectral image, and selecting two or more narrow-band images whose central wavelength difference reaches a preset difference from multiple narrow-band images of the hyperspectral image; and, based on at least any two narrow-band images of the two or more narrow-band images, performing fusion of promotion and inhibition mechanisms and color channel combination to generate a pseudo-color image.

[0082] The pseudo-color images are used to grasp the details of the illuminated objects, such as basic structural information of the illuminated objects such as the surface of the tissue, deep tissue, etc. These photographed objects can be called inspected objects.

[0083] The multiple narrowband images of the above-mentioned hyperspectral image may include but are not limited to more than 30 bands whose central wavelength differences reach a preset difference. These bands are analyzed to obtain at least two arbitrary narrowband images of the hyperspectral image. In this way, pseudo-color images can be generated from more than two narrowband images whose central wavelength differences of more than 30 bands reach a preset difference. In this way, all bands can be applied, and the narrowband images have richer levels, realizing functional imaging and diagnosis. In addition, by exploring and testing different wavelength combinations, spectral combination optimization is achieved, and the imaging effect is further optimized to meet the needs of various lesion types.

[0084] There are several ways to obtain the above standard white light images:

[0085] In an optional manner, a white light image is collected and subjected to standard processing to obtain a standard white light image as a standard white light image. The standard processing may include but is not limited to steps such as denoising, contrast enhancement, exposure adjustment, sharpening, and color correction, so that the visual effect and readability of the image can be improved. The image after standard processing can display richer details and perform better on different display devices.

[0086] In another optional method, a standard white light image is directly acquired. When the camera's shooting effect can automatically adjust exposure, focus, white balance, etc. according to the environment, so that the final image is almost lossless and close to the effect after standard processing, the standard white light image can be directly used.

[0087] In addition, the fusion of the promotion and suppression mechanisms in this article is used to process the specific features of each of the two narrow-band images to obtain the suppressed image features and the enhanced image features, and these suppressed image features and the enhanced image features are fused. The specific features are used to represent features with different absorption of light.

[0088] It should be noted that the fusion of the above-mentioned promotion and inhibition mechanisms may include but is not limited to the fusion of the promotion and inhibition mechanisms of biomimetic vision. Specifically, the promotion and inhibition mechanisms of biomimetic vision are fused based on at least any two narrow-band images of the two or more narrow-band images to generate a fused image.

[0089] Among them, the fusion of the promotion and inhibition mechanisms of biomimetic vision includes, for example, but is not limited to: ON-center and OFF-center receptive field models in the visual system of rattlesnakes. Accordingly, based on the ON-center and OFF-center receptive field models in the visual system of rattlesnakes, image processing is performed on specific features of each of two or more narrow-band images to obtain suppressed image features and enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0090] In this regard, each two narrow-band images are fused through the promotion and inhibition mechanism of bionic vision to generate a new monochrome image (also called a fused image). In this way, the interaction between each two narrow-band images is fully considered to enhance the contrast and depth resolution. And, the generated new monochrome image is sent to one of the RGB (Red Green Blue) channels for display, and the other color channels are operated in the same way. See below for detailed description.

[0091] The fusion of the promotion and inhibition mechanisms of the above-mentioned biomimetic vision includes, but is not limited to, the positive and negative feedback mechanisms in biological perception and behavior learning, and the corresponding fusion of the promotion and inhibition mechanisms is realized. In this way, sample training and learning are required to obtain the suppressed image features and the enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0092] It should also be noted that the fusion of the above-mentioned promotion and inhibition mechanisms may also include but is not limited to the fusion of the promotion and inhibition mechanisms of the neural network model based on deep learning. Among them, the neural network model based on deep learning includes an activation function for realizing promotion and an attention mechanism for realizing inhibition. The promotion and inhibition mechanisms in biological perception and behavioral learning correspondingly realize the fusion of promotion and inhibition mechanisms. In this way, sample training and learning are required to obtain the suppressed image features and the enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0093] Step 140: display the standard white light image, the hyperspectral image, and the pseudo color image.

[0094] The above-mentioned standard white light image, hyperspectral image and pseudo color image are used to respectively display the real-time situation of the inspected object. Thus, by using the standard normal image display, hyperspectral image display and high-contrast pseudo color image display for the same inspected object, it is more conducive to analysis and observation.

[0095] The pseudo color image herein can be a high contrast pseudo color image. The difference between the brightest and darkest areas in the high contrast pseudo color image is relatively large. In this way, the high contrast pseudo color image is more vivid and has distinct light and dark. Exemplarily, the contrast ratio of the high contrast pseudo color image is usually greater than 5, and can even reach 10 or higher.

[0096] Combination Figure 1 As shown, in the above step 130, two or more narrow-band images whose central wavelength difference reaches a preset difference are selected from multiple narrow-band images of the hyperspectral image, wherein the preset difference can be used to indicate that the difference between these narrow-band images is large. In this way, the subsequent fusion of the promotion and inhibition mechanisms is more meaningful.

[0097] The preset difference in this article may be, but is not limited to, greater than or equal to 50 nm (nanometers). In order to have a sufficiently large information gap, the fusion is more effective. The preset difference may be, but is not limited to, greater than or equal to 80 nm (nanometers).

[0098] Based on this, combined Figure 1 As shown, the fusion of the promotion and suppression mechanisms and the combination of color channels based on at least any two of the two or more narrow-band images in step 130 of the above imaging method to generate a pseudo-color image can be achieved through the following multiple embodiments:

[0099] In a first optional embodiment, the two or more narrow-band images include any two narrow-band images; each time any color channel of the RGB channel is mapped (which can be referred to as each mapping), a monochrome image is generated by fusing the promotion and inhibition mechanisms based on the same two narrow-band images. Accordingly, (1) a monochrome image is generated by fusing the promotion and inhibition mechanisms of biomimetic vision based on the same two narrow-band images. (2) The monochrome image generated based on the same two narrow-band images is respectively mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel. (3) The images mapped to the first color channel, the second color channel, and the third color channel are combined by color channels to obtain a pseudo-color image.

[0100] The “first” in the above-mentioned “first color channel”, the “second” in the “second color channel” and the “third” in the “third color channel” are used to distinguish three different color channels.

[0101] In this embodiment, two narrow-band images interact with each other to generate an image that is directly mapped to one of the RGB channels. In this way, an image fusion method based on biomimetic vision mechanism is adopted to enhance the contrast and depth resolution of the image.

[0102] In a second optional embodiment, the two or more narrow-band images include any two or more narrow-band images; each time any color channel of the RGB channel is mapped (which can be called each mapping), a fusion of promotion and inhibition mechanisms is performed based on at least two different narrow-band images to generate a corresponding monochrome image. Accordingly, 1) the fusion of promotion and inhibition mechanisms of biomimetic vision based on at least two different narrow-band images is performed to generate a corresponding monochrome image. 2) the monochrome image generated based on each of the two different narrow-band images is respectively mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel. 3) the images mapped to the first color channel, the second color channel, and the third color channel are combined by color channels to obtain a pseudo-color image.

[0103] It should be noted that the narrowband images of each two bands of the at least two bands are acted on in pairs, and then the images acted on in pairs are acted on again to obtain an image, which is mapped to one of the RGB channels.

[0104] In addition, the narrowband images of each two bands of the at least two bands are acted on in pairs, and the images acted on in pairs are mapped to one of the RGB channels.

[0105] In this regard, each time a monochrome image is mapped to one of the RGB channels, all the mapped images are finally synthesized, and the image is not directly output through the real RGB channels, but the above-mentioned fused image is generated. The fused image can be called a pseudo-color image.

[0106] For example, the first narrowband image O and the second narrowband image P interact with each other to a new monochrome image, mapped to one of the RGB channels;

[0107] The third narrowband image Q and the fourth narrowband image R interact with each other to a new monochrome image mapped to one of the RGB channels;

[0108] The fifth narrowband image S and the fourth narrowband image T are interacted with each other to a new monochrome image mapped to one of the RGB channels.

[0109] In the embodiment, a pseudo-color fusion algorithm is used, combined with the advantages of the above-mentioned multi-band narrow-band imaging, so that the image contrast and depth resolution are significantly improved. In addition, a pseudo-color image is generated by mutual promotion and inhibition of the image, and it is mapped to one of the RGB channels, and the operation is repeated to finally generate a fused image. In this way, the hyperspectral image data is mapped to the RGB color space using a fusion algorithm based on biological vision mechanism to enhance the image contrast and resolution.

[0110] Continue as Figure 1 As shown, the acquisition of the hyperspectral image in the above step 130 can be implemented in at least any of the following implementations:

[0111] In the first optional implementation, a hyperspectral camera is used to obtain a hyperspectral image; the hyperspectral image includes multiple narrow-band images generated by the hyperspectral camera acquiring the reflected light in the entire spectral range at one time. In this way, the hyperspectral camera is suitable for applications that require real-time imaging, covering the visible light to near-infrared band.

[0112] In a second optional implementation, an image sensor is combined with a color filter wheel to obtain a hyperspectral image; the color filter wheel includes filters of different wavelengths, and multiple narrowband images are obtained by rotating the color filter wheel to filter light of different wavelengths sequentially onto a monochrome sensor. The image sensor is used to capture light and convert it into an electronic signal. The image sensor may include but is not limited to a CMOS (Complementary Metal-Oxide-Semiconductor) sensor, a monochrome image sensor camera, and the like, which will not be listed one by one here. In this way, the combination of an image sensor and a color filter wheel can be used in scenes with high resolution requirements and cost sensitivity to provide high-quality image resolution.

[0113] In the third optional implementation, a grating spectrometer is used to obtain a hyperspectral image; a hyperspectral image is a grating spectrometer that decomposes incoming white light into multiple spectra of different wavelengths through a grating and records the spectral information of each wavelength separately. In this way, the grating spectrometer can provide high spectral resolution, which is suitable for fine analysis of the tissue characteristics of the object under inspection.

[0114] Figure 2 Shown Figure 1 Schematic diagram of the specific process of the imaging method shown.

[0115] Continue as Figure 1 and Figure 2 As shown, the above step 140 may include but is not limited to the following two embodiments:

[0116] In an optional embodiment of the above step 140, in step 141, a standard white light image, a hyperspectral image, and a pseudo-color image are displayed on a display. Accordingly, the above imaging method may further include the following step 150, receiving a switching instruction for the displayed standard white light image, hyperspectral image, and pseudo-color image. And, in step 160, in response to the switching instruction, switching to the image to be displayed.

[0117] To this end, a standard white light image, a hyperspectral image, and a pseudo-color image can be displayed separately on one display, and by switching these images, the desired image can be displayed.

[0118] Furthermore, a standard white light image, a hyperspectral image and a pseudo-color image can be displayed on a display in multiple images at the same time. By switching these images, multiple images are switched to one image, and the image to be displayed is enlarged or displayed more specifically.

[0119] In this embodiment, an image required to be displayed among a plurality of images may be controlled to be displayed.

[0120] In another optional embodiment of the above step 140, the standard white light image, the hyperspectral image and the pseudo-color image are displayed on three displays respectively. In this way, each image is directly displayed on the three displays without the user having to switch each image, and the image to be displayed is used and displayed.

[0121] Based on the same inventive concept as the above method, the embodiment of the present application also provides an imaging device, such as Figure 3 As shown, the imaging device may include the following lighting module 51, beam splitting module 52, white light imaging processing module 53, hyperspectral imaging processing module 54 and image display module 55:

[0122] An illumination module 51 is used to obtain standard white light to illuminate the object under inspection;

[0123] A beam splitting module 52, used for splitting the reflected light of the white light into a first light beam and a second light beam;

[0124] The white light imaging processing module 53 is used to obtain a standard white light image using the first light beam; at the same time,

[0125] The hyperspectral imaging processing module 54 is used to collect a hyperspectral image using the second light beam, and select two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generate a pseudo-color image by fusing the promotion and suppression mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images;

[0126] The image display module 55 is used to display standard white light images, hyperspectral images and pseudo color images.

[0127] As an embodiment, the image display module 55 is specifically used to display a standard white light image, a hyperspectral image, and a pseudo-color image on a display. Accordingly, the imaging device further includes: a control module, which is used to receive a switching instruction for the displayed standard white light image, hyperspectral image, and pseudo-color image, and a switching module, which is used to respond to the switching instruction and switch to the image to be displayed.

[0128] The imaging device provided in the embodiment of the present application has the same inventive concept as the above method and can achieve the same or similar technical effects, which will not be listed one by one here.

[0129] At present, endoscopic technology has become an indispensable tool in clinical diagnosis and treatment, especially in the fields of digestive tract, respiratory tract, etc., where endoscopes using white light as the mainstream technology have been widely used. Specifically, endoscopes mainly rely on visible light (400-750 nanometer band) imaging, which limits their ability to detect tiny features and physiological changes in tissues, especially in the identification of early cancer, inflammation and other lesions. Since white light images can only present information on the surface structure of tissues and lack sensitivity to the chemical composition and function of deep tissues, subtle lesions are often ignored or misjudged, affecting the accuracy of clinical diagnosis. In addition, the use of hyperspectral imaging (HSI) endoscope systems usually requires switching different light sources to collect multi-band images. Users need to switch light sources multiple times, and the user operation process is complicated, which affects imaging efficiency and real-time performance, and increases the difficulty of clinical operations.

[0130] In order to solve the above-mentioned technical problems that white light images can only present the surface structural information of tissues and lack sensitivity to the chemical composition and function of deep tissues, which often leads to the neglect or misjudgment of subtle lesions, affecting the accuracy of clinical diagnosis, and the need for users to switch light sources multiple times, the user operation process is complicated, affecting the imaging efficiency and real-time performance, and increasing the difficulty of clinical operation, the embodiment of the present application provides an endoscopic imaging system for implementing the imaging method as described above. White light analysis is performed through standard white light image display, and spectral imaging analysis is performed through hyperspectral images, and two or more narrow-band images with a central wavelength difference reaching a preset difference are fused by a promotion and inhibition mechanism to generate a fused image. In this way, through the design of a beam splitting module and a single white light source, the synchronous acquisition of white light images and hyperspectral images can be achieved without switching light sources, which improves the operating efficiency as well as the imaging efficiency and real-time performance, and is suitable for real-time clinical operations.

[0131] Based on the same inventive concept as the above method, the embodiment of the present application also provides an imaging device, such as Figure 4 and Figure 5 The embodiment shown is similar to Figures 1 to 3 The embodiment shown, compared to Figures 1 to 3 In the embodiment shown, Figure 4 In the embodiment of the present application, an endoscopic imaging system is provided for implementing the endoscopic imaging method as described below. In addition, the system may include the following structures: a white light illumination module 511, a beam splitting module 52, a white light imaging processing module 53, a hyperspectral imaging processing module 54 and an image display module 55:

[0132] Among them, the white light illumination module 511 is used to obtain standard white light to illuminate the target tissue of the endoscope. The target tissue can be used to represent the tissue characteristics of a specific object under examination. In this way, the white light illumination module 511 uses standard white light to illuminate the target tissue under the endoscope, without the need for a dedicated multi-band light source, simplifying the system design. White light is not only used for normal observation, but also for subsequent image acquisition.

[0133] The beam splitting module 52 is used to split the reflected light of the white light into a first light beam and a second light beam. The beam splitting module 52 may include but is not limited to one or more of a beam splitter, a beam splitter, a fiber distributor and an optical grating. In this way, the light source is standard white light, and as the light source, the light signal can be used for both white light imaging and hyperspectral imaging through the beam splitting module 52 without the need for special light source conversion. In this way, through standard white light illumination and combined with the design of the beam splitting module 52, the white light image acquisition unit and the HSI image acquisition unit can work at the same time. In this way, dual-modal imaging can be completed without switching the light source, simplifying the operation and improving the imaging efficiency.

[0134] In this regard, after white light illumination, the reflected light is split into two paths through the beam splitting module 52, one path enters the white light image acquisition unit, and the other path enters the HSI image acquisition unit. This structure ensures that the two images are independently acquired without interfering with each other.

[0135] The white light imaging processing module 53 is used to obtain a standard white light image using the first light beam; at the same time,

[0136] The hyperspectral imaging processing module 54 is used to use the second light beam to collect a hyperspectral image, and select two or more narrow-band images whose central wavelength difference reaches a preset difference from multiple narrow-band images of the hyperspectral image; and generate a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images.

[0137] In this way, white light and hyperspectral images can be collected simultaneously without switching the light source. The endoscopic imaging system adopts a single white light source, which is designed with a beam splitting module 52 to allow white light signals to enter different image acquisition modules (such as white light image acquisition units and HSI image acquisition units), thereby achieving efficient dual-modality simultaneous imaging.

[0138] Among them, the above-mentioned HSI technology, as a technology that can capture tissue characteristics under different spectra, shows great potential in improving the sensitivity of lesion detection. HSI technology can collect information from multiple bands from visible light to near-infrared light, and generate detailed spectral data for each pixel, enabling it to identify and distinguish different tissue components, such as the absorption differences between oxygenated hemoglobin and deoxygenated hemoglobin at different wavelengths. This feature gives HSI a significant advantage in detecting tiny lesions, distinguishing benign and malignant tissues, and monitoring surgical margins.

[0139] In addition, the two or more narrow-band images with a preset central wavelength difference may be, but are not limited to, two narrow-band images with a central wavelength difference greater than 50 nm, or two narrow-band images with a central wavelength difference greater than 80 nm. In this way, a fusion algorithm based on biological vision is introduced to fuse two narrow-band images with a large central wavelength difference through a promotion and inhibition mechanism and map them to the RGB color space, thereby significantly enhancing the contrast and depth resolution of tissues.

[0140] Compared with the imaging systems in related technologies, the image fusion method is relatively simple and cannot effectively improve the hierarchical information and contrast of the tissue, which limits the application effect of the system in the observation of complex tissue structures.

[0141] Compared with the related art, in the embodiment of the present application, by selecting two or more narrow-band images with a central wavelength difference that reaches a preset difference and adopting a promotion and inhibition image fusion method, the visualization effect of the tissue is significantly improved, especially in lesion identification, early lesion detection and disease treatment.

[0142] The image display module 55 is used to display standard white light images, hyperspectral images and pseudo-color images, so as to realize real-time inspection of the endoscope. The image display module 55 may include but is not limited to a display or a display screen.

[0143] The endoscope system of the related art can usually only obtain different types of images by switching the light source, while the endoscope imaging system of the present invention can simultaneously obtain white light and hyperspectral images through the beam splitting module 52, which greatly improves the convenience of operation and imaging efficiency. In addition, the introduction of the HSI image fusion method enables the system to not only provide white light images, but also provide richer tissue information through hyperspectral imaging. This combination of multimodal images provides more dimensional information for clinical diagnosis, which helps to improve the accuracy of diagnosis.

[0144] Compared with the related art, the design of the endoscopic imaging system in the embodiment of the present application combines the advantages of WLI and HSI imaging methods to provide a flexible image acquisition and processing solution, especially showing great potential in lesion diagnosis and treatment and early lesion identification. This system has significant clinical application prospects by simplifying the operation process and improving the imaging quality.

[0145] Continue as Figure 5 As shown, the white light imaging processing module 53 includes a white light image acquisition unit and a white light image processing unit; the hyperspectral imaging processing module 54 includes an HSI image acquisition unit and an HSI image processing unit. It should be noted that the HSI image acquisition unit obtains hyperspectral image data through a hyperspectral sensor, covering image information of multiple narrow-band light bands, and providing reflectance characteristics of tissues under different spectra.

[0146] The white light image acquisition unit is used to acquire a standard white light image using the first light beam; meanwhile, the HSI image acquisition unit is used to acquire a hyperspectral image using the second light beam;

[0147] The white light image processing unit is used to process the standard white light image. Meanwhile, the HSI image processing unit is used to select two or more narrow band images whose central wavelength difference reaches a preset difference from the images of multiple narrow band light bands in the hyperspectral image. Furthermore, the biomimetic visual mechanism is used to interact with each other of the two or more narrow band images to generate a monochrome image.

[0148] It should be noted that the white light image processing unit and the HSI image processing unit in this article can be collectively referred to as a dual-path image processing unit. The dual-path image processing unit performs independent processing based on the WLI (white light) image and the HSI image acquired simultaneously.

[0149] The above-mentioned white light image acquisition unit and the above-mentioned HSI image acquisition unit are relatively independent acquisition channels, and image acquisition is performed simultaneously. The above-mentioned white light image acquisition unit can realize standard white light imaging, and the acquisition frame rate is not less than 30fps, ensuring the smooth imaging effect of real-time inspection of the endoscope. The white light image acquisition unit can be but not limited to CMOS (Complementary Metal-Oxide-Semiconductor Image Sensor). The white light image acquired by the CMOS is transmitted to the image processing module. The image processing module may include the above-mentioned dual-channel image processing unit. The image processing module may be a chip for realizing the image processing function.

[0150] The above-mentioned HSI image acquisition unit, under hyperspectral imaging, acquires narrowband images of different wavelengths at a frame rate of not less than 15fps, and the band interval is not less than 5nm, thereby ensuring image resolution while improving the acquisition of deep tissue information.

[0151] In addition, the acquisition band interval of the HSI image acquisition unit is not less than 5nm, and the narrowband spectrum bandwidth is controlled within 10nm-30nm.

[0152] In addition, when the HSI image acquisition unit selects narrowband images, the band interval is ensured to be above 5nm to ensure the difference and accuracy of spectral images in different bands.

[0153] The above-mentioned standard white light image, hyperspectral image and fusion image are all used to realize real-time inspection of the endoscope.

[0154] In another embodiment, the hyperspectral imaging processing module 54 may also include but is not limited to: an HSI image fusion module selects two narrowband images with a wavelength difference greater than 80 nm from the HSI data for fusion, such as mapping the HSI image to the RGB color space based on a fusion algorithm and a fusion based on the promotion and inhibition mechanism of biological vision.

[0155] Continue as Figure 5 As shown, in an optional embodiment, the above-mentioned more than two narrow-band images include any two narrow-band images; each time mapping is performed, the promotion and inhibition mechanisms are fused based on the same two narrow-band images to generate a monochrome image.

[0156] Accordingly, the hyperspectral imaging processing module 54 may include but is not limited to: a mapping unit, and a promotion and inhibition fusion unit; the promotion and inhibition fusion unit is used to fuse the promotion and inhibition mechanisms of biomimetic vision based on the same two narrow-band images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on the same two narrow-band images to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel; the images mapped to the first color channel, the second color channel, and the third color channel are combined by color channels to obtain a pseudo-color image.

[0157] In the embodiment of the present application, the fusion of pseudo-color images combines the advantages of multi-band narrow-band imaging, which significantly improves the image contrast and deep-level resolution, making it easier for doctors to identify early lesions during observation.

[0158] In another optional embodiment, the above-mentioned two or more narrow-band images include any two or more narrow-band images; each time mapping is performed, the promotion and inhibition mechanisms are fused based on different at least every two narrow-band images to generate a monochrome image accordingly.

[0159] Accordingly, the hyperspectral imaging processing module 54 may include but is not limited to: a mapping unit, and a promotion and inhibition fusion unit; the promotion and inhibition fusion unit is used to fuse the promotion and inhibition mechanisms of biomimetic vision based on each of the two narrow-band images to generate the monochrome image; the mapping unit is used to map the monochrome image generated accordingly based on each of the two narrow-band images to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel; the images mapped to the first color channel, the second color channel, and the third color channel are combined in color channels to obtain a pseudo-color image.

[0160] Continue as Figure 5 As shown, the image display module 55 may include but is not limited to one or two displays; the endoscope imaging system also includes: a control module for receiving switching instructions for standard white light images, hyperspectral images, and pseudo-color images, and switching to the image to be displayed. In this way, doctors can choose to display different types of images as needed to better observe and diagnose lesions.

[0161] As an embodiment, the control module is further used to receive any screenshot instruction of a standard white light image, a hyperspectral image and a pseudo-color image, take a screenshot of the current image; save the screenshot and generate a report; the report is used to display the diagnosis result.

[0162] The control module can be controlled by the chip. The control module can include a foot controller and a controller. Exemplarily, the screenshot instruction is the screenshot instruction input by the doctor stepping on the foot controller.

[0163] The image display module 55 includes three displays, which are used to display a standard white light image, a hyperspectral image and a pseudo-color image respectively.

[0164] Continue to combine Figure 5 As shown, in the embodiment of the present application, the above-mentioned hyperspectral imaging processing module 54 can be implemented by the HSI image acquisition unit of at least any one of the following three embodiments: 1) hyperspectral camera, suitable for clinical environments requiring rapid response; 2) monochrome sensor plus filter color wheel, suitable for high-resolution imaging and low-cost applications; 3) grating spectrometer, suitable for tissue analysis scenarios requiring high spectral accuracy. The three methods provide flexibility and economical options for different diagnosis and treatment needs.

[0165] 1) Hyperspectral Camera

[0166] In a first optional embodiment, the hyperspectral imaging processing module 54 includes an HSI image acquisition unit: the HSI image acquisition unit is used to obtain a hyperspectral image using a hyperspectral camera; the hyperspectral image includes multiple narrow-band images generated by the hyperspectral camera acquiring reflected light in the entire spectral range at one time.

[0167] The first optional embodiment described above uses a hyperspectral camera to acquire images quickly and can obtain hyperspectral data in real time, which is suitable for clinical applications that require rapid imaging and real-time analysis. Hyperspectral cameras can cover the spectrum from visible light to near-infrared range, thereby providing a wider range of tissue information. In this way, hyperspectral cameras are suitable for real-time imaging and rapid response clinical environments.

[0168] 2) Monochrome sensor plus filter color wheel

[0169] In a second optional embodiment, the HSI image acquisition unit is used to obtain a hyperspectral image by combining an image sensor with a filter color wheel; the filter color wheel includes filters of different bands, and multiple narrow-band images are obtained by rotating the filter color wheel to filter light of different wavelengths in turn onto a monochrome sensor.

[0170] The above second optional embodiment, the combination of the image sensor and the filter color wheel, makes the cost relatively low, and is suitable for obtaining high-quality image data through the image sensor in the case of requiring higher resolution. Although the imaging speed of this method is relatively slow, its flexibility and cost-effectiveness make it one of the better choices. In this way, the combination of the image sensor and the filter color wheel is suitable for scenes that require high-quality image resolution and are relatively cost-sensitive.

[0171] 3) Grating spectrometer

[0172] In a third optional embodiment, the HSI image acquisition unit is used to obtain a hyperspectral image using a grating spectrometer; the hyperspectral image is a grating spectrometer that decomposes incoming white light into multiple spectra of different wavelengths through a grating and records the spectral information of each wavelength separately.

[0173] The third optional embodiment described above can provide extremely high spectral resolution and is suitable for scenarios where high precision is required for the spectral characteristics of tissues. The grating spectrometer has excellent wavelength accuracy and sensitivity, can capture the weak spectral changes of tissues, and is very suitable for fine imaging and analysis of lesions. In this way, the grating spectrometer is designed for high-precision spectral resolution applications and is suitable for application scenarios that require fine analysis of the spectral characteristics of tissues.

[0174] In the embodiment of the present application, three acquisition implementation schemes are provided to optimize the imaging effect under different clinical needs, thereby realizing diversified HSI image acquisition. Moreover, through these diversified image acquisition methods, the system can provide highly adaptable solutions to meet a wide range of medical imaging needs from ordinary endoscopy to hyperspectral diagnosis.

[0175] The three hyperspectral acquisition methods in this article provide solutions for different clinical application needs, ensuring that the system can adapt to a wide range of medical scenarios. This makes the system flexible and adaptable.

[0176] The implementation process of the functions and effects of each module / unit in this article is specifically described in the implementation process of the corresponding steps in the above method, and the same technical effect can be achieved, so it will not be repeated here.

[0177] Figure 6 Shown is a schematic flow chart of an endoscopic imaging method according to an example of the present application.

[0178] like Figure 6 As shown, the endoscopic imaging method may include but is not limited to the following steps 210 to 240:

[0179] Step 210: The white light illumination module obtains standard white light to illuminate the target tissue of the endoscope.

[0180] Step 220: The beam splitting module splits the reflected light of the white light into a first light beam and a second light beam.

[0181] In step 230, the white light imaging processing module uses the first light beam to obtain a standard white light image; at the same time, the hyperspectral imaging processing module uses the second light beam to collect a hyperspectral image, and selects two or more narrow-band images whose central wavelength difference reaches a preset difference from multiple narrow-band images of the hyperspectral image; and, based on at least any two of the two or more narrow-band images, the fusion of the promotion and inhibition mechanisms and the combination of color channels are performed to generate a pseudo-color image.

[0182] Step 230 can simultaneously acquire a white light image (WLI) and a hyperspectral image (HSI) of the same location without switching the light source between the two. In this way, white light and HSI images are acquired simultaneously, improving operation efficiency and avoiding the possibility of missing important tissue information when switching modes.

[0183] Furthermore, the WLI image and the HSI image go through independent processing flows, and doctors can choose to display one or a combination of the two in the image display module as needed.

[0184] Step 240: The image display module displays the standard white light image, the hyperspectral image and the pseudo-color image to realize the real-time inspection of the endoscope.

[0185] In the embodiment of the present application, WLI and HSI dual-channel independent imaging is achieved through white light illumination without switching the light source. At the same time, the HSI image fusion module adopts a multi-band narrowband image fusion method to significantly improve the image depth resolution and tissue visualization effect. The system not only improves diagnostic efficiency, but also helps to assist doctors in subsequent precise guidance.

[0186] The endoscopic imaging method and system in this article can achieve the following effects:

[0187] 1. Clinical effect verification: Conduct experiments in different clinical scenarios, accumulate data and verify the effectiveness of the fusion algorithm of promotion and inhibition mechanisms in early lesion detection, and provide quantitative data support.

[0188] 2. Modular design and cost control: Through modular design and low-cost solutions, the system can be improved in terms of applicability and promotion in different medical scenarios, thereby enhancing its market potential.

[0189] An embodiment of the present application provides an electronic device, including one or more processors, for implementing the above imaging method.

[0190] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, a handheld terminal, etc. Any electronic device that can implement the embodiment of the present invention belongs to the protection scope of the present invention and is not limited here.

[0191] The endoscopic imaging method and system, imaging method and device in this article all have the same inventive concept, and the same or similar steps can achieve the same effect.

[0192] Figure 7 Shown is a schematic structural diagram of an electronic device 70 provided in an embodiment of the present application.

[0193] like Figure 7 As shown, the electronic device 70 includes one or more processors 71 for implementing the imaging method as described above.

[0194] In some embodiments, the electronic device 70 may include a storage medium 79. For example, the computer-readable storage medium may store a program that can be called by the processor 71, and may include a non-volatile storage medium. In some embodiments, the electronic device 70 may include a memory 78 and an interface 77. In some embodiments, the electronic device 70 may also include other hardware according to actual applications.

[0195] The computer-readable storage medium of the embodiment of the present application stores a program thereon, and when the program is executed by the processor 71, it is used to implement the imaging method described above.

[0196] The present application provides a computer program product, comprising a computer program / instruction, which implements any of the above methods when executed by a processor.

[0197] The present application also provides a computer program, which is stored in a computer-readable storage medium, such as Figure 7 storage medium 79, and when the processor executes the computer program, it causes the processor 71 to execute the method described above.

[0198] The present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer-readable storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0199] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0200] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the phrase "includes a ..." defines an element, and does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

Claims

1. An endoscopic imaging system, characterized in that: include: A white light illumination module, used to obtain standard white light to illuminate the target tissue of the endoscope; A beam splitting module, used for splitting the reflected light of the white light into a first light beam and a second light beam; A white light imaging processing module is used to obtain a standard white light image using the first light beam; at the same time, A hyperspectral imaging processing module is used to collect a hyperspectral image using the second light beam, and select two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generate a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images; The image display module is used to display the standard white light image, the hyperspectral image and the pseudo-color image, so as to realize real-time inspection of the endoscope.

2. The endoscopic imaging system according to claim 1, characterized in that: The two or more narrow-band images include any two narrow-band images; each time the mapping is performed, the promotion and inhibition mechanisms are fused based on the two narrow-band images to generate a monochrome image; The hyperspectral imaging processing module includes: a mapping unit, and a promotion and inhibition fusion unit; the promotion and inhibition fusion unit is used to fuse the promotion and inhibition mechanism of biomimetic vision based on the same two narrow-band images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on the same two narrow-band images to a first color channel of the RGB channel, a second color channel mapped to the RGB channel, and a third color channel mapped to the RGB channel; the images mapped to the first color channel, the second color channel, and the third color channel are combined by color channels to obtain a pseudo-color image; or, The two or more narrow-band images include any two or more narrow-band images; each time mapping is performed, a fusion of promotion and inhibition mechanisms is performed based on at least every two narrow-band images to generate a monochrome image accordingly; the hyperspectral imaging processing module includes: a mapping unit, and a fusion unit of promotion and inhibition; the fusion unit of promotion and inhibition is used to fuse the promotion and inhibition mechanisms of biomimetic vision based on the different two narrow-band images to generate the monochrome image accordingly; the mapping unit is used to map the monochrome image generated accordingly based on the different two narrow-band images to the first color channel of the RGB channel, the second color channel mapped to the RGB channel, and the third color channel mapped to the RGB channel; the images mapped by the first color channel, the second color channel and the third color channel are combined by color channels to obtain a pseudo-color image.

3. The endoscopic imaging system according to claim 1 or 2, characterized in that: The image display module includes one or two displays; the endoscope imaging system also includes: a control module, which is used to receive switching instructions for the standard white light image, the hyperspectral image and the pseudo-color image, and switch to the image to be displayed; or, The image display module includes three displays; the three displays are used to display the standard white light image, the hyperspectral image and the pseudo-color image respectively.

4. The endoscopic imaging system according to claim 1 or 2, characterized in that: The hyperspectral imaging processing module includes an HSI image acquisition unit: The HSI image acquisition unit is used to obtain a hyperspectral image using a hyperspectral camera; the hyperspectral image includes a plurality of narrow-band light images generated by the hyperspectral camera acquiring reflected light within the entire spectral range at one time; and / or, The HSI image acquisition unit is used to obtain a hyperspectral image by combining an image sensor with a filter color wheel; the filter color wheel includes filters of different wavelengths, and the multiple narrow-band light images are obtained by rotating the filter color wheel to filter light of different wavelengths sequentially onto a monochromatic sensor; and / or, The HSI image acquisition unit is used to obtain a hyperspectral image using a grating spectrometer; the hyperspectral image is obtained by the grating spectrometer decomposing the incoming white light into a plurality of spectra of different wavelengths through a grating and recording the spectral information of each wavelength respectively.

5. An endoscopic imaging method, characterized in that: include: The white light illumination module obtains standard white light to illuminate the target tissue of the endoscope; The beam splitting module splits the reflected light of the white light into a first light beam and a second light beam; The white light imaging processing module uses the first light beam to obtain a standard white light image; at the same time, The hyperspectral imaging processing module uses the second light beam to collect a hyperspectral image, and selects two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generates a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images; The image display module displays the standard white light image, the hyperspectral image and the pseudo-color image, so as to realize real-time inspection of the endoscope.

6. An imaging method, characterized in that: include: Obtain standard white light to illuminate the object to be inspected; Splitting the reflected light of the white light into a first light beam and a second light beam; Using the first light beam, a standard white light image is obtained; at the same time, Using the second light beam, a hyperspectral image is collected, and two or more narrow-band images whose central wavelength difference reaches a preset difference are selected from a plurality of narrow-band images of the hyperspectral image; and based on at least any two narrow-band images of the two or more narrow-band images, a fusion of promotion and inhibition mechanisms and a combination of color channels are performed to generate a pseudo-color image; The standard white light image, the hyperspectral image, and the pseudo color image are displayed.

7. The imaging method according to claim 6, characterized in that: The two or more narrow-band images include any two narrow-band images; each time any color channel of the RGB channels is mapped, a monochrome image will be generated based on the same two narrow-band images; The step of fusing the promotion and suppression mechanisms and combining the color channels based on at least any two of the two or more narrow-band images to generate a pseudo-color image includes: The promotion and inhibition mechanisms of biomimetic vision are integrated based on the two narrow-band images to generate a monochromatic image; Mapping the monochrome image generated based on the two narrow-band images to a first color channel of an RGB channel, to a second color channel of the RGB channel, and to a third color channel of the RGB channel; Color channel combination is performed on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image.

8. The imaging method according to claim 6, characterized in that: The two or more narrow-band images include any two or more narrow-band images; each time any color channel of the RGB channel is mapped, a monochrome image is generated based on each of the two narrow-band images; The step of fusing the promotion and suppression mechanisms and combining the color channels based on at least any two of the two or more narrow-band images to generate a pseudo-color image includes: The promotion and inhibition mechanisms of biomimetic vision are integrated based on at least the two narrow-band images to generate a monochromatic image accordingly; Mapping the monochrome images correspondingly generated based on each of the two narrow-band images to a first color channel of an RGB channel, to a second color channel of the RGB channel, and to a third color channel of the RGB channel; Color channel combination is performed on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image.

9. The imaging method according to any one of claims 6 to 8, characterized in that: The collecting of the hyperspectral image comprises: using a hyperspectral camera to obtain a hyperspectral image; the hyperspectral image comprises a plurality of narrow-band light images generated by the hyperspectral camera acquiring reflected light in the entire spectral range at one time; and / or, The method for collecting hyperspectral images comprises: using an image sensor and a color filter wheel to obtain a hyperspectral image; the color filter wheel comprises filters of different wavelengths, and the multiple narrow-band light images are obtained by rotating the color filter wheel to filter light of different wavelengths sequentially onto a monochromatic sensor; and / or, The collecting of the hyperspectral image comprises: using a grating spectrometer to obtain the hyperspectral image; the hyperspectral image is obtained by the grating spectrometer decomposing the incoming white light into a plurality of spectra of different wavelengths through a grating, and recording the spectral information of each wavelength respectively.

10. The imaging method according to any one of claims 6 to 8, characterized in that: The displaying of the standard white light image, the hyperspectral image and the pseudo color image comprises: displaying the standard white light image, the hyperspectral image and the pseudo color image on one display; the imaging method further comprises: receiving a switching instruction for the displayed standard white light image, the hyperspectral image and the pseudo color image, and, in response to the switching instruction, switching to the image to be displayed; or, The displaying of the standard white light image, the hyperspectral image and the pseudo color image comprises: displaying the standard white light image, the hyperspectral image and the pseudo color image on three displays respectively.

11. An imaging device, characterized in that: For implementing the imaging method according to any one of claims 6 to 10, the imaging device comprises: An illumination module, used to obtain standard white light to illuminate the object under inspection; A beam splitting module, used for splitting the reflected light of the white light into a first light beam and a second light beam; A white light imaging processing module is used to obtain a standard white light image using the first light beam; at the same time, A hyperspectral imaging processing module is used to collect a hyperspectral image using the second light beam, and select two or more narrow-band images whose central wavelength difference reaches a preset difference from a plurality of narrow-band images of the hyperspectral image; and generate a pseudo-color image by fusing the promotion and inhibition mechanisms and combining the color channels based on at least any two narrow-band images of the two or more narrow-band images; An image display module is used to display the standard white light image, the hyperspectral image and the pseudo-color image.

12. An electronic device, characterized in that: The method comprises one or more processors for implementing the imaging method according to any one of claims 6 to 10.

Images