Endoscopic imaging method and system, imaging method and apparatus, and electronic device

Abstract

The application provides an endoscope imaging method and system, an imaging method and device, and an electronic device. The system comprises: a white light illumination module for obtaining standard white light to illuminate a target tissue of an endoscope; a beam splitting module for splitting reflected light of the white light into a first light beam and a second light beam; a white light imaging processing module for obtaining a standard white light image using the first light beam; simultaneously, a hyperspectral imaging processing module for collecting a hyperspectral image using the second light beam, and selecting two or more narrowband images with a center wavelength difference reaching a preset difference from a plurality of narrowband images of the hyperspectral image; and, based on at least any two narrowband images of the two or more narrowband images, promoting and inhibiting mechanism fusion and color channel combination are performed to generate a false color image; and an image display module for displaying the standard white light image, the hyperspectral image, and the false color image, so as to realize real-time examination of the endoscope.

Description

Technical Field

[0001] This invention relates to the field of imaging technology, and more particularly to an endoscopic imaging method and system, an imaging method and device, and an electronic device. Background Technology

[0002] Image imaging in related technologies requires the use of multiple light sources with different spectral characteristics to generate different images corresponding to each light source. For example, light sources of different wavelengths include a first wavelength light source and a second wavelength light source. First, the user manually switches to the first wavelength light source and uses it to generate the image corresponding to that wavelength. Then, the user manually switches to the second wavelength light source and uses it to generate the image corresponding to that wavelength.

[0003] This requires users to switch light sources multiple times, making the user operation process complex and affecting imaging efficiency and real-time performance. Summary of the Invention

[0004] This application provides an improved endoscopic imaging method and system, imaging method and apparatus, and electronic device.

[0005] This application provides an endoscopic imaging system, comprising:

[0006] The white light illumination module is used to acquire standard white light to illuminate the target tissue of the endoscope;

[0007] The beam splitting module is used to split the reflected light of the white light into a first beam and a second beam.

[0008] The white light imaging processing module is used to acquire a standard white light image using the first beam; simultaneously...

[0009] The hyperspectral imaging processing module is used to acquire a hyperspectral image using the second beam, and select two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and to perform fusion of promotion and suppression mechanisms and color channel combination based on at least any two of the two or more narrowband images to generate a pseudo-color image.

[0010] The image display module is used to display the standard white light image, the hyperspectral image, and the pseudo-color image, for enabling real-time endoscopic examination.

[0011] Furthermore, the two or more narrowband images include any two narrowband images; during each mapping, a fusion of facilitation and inhibition mechanisms is performed based on the same two narrowband images to generate a monochrome image; the hyperspectral imaging processing module includes: a mapping unit, and a fusion unit for facilitating and inhibiting; the fusion unit for facilitating and inhibiting is used to fuse the facilitation and inhibition mechanisms of biomimetic vision based on the same two narrowband images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on the same two narrowband images to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel, respectively; the images mapped by the first color channel, the second color channel, and the third color channel are combined to obtain a pseudo-color image;

[0012] or,

[0013] The two or more narrowband images include any two or more narrowband images; during each mapping, a fusion of promotion and inhibition mechanisms is performed based on at least two different narrowband images to generate a monochrome image accordingly; the hyperspectral imaging processing module includes: a mapping unit and a fusion unit for promotion and inhibition; the fusion unit for promotion and inhibition is used to fuse the biomimetic vision promotion and inhibition mechanisms based on each pair of different narrowband images to generate the monochrome image accordingly; the mapping unit is used to map the monochrome images generated based on each pair of different narrowband images to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel, respectively; the images mapped to the first color channel, the second color channel, and the third color channel are combined to obtain a pseudo-color image.

[0014] Furthermore, the image display module includes one or two displays; the endoscopic imaging system further includes: a control module, used to receive switching instructions for the standard white light image, the hyperspectral image, and the pseudo-color image, and switch to the desired image for display;

[0015] or,

[0016] The image display module includes three displays; the three displays are used to display the standard white light image, the hyperspectral image, and the pseudo-color image, respectively.

[0017] Furthermore, the hyperspectral imaging processing module includes an HSI image acquisition unit:

[0018] The HSI image acquisition unit is used to obtain hyperspectral images using a hyperspectral camera; the hyperspectral images include multiple narrowband light images generated by the hyperspectral camera acquiring reflected light across the entire spectral range at once.

[0019] And / or,

[0020] The HSI image acquisition unit is used to obtain hyperspectral images by combining an image sensor with a color filter wheel; the color filter wheel contains filters of different wavelengths, and the multiple narrowband light images are obtained by rotating the color filter wheel to sequentially filter light of different wavelengths onto a monochromatic sensor;

[0021] And / or,

[0022] The HSI image acquisition unit is used to acquire hyperspectral images using a grating spectrometer; the hyperspectral image is the spectral information of each wavelength recorded by the grating spectrometer, which decomposes incoming white light into multiple spectra of different wavelengths through a grating.

[0023] This application provides an endoscopic imaging method, including:

[0024] The white light illumination module acquires standard white light to illuminate the target tissue of the endoscope;

[0025] The beam splitting module divides the reflected light of the white light into a first beam and a second beam.

[0026] The white light imaging processing module uses the first beam to acquire a standard white light image; simultaneously,

[0027] The hyperspectral imaging processing module uses the second beam to acquire a hyperspectral image, and selects two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and, based on at least any two of the two or more narrowband images, performs a fusion of promotion and suppression mechanisms and color channel combination to generate a pseudo-color image.

[0028] The image display module displays the standard white light image, the hyperspectral image, and the pseudo-color image to enable real-time endoscopic examination.

[0029] This application provides an imaging method, including:

[0030] Obtain standard white light to illuminate the object being inspected;

[0031] The reflected light of the white light is divided into a first beam and a second beam;

[0032] Using the first beam, a standard white light image is acquired; simultaneously...

[0033] Using the second beam, a hyperspectral image is acquired, and from multiple narrowband images of the hyperspectral image, two or more narrowband images with a center wavelength difference reaching a preset difference are selected; and, based on at least any two of the two or more narrowband images, a fusion of promotion and suppression mechanisms and color channel combination are performed to generate a pseudo-color image.

[0034] Display the standard white light image, the hyperspectral image, and the pseudo-color image.

[0035] Furthermore, the two or more narrowband images include any two narrowband images; each time any color channel of the RGB channel is mapped, the monochrome image generated based on the same two narrowband images will be generated.

[0036] The process of fusing at least two narrowband images based on the above two or more narrowband images using promotion and suppression mechanisms and combining color channels to generate a pseudo-color image includes:

[0037] A monochrome image is generated by fusing the facilitation and inhibition mechanisms of biomimetic vision based on the two narrowband images mentioned above.

[0038] The monochrome image generated based on the same two narrowband images is respectively mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel;

[0039] A pseudo-color image is obtained by combining the color channels of the image mapped by the first color channel, the second color channel, and the third color channel.

[0040] Furthermore, the two or more narrowband images include any two or more narrowband images; each time any color channel of the RGB channel is mapped, a monochrome image is generated based on each of the two narrowband images respectively;

[0041] The process of fusing at least two narrowband images based on the above two or more narrowband images using promotion and suppression mechanisms and combining color channels to generate a pseudo-color image includes:

[0042] The fusion of biomimetic vision enhancement and inhibition mechanisms based on at least two different narrowband images generates a monochrome image accordingly.

[0043] The monochrome images generated based on each pair of narrowband images are respectively mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel;

[0044] A pseudo-color image is obtained by combining the color channels of the image mapped by the first color channel, the second color channel, and the third color channel.

[0045] Furthermore, the acquisition of hyperspectral images includes: using a hyperspectral camera to obtain hyperspectral images; the hyperspectral images include multiple narrowband light images generated by the hyperspectral camera acquiring reflected light across the entire spectral range in a single operation;

[0046] And / or,

[0047] The acquisition of hyperspectral images includes: using an image sensor and a color filter wheel to obtain hyperspectral images; the color filter wheel contains filters of different wavelengths, and the multiple narrowband light images are obtained by rotating the color filter wheel to sequentially filter light of different wavelengths onto a monochromatic sensor;

[0048] And / or,

[0049] The acquisition of hyperspectral images includes: using a grating spectrophotometer to acquire hyperspectral images; the hyperspectral images are the spectral information of each wavelength recorded by the grating spectrophotometer, which decomposes incoming white light into multiple spectra of different wavelengths through a grating.

[0050] Furthermore, the display of the standard white light image, the hyperspectral image, and the pseudo-color image includes: displaying the standard white light image, the hyperspectral image, and the pseudo-color image on a display; the imaging method further includes: receiving a switching instruction for the displayed standard white light image, the hyperspectral image, and the pseudo-color image, and, in response to the switching instruction, switching to the desired image;

[0051] or,

[0052] The display of the standard white light image, the hyperspectral image, and the pseudo-color image includes: displaying the standard white light image, the hyperspectral image, and the pseudo-color image on three separate displays.

[0053] This application provides an imaging apparatus for implementing the imaging method described above, the imaging apparatus comprising:

[0054] The lighting module is used to acquire standard white light to illuminate the object being inspected;

[0055] The beam splitting module is used to split the reflected light of the white light into a first beam and a second beam.

[0056] The white light imaging processing module is used to acquire a standard white light image using the first beam; simultaneously...

[0057] The hyperspectral imaging processing module is used to acquire a hyperspectral image using the second beam, and select two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and to perform fusion of promotion and suppression mechanisms and color channel combination based on at least any two of the two or more narrowband images to generate a pseudo-color image.

[0058] An image display module is used to display the standard white light image, the hyperspectral image, and the pseudo-color image.

[0059] This application provides an electronic device including one or more processors for implementing the imaging method as described in any of the preceding claims.

[0060] This application provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the imaging method as described in any of the preceding claims.

[0061] This application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the imaging method as described in any of the preceding claims.

[0062] In some embodiments, the imaging method of this application does not require user intervention, automatically generates standard white light images and pseudo-color images, and displays hyperspectral images, standard white light images and pseudo-color images, thereby improving imaging efficiency and real-time performance. Attached Figure Description

[0063] Figure 1 The diagram shown is a flowchart illustrating the imaging method according to an embodiment of this application.

[0064] Figure 2 As shown Figure 1 The diagram shows a detailed flowchart of the imaging method.

[0065] Figure 3 The diagram shown is a schematic representation of the imaging device provided in an embodiment of this application.

[0066] Figure 4 The diagram shown is a structural schematic of the endoscopic imaging system provided in an embodiment of this application;

[0067] Figure 5 The diagram shown is a schematic representation of the endoscopic imaging system provided in an embodiment of this application.

[0068] Figure 6 The diagram shown is a flowchart of the endoscopic imaging method of this application.

[0069] Figure 7 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0071] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0072] In related technologies, users manually switch between light sources with different spectral characteristics. This requires users to switch light sources multiple times, making the user operation process complex and affecting imaging efficiency and real-time performance.

[0073] To address the technical issues mentioned above, such as the need for users to switch light sources multiple times, resulting in complex user operations and impacting imaging efficiency and real-time performance, this application provides an imaging method that splits the reflected white light into two paths. It automatically uses both light sources simultaneously to acquire both white light and hyperspectral images in real time, thereby generating standard white light and pseudo-color images accordingly. The hyperspectral, standard white light, and pseudo-color images are then displayed in real time. This eliminates the need for user intervention, automatically generating and displaying standard white light and pseudo-color images, thus improving imaging efficiency and real-time performance.

[0074] The imaging method of this application can be applied to the field of endoscopic tissue detection, endoscopic functional imaging such as blood oxygen detection, and remote sensing. Detailed explanation follows.

[0075] Figure 1 The diagram shown is a flowchart of the imaging method according to an embodiment of this application.

[0076] like Figure 1 As shown, the imaging method may include, but is not limited to, the following steps 110 to 140:

[0077] Step 110: Obtain standard white light to illuminate the object under inspection.

[0078] When taking photos, white light illumination can help you understand the condition of the illuminated object.

[0079] Step 120: Divide the reflected white light into a first beam and a second beam.

[0080] The "first" in "first beam" and the "second" in "second beam" are used to distinguish between the two white lights. This allows for the simultaneous and automatic acquisition of both standard white light and hyperspectral images, avoiding multiple light source switching operations and reducing user intervention.

[0081] Step 130: Using the first beam, a standard white light image is acquired; simultaneously, using the second beam, a hyperspectral image is acquired, and from multiple narrowband images of the hyperspectral image, two or more narrowband images with a center wavelength difference reaching a preset difference are selected; and, based on at least any two of the two or more narrowband images, a fusion of promotion and suppression mechanisms and color channel combination are performed to generate a pseudo-color image.

[0082] The aforementioned pseudo-color images are used to obtain detailed information about the illuminated object. This detailed information includes, for example, basic structural information of the surface of the illuminated object, deep tissues, etc. These photographed objects can be referred to as the objects under inspection.

[0083] The aforementioned hyperspectral images can include, but are not limited to, more than 30 bands with center wavelength differences reaching a preset difference. Analyzing these bands yields at least two arbitrary narrowband images from the hyperspectral image. Thus, pseudo-color images can be generated from two or more narrowband images with center wavelength differences reaching a preset difference from the more than 30 bands. This allows for application across the entire spectrum, providing richer detail in the narrowband images and enabling functional imaging and diagnosis. Furthermore, by exploring and testing different wavelength combinations, spectral combination optimization can be achieved, further refining the imaging effect to meet the needs of various lesion types.

[0084] There are several ways to obtain the above-mentioned standard white light image:

[0085] In one alternative approach, a white light image is acquired and then subjected to standard processing to obtain a standard white light image, which serves as the standard white light image. This standard processing may include, but is not limited to, steps such as noise reduction, contrast enhancement, exposure adjustment, sharpening, and color correction. This improves the visual appeal and readability of the image. The image after standard processing displays richer details and performs better on different display devices.

[0086] In another alternative approach, a standard white light image can be directly captured. When the camera's shooting effect can automatically adjust exposure, focus, white balance, etc., according to the environment, so that the final image is almost lossless and close to the effect of standard processing, the standard white light image can be used directly.

[0087] Furthermore, the fusion of promotion and inhibition mechanisms in this paper involves image processing of specific features from every two narrowband images to obtain suppressed and enhanced image features, which are then fused together. Specific features are used to represent differences in light absorption.

[0088] It should be noted that the fusion of the aforementioned facilitation and inhibition mechanisms may include, but is not limited to, the fusion of facilitation and inhibition mechanisms in biomimetic vision. Specifically, the fusion of facilitation and inhibition mechanisms in biomimetic vision is performed on at least two of two or more narrowband images to generate a fused image.

[0089] Among these, the fusion of facilitation and inhibition mechanisms in biomimetic vision includes, but is not limited to, the ON-center and OFF-center receptive field models in the rattlesnake's visual system. Correspondingly, based on the ON-center and OFF-center receptive field models in the rattlesnake's visual system, image processing is performed on specific features of each pair of narrowband images in two or more narrowband images to obtain suppressed and enhanced image features. These suppressed and enhanced image features are then fused to obtain a fused image.

[0090] To address this, every two narrow-band images are fused using a biomimetic vision-based facilitation and inhibition mechanism to generate a new monochrome image (also known as a fused image). This fully considers the interaction between the two narrow-band images to enhance contrast and depth resolution. The generated monochrome image is then fed into one of the RGB (Red, Green, Blue) channels for display, with the other color channels processed in the same way. See below for a detailed explanation.

[0091] The fusion of the aforementioned facilitation and inhibition mechanisms in biomimetic vision includes, but is not limited to, positive and negative feedback mechanisms in biological perception and behavioral learning, thereby achieving the fusion of facilitation and inhibition mechanisms. This requires sample training to obtain both suppressed and enhanced image features. These suppressed and enhanced image features are then fused to obtain a fused image.

[0092] It should also be noted that the fusion of the aforementioned facilitation and inhibition mechanisms may include, but is not limited to, the fusion of facilitation and inhibition mechanisms based on deep learning neural network models. These deep learning-based neural network models include activation functions for facilitation and attention mechanisms for inhibition. The facilitation and inhibition mechanisms in biological perception and behavioral learning are thus fused. This requires sample training to obtain both suppressed and enhanced image features. These suppressed and enhanced image features are then fused to obtain a fused image.

[0093] Step 140: Display the standard white light image, hyperspectral image, and pseudo-color image.

[0094] The aforementioned standard white light image, hyperspectral image, and pseudo-color image are used to display the real-time condition of the object under examination. Thus, using standard normal image display, hyperspectral image display, and high-contrast pseudo-color image display for the same object under examination facilitates analysis and observation.

[0095] The pseudo-color image used in this article can be a high-contrast pseudo-color image. In a high-contrast pseudo-color image, the difference between the brightest and darkest areas is greater. This makes the high-contrast pseudo-color image more vivid and clearly distinguishable in terms of light and dark. For example, the contrast ratio of a high-contrast pseudo-color image is typically greater than 5, and can even reach 10 or higher.

[0096] Combination Figure 1 As shown, in step 130 above, two or more narrowband images with center wavelength differences reaching a preset difference are selected from multiple narrowband images of the hyperspectral image. The preset difference indicates that these narrowband images have significant differences. This makes the subsequent fusion of promotion and inhibition mechanisms more meaningful.

[0097] The presupposed difference in this paper can be, but is not limited to, greater than or equal to 50 nm. For a sufficiently large information gap to make fusion more effective, the presupposed difference can be, but is not limited to, greater than or equal to 80 nm.

[0098] Based on this, combined Figure 1 As shown, step 130 of the above imaging method, which involves fusing at least two narrowband images based on two or more narrowband images using promotion and suppression mechanisms and combining color channels to generate a pseudo-color image, can be implemented through the following multiple embodiments:

[0099] In a first optional embodiment, the two or more narrowband images include any two narrowband images; each time any color channel of the RGB channel is mapped (which may be referred to as each mapping), a monochrome image is generated by fusing the facilitation and inhibition mechanisms based on the same two narrowband images. Accordingly, (1) a monochrome image is generated by fusing the facilitation and inhibition mechanisms of biomimetic vision based on the same two narrowband images. (2) The monochrome image generated based on the same two narrowband images is mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel, respectively. (3) The images mapped by the first color channel, the second color channel, and the third color channel are combined to obtain a pseudo-color image.

[0100] The "first" in "first color channel", the "second" in "second color channel", and the "third" in "third color channel" are used to distinguish the three different color channels.

[0101] In this embodiment, two narrowband images interact to produce a single image, which is directly mapped to one of the RGB channels. Thus, an image fusion method based on biomimetic visual mechanisms enhances the image's contrast and depth resolution.

[0102] In a second optional embodiment, the two or more narrowband images include any two or more narrowband images; each time any color channel of the RGB channel is mapped (which may be referred to as each mapping), a fusion of promotion and inhibition mechanisms is performed based on at least two different narrowband images to generate a corresponding monochrome image. Accordingly, 1) a fusion of biomimetic visual promotion and inhibition mechanisms is performed based on at least two different narrowband images to generate a corresponding monochrome image. 2) The monochrome images generated based on each pair of different narrowband images are respectively mapped to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel. 3) The images mapped to the first color channel, the second color channel, and the third color channel are combined to obtain a pseudo-color image.

[0103] It should be noted that the narrowband images of each pair of the above at least two bands are processed in pairs, and then the processed images are processed again to obtain an image, which is then mapped to one of the RGB channels.

[0104] In addition, the narrowband images of each pair of the above at least two bands are processed in pairs, and the processed images are mapped to one of the RGB channels.

[0105] To address this, each time a monochrome image is mapped to one of the RGB channels, and finally all mapped images are synthesized, the image is not directly output through the actual RGB channels, but rather the aforementioned fused image is generated. This fused image can be called a pseudo-color image.

[0106] For example, the first narrowband image O and the second narrowband image P interact to form a new monochrome image, mapped to one of the RGB channels;

[0107] The third narrowband image Q and the fourth narrowband image R interact to form a new monochrome image, which is mapped to one of the RGB channels.

[0108] The fifth narrowband image S and the fourth narrowband image T interact to form a new monochrome image, which is mapped to one of the RGB channels.

[0109] In this embodiment, a pseudo-color fusion algorithm is used, combining the advantages of multi-band narrowband imaging, to significantly improve image contrast and depth resolution. Furthermore, a pseudo-color image is generated through the mutual promotion and suppression of images, and mapped to one of the RGB channels; this process is repeated to finally generate the fused image. Thus, by utilizing a fusion algorithm based on biological visual mechanisms, hyperspectral image data is mapped to the RGB color space to enhance image contrast and resolution.

[0110] Continue as Figure 1 As shown, the acquisition of hyperspectral images in step 130 above can be implemented using at least any of the following methods:

[0111] In the first alternative implementation, a hyperspectral camera is used to acquire hyperspectral images. These hyperspectral images consist of multiple narrowband images generated from reflected light captured in a single pass across the entire spectral range by the hyperspectral camera. Thus, the hyperspectral camera is suitable for applications requiring real-time imaging, covering the visible to near-infrared bands.

[0112] In the second optional implementation, an image sensor and a color filter wheel are combined to obtain hyperspectral images. The color filter wheel contains filters for different wavelengths, and multiple narrowband images are obtained by rotating the color filter wheel to sequentially filter light of different wavelengths onto a monochrome sensor. The image sensor is used to capture light and convert it into electronic signals. This image sensor can be, but is not limited to, CMOS (Complementary Metal-Oxide-Semiconductor) sensors, monochrome image sensor cameras, etc., which will not be listed here. Thus, the combination of an image sensor and a color filter wheel can be used in scenarios with high resolution requirements and cost sensitivity, providing high-quality image resolution.

[0113] In the third optional implementation, a grating spectrophotometer is used to acquire hyperspectral images. A hyperspectral image is obtained by the grating spectrophotometer decomposing incoming white light into multiple wavelengths using a grating, and recording the spectral information of each wavelength separately. Thus, the grating spectrophotometer can provide high spectral resolution, suitable for detailed analysis of the tissue characteristics of the examined object.

[0114] Figure 2 As shown Figure 1 The diagram shows the specific process flow of the imaging method.

[0115] Continue as Figure 1 and Figure 2 As shown, step 140 above may include, but is not limited to, the following two embodiments:

[0116] In an optional embodiment of step 140 above, step 141 involves displaying a standard white light image, a hyperspectral image, and a pseudo-color image on a display. Correspondingly, the imaging method may further include step 150, receiving a switching command for the displayed standard white light image, hyperspectral image, and pseudo-color image. And, in step 160, responding to the switching command, switching to the desired image for display.

[0117] To address this, a single monitor can display standard white light images, hyperspectral images, and pseudo-color images separately. By switching between these images, the desired image can be displayed.

[0118] Furthermore, a single monitor can simultaneously display multiple images, including standard white light images, hyperspectral images, and pseudo-color images. By switching between these images, multiple images can be combined into a single display, allowing for magnification or more targeted display of the desired image.

[0119] In this embodiment, the desired image from a variety of images can be displayed.

[0120] In another optional embodiment of step 140 above, a standard white light image, a hyperspectral image, and a pseudo-color image are displayed on three separate displays. This allows for direct display of the images on the three displays without requiring the user to switch between them, enabling the use and display of the desired image.

[0121] Based on the same inventive concept as the methods described above, embodiments of this application also provide an imaging device, such as... Figure 3 As shown, the imaging device may include the following: illumination module 51, beam splitting module 52, white light imaging processing module 53, hyperspectral imaging processing module 54, and image display module 55:

[0122] Illumination module 51 is used to acquire standard white light to illuminate the object under inspection;

[0123] The beam splitting module 52 is used to split the reflected light of white light into a first beam and a second beam.

[0124] The white light imaging processing module 53 is used to acquire a standard white light image using the first beam; simultaneously,

[0125] The hyperspectral imaging processing module 54 is used to acquire a hyperspectral image using a second beam, and select two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and to perform fusion of promotion and suppression mechanisms and color channel combination based on at least two of the two or more narrowband images to generate a pseudo-color image.

[0126] Image display module 55 is used to display standard white light images, hyperspectral images and pseudo-color images.

[0127] As one embodiment, the image display module 55 is specifically used to display a standard white light image, a hyperspectral image, and a pseudo-color image on a display. Correspondingly, the imaging device further includes: a control module for receiving switching commands for the displayed standard white light image, hyperspectral image, and pseudo-color image; and a switching module for responding to the switching commands and switching to the desired image for display.

[0128] The imaging device provided in this application embodiment has the same inventive concept as the above method and can achieve the same or similar technical effects, and will not be listed one by one here.

[0129] Endoscopic technology has become an indispensable tool in clinical diagnosis and treatment, especially in the digestive and respiratory tracts, where white light is widely used as the mainstream technique. Specifically, endoscopy primarily relies on visible light (400-750 nm wavelength) imaging, which limits its ability to detect subtle tissue features and physiological changes, particularly in the identification of early-stage cancer, inflammation, and other lesions. Because white light images only present surface structural information and lack sensitivity to the chemical composition and function of deeper tissues, subtle lesions are often overlooked or misdiagnosed, affecting the accuracy of clinical diagnosis. Furthermore, using hyperspectral imaging (HSI) endoscopic systems typically requires switching between different light sources to acquire multi-band images, necessitating multiple light source switching operations. This complex user process impacts imaging efficiency and real-time performance, increasing the difficulty of clinical operation.

[0130] To address the aforementioned technical issues—that white light images only present surface tissue structure information and lack sensitivity to the chemical composition and function of deep tissues, often leading to the overlooking or misdiagnosis of subtle lesions and affecting the accuracy of clinical diagnosis; and that users need to switch light sources multiple times, resulting in complex user procedures, impacting imaging efficiency and real-time performance, and increasing the difficulty of clinical operation—this application provides an endoscopic imaging system to achieve the imaging method described above. It displays a standard white light image, performs white light analysis, and performs spectral imaging analysis using a hyperspectral image. It fuses two or more narrowband images with center wavelength differences reaching a preset difference using a facilitation and suppression mechanism to generate a fused image. Thus, through a beam-splitting module and a single white light source design, simultaneous acquisition of white light and hyperspectral images can be achieved without switching light sources, improving operational efficiency, imaging efficiency, and real-time performance, making it suitable for real-time clinical operations.

[0131] Based on the same inventive concept as the methods described above, embodiments of this application also provide an imaging device, such as... Figure 4 and Figure 5 The illustrated embodiments are similar to Figures 1 to 3 The illustrated embodiment, compared to Figures 1 to 3 The illustrated embodiment, in Figure 4 In this embodiment, an endoscopic imaging system is provided to implement the endoscopic imaging method described below. The system may include the following components: a white light illumination module 511, a beam splitting module 52, a white light imaging processing module 53, a hyperspectral imaging processing module 54, and an image display module 55.

[0132] The white light illumination module 511 is used to acquire standard white light to illuminate the target tissue under the endoscope. The target tissue can be used to represent the tissue characteristics of a specific object being examined. Thus, the white light illumination module 511 uses standard white light to illuminate the target tissue under the endoscope, eliminating the need for a dedicated multi-band light source and simplifying system design. The white light is used not only for normal observation but also for subsequent image acquisition.

[0133] The beam-splitting module 52 is used to split the reflected white light into a first beam and a second beam. The beam-splitting module 52 can be, but is not limited to, one or more of a beam splitter, beam-splitting mirror, fiber optic distributor, and optical grating. Thus, using standard white light as the light source, no special light source conversion is required; the light signal can be simultaneously used for white light imaging and hyperspectral imaging via the beam-splitting module 52. In this way, by using standard white light illumination and combining it with the beam-splitting module 52, the white light image acquisition unit and the HSI image acquisition unit can operate simultaneously. This allows for dual-modal imaging without switching light sources, simplifying operation and improving imaging efficiency.

[0134] To address this, after white light illumination, the reflected light is split into two paths by the beam splitter module 52. One path enters the white light image acquisition unit, and the other enters the HSI image acquisition unit. This structure ensures independent acquisition of the two types of images without interference.

[0135] The white light imaging processing module 53 is used to acquire a standard white light image using the first beam; simultaneously,

[0136] The hyperspectral imaging processing module 54 is used to acquire a hyperspectral image using a second beam, and select two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and to perform fusion of promotion and suppression mechanisms and color channel combination based on at least two of the two or more narrowband images to generate a pseudo-color image.

[0137] In this way, white light and hyperspectral images can be acquired simultaneously without switching light sources. The endoscopic imaging system uses a single white light source and is designed with a beam splitter module 52 to allow white light signals to enter different image acquisition modules (such as white light image acquisition unit and HSI image acquisition unit), thus achieving efficient dual-modal simultaneous imaging.

[0138] Among these technologies, HSI (High-Speed ​​Injection) technology, capable of capturing tissue characteristics across different spectra, shows great potential in improving the sensitivity of lesion detection. HSI technology can acquire information across multiple wavelengths from visible to near-infrared light, generating detailed spectral data for each pixel. This allows it to identify and distinguish different tissue components, such as the absorption differences between oxyhemoglobin and deoxyhemoglobin at different wavelengths. This characteristic gives HSI significant advantages in detecting minute lesions, differentiating between benign and malignant tissues, and monitoring surgical margins.

[0139] Furthermore, the aforementioned two or more narrowband images with a center wavelength difference reaching a preset difference can be, but are not limited to, two narrowband images with a center wavelength difference greater than 50 nm, or two narrowband images with a center wavelength difference greater than 80 nm. Thus, by introducing a bio-vision-based fusion algorithm, two narrowband images with a large center wavelength difference are fused through a facilitation and inhibition mechanism and mapped to the RGB color space, significantly enhancing the contrast and depth resolution of tissues.

[0140] Compared to imaging systems in related technologies, image fusion methods are relatively simple and cannot effectively improve the hierarchical information and contrast of tissues, thus limiting the application effect of the system in the observation of complex tissue structures.

[0141] Compared with related technologies, in the embodiments of this application, by selecting two or more narrowband images with a preset difference in center wavelength, and using an image fusion method that promotes and inhibits, the visualization effect of tissues is significantly improved, especially in lesion identification, early lesion detection and disease treatment.

[0142] The image display module 55 is used to display standard white light images, hyperspectral images, and pseudo-color images for real-time endoscopic examination. The image display module 55 may include, but is not limited to, a monitor or display screen.

[0143] Endoscopic systems using related technologies typically acquire different types of images only by switching light sources. However, this endoscopic imaging system, through its beam-splitting module 52, achieves simultaneous acquisition of white light and hyperspectral images, significantly improving operational convenience and imaging efficiency. Furthermore, the introduction of the HSI image fusion method allows the system to provide not only white light images but also richer tissue information through hyperspectral imaging. This combination of multimodal images provides more dimensional information for clinical diagnosis, contributing to improved diagnostic accuracy.

[0144] Compared to related technologies, the endoscopic imaging system in this embodiment combines the advantages of both WLI and HSI imaging methods, providing a flexible image acquisition and processing solution, and showing great potential, especially in lesion diagnosis and treatment and early lesion identification. This system simplifies the operation process while improving image quality, and has significant clinical application prospects.

[0145] Continue as Figure 5 As shown, the white light imaging processing module 53 includes a white light image acquisition unit and a white light image processing unit; the hyperspectral imaging processing module 54 includes an HSI image acquisition unit and an HSI image processing unit. It should be noted that the HSI image acquisition unit acquires hyperspectral image data through a hyperspectral sensor, covering image information from multiple narrowband light bands, providing information on the tissue's reflectance characteristics under different spectra.

[0146] The aforementioned white light image acquisition unit is used to acquire a standard white light image when using the first beam; simultaneously, the HSI image acquisition unit is used to acquire a hyperspectral image when using the second beam.

[0147] The aforementioned white light image processing unit is used to process a standard white light image; simultaneously, the aforementioned HSI image processing unit is used to select two or more narrowband images from multiple narrowband light band images in the hyperspectral image whose center wavelength difference reaches a preset difference; and to perform biomimetic visual mechanism interaction on each pair of the two or more narrowband images to generate a monochrome image.

[0148] It should be noted that the white light image processing unit and the HSI image processing unit mentioned above in this paper can be collectively referred to as the dual-channel image processing unit. The dual-channel image processing unit performs independent processing on the simultaneously acquired WLI (white light) image and HSI image.

[0149] The aforementioned white light image acquisition unit and the aforementioned HSI image acquisition unit are relatively independent acquisition channels, performing image acquisition simultaneously. The aforementioned white light image acquisition unit can achieve standard white light imaging with an acquisition frame rate of no less than 30fps, ensuring smooth imaging during real-time endoscopic examination. This white light image acquisition unit can be, but is not limited to, a CMOS (Complementary Metal-Oxide-Semiconductor Image Sensor). The white light image acquired using this CMOS is transmitted to the image processing module. This image processing module can include the aforementioned dual-channel image processing unit. This image processing module can be a chip used to implement image processing functions.

[0150] The HSI image acquisition unit described above acquires narrowband images of different wavelengths at a frame rate of no less than 15fps under hyperspectral imaging, with a band spacing of no less than 5nm, ensuring image resolution while improving the acquisition of deep tissue information.

[0151] In addition, the acquisition band spacing of the HSI image acquisition unit is not less than 5nm, and the narrowband spectral bandwidth is controlled within 10nm-30nm.

[0152] Furthermore, when selecting narrowband images, the HSI image acquisition unit ensures that the band spacing is greater than 5nm to guarantee the difference and accuracy of spectral images of different bands.

[0153] The aforementioned standard white light image, hyperspectral image, and fused image are all used to achieve real-time endoscopic examination.

[0154] In another embodiment, the hyperspectral imaging processing module 54 may also include, but is not limited to, an HSI image fusion module that selects two narrowband images with a wavelength difference greater than 80 nm from the HSI data for fusion. For example, based on fusion algorithms and fusion based on biological vision's promotion and inhibition mechanisms, the HSI image is mapped to the RGB color space.

[0155] Continue as Figure 5 As shown, in an optional embodiment, the two or more narrowband images mentioned above include any two narrowband images; each time mapping is performed, a fusion of the facilitation and suppression mechanisms is performed based on the same two narrowband images to generate a monochrome image.

[0156] Correspondingly, the hyperspectral imaging processing module 54 may include, but is not limited to: a mapping unit, and a fusion unit for promoting and inhibiting; the fusion unit for promoting and inhibiting is used to fuse the biomimetic vision promotion and inhibition mechanisms based on the same two narrowband images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on the same two narrowband images to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel, respectively; and to perform color channel combination on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image.

[0157] In the embodiments of this application, the fusion of pseudo-color images combines the advantages of multi-band narrowband imaging, which significantly improves the contrast and depth resolution of the images, making it easier for doctors to identify early lesions during observation.

[0158] In another alternative embodiment, the two or more narrowband images mentioned above include any two or more narrowband images; during each mapping, a fusion of promotion and suppression mechanisms is performed based on at least every two different narrowband images to generate a monochrome image accordingly.

[0159] Correspondingly, the hyperspectral imaging processing module 54 may include, but is not limited to: a mapping unit, and a fusion unit for promoting and inhibiting; the fusion unit for promoting and inhibiting is used to fuse the biomimetic vision promotion and inhibition mechanisms based on each pair of different narrowband images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on each pair of different narrowband images to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel, respectively; and to perform color channel combination on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image.

[0160] Continue as Figure 5 As shown, the image display module 55 may include, but is not limited to, one or two displays; the endoscopic imaging system also includes a control module for receiving switching commands for standard white light images, hyperspectral images, and pseudo-color images, and switching to the desired image. In this way, doctors can select different types of images to display as needed for better lesion observation and diagnosis.

[0161] As an example, the control module described above is also used to receive any screenshot command for a standard white light image, a hyperspectral image, and a pseudo-color image, to take a screenshot of the current image; to save the screenshot and generate a report; the report is used to display the diagnostic results.

[0162] The control module can implement control via the aforementioned chip. This control module may include a foot pedal controller and a controller. For example, the screenshot command is input by the doctor pressing the foot pedal controller.

[0163] The image display module 55 includes three displays; each of the three displays is used to display a standard white light image, a hyperspectral image, and a pseudo-color image, respectively.

[0164] Continue to combine Figure 5 As shown in the embodiments of this application, the hyperspectral imaging processing module 54 can be implemented by at least any one of the following three embodiments of the HSI image acquisition unit: 1) a hyperspectral camera, suitable for clinical environments requiring rapid response; 2) a monochromatic sensor with a color filter wheel, suitable for applications requiring high-resolution imaging and low cost; 3) a grating spectrometer, suitable for tissue analysis scenarios requiring high spectral accuracy. These three methods provide flexibility and economical options for different diagnostic and treatment needs.

[0165] 1) Hyperspectral camera

[0166] In a first optional embodiment, the hyperspectral imaging processing module 54 includes an HSI image acquisition unit: the HSI image acquisition unit is used to obtain hyperspectral images using a hyperspectral camera; the hyperspectral images include multiple narrowband images generated by the hyperspectral camera acquiring reflected light across the entire spectral range at once.

[0167] The first optional embodiment described above utilizes a hyperspectral camera for rapid image acquisition, enabling real-time acquisition of hyperspectral data and images, making it suitable for clinical applications requiring rapid imaging and real-time analysis. Hyperspectral cameras can cover the spectral range from visible to near-infrared, thus providing broader tissue information. Therefore, hyperspectral cameras are well-suited for clinical environments demanding real-time imaging and rapid response.

[0168] 2) Monochrome sensor with color filter wheel

[0169] In a second optional embodiment, the HSI image acquisition unit is used to obtain a hyperspectral image by combining an image sensor with a color filter wheel; the color filter wheel contains filters of different wavelengths, and multiple narrowband images are obtained by rotating the color filter wheel to sequentially filter light of different wavelengths onto a monochromatic sensor.

[0170] The second optional embodiment described above, which combines an image sensor with a color filter wheel, offers relatively low cost and is suitable for applications requiring high resolution, enabling the acquisition of high-quality image data via the image sensor. While this method is relatively slow in imaging speed, its flexibility and cost-effectiveness make it a good choice. Thus, the combination of an image sensor and a color filter wheel is suitable for scenarios requiring high-quality image resolution and where cost is relatively sensitive.

[0171] 3) Grating spectrometer

[0172] In a third optional embodiment, the HSI image acquisition unit is used to acquire a hyperspectral image using a grating spectrometer; the hyperspectral image is the spectral information of each wavelength recorded by the grating spectrometer after decomposing incoming white light into multiple spectra of different wavelengths through a grating.

[0173] The third optional embodiment described above provides extremely high spectral resolution, making it suitable for scenarios requiring high precision in analyzing tissue spectral properties. Grating spectrophotometers possess excellent wavelength accuracy and sensitivity, enabling them to capture subtle spectral changes in tissues, making them ideal for fine imaging and analysis of lesions. Thus, grating spectrophotometers are designed for high-precision spectral resolution applications, suitable for scenarios requiring detailed analysis of tissue spectral properties.

[0174] This application provides three acquisition schemes to optimize imaging effects under different clinical needs, thereby achieving diversified HSI image acquisition. Furthermore, through these diverse image acquisition methods, this system can provide highly adaptable solutions to meet a wide range of medical imaging needs, from routine endoscopy to hyperspectral diagnostics.

[0175] The three hyperspectral acquisition methods presented in this article provide solutions for different clinical application needs, ensuring that the system can adapt to a wide range of medical scenarios. This enhances the system's flexibility and adaptability.

[0176] The implementation process of the functions and roles of each module / unit in this article can be found in the implementation process of the corresponding steps in the above method, which can achieve the same technical effect, and will not be repeated here.

[0177] Figure 6 The diagram shown is a flowchart of the endoscopic imaging method of this application.

[0178] like Figure 6 As shown, the endoscopic imaging method may include, but is not limited to, the following steps 210 to 240:

[0179] Step 210: The white light illumination module acquires standard white light to illuminate the target tissue of the endoscope.

[0180] Step 220: The beam splitting module splits the reflected light of the white light into a first beam and a second beam.

[0181] Step 230: The white light imaging processing module uses the first beam to acquire a standard white light image; simultaneously, the hyperspectral imaging processing module uses the second beam to acquire a hyperspectral image, and selects two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and, based on at least any two of the two or more narrowband images, performs a fusion of promotion and suppression mechanisms and color channel combination to generate a pseudo-color image.

[0182] Step 230 can simultaneously acquire white light (WLI) and hyperspectral (HSI) images of the same location without switching between the two light sources. This simultaneous acquisition of white light and HSI images improves operational efficiency and avoids potentially missing important tissue information when switching modes.

[0183] Furthermore, WLI and HSI images undergo independent processing, allowing doctors to select one or a combination of both images to display in the image display module as needed.

[0184] Step 240: The image display module displays a standard white light image, a hyperspectral image, and a pseudo-color image to enable real-time endoscopic examination.

[0185] In this embodiment, dual-path independent imaging (WLI and HSI) is achieved through white light illumination, eliminating the need to switch light sources. Simultaneously, the HSI image fusion module employs a multi-band narrowband image fusion method, significantly improving image depth resolution and tissue visualization. This system not only enhances diagnostic efficiency but also assists physicians in providing precise subsequent guidance.

[0186] The endoscopic imaging method and system described in this article can achieve the following effects:

[0187] 1. Clinical efficacy verification: Conduct experiments in different clinical scenarios, accumulate data and verify the effectiveness of the fusion algorithm of promotion and inhibition mechanisms in early lesion detection, and provide quantitative data support.

[0188] 2. Modular design and cost control: By using modular design and low-cost solutions, the applicability and scalability of the system in different medical scenarios are improved, thereby enhancing its market potential.

[0189] This application provides an electronic device, including one or more processors, for implementing the imaging method described above.

[0190] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, or a handheld terminal, etc. Any electronic device that can implement the embodiments of this invention falls within the protection scope of this invention and is not limited thereto.

[0191] The endoscopic imaging methods and systems, imaging methods and devices described in this article all share the same inventive concept, and the same or similar steps can achieve the same effect.

[0192] Figure 7 The diagram shown is a structural schematic of the electronic device 70 provided in an embodiment of this application.

[0193] like Figure 7 As shown, the electronic device 70 includes one or more processors 71 for implementing the imaging method described above.

[0194] In some embodiments, electronic device 70 may include storage medium 79. For example, computer-readable storage medium may store a program that can be invoked by processor 71, and may include non-volatile storage medium. In some embodiments, electronic device 70 may include memory 78 and interface 77. In some embodiments, electronic device 70 may also include other hardware depending on the specific application.

[0195] The computer-readable storage medium of this application embodiment stores a program that, when executed by processor 71, is used to implement the imaging method described above.

[0196] This application provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described in any of the preceding claims.

[0197] This application also provides a computer program stored in a computer-readable storage medium, such as... Figure 7 The storage medium 79, and when the processor executes the computer program, causes the processor 71 to perform the method described above.

[0198] This application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented using any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0199] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

[0200] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element qualified by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. An endoscopic imaging system, characterized in that, include: The white light illumination module is used to acquire standard white light to illuminate the target tissue of the endoscope; The beam splitting module is used to split the reflected light of white light into a first beam and a second beam. The white light imaging processing module is used to acquire a standard white light image using the first beam; simultaneously, The hyperspectral imaging processing module is used to acquire a hyperspectral image using a second beam, and select two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength difference reaches a preset difference; and to perform a fusion of promotion and suppression mechanisms and color channel combination based on at least any two of the two or more narrowband images to generate a pseudo-color image; the fusion of promotion and suppression mechanisms is used to perform image processing on specific features of each pair of narrowband images that represent differences in light absorption to obtain suppressed image features and enhanced image features, and to fuse the suppressed image features and enhanced image features; It includes the fusion of facilitation and inhibition mechanisms in biomimetic vision, and / or the fusion of facilitation and inhibition mechanisms based on a deep learning-based neural network model; the fusion of facilitation and inhibition mechanisms in biomimetic vision includes receptive field models of ON-center and OFF-center in the rattlesnake's visual system; based on the receptive field model, image processing is performed on specific features of each pair of narrowband images in two or more narrowband images to obtain suppressed image features and enhanced image features, and the suppressed image features and enhanced image features are fused to obtain a monochrome image; based on the neural network model including activation functions for facilitation and attention mechanisms for inhibition, a monochrome image is obtained through facilitation and inhibition mechanisms in biological perception and behavioral learning; The image display module is used to display standard white light images, hyperspectral images, and pseudo-color images to enable real-time endoscopic examination.

2. The endoscopic imaging system as described in claim 1, characterized in that, The two or more narrowband images include any two narrowband images; during each mapping, a fusion of promotion and suppression mechanisms is performed based on the same two narrowband images to generate a monochrome image; The hyperspectral imaging processing module includes: a mapping unit and a fusion unit for promoting and inhibiting; the fusion unit for promoting and inhibiting is used to fuse the biomimetic vision promotion and inhibition mechanisms based on the same two narrowband images to generate the monochrome image; the mapping unit is used to map the monochrome image generated based on the same two narrowband images to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel, respectively; and to perform color channel combination on the images mapped by the first color channel, the second color channel, and the third color channel to obtain a pseudo-color image; or, The two or more narrowband images include any two or more narrowband images; during each mapping, a fusion of promotion and inhibition mechanisms is performed based on at least two different narrowband images to generate a monochrome image accordingly; the hyperspectral imaging processing module includes: a mapping unit and a fusion unit for promotion and inhibition; the fusion unit for promotion and inhibition is used to fuse the biomimetic vision promotion and inhibition mechanisms based on each pair of different narrowband images to generate the monochrome image accordingly; the mapping unit is used to map the monochrome images generated based on each pair of different narrowband images to a first color channel of the RGB channel, a second color channel of the RGB channel, and a third color channel of the RGB channel, respectively; the images mapped to the first color channel, the second color channel, and the third color channel are combined to obtain a pseudo-color image.

3. The endoscopic imaging system as described in claim 1 or 2, characterized in that, The image display module includes one or two displays; the endoscopic imaging system further includes a control module for receiving switching instructions for the standard white light image, the hyperspectral image, and the pseudo-color image, and switching to the desired image for display. or, The image display module includes three displays; the three displays are used to display the standard white light image, the hyperspectral image, and the pseudo-color image, respectively.

4. The endoscopic imaging system as described in claim 1 or 2, characterized in that, The hyperspectral imaging processing module includes an HSI image acquisition unit: The HSI image acquisition unit is used to obtain hyperspectral images using a hyperspectral camera; the hyperspectral images include multiple narrowband images generated by the hyperspectral camera acquiring reflected light across the entire spectral range at once.

5. The endoscopic imaging system as described in claim 1 or 2, characterized in that, The hyperspectral imaging processing module includes an HSI image acquisition unit: The HSI image acquisition unit is used to obtain hyperspectral images by combining an image sensor with a color filter wheel; the color filter wheel contains filters of different wavelengths, and the multiple narrowband images are obtained by rotating the color filter wheel to sequentially filter light of different wavelengths onto a monochromatic sensor.

6. The endoscopic imaging system as described in claim 1 or 2, characterized in that, The hyperspectral imaging processing module includes an HSI image acquisition unit: The HSI image acquisition unit is used to acquire hyperspectral images using a grating spectrometer; the hyperspectral image is the spectral information of each wavelength recorded by the grating spectrometer, which decomposes incoming white light into multiple spectra of different wavelengths through a grating.

7. An endoscopic imaging method, characterized in that, include: The white light illumination module acquires standard white light to illuminate the target tissue of the endoscope; The beam splitting module divides the reflected light of the white light into a first beam and a second beam. The white light imaging processing module uses the first beam to acquire a standard white light image; simultaneously, The hyperspectral imaging processing module uses the second beam to acquire a hyperspectral image, and selects two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength differences reach a preset difference; and, based on at least any two of the two or more narrowband images, performs a fusion of promotion and suppression mechanisms and color channel combination to generate a pseudo-color image; the fusion of promotion and suppression mechanisms is used to perform image processing on specific features of each pair of narrowband images that represent differences in light absorption, to obtain suppressed image features and enhanced image features, and then fuses the suppressed image features and enhanced image features; It includes the fusion of facilitation and inhibition mechanisms in biomimetic vision, and / or the fusion of facilitation and inhibition mechanisms based on a deep learning-based neural network model; the fusion of facilitation and inhibition mechanisms in biomimetic vision includes receptive field models of ON-center and OFF-center in the rattlesnake's visual system; based on the receptive field model, image processing is performed on specific features of each pair of narrowband images in two or more narrowband images to obtain suppressed image features and enhanced image features, and the suppressed image features and enhanced image features are fused to obtain a monochrome image; based on the neural network model including activation functions for facilitation and attention mechanisms for inhibition, a monochrome image is obtained through facilitation and inhibition mechanisms in biological perception and behavioral learning; The image display module displays the standard white light image, the hyperspectral image, and the pseudo-color image to enable real-time endoscopic examination.

8. An imaging method, characterized in that, include: Obtain standard white light to illuminate the object being inspected; The reflected light of the white light is divided into a first beam and a second beam; Using the first beam, a standard white light image is acquired; simultaneously... Using the second beam, a hyperspectral image is acquired, and from multiple narrowband images of the hyperspectral image, two or more narrowband images with a center wavelength difference reaching a preset difference are selected; and, based on at least any two of the two or more narrowband images, a fusion of promotion and suppression mechanisms and color channel combination are performed to generate a pseudo-color image; the fusion of promotion and suppression mechanisms is used to perform image processing on specific features of each pair of narrowband images that represent differences in light absorption, to obtain suppressed image features and enhanced image features, and the suppressed image features and enhanced image features are fused; It includes the fusion of facilitation and inhibition mechanisms in biomimetic vision, and / or the fusion of facilitation and inhibition mechanisms based on a deep learning-based neural network model; the fusion of facilitation and inhibition mechanisms in biomimetic vision includes receptive field models of ON-center and OFF-center in the rattlesnake's visual system; based on the receptive field model, image processing is performed on specific features of each pair of narrowband images in two or more narrowband images to obtain suppressed image features and enhanced image features, and the suppressed image features and enhanced image features are fused to obtain a monochrome image; based on the neural network model including activation functions for facilitation and attention mechanisms for inhibition, a monochrome image is obtained through facilitation and inhibition mechanisms in biological perception and behavioral learning; Display the standard white light image, the hyperspectral image, and the pseudo-color image.

9. The imaging method as described in claim 8, characterized in that, The two or more narrowband images include any two narrowband images; each time any color channel of the RGB channel is mapped, a monochrome image will be generated based on the same two narrowband images; The process of fusing at least two narrowband images based on the above two or more narrowband images using promotion and suppression mechanisms and combining color channels to generate a pseudo-color image includes: A monochrome image is generated by fusing the facilitation and inhibition mechanisms of biomimetic vision based on the two narrowband images mentioned above. The monochrome image generated based on the same two narrowband images is respectively mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel; A pseudo-color image is obtained by combining the color channels of the image mapped by the first color channel, the second color channel, and the third color channel.

10. The imaging method as described in claim 8, characterized in that, The two or more narrowband images include any two or more narrowband images; each time any color channel of the RGB channel is mapped, a monochrome image is generated based on each pair of narrowband images; The process of fusing at least two narrowband images based on the above two or more narrowband images using promotion and suppression mechanisms and combining color channels to generate a pseudo-color image includes: The fusion of biomimetic vision enhancement and inhibition mechanisms based on at least two different narrowband images generates a monochrome image accordingly. The monochrome images generated based on each pair of narrowband images are respectively mapped to the first color channel of the RGB channel, the second color channel of the RGB channel, and the third color channel of the RGB channel; A pseudo-color image is obtained by combining the color channels of the image mapped by the first color channel, the second color channel, and the third color channel.

11. The imaging method according to any one of claims 8 to 10, characterized in that, The acquisition of hyperspectral images includes: using a hyperspectral camera to obtain hyperspectral images; the hyperspectral images include multiple narrowband images generated by the hyperspectral camera acquiring reflected light across the entire spectral range at once.

12. The imaging method according to any one of claims 8 to 10, characterized in that, The acquisition of hyperspectral images includes: using an image sensor and a color filter wheel to obtain hyperspectral images; the color filter wheel contains filters of different wavelengths, and the multiple narrowband images are obtained by rotating the color filter wheel to sequentially filter light of different wavelengths onto a monochromatic sensor.

13. The imaging method according to any one of claims 8 to 10, characterized in that, The acquisition of hyperspectral images includes: using a grating spectrophotometer to acquire hyperspectral images; the hyperspectral images are the spectral information of each wavelength recorded by the grating spectrophotometer, which decomposes incoming white light into multiple spectra of different wavelengths through a grating.

14. The imaging method according to any one of claims 8 to 10, characterized in that, The method of displaying the standard white light image, the hyperspectral image, and the pseudo-color image includes: displaying the standard white light image, the hyperspectral image, and the pseudo-color image on a display; the imaging method further includes: receiving a switching instruction for the displayed standard white light image, the hyperspectral image, and the pseudo-color image, and, in response to the switching instruction, switching to the desired image to be displayed; or, The display of the standard white light image, the hyperspectral image, and the pseudo-color image includes: displaying the standard white light image, the hyperspectral image, and the pseudo-color image on three separate displays.

15. An imaging device, characterized in that, For implementing the imaging method as described in any one of claims 8 to 14, the imaging apparatus comprises: The lighting module is used to acquire standard white light to illuminate the object being inspected; The beam splitting module is used to split the reflected light of the white light into a first beam and a second beam. The white light imaging processing module is used to acquire a standard white light image using the first beam; simultaneously... A hyperspectral imaging processing module is used to acquire a hyperspectral image using the second beam, and select two or more narrowband images from multiple narrowband images of the hyperspectral image whose center wavelength differences reach a preset difference; and to perform a fusion of promotion and inhibition mechanisms and color channel combination based on at least any two of the two or more narrowband images to generate a pseudo-color image; the fusion of promotion and inhibition mechanisms is used to process specific features representing differences in light absorption in each pair of narrowband images to obtain suppressed image features and enhanced image features, and to fuse the suppressed image features and enhanced image features; it includes the fusion of promotion and inhibition mechanisms in biomimetic vision, and / Or, a fusion of facilitation and inhibition mechanisms based on a deep learning-based neural network model; the fusion of facilitation and inhibition mechanisms in biomimetic vision includes receptive field models of ON-center and OFF-center in the rattlesnake's visual system; based on the receptive field model, image processing is performed on specific features of each pair of narrowband images in two or more narrowband images to obtain suppressed image features and enhanced image features, and the suppressed image features and enhanced image features are fused to obtain a monochrome image; based on the neural network model including activation functions for facilitation and attention mechanisms for inhibition, a monochrome image is obtained through facilitation and inhibition mechanisms in biological perception and behavioral learning; An image display module is used to display the standard white light image, the hyperspectral image, and the pseudo-color image.

16. An electronic device, characterized in that, It includes one or more processors for implementing the imaging method as described in any one of claims 8 to 14.

Images