Endoscopic image fusion system and method, image fusion method and apparatus, and device

Abstract

The application provides an endoscope image fusion system and method, an image fusion method and device, and equipment. The system comprises a narrow-band light acquisition module, an image extraction module, an image processing module, and a pseudo-color image generation module. The narrow-band light acquisition module is configured to acquire first narrow-band light and second narrow-band light with a preset difference in central wavelength. The image extraction module is configured to use the first narrow-band light and the second narrow-band light to irradiate a target tissue, and extract shallow images and deep images of different depths. The image processing module is configured to perform fusion of a mutual promotion and inhibition mechanism of biological bionic vision based on the shallow images and the deep images of different depths, and obtain inhibited image features and enhanced image features. The pseudo-color image generation module is configured to perform color channel combination based on the inhibited image features and the enhanced image features, and generate a pseudo-color image. The pseudo-color image is used to represent tissue features of the target tissue.

Description

Technical Field

[0001] The present invention relates to the field of image fusion technology, and in particular to an endoscopic image fusion system and method, an image fusion method and device, and equipment. Background Art

[0002] Narrow Band Imaging (NBI) technology uses narrowband light wavelengths of 415 nm and 540 nm. The 540 nm image is placed in the R channel of the RGB (Red, Green, Blue) system, while the 415 nm image is placed in the G and B channels of the RGB system. This results in a monotonous image with limited depth distinction. Summary of the Invention

[0003] The present application provides an improved endoscopic image fusion system and method, image fusion method and apparatus, and equipment.

[0004] The present application provides an endoscopic image fusion system, comprising:

[0005] a narrowband light acquisition module, configured to acquire first narrowband light of a first central wavelength and second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaching a preset difference; the first narrowband light being used to capture tissue features of a shallow layer in a target tissue of an endoscope, and the second narrowband light being used to capture tissue features of a deep layer in the target tissue;

[0006] an image extraction module, configured to respectively illuminate the target tissue using the first narrowband light and the second narrowband light, and extract shallow layer images and deep layer images at different depths;

[0007] An image processing module is configured to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and enhanced image features both include specific features; the specific features are used to represent features with different light absorption;

[0008] The pseudo-color image generation module is used to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

[0009] Furthermore, the image processing module includes an image processing submodule;

[0010] The image processing submodule is used to: based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, perform image processing on the specific features of the shallow image and the specific features of the deep image respectively to obtain suppressed image features and enhanced image features.

[0011] Furthermore, the image processing submodule includes a first image processing unit and a second image processing unit:

[0012] The first image processing unit is configured to apply ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G);

[0013] The second image processing unit is used to apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B), and to apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0014] Furthermore, the image processing module includes a first fusion image generation submodule and a second fusion image generation submodule; the pseudo-color image generation module includes a first mapping submodule and a second mapping submodule;

[0015] The first fusion image generation submodule is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, and use the deep enhanced image features (+G), the shallow suppressed image features (-B), and the shallow enhanced image features (+B) to generate monochrome images accordingly; the first mapping submodule is used to map the corresponding monochrome images to the red channel and the blue channel of the RGB channel respectively;

[0016] The second fusion image generation submodule is used to combine the promoting effect of the deep-layer enhanced image features (+G) on the shallow-layer suppressed image features (-B), and the suppressing effect of the shallow-layer enhanced image features (+B) on the deep-layer enhanced image features (+G), to generate a monochrome image; the second mapping submodule is used to map the monochrome image to the green channel in the RGB channel.

[0017] Furthermore, the first fused image generation submodule includes a first fused image generation unit and a second fused image generation unit; the first mapping submodule includes a first mapping unit and a second mapping unit;

[0018] The first fusion image generation unit is configured to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the deep layer enhanced image feature (+G) and the shallow layer suppressed image feature (-B) to achieve the effect of the shallow layer image promoting the deep layer image, and generate a monochrome image; the first mapping unit is configured to map the monochrome image to a red channel in an RGB channel;

[0019] The second fusion image generation unit is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the deep enhanced image features (+G) and the shallow enhanced image features (+B) to achieve the inhibitory effect of the deep image on the shallow image, and generate a monochrome image; the second mapping unit is used to map the monochrome image to the blue channel in the RGB channel.

[0020] The present invention provides an endoscopic image fusion method, comprising:

[0021] The narrowband light acquisition module acquires first narrowband light of a first central wavelength and second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture tissue features of a shallow layer in a target tissue of an endoscope, and the second narrowband light is used to capture tissue features of a deep layer in the target tissue;

[0022] The image extraction module uses the first narrowband light and the second narrowband light to illuminate the target tissue respectively, and extracts shallow layer images and deep layer images at different depths;

[0023] The image processing module integrates the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and enhanced image features both include specific features; the specific features are used to represent features with different light absorption;

[0024] The pseudo-color image generation module combines color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

[0025] The present invention provides an image fusion method, including:

[0026] Acquire a first narrowband light having a first central wavelength and a second narrowband light having a second central wavelength; wherein a central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; wherein the first narrowband light is used to capture features of a shallow layer of an object under inspection, and the second narrowband light is used to capture features of a deep layer of the object under inspection;

[0027] irradiating the subject with the first narrowband light and the second narrowband light respectively, and extracting shallow layer images and deep layer images at different depths;

[0028] Based on shallow and deep images at different depths, a fusion of mutual promotion and inhibition mechanisms of biomimetic vision is performed to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different light absorption;

[0029] Color channels are combined based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under inspection.

[0030] Furthermore, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated based on shallow and deep images of different depths to obtain suppressed and enhanced image features, including:

[0031] Based on the ON-center and OFF-center receptive field models in the visual system of the rattlesnake, image processing is performed on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features.

[0032] Furthermore, the ON-center type and OFF-center type receptive field models in the rattlesnake-based visual system respectively perform image processing on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features, including:

[0033] Applying ON-center receptive field processing to the deep image to enhance specific features of the deep image to obtain deep enhanced image features (+G);

[0034] Applying OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B); and applying ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0035] Furthermore, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated based on shallow and deep images of different depths to obtain suppressed and enhanced image features, including:

[0036] Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated. The deep enhanced image features (+G), shallow suppressed image features (-B), and shallow enhanced image features (+B) are used to generate monochrome images. The corresponding monochrome images are mapped to the red channel and blue channel of the RGB channel respectively.

[0037] The promoting effect of the deep-layer enhanced image features (+G) on the shallow-layer suppressed image features (-B) and the suppressing effect of the shallow-layer enhanced image features (+B) on the deep-layer enhanced image features (+G) are merged to generate a monochrome image; and the monochrome image is mapped to the green channel in the RGB channel.

[0038] Furthermore, the Gaussian difference function model is used to simulate the antagonistic receptive field of ganglion cells, and the deep enhanced image features (+G), the shallow suppressed image features (-B), and the shallow enhanced image features (+B) are used to generate monochrome images; and the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively, including:

[0039] Simulating the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, using the deep layer enhanced image feature (+G) and the shallow layer suppressed image feature (-B) to achieve the effect of the shallow layer image promoting the deep layer image, thereby generating a monochrome image; the first mapping unit is used to map the monochrome image to the red channel of the RGB channel; and mapping the monochrome image to the red channel of the RGB channel;

[0040] The antagonistic receptive field of ganglion cells is simulated based on a Gaussian difference function model, and the deep-layer enhanced image features (+G) and the shallow-layer enhanced image features (+B) are used to achieve the inhibitory effect of the deep-layer image on the shallow-layer image, thereby generating a monochrome image; and mapping the monochrome image to the blue channel in the RGB channel.

[0041] The present application provides an image fusion device for implementing the above-mentioned image fusion method, the image fusion device comprising:

[0042] a narrowband light acquisition module, configured to acquire first narrowband light of a first central wavelength and second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaching a preset difference; the first narrowband light being used to capture shallow features of an object under inspection, and the second narrowband light being used to capture deep features of the object under inspection;

[0043] an image extraction module, configured to respectively illuminate a subject using the first narrowband light and the second narrowband light to extract shallow layer images and deep layer images at different depths;

[0044] An image processing module is configured to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and enhanced image features both include specific features; the specific features are used to represent features with different light absorption;

[0045] The pseudo-color image generation module is used to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under inspection.

[0046] The present application provides an electronic device, comprising one or more processors, for implementing the image fusion method as described in any one of the above items.

[0047] The present application provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method described in any one of the above items is implemented.

[0048] The present application provides a computer program product, comprising a computer program / instruction, which implements any of the above methods when executed by a processor.

[0049] In some embodiments, the endoscopic image fusion system of the present application generates pseudo-color images with enhanced contrast and depth resolution, which can enhance the depth resolution of the image, enhance the color contrast and depth level of the image, and make the color representation richer and have sufficient contrast (depth differentiation capability). This makes the depth and color differentiation more obvious, thereby providing a clearer visualization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure shows a module schematic diagram of an endoscopic image fusion system provided by an embodiment of the application;

[0051] Figure 2 Shown Figure 1 Schematic diagram of the ON-center type in the endoscopic image fusion system shown;

[0052] Figure 3 Shown Figure 1 Schematic diagram of the OFF-center type in the endoscopic image fusion system shown;

[0053] Figure 4 Shown Figure 1 Schematic diagram of the application of the endoscopic image fusion system shown;

[0054] Figure 5 FIG2 is a flow chart of an endoscopic image fusion method according to an embodiment of the present application;

[0055] Figure 6 Schematic diagram of the process of the image fusion method provided in the embodiment of the present application;

[0056] Figure 7 Shown is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with one or more embodiments of this specification. Rather, they are merely examples of apparatuses and methods consistent with certain aspects of one or more embodiments of this specification, as detailed in the appended claims.

[0058] It should be noted that in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the method may include more or fewer steps than those described in this specification. In addition, a single step described in this specification may be broken down into multiple steps for description in other embodiments, and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0059] In order to solve the technical problem that the color of the above-mentioned displayed image is relatively monotonous and lacks sufficient contrast, an embodiment of the present application provides an endoscope image fusion system, including a narrowband light acquisition module, an image extraction module, an image processing module, and a pseudo-color image generation module. Among them, the narrowband light acquisition module acquires a first narrowband light for capturing shallow features and a second narrowband light for capturing deep features, and extracts shallow images and deep images of different depths through the image extraction module; the image processing module is used to fuse the mutual promotion and inhibition mechanism of biomimetic vision based on the shallow images and deep images of different depths, and obtains suppressed image features and enhanced image features; finally, the pseudo-color image generation module is used to combine color channels based on the suppressed image features and enhanced image features to generate a pseudo-color image.

[0060] In the embodiments of the present application, the pseudo-color image has enhanced contrast and depth resolution, which can enhance the depth resolution of the image, enhance the color contrast and depth level of the image, make the color representation richer, and have sufficient contrast (depth differentiation ability). This makes the depth and color differentiation more obvious, thereby providing a clearer visualization effect.

[0061] With the rapid development of endoscopic technology, narrow-band light imaging technology (NBI) has become a common imaging method in clinical practice, playing an important role in the early screening and lesion detection of digestive and respiratory diseases. Current endoscopic staining technology typically uses narrow-band light wavelengths corresponding to the hemoglobin absorption peak, specifically 415nm and 540nm. By enhancing the visualization of blood vessels and mucosal tissues with these wavelengths, doctors can observe vascular structures and possible lesions more clearly. However, the related technology of narrow-band light imaging technology NBI technology has great limitations in image processing.

[0062] Most current narrow-band imaging (NBI) systems use a simple image fusion method, directly assigning different narrow-band images to RGB color channels. While this processing method can enhance the observation of tissue structure to a certain extent, the image color is relatively monotonous and lacks sufficient contrast. Furthermore, this method has shortcomings in depth resolution, making it difficult to accurately distinguish structures at different levels of tissue, limiting its application in complex tissues and early-stage lesions.

[0063] In this context, in order to solve the technical problems that the above-mentioned images have relatively monotonous color representation, lack of sufficient contrast, and insufficient depth resolution, making it difficult to accurately distinguish the structures of different levels of tissues, which limits their application effect in complex tissues and early lesions, the embodiment of the present application provides an endoscopic image fusion system, and develops a new image fusion method that can fully utilize the interaction between narrow-band light images to improve the color contrast and depth resolution of images, which has become the focus of current endoscopic imaging technology research. At the same time, based on bionic vision inspiration, a new image processing method is proposed, which aims to improve the overall performance of the narrow-band light imaging system, provide higher quality image support for clinical diagnosis, and break through the limitations of narrow-band light imaging technology in related technologies.

[0064] Figure 1 Shown is a module schematic diagram of an endoscopic image fusion system provided in an embodiment of the application.

[0065] First, as Figure 1 As shown, the endoscopic image fusion system may include but is not limited to the following narrow-band light acquisition module 11, image extraction module 12, image processing module 13 and pseudo-color image generation module 14:

[0066] The narrowband light acquisition module 11 is configured to acquire first narrowband light having a first central wavelength and second narrowband light having a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light being a predetermined difference; the first narrowband light being configured to capture superficial tissue features within the target tissue of the endoscope, and the second narrowband light being configured to capture deeper tissue features within the target tissue. The target tissue is tissue of the subject being examined.

[0067] The first and second narrowband lights herein have different central wavelengths, with a certain difference. This makes the subsequent integration of the mutual promotion and inhibition mechanisms of biomimetic vision more meaningful. For example, the preset difference can be used to indicate that these narrowband images are significantly different. This further enhances the integration of the subsequent promotion and inhibition mechanisms.

[0068] The preset difference in this article can be, but is not limited to, greater than or equal to 50 nm (nanometers). In order to have a sufficiently large information gap to make the fusion more effective, the preset difference can be, but is not limited to, greater than or equal to 80 nm (nanometers).

[0069] Image extraction module 12 is configured to illuminate target tissue using first and second narrowband light, respectively, to extract shallow and deep layer images at different depths. The shallow and deep layer images have different image depths. Deep layer tissue features in the deep layer image have a deeper image depth than shallow layer tissue features in the shallow layer image. The object illuminated by the two narrowband light beams can be referred to as an inspected object.

[0070] The image processing module 13 is used to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different light absorption.

[0071] The pseudo-color image generating module 14 is configured to generate a pseudo-color image by combining color channels based on the suppressed image features and the enhanced image features; the pseudo-color image is used to represent the tissue features of the target tissue.

[0072] The color images mentioned in this article are not directly generated from actual RGB, but rather mapped, and are referred to as pseudo-color images. These pseudo-color images are used to capture detailed information about the illuminated object. This information includes, for example, basic structural information about the surface of tissues and underlying tissue layers, as seen by the two narrowband lights.

[0073] The pseudo-color image has enhanced contrast and depth resolution. Pseudo-color images are at the same scale as ordinary white light images, but with richer image information and greater differences. For example, the pseudo-color image herein can be a high-contrast pseudo-color image. The difference between the brightest and darkest areas in a high-contrast pseudo-color image is greater. This makes the high-contrast pseudo-color image more vivid, with distinct light and dark areas. Exemplarily, the contrast ratio of a high-contrast pseudo-color image is typically greater than 5, and can even reach 10 or higher.

[0074] Endoscopic technology is widely used in clinical screening and diagnosis, especially in the early detection of gastrointestinal and respiratory diseases. Narrowband imaging technology (NBI) relies on a specific wavelength light source to enhance tissue contrast and visualization. The current related technology uses narrowband wavelengths of 415nm (blue light) and 540nm (green light) to enhance blood vessel and mucosal details. However, the image contrast and depth resolution in the NBI imaging system of the related technology are insufficient: the NBI imaging method of the related technology uses a simple RGB channel allocation for image fusion, which cannot fully explore the different levels of information of the tissue, and the color contrast is not obvious enough.

[0075] In this article, the first narrowband light and the second narrowband light are generated by directly emitting from the light source or filtering out dual narrowband lights from white light for simultaneous illumination or switching illumination separately to generate two narrowband lights. Exemplarily, the narrowband light bandwidth is within 30nm, and the center wavelength difference is above 80nm. The specific embodiment adopts: 415nm (blue light) and 540nm (green light). In this way, the system projects narrowband light of 415nm (blue light) and 540nm (green light) wavelengths through the filter, while capturing the shallow and deep tissue features, using different light source characteristics to distinguish tissue layers and improve the visualization of lesions.

[0076] In contrast, when dual narrow-band light sources are used simultaneously, a color image processing sensor is used, and the image acquisition frame rate must be no less than 30 fps. When dual narrow-band light sources are used independently and in a time-sharing manner, a monochrome image processing sensor is used, and the image acquisition frame rate must be no less than 60 fps. Therefore, setting a numerical value for the image acquisition frame rate is essential to ensure accurate data collection.

[0077] In the embodiment of the present application, clinical imaging requires the system to maintain imaging quality at a high frame rate, which can achieve the best balance between clarity and operational smoothness, satisfying the balance between real-time performance and imaging efficiency.

[0078] Continue to combine Figure 1As shown, the fusion of the promotion and suppression mechanisms in this paper is used to process specific features of each of the two narrowband images to obtain suppressed image features and enhanced image features, and then these suppressed image features and enhanced image features are fused. The specific features are used to represent features that have different light absorption.

[0079] It should be noted that the aforementioned fusion of promotion and inhibition mechanisms may include, but is not limited to, the fusion of promotion and inhibition mechanisms of biomimetic vision. Specifically, the mutual promotion and inhibition mechanisms of biomimetic vision are fused based on at least any two of the two or more narrow-band images to generate a fused image.

[0080] The fusion of the promotion and inhibition mechanisms of biomimetic vision includes, for example, but is not limited to, the on-center and off-center receptive field models of the rattlesnake's visual system. Accordingly, based on the on-center and off-center receptive field models of the rattlesnake's visual system, image processing is performed on specific features of each of two or more narrow-band images to obtain suppressed image features and enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0081] To this end, each of the two narrowband images is fused using the facilitation and inhibition mechanisms of biomimetic vision to generate a new monochrome image (also called a fused image). This fully accounts for the interaction between the two narrowband images, enhancing contrast and depth resolution. This new monochrome image is then fed into one of the RGB channels for display, with the same process applied to the other color channels. See below for details.

[0082] The integration of the aforementioned promotion and inhibition mechanisms of biomimetic vision includes, but is not limited to, the positive and negative feedback mechanisms in biological perception and behavioral learning, which can be used to achieve the integration of promotion and inhibition mechanisms. This requires sample training and learning to obtain suppressed and enhanced image features, which are then integrated to produce a fused image.

[0083] It should also be noted that the fusion of the aforementioned promotion and inhibition mechanisms may also include, but is not limited to, the fusion of promotion and inhibition mechanisms based on a deep learning-based neural network model. Specifically, the deep learning-based neural network model includes an activation function for implementing promotion and an attention mechanism for implementing inhibition. The promotion and inhibition mechanisms in biological perception and behavioral learning can correspondingly achieve the fusion of promotion and inhibition mechanisms. In this way, sample training and learning are required to obtain suppressed image features and enhanced image features, which are then fused to obtain a fused image.

[0084] Figure 2 Shown Figure 1 Schematic diagram of the ON-center type in the endoscopic image fusion system shown. Figure 3 Shown Figure 1 Schematic diagram of the OFF-center type in the endoscopic image fusion system shown.

[0085] like Figure 2 and Figure 3 As shown, the shallow layer image may also be referred to as a shallow surface layer image B. The deep layer image may also be referred to as a deeper layer image G. The above-mentioned image processing module 13 may include but is not limited to image processing submodules, which are described in detail as follows.

[0086] The image processing submodule is used to: perform image processing on specific features of shallow images and specific features of deep images based on the ON-center and OFF-center receptive field models in the visual system of the rattlesnake, respectively, to obtain suppressed image features and enhanced image features.

[0087] In the present embodiment, to achieve effective fusion of these images, image processing is performed based on the on-center and off-center receptive field models in the rattlesnake's visual system. The main function of these models is to enhance feature contrast in the image while maintaining accurate resolution of tissue depth.

[0088] Combine Figure 1 and Figure 2 As shown, the above-mentioned image processing submodule may include but is not limited to a first image processing unit and a second image processing unit:

[0089] a first image processing unit, configured to apply ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G);

[0090] The second image processing unit is used to apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B), and to apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0091] In the embodiment of the present application, ON-center receptive field enhancement: ON-center processing is performed on blue light images (415nm) and green light images (540nm) to enhance the contrast of local tissue features and make tissue layers clearer; OFF-center receptive field suppression: OFF-center suppression is applied to blue light images to further extract deep tissue information and help users better identify key areas.

[0092] Combine Figure 1 and Figure 2 As shown, the above-mentioned image processing module 13 may include but is not limited to a first fusion image generation submodule and a second fusion image generation submodule; the pseudo-color image generation module includes: a first mapping submodule and a second mapping submodule;

[0093] The first fusion image generation submodule is used as a submodule to simulate the antagonistic receptive field of ganglion cells based on the Gaussian difference function model, and use the deep enhanced image features (+G), the shallow suppressed image features (-B) and the shallow enhanced image features (+B) to generate monochrome images accordingly; the first mapping submodule is used to map the corresponding monochrome image to the red channel and the blue channel of the RGB channel respectively; the Gaussian difference function model simulates the antagonistic receptive field of ganglion cells to realize the pseudo-color fusion of the bionic vision mechanism.

[0094] The second fusion image generation submodule is used to combine the promoting effect of the deep-layer enhanced image features (+G) on the shallow-layer suppressed image features (-B), and the suppressing effect of the shallow-layer enhanced image features (+B) on the deep-layer enhanced image features (+G), to generate a monochrome image; the second mapping submodule is used to map the monochrome image to the green channel in the RGB channel.

[0095] In an embodiment of the present application, dual narrowband images based on biological vision (Gaussian difference model) are used to promote and inhibit each other, and the promotion and inhibition results are mapped to RGB space. In this way, through the visual perception system derived from rattlesnakes, ON-center type (center) and OFF-center type (center) receptive field models are introduced, and the contrast and depth resolution are enhanced by the image promotion and inhibition mechanism. In addition, the pseudo-color presentation of multimodal images is achieved by mapping different channels of RGB space, combining different light wave imaging information to form images with high contrast and prominent layering, so as to enhance the visualization of mucosal and vascular structures.

[0096] Figure 4 Shown Figure 1 Schematic diagram of the application of the endoscopic image fusion system shown.

[0097] like Figure 4 As shown, the shallow layer image may also be referred to as a shallow surface layer image B. The deep layer image may also be referred to as a deeper layer image G.

[0098] Combine Figure 1 、 Figure 2 、 Figure 3 and Figure 4 As shown, the first fusion image generation submodule includes a first fusion image generation unit and a second fusion image generation unit; the first mapping submodule includes a first mapping unit and a second mapping unit;

[0099] a first fusion image generation unit, configured to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, using the deep layer enhanced image feature (+G) and the shallow layer suppressed image feature (-B) to achieve a promoting effect of the shallow layer image on the deep layer image, thereby generating a monochrome image; and a first mapping unit, configured to map the monochrome image to a red channel in an RGB channel.

[0100] The second fusion image generation unit is configured to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, using the deep layer enhanced image features (+G) and the shallow layer enhanced image features (+B) to suppress the shallow layer image by the deep layer image, thereby generating a monochrome image. The second mapping unit is configured to map the monochrome image to the blue channel of the RGB channel. The image formed by the +G+B combination can be referred to as a shallow layer-enhanced deep feature image.

[0101] In the embodiment of the present application, two narrowband images interact to generate an image that is directly mapped to one of the RGB channels. In this way, an image fusion method based on biomimetic vision mechanism is adopted to enhance the image contrast and depth resolution capability.

[0102] In biological vision, the passive membrane equation of the ON-center receptive field is a physical model that describes the dynamics of the center-surround inhibition field as follows:

[0103]

[0104] Among them, the pixel corresponding to the coordinate position (i, j) in an image, at the coordinate position (i, j), assumes that the cell activity of the ON-center is x i,j , t is the response time. Among them, C i,j and S i,j is the total excitatory input and inhibitory input of the input, which is the discrete convolution of the input image and the Gaussian kernel. A represents the decay rate, B and D are the maximum and minimum activation levels respectively. The dynamic process is not considered when fusion is performed, so only the equation when equilibrium is reached is studied, that is, dx i,j / dt=0, in image processing, B and D need to be set to 1, and this model needs to be revised to the ON-center system response for image fusion:

[0105]

[0106] Among them, x i,j represents the response of the ON-center system, a is the coefficient of the excitatory core, and b is the coefficient of the inhibitory core. i,j is the input excitatory information, S i,j is the inhibitory information of the input. The above formula cannot show the operation between the image and the input.

[0107] Similarly, the OFF-center receptive field is processed in a similar way, as follows:

[0108]

[0109] Among them, x i ′ ,j represents the response of the OFF-center receptive field, however, C i,j and S i,j The calculation of requires convolution of the image with the Gaussian function, as shown in the following formula (4) and formula (5), where I(i, j) represents the image, which can be G or B in this study. i,j and S i,j The two are convolved using the Gaussian function formula (4) and formula (5), with I(i, j) representing the center image or the surrounding image. Here, one of the center image and the surrounding image is selected as G or B, and the other as the surround.

[0110] in:

[0111]

[0112] Where I(i,j) is the input image, C1 and C2 are the Gaussian distribution functions of the central area and the surrounding area, m×n and p×q are the sizes of the template, σ1 and σ2 are constants related to the central area and the surrounding area, which determine the shape of the Gaussian function.

[0113] Formula (2) and Formula (3) can reflect the final result after the two images are interacted. Their denominators are the same, and the main difference is the numerator, that is, the difference between one image and the other. However, the difference between one image and the other results in the ON-center type (center) and the OFF-center type (center). The final intuitive expression can be simulated by the Gaussian difference function (Difference of Gaussian, DOG) model to simulate the antagonistic receptive field of the ganglion cell. Formula (6) is to more intuitively display these two forms of receptive field, but it can be considered that the actual calculation is involved in Formula (2) and Formula (3):

[0114]

[0115] Among them, DOG represents the Gaussian difference function, C1 and C2 are two-dimensional Gaussian kernel functions, and Gaussian distribution can better describe the activities of cells in different regions, and its sensitivity decreases from the center to the periphery.

[0116] Related image processing technology fails to effectively utilize the interaction of multi-wavelength narrow-band light, limiting the application effect of the system in observing complex tissue structures.

[0117] Compared with the related art, in the embodiment of the present application, the interaction between the two narrow-band lights, the enhanced image feature (+G) and the suppressed image feature (-B), can be fully utilized to make the tissue information integration more complete and enhance the application effect of the system in the observation of complex tissue structures.

[0118] Continue to combine Figure 4 As shown in the figure, the specific application implementation of the system is divided into three stages:

[0119] (1) Preprocessing stage:

[0120] First, the blue (B, 415nm) and green (G, 540nm) images are brightness-equalized to ensure consistency between the two images. This step ensures that subsequent image fusion does not cause unnecessary distortion due to brightness differences.

[0121] (2) Phase 1: Application of receptive field model:

[0122] In this stage, the ON-center receptive field model is applied to the blue and green light images to enhance tissue features. The specific steps are as follows:

[0123] ON-center receptive field processing is applied to the green light image (G) and the blue light image (B) to enhance their local specific features.

[0124] The OFF-center receptive field processing is applied to the blue light image (B) to suppress some unnecessary features and further extract the depth information.

[0125] (3) The second stage: image fusion stage:

[0126] In this stage, the pseudo-color image is fused based on the image processed in the previous stage:

[0127] Red channel: The boost of the blue light image to the green light image is enhanced by calculating (+G+B) and mapping it to the red channel.

[0128] Blue channel: The suppression effect of the green light image on the blue light image is reflected by calculating (+BG) and mapping it to the blue channel.

[0129] Green channel: Perform a merge (AND) operation on the (+GB) image and the (+BG) image, obtain the result of the AND operation, and map the result to the green channel.

[0130] Finally, after the above steps, a pseudo-color fused image with enhanced contrast and depth resolution is generated. Compared with the related NBI image fusion method, this method can better display the depth layer information of the tissue, especially in vascular visualization and lesion detection.

[0131] In this paper, an endoscopic image fusion system is designed using dual narrowband light (415nm and 540nm) and a pseudo-color fusion algorithm based on bionic vision mechanisms is introduced. This achieves the following technical effects:

[0132] 1. Significantly improve contrast and layer resolution: The pseudo-color fusion algorithm is based on image processing of dual narrow-band light wavelengths, which enhances the color contrast and depth of the image, providing clearer visualization effects, especially in the detection of early lesions and observation of complex tissues.

[0133] 2. Real-time and efficient imaging system: The system achieves high-quality imaging without sacrificing frame rate (≥30fps under simultaneous dual narrow-band light acquisition), ensuring smooth clinical operations.

[0134] 3. Enhanced presentation of tissue information: Pseudo-color fusion images fully integrate superficial and deep tissue information, increase the details of structures such as mucosa and blood vessels, and provide richer imaging support for clinical diagnosis.

[0135] In the second aspect, based on the same application concept as the device in the first aspect, the embodiment of the present application further provides an endoscopic image fusion method, such as Figure 5 As shown, the endoscopic image fusion method may include the following steps 110 to 140:

[0136] Step 110: The narrowband light acquisition module acquires first narrowband light of a first central wavelength and second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture tissue features of a shallow layer in the target tissue of the endoscope, and the second narrowband light is used to capture tissue features of a deep layer in the target tissue;

[0137] Step 120 , the image extraction module uses the first narrowband light and the second narrowband light to illuminate the target tissue respectively, and extracts shallow layer images and deep layer images at different depths;

[0138] Step 130: The image processing module integrates the mutual promotion and inhibition mechanisms of biomimetic vision based on the shallow layer images and the deep layer images at different depths to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different light absorption;

[0139] Step 140 : The pseudo-color image generation module combines color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

[0140] To ensure the practicality and scalability of the technical solution, the present application embodiment also recommends the following verification and optimization directions:

[0141] 1. Clinical effect verification: Conduct comparative experiments in real clinical applications to verify the imaging advantages of the algorithm in different lesion types, collect data such as detection accuracy and recognition efficiency, and use data to support the technical effects of the embodiments of this application.

[0142] 2. Spectral parameter optimization: According to different clinical needs, the combination parameters of dual wavelengths are optimized, and the imaging effects of different wavelengths are studied to improve the adaptability of the technology and image clarity.

[0143] 3. Modular design and cost control: Based on system flexibility and cost-effectiveness, the dual narrowband light source and image processing module 13 are optimized to ensure that the modular design of the system is easy to replace and expand, so as to promote its application in different medical scenarios.

[0144] The endoscopic image fusion method provided in the embodiment of the present application has the same inventive concept as the above-mentioned endoscopic image fusion system, and can achieve the same or similar technical effects, so examples will not be given one by one here.

[0145] In a third aspect, based on the same application concept as the device in the first aspect, the present application embodiment further provides an image fusion method, such as Figure 6 As shown, the image fusion method may include the following steps 210 to 240:

[0146] Step 210: Acquire a first narrowband light having a first central wavelength and a second narrowband light having a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture shallow features of the object under inspection, and the second narrowband light is used to capture deep features of the object under inspection.

[0147] In step 220 , the first narrowband light and the second narrowband light are used to illuminate the subject respectively, and shallow layer images and deep layer images at different depths are extracted.

[0148] Step 230, based on the shallow images and deep images of different depths, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different light absorption.

[0149] Step 240 : performing color channel combination based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination.

[0150] As an embodiment, the above step 230 may further include but is not limited to: based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, image processing is performed on specific features of the shallow image and specific features of the deep image respectively to obtain suppressed image features and enhanced image features.

[0151] As an embodiment, based on the ON-center and OFF-center receptive field models in the visual system of a rattlesnake, image processing is performed on specific features of shallow images and specific features of deep images, respectively, to obtain suppressed image features and enhanced image features, including:

[0152] Applying ON-center receptive field processing to the deep image to enhance specific features of the deep image to obtain deep enhanced image features (+G);

[0153] Applying OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B); and applying ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0154] As an embodiment, based on shallow images and deep images of different depths, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated to obtain suppressed image features and enhanced image features, including:

[0155] Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated. The deep enhanced image features (+G), shallow suppressed image features (-B), and shallow enhanced image features (+B) are used to generate monochrome images. The corresponding monochrome images are mapped to the red channel and blue channel of the RGB channel respectively.

[0156] The promoting effect of the deep-layer enhanced image features (+G) on the shallow-layer suppressed image features (-B) and the suppressing effect of the shallow-layer enhanced image features (+B) on the deep-layer enhanced image features (+G) are merged to generate a monochrome image; and the monochrome image is mapped to the green channel in the RGB channel.

[0157] As an embodiment, based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the deep enhanced image features (+G), the shallow suppressed image features (-B), and the shallow enhanced image features (+B) are used to generate monochrome images accordingly; and the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively, including:

[0158] Simulating the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, using the deep layer enhanced image feature (+G) and the shallow layer suppressed image feature (-B) to achieve the effect of the shallow layer image promoting the deep layer image, thereby generating a monochrome image; the first mapping unit is used to map the monochrome image to the red channel of the RGB channel; and mapping the monochrome image to the red channel of the RGB channel;

[0159] The antagonistic receptive field of ganglion cells is simulated based on a Gaussian difference function model, and the deep-layer enhanced image features (+G) and the shallow-layer enhanced image features (+B) are used to achieve the inhibitory effect of the deep-layer image on the shallow-layer image, thereby generating a monochrome image; and mapping the monochrome image to the blue channel in the RGB channel.

[0160] The image fusion method provided in the embodiment of the present application has the same inventive concept as the above-mentioned endoscopic image fusion system, and can achieve the same or similar technical effects, so examples will not be given one by one here.

[0161] Fourthly, based on the same application concept as the above-mentioned image fusion method, continue to combine Figure 1 As shown, the embodiment of the present application further provides an image fusion device for implementing the above image fusion method, which may include the following narrow-band light acquisition module 11, image extraction module 12, image processing module 13 and pseudo-color image generation module 14:

[0162] The narrowband light acquisition module 11 is configured to acquire a first narrowband light having a first central wavelength and a second narrowband light having a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaching a preset difference; the first narrowband light being configured to capture shallow features of the object under inspection and the second narrowband light being configured to capture deep features of the object under inspection;

[0163] An image extraction module 12 is configured to illuminate a subject using the first narrowband light and the second narrowband light respectively, and extract shallow layer images and deep layer images at different depths;

[0164] An image processing module 13 is configured to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different light absorption;

[0165] The pseudo-color image generating module 14 is configured to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination.

[0166] The image fusion device provided in the embodiment of the present application has the same inventive concept as the above-mentioned image fusion method and can achieve the same or similar technical effects, so examples will not be given one by one here.

[0167] An embodiment of the present application provides an electronic device, including an image fusion device or an endoscope image fusion system.

[0168] The methods provided in the embodiments of the present invention can be applied to electronic devices. Specifically, the electronic devices can include desktop computers, portable computers, smart mobile terminals, servers, and handheld terminals. Any electronic device that can implement the embodiments of the present invention falls within the scope of protection of the present invention and is not limited herein.

[0169] The endoscopic imaging method and system, and the imaging method and device in this article all have the same inventive concept, and the same or similar steps can achieve the same effect.

[0170] Figure 7 Shown is a structural diagram of an electronic device 30 provided in an embodiment of the present application.

[0171] like Figure 7 As shown, the electronic device 30 includes one or more processors 31 for implementing the above image fusion method.

[0172] In some embodiments, the electronic device 30 may include a storage medium 39. For example, the computer-readable storage medium may store a program that can be called by the processor 31, and may include a non-volatile storage medium. In some embodiments, the electronic device 30 may include a memory 38 and an interface 37. In some embodiments, the electronic device 30 may also include other hardware depending on the actual application.

[0173] The computer-readable storage medium of the embodiment of the present application stores a program thereon, and when the program is executed by the processor 31, it is used to implement the image fusion method described above.

[0174] The present application provides a computer program product, comprising a computer program / instruction, which implements any of the above methods when executed by a processor.

[0175] The present application also provides a computer program, which is stored in a computer-readable storage medium, for example, Figure 7 and when the processor executes the computer program, it causes the processor 31 to execute the method described above.

[0176] The present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. Information may be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer-readable storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0177] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0178] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, the phrase "comprises a ..." defining an element does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

Claims

1. An endoscopic image fusion system, characterized in that: include: a narrowband light acquisition module, configured to acquire first narrowband light of a first central wavelength and second narrowband light of a second central wavelength; The difference in central wavelengths between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture tissue features of a shallow layer in a target tissue of an endoscope, and the second narrowband light is used to capture tissue features of a deep layer in the target tissue; an image extraction module, configured to respectively illuminate the target tissue using the first narrowband light and the second narrowband light, and extract shallow layer images and deep layer images at different depths; An image processing module is configured to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and enhanced image features both include specific features; the specific features are used to represent features with different light absorption; a pseudo-color image generation module, configured to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue; The image processing module includes an image processing submodule; The image processing submodule is used to: perform image processing on the specific features of the shallow image and the specific features of the deep image based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, respectively, to obtain suppressed image features and enhanced image features; The image processing submodule includes a first image processing unit and a second image processing unit: The first image processing unit is configured to apply ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G); The second image processing unit is used to apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B), and to apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

2. The endoscopic image fusion system according to claim 1, wherein: The image processing module includes a first fusion image generation submodule and a second fusion image generation submodule; the pseudo-color image generation module includes a first mapping submodule and a second mapping submodule; The first fusion image generation submodule is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, and use the deep layer enhanced image features (+G), the shallow layer suppressed image features (-B) and the shallow layer enhanced image features (+B) to generate monochrome images accordingly; the first mapping submodule is used to map the corresponding monochrome images to the red channel and the blue channel of the RGB channel respectively; The second fusion image generation submodule is used to combine the promoting effect of the deep-layer enhanced image features (+G) on the shallow-layer suppressed image features (-B), and the suppressing effect of the shallow-layer enhanced image features (+B) on the deep-layer enhanced image features (+G), to generate a monochrome image; the second mapping submodule is used to map the monochrome image to the green channel in the RGB channel.

3. The endoscopic image fusion system according to claim 2, wherein: The first fused image generation submodule includes a first fused image generation unit and a second fused image generation unit; the first mapping submodule includes a first mapping unit and a second mapping unit; The first fusion image generation unit is configured to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the deep layer enhanced image feature (+G) and the shallow layer suppressed image feature (-B) to achieve the effect of the shallow layer image promoting the deep layer image, and generate a monochrome image; the first mapping unit is configured to map the monochrome image to a red channel in an RGB channel; The second fusion image generating unit is configured to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the deep layer enhanced image features (+G) and the shallow layer enhanced image features (+B) to achieve the inhibitory effect of the deep layer image on the shallow layer image, and generate a monochrome image; The second mapping unit is configured to map the monochrome image to a blue channel in an RGB channel.

4. An endoscopic image fusion method, characterized in that: include: The narrowband light acquisition module acquires a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; The difference in central wavelengths between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture tissue features of a shallow layer in a target tissue of an endoscope, and the second narrowband light is used to capture tissue features of a deep layer in the target tissue; The image extraction module uses the first narrowband light and the second narrowband light to illuminate the target tissue respectively, and extracts shallow layer images and deep layer images at different depths; The image processing module integrates the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light; wherein, the image processing module includes an image processing submodule; the image processing submodule is used to: based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, perform image processing on the specific features of the shallow image and the specific features of the deep image respectively to obtain suppressed image features and enhanced image features; the image processing submodule includes a first image processing unit and a second image processing unit: the first image processing unit is used to apply ON-center receptive field processing to the deep image, enhance specific features of the deep image, and obtain deep-enhanced image features (+G); the second image processing unit is used to apply OFF-center receptive field processing to the shallow image, suppress specific features of the shallow image, and obtain shallow-suppressed image features (-B), and apply ON-center receptive field processing to the shallow image, enhance specific features of the shallow image, and obtain shallow-enhanced image features (+B); The pseudo-color image generation module combines color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

5. An image fusion method, characterized in that: include: Acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; The difference in central wavelengths between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture features of a shallow layer of the object under inspection, and the second narrowband light is used to capture features of a deep layer of the object under inspection; irradiating the subject with the first narrowband light and the second narrowband light respectively, and extracting shallow layer images and deep layer images at different depths; Based on shallow and deep images at different depths, a fusion of mutual promotion and inhibition mechanisms of biomimetic vision is performed to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different light absorption; Combining color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination; The shallow layer images and deep layer images at different depths are used to fuse the mutual promotion and inhibition mechanisms of biomimetic vision to obtain suppressed image features and enhanced image features, including: Based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, image processing is performed on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features; The ON-center type and OFF-center type receptive field models in the rattlesnake-based visual system respectively perform image processing on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features, including: Applying ON-center receptive field processing to the deep image to enhance specific features of the deep image to obtain deep enhanced image features (+G); Applying OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B); and applying ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

6. The image fusion method according to claim 5, wherein: The shallow layer images and deep layer images at different depths are used to fuse the mutual promotion and inhibition mechanisms of biomimetic vision to obtain suppressed image features and enhanced image features, including: Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated. The deep enhanced image features (+G), the shallow suppressed image features (-B), and the shallow enhanced image features (+B) are used to generate monochrome images. The corresponding monochrome images are mapped to the red channel and blue channel of the RGB channel respectively. The promoting effect of the deep-layer enhanced image features (+G) on the shallow-layer suppressed image features (-B) and the suppressing effect of the shallow-layer enhanced image features (+B) on the deep-layer enhanced image features (+G) are merged to generate a monochrome image; and the monochrome image is mapped to the green channel in the RGB channel.

7. The image fusion method according to claim 6, wherein: The Gaussian difference function model is based on simulating the antagonistic receptive field of ganglion cells, and uses the deep layer enhanced image features (+G), the shallow layer suppressed image features (-B) and the shallow layer enhanced image features (+B) to generate monochrome images accordingly; Map the corresponding monochrome image to the red channel and blue channel of the RGB channel respectively, including: Simulating the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, using the deep layer enhanced image feature (+G) and the shallow layer suppressed image feature (-B) to achieve the effect of the shallow layer image promoting the deep layer image, thereby generating a monochrome image; a first mapping unit, configured to map the monochrome image to a red channel in an RGB channel; and mapping the monochrome image to a red channel in an RGB channel; The antagonistic receptive field of ganglion cells is simulated based on a Gaussian difference function model, and the deep-layer enhanced image features (+G) and the shallow-layer enhanced image features (+B) are used to achieve the inhibitory effect of the deep-layer image on the shallow-layer image, thereby generating a monochrome image; and mapping the monochrome image to the blue channel in the RGB channel.

8. An image fusion device, characterized in that: For implementing the image fusion method according to any one of claims 5 to 7, the image fusion device comprises: a narrowband light acquisition module, configured to acquire first narrowband light of a first central wavelength and second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaching a preset difference; the first narrowband light being used to capture shallow features of an object under inspection, and the second narrowband light being used to capture deep features of the object under inspection; an image extraction module, configured to respectively illuminate a subject using the first narrowband light and the second narrowband light to extract shallow layer images and deep layer images at different depths; An image processing module is configured to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and enhanced image features both include specific features; the specific features are used to represent features with different light absorption; The pseudo-color image generation module is used to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under inspection.

9. An electronic device, characterized in that: The method comprises one or more processors, and is configured to implement the image fusion method according to any one of claims 5 to 7.

Images