Endoscope image fusion system and method, image fusion method and device and equipment

Abstract

The invention provides an endoscope image fusion system and method, an image fusion method and device and equipment. The system comprises a narrow-band light acquisition module, an image extraction module, an image processing module and a pseudo-color image generation module. The narrow-band light acquisition module is used for acquiring first narrow-band light and second narrow-band light of which the central wavelength difference reaches a preset difference; the image extraction module is used for irradiating the target tissue by using the first narrow-band light and the second narrow-band light respectively and extracting shallow layer images and deep layer images with different depths; the image processing module carries out mutual promotion of biological bionic vision and fusion of an inhibition mechanism based on shallow layer images and deep layer images of different depths to obtain inhibited image features and enhanced image features; the pseudo-color image generation module is used for performing color channel combination based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used for representing tissue characteristics of the target tissue.

Description

Technical Field

[0001] The present invention relates to the field of image fusion technology, and in particular to an endoscope image fusion system and method, an image fusion method and device, and equipment. Background Art

[0002] The narrow band imaging technology (NBI) of the related art includes a narrow band light wavelength of 415nm and a narrow band light wavelength of 540nm. The image of the narrow band light wavelength of 540nm is placed in the R color channel of the RGB (Red Green Blue) channel, and the image of the narrow band light wavelength of 415nm is placed in the G color channel and the B color channel of the RGB channel, and then the image is displayed. The color of the displayed image is relatively monotonous and lacks sufficient depth distinction capability. Summary of the invention

[0003] The present application provides an improved endoscopic image fusion system and method, image fusion method and device, and equipment.

[0004] The present application provides an endoscopic image fusion system, comprising:

[0005] A narrowband light acquisition module, used to acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the tissue features of a shallow layer in a target tissue of an endoscope and the second narrowband light is used to capture the tissue features of a deep layer in the target tissue;

[0006] An image extraction module, configured to use the first narrow-band light and the second narrow-band light to irradiate the target tissue respectively, and extract shallow images and deep images at different depths;

[0007] An image processing module, for fusing the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths, to obtain inhibited image features and enhanced image features; the inhibited image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light;

[0008] The pseudo-color image generation module is used to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

[0009] Furthermore, the image processing module includes an image processing submodule;

[0010] The image processing submodule is used to: based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, perform image processing on the specific features of the shallow image and the specific features of the deep image respectively to obtain suppressed image features and enhanced image features.

[0011] Furthermore, the image processing submodule includes a first image processing unit and a second image processing unit:

[0012] The first image processing unit is used to apply ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G);

[0013] The second image processing unit is used to apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B), and to apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0014] Further, the image processing module includes a first fused image generation submodule and a second fused image generation submodule; the pseudo-color image generation module includes a first mapping submodule and a second mapping submodule;

[0015] The first fusion image generation submodule is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, and use the deep layer enhanced image features (+G), the shallow layer suppressed image features (-B) and the shallow layer enhanced image features (+B) to generate monochrome images accordingly; the first mapping submodule is used to map the corresponding monochrome images to the red channel and the blue channel of the RGB channel respectively;

[0016] The second fused image generation submodule is used to combine the promoting effect of the deep enhanced image features (+G) on the shallow suppressed image features (-B), and the suppressing effect of the shallow enhanced image features (+B) on the deep enhanced image features (+G), to generate a monochrome image; the second mapping submodule is used to map the monochrome image to the green channel in the RGB channel.

[0017] Further, the first fused image generation submodule includes a first fused image generation unit and a second fused image generation unit; the first mapping submodule includes a first mapping unit and a second mapping unit;

[0018] The first fusion image generating unit is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the image features of the deep layer enhancement (+G) and the image features of the shallow layer suppression (-B) to achieve the promotion effect of the shallow layer image on the deep layer image, and generate a monochrome image; the first mapping unit is used to map the monochrome image to a red channel in the RGB channel;

[0019] The second fusion image generating unit is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the deep enhanced image features (+G) and the shallow enhanced image features (+B) to achieve the inhibitory effect of the deep image on the shallow image, and generate a monochrome image; the second mapping unit is used to map the monochrome image to a blue channel in the RGB channel.

[0020] The present application provides an endoscopic image fusion method, comprising:

[0021] The narrowband light acquisition module acquires a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the tissue features of the shallow layer in the target tissue of the endoscope and the second narrowband light is used to capture the tissue features of the deep layer in the target tissue;

[0022] The image extraction module uses the first narrow-band light and the second narrow-band light to illuminate the target tissue respectively, and extracts shallow images and deep images at different depths;

[0023] The image processing module integrates the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths to obtain inhibited image features and enhanced image features; the inhibited image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light;

[0024] The pseudo-color image generation module combines color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

[0025] The present application provides an image fusion method, including:

[0026] Acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the shallow characteristics of the object under inspection and the second narrowband light is used to capture the deep characteristics of the object under inspection;

[0027] Using the first narrowband light and the second narrowband light to illuminate the subject respectively, and extracting shallow layer images and deep layer images at different depths;

[0028] Based on shallow images and deep images of different depths, mutual promotion and inhibition mechanisms of biomimetic vision are integrated to obtain inhibited image features and enhanced image features; both the inhibited image features and the enhanced image features include specific features; the specific features are used to represent features with different absorption of light;

[0029] Color channels are combined based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination.

[0030] Furthermore, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated based on shallow images and deep images of different depths to obtain suppressed image features and enhanced image features, including:

[0031] Based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, image processing is performed on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features.

[0032] Furthermore, the ON-center type and OFF-center type receptive field models in the rattlesnake-based visual system respectively perform image processing on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features, including:

[0033] Applying ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G);

[0034] Apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B); and apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0035] Furthermore, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated based on shallow images and deep images of different depths to obtain suppressed image features and enhanced image features, including:

[0036] Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the deep enhanced image features (+G), the shallow suppressed image features (-B) and the shallow enhanced image features (+B) are used to generate monochrome images accordingly; the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively;

[0037] The promoting effect of the deep enhanced image features (+G) on the shallow suppressed image features (-B), and the suppressing effect of the shallow enhanced image features (+B) on the deep enhanced image features (+G) are combined to generate a monochrome image; and the monochrome image is mapped to the green channel in the RGB channel.

[0038] Furthermore, the Gaussian difference function model is used to simulate the antagonistic receptive field of ganglion cells, and the deep enhanced image features (+G), the shallow suppressed image features (-B) and the shallow enhanced image features (+B) are used to generate monochrome images accordingly; and the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively, including:

[0039] Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the image features of the deep layer enhancement (+G) and the image features of the shallow layer suppression (-B) are used to achieve the promotion effect of the shallow layer image on the deep layer image, so as to generate a monochrome image; the first mapping unit is used to map the monochrome image to the red channel in the RGB channel; and the monochrome image is mapped to the red channel in the RGB channel;

[0040] The antagonistic receptive field of ganglion cells is simulated based on a Gaussian difference function model, and the deep enhanced image features (+G) and the shallow enhanced image features (+B) are used to achieve the inhibitory effect of the deep image on the shallow image to generate a monochrome image; and the monochrome image is mapped to the blue channel in the RGB channel.

[0041] The present application provides an image fusion device for implementing the above-mentioned image fusion method, the image fusion device comprising:

[0042] A narrowband light acquisition module, used to acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the shallow layer characteristics of the object under inspection and the second narrowband light is used to capture the deep layer characteristics of the object under inspection;

[0043] An image extraction module, configured to use the first narrowband light and the second narrowband light to illuminate the subject respectively, and extract shallow images and deep images at different depths;

[0044] An image processing module, for fusing the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths, to obtain inhibited image features and enhanced image features; the inhibited image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light;

[0045] The pseudo-color image generation module is used to combine color channels based on suppressed image features and enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under inspection.

[0046] The present application provides an electronic device, comprising one or more processors, for implementing the image fusion method as described in any one of the above items.

[0047] The present application provides a computer-readable storage medium having a program stored thereon. When the program is executed by a processor, the method described in any one of the above items is implemented.

[0048] The present application provides a computer program product, comprising a computer program / instruction, which implements any of the above methods when executed by a processor.

[0049] In some embodiments, the pseudo-color image generated by the endoscope image fusion system of the present application has enhanced contrast and depth resolution, can enhance the depth resolution of the image, enhance the color contrast and depth level of the image, make the color expression richer, and have sufficient contrast (depth differentiation ability). This makes the depth and color differentiation more obvious, thereby providing a clearer visualization effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure shows a module schematic diagram of an endoscope image fusion system provided in an embodiment of the application;

[0051] Figure 2 Shown Figure 1 A schematic diagram of an ON-center type in an endoscopic image fusion system is shown;

[0052] Figure 3 Shown Figure 1 A schematic diagram of an OFF-center type in an endoscopic image fusion system is shown;

[0053] Figure 4 Shown Figure 1 A schematic diagram of an endoscope image fusion system application is shown;

[0054] Figure 5 The figure is a schematic diagram of the process of the endoscopic image fusion method provided in the embodiment of the present application;

[0055] Figure 6 The figure is a schematic diagram of the process of the image fusion method provided in the embodiment of the present application;

[0056] Figure 7 Shown is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of the present specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of the present specification as detailed in the appended claims.

[0058] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0059] In order to solve the technical problem that the color of the above-mentioned displayed image is relatively monotonous and lacks sufficient contrast, an endoscope image fusion system is provided in an embodiment of the present application, including a narrowband light acquisition module, an image extraction module, an image processing module and a pseudo-color image generation module. Among them, the narrowband light acquisition module acquires a first narrowband light for capturing shallow features and a second narrowband light for capturing deep features, and extracts shallow images and deep images of different depths through the image extraction module; the image processing module is used to fuse the mutual promotion and inhibition mechanism of biomimetic vision based on shallow images and deep images of different depths to obtain suppressed image features and enhanced image features; finally, the pseudo-color image generation module is used to combine color channels based on the suppressed image features and enhanced image features to generate a pseudo-color image.

[0060] In the embodiment of the present application, the pseudo-color image has enhanced contrast and depth resolution, can enhance the depth resolution of the image, enhance the color contrast and depth level of the image, make the color performance richer, and have sufficient contrast (depth differentiation ability). In this way, the depth and color differentiation are more obvious, thereby providing a clearer visualization effect.

[0061] With the rapid development of endoscopic technology, narrow-band imaging (NBI) has become a common imaging method in clinical practice, playing an important role in the early screening and lesion detection of digestive and respiratory diseases. Current endoscopic staining technology usually uses narrow-band light wavelengths corresponding to the hemoglobin absorption peak, specifically 415nm and 540nm. By enhancing the visualization of blood vessels and mucosal tissues with these wavelengths, doctors can more clearly observe vascular structures and possible lesions. However, the related art narrow-band imaging technology NBI technology has great limitations in image processing.

[0062] Most current narrow-band imaging technology NBI imaging systems use a simple image fusion method, which is to directly assign different narrow-band light images to the RGB color channel. Although this processing method can enhance the observation of tissue structure to a certain extent, the color of the image is relatively monotonous and lacks sufficient contrast. In addition, this method is insufficient in depth resolution and it is difficult to accurately distinguish the structures of different levels of tissues, which limits its application effect in complex tissues and early lesions.

[0063] In this context, in order to solve the technical problems that the above-mentioned images have relatively monotonous color performance, lack of sufficient contrast, and insufficient depth resolution, making it difficult to accurately distinguish the structures of different levels of tissues, limiting their application effect in complex tissues and early lesions, the embodiment of this application provides an endoscopic image fusion system, and develops a new image fusion method that can fully utilize the interaction between narrow-band light images to improve the color contrast and depth resolution of images, which has become the focus of current endoscopic imaging technology research. At the same time, based on bionic visual inspiration, a new image processing method is proposed to improve the overall performance of the narrow-band light imaging system, provide higher quality image support for clinical diagnosis, and break through the limitations of narrow-band light imaging technology in related technologies.

[0064] Figure 1 Shown is a module schematic diagram of an endoscopic image fusion system provided in an embodiment of the application.

[0065] First, as Figure 1 As shown, the endoscope image fusion system may include but is not limited to the following narrow-band light acquisition module 11, image extraction module 12, image processing module 13 and pseudo-color image generation module 14:

[0066] The narrowband light acquisition module 11 is used to acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the tissue characteristics of the shallow layer in the target tissue of the endoscope and the second narrowband light is used to capture the tissue characteristics of the deep layer in the target tissue. The target tissue is the tissue of the subject to be examined.

[0067] The central wavelengths of the first narrowband light and the second narrowband light in this article are different and have certain differences. In this way, the subsequent fusion of the mutual promotion and inhibition mechanisms of biomimetic vision has more fusion significance. For example, the preset difference can be used to indicate that these narrowband images are quite different. In this way, the subsequent fusion of the promotion and inhibition mechanisms has more fusion significance.

[0068] The preset difference in this article may be, but is not limited to, greater than or equal to 50 nm (nanometers). In order to have a sufficiently large information gap, the fusion is more effective. The preset difference may be, but is not limited to, greater than or equal to 80 nm (nanometers).

[0069] The image extraction module 12 is used to use the first narrowband light and the second narrowband light to illuminate the target tissue respectively, and extract shallow images and deep images of different depths. The shallow images and the deep images have different image depths. The deep tissue features of the deep image have a deeper image depth than the shallow tissue features of the shallow image. These objects illuminated by the two narrowband lights can be called inspected objects.

[0070] The image processing module 13 is used to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light.

[0071] The pseudo-color image generation module 14 is used to generate a pseudo-color image by combining color channels based on the suppressed image features and the enhanced image features; the pseudo-color image is used to represent the tissue features of the target tissue.

[0072] The color image mentioned in this article is not directly generated by actual RGB, but a mapped color image, which is called a pseudo-color image. The pseudo-color image is used to grasp the details of the illuminated object. The details include the basic structural information of the surface of the tissue, deep tissue, etc. of the objects illuminated by two narrow-band lights.

[0073] The pseudo color image has enhanced contrast and depth resolution. The pseudo color image has the same scale level as the ordinary white light image, and the image information is richer and the difference is greater. For example, the pseudo color image of this article can be a high-contrast pseudo color image. The difference between the brightest and darkest areas in the high-contrast pseudo color image is relatively large. In this way, the high-contrast pseudo color image is more vivid and the light and dark are distinct. Exemplary, the contrast ratio of the high-contrast pseudo color image is usually greater than 5, and can even reach 10 or higher.

[0074] Endoscopic technology is widely used in clinical screening and diagnosis, especially in the early detection of digestive and respiratory diseases. Narrowband imaging technology NBI relies on a specific wavelength light source to enhance tissue contrast and visualization. The current related technology uses narrowband wavelengths of 415nm (blue light) and 540nm (green light) to enhance blood vessel and mucosal details. However, the image contrast and depth resolution in the NBI imaging system of the related technology are insufficient: the NBI imaging method of the related technology uses a simple RGB channel allocation for image fusion, which cannot fully mine the different levels of information of the tissue, and the color contrast is not obvious enough.

[0075] In this article, the first narrowband light and the second narrowband light are generated by directly emitting from the light source or filtering out dual narrowband lights from white light for simultaneous illumination or switching illumination separately to generate two narrowband lights. Exemplarily, the bandwidth of the narrowband light is within 30nm, and the difference in center wavelength is above 80nm. The specific embodiment adopts: 415nm (blue light) and 540nm (green light). In this way, the system projects narrowband light with wavelengths of 415nm (blue light) and 540nm (green light) through filters, while capturing the characteristics of shallow and deep tissues, using different light source characteristics to distinguish tissue levels and improve the visualization of lesions.

[0076] In contrast, when dual narrow-band lights are illuminated simultaneously, a color image processing sensor is used, and the image acquisition frame rate is not less than 30fps. When dual narrow-band lights are individually switched in time, a monochrome image processing sensor is used, and the image acquisition frame rate is not less than 60fps. In this way, it is necessary to set the numerical value of the image acquisition frame rate to ensure accurate data collection.

[0077] In the embodiment of the present application, clinical imaging requires the system to maintain imaging quality at a high frame rate, which can achieve an optimal balance between clarity and operational smoothness, satisfying the balance between real-time performance and imaging efficiency.

[0078] Continue to combine Figure 1As shown, the fusion of the promotion and suppression mechanisms in this article is used to process the specific features of each of the two narrow-band images, obtain suppressed image features and enhanced image features, and fuse these suppressed image features and enhanced image features. The specific features are used to represent features with different absorption of light.

[0079] It should be noted that the fusion of the above-mentioned promotion and inhibition mechanisms may include but is not limited to the fusion of the promotion and inhibition mechanisms of biomimetic vision. Specifically, based on at least any two of the two or more narrow-band images, the mutual promotion and inhibition mechanisms of biomimetic vision are fused to generate a fused image.

[0080] Among them, the fusion of the promotion and inhibition mechanisms of biomimetic vision includes, for example, but is not limited to: ON-center (center) (center) and OFF-center (center) receptive field models in the visual system of rattlesnakes. Accordingly, based on the ON-center (center) and OFF-center (center) receptive field models in the visual system of rattlesnakes, image processing is performed on specific features of each of two or more narrow-band images to obtain suppressed image features and enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0081] In this regard, each two narrow-band images are fused through the promotion and inhibition mechanism of bionic vision to generate a new monochrome image (also called a fused image). In this way, the interaction between each two narrow-band images is fully considered to enhance the contrast and depth resolution. And, the generated new monochrome image is sent to one of the RGB channels for display, and the other color channels are operated in the same way. See below for detailed description.

[0082] The fusion of the promotion and inhibition mechanisms of the above-mentioned biomimetic vision includes, but is not limited to, the positive feedback and negative feedback mechanisms in biological perception and behavior learning, and the corresponding fusion of the promotion and inhibition mechanisms is realized. In this way, sample training and learning are required to obtain suppressed image features and enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0083] It should also be noted that the fusion of the above-mentioned promotion and inhibition mechanisms may also include but is not limited to the fusion of the promotion and inhibition mechanisms of the neural network model based on deep learning. Among them, the neural network model based on deep learning includes an activation function for realizing promotion and an attention mechanism for realizing inhibition. The promotion and inhibition mechanisms in biological perception and behavioral learning correspondingly realize the fusion of promotion and inhibition mechanisms. In this way, sample training and learning are required to obtain suppressed image features and enhanced image features, and these suppressed image features and enhanced image features are fused to obtain a fused image.

[0084] Figure 2 Shown Figure 1 A schematic diagram of an ON-center type endoscopic image fusion system is shown. Figure 3 Shown Figure 1 A schematic diagram of an OFF-center type in an endoscopic image fusion system is shown.

[0085] like Figure 2 and Figure 3 As shown, the shallow layer image may also be referred to as a shallow surface layer image B. The deep layer image may also be referred to as a deeper layer image G. The above-mentioned image processing module 13 may include but is not limited to image processing submodules, which are described in detail as follows.

[0086] The image processing submodule is used to: based on the ON-center and OFF-center receptive field models in the visual system of the rattlesnake, perform image processing on specific features of shallow images and specific features of deep images respectively to obtain suppressed image features and enhanced image features.

[0087] In the embodiment of the present application, in order to achieve effective fusion of these images, image processing is performed based on the ON-center and OFF-center receptive field models in the visual system of rattlesnakes. The main function of these models is to enhance the feature contrast in the image while maintaining accurate resolution of tissue depth.

[0088] Combination Figure 1 and Figure 2 As shown, the above-mentioned image processing submodule may include but is not limited to a first image processing unit and a second image processing unit:

[0089] A first image processing unit, configured to apply ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G);

[0090] The second image processing unit is used to apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B), and to apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0091] In the embodiment of the present application, ON-center receptive field enhancement: ON-center processing is performed on blue light images (415nm) and green light images (540nm) to enhance the contrast of local tissue features and make the tissue hierarchy clearer; OFF-center receptive field suppression: OFF-center suppression is applied to blue light images to further extract deep tissue information and help users better identify key areas.

[0092] Combination Figure 1 and Figure 2 As shown, the above-mentioned image processing module 13 may include but is not limited to a first fusion image generation submodule and a second fusion image generation submodule; the pseudo-color image generation module includes: a first mapping submodule and a second mapping submodule;

[0093] The first fusion image generation submodule is used for a submodule, which is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, and uses deep enhanced image features (+G), shallow suppressed image features (-B) and shallow enhanced image features (+B) to generate monochrome images accordingly; the first mapping submodule is used to map the corresponding monochrome image to the red channel and blue channel of the RGB channel respectively; the Gaussian difference function model simulates the antagonistic receptive field of ganglion cells to realize the pseudo-color fusion of the bionic vision mechanism.

[0094] The second fused image generation submodule is used to combine the promoting effect of the deep enhanced image features (+G) on the shallow suppressed image features (-B), and the suppressing effect of the shallow enhanced image features (+B) on the deep enhanced image features (+G), to generate a monochrome image; the second mapping submodule is used to map the monochrome image to the green channel in the RGB channel.

[0095] In an embodiment of the present application, dual narrow-band images based on biological vision (Gaussian difference model) are used to promote and inhibit each other, and the promotion and inhibition results are mapped to RGB space. In this way, through the visual perception system derived from rattlesnakes, ON-center and OFF-center receptive field models are introduced, and contrast and depth resolution are enhanced through image promotion and inhibition mechanisms. In addition, pseudo-color presentation of multimodal images is achieved through different channel mappings in RGB space, and imaging information of different light waves is combined to form images with high contrast and prominent sense of hierarchy to enhance the visualization of mucosal and vascular structures.

[0096] Figure 4 Shown Figure 1 Schematic diagram of the application of the endoscopic image fusion system shown.

[0097] like Figure 4 As shown, the shallow layer image may also be referred to as a shallow layer image B. The deep layer image may also be referred to as a deeper layer image G.

[0098] Combination Figure 1 , Figure 2 , Figure 3 and Figure 4 As shown, the first fused image generation submodule includes a first fused image generation unit and a second fused image generation unit; the first mapping submodule includes a first mapping unit and a second mapping unit;

[0099] The first fusion image generating unit is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the image features of the deep layer enhancement (+G) and the image features of the shallow layer suppression (-B) to achieve the promotion effect of the shallow layer image on the deep layer image, and generate a monochrome image; the first mapping unit is used to map the monochrome image to a red channel in the RGB channel;

[0100] The second fusion image generation unit is used to simulate the antagonistic receptive field of ganglion cells based on the Gaussian difference function model, and use the deep enhanced image features (+G) and the shallow enhanced image features (+B) to achieve the inhibitory effect of the deep image on the shallow image to generate a monochrome image; the second mapping unit is used to map the monochrome image to the blue channel in the RGB channel. The image formed by +G+B can be called a shallow layer promoting deep feature image.

[0101] In the embodiment of the present application, two narrow-band images interact with each other to generate an image that is directly mapped to one of the RGB channels. In this way, an image fusion method based on biomimetic vision mechanism is adopted to enhance the contrast and depth resolution of the image.

[0102] In biological vision, the passive membrane equation of the ON-center receptive field is a physical model that describes the dynamics of the center-surround inhibition domain as follows:

[0103]

[0104] Among them, the pixel corresponding to the coordinate position (i, j) in an image, at the coordinate position (i, j), assume that the cell activity of the ON-center is x i,j , t is the response time. Among them, C i,j and S i,j is the total excitatory input and inhibitory input of the input, is the discrete convolution of the input image and the Gaussian kernel, A represents the decay rate, B and D are the maximum and minimum activation levels respectively. The dynamic process is not considered when fusion of images, so only the equation when reaching equilibrium is studied, that is, dx i,j / dt=0, when processing images, B and D need to be set to 1, and this model needs to be revised to obtain the ON-center system response when applied to image fusion:

[0105]

[0106] Among them, x i,j represents the response of the ON-center system, a is the coefficient of the excitatory core, and b is the coefficient of the inhibitory core. i,j is the input excitatory information, S i,j is the inhibitory information of the input. The above formula cannot show the operation between the image and the input.

[0107] Similarly, the OFF-center receptive field is processed in a similar way, as follows:

[0108]

[0109] Among them, x i ′ ,j represents the response of the OFF-center receptive field, however, C i,j and S i,j The calculation of requires convolution of the image with the Gaussian function, as shown in the following formula (4) and formula (5), where I(i,j) represents the image, which can be G or B in this study. i,j and S i,j The two are convolved by Gaussian functions as shown in the following formula (4) and the following formula (5), where I(i, j) represents the center image or the surrounding image. Here, the center image and the surrounding image are selected from G and B as the center (center) and the other as the surrounding (surround).

[0110] in:

[0111]

[0112] Among them, I(i,j) is the input image, C1 and C2 are the Gaussian distribution functions of the central area and the surrounding area, m×n and p×q are the sizes of the template, σ1 and σ2 are constants related to the central area and the surrounding area, which determine the shape of the Gaussian function.

[0113] Formula (2) and formula (3) can reflect the final result after the two images are acted upon. The denominators are the same, and the main difference is the numerator, that is, the difference between one image and the other. However, the difference between one image and the other creates the ON-center type (center) and the OFF-center type (center). The final intuitive expression can be simulated by the Gaussian difference function (Difference of Gaussian, DOG) model of the antagonistic receptive field of the ganglion cell. Formula (6) is to more intuitively display the receptive field of these two forms, but it can be considered that the actual calculations are involved in formula (2) and formula (3):

[0114]

[0115] Among them, DOG represents the Gaussian difference function, C1 and C2 are two-dimensional Gaussian kernel functions, and Gaussian distribution can better describe the activities of cells in different areas, and its sensitivity decreases from the center to the periphery.

[0116] Related art image processing technology fails to effectively utilize the interaction of multi-wavelength narrow-band light, limiting the application effect of the system in the observation of complex tissue structures.

[0117] Compared with the related art, in the embodiment of the present application, the interaction between the two narrow-band lights, the enhanced image feature (+G) and the suppressed image feature (-B), can be fully utilized to make the tissue information integration more complete and enhance the application effect of the system in the observation of complex tissue structures.

[0118] Continue to combine Figure 4 As shown in the figure, the specific application implementation of the system is divided into three stages:

[0119] (1) Preprocessing stage:

[0120] First, the blue light (B, 415nm) and green light (G, 540nm) images are brightness-equalized to ensure the consistency of the brightness of the two images. This step is to ensure that the subsequent image fusion will not cause unnecessary distortion due to brightness differences.

[0121] (2) Phase 1: Application of receptive field model:

[0122] In this stage, the ON-center receptive field model is applied to the blue light and green light images to enhance the tissue features. The specific steps are as follows:

[0123] ON-center receptive field processing is applied to the green light image (G) and the blue light image (B) to enhance their local specific features.

[0124] The OFF-center receptive field processing is applied to the blue light image (B) to suppress some unnecessary features and further extract the depth information.

[0125] (3) The second stage: image fusion stage:

[0126] In this stage, the pseudo-color image is fused based on the image processed in the previous stage:

[0127] Red channel: The boost of the blue light image to the green light image is enhanced by calculating (+G+B) and mapping it to the red channel.

[0128] Blue channel: The suppression effect of the green light image on the blue light image is reflected by calculating (+BG) and mapping it to the blue channel.

[0129] Green channel: Perform a merge (AND) operation on the (+GB) image and the (+BG) image, obtain the result of the AND operation, and map the result to the green channel.

[0130] Finally, after the above steps, a pseudo-color fusion image with enhanced contrast and depth resolution is generated. Compared with the NBI image fusion method of related technologies, this method can better display the depth layer information of the tissue, especially in vascular visualization and lesion detection.

[0131] In this paper, an endoscope image fusion system is designed by using dual narrow-band light (415nm and 540nm) and introducing a pseudo-color fusion algorithm based on bionic vision mechanism. In this way, the following technical effects can be achieved:

[0132] 1. Significantly improve contrast and layer resolution: The pseudo-color fusion algorithm is based on image processing of dual narrow-band light wavelengths, which enhances the color contrast and depth level of the image, especially providing clearer visualization in the detection of early lesions and observation of complex tissues.

[0133] 2. Real-time and efficient imaging system: The system achieves high-quality imaging without sacrificing frame rate (≥30fps under simultaneous acquisition of dual narrow-band light), ensuring smooth clinical operations.

[0134] 3. Enhanced presentation of tissue information: Pseudo-color fusion images fully integrate the superficial and deep tissue information, increase the details of structures such as mucosa and blood vessels, and provide richer imaging support for clinical diagnosis.

[0135] In the second aspect, based on the same application concept as the device in the first aspect, the embodiment of the present application also provides an endoscopic image fusion method, such as Figure 5 As shown, the endoscopic image fusion method may include the following steps 110 to 140:

[0136] Step 110, the narrowband light acquisition module acquires a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the tissue features of the shallow layer in the target tissue of the endoscope, and the second narrowband light is used to capture the tissue features of the deep layer in the target tissue;

[0137] Step 120, the image extraction module uses the first narrowband light and the second narrowband light to illuminate the target tissue respectively, and extracts shallow images and deep images at different depths;

[0138] Step 130, the image processing module integrates the mutual promotion and inhibition mechanisms of biomimetic vision based on the shallow layer images and deep layer images of different depths to obtain the suppressed image features and the enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent the features with different absorption of light;

[0139] Step 140: The pseudo-color image generation module combines color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

[0140] To ensure the practicability and scalability of the technical solution, the present application embodiment also recommends the following verification and optimization directions:

[0141] 1. Clinical effect verification: Conduct comparative experiments in real clinical applications to verify the imaging advantages of the algorithm in different lesion types, collect data such as detection accuracy and recognition efficiency, and use data to support the technical effects of the embodiments of this application.

[0142] 2. Spectral parameter optimization: According to different clinical needs, the combination parameters of dual wavelengths are optimized, and the imaging effects of different wavelengths are studied to improve the adaptability of the technology and image clarity.

[0143] 3. Modular design and cost control: Based on system flexibility and cost-effectiveness, the dual narrow-band light source and image processing module 13 are optimized to ensure that the modular design of the system is easy to replace and expand, so as to promote its application in different medical scenarios.

[0144] The endoscopic image fusion method provided in the embodiment of the present application has the same inventive concept as the above-mentioned endoscopic image fusion system, and can achieve the same or similar technical effects, so examples will not be given one by one here.

[0145] In a third aspect, based on the same application concept as the device in the first aspect, the present application embodiment further provides an image fusion method, such as Figure 6 As shown, the image fusion method may include the following steps 210 to 240:

[0146] Step 210, obtaining a first narrowband light with a first central wavelength and a second narrowband light with a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture shallow features of the object under test and the second narrowband light is used to capture deep features of the object under test.

[0147] Step 220: Use the first narrowband light and the second narrowband light to illuminate the subject respectively, and extract shallow images and deep images at different depths.

[0148] Step 230, based on the shallow images and deep images of different depths, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated to obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light.

[0149] Step 240 , combining color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination.

[0150] As an embodiment, the above step 230 may further include but is not limited to: based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, image processing is performed on specific features of the shallow image and specific features of the deep image respectively to obtain suppressed image features and enhanced image features.

[0151] As an embodiment, based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, image processing is performed on specific features of the shallow image and specific features of the deep image respectively to obtain suppressed image features and enhanced image features, including:

[0152] Applying ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G);

[0153] Apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B); and apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

[0154] As an embodiment, based on shallow images and deep images of different depths, the mutual promotion and inhibition mechanisms of biomimetic vision are integrated to obtain suppressed image features and enhanced image features, including:

[0155] Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the deep enhanced image features (+G), the shallow suppressed image features (-B) and the shallow enhanced image features (+B) are used to generate monochrome images accordingly; the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively;

[0156] The promoting effect of the deep enhanced image features (+G) on the shallow suppressed image features (-B), and the suppressing effect of the shallow enhanced image features (+B) on the deep enhanced image features (+G) are combined to generate a monochrome image; and the monochrome image is mapped to the green channel in the RGB channel.

[0157] As an embodiment, based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the deep enhanced image features (+G), the shallow suppressed image features (-B) and the shallow enhanced image features (+B) are used to generate monochrome images accordingly; and the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively, including:

[0158] Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the image features of the deep layer enhancement (+G) and the image features of the shallow layer suppression (-B) are used to achieve the promotion effect of the shallow layer image on the deep layer image, so as to generate a monochrome image; the first mapping unit is used to map the monochrome image to the red channel in the RGB channel; and the monochrome image is mapped to the red channel in the RGB channel;

[0159] The antagonistic receptive field of ganglion cells is simulated based on a Gaussian difference function model, and the deep enhanced image features (+G) and the shallow enhanced image features (+B) are used to achieve the inhibitory effect of the deep image on the shallow image to generate a monochrome image; and the monochrome image is mapped to the blue channel in the RGB channel.

[0160] The image fusion method provided in the embodiment of the present application has the same inventive concept as the above-mentioned endoscopic image fusion system, and can achieve the same or similar technical effects, so examples will not be given one by one here.

[0161] Fourthly, based on the same application concept as the above-mentioned image fusion method, continue to combine Figure 1 As shown, the embodiment of the present application also provides an image fusion device for implementing the above image fusion method, which may include the following narrow-band light acquisition module 11, image extraction module 12, image processing module 13 and pseudo-color image generation module 14:

[0162] The narrowband light acquisition module 11 is used to acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the shallow characteristics of the object under inspection and the second narrowband light is used to capture the deep characteristics of the object under inspection;

[0163] An image extraction module 12, configured to use the first narrowband light and the second narrowband light to illuminate the subject respectively, and extract shallow images and deep images at different depths;

[0164] The image processing module 13 is used to integrate the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths, and obtain suppressed image features and enhanced image features; the suppressed image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light;

[0165] The pseudo-color image generation module 14 is used to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination.

[0166] The image fusion device provided in the embodiment of the present application has the same inventive concept as the above-mentioned image fusion method, and can achieve the same or similar technical effects, which will not be exemplified one by one here.

[0167] An embodiment of the present application provides an electronic device, including an image fusion device or an endoscope image fusion system.

[0168] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, a handheld terminal, etc. Any electronic device that can implement the embodiment of the present invention belongs to the protection scope of the present invention and is not limited here.

[0169] The endoscopic imaging method and system, imaging method and device in this article all have the same inventive concept, and the same or similar steps can achieve the same effect.

[0170] Figure 7 Shown is a schematic structural diagram of an electronic device 30 provided in an embodiment of the present application.

[0171] like Figure 7 As shown, the electronic device 30 includes one or more processors 31 for implementing the above image fusion method.

[0172] In some embodiments, the electronic device 30 may include a storage medium 39. For example, the computer-readable storage medium may store a program that can be called by the processor 31, and may include a non-volatile storage medium. In some embodiments, the electronic device 30 may include a memory 38 and an interface 37. In some embodiments, the electronic device 30 may also include other hardware according to actual applications.

[0173] The computer-readable storage medium of the embodiment of the present application stores a program thereon, and when the program is executed by the processor 31, it is used to implement the image fusion method described above.

[0174] The present application provides a computer program product, comprising a computer program / instruction, which implements any of the above methods when executed by a processor.

[0175] The present application also provides a computer program, which is stored in a computer-readable storage medium, for example, Figure 7 The computer program is stored in a storage medium 39, and when the processor executes the computer program, the processor 31 is prompted to execute the method described above.

[0176] The present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer-readable storage media include but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0177] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0178] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the phrase "includes a ..." defines an element, and does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

Claims

1. An endoscope image fusion system, characterized in that: include: A narrowband light acquisition module, used for acquiring a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; The difference in central wavelengths between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the tissue features of a shallow layer in a target tissue of an endoscope, and the second narrowband light is used to capture the tissue features of a deep layer in the target tissue; An image extraction module, configured to use the first narrow-band light and the second narrow-band light to irradiate the target tissue respectively, and extract shallow images and deep images at different depths; An image processing module, for fusing the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths, to obtain inhibited image features and enhanced image features; the inhibited image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light; The pseudo-color image generation module is used to combine color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

2. The endoscopic image fusion system according to claim 1, characterized in that: The image processing module includes an image processing submodule; The image processing submodule is used to: based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, perform image processing on the specific features of the shallow image and the specific features of the deep image respectively to obtain suppressed image features and enhanced image features.

3. The endoscopic image fusion system according to claim 2, characterized in that: The image processing submodule includes a first image processing unit and a second image processing unit: The first image processing unit is used to apply ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G); The second image processing unit is used to apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B), and to apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

4. The endoscopic image fusion system according to any one of claims 1 to 3, characterized in that: The image processing module includes a first fused image generation submodule and a second fused image generation submodule; the pseudo color image generation module includes a first mapping submodule and a second mapping submodule; The first fusion image generation submodule is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, and use the deep layer enhanced image features (+G), the shallow layer suppressed image features (-B) and the shallow layer enhanced image features (+B) to generate monochrome images accordingly; the first mapping submodule is used to map the corresponding monochrome images to the red channel and the blue channel of the RGB channel respectively; The second fused image generation submodule is used to combine the promoting effect of the deep enhanced image features (+G) on the shallow suppressed image features (-B), and the suppressing effect of the shallow enhanced image features (+B) on the deep enhanced image features (+G), to generate a monochrome image; the second mapping submodule is used to map the monochrome image to the green channel in the RGB channel.

5. The endoscopic image fusion system according to claim 4, characterized in that: The first fused image generation submodule includes a first fused image generation unit and a second fused image generation unit; the first mapping submodule includes a first mapping unit and a second mapping unit; The first fusion image generating unit is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the image features of the deep layer enhancement (+G) and the image features of the shallow layer suppression (-B) to achieve the promotion effect of the shallow layer image on the deep layer image, and generate a monochrome image; the first mapping unit is used to map the monochrome image to a red channel in the RGB channel; The second fused image generating unit is used to simulate the antagonistic receptive field of ganglion cells based on a Gaussian difference function model, use the deep layer enhanced image feature (+G) and the shallow layer enhanced image feature (+B) to achieve the inhibitory effect of the deep layer image on the shallow layer image, and generate a monochrome image; The second mapping unit is used to map the monochrome image to a blue channel in an RGB channel.

6. An endoscopic image fusion method, characterized in that: include: The narrowband light acquisition module acquires a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; The difference in central wavelengths between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the tissue features of a shallow layer in a target tissue of an endoscope, and the second narrowband light is used to capture the tissue features of a deep layer in the target tissue; The image extraction module uses the first narrow-band light and the second narrow-band light to illuminate the target tissue respectively, and extracts shallow images and deep images at different depths; The image processing module integrates the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths to obtain inhibited image features and enhanced image features; the inhibited image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light; The pseudo-color image generation module combines color channels based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the target tissue.

7. An image fusion method, characterized in that: include: Acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; The difference in central wavelengths between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture shallow features of the object under inspection and the second narrowband light is used to capture deep features of the object under inspection; Using the first narrowband light and the second narrowband light to illuminate the subject respectively, and extracting shallow layer images and deep layer images at different depths; Based on shallow images and deep images of different depths, mutual promotion and inhibition mechanisms of biomimetic vision are integrated to obtain inhibited image features and enhanced image features; both the inhibited image features and the enhanced image features include specific features; the specific features are used to represent features with different absorption of light; Color channels are combined based on the suppressed image features and the enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under examination.

8. The image fusion method according to claim 7, characterized in that: The mutual promotion and inhibition mechanisms of biomimetic vision are integrated based on shallow images and deep images at different depths to obtain suppressed image features and enhanced image features, including: Based on the ON-center type and OFF-center type receptive field models in the visual system of the rattlesnake, image processing is performed on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features.

9. The image fusion method according to claim 8, characterized in that: The ON-center type and OFF-center type receptive field models in the visual system based on the rattlesnake respectively perform image processing on the specific features of the shallow image and the specific features of the deep image to obtain suppressed image features and enhanced image features, including: Applying ON-center receptive field processing to the deep image to enhance specific features of the deep image and obtain deep enhanced image features (+G); Apply OFF-center receptive field processing to the shallow image to suppress specific features of the shallow image and obtain shallow suppressed image features (-B); and apply ON-center receptive field processing to the shallow image to enhance specific features of the shallow image and obtain shallow enhanced image features (+B).

10. The image fusion method according to any one of claims 7 to 9, characterized in that: The mutual promotion and inhibition mechanisms of biomimetic vision are integrated based on shallow images and deep images at different depths to obtain suppressed image features and enhanced image features, including: Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the deep enhanced image features (+G), the shallow suppressed image features (-B) and the shallow enhanced image features (+B) are used to generate monochrome images accordingly; the corresponding monochrome images are mapped to the red channel and the blue channel of the RGB channel respectively; The promoting effect of the deep enhanced image features (+G) on the shallow suppressed image features (-B), and the suppressing effect of the shallow enhanced image features (+B) on the deep enhanced image features (+G) are combined to generate a monochrome image; and the monochrome image is mapped to the green channel in the RGB channel.

11. The image fusion method according to claim 10, characterized in that: The Gaussian difference function model is based on simulating the antagonistic receptive field of ganglion cells, using deep layer enhanced image features (+G), shallow layer suppressed image features (-B) and shallow layer enhanced image features (+B) to generate monochrome images accordingly; Map the corresponding monochrome image to the red channel and blue channel of the RGB channel respectively, including: Based on the Gaussian difference function model, the antagonistic receptive field of ganglion cells is simulated, and the image features of the deep layer enhancement (+G) and the image features of the shallow layer suppression (-B) are used to achieve the promotion effect of the shallow layer image on the deep layer image, so as to generate a monochrome image; the first mapping unit is used to map the monochrome image to the red channel in the RGB channel; and the monochrome image is mapped to the red channel in the RGB channel; The antagonistic receptive field of ganglion cells is simulated based on a Gaussian difference function model, and the deep enhanced image features (+G) and the shallow enhanced image features (+B) are used to achieve the inhibitory effect of the deep image on the shallow image to generate a monochrome image; and the monochrome image is mapped to the blue channel in the RGB channel.

12. An image fusion device, characterized in that: For implementing the image fusion method according to any one of claims 7 to 11, the image fusion device comprises: A narrowband light acquisition module, used to acquire a first narrowband light of a first central wavelength and a second narrowband light of a second central wavelength; the central wavelength difference between the first narrowband light and the second narrowband light reaches a preset difference; the first narrowband light is used to capture the shallow layer characteristics of the object under inspection and the second narrowband light is used to capture the deep layer characteristics of the object under inspection; An image extraction module, configured to use the first narrowband light and the second narrowband light to illuminate the subject respectively, and extract shallow images and deep images at different depths; An image processing module, for fusing the mutual promotion and inhibition mechanisms of biomimetic vision based on shallow images and deep images of different depths, to obtain inhibited image features and enhanced image features; the inhibited image features and the enhanced image features both include specific features; the specific features are used to represent features with different absorption of light; The pseudo-color image generation module is used to combine color channels based on suppressed image features and enhanced image features to generate a pseudo-color image; the pseudo-color image is used to represent the tissue features of the object under inspection.

13. An electronic device, characterized in that: It comprises one or more processors, used to implement the image fusion method as described in any one of claims 7-11.

Images