Image fusion method and endoscope device

Abstract

The present disclosure provides an image fusion method and an endoscope device. The method includes: determining color photosensitive characteristic parameters of a white light image based on the white light image of a human tissue, where the white light image is an image obtained by irradiating the human tissue with light emitted by a white light source; performing pseudo-color conversion on the black-and-white image by using pixel values of each pixel point in the black-and-white image of the human tissue and the color photosensitive characteristic parameters of the white light image to obtain a fluorescence image, where the black-and-white image is an image obtained by irradiating the human tissue with light emitted by a fluorescence light source; and performing image fusion on the fluorescence image and the white light image to obtain a fluorescence image with restored color. Thus, in the present disclosure, with reference to the color photosensitive characteristic parameters of the white light image, the obtained fluorescence image is consistent with the actual fluorescence color, and it is ensured that the white light details of the fused image remain unchanged, improving the quality of the fused image.

Description

Background Art

[0002] Fluorescence endoscope is a spectral technology that applies the inherent fluorescence of laser. It is a new type of device that enters the human body cavity through various endoscopes for diagnosis. It automatically identifies and diagnoses based on the inherent fluorescence spectral characteristics of human tissues, can immediately indicate whether the measured tissue is normal tissue, can distinguish between benign and malignant lesions of the measured tissue, and improve the diagnosis rate of early cancer and dysplasia.

[0003] In the current technical solutions of fluorescence endoscopy imaging, the pseudo-color conversion method of fluorescence images is to replace one of the channels in the RGB image, or map the information in the fluorescence image to the hue information of the HIS color gamut space. However, this method results in the color of the obtained fluorescence image being inconsistent with the actual fluorescence performance. The fluorescence concentration can only be reflected by multiple colors (gradient fluorescence mode), and the change in fluorescence concentration cannot be reflected in monochromatic pseudo-color. Moreover, after the two pseudo-color images are fused with the white light image, the details of the main white light image will be covered. The difference is that different colors in the gradient fluorescence mode will cause more details of the white light image to be lost after being superimposed on the white light image, resulting in a lower quality of the fused image. Summary of the Invention

[0004] An image fusion method and an endoscope device are provided in an exemplary embodiment of the present disclosure to improve the quality of the fused image.

[0005] A first aspect of the present disclosure provides an image fusion method, the method comprising:

[0006] Based on a white light image of human tissue, determining a color light-sensitive characteristic parameter of the white light image, wherein the white light image is an image obtained by irradiating the human tissue with light emitted by a white light source;

[0007] Performing pseudo-color conversion on the black and white image by using the pixel values of each pixel point in the black and white image of the human tissue and the color light-sensitive characteristic parameter of the white light image to obtain a fluorescence image, wherein the black and white image is an image obtained by irradiating the human tissue with light emitted by a fluorescence light source;

[0008] Performing image fusion on the fluorescence image and the white light image to obtain a fluorescence image with restored color.

[0009] In this embodiment, a fluorescence image is obtained by performing pseudo-color conversion on a black and white image by combining the color light-sensitive characteristic parameter of the white light image, and the fluorescence image and the white light image are fused. In this embodiment, with reference to the color light-sensitive characteristic parameter of the white light image, the color that the fluorescence should present in the invisible band of the human eye is obtained, so that the obtained fluorescence image is consistent with the actual fluorescence color, and it is ensured that the white light details of the fused image remain unchanged, improving the quality of the fused image.

[0010] In one embodiment, determining the color light-sensitive characteristic parameters of the white light image based on the white light image of human tissue includes:

[0011] Converting the white light image into an RGB image by using an interpolation algorithm, and obtaining the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, where the color component values include a red component value, a green component value, and a blue component value; or,

[0012] Obtaining the color light-sensitive parameters of the white light image according to the color component values of each pixel point in the white light image.

[0013] In this embodiment, the color light-sensitive parameters of the white light image are obtained through the color component values of each pixel point in the white light image or the color component values of each pixel point in the RGB image corresponding to the white light image. Thus, it is ensured that the color light-sensitive parameters of the white light image can be obtained.

[0014] In one embodiment, the color light-sensitive characteristic parameters include a red light-sensitive characteristic parameter, a green light-sensitive characteristic parameter, and a blue light-sensitive characteristic parameter;

[0015] The obtaining of the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image includes:

[0016] Obtaining the red light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the red component value of each pixel point in the RGB image corresponding to the white light image; and obtaining the green light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the green component value of each pixel point in the RGB image; and obtaining the blue light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the blue component value of each pixel point in the RGB image; or,

[0017] Obtaining the red light-sensitive characteristic parameter of the white light image by using the red component value of each pixel point in the RGB image corresponding to the white light image; and obtaining the green light-sensitive characteristic parameter by using the green component value of each pixel point in the RGB image corresponding to the white light image; and obtaining the blue light-sensitive characteristic parameter by using the blue component value of each pixel point in the RGB image corresponding to the white light image; or,

[0018] Obtain the red light-sensing characteristic parameter of the white light image based on the green component value and the red component value of each pixel in the RGB image corresponding to the white light image; and set the green light-sensing characteristic parameter of the white light image to a specified parameter; and obtain the blue light-sensing characteristic parameter of the white light image based on the green component value and the blue component value of each pixel in the RGB image.

[0019] In one embodiment, the obtaining the red light-sensing characteristic parameter of the white light image according to the pixel value of each pixel in the white light image and the red component value of each pixel in the RGB image corresponding to the white light image includes:

[0020] Add the pixel values of each pixel in the white light image to obtain the total pixel value; add the red component values of each pixel in the RGB image corresponding to the white light image to obtain the total red component value, multiply the total red component value by a first preset threshold to obtain the enlarged total red component value, and divide the total pixel value by the enlarged total red component value to obtain the red light-sensing characteristic parameter of the white light image;

[0021] The obtaining the green light-sensing characteristic parameter of the white light image according to the pixel value of each pixel in the white light image and the green component value of each pixel in the RGB image corresponding to the white light image includes:

[0022] Add the green component values of each pixel in the RGB image corresponding to the white light image to obtain the total green component value, multiply the total green light-sensing characteristic parameter by a second preset threshold to obtain the enlarged total green component value, and divide the total pixel value by the enlarged total green component value to obtain the green light-sensing characteristic parameter of the white light image;

[0023] The obtaining the blue light-sensing characteristic parameter of the white light image according to the pixel value of each pixel in the white light image and the blue component value of each pixel in the RGB image corresponding to the white light image includes:

[0024] Add the blue component values of each pixel in the RGB image corresponding to the white light image to obtain the total blue component value, multiply the total blue component value by a first preset threshold to obtain the enlarged total blue component value, and divide the total pixel value by the enlarged total blue component value to obtain the blue light-sensing characteristic parameter of the white light image.

[0025] In one embodiment, the obtaining the red light-sensing characteristic parameter of the white light image by using the red component value of each pixel in the RGB image corresponding to the white light image includes:

[0026] For any pixel point in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel point to obtain the total color component value of the pixel point. Add the total color component values of all pixel points in the RGB image to obtain the total color component value of the RGB image. Divide the total color component value of the RGB image by the total red component value of all pixel points in the RGB image to obtain the red light-sensitive characteristic parameter in the color light-sensitive characteristic parameters, where the total red component value is obtained based on the red component values of all pixel points in the RGB image and a third preset threshold;

[0027] The obtaining of the green light-sensitive characteristic parameter by using the green component values of all pixel points in the RGB image corresponding to the white light image includes:

[0028] Divide the total color component value of the RGB image by the total green component value of all pixel points in the RGB image to obtain the green light-sensitive characteristic parameter in the color light-sensitive characteristic parameters; where the total red component value is obtained based on the green component values of all pixel points in the RGB image and a third preset threshold;

[0029] The obtaining of the blue light-sensitive characteristic parameter by using the blue component values of all pixel points in the RGB image corresponding to the white light image includes:

[0030] Divide the total color component value of the RGB image by the total blue component value of all pixel points in the RGB image to obtain the blue light-sensitive characteristic parameter in the color light-sensitive characteristic parameters, where the total red component value is obtained based on the blue component values of all pixel points in the RGB image and a third preset threshold.

[0031] In one embodiment, the obtaining of the red light-sensitive characteristic parameter of the white light image based on the green component values and red component values of all pixel points in the RGB image corresponding to the white light image includes:

[0032] Add the green component values of all pixel points in the RGB image corresponding to the white light image to obtain the total green component value of the RGB image, and add the red component values of all pixel points in the RGB image to obtain the total red component value of the RGB image, and divide the total green component value by the total red component value to obtain the red light-sensitive characteristic parameter;

[0033] The obtaining of the blue light-sensitive characteristic parameter of the white light image based on the green component values and blue component values of all pixel points in the RGB image includes:

[0034] Add the blue component values of each pixel in the RGB image to obtain the total blue component value of the RGB image, and divide the total green component value of the RGB image by the total blue component value to obtain the blue photosensitive characteristic parameter.

[0035] In one embodiment, the color photosensitive characteristic parameters include a red photosensitive characteristic parameter, a green photosensitive characteristic parameter, and a blue photosensitive characteristic parameter;

[0036] Obtaining the color photosensitive parameters of the white light image according to the color component values of each pixel in the white light image includes:

[0037] For any pixel in the white light image, perform a weighted sum of the color component values of the pixel in the white light image to obtain the total color component value of the pixel, and add the total color component values of each pixel to obtain the total color component value of the white light image. Divide the total color component value of the white light image by the total red component value of each pixel in the white light image to obtain the red photosensitive characteristic parameter, where the total red component value of each pixel in the white light image is based on the red component value of each pixel in the white light image and a fourth preset threshold; and,

[0038] Divide the total color component value of the white light image by the total green component value of each pixel in the white light image to obtain the green photosensitive characteristic parameter, where the total green component value of each pixel in the white light image is based on the green component value of each pixel in the white light image and a fourth preset threshold; and,

[0039] Divide the total color component value of the white light image by the total blue component value of each pixel in the white light image to obtain the blue photosensitive characteristic parameter, where the total blue component value of each pixel in the white light image is based on the blue component value of each pixel in the white light image and a fourth preset threshold.

[0040] In one embodiment, performing pseudo-color conversion on the black-and-white image by using the pixel values of each pixel in the black-and-white image of the human tissue and the color photosensitive characteristic parameters of the white light image to obtain a fluorescence image includes:

[0041] For any pixel in the black-and-white image, multiply the pixel value of the pixel in the black-and-white image by the red photosensitive characteristic parameter to obtain an intermediate red photosensitive characteristic parameter, and multiply the intermediate red photosensitive characteristic parameter by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel; and,

[0042] Multiply the pixel value of the pixel point in the black-and-white image by the green photosensitive characteristic parameter to obtain an intermediate green photosensitive characteristic parameter, and multiply the intermediate green photosensitive characteristic parameter by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point; and,

[0043] Multiply the pixel value of the pixel point in the black-and-white image by the blue photosensitive characteristic parameter to obtain an intermediate blue photosensitive characteristic parameter, and multiply the intermediate blue photosensitive characteristic parameter by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel point;

[0044] Obtain the fluorescence image according to the red component value, green component value and blue component value after pseudo-color conversion of each pixel point.

[0045] In this embodiment, the black-and-white image is pseudo-color converted into a fluorescence image through the color photosensitive characteristic parameters of the white light image, so that the obtained fluorescence image is more consistent with the color of the actual fluorescence, and the white light details of the fused image are ensured to remain unchanged.

[0046] A second aspect of the present disclosure provides an endoscope device, including a storage unit and a processor, wherein:

[0047] The storage unit is configured to store a white light image and a black-and-white image of a human tissue, wherein the white light image is an image obtained after the light emitted by a white light source irradiates the human tissue, and the black-and-white image is an image obtained after the light emitted by a fluorescence light source irradiates the human tissue;

[0048] The processor is configured to:

[0049] Based on the white light image, determine the color photosensitive characteristic parameters of the white light image;

[0050] Perform pseudo-color conversion on the black-and-white image by using the pixel values of the pixel points in the black-and-white image and the color photosensitive characteristic parameters of the white light image to obtain a fluorescence image;

[0051] Fuse the fluorescence image and the white light image to obtain a fluorescence image with color restored.

[0052] In one embodiment, when the processor executes the step of determining the color photosensitive characteristic parameters of the white light image based on the white light image, it is specifically configured to:

[0053] Convert the white light image into an RGB image by using an interpolation algorithm, and obtain the color photosensitive parameters of the white light image based on the color component values of the pixel points in the RGB image, wherein the color component values include red component values, green component values and blue component values; or,

[0054] Obtain the color light-sensitive parameters of the white light image according to the color component values of each pixel point in the white light image.

[0055] In one embodiment, the color light-sensitive characteristic parameters include a red light-sensitive characteristic parameter, a green light-sensitive characteristic parameter, and a blue light-sensitive characteristic parameter;

[0056] The processor executes obtaining the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, and is specifically configured as:

[0057] Obtain the red light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the red component value of each pixel point in the RGB image corresponding to the white light image; and obtain the green light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the green component value of each pixel point in the RGB image; and obtain the blue light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the blue component value of each pixel point in the RGB image; or,

[0058] Obtain the red light-sensitive characteristic parameter of the white light image by using the red component value of each pixel point in the RGB image corresponding to the white light image; and obtain the green light-sensitive characteristic parameter by using the green component value of each pixel point in the RGB image corresponding to the white light image; and obtain the blue light-sensitive characteristic parameter by using the blue component value of each pixel point in the RGB image corresponding to the white light image; or,

[0059] Obtain the red light-sensitive characteristic parameter of the white light image based on the green component value and the red component value of each pixel point in the RGB image corresponding to the white light image; and set the green light-sensitive characteristic parameter of the white light image to a specified parameter; and obtain the blue light-sensitive characteristic parameter of the white light image based on the green component value and the blue component value of each pixel point in the RGB image.

[0060] In one embodiment, the processor executes obtaining the red light-sensitive characteristic parameter of the white light image according to the pixel value of each pixel point in the white light image and the red component value of each pixel point in the RGB image corresponding to the white light image, and is specifically configured as:

[0061] Add the pixel values of each pixel point in the white light image to obtain the total pixel value; add the red component values of each pixel point in the RGB image corresponding to the white light image to obtain the total red component value, multiply the total red component value by a first preset threshold to obtain an enlarged total red component value, and divide the total pixel value by the enlarged total red component value to obtain the red light-sensitive characteristic parameter of the white light image;

[0062] The obtaining of the green light-sensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the green component values of each pixel point in the RGB image corresponding to the white light image includes:

[0063] Add the green component values of each pixel point in the RGB image corresponding to the white light image to obtain the total green component value, multiply the total green light-sensitive characteristic parameter by a second preset threshold to obtain an enlarged total green component value, and divide the total pixel value by the enlarged total green component value to obtain the green light-sensitive characteristic parameter of the white light image;

[0064] The obtaining of the blue light-sensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the blue component values of each pixel point in the RGB image corresponding to the white light image includes:

[0065] Add the blue component values of each pixel point in the RGB image corresponding to the white light image to obtain the total blue component value, multiply the total blue component value by a first preset threshold to obtain an enlarged total blue component value, and divide the total pixel value by the enlarged total blue component value to obtain the blue light-sensitive characteristic parameter of the white light image.

[0066] In one embodiment, the processor executes the obtaining of the red light-sensitive characteristic parameter of the white light image by using the red component values of each pixel point in the RGB image corresponding to the white light image, and is specifically configured as:

[0067] For any pixel point in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel point to obtain the total color component value of the pixel point, add the total color component values of each pixel point in the RGB image to obtain the total color component value of the RGB image, and divide the total color component value of the RGB image by the total red component value of each pixel point in the RGB image to obtain the red light-sensitive characteristic parameter in the color light-sensitive characteristic parameter, where the total red component value is obtained based on the red component values of each pixel point in the RGB image and a third preset threshold;

[0068] Obtaining the green photosensitive feature parameter by using the green component values of each pixel point in the RGB image corresponding to the white light image includes:

[0069] Dividing the total color component value of the RGB image by the total green component value of each pixel point in the RGB image to obtain the green photosensitive feature parameter in the color photosensitive feature parameter; wherein, the total red component value is obtained based on the green component value of each pixel point in the RGB image and a third preset threshold.

[0070] Obtaining the blue photosensitive feature parameter by using the blue component values of each pixel point in the RGB image corresponding to the white light image includes:

[0071] Dividing the total color component value of the RGB image by the total blue component value of each pixel point in the RGB image to obtain the blue photosensitive feature parameter in the color photosensitive feature parameter, wherein, the total red component value is obtained based on the blue component value of each pixel point in the RGB image and a third preset threshold.

[0072] In one embodiment, the processor executes obtaining the red photosensitive feature parameter of the white light image based on the green component value and the red component value of each pixel point in the RGB image corresponding to the white light image, and is specifically configured as:

[0073] Adding the green component values of each pixel point in the RGB image corresponding to the white light image to obtain the total green component value of the RGB image, and adding the red component values of each pixel point in the RGB image to obtain the total red component value of the RGB image, and dividing the total green component value by the total red component value to obtain the red photosensitive feature parameter.

[0074] Obtaining the blue photosensitive feature parameter of the white light image based on the green component value and the blue component value of each pixel point in the RGB image includes:

[0075] Adding the blue component values of each pixel point in the RGB image to obtain the total blue component value of the RGB image, and dividing the total green component value of the RGB image by the total blue component value to obtain the blue photosensitive feature parameter.

[0076] In one embodiment, the color photosensitive feature parameter includes a red photosensitive feature parameter, a green photosensitive feature parameter, and a blue photosensitive feature parameter.

[0077] The processor executes obtaining the color photosensitive parameter of the white light image according to the color component values of each pixel point in the white light image, and is specifically configured as:

[0078] For any pixel point in the white light image, the color component values of the pixel point in the white light image are weighted and summed to obtain the total color component value of the pixel point, and the total color component values of all pixel points are added to obtain the total color component value of the white light image. The total color component value of the white light image is divided by the total red component value of each pixel point in the white light image to obtain the red light-sensitive characteristic parameter, where the total red component value of each pixel point in the white light image is obtained based on the red component value of each pixel point in the white light image and a fourth preset threshold; and,

[0079] The total color component value of the white light image is divided by the total green component value of each pixel point in the white light image to obtain the green light-sensitive characteristic parameter, where the total green component value of each pixel point in the white light image is obtained based on the green component value of each pixel point in the white light image and a fourth preset threshold; and,

[0080] The total color component value of the white light image is divided by the total blue component value of each pixel point in the white light image to obtain the blue light-sensitive characteristic parameter, where the total blue component value of each pixel point in the white light image is obtained based on the blue component value of each pixel point in the white light image and a fourth preset threshold.

[0081] In one embodiment, the processor performs pseudo-color conversion on the black-and-white image by using the pixel values of each pixel point in the black-and-white image and the color light-sensitive characteristic parameters of the white light image to obtain a fluorescence image, and is specifically configured as:

[0082] For any pixel point in the black-and-white image, the pixel value of the pixel point in the black-and-white image is multiplied by the red light-sensitive characteristic parameter to obtain an intermediate red light-sensitive characteristic parameter, and the intermediate red light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel point; and,

[0083] The pixel value of the pixel point in the black-and-white image is multiplied by the green light-sensitive characteristic parameter to obtain an intermediate green light-sensitive characteristic parameter, and the intermediate green light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point; and,

[0084] The pixel value of the pixel point in the black-and-white image is multiplied by the blue light-sensitive characteristic parameter to obtain an intermediate blue light-sensitive characteristic parameter, and the intermediate blue light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel point;

[0085] The fluorescence image is obtained according to the red component value after pseudo-color conversion, the green component value after color conversion, and the blue component value after color conversion of each pixel point.

[0086] According to the third aspect provided by the embodiments of the present disclosure, a computer storage medium is provided. The computer storage medium stores a computer program, and the computer program is used to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0088] Figure 1A-1B It is a schematic diagram of an applicable scenario according to an embodiment of the present disclosure;

[0089] Figure 2 It is one of the schematic flowcharts of the image fusion method according to an embodiment of the present disclosure;

[0090] Figure 3 It is one of the schematic flowcharts of determining the red photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0091] Figure 4 It is one of the schematic flowcharts of determining the green photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0092] Figure 5 It is one of the schematic flowcharts of determining the blue photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0093] Figure 6 It is one of the schematic flowcharts of determining the red photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0094] Figure 7 It is one of the schematic flowcharts of determining the blue photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0095] Figure 8 It is one of the schematic flowcharts of determining the red photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0096] Figure 9 It is one of the schematic flowcharts of determining the green photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0097] Figure 10 It is one of the schematic flowcharts of determining the blue photosensitive characteristic parameters according to an embodiment of the present disclosure;

[0098] Figure 11 Schematic flowchart of determining a fluorescence image according to an embodiment of the present disclosure;

[0099] Figure 12 Schematic flowchart of image fusion according to an embodiment of the present disclosure;

[0100] Figure 13 Second schematic flowchart of an image fusion method according to an embodiment of the present disclosure;

[0101] Figure 14 Image fusion device according to an embodiment of the present disclosure;

[0102] Figure 15 Schematic structural diagram of an endoscope device according to an embodiment of the present disclosure. Detailed implementation manners

[0103] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the scope of protection of the present disclosure.

[0104] The term "and / or" in the embodiments of the present disclosure describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0105] The application scenarios described in the embodiments of the present disclosure are for more clearly explaining the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those of ordinary skill in the art know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems. Among them, in the description of the present disclosure, unless otherwise specified, the meaning of "a plurality" is two or more.

[0106] In the prior art, in the technical solution of fluorescence endoscopy imaging, the false color conversion method of the fluorescence image is to replace one of the channels in the RGB image, or map the information in the fluorescence image to the hue information in the HIS color gamut space. However, this method results in the color of the obtained fluorescence image being inconsistent with the actual fluorescence performance. The fluorescence concentration can only be represented by multiple colors (gradient fluorescence mode), and the change in fluorescence concentration cannot be reflected in monochromatic false color. Moreover, after the two false color images are fused with the white light image, the details of the main white light image will be covered. The difference is that different colors in the gradient fluorescence mode will cause more details of the white light image to be lost after being superimposed on the white light image, resulting in a lower quality of the fused image.

[0107] Therefore, the present disclosure provides an image fusion method, which performs false color conversion on a black-and-white image by combining the color light-sensitive characteristic parameters of a white light image to obtain a fluorescence image, and fuses the fluorescence image and the white light image. In the present disclosure, with reference to the color light-sensitive characteristic parameters of the white light image, the color that the fluorescence should present in the invisible band of the human eye is determined, so that the obtained fluorescence image is consistent with the actual fluorescence color, and the details of the white light in the fused image are ensured to remain unchanged, improving the quality of the fused image. Next, the solution of the present disclosure will be introduced in detail with reference to the accompanying drawings.

[0108] As Figure 1A shown, an application scenario of an image fusion method. The figure includes: a display device 10, an endoscope device 20, an endoscope 21, and a memory 30; wherein:

[0109] In a possible application scenario, the endoscope device 20 obtains a white light image and a black-and-white image of a human tissue through the endoscope 21, stores the white light image and the black-and-white image in the memory 30, and then the endoscope device 20 determines the color light-sensitive characteristic parameters of the white light image based on the white light image of the human tissue, wherein the white light image is an image obtained by irradiating the human tissue with light emitted by a white light source; the endoscope device 20 performs false color conversion on the black-and-white image by using the pixel values of each pixel point in the black-and-white image of the human tissue and the color light-sensitive characteristic parameters of the white light image to obtain a fluorescence image, wherein the black-and-white image is an image obtained by irradiating the human tissue with light emitted by a fluorescence light source; and performs image fusion on the fluorescence image and the white light image to obtain a fluorescence image with color restored, and the endoscope device 20 sends the fluorescence image with color restored to the display device 10 for display.

[0110] In another possible application scenario, as Figure 1BAs shown, the application scenario includes an endoscopic device 20, an endoscope 21, and a display device 10. The endoscopic device 20 acquires white light images and black-and-white images of human tissues through the endoscope 21. Then, based on the white light image of the human tissue, the endoscopic device 20 determines the color light-sensitive characteristic parameters of the white light image, where the white light image is an image obtained after the light emitted by a white light source irradiates the human tissue. The endoscopic device 20 performs pseudo-color conversion on the black-and-white image by using the pixel values of each pixel point in the black-and-white image of the human tissue and the color light-sensitive characteristic parameters of the white light image to obtain a fluorescence image, where the black-and-white image is an image obtained after the light emitted by a fluorescence light source irradiates the human tissue. And the endoscopic device 20 fuses the fluorescence image and the white light image to obtain a fluorescence image with restored color, and the endoscopic device 20 sends the fluorescence image with restored color to the display device 10 for display.

[0111] Wherein, Figure 1A Between the endoscopic device 20 and the display device 10 in it, information interaction can be carried out through a communication network. Among them, the communication method adopted by the communication network can be divided into a wireless communication method or a wired communication method.

[0112] Exemplarily, the endoscopic device 20 can access the network through cellular mobile communication technology and communicate with the display device 10. Among them, the cellular mobile communication technology, for example, includes the fifth-generation mobile communication (5th Generation Mobile Networks, 5G) technology.

[0113] Optionally, the endoscopic device 20 can access the network through a short-range wireless communication method and communicate with the display device 10. Among them, the short-range wireless communication method, for example, includes wireless fidelity (Wireless Fidelity, Wi-Fi) technology.

[0114] Moreover, in the description of this application, only a single display device 10, endoscopic device 20, and memory 30 are described in detail. However, those skilled in the art should understand that the shown display device 10, endoscopic device 20, and memory 30 are intended to represent the operations of the display device 10, endoscopic device 20, and memory 30 involved in the technical solution of this application. Instead of implying limitations on the number, type, or location of the display device 10, endoscopic device 20, and memory 30. It should be noted that if additional modules are added to or individual modules are removed from the illustrated environment, the underlying concept of the exemplary embodiments of this application will not be changed. Additionally, although a two-way arrow from the memory 30 to the endoscopic device 20 is shown in Figure 1A for convenience of illustration, those skilled in the art can understand that the above data transmission and reception also need to be realized through the network.

[0115] It should be noted that the memory in the embodiments of the present application can be, for example, a cache system, a hard disk storage, a memory storage, etc. In addition, the image fusion method proposed in the present application is not only applicable to Figure 1A and Figure 1B the application scenarios shown, but also applicable to any device with an interface display requirement.

[0116] Exemplarily, the display device 10 includes but is not limited to: a visualization large screen, a tablet computer, a laptop computer, a palm computer, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal device in industrial control, a wireless terminal device in unmanned driving, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, or a wireless terminal device in a smart home, etc.; a relevant client can be installed on the terminal device, and the client can be software (for example, a browser, a short video software, etc.), or a web page, a small program, etc.

[0117] The endoscope device 20 can be implemented by a single server or by multiple servers. The endoscope device 20 can be implemented by a physical server or by a virtual server.

[0118] As Figure 2 shown, it is a schematic flowchart of the image fusion method of the present disclosure, which may include the following steps:

[0119] Step 201: Based on the white light image of the human tissue, determine the color photosensitive characteristic parameters of the white light image, where the white light image is an image obtained after the light emitted by a white light source irradiates the human tissue;

[0120] In one embodiment, the color photosensitive characteristic parameters of the white light image are mainly determined in the following two ways:

[0121] Method 1: Use an interpolation algorithm to convert the white light image into an RGB image, and based on the color component values of each pixel point in the RGB image, obtain the color photosensitive parameters of the white light image, where the color component values include a red component value, a green component value, and a blue component value.

[0122] It should be noted that: The specific method of using the interpolation algorithm to convert the white light image into an RGB image can be set according to the actual situation, and this embodiment does not limit it here.

[0123] Among them, the color light-sensitive characteristic parameters include red light-sensitive characteristic parameters, green light-sensitive characteristic parameters, and blue light-sensitive characteristic parameters. And the methods for determining the color light-sensitive parameters of the white light image in Method 1 mainly include the following three methods:

[0124] Method 1:

[0125] Red light-sensitive characteristic parameter: According to the pixel values of each pixel point in the white light image and the red component values of each pixel point in the RGB image corresponding to the white light image, the red light-sensitive characteristic parameter of the white light image is obtained.

[0126] In one embodiment, the above method for determining the red light-sensitive characteristic parameter can be specifically implemented as follows: Add the pixel values of each pixel point in the white light image to obtain the total pixel value; add the red component values of each pixel point in the RGB image corresponding to the white light image to obtain the total red component value; multiply the total red component value by a first preset threshold to obtain an enlarged total red component value; divide the total pixel value by the enlarged total red component value to obtain the red light-sensitive characteristic parameter of the white light image. Among them, the red light-sensitive characteristic parameter R can be obtained through formula (1) av :

[0127]

[0128] Among them, BAYER ij is the pixel value of the pixel point at the i-th row and j-th column in the white light image, R ij is the red component value of the pixel point at the i-th row and j-th column in the RGB image, A is the first preset threshold, m is the total number of rows of pixel points in the white light image, and n is the total number of columns of pixel points in the white light image.

[0129] Green light-sensitive characteristic parameter: According to the pixel values of each pixel point in the white light image and the green component values of each pixel point in the RGB image, the green light-sensitive characteristic parameter of the white light image is obtained.

[0130] In one embodiment, the above method for determining the green light-sensitive characteristic parameter can be specifically implemented as follows: Add the green component values of each pixel point in the RGB image corresponding to the white light image to obtain the total green component value, multiply the total green light-sensitive characteristic parameter by a second preset threshold to obtain an enlarged total green component value, and divide the total pixel value by the enlarged total green component value to obtain the green light-sensitive characteristic parameter of the white light image. Among them, the green light-sensitive characteristic parameter G can be obtained through formula (2) av :

[0131]

[0132] Among them, Gav is the green photosensitive characteristic parameter of the white light image, BAYER ij is the pixel value of the pixel at the i-th row and j-th column in the white light image, G ij is the green component value of the pixel at the i-th row and j-th column in the RGB image, B is the second preset threshold, m is the total number of rows of pixels in the white light image, and n is the total number of columns of pixels in the white light image.

[0133] Blue photosensitive characteristic parameter: According to the pixel values of each pixel in the white light image and the blue component values of each pixel in the RGB image, the blue photosensitive characteristic parameter of the white light image is obtained.

[0134] In one embodiment, the step of determining the blue photosensitive characteristic parameter described above can be implemented as follows: adding the blue component values of each pixel in the RGB image corresponding to the white light image to obtain the total blue component value, multiplying the total blue component value by the first preset threshold to obtain the enlarged total blue component value, and dividing the total pixel value by the enlarged total blue component value to obtain the blue photosensitive characteristic parameter of the white light image. Among them, the blue photosensitive characteristic parameter B can be obtained through formula (3). av :

[0135]

[0136] Among them, B av is the blue photosensitive characteristic parameter of the white light image, BAYER ij is the pixel value of the pixel at the i-th row and j-th column in the white light image, B ij is the blue component value of the pixel at the i-th row and j-th column in the RGB image, A is the first preset threshold, m is the total number of rows of pixels in the white light image, and n is the total number of columns of pixels in the white light image.

[0137] It should be noted that: the first preset threshold in this embodiment is 4 and the second preset threshold is 2, but the specific values of the first preset threshold and the second preset threshold in this embodiment are not limited, and the first preset threshold and the second preset threshold can be set according to the actual situation.

[0138] Method 2:

[0139] Red photosensitive characteristic parameter: Using the red component values of each pixel in the RGB image corresponding to the white light image, the red photosensitive characteristic parameter of the white light image is obtained. Specifically, as Figure 3 shown, it is a schematic flow chart of using Method 2 to determine the red photosensitive characteristic parameter, including the following steps:

[0140] Step 301: For any pixel point in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel point to obtain the total color component value of the pixel point;

[0141] Step 302: Add the total color component values of all pixel points in the RGB image to obtain the total color component value of the RGB image;

[0142] Step 303: Divide the total color component value of the RGB image by the total red component value of all pixel points in the RGB image to obtain the red light-sensing characteristic parameter in the color light-sensing characteristic parameters, where the total red component value is obtained based on the red component value of all pixel points in the RGB image and a third preset threshold. Among them, the red light-sensing characteristic parameter R can be determined by formula (4) av :

[0143]

[0144] Among them, R ij is the red component value of the pixel point in the i-th row and j-th column in the RGB image, G ij is the green component value of the pixel point in the i-th row and j-th column in the RGB image, B ij is the blue component value of the pixel point in the i-th row and j-th column in the RGB image, C is the third preset threshold, p is the total number of rows of pixel points in the RGB image, and q is the total number of columns of pixel points in the RGB image.

[0145] Green light-sensing characteristic parameter: The green light-sensing characteristic parameter is obtained by using the green component values of all pixel points in the RGB image corresponding to the white light image. Specifically, as Figure 4 shown, it is a schematic flow chart for determining the green light-sensing characteristic parameter using Method 2, including the following steps:

[0146] Step 401: For any pixel point in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel point to obtain the total color component value of the pixel point;

[0147] Step 402: Add the total color component values of all pixel points in the RGB image to obtain the total color component value of the RGB image;

[0148] Step 403: Divide the total color component value of the RGB image by the total green component value of each pixel in the RGB image to obtain the green photosensitive characteristic parameter in the color photosensitive characteristic parameters; wherein, the total red component value is obtained based on the green component value of each pixel in the RGB image and a third preset threshold. Among them, the green photosensitive characteristic parameter G can be obtained through formula (5). av :

[0149]

[0150] wherein, R ij is the red component value of the pixel at the i-th row and j-th column in the RGB image, G ij is the green component value of the pixel at the i-th row and j-th column in the RGB image, B ij is the blue component value of the pixel at the i-th row and j-th column in the RGB image, C is the third preset threshold, p is the total number of rows of pixels in the RGB image, and q is the total number of columns of pixels in the RGB image.

[0151] Blue photosensitive characteristic parameter: Use the blue component value of each pixel in the RGB image corresponding to the white light image to obtain the blue photosensitive characteristic parameter. Specifically, as Figure 5 shown, it is a schematic flow chart for determining the blue photosensitive characteristic parameter using Method 2, including the following steps:

[0152] Step 501: For any pixel in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel to obtain the total color component value of the pixel;

[0153] Step 502: Add the total color component values of all pixels in the RGB image to obtain the total color component value of the RGB image;

[0154] Step 503: Divide the total color component value of the RGB image by the total blue component value of each pixel in the RGB image to obtain the blue photosensitive characteristic parameter in the color photosensitive characteristic parameters, wherein the total red component value is obtained based on the blue component value of each pixel in the RGB image and a third preset threshold. Among them, the blue photosensitive characteristic parameter B can be obtained through formula (6). av :

[0155]

[0156] wherein, R ij is the red component value of the pixel at the i-th row and j-th column in the RGB image, G ij is the green component value of the pixel at the i-th row and j-th column in the RGB image, Bij is the blue component value of the pixel at the i-th row and j-th column in the RGB image, C is the third preset threshold, p is the total number of rows of pixels in the RGB image, and q is the total number of columns of pixels in the RGB image.

[0157] It should be noted that: the third preset threshold C in this embodiment is 3, but the third preset threshold in this embodiment is not limited, and the specific value of the third preset threshold can be set according to the actual situation.

[0158] Method 3:

[0159] Red photosensitive feature parameter: Based on the green component value and red component value of each pixel in the RGB image corresponding to the white light image, the red photosensitive feature parameter of the white light image is obtained. Specifically, as Figure 6 shown, it is a schematic flow chart for determining the red photosensitive feature parameter using Method 3, including the following steps:

[0160] Step 601: Add up the green component values of each pixel in the RGB image corresponding to the white light image to obtain the total green component value of the RGB image;

[0161] Step 602: Add up the red component values of each pixel in the RGB image to obtain the total red component value of the RGB image;

[0162] Step 603: Divide the total green component value by the total red component value to obtain the red photosensitive feature parameter.

[0163] Specifically, the red photosensitive feature parameter R can be obtained through formula (7) av :

[0164]

[0165] where, R ij is the red component value of the pixel at the i-th row and j-th column in the RGB image, G ij is the green component value of the pixel at the i-th row and j-th column in the RGB image, p is the total number of rows of pixels in the RGB image, and q is the total number of columns of pixels in the RGB image.

[0166] Green photosensitive feature parameter: Set the green photosensitive feature parameter of the white light image to a specified parameter.

[0167] The specified parameter in this embodiment is 1, but this embodiment does not limit the specified parameter, and the specific value of the specified parameter can be set according to the actual situation.

[0168] Blue photosensitive characteristic parameter: Based on the green component value and blue component value of each pixel in the RGB image, the blue photosensitive characteristic parameter of the white light image is obtained. As Figure 7 shown, it is a schematic flowchart for determining the blue photosensitive characteristic parameter using Method 2, including the following steps:

[0169] Step 701: Add the green component values of each pixel in the RGB image corresponding to the white light image to obtain the total green component value of the RGB image;

[0170] Step 702: Add the blue component values of each pixel in the RGB image to obtain the total blue component value of the RGB image;

[0171] Step 703: Divide the total green component value of the RGB image by the total blue component value to obtain the blue photosensitive characteristic parameter.

[0172] Specifically, the blue photosensitive characteristic parameter B can be obtained through formula (8) av :

[0173]

[0174] where G ij is the green component value of the pixel at the i-th row and j-th column in the RGB image, B ij is the blue component value of the pixel at the i-th row and j-th column in the RGB image, p is the total number of rows of pixels in the RGB image, and q is the total number of columns of pixels in the RGB image.

[0175] Method 2: According to the color component values of each pixel in the white light image, the color photosensitive parameter of the white light image is obtained.

[0176] Next, the specific methods for determining the red photosensitive characteristic parameter, green photosensitive characteristic parameter, and blue photosensitive characteristic parameter in the color photosensitive parameter using Method 2 are introduced respectively:

[0177] 1. Red photosensitive characteristic parameter:

[0178] As Figure 8 shown, it is a schematic flowchart for determining the red photosensitive characteristic parameter using Method 2, including the following steps:

[0179] Step 801: For any pixel in the white light image, perform a weighted sum of the color component values of the pixel in the white light image to obtain the total color component value of the pixel;

[0180] Step 802: Add the total color component values of each pixel to obtain the total color component value of the white light image;

[0181] Step 803: Divide the total color component value of the white light image by the total red component value of each pixel point in the white light image to obtain the red photosensitive characteristic parameter, where the total red component value of each pixel point in the white light image is obtained based on the red component value of each pixel point in the white light image and a fourth preset threshold value.

[0182] Specifically, the red photosensitive characteristic parameter R can be obtained through formula (9) av :

[0183]

[0184] where R ij’ is the red component value of the pixel point at the i-th row and j-th column in the white light image, B ij’ is the blue component value of the pixel point at the i-th row and j-th column in the white light image, G ij’ is the green component value of the pixel point at the i-th row and j-th column in the white light image, H is the fourth preset threshold value, and D, E, and F are preset weight values respectively.

[0185] 2. Green photosensitive characteristic parameter:

[0186] As Figure 9 shown, it is a schematic flowchart for determining the green photosensitive characteristic parameter using Method 2, including the following steps:

[0187] Step 901: For any pixel point in the white light image, perform weighted summation on the color component values of the pixel point in the white light image to obtain the total color component value of the pixel point;

[0188] Step 902: Add up the total color component values of all pixel points to obtain the total color component value of the white light image;

[0189] Step 903: Divide the total color component value of the white light image by the total green component value of each pixel point in the white light image to obtain the green photosensitive characteristic parameter, where the total green component value of each pixel point in the white light image is obtained based on the green component value of each pixel point in the white light image and a fourth preset threshold value.

[0190] Specifically, the green photosensitive characteristic parameter G can be obtained through formula (10) av :

[0191]

[0192] where R ij’ is the red component value of the pixel point at the i-th row and j-th column in the white light image, B ij’is the blue component value of the pixel at the i-th row and j-th column in the white light image, G ij’ is the green component value of the pixel at the i-th row and j-th column in the white light image, H is the fourth preset threshold, and D, E, and F are preset weight values respectively.

[0193] 3. Blue photosensitive characteristic parameter:

[0194] As Figure 10 shown, it is a schematic flowchart for determining the blue photosensitive characteristic parameter using Method 2, including the following steps:

[0195] Step 1001: For any pixel in the white light image, perform weighted summation on the color component values of the pixel in the white light image to obtain the total color component value of the pixel;

[0196] Step 1002: Add up the total color component values of all pixels to obtain the total color component value of the white light image;

[0197] Step 1003: Divide the total color component value of the white light image by the total blue component value of each pixel in the white light image to obtain the blue photosensitive characteristic parameter, where the total blue component value of each pixel in the white light image is obtained based on the blue component value of each pixel in the white light image and the fourth preset threshold.

[0198] Specifically, the blue photosensitive characteristic parameter B can be obtained through formula (11) av :

[0199]

[0200] where, R ij’ is the red component value of the pixel at the i-th row and j-th column in the white light image, B ij’ is the blue component value of the pixel at the i-th row and j-th column in the white light image, G ij’ is the green component value of the pixel at the i-th row and j-th column in the white light image, D is the fourth preset threshold, and D, E, and F are preset weight values respectively.

[0201] It should be noted that in this embodiment, the preset weight D is 263, the preset weight E is 660, the preset weight F is 100, and the fourth preset threshold H is 1024. However, the preset weights and the fourth preset threshold in this embodiment are not limited, and the preset weights and the fourth preset threshold can be set according to the actual situation.

[0202] Step 202: Perform pseudo-color conversion on the black-and-white image using the pixel values of each pixel point in the black-and-white image of the human tissue and the color photosensitive characteristic parameters of the white light image to obtain a fluorescence image, where the black-and-white image is an image obtained by irradiating the human tissue with light emitted by a fluorescence light source;

[0203] In one embodiment, as Figure 11 shown, it is a schematic flow chart for obtaining a fluorescence image, including the following steps:

[0204] Step 1101: For any pixel point in the black-and-white image, multiply the pixel value of the pixel point in the black-and-white image by the red photosensitive characteristic parameter to obtain an intermediate red photosensitive characteristic parameter, and multiply the intermediate red photosensitive characteristic parameter by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel point;

[0205] Among them, the preset image adjustment parameter includes a preset fluorescence image enhancement coefficient and a preset fluorescence image adjustment parameter. Specifically, the red component value after pseudo-color conversion of the pixel point can be obtained through formula (12):

[0206] R grayij =Gray ij *k1*R av *k2……(12);

[0207] Among them, R grayij is the red component value after pseudo-color conversion of the pixel point in the i-th row and j-th column, Gray ij is the pixel value of the pixel point in the i-th row and j-th column in the black-and-white image, k1 is the preset fluorescence image enhancement coefficient, k2 is the preset fluorescence image adjustment parameter, and R av is the red photosensitive characteristic parameter.

[0208] Step 1102: Multiply the pixel value of the pixel point in the black-and-white image by the green photosensitive characteristic parameter to obtain an intermediate green photosensitive characteristic parameter, and multiply the intermediate green photosensitive characteristic parameter by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point; specifically, the green component value after pseudo-color conversion of the pixel point can be obtained through formula (13):

[0209] G grayij =Gray ij *k1*G av *k2……(13);

[0210] Among them, G grayij is the green component value after pseudo-color conversion of the pixel point in the i-th row and j-th column, Gray ijis the pixel value of the pixel at the i-th row and j-th column in the black-and-white image, k1 is the preset fluorescence image enhancement coefficient, k2 is the preset fluorescence image adjustment parameter, and G av is the green light-sensitive characteristic parameter.

[0211] Step 1103: Multiply the pixel value of the pixel in the black-and-white image by the blue light-sensitive characteristic parameter to obtain an intermediate blue light-sensitive characteristic parameter, and multiply the intermediate blue light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel; specifically, the blue component value after pseudo-color conversion of the pixel can be obtained through formula (14):

[0212] B grayij = Gray ij * k1 * B av * k2……(14);

[0213] where B grayij is the blue component value after pseudo-color conversion of the pixel at the i-th row and j-th column, Gray ij is the pixel value of the pixel at the i-th row and j-th column in the black-and-white image, k1 is the preset fluorescence image enhancement coefficient, k2 is the preset fluorescence image adjustment parameter, and B av is the blue light-sensitive characteristic parameter.

[0214] Step 1104: Obtain the fluorescence image according to the red component value after pseudo-color conversion, the green component value after pseudo-color conversion, and the blue component value after pseudo-color conversion of each pixel.

[0215] The fluorescence image is a black-and-white image after pseudo-color conversion.

[0216] Step 203: Perform image fusion on the fluorescence image and the white light image to obtain a fluorescence image with color restored.

[0217] In order to improve the accuracy of image fusion, before performing Step 203, it is necessary to perform image registration on the fluorescence image and the white light image. The method of image registration can use the image registration method in the prior art, and the specific image registration method can be set according to the actual situation. This embodiment does not limit the method of image registration here.

[0218] In one embodiment, as Figure 12 shown, is a schematic flow diagram of image fusion, including the following steps:

[0219] Step 1201: For any pixel, based on the red component value after pseudo-color conversion of the pixel in the fluorescence image and the red component value of the pixel in the RGB image of the white light image, obtain the red component value after fusion of the pixel;

[0220] In one embodiment, the red component value after fusion of the pixel is obtained by the following method:

[0221] The red component value after false color conversion of the pixel in the fluorescence image and the red component value of the pixel in the RGB image of the white light image are weighted and summed to obtain a first intermediate fusion red component value. The first intermediate fusion red component value is multiplied by a preset enhancement coefficient to obtain a second intermediate fusion red component value. If the second intermediate fusion red component value is less than a preset red component threshold, the second intermediate fusion red component value is determined as the red component value after fusion of the pixel; otherwise, the red component value after fusion of the pixel is determined as the preset red component threshold. Among them, the second intermediate fusion red component value can be obtained by formula (15):

[0222] R 融ij =a*(R ij *b+(1 - b)R grayij )……(15);

[0223] Among them, R 融ij is the second intermediate fusion red component value of the pixel at the i-th row and j-th column, a is the preset enhancement coefficient, R i is the red component value of the pixel at the i-th row and j-th column in the RGB image of the white light image, and R grayij is the red component value after false color conversion of the pixel in the fluorescence image.

[0224] Step 1202: Based on the green component value after false color conversion of the pixel in the fluorescence image and the green component value of the pixel in the RGB image of the white light image, obtain the green component value after fusion of the pixel.

[0225] In one embodiment, the green component value after fusion of the pixel is obtained by the following method:

[0226] The green component value after false color conversion of the pixel in the fluorescence image and the green component value of the pixel in the RGB image of the white light image are weighted and summed to obtain a first intermediate fusion green component value. The first intermediate fusion green component value is multiplied by a preset enhancement coefficient to obtain a second intermediate fusion green component value. If the second intermediate fusion green component value is less than a preset green component threshold, the second intermediate fusion green component value is determined as the green component value after fusion of the pixel; otherwise, the green component value after fusion of the pixel is determined as the preset green component threshold. Among them, the second intermediate fusion green component value can be obtained by formula (16):

[0227] G 融ij = a * (G ij * b + (1 - b)G grayij )……(16);

[0228] Wherein, G 融ij is the second intermediate fusion green component value of the pixel at the i-th row and j-th column, a is the preset enhancement coefficient, G ij is the green component value of the pixel at the i-th row and j-th column in the RGB image of the white light image, and G grayij is the green component value after pseudo-color conversion of the pixel in the fluorescence image.

[0229] Step 1203: Based on the blue component value after pseudo-color conversion of the pixel in the fluorescence image and the blue component value of the pixel in the RGB image of the white light image, obtain the fused blue component value of the pixel;

[0230] In one embodiment, the fused blue component value of the pixel is obtained in the following manner:

[0231] Perform weighted summation on the blue component value after pseudo-color conversion of the pixel in the fluorescence image and the blue component value of the pixel in the RGB image of the white light image to obtain the first intermediate fusion green component value, multiply the first intermediate fusion blue component value by the preset enhancement coefficient to obtain the second intermediate fusion blue component value. If the second intermediate fusion blue component value is less than the preset blue component threshold, then determine the second intermediate fusion blue component value as the fused blue component value of the pixel; otherwise, determine the fused blue component value of the pixel as the preset blue component threshold. Wherein, the second intermediate fusion blue component value can be obtained through formula (17):

[0232] B 融ij = a * (B ij * b + (1 - b)B grayij )……(17);

[0233] Wherein, B 融ij is the second intermediate fusion blue component value of the pixel at the i-th row and j-th column, a is the preset enhancement coefficient, B ij is the blue component value of the pixel at the i-th row and j-th column in the RGB image of the white light image, and B grayij is the blue component value after pseudo-color conversion of the pixel in the fluorescence image.

[0234] Step 1204: According to the fused red component value, fused green component value, and fused blue component value of each pixel, obtain the fluorescence image with color restored.

[0235] To further understand the technical solution of the present disclosure, the following is a detailed description in conjunction with Figure 13 which may include the following steps:

[0236] Step 1301: Convert the white light image into an RGB image by using an interpolation algorithm, and obtain the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, where the color component values include a red component value, a green component value, and a blue component value;

[0237] Step 1302: For any pixel point in the black and white image, multiply the pixel value of the pixel point in the black and white image by the red light-sensitive characteristic parameter to obtain an intermediate red light-sensitive characteristic parameter, and multiply the intermediate red light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel point;

[0238] Step 1303: Multiply the pixel value of the pixel point in the black and white image by the green light-sensitive characteristic parameter to obtain an intermediate green light-sensitive characteristic parameter, and multiply the intermediate green light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point;

[0239] Step 1304: Multiply the pixel value of the pixel point in the black and white image by the blue light-sensitive characteristic parameter to obtain an intermediate blue light-sensitive characteristic parameter, and multiply the intermediate blue light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel point;

[0240] Step 1305: Obtain the fluorescence image according to the red component value after pseudo-color conversion, the green component value after color conversion, and the blue component value after color conversion of each pixel point;

[0241] Step 1306: For any pixel point, obtain the fused red component value of the pixel point based on the red component value after pseudo-color conversion of the pixel point in the fluorescence image and the red component value of the pixel point in the RGB image of the white light image;

[0242] Step 1307: Obtain the fused green component value of the pixel point based on the green component value after pseudo-color conversion of the pixel point in the fluorescence image and the green component value of the pixel point in the RGB image of the white light image;

[0243] Step 1308: Obtain the fused blue component value of the pixel point based on the blue component value after pseudo-color conversion of the pixel point in the fluorescence image and the blue component value of the pixel point in the RGB image of the white light image;

[0244] Step 1309: Obtain the fluorescence image with color restored based on the fused red component value, fused green component value, and fused blue component value of each pixel point.

[0245] Based on the same inventive concept, the image fusion method of the present disclosure as described above can also be implemented by an image fusion device. The effects of this image fusion device are similar to those of the foregoing method and will not be elaborated here.

[0246] Figure 14 It is a schematic structural diagram of an image fusion device according to an embodiment of the present disclosure.

[0247] As Figure 14 shown, the image fusion device 1400 of the present disclosure may include a color photosensitive feature parameter determination module 1410, a false color conversion module 1420, and an image fusion module 1430.

[0248] The color photosensitive feature parameter determination module 1410 is configured to determine the color photosensitive feature parameters of the white light image based on the white light image of the human tissue, where the white light image is an image obtained by irradiating the human tissue with light emitted by a white light source;

[0249] The false color conversion module 1420 is configured to perform false color conversion on the black and white image by using the pixel values of each pixel point in the black and white image of the human tissue and the color photosensitive feature parameters of the white light image to obtain a fluorescence image, where the black and white image is an image obtained by irradiating the human tissue with light emitted by a fluorescence light source;

[0250] The image fusion module 1430 is configured to perform image fusion on the fluorescence image and the white light image to obtain a fluorescence image with color restored.

[0251] In one embodiment, the color photosensitive feature parameter determination module 1410 is specifically configured to:

[0252] Convert the white light image into an RGB image by using an interpolation algorithm, and obtain the color photosensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, where the color component values include a red component value, a green component value, and a blue component value; or,

[0253] Obtain the color photosensitive parameters of the white light image according to the color component values of each pixel point in the white light image.

[0254] In one embodiment, the color photosensitive feature parameters include a red photosensitive feature parameter, a green photosensitive feature parameter, and a blue photosensitive feature parameter;

[0255] The color photosensitive feature parameter determination module 1410 is further configured to:

[0256] Obtain the red photosensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the red component values of each pixel point in the RGB image corresponding to the white light image; and obtain the green photosensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the green component values of each pixel point in the RGB image; and obtain the blue photosensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the blue component values of each pixel point in the RGB image; or,

[0257] Use the red component values of each pixel point in the RGB image corresponding to the white light image to obtain the red photosensitive characteristic parameter of the white light image; and use the green component values of each pixel point in the RGB image corresponding to the white light image to obtain the green photosensitive characteristic parameter; and use the blue component values of each pixel point in the RGB image corresponding to the white light image to obtain the blue photosensitive characteristic parameter; or,

[0258] Based on the green component values and red component values of each pixel point in the RGB image corresponding to the white light image, obtain the red photosensitive characteristic parameter of the white light image; and set the green photosensitive characteristic parameter of the white light image to a specified parameter; and based on the green component values and blue component values of each pixel point in the RGB image, obtain the blue photosensitive characteristic parameter of the white light image.

[0259] In one embodiment, the color photosensitive characteristic parameter determination module 1410 executes obtaining the red photosensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the red component values of each pixel point in the RGB image corresponding to the white light image, and specifically is used for:

[0260] Add the pixel values of each pixel point in the white light image to obtain a total pixel value; add the red component values of each pixel point in the RGB image corresponding to the white light image to obtain a total red component value, multiply the total red component value by a first preset threshold to obtain an enlarged total red component value, and divide the total pixel value by the enlarged total red component value to obtain the red photosensitive characteristic parameter of the white light image;

[0261] The color photosensitive characteristic parameter determination module 1410 executes obtaining the green photosensitive characteristic parameter of the white light image according to the pixel values of each pixel point in the white light image and the green component values of each pixel point in the RGB image corresponding to the white light image, and specifically is used for:

[0262] Add the green component values of each pixel in the RGB image corresponding to the white light image to obtain the total green component value. Multiply the total green photosensitive characteristic parameter by a second preset threshold to obtain an enlarged total green component value. Divide the total pixel value by the enlarged total green component value to obtain the green photosensitive characteristic parameter of the white light image.

[0263] The color photosensitive characteristic parameter determination module 1410 executes obtaining the blue photosensitive characteristic parameter of the white light image according to the pixel values of each pixel in the white light image and the blue component values of each pixel in the RGB image corresponding to the white light image, specifically for:

[0264] Add the blue component values of each pixel in the RGB image corresponding to the white light image to obtain the total blue component value. Multiply the total blue component value by a first preset threshold to obtain an enlarged total blue component value. Divide the total pixel value by the enlarged total blue component value to obtain the blue photosensitive characteristic parameter of the white light image.

[0265] In one embodiment, the color photosensitive characteristic parameter determination module 1410 executes obtaining the red photosensitive characteristic parameter of the white light image by using the red component values of each pixel in the RGB image corresponding to the white light image, specifically for:

[0266] For any pixel in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel to obtain the total color component value of the pixel. Add the total color component values of each pixel in the RGB image to obtain the total color component value of the RGB image. Divide the total color component value of the RGB image by the total red component value of each pixel in the RGB image to obtain the red photosensitive characteristic parameter in the color photosensitive characteristic parameter, where the total red component value is obtained based on the red component values of each pixel in the RGB image and a third preset threshold;

[0267] The color photosensitive characteristic parameter determination module 1410 executes obtaining the green photosensitive characteristic parameter by using the green component values of each pixel in the RGB image corresponding to the white light image, specifically for:

[0268] Divide the total color component value of the RGB image by the total green component value of each pixel in the RGB image to obtain the green photosensitive characteristic parameter in the color photosensitive characteristic parameter; where the total red component value is obtained based on the green component values of each pixel in the RGB image and a third preset threshold;

[0269] The color light-sensing characteristic parameter determination module 1410 executes obtaining the blue light-sensing characteristic parameter by using the blue component values of each pixel in the RGB image corresponding to the white light image, specifically for:

[0270] Dividing the total color component value of the RGB image by the total blue component value of each pixel in the RGB image to obtain the blue light-sensing characteristic parameter in the color light-sensing characteristic parameter, where the total red component value is obtained based on the blue component value of each pixel in the RGB image and a third preset threshold.

[0271] In one embodiment, the color light-sensing characteristic parameter determination module 1410 executes obtaining the red light-sensing characteristic parameter of the white light image based on the green component value and the red component value of each pixel in the RGB image corresponding to the white light image, specifically for:

[0272] Adding the green component values of each pixel in the RGB image corresponding to the white light image to obtain the total green component value of the RGB image, and adding the red component values of each pixel in the RGB image to obtain the total red component value of the RGB image, and dividing the total green component value by the total red component value to obtain the red light-sensing characteristic parameter;

[0273] The color light-sensing characteristic parameter determination module 1410 executes obtaining the blue light-sensing characteristic parameter of the white light image based on the green component value and the blue component value of each pixel in the RGB image, specifically for:

[0274] Adding the blue component values of each pixel in the RGB image to obtain the total blue component value of the RGB image, and dividing the total green component value of the RGB image by the total blue component value to obtain the blue light-sensing characteristic parameter.

[0275] In one embodiment, the color light-sensing characteristic parameter includes a red light-sensing characteristic parameter, a green light-sensing characteristic parameter, and a blue light-sensing characteristic parameter;

[0276] The color light-sensing characteristic parameter determination module 1410 executes obtaining the color light-sensing parameter of the white light image according to the color component values of each pixel in the white light image, specifically for:

[0277] For any pixel point in the white light image, the color component values of the pixel point in the white light image are weighted and summed to obtain the total color component value of the pixel point, and the total color component values of all pixel points are added to obtain the total color component value of the white light image. The total color component value of the white light image is divided by the total red component value of each pixel point in the white light image to obtain the red light-sensitive characteristic parameter, where the total red component value of each pixel point in the white light image is obtained based on the red component value of each pixel point in the white light image and a fourth preset threshold; and,

[0278] The total color component value of the white light image is divided by the total green component value of each pixel point in the white light image to obtain the green light-sensitive characteristic parameter, where the total green component value of each pixel point in the white light image is obtained based on the green component value of each pixel point in the white light image and a fourth preset threshold; and,

[0279] The total color component value of the white light image is divided by the total blue component value of each pixel point in the white light image to obtain the blue light-sensitive characteristic parameter, where the total blue component value of each pixel point in the white light image is obtained based on the blue component value of each pixel point in the white light image and a fourth preset threshold.

[0280] In one embodiment, the pseudo-color conversion module 1420 is specifically configured to:

[0281] For any pixel point in the black-and-white image, the pixel value of the pixel point in the black-and-white image is multiplied by the red light-sensitive characteristic parameter to obtain an intermediate red light-sensitive characteristic parameter, and the intermediate red light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel point; and,

[0282] The pixel value of the pixel point in the black-and-white image is multiplied by the green light-sensitive characteristic parameter to obtain an intermediate green light-sensitive characteristic parameter, and the intermediate green light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point; and,

[0283] The pixel value of the pixel point in the black-and-white image is multiplied by the blue light-sensitive characteristic parameter to obtain an intermediate blue light-sensitive characteristic parameter, and the intermediate blue light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel point;

[0284] According to the red component value, green component value, and blue component value after pseudo-color conversion of each pixel point, the fluorescence image is obtained.

[0285] After introducing an image fusion method and device according to an exemplary embodiment of the present disclosure, next, an endoscope device according to another exemplary embodiment of the present disclosure will be introduced.

[0286] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "system" here.

[0287] In some possible embodiments, the endoscope device according to the present disclosure may at least include at least one processor and at least one computer storage medium. Among them, the computer storage medium stores program code, and when the program code is executed by the processor, it causes the processor to execute the steps in the image fusion method according to various exemplary embodiments of the present disclosure described above in this specification. For example, the processor can execute steps such as Figure 2 shown in step 201-203.

[0288] Next, refer to Figure 15 to describe the endoscope device 1500 according to this embodiment of the present disclosure. Figure 15 The endoscope device 1500 shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0289] As Figure 15 shown, the endoscope device 1500 is presented in the form of a general endoscope device. The components of the endoscope device 1500 may include but are not limited to: the above-mentioned at least one processor 1501, the above-mentioned at least one computer storage medium 1502, and a bus 1503 connecting different system components (including the computer storage medium 1502 and the processor 1501).

[0290] The bus 1503 represents one or more of several types of bus structures, including a computer storage medium bus or a computer storage medium controller, a peripheral bus, a processor, or a local bus using any bus structure in a variety of bus structures.

[0291] The computer storage medium 1502 may include a readable medium in the form of a volatile computer storage medium, such as a random access computer storage medium (RAM) 1521 and / or a cache storage medium 1522, and may further include a read-only computer storage medium (ROM) 1523.

[0292] The computer storage medium 1502 may also include a program / utilities 1525 having a set (at least one) of program modules 1524. Such program modules 1524 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0293] The endoscope device 1500 may also communicate with one or more external devices 1504 (such as a keyboard, a pointing device, etc.), and may also communicate with one or more devices that enable a user to interact with the endoscope device 1500, and / or communicate with any device (such as a router, a modem, etc.) that enables the endoscope device 1500 to communicate with one or more other endoscope devices. Such communication may be performed through an input / output (I / O) interface 1505. Moreover, the endoscope device 1500 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 1506. As shown in the figure, the network adapter 1506 communicates with other modules for the endoscope device 1500 through a bus 1503. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in conjunction with the endoscope device 1500, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0294] In some possible implementation manners, various aspects of an image fusion method provided by the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product runs on a computer device, the program code is used to cause the computer device to execute the steps in the image fusion method according to various exemplary implementation manners of the present disclosure described above in this specification.

[0295] The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access computer storage medium (RAM), a read-only computer storage medium (ROM), an erasable programmable read-only computer storage medium (EPROM or flash memory), an optical fiber, a portable compact disk read-only computer storage medium (CD-ROM), an optical computer storage medium, a magnetic computer storage medium, or any suitable combination of the above.

[0296] The image fusion program product according to an embodiment of the present disclosure may be embodied on a portable compact disc read-only computer storage medium (CD-ROM) and include program code, and may run on an endoscopic device. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0297] A readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0298] The program code contained on the readable medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0299] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's endoscopic device, partially on the user's device, executed as a stand-alone software package, partially on the user's endoscopic device and partially on a remote endoscopic device, or entirely on a remote endoscopic device or server. In the case of a remote endoscopic device, the remote endoscopic device may be connected to the user's endoscopic device through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external endoscopic device (e.g., by using an Internet service provider to connect through the Internet).

[0300] It should be noted that although several modules of the apparatus are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more of the above-described modules may be embodied in one module. Conversely, the features and functions of one module described above may be further divided and embodied by a plurality of modules.

[0301] In addition, although the operations of the method of the present disclosure are described in a specific order in the drawings, this is not required or implied that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step for execution, and / or one step may be decomposed into multiple steps for execution.

[0302] Those skilled in the art should understand that the embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic computer storage media, CD-ROM, optical computer storage media, etc.) that contain computer-usable program code.

[0303] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices produce a device for realizing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0304] These computer program instructions can also be stored in a computer-readable computer storage medium that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable computer storage medium produce a manufactured article including an instruction device that realizes the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0305] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in one or more flows of the flowchart and / or one or more blocks of the block diagram.

[0306] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalent technologies, the present disclosure is also intended to cover these modifications and variations.

Claims

1. An endoscope device, characterized in that, Comprising a storage unit and a processor, wherein: The storage unit is configured to store a white light image and a black-and-white image of a human tissue, wherein the white light image is an image obtained by irradiating the human tissue with light emitted by a white light source, and the black-and-white image is an image obtained by irradiating the human tissue with light emitted by a fluorescent light source; The processor is configured to: Based on the white light image, determine the color light-sensitive characteristic parameters of the white light image, wherein the color light-sensitive characteristic parameters include red light-sensitive characteristic parameters, green light-sensitive characteristic parameters, and blue light-sensitive characteristic parameters; For any pixel point in the black-and-white image, multiply the pixel value of the pixel point in the black-and-white image by the red light-sensitive characteristic parameter to obtain an intermediate red light-sensitive characteristic parameter, and multiply the intermediate red light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel point; and, Multiply the pixel value of the pixel point in the black-and-white image by the green light-sensitive characteristic parameter to obtain an intermediate green light-sensitive characteristic parameter, and multiply the intermediate green light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point; and, Multiply the pixel value of the pixel point in the black-and-white image by the blue light-sensitive characteristic parameter to obtain an intermediate blue light-sensitive characteristic parameter, and multiply the intermediate blue light-sensitive characteristic parameter by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel point; Obtain a fluorescence image according to the red component value, the green component value, and the blue component value after pseudo-color conversion of each pixel point; Perform image fusion on the fluorescence image and the white light image to obtain a fluorescence image with restored color.

2. The endoscope device according to claim 1, characterized in that, When the processor executes the operation of determining the color light-sensitive characteristic parameters of the white light image based on the white light image, it is specifically configured to: Convert the white light image into an RGB image by using an interpolation algorithm, and obtain the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, wherein the color component values include red component values, green component values, and blue component values; or, Obtain the color light-sensitive parameters of the white light image according to the color component values of each pixel point in the white light image.

3. The endoscope device according to claim 2, characterized in that, When the processor executes the operation of obtaining the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, it is specifically configured to: Obtain the red light-sensitive characteristic parameters of the white light image according to the pixel values of each pixel point in the white light image and the red component values of each pixel point in the RGB image corresponding to the white light image; And obtain the green light-sensitive characteristic parameters of the white light image according to the pixel values of each pixel point in the white light image and the green component values of each pixel point in the RGB image; And obtain the blue light-sensitive characteristic parameters of the white light image according to the pixel values of each pixel point in the white light image and the blue component values of each pixel point in the RGB image; Or, Obtain the red photosensitive characteristic parameter of the white light image by using the red component values of each pixel in the RGB image corresponding to the white light image; And obtain the green photosensitive characteristic parameter by using the green component values of each pixel in the RGB image corresponding to the white light image; And obtain the blue photosensitive characteristic parameter by using the blue component values of each pixel in the RGB image corresponding to the white light image; Or, Based on the green component values and red component values of each pixel in the RGB image corresponding to the white light image, obtain the red photosensitive characteristic parameter of the white light image; And set the green photosensitive characteristic parameter of the white light image to a specified parameter; and based on the green component values and blue component values of each pixel in the RGB image, obtain the blue photosensitive characteristic parameter of the white light image.

4. The endoscope device according to claim 3, characterized in that, The processor executes obtaining the red photosensitive characteristic parameter of the white light image according to the pixel values of each pixel in the white light image and the red component values of each pixel in the RGB image corresponding to the white light image, and is specifically configured to: Add the pixel values of each pixel in the white light image to obtain a total pixel value; add the red component values of each pixel in the RGB image corresponding to the white light image to obtain a total red component value, multiply the total red component value by a first preset threshold to obtain an enlarged total red component value, and divide the total pixel value by the enlarged total red component value to obtain the red photosensitive characteristic parameter of the white light image; The processor executes obtaining the green photosensitive characteristic parameter of the white light image according to the pixel values of each pixel in the white light image and the green component values of each pixel in the RGB image corresponding to the white light image, and is specifically configured to: Add the green component values of each pixel in the RGB image corresponding to the white light image to obtain a total green component value, multiply the total green photosensitive characteristic parameter by a second preset threshold to obtain an enlarged total green component value, and divide the total pixel value by the enlarged total green component value to obtain the green photosensitive characteristic parameter of the white light image; The processor executes obtaining the blue photosensitive characteristic parameter of the white light image according to the pixel values of each pixel in the white light image and the blue component values of each pixel in the RGB image corresponding to the white light image, and is specifically configured to: Add the blue component values of each pixel in the RGB image corresponding to the white light image to obtain a total blue component value, multiply the total blue component value by a first preset threshold to obtain an enlarged total blue component value, and divide the total pixel value by the enlarged total blue component value to obtain the blue photosensitive characteristic parameter of the white light image.

5. The endoscope device according to claim 3, characterized in that, The processor executes obtaining the red photosensitive characteristic parameter of the white light image by using the red component values of each pixel in the RGB image corresponding to the white light image, and is specifically configured to: For any pixel point in the RGB image corresponding to the white light image, add the red component value, green component value, and blue component value of the pixel point to obtain the total color component value of the pixel point. Add the total color component values of all pixel points in the RGB image to obtain the total color component value of the RGB image. Divide the total color component value of the RGB image by the total red component value of all pixel points in the RGB image to obtain the red light-sensing characteristic parameter in the color light-sensing characteristic parameters, where the total red component value is obtained based on the red component value of each pixel point in the RGB image and a third preset threshold; The processor executes obtaining the green light-sensing characteristic parameter by using the green component value of each pixel point in the RGB image corresponding to the white light image, and is specifically configured as: Divide the total color component value of the RGB image by the total green component value of all pixel points in the RGB image to obtain the green light-sensing characteristic parameter in the color light-sensing characteristic parameters; where the total red component value is obtained based on the green component value of each pixel point in the RGB image and a third preset threshold; The processor executes obtaining the blue light-sensing characteristic parameter by using the blue component value of each pixel point in the RGB image corresponding to the white light image, and is specifically configured as: Divide the total color component value of the RGB image by the total blue component value of all pixel points in the RGB image to obtain the blue light-sensing characteristic parameter in the color light-sensing characteristic parameters, where the total red component value is obtained based on the blue component value of each pixel point in the RGB image and a third preset threshold.

6. The endoscope device according to claim 3, characterized in that, The processor executes obtaining the red light-sensing characteristic parameter of the white light image based on the green component value and red component value of each pixel point in the RGB image corresponding to the white light image, and is specifically configured as: Add the green component values of all pixel points in the RGB image corresponding to the white light image to obtain the total green component value of the RGB image, and add the red component values of all pixel points in the RGB image to obtain the total red component value of the RGB image, and divide the total green component value by the total red component value to obtain the red light-sensing characteristic parameter; The processor executes obtaining the blue light-sensing characteristic parameter of the white light image based on the green component value and blue component value of each pixel point in the RGB image, and is specifically configured as: Add the blue component values of all pixel points in the RGB image to obtain the total blue component value of the RGB image, and divide the total green component value of the RGB image by the total blue component value to obtain the blue light-sensing characteristic parameter.

7. The endoscope device according to claim 2, characterized in that, The color light-sensing characteristic parameters include a red light-sensing characteristic parameter, a green light-sensing characteristic parameter, and a blue light-sensing characteristic parameter; The processor executes obtaining the color light-sensing parameter of the white light image according to the color component values of each pixel point in the white light image, and is specifically configured as: For any pixel point in the white light image, the color component values of the pixel point in the white light image are weighted and summed to obtain the total color component value of the pixel point, and the total color component values of all pixel points are added to obtain the total color component value of the white light image. The total color component value of the white light image is divided by the total red component value of each pixel point in the white light image to obtain the red light-sensitive characteristic parameter, where the total red component value of each pixel point in the white light image is obtained based on the red component value of each pixel point in the white light image and a fourth preset threshold; and, The total color component value of the white light image is divided by the total green component value of each pixel point in the white light image to obtain the green light-sensitive characteristic parameter, where the total green component value of each pixel point in the white light image is obtained based on the green component value of each pixel point in the white light image and a fourth preset threshold; and, The total color component value of the white light image is divided by the total blue component value of each pixel point in the white light image to obtain the blue light-sensitive characteristic parameter, where the total blue component value of each pixel point in the white light image is obtained based on the blue component value of each pixel point in the white light image and a fourth preset threshold.

8. An image fusion method, characterized in that, The method includes: Based on the white light image of the human tissue, determining the color light-sensitive characteristic parameters of the white light image, where the white light image is an image obtained by irradiating the human tissue with light emitted by a white light source, and the color light-sensitive characteristic parameters include a red light-sensitive characteristic parameter, a green light-sensitive characteristic parameter, and a blue light-sensitive characteristic parameter; For any pixel point in the black and white image, the pixel value of the pixel point in the black and white image is multiplied by the red light-sensitive characteristic parameter to obtain an intermediate red light-sensitive characteristic parameter, and the intermediate red light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the red component value after pseudo-color conversion of the pixel point, where the black and white image is an image obtained by irradiating the human tissue with light emitted by a fluorescent light source; and, The pixel value of the pixel point in the black and white image is multiplied by the green light-sensitive characteristic parameter to obtain an intermediate green light-sensitive characteristic parameter, and the intermediate green light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the green component value after pseudo-color conversion of the pixel point; and, The pixel value of the pixel point in the black and white image is multiplied by the blue light-sensitive characteristic parameter to obtain an intermediate blue light-sensitive characteristic parameter, and the intermediate blue light-sensitive characteristic parameter is multiplied by a preset image adjustment parameter to obtain the blue component value after pseudo-color conversion of the pixel point; According to the red component value, green component value, and blue component value after pseudo-color conversion of each pixel point, a fluorescence image is obtained; The fluorescence image and the white light image are fused to obtain a fluorescence image with color restored.

9. The method according to claim 8, characterized in that, The determining the color light-sensitive characteristic parameters of the white light image based on the white light image of the human tissue includes: Convert the white light image into an RGB image by using an interpolation algorithm, and obtain the color light-sensitive parameters of the white light image based on the color component values of each pixel point in the RGB image, where the color component values include a red component value, a green component value, and a blue component value; or, Obtain the color light-sensitive parameters of the white light image according to the color component values of each pixel point in the white light image.

Images