Image fusion method and device, computer device and readable storage medium

Abstract

The application relates to an image fusion method and device, computer equipment and a readable storage medium. The method comprises the following steps: acquiring a visible light image and a fluorescent image; determining an overexposure area in an overlapping area of the visible light image and the fluorescent image; filtering the overexposure area to obtain a to-be-fused fluorescent image; and fusing the to-be-fused fluorescent image with the visible light image to obtain a target image. By determining the overexposure area in the overlapping area of the visible light image and the fluorescent image and filtering the overexposure area to obtain the to-be-fused fluorescent image, the exposure area in the to-be-fused fluorescent image can be effectively processed. After the to-be-fused fluorescent image is fused with the visible light image to obtain the target image, the imaging quality of the target image is improved, the effective information in the collected image can be more accurately acquired, the possibility of misjudgment of a lesion caused by low imaging quality is further reduced, and the accuracy of disease diagnosis based on endoscope imaging is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image fusion method, apparatus, computer device, and readable storage medium. Background Technology

[0002] With advancements in medical technology and the widespread adoption of advanced medical equipment, endoscopes have found extensive applications in the medical field. Currently, endoscopic imaging primarily includes visible light imaging based on color visible light sensors and fluorescence imaging based on near-infrared cameras that acquire signals beyond the visible light wavelength. In the generated endoscopic images, visible light cameras can only produce visible light images and cannot capture image information beyond the visible spectrum, while fluorescence imaging technology can reveal details that are difficult to observe clearly under visible light. Therefore, the fusion imaging technology of visible light and fluorescence images has emerged.

[0003] However, in traditional fusion imaging techniques, the overexposed areas in the fluorescence image are not considered during the fusion of visible light and fluorescence images. This may result in overexposed areas in the fused image, leading to a decrease in the quality of the fused image and potentially causing misdiagnosis of lesion areas. Summary of the Invention

[0004] Therefore, it is necessary to provide an image fusion method, apparatus, computer device, and readable storage medium that can improve the quality of fused images, addressing the aforementioned technical problems.

[0005] Firstly, this application provides an image fusion method. The method includes:

[0006] Acquire visible light and fluorescence images;

[0007] Identify the overexposed areas in the overlapping region of the visible light image and the fluorescence image;

[0008] The overexposed areas are filtered to obtain the fluorescence image to be fused;

[0009] The fluorescence image to be fused is fused with the visible light image to obtain the target image.

[0010] In one embodiment, prior to filtering the overexposed region to obtain the fluorescence image to be fused, the following steps are included:

[0011] The dark areas in the visible light image are determined based on a preset dark area threshold.

[0012] The overlapping area between the dark area and the fluorescence image is determined based on a preset overlap threshold;

[0013] A fluorescence superimposed image is obtained based on the overlapping region and its corresponding first superimposed weight, the fluorescence image and its corresponding second superimposed weight, and a preset superimposed threshold.

[0014] In one embodiment, filtering the overexposed area to obtain the fluorescence image to be fused includes:

[0015] The overexposed areas are filtered to obtain a filtered image;

[0016] The fluorescence image to be fused is obtained based on the filtered image, the fluorescence superimposed image, and the preset fusion threshold.

[0017] In one embodiment, the process prior to fusing the fluorescence image to be fused with the visible light image includes:

[0018] The visible light image is processed using a histogram equalization algorithm to obtain a histogram equalized image.

[0019] The histogram-equalized image is filtered to obtain a visible light filtered image.

[0020] In one embodiment, after filtering the histogram-equalized image to obtain a visible light filtered image, the method further includes:

[0021] A visible light superimposed image is obtained based on the histogram equalized image, the visible light filtered image, and the superposition ratio;

[0022] A visible light fused image is obtained based on the visible light superimposed image and the corresponding first fusion threshold, the preset visible light fusion threshold and the corresponding second fusion threshold;

[0023] The visible light fusion image is processed using a pseudo-color algorithm to obtain the visible light image to be fused.

[0024] In one embodiment, fusing the fluorescence image to be fused with the visible light image to obtain the target image includes:

[0025] The target image is obtained based on the fluorescence image to be fused, the visible light image, and the fusion weights.

[0026] In one embodiment, the process of obtaining the target image based on the fluorescence image to be fused, the visible light image, and the fusion weights includes:

[0027] In response to user instructions, the fusion weights are determined.

[0028] Secondly, this application also provides an image fusion apparatus. The apparatus includes:

[0029] The image acquisition module is used to acquire visible light images and fluorescence images;

[0030] An overexposed area determination module is used to determine the overexposed areas in the overlapping area of ​​the visible light image and the fluorescence image;

[0031] The image processing module is used to filter the overexposed areas to obtain the fluorescence image to be fused.

[0032] An image fusion module is used to fuse the fluorescence image to be fused with the visible light image to obtain a target image.

[0033] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in any one of the first aspects above.

[0034] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0035] The aforementioned image fusion method, apparatus, computer equipment, and readable storage medium effectively process exposed areas in the fluorescence image by identifying overexposed regions within the overlapping area of ​​the visible light image and the fluorescence image, and filtering these overexposed regions to obtain the fluorescence image to be fused. The target image is then obtained by fusing the fluorescence image to be fused with the visible light image, improving the imaging quality of the target image. This allows for a more accurate display of the effective information from the images acquired by the endoscope, further reducing the possibility of misdiagnosis of lesions due to low imaging quality, and improving the accuracy of disease diagnosis based on endoscopic imaging.

[0036] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0037] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0038] Figure 1 This is an application environment diagram of the image fusion method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating an image fusion method in one embodiment;

[0040] Figure 3 This is a flowchart illustrating an image fusion method in a specific embodiment;

[0041] Figure 4 This is a structural block diagram of an image fusion device in one embodiment. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0043] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0044] The terms “module”, “unit”, etc., used below refer to a combination of software and / or hardware that can perform a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in hardware, implementation in software, or a combination of software and hardware, is also possible and contemplated.

[0045] The image fusion method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated on server 104 or placed in the cloud or on other network servers. In this embodiment, visible light and fluorescence images can be acquired by terminal 102 and transmitted to server 104 via a communication network. Server 104 stores the visible light and fluorescence images in a data storage system. Server 104 determines the overexposed areas in the overlapping region of the visible light and fluorescence images; filters the overexposed areas to obtain a fluorescence image to be fused; and fuses the fluorescence image to be fused with the visible light image to obtain a target image. It is understood that the above steps can also be completed by terminal 102 and server 104 in cooperation. For example, in some embodiments, terminal 102 can fuse the fluorescence image to be fused with the visible light image to obtain a target image, and then send the target image to server 104 and store it in a data storage system. This application does not limit this. Server 104 can be implemented using a standalone server or a server cluster composed of multiple servers.

[0046] In the embodiments of this application, such as Figure 2 As shown, an image fusion method is provided. This embodiment illustrates the method applied to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0047] S201: Acquire visible light images and fluorescence images.

[0048] In this embodiment, the visible light image and fluorescence image can be acquired by an endoscopic imaging system. The endoscopic imaging system is prior art and may include components such as a light source, camera device, host computer, and display. In some embodiments, the camera device of the endoscopic imaging system may include a white light camera and a fluorescence camera. The visible light image may include a white light image acquired by the white light camera, and the fluorescence image may include a fluorescence image acquired by irradiating human tissue containing a contrast agent with a preset wavelength fluorescence light source. In other embodiments, the visible light image and fluorescence image can also be acquired by the camera device through an optical path, and then the image can be divided into a visible light image and a fluorescence image using a spectrophotometer. The visible light image and fluorescence image can be acquired simultaneously, including simultaneously acquiring visible light images and fluorescence images for the same area. In this embodiment, the visible light image may include an RGB image or a YUV image, and the fluorescence image may include a grayscale image, a color image with saturation exceeding a preset threshold, or a YUV image, etc., wherein the grayscale image and the YUV image can be converted from an RGB image.

[0049] S203: Determine the overexposed area in the overlapping region of the visible light image and the fluorescence image.

[0050] In this embodiment, the overlapping region may include the overlapping region after registration of the visible light image and the fluorescence image of the same target location. Registration refers to aligning the physical pixels of the fluorescence image and the visible light image. It is understood that the fluorescence image and the visible light image used for registration should be images acquired for the same target location, and the target features should be consistent in direction. For example, if the target feature of a lesion region is approximately trapezoidal, regardless of whether the acquisition directions of the fluorescence image and the visible light image are consistent, the direction of the lesion region in the images should be kept consistent during registration. For instance, during registration, the lesion region in both the fluorescence image and the visible light image should be a trapezoid with the shorter base on top and the longer base on the bottom.

[0051] In this embodiment, overexposure refers to a fluorescence brightness value in the overlapping region that is significantly higher than a preset fluorescence brightness value and approaches saturation. Specifically, overexposure means that the fluorescence brightness value in the overlapping region is higher than a preset fluorescence brightness threshold. Fluorescence images acquired under overexposure will result in low image quality. Typically, overexposure is caused by reflections from the target area, the light source, or the instrument. Therefore, the overexposure area is a "false fluorescence area," not a true fluorescence area. If the overexposure area is not removed, it will affect the authenticity of the fluorescence image, potentially leading to misdiagnosis by doctors. Therefore, if an overexposure area exists in the fluorescence image, the quality of the target image after fusion of the fluorescence image and the visible light image will be low. It is necessary to identify the overexposure area in the fluorescence image and remove it accordingly. Of course, in some embodiments, the overexposure area in the fused target image can also be identified, as well as the overexposure area in the overlapping region of the visible light image and the fluorescence image. In other embodiments, overexposure may also exist in the visible light image. Therefore, the overexposure areas in the visible light image and the fluorescence image can be identified separately, thereby identifying the overexposure area in the overlapping region.

[0052] In one embodiment of this application, the brightness parameter of the visible light image can be represented as Y_VIS, and the brightness parameter of the fluorescence image can be represented as Y_IR. Then, the brightness parameter Y_VIS_highlight of the overexposed area in the visible light image and the brightness parameter Y_IR_highlight of the overexposed area in the fluorescence image can be determined by equation (1):

[0053]

[0054] Where VIS_exposure_threshold is the preset threshold for overexposed areas in visible light images, and IR_exposure_threshold is the preset threshold for overexposed areas in fluorescence images. The overexposed area EX_AREA can then be determined using equation (2):

[0055]

[0056] S205: Filter the overexposed area to obtain the fluorescence image to be fused.

[0057] S207: The fluorescence image to be fused is fused with the visible light image to obtain the target image.

[0058] In this embodiment, filtering the overexposed areas can achieve smooth image denoising and reduce the impact of interference signals on image quality. The filtering can be implemented using a low-pass filter from the prior art, which will not be elaborated here. After filtering the overexposed areas, a filtered fluorescence image to be fused can be obtained, which includes the filtered overexposed areas.

[0059] In one embodiment of this application, the overexposed region in the overlapping region can be Gaussian filtered using equations (3) and (4) to obtain the filtered overexposed region EX_AREA_OUT:

[0060]

[0061] Where EX_AREA1 is a process variable, and kernel2 is the convolution kernel in the Gaussian filtering of the overexposed region.

[0062] In this embodiment, by filtering the overexposed areas, the overexposed areas in the fluorescence image to be fused are transformed into filtered overexposed areas, which can effectively reduce the impact of exposure on the imaging quality of the fluorescence image to be fused. Furthermore, fusing the fluorescence image to be fused with the visible light image yields the target image. In some embodiments, the fluorescence image and the visible light image can be registered and then directly fused to obtain the target image. In other embodiments, at least one pseudo-color can be mixed into the visible light image and then fused with the registered fluorescence image to be fused, making the fluorescent colored areas clearer and more natural in the target image, thus improving the recognition of the target area.

[0063] The image fusion method provided in this application identifies overexposed areas in the overlapping region of a visible light image and a fluorescence image, and filters these overexposed areas to obtain a fluorescence image to be fused. This effectively processes the exposed areas in the fluorescence image to be fused. The target image is then obtained by fusing the fluorescence image to be fused with the visible light image, improving the imaging quality of the target image. This allows for a more accurate display of the effective information from the images acquired by the endoscope, further reducing the possibility of misdiagnosis of lesions due to low imaging quality, and improving the accuracy of disease diagnosis based on endoscopic imaging.

[0064] To further improve the imaging quality of the fused target image, the visible light image can also be processed accordingly in this embodiment of the application. Before fusing the fluorescence image to be fused with the visible light image in step S207, the following steps are included:

[0065] S301: The visible light image is processed based on the histogram equalization algorithm to obtain a histogram equalized image.

[0066] S303: Filter the histogram equalized image to obtain a visible light filtered image.

[0067] In this embodiment, the histogram equalization optimizes the brightness distribution in a visible light image, thereby enhancing the contrast of the target area without affecting the overall contrast, which is beneficial for improving the recognition of the target area. The histogram equalization algorithm can refer to relevant algorithms in the prior art, and will not be elaborated here. After processing the visible light image to obtain a histogram equalized image, the histogram equalized image can be filtered to obtain a visible light filtered image, further reducing the impact of interference signals on the imaging quality of the visible light image.

[0068] In one embodiment of this application, the brightness parameter Y_VIS of the visible light image can be histogram equalized using equation (5) to obtain the brightness parameter Y_VIS_histeq of the histogram equalized image:

[0069]

[0070] Where histeq() is the histogram algorithm function. Furthermore, the brightness parameter Y_VIS_histeq of the histogram-equalized image can be filtered using equation (6) to obtain the brightness parameter Y_GAUSS of the visible light filtered image:

[0071]

[0072] Where Y1 is the process variable and kernel1 is the convolution kernel in the Gaussian filtering of the visible light image.

[0073] To further improve the recognition accuracy of the fused target image, in this embodiment of the application, pseudo-color processing can be performed on the visible light image. After filtering the histogram equalization image in step S205 to obtain the visible light filtered image, the method further includes:

[0074] S401: A visible light superimposed image is obtained based on the histogram equalized image, the visible light filtered image, and the superposition ratio.

[0075] S403: A visible light fused image is obtained based on the visible light superimposed image and the corresponding first fusion threshold, the preset visible light fusion threshold and the corresponding second fusion threshold.

[0076] S405: Perform pseudo-color processing on the visible light fusion image based on the pseudo-color algorithm to obtain the visible light image to be fused.

[0077] In one embodiment of this application, after the visible light image undergoes histogram equalization and Gaussian filtering, the brightness parameter Y_VIS_FINAL of the visible light superimposed image can be obtained according to equation (7):

[0078]

[0079] Where Y_VIS_WEIGHT is the overlay ratio, Y_GAUSS is the brightness parameter of the visible light filtered image, and Y_VIS_histeq is the brightness parameter of the histogram equalized image. Therefore, the brightness parameter Y_FINAL of the visible light fused image can be obtained according to equation (8):

[0080]

[0081] Where Y_VIS_FINAL is the brightness parameter of the visible light overlay image, Y_IR-IR_threshold2 is the preset visible light fusion threshold, Y_WEIGHT1 is the first fusion threshold corresponding to the visible light overlay image, and Y_WEIGHT2 is the second fusion threshold corresponding to the preset visible light fusion threshold.

[0082] In this embodiment of the application, by performing proportional superposition and fusion processing on the visible light image, the details of the fused target image can be made clearer and easier to distinguish, and the false color area of ​​the subsequent visible light image can be made more natural, avoiding the appearance of a coating or oil painting effect.

[0083] After obtaining the visible light fused image, pseudo-color processing can be performed on the visible light fused image using equation (9) to obtain the visible light image to be fused, wherein the visible light image to be fused includes a YUV (a color coding method) image. The visible light image to be fused can be obtained from equations (1), (2), and (3):

[0084]

[0085] Wherein, color_map is a set of n rows and 3 columns of matrix, the first column is the Y parameter, the second column is the U parameter, and the third column is the V parameter. The matrix is ​​used to generate pseudo-color YUV image. color_map(Y_FINAL).[1] represents the Y parameter value corresponding to Y_FINAL in the first column of the matrix. The Y parameter value is represented as Y_IR_color. Similarly, color_map(Y_FINAL).[2] and color_map(Y_FINAL).[3] represent the U and V parameter values ​​U_IR_color and V_IR_color in the second and third columns of the matrix, respectively. The color_map matrix includes attributes such as luminance information and chrominance information.

[0086] In this embodiment, by processing the brightness and chromaticity information of the visible light image, the colors of the visible light image to be fused are made more natural and clear, and the information features of the original fluorescence image are not affected during image fusion. This also avoids the appearance of a coating or oil painting effect in the fused target image. Through pseudo-color processing of the visible light image, the recognizability of the fused target image is further improved, i.e., the quality of the fused target image is enhanced, thereby improving the efficiency and accuracy of disease diagnosis based on the target image.

[0087] In this embodiment of the application, the fluorescence image can also be superimposed on the visible light image to make the details in the fluorescence image more obvious and clear. Before filtering the overexposed area in step S205 to obtain the fluorescence image to be fused, the following steps are included:

[0088] S501: Determine the dark area region in the visible light image based on a preset dark area threshold.

[0089] S503: Determine the overlapping area between the dark area and the fluorescence image based on a preset overlap threshold.

[0090] S505: A fluorescence superimposed image is obtained based on the overlapping region and the corresponding first superimposed weight, the fluorescence image and the corresponding second superimposed weight, and a preset superimposed threshold.

[0091] In this embodiment, the dark area includes the corresponding region in the visible light image of a light-absorbing region. For example, if the target area is a hemorrhage area or a red part, which are light-absorbing areas, the corresponding region in the visible light image generated from the target area will appear as a dark area. The dark area affects the quality of the fused target image. Therefore, it is necessary to process the dark area to make the details of the corresponding region after the visible light image and fluorescence image are fused clearer, thereby improving the imaging quality of the fused image.

[0092] In one embodiment of this application, after the visible light image is filtered, dark area fusion can be performed using equation (10) to obtain the brightness parameter Y_VIS_OUT of the dark area in the visible light image:

[0093]

[0094] Where Y_GAUSS is the visible light filtered image, and VIS_DARK_WEIGHT1 is the visible light dark area intensity coefficient. It is understood that the larger the visible light dark area intensity coefficient value, the greater the dark area fusion ratio. Further, the brightness parameter Y_OUT of the overlapping area between the dark area and the fluorescence image is obtained through equation (11):

[0095]

[0096] Where Y_VIS_OUT is the brightness parameter of the dark area in the visible light image, Y_IR is the brightness parameter of the fluorescence image, and IR_threshold1 is the preset overlap threshold of the overlapping area between the dark area in the visible light image and the fluorescence image. It can be understood that the larger this value is, the less the overlap between the dark area in the visible light image and the fluorescence image. By superimposing the fluorescence image onto the overlapping area Y_OUT according to equation (12), the brightness parameter Y_OUT2 of the fluorescence superimposed image can be obtained:

[0097]

[0098] Where Y_OUT is the overlapping region obtained by equation (11), mask_weight1 is the first superposition weight corresponding to the overlapping region, mask_weight2 is the fluorescence image and the corresponding second superposition weight, Y_IR is the brightness parameter of the fluorescence image, and IR_threshold2 is the preset superposition threshold. It can be understood that the larger the first superposition weight value, the larger the proportion of the visible light image brightness during image superposition; the larger the second superposition weight value, the larger the proportion of the fluorescence image brightness during image superposition; and the larger the preset superposition threshold, the smaller the proportion of the fluorescence region size in the superimposed fluorescence image during fusion. It can be understood that superposition of the visible light image and the fluorescence image does not necessarily require the visible light image to be filtered. The visible light image can also be directly superimposed with the fluorescence image according to the above steps.

[0099] Based on the above fluorescence superimposed image, step S205 of this embodiment of the application, which involves filtering the overexposed area to obtain the fluorescence image to be fused, includes:

[0100] S601: Filter the overexposed area to obtain a filtered image.

[0101] S603: The fluorescence image to be fused is obtained based on the filtered image, the fluorescence superimposed image, and the preset fusion threshold.

[0102] In this embodiment, the filtered image is the overexposed region EX_AREA_OUT after filtering, which can be obtained by referring to the above equations (3) and (4), and will not be repeated here. Based on the filtered image EX_AREA_OUT, the fluorescence superimposed image Y_OUT2, and the preset fusion threshold, the fluorescence image mask_1 to be fused can be obtained according to equation (13):

[0103]

[0104] In some embodiments, the size of the fluorescence image to be fused can also be adjusted according to equation (14):

[0105]

[0106] Wherein, mask is the fluorescence image to be fused after resizing, mask_1 is the fluorescence image to be fused before resizing, and mask_threshold is the preset fusion threshold. It can be understood that the larger the preset fusion threshold, the smaller the size of the fluorescence image to be fused.

[0107] In this embodiment, a fluorescence image is superimposed on a visible light image to obtain a fluorescence superimposed image, based on a visible light image. This allows the regional features of the fluorescence image to be reflected in the fluorescence superimposed image. The fluorescence superimposed image can serve as the basis for generating the fluorescence image to be fused, thereby enabling the fused target image to reflect the regional features of the fluorescence image. This makes the details of the target location displayed in the fluorescence image clearer, increases the recognizability, and further improves the quality of the fused image.

[0108] In this embodiment of the application, fusing the fluorescence image to be fused with the visible light image in step S207 to obtain the target image includes:

[0109] S701: Obtain the target image based on the fluorescence image to be fused, the visible light image, and the fusion weights.

[0110] In this embodiment, the fluorescence image to be fused and the visible light image can be fused using a preset fusion weight to obtain a target image. The preset fusion weight is used to adjust the fusion ratio of the fluorescence image to be fused and the visible light image in the target image. In a specific embodiment of this application, the target image can be obtained using equation (15):

[0111]

[0112] Where Y, U, and V are three parameters of the YUV image, FUSE_WEIGHT is the fusion weight, mask is the resized fluorescence image to be fused, Y_IR_color, U_IR_color, and V_IR_color are parameters of the visible light YUV images to be fused, and Y_VIS, U_VIS, and V_VIS are the corresponding parameters of the visible light YUV images. It can be understood that the larger the fusion weight value, the greater the fusion ratio of the fluorescence image to be fused in the target image.

[0113] Furthermore, in this embodiment of the application, before obtaining the target image based on the fluorescence image to be fused, the visible light image, and the fusion weights in step S701, the following steps are included:

[0114] S801: In response to a user instruction, determine the fusion weights.

[0115] In this embodiment, to allow users to adjust the fusion ratio of the visible light image and the fluorescence image to be fused in the target image, the fusion weight can be determined in response to user commands. When diagnosing a disease based on the target image, the user can adjust the fusion weight value according to actual needs. If it is necessary to observe the details of the target area in the fluorescence image, the fusion weight value can be increased; if it is necessary to observe the actual image of the target area under visible light, the fusion weight value can be decreased. This increases the flexibility of target image fusion, thereby making the diagnostic process more detailed and accurate.

[0116] The image fusion method provided in this application is illustrated below through a specific embodiment. Figure 3 As shown, after acquiring the visible light image and the fluorescence image, the visible light image and the fluorescence image are processed respectively. Firstly, the visible light image is processed based on a histogram equalization algorithm to obtain a histogram equalized image, and then Gaussian filtering is applied to the histogram equalized image to obtain a visible light filtered image. Visible light ratio superposition includes obtaining a visible light superimposed image based on the histogram equalized image, the visible light filtered image, and the superposition ratio; ratio fusion includes obtaining a visible light fused image based on the visible light superimposed image and a corresponding first fusion threshold, a preset visible light fusion threshold, and a corresponding second fusion threshold; pseudo-color lookup includes performing pseudo-color processing on the visible light fused image based on a pseudo-color algorithm to obtain a visible light image to be fused. Secondly, multiplying by dark area weights includes determining the dark area region in the visible light image based on a preset dark area threshold; multiplying by fluorescence data includes determining the overlapping area between the dark area region and the fluorescence image based on a preset overlap threshold; superimposing fluorescence includes obtaining a fluorescence superimposed image based on the overlapping area and the corresponding first superposition weight, the fluorescence image and the corresponding second superposition weight, and a preset superposition threshold. Thirdly, after registering the fluorescence image with the visible light image, the determination of the overexposed overlapping region of the fluorescence and visible light images includes identifying the overexposed region within the overlapping region of the visible light image and the fluorescence image. Gaussian filtering of the overexposed region includes filtering the overexposed region to obtain a filtered image. Mask region determination includes obtaining the fluorescence image to be fused based on the filtered image, the superimposed fluorescence image, and a preset fusion threshold. After determining the visible light image and the fluorescence image to be fused, a target image is obtained based on the fluorescence image to be fused, the visible light image, and fusion weights to achieve the fusion of the two images.

[0117] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0118] Based on the same inventive concept, this application also provides an image fusion apparatus 900 for implementing the image fusion method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the image fusion apparatus 900 provided below can be found in the limitations of the image fusion method described above, and will not be repeated here.

[0119] In one embodiment, such as Figure 4 As shown, an image fusion apparatus 900 is provided, comprising:

[0120] Image acquisition module 901 is used to acquire visible light images and fluorescence images;

[0121] The overexposed area determination module 902 is used to determine the overexposed area in the overlapping area of ​​the visible light image and the fluorescence image;

[0122] Image processing module 903 is used to filter the overexposed area to obtain the fluorescence image to be fused;

[0123] The image fusion module 904 is used to fuse the fluorescence image to be fused with the visible light image to obtain the target image.

[0124] Each module in the aforementioned image fusion device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0125] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the image fusion method described in any of the preceding embodiments.

[0126] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the image fusion method described in any of the preceding embodiments.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0128] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0129] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0130] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image fusion method, characterized in that, The method includes: Acquire visible light and fluorescence images; Identify the overexposed areas in the overlapping region of the visible light image and the fluorescence image; The overexposed areas are filtered to obtain the fluorescence image to be fused; The fluorescence image to be fused is fused with the visible light image to obtain the target image; Before filtering the overexposed area to obtain the fluorescence image to be fused, the following steps are included: The dark areas in the visible light image are determined based on a preset dark area threshold. The overlapping area between the dark area and the fluorescence image is determined based on a preset overlap threshold; A fluorescence superimposed image is obtained based on the overlapping area between the dark area and the fluorescence image and the corresponding first superimposed weight, the fluorescence image and the corresponding second superimposed weight, and a preset superimposed threshold. The step of filtering the overexposed areas to obtain the fluorescence image to be fused includes: The overexposed areas are filtered to obtain a filtered image; The fluorescence image to be fused is obtained based on the filtered image, the fluorescence superimposed image, and the preset fusion threshold.

2. The method according to claim 1, characterized in that, Prior to fusing the fluorescence image to be fused with the visible light image, the following steps are included: The visible light image is processed using a histogram equalization algorithm to obtain a histogram equalized image. The histogram-equalized image is filtered to obtain a visible light filtered image.

3. The method according to claim 2, characterized in that, After filtering the histogram-equalized image to obtain the visible light filtered image, the method further includes: A visible light superimposed image is obtained based on the histogram equalized image, the visible light filtered image, and the superposition ratio; A visible light fused image is obtained based on the visible light superimposed image and the corresponding first fusion threshold, the preset visible light fusion threshold and the corresponding second fusion threshold; The visible light fusion image is processed using a pseudo-color algorithm to obtain the visible light image to be fused.

4. The method according to claim 1, characterized in that, The step of fusing the fluorescence image to be fused with the visible light image to obtain the target image includes: The target image is obtained based on the fluorescence image to be fused, the visible light image, and the fusion weights.

5. The method according to claim 4, characterized in that, Before obtaining the target image based on the fluorescence image to be fused, the visible light image, and the fusion weights, the following steps are included: In response to user instructions, the fusion weights are determined.

6. An image fusion apparatus, characterized in that, The device includes: Image acquisition module, used to acquire visible light images and fluorescence images; An overexposed area determination module is used to determine the overexposed areas in the overlapping area of ​​the visible light image and the fluorescence image; The image processing module is used to filter the overexposed areas to obtain the fluorescence image to be fused. An image fusion module is used to fuse the fluorescence image to be fused with the visible light image to obtain a target image; The image processing module is further configured to determine the dark area region in the visible light image based on a preset dark area threshold; determine the overlapping area between the dark area region and the fluorescence image based on a preset overlap threshold; and obtain a fluorescence superimposed image based on the overlapping area between the dark area region and the fluorescence image and the corresponding first superimposed weight, the fluorescence image and the corresponding second superimposed weight, and the preset superimposed threshold. The image processing module is also used to filter the overexposed area to obtain a filtered image; and to obtain the fluorescence image to be fused based on the filtered image, the fluorescence superimposed image, and a preset fusion threshold.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images