Multi-branch fused endoscopic image enhancement method, system, device, and medium

Abstract

This invention discloses a multi-branch fusion method, system, apparatus, and medium for endoscopic image enhancement. The method includes acquiring an original endoscopic image and performing noise suppression processing on the original endoscopic image to obtain an RGB image to be enhanced; simultaneously performing multi-dimensional enhancement processing on the RGB image to be enhanced to obtain multiple branch images; the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; and fusing all branch images to obtain the target endoscopic image. This invention, based on the simultaneous execution and fusion of multi-dimensional enhancement processing, effectively balances multiple requirements such as illumination balance and noise suppression, ensuring a significant improvement in the quality of the final endoscopic image. It meets the high standards and stability requirements for endoscopic image quality, providing clearer and more accurate image evidence for related medical diagnoses, and contributing to improved accuracy and efficiency in diagnosis and detection.

Description

Technical Field

[0001] This invention relates to the field of medical image processing, and more specifically to a multi-branch fusion method, system, device, and medium for enhancing endoscopic images. Background Technology

[0002] Endoscopic imaging technology, as a key tool in modern clinical medical diagnosis, can directly present the morphology and structure of internal human tissues, providing important evidence for the early detection and accurate diagnosis of diseases. However, endoscopic images often suffer from uneven lighting, blurred details, mucosal reflection, noise interference, and excessive reflection, resulting in quality that fails to meet the high standards required for clinical diagnosis.

[0003] In existing technologies, common image enhancement methods to improve the aforementioned image quality issues include gamma correction and histogram equalization (CLAHE). While these methods can partially improve image quality, they still have shortcomings in preserving local details, suppressing noise, and handling reflections. (1) Limited ability to preserve local details: When processing complex textures and edge areas, it is easy to over-enhance or lose details, resulting in visual distortion; (2) Insufficient noise suppression: While enhancing low-contrast areas, image noise is often amplified simultaneously, affecting image purity; (3) Poor reflection and highlight processing: There is a lack of targeted suppression mechanism for specular reflection and highlight areas, which can easily cause information loss or artifacts.

[0004] Therefore, current technologies for endoscopic image enhancement have not been able to effectively address the multiple requirements of balanced illumination, enhanced detail, noise suppression, and reflection suppression, making it difficult to meet the high standards and stability demands for image quality in clinical diagnosis. There is an urgent need to develop a novel image enhancement technology that can comprehensively improve the brightness, contrast, local detail, and noise suppression of endoscopic images. Summary of the Invention

[0005] In view of this, the present invention provides a multi-branch fusion endoscopic image enhancement method, system, device and medium to solve the problem that existing endoscopic image enhancement technologies cannot effectively balance multiple requirements such as illumination balance and noise suppression, which makes it difficult to meet the high standards and stability requirements of endoscopic image quality.

[0006] This invention provides a multi-branch fusion method for enhancing endoscopic images, the method comprising: Acquire the original endoscopic image and perform noise suppression processing on the original endoscopic image to obtain the RGB image to be enhanced; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images; wherein, the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; All branch images are fused to obtain the target endoscopic image.

[0007] Optionally, the number of branch images must include at least two; The images from all branches are fused to obtain the target endoscopic image, including: Obtain the weight map corresponding to each branch image; Construct an image pyramid for each branch image and a weight pyramid for each weight image; the total number of image pyramids is the same as the total number of weight pyramids, and all image pyramids correspond one-to-one with all weight pyramids, with the same number of layers for each image pyramid and each weight pyramid. Based on all weighted pyramids, a weighted fusion of all image pyramids is performed to obtain a fused pyramid. The fused pyramid is upsampled and reconstructed to obtain the target endoscopic image.

[0008] Optionally, based on all weight pyramids, a weighted fusion of all image pyramids is performed to obtain a fused pyramid, including: The i-th layer is selected from each image pyramid and each weight pyramid respectively. Image data is extracted from each i-th layer image pyramid to obtain the corresponding i-th layer image data, and weight data is extracted from each i-th layer weight pyramid to obtain the corresponding i-th layer weight data. Among them, the i-th layer image data in each image pyramid corresponds one-to-one with the i-th layer weight data in the corresponding weight pyramid. Using all the weight data of the i-th layer, perform a weighted summation operation on all the image data of the i-th layer to obtain the fused pyramid layer data of the i-th layer; Traverse each layer of the image pyramid and weight pyramid, and obtain the fusion pyramid layer data for each layer using the same method; and obtain the fusion pyramid based on all the fusion pyramid layer data.

[0009] Optionally, the calculation formula for the fused pyramid layer data of the i-th layer is as follows: ; Among them, F i (x,y) represents the fused pyramid layer data of the i-th layer. For the j-th image pyramid, the i-th layer image data is... Let be the weight data of the i-th layer of the j-th weight pyramid, and m be the total number of image pyramids or the total number of weight pyramids.

[0010] Optionally, the fused pyramid is upsampled and reconstructed to obtain the target endoscopic image, including: Starting from the top image of the fusion pyramid, upsampling is performed layer by layer and superimposed with the next layer image in the fusion pyramid to obtain the target endoscopic image.

[0011] Optionally, obtain the weight map corresponding to each branch image, including: Choose any branch image and calculate the saturation weight, contrast weight, and exposure weight of each pixel in the selected branch image. Based on the calculated saturation weight, contrast weight, and exposure weight of each pixel, the composite weight of each pixel in the selected branch image is obtained. Generate a weight map of the selected branch image based on the composite weights of all pixels in the selected branch image. Traverse each branch image and obtain the weight map for each branch image using the same method.

[0012] Optionally, when selecting the j-th branch image, the formula for calculating the composite weight at pixel coordinates (x, y) in the j-th branch image is: ; in, Let be the composite weight at pixel coordinates (x, y) in the j-th branch image. , and These are the saturation weight, contrast weight, and exposure weight at pixel coordinates (x, y) in the j-th branch image, respectively.

[0013] Optionally, an image pyramid corresponding to each branch image and a weight pyramid corresponding to each weight map are constructed, including: Choose any branch image and perform layer-by-layer downsampling on the selected branch image to obtain the Gaussian pyramid corresponding to the selected branch image; also perform layer-by-layer downsampling on the weight map corresponding to the selected branch image to obtain the weight pyramid corresponding to the weight map of the selected branch image. Based on the Gaussian pyramid of the selected branch image, an image pyramid of the selected branch image is constructed; Traverse each branch image and construct the image pyramid corresponding to each branch image and the weight pyramid corresponding to each weight image using the same method.

[0014] Optionally, when selecting the j-th branch image, the expression for the i-th layer image data of the image pyramid of the j-th branch image is as follows: ; in, For the image data of the i-th layer of the image pyramid of the j-th branch image, and Let be the data of the i-th and (i+1)-th layers of the Gaussian pyramid in the j-th branch image, respectively, and g be the interpolation kernel. This is the upsampling operation function. This is a convolution operation.

[0015] Optionally, the branch image specifically includes a first branch image and a second branch image; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing, including: The first branch image is obtained by performing brightness enhancement processing on the RGB image to be enhanced using the multi-scale Retinex enhancement method. The second branch image is obtained by using a histogram equalization method to adjust the contrast of the RGB image to be enhanced.

[0016] Optionally, a multi-scale Retinex enhancement method is used to perform brightness enhancement processing on the RGB image to be enhanced, resulting in the first branch image, including: The RGB image to be enhanced is converted to grayscale to obtain a grayscale image; The grayscale image is enhanced using a multi-scale Retinex enhancement method to obtain a brightness-enhanced image; The three color components of the RGB image to be enhanced are associated and reconstructed with the brightness enhancement image to obtain the three enhanced color components corresponding to the three color components respectively; The three enhanced color components are merged to obtain the first branch image.

[0017] Optionally, a histogram equalization method is used to perform contrast adjustment processing on the RGB image to be enhanced, resulting in the second branch image, including: The RGB image to be enhanced is converted to the LAB color space to obtain the first LAB color image; Extract the L channel component from the first LAB color image, and enhance the extracted L channel component using a histogram equalization method to obtain the first enhanced L channel component. Extract the A-channel and B-channel components from the first LAB color image, and merge the first enhanced L-channel component with the extracted A-channel and B-channel components to obtain the first enhanced LAB color image. The first enhanced color LAB image is converted back to the RGB color space to obtain the second branch image.

[0018] Optionally, the branch image further includes a third branch image; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images, and the process also includes: The RGB image to be enhanced is converted to the HSV color space to obtain an HSV color image; The HSV color image is detected using a preset brightness threshold and / or saturation threshold to obtain the highlight area; Based on the highlighted area, a texture synthesis and repair method is used to perform texture synthesis and repair on the area corresponding to the highlighted area in the RGB image to be enhanced, thereby obtaining the third branch image.

[0019] Optionally, the branch image further includes a fourth branch image; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images, and the process also includes: The RGB image to be enhanced is converted to the LAB color space to obtain a second LAB color image; Extract the L channel component from the second LAB color image, and decompose the extracted L channel component to obtain the base layer and detail layer; An adaptive gain map of the detail layer is constructed, and the detail layer is modulated using the adaptive gain map to obtain an enhanced detail layer; The base layer and the enhanced detail layer are fused to obtain the second enhanced L-channel component; Extract the A channel component and B channel component from the second LAB color image, and merge the second enhanced L channel component with the extracted A channel component and the B channel component to obtain the second enhanced LAB color image; The second enhanced color LAB image is converted back to the RGB color space to obtain the fourth branch image.

[0020] Optionally, noise suppression processing is performed on the original endoscopic image to obtain an RGB image to be enhanced, including: The original endoscopic image is converted to the LAB color space to obtain a third LAB color image; Extract the L channel component from the third LAB color image, and denoise the extracted L channel component to obtain the denoised L channel component. Extract the A-channel and B-channel components from the third LAB color image, and merge the denoised L-channel component with the extracted A-channel and B-channel components to obtain the denoised LAB color image. The denoised LAB color image is converted back to the RGB color space to obtain the RGB image to be enhanced.

[0021] Furthermore, the present invention also provides a multi-branch fusion endoscopic image enhancement system, applied to the aforementioned multi-branch fusion endoscopic image enhancement method, comprising: The image noise suppression module is used to acquire the original endoscope image and perform noise suppression processing on the original endoscope image to obtain the RGB image to be enhanced; A multi-dimensional synchronous enhancement module is used to synchronously perform multi-dimensional enhancement processing on the RGB image to be enhanced to obtain multiple branch images; wherein, the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; The multi-branch fusion module is used to fuse all branch images to obtain the target endoscopic image.

[0022] Furthermore, the present invention also provides a multi-branch fusion endoscopic image enhancement device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the method steps in the aforementioned multi-branch fusion endoscopic image enhancement method.

[0023] Furthermore, the present invention also provides a computer storage medium comprising: at least one instruction that, when executed by a computer, implements the method steps of the aforementioned multi-branch fusion endoscopic image enhancement method.

[0024] The beneficial effects of this invention are as follows: By performing noise suppression processing on the acquired original endoscopic images, noise interference in the images can be effectively reduced, and the purity of the images can be improved; then, based on the RGB image to be enhanced obtained from the noise suppression processing, multi-dimensional enhancement processing is performed simultaneously to generate multiple branch images. Each branch image can enhance and optimize the image from different angles, including but not limited to brightness enhancement, contrast adjustment, and other effects; then, all these branch images with different enhancement effects are fused, which can give full play to the advantages of different enhancement methods and effectively take into account multiple requirements such as illumination balance and noise suppression; The multi-branch fusion endoscopic image enhancement method, system, device, and medium of the present invention, based on the simultaneous and fusion of multi-dimensional enhancement processing, can effectively take into account multiple requirements such as illumination balance and noise suppression, ensuring that the quality of the final endoscopic image is significantly improved, meeting the high standards and stability requirements for endoscopic image quality, and providing clearer and more accurate image evidence for related medical diagnoses, thus helping to improve the accuracy and efficiency of diagnosis and detection. Attached Figure Description

[0025] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings: Figure 1 A flowchart of a multi-branch fusion endoscopic image enhancement method according to Embodiment 1 of the present invention is shown; Figure 2 The flowchart illustrating the fusion of all branch images in Embodiment 1 of the present invention is shown; Figure 3 A complete flowchart of the multi-branch fusion endoscopic image enhancement method in Embodiment 1 of the present invention is shown; Figure 4 The diagram shows a structure of a multi-branch fusion endoscopic image enhancement system according to Embodiment 2 of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] In this embodiment of the invention, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0029] In this embodiment of the invention, the term "multiple" refers to two or more, and other quantifiers are similar.

[0030] Example 1 This embodiment provides a multi-branch fusion method for enhancing endoscopic images, such as... Figure 1 As shown, the method includes: S1: Acquire the original endoscopic image and perform noise suppression processing on the original endoscopic image to obtain the RGB image to be enhanced; S2: Simultaneously perform multi-dimensional enhancement processing on the RGB image to be enhanced to obtain multiple branch images; wherein, the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; S3: Fuse all branch images to obtain the target endoscope image.

[0031] In this embodiment, noise suppression processing is applied to the acquired original endoscopic image to effectively reduce noise interference and improve image purity. Then, based on the RGB image to be enhanced obtained from the noise suppression processing, multi-dimensional enhancement processing is performed simultaneously to generate multiple branch images. Each branch image can enhance and optimize the image from different angles, including but not limited to brightness enhancement and contrast adjustment. Then, all these branch images with different enhancement effects are fused to give full play to the advantages of different enhancement methods and effectively take into account multiple requirements such as illumination balance and noise suppression.

[0032] The multi-branch fusion endoscopic image enhancement method in this embodiment, based on the simultaneous and fusion of multi-dimensional enhancement processing, can effectively take into account multiple requirements such as illumination balance and noise suppression, ensuring that the quality of the final endoscopic image is significantly improved. It meets the high standards and stability requirements for endoscopic image quality, and can provide clearer and more accurate image evidence for related medical diagnoses, which helps to improve the accuracy and efficiency of diagnosis and detection.

[0033] The following provides a detailed description of each step in the multi-branch fusion endoscopic image enhancement method of this embodiment.

[0034] In step S1, the raw endoscopic images can be acquired in real time by an image sensor (such as a CMOS sensor), or they can be collected in an endoscopy-related data system or data platform using big data technology.

[0035] In step S1, noise suppression processing is performed on the original endoscopic image to obtain the RGB image to be enhanced, including: S11: Convert the original endoscope image to the LAB color space to obtain a third LAB color image; S12: Extract the L channel component from the third LAB color image, and denoise the extracted L channel component to obtain the denoised L channel component. S13: Extract the A channel component and B channel component from the third LAB color image, and merge the denoised L channel component with the extracted A channel component and the B channel component to obtain the denoised LAB color image. S14: Convert the denoised LAB color image back to the RGB color space to obtain the RGB image to be enhanced.

[0036] In the noise suppression process, the original endoscope image is first converted from the RGB color space to the LAB color space in step S11, enabling the separation of luminance (L channel) and chrominance (A and B channels). Since the noise in the endoscope image is mainly concentrated in the luminance channel (L channel), while the chrominance channels (A and B channels) are relatively stable, after separating the L channel components, targeted denoising of the L channel components in step S12 effectively avoids over-processing of color information, ensuring the accuracy and naturalness of subsequent image colors. After denoising the L channel components, in step S13, the A and B channel components of the original endoscope image are directly reused, and the denoised L channel (i.e., the denoised L channel component) is merged with the original A and B channels. This ensures the denoising effect while reducing the processing cost of the chrominance channels, improving overall processing efficiency, avoiding color distortion, and ensuring the authenticity of image colors. After the denoised LAB color image is obtained after merging, in step S14, the denoised LAB color image is converted back to RGB space. This ensures that the output image meets the format requirements of conventional display and subsequent processing, provides a high-quality base image for subsequent multi-dimensional enhancement processing, and helps to fuse subsequent branch images, thereby achieving a comprehensive improvement in image quality.

[0037] Specifically, the denoising in step S12 can be performed using the nonlocal means denoising method.

[0038] In step S2, the branch image specifically includes a first branch image and a second branch image; Then step S2 includes: S21: Using the multi-scale Retinex enhancement processing method, the brightness of the RGB image to be enhanced is increased to obtain the first branch image; S22: The histogram equalization method is used to perform contrast adjustment processing on the RGB image to be enhanced to obtain the second branch image.

[0039] In step S21, the multi-scale Retinex enhancement method (MSR method) simulates the characteristics of the human visual system, enabling adjustments to the brightness of the RGB image to be enhanced at different scales. This method effectively preserves the detailed information of the RGB image to be enhanced, avoiding detail loss during brightness enhancement, ensuring that the texture and edge details of the first branch image remain clearly discernible even as brightness is increased.

[0040] In step S22, the histogram equalization processing method adjusts the grayscale histogram of the RGB image to be enhanced, making the grayscale distribution of the image more uniform. The resulting second branch image has brighter highlights, darker shadows, and clearer boundaries between different objects, which helps doctors observe lesion features in the image more accurately.

[0041] Preferably, step S21 includes: S211: Convert the RGB image to be enhanced to grayscale to obtain a grayscale image; S212: The grayscale image is enhanced using the multi-scale Retinex enhancement method to obtain a brightness-enhanced image; S213: The three color components of the RGB image to be enhanced are respectively associated with the brightness enhancement image to reconstruct the image, so as to obtain the three enhanced color components corresponding to the three color components respectively; S214: The three enhanced color components are merged to obtain the first branch image.

[0042] In step S211, the RGB image to be enhanced is converted to grayscale. This removes the color information from the RGB image and retains only the brightness information, making it easier to perform subsequent brightness enhancement processing. Grayscale conversion can be performed using a weighted average method, which calculates grayscale values ​​by assigning different weights to the three color channels of the RGB image to be enhanced.

[0043] In step S212, the grayscale image is enhanced using a multi-scale Retinex enhancement method. The multi-scale Retinex method combines Gaussian filtering at different scales, enabling adjustments to image brightness at various scales. It enhances image details at small scales and adjusts global brightness at large scales. This method effectively preserves image details while increasing brightness, avoiding over-enhancement or loss of detail.

[0044] In step S213, the three color components of the RGB image to be enhanced are associated with the brightness enhancement image and reconstructed respectively. This allows the grayscale information after brightness enhancement to be reintegrated into the original color image, so that the enhanced image not only has improved brightness, but also maintains relative accuracy and naturalness of color.

[0045] In step S214, the three enhanced color components after association and reconstruction are merged to obtain the first branch image. This first branch image enhances brightness while maintaining the original color of the image, providing a branch image with good brightness enhancement effect for subsequent multi-branch image fusion, thereby ensuring a comprehensive improvement in brightness, contrast, local details, and noise suppression of the endoscopic image.

[0046] Specifically, in step S212, the grayscale image is enhanced using the multi-scale Retinex enhancement method, and the expression for the resulting brightness-enhanced image is as follows: ; Where R(x,y) is the brightness-enhanced image, F k (x,y) is the Gaussian wrapping function at the k-th scale, w k Let be the scale weight corresponding to the k-th scale, n be the total number of scales, and I(x,y) be the grayscale image.

[0047] Furthermore, in actual image processing, after obtaining the brightness-enhanced image R(x,y), linear or gain compensation adjustments can be made to further optimize its brightness performance. Linear compensation adjustment can improve the overall brightness level of the image by linearly transforming each pixel value in the brightness-enhanced image, for example, by multiplying it by a preset fixed coefficient or adding a preset fixed offset. Gain compensation adjustment, on the other hand, can adjust pixel values ​​in different regions to varying degrees based on the local features of the image, resulting in a more uniform brightness distribution and avoiding localized over-brightness or under-brightness.

[0048] Specifically, in step S213, taking the R color component of the RGB image to be enhanced as an example, the calculation formula for its association and reconstruction with the brightness enhancement image is as follows: ; in, To enhance the R color components, The R color component is the RGB image to be enhanced. The calculation of the G and B color components of the RGB image to be enhanced and their association with the brightness enhancement image is similar and will not be described in detail here.

[0049] Preferably, step S22 includes: S221: Convert the RGB image to be enhanced to the LAB color space to obtain a first LAB color image; S222: Extract the L channel component from the first LAB color image, and enhance the extracted L channel component using a histogram equalization method to obtain the first enhanced L channel component. S223: Extract the A channel component and B channel component from the first LAB color image, and merge the first enhanced L channel component with the extracted A channel component and B channel component to obtain the first enhanced LAB color image; S224: Convert the first enhanced color LAB image back to the RGB color space to obtain the second branch image.

[0050] In the LAB color space, the L channel represents luminance, and the A and B channels represent color. In step S221, the RGB image to be enhanced is converted to the LAB color space, which can also separate luminance and color, making it easier to perform more targeted contrast adjustment on the luminance channel in the future.

[0051] In step S222, the extracted L channel components are enhanced by histogram equalization, which makes the gray distribution of the image more uniform, improves the contrast of the image, makes the bright parts brighter and the dark parts darker, thereby highlighting the details and features of the mucosal texture, blood vessels and lesion edges in the image.

[0052] In step S223, the enhanced first enhanced L channel component is merged with the original A channel component and B channel component, which can preserve the color information of the original image while ensuring the contrast adjustment effect and avoiding color distortion.

[0053] Finally, in step S224, the merged first enhanced LAB color image is converted back to the RGB color space to obtain a second branch image that meets the requirements of conventional display and processing formats. This second branch image has a good contrast adjustment effect and also provides a high-quality branch image for subsequent multi-branch image fusion, which in turn helps to achieve a comprehensive improvement in brightness, contrast, local details and noise suppression of endoscopic images.

[0054] In step S2, the branch image further includes a third branch image; Then step S2 also includes: S23: Perform reflection suppression processing on the RGB image to be enhanced to obtain the third branch image.

[0055] Preferably, step S23 includes: S231: Convert the RGB image to be enhanced to the HSV color space to obtain an HSV color image; S232: Using a preset brightness threshold and / or saturation threshold, the HSV color image is detected to obtain the highlight area; S233: Based on the highlight area, a texture synthesis and repair method is used to perform texture synthesis and repair on the area corresponding to the highlight area in the RGB image to be enhanced, so as to obtain the third branch image.

[0056] In the HSV color space, H represents hue, S represents saturation, and V represents brightness. In step S231, the RGB image to be enhanced is converted to the HSV color space, which can more intuitively represent the attributes of color and further separate color information and brightness information, making it easier to detect the highlight area in the future.

[0057] In step S232, the HSV color image is detected using preset brightness and / or saturation thresholds. When the brightness value of a pixel in the image is higher than the preset brightness threshold, or the saturation value is lower than the preset saturation threshold, or when the brightness value of a pixel is higher than the preset brightness threshold and the saturation value is lower than the preset saturation threshold, the area containing that pixel is determined to be a highlight area. By pre-setting the brightness and saturation thresholds appropriately, reflective parts in the image can be accurately identified, providing a basis for subsequent repair. Both the brightness and saturation thresholds can be pre-set and adjusted according to actual conditions; no restrictions are imposed here.

[0058] In step S233, based on the detected highlight areas, a texture synthesis inpainting method is used to repair the corresponding areas in the RGB image to be enhanced. Specifically, the texture synthesis inpainting method utilizes texture information from other non-highlight areas in the image (specifically, image areas in the RGB image to be enhanced other than those corresponding to the highlight areas) to generate a texture similar to the highlight areas using a specific algorithm. This texture is then filled into the highlight areas, thereby removing reflections. This method can effectively eliminate the impact of reflections on image quality without destroying the overall image structure and details.

[0059] This embodiment effectively removes reflections in endoscopic images through the reflection suppression processing in steps S231-S233, preventing reflections from interfering with the doctor's image observation. Reflections can obscure important lesion features, blur blood vessels, and obscure mucosal textures. The third branch image obtained after reflection suppression clearly presents the areas originally obscured by reflections, allowing doctors to more accurately identify the location, shape, and boundaries of lesions. Simultaneously, texture synthesis restoration is used to remove reflections while preserving the original texture and details of the image, ensuring overall image quality. Like the first and second branch images, this third branch image provides a high-quality branch image for subsequent multi-branch image fusion, further contributing to a comprehensive improvement in brightness, contrast, local detail, and noise suppression of endoscopic images. This provides doctors with clearer and more accurate endoscopic images, assisting them in making more precise diagnostic decisions.

[0060] In step S2, the branch image further includes a fourth branch image; Then step S2 also includes: S24: Perform local detail enhancement processing on the RGB image to be enhanced to obtain the fourth branch image.

[0061] Preferably, step S24 includes: S241: Convert the RGB image to be enhanced to the LAB color space to obtain a second LAB color image; S242: Extract the L channel component from the second LAB color image, and decompose the extracted L channel component to obtain the base layer and the detail layer; S243: Construct an adaptive gain map of the detail layer, and use the adaptive gain map to modulate the detail layer to obtain an enhanced detail layer; S244: The base layer and the enhanced detail layer are fused to obtain the second enhanced L-channel component; S245: Extract the A channel component and B channel component from the second LAB color image, and merge the second enhanced L channel component with the extracted A channel component and the B channel component to obtain the second enhanced LAB color image; S246: Convert the second enhanced color LAB image back to the RGB color space to obtain the fourth branch image.

[0062] In step S241, the RGB image to be enhanced is converted to the LAB color space. Similar to the previous steps, this is also to separate brightness and color, laying the foundation for subsequent independent processing of local detail enhancement of the brightness channel.

[0063] In step S242, the extracted L-channel components are decomposed into a base layer and a detail layer. The base layer mainly contains low-frequency information of the image, reflecting the overall brightness and wide-range illumination changes of the image; while the detail layer contains high-frequency information of the image, namely the local textures, edges, contours, and other details in the image. Through this decomposition, the details can be enhanced more effectively without affecting the overall illumination and color information of the image, while also avoiding the misjudgment of noise as details.

[0064] In step S243, the constructed adaptive gain map can automatically adjust the enhancement level of different regions in the detail layer based on their characteristics. For example, in regions rich in detail, the gain factor can be increased (e.g., 1.5 to 2 times) to highlight these details; while in smooth regions with less detail, the gain factor can be decreased (e.g., 0.8 to 1.0 times) to avoid over-enhancement that generates noise or distortion. After modulating the detail layer using the adaptive gain map, the resulting enhanced detail layer can significantly improve the local detail representation of the image.

[0065] In step S244, the base layer and the enhanced detail layer are fused to obtain the second enhanced L channel component, which combines the processed detail information with the original overall brightness information, so that the image retains the overall lighting and color atmosphere while enhancing local details.

[0066] In step S245, the second enhanced L channel component is merged with the extracted A channel component and B channel component to obtain the second enhanced LAB color image. This ensures that while enhancing local details, the color information of the original image is preserved, thus ensuring the accuracy and naturalness of the image colors.

[0067] In step S246, the second enhanced LAB color image is converted back to the RGB color space to obtain the fourth branch image. This image has good local detail enhancement and also provides a high-quality branch image for subsequent multi-branch image fusion. After fusing it with the first branch image (brightness enhancement), the second branch image (contrast adjustment), and the third branch image (reflection suppression), the image can be optimized from multiple dimensions, ultimately achieving a comprehensive improvement in the brightness, contrast, local detail, and noise suppression of the endoscopic image, providing doctors with clearer and more accurate endoscopic images.

[0068] Specifically, in step S242, bilateral filtering, weighted least squares filtering, and other methods can be used to process the L-channel components to achieve decomposition.

[0069] Furthermore, in step S243, before constructing the adaptive gain map, a Laplace pyramid or wavelet transform can be used to decompose the detail layer into multiple detail sub-bands of different scales (frequency bands), and then construct the adaptive gain map of all detail sub-bands, which can further refine the processing and achieve higher precision local detail enhancement.

[0070] In step S3, the number of branch images includes at least two (specifically, the first branch image and the second branch image mentioned above). Then as Figure 2 As shown, step S3 includes: S31: Obtain the weight map corresponding to each branch image; S32: Construct the image pyramid corresponding to each branch image and the weight pyramid corresponding to each weight image respectively; wherein, the total number of image pyramids is the same as the total number of weight pyramids, and all image pyramids correspond one-to-one with all weight pyramids, and the number of layers of each image pyramid and each weight pyramid is the same. S33: Based on all weighted pyramids, perform weighted fusion of all image pyramids to obtain a fused pyramid; S34: Upsample and reconstruct the fused pyramid to obtain the target endoscope image.

[0071] In step S31, the weight map can reflect the proportion of each branch image in the final fused image. Obtaining the weight map corresponding to each branch image facilitates the subsequent construction of the weight pyramid, which in turn facilitates the weighted fusion of the image pyramid, ensuring that the features of the dominant branches are preferentially retained during the fusion process, improving the quality of the fused image, and obtaining the desired target endoscope image.

[0072] In practice, weights can be assigned based on the characteristics and advantages of different branch images. For example, for the first branch image obtained after brightness enhancement processing, when the overall endoscopic scene is dark, its weight can be appropriately increased to enhance the brightness of the final image. For the second branch image obtained after contrast adjustment processing, if details such as mucosal texture, blood vessels, and lesion edges in the image need to be highlighted, its weight can be increased.

[0073] In step S32, the image pyramid generates image sets of different resolutions through multiple downsampling operations, capturing image features at both large-scale (overall structure) and small-scale (local details) levels. The weight pyramid simultaneously matches the multi-scale distribution of weight information, ensuring that the weights at different scales accurately correspond to the image features. This pyramid structure preserves image details at different resolutions, avoiding the loss of local features or global structure caused by single-scale fusion, thus laying the foundation for subsequent multi-scale fusion.

[0074] In step S33, weighted fusion of all image pyramids based on all weight pyramids can integrate information from different branch images at various scales according to weights; in step S34, upsampling reconstruction of the fused pyramid can restore the fused information at different scales to the resolution of the original image.

[0075] The target endoscopic image obtained through the above steps S31 to S34 in this embodiment integrates the enhancement effects of various branch images in terms of brightness, contrast, reflection suppression, and local details, and can present the information in the endoscopic image more clearly and accurately, providing doctors with a more reliable basis for diagnosis.

[0076] Preferably, step S31 includes: S311: Select any branch image and calculate the saturation weight, contrast weight, and exposure weight of each pixel in the selected branch image. S312: Based on the calculated saturation weight, contrast weight, and exposure weight of each pixel, obtain the composite weight of each pixel in the selected branch image; S313: Generate a weight map of the selected branch image based on the composite weight of all pixels in the selected branch image; S314: Traverse each branch image and obtain the weight map of each branch image in the same way.

[0077] In step S311, the saturation weight reflects the vibrancy of the image colors. Higher saturation indicates more vibrant colors, and the pixel is likely to be more valuable in the final fused image; therefore, the saturation weight will be relatively high. The contrast weight reflects the contrast of the region where the pixel is located. Regions with high contrast often contain more detail, so the contrast weight will also increase accordingly. The exposure weight is related to the pixel's exposure. Pixels with proper exposure can more accurately present information in the image, and their exposure weight will be higher. By calculating these three weights separately, the importance of each pixel in the fusion process can be more comprehensively measured.

[0078] In step S312, the saturation weight, contrast weight, and exposure weight are combined to obtain the composite weight of each pixel. This can comprehensively consider the influence of multiple factors such as color, contrast, and exposure on the importance of pixels, making the weight allocation of each pixel in the fusion process more reasonable and accurate.

[0079] In step S313, a weight map is generated based on the composite weights of all pixels. The weight map visually displays the importance distribution of each pixel in the fusion process, providing a foundation for the subsequent construction of the weight pyramid.

[0080] In step S314, the above operation is performed on each branch image to obtain a weight map of all branch images. This ensures that each branch image can be reasonably assigned weights according to its own characteristics during the fusion process, so that the final fused image can give full play to the advantages of each branch image and achieve optimization in many aspects such as brightness, contrast, reflection suppression, and local details.

[0081] Specifically, in step S311, the saturation weight, contrast weight, and exposure weight of each pixel are calculated, which can be done using conventional methods.

[0082] For example, when selecting the j-th branch image, the formula for calculating the saturation weight at pixel coordinates (x, y) in the j-th branch image is: ; in, S represents the saturation weight at pixel coordinates (x, y) in the j-th branch image. j (x,y) represents the saturation value at pixel coordinates (x,y) in the j-th branch image, S max The maximum saturation value in the j-th branch image. This is the saturation weight decay constant (used to control the decay rate of the saturation weight, which can be preset according to the actual situation, for example, 0.3).

[0083] For the above saturation weight, when the saturation value S at pixel coordinates (x, y) j (x,y) is close to S max When the saturation weight is close to 1, it indicates that the high-saturation area is emphasized; when the saturation value S at pixel coordinates (x,y) is close to 1, it indicates that the high-saturation area is emphasized. j When (x,y) is low, the saturation weight decays rapidly, indicating that the low saturation region is suppressed.

[0084] When calculating the contrast weights, the branch image needs to be converted into a grayscale image first, and then the grayscale image is filtered (e.g., Laplacian filtering) to obtain the filtered response map.

[0085] The formula for calculating the contrast weight at pixel coordinates (x, y) in the j-th branch image is: ; in, X represents the contrast weight at pixel coordinates (x, y) in the j-th branch image. j (x,y) is the response value at pixel coordinates (x,y) in the filtered response map corresponding to the j-th branch image.

[0086] The formula for calculating the exposure weight at pixel coordinates (x, y) in the j-th branch image is: ; in, Let L be the exposure weight at pixel coordinates (x, y) in the j-th branch image. j (x,y) represents the brightness value at pixel coordinates (x,y) in the j-th branch image. This is the exposure weight decay constant (used to control the decay rate of exposure weight, which can be preset according to actual conditions, for example, 0.3).

[0087] For the exposure weights mentioned above, when the exposure value L at pixel coordinates (x, y) j When (x,y) is close to 0.5, the exposure weight is close to 1, indicating that the exposed area is emphasized; when the exposure value L at pixel coordinates (x,y) is close to 1, the exposure weight is close to 1. j When (x,y) approaches 0 or 1, the exposure weight decays rapidly, indicating that overexposed / underexposed areas are suppressed.

[0088] After calculating the saturation weight, contrast weight, and exposure weight, the corresponding composite weight can be calculated. In step S312, the formula for calculating the composite weight at pixel coordinates (x, y) in the j-th branch image is as follows: ; in, Let be the composite weight at pixel coordinates (x, y) in the j-th branch image. , and These are the saturation weight, contrast weight, and exposure weight at pixel coordinates (x, y) in the j-th branch image, respectively.

[0089] According to the above formula, the composite weight of each pixel in the j-th branch image is calculated pixel by pixel, and in step S313, a weight map of the corresponding branch image is generated based on the composite weights of all pixels. Further, in step S314, a corresponding weight map is generated for each branch image according to the above method.

[0090] Preferably, step 32 includes: S321: Select any branch image, perform layer-by-layer downsampling on the selected branch image to obtain the Gaussian pyramid corresponding to the selected branch image; also perform layer-by-layer downsampling on the weight map corresponding to the selected branch image to obtain the weight pyramid corresponding to the weight map of the selected branch image. S322: Based on the Gaussian pyramid of the selected branch image, construct the image pyramid of the selected branch image; S323: Traverse each branch image and construct the image pyramid corresponding to each branch image and the weight pyramid corresponding to each weight image in the same way.

[0091] In step S321, a Gaussian pyramid is generated by downsampling the branch image layer by layer. This gradually reduces the resolution of the branch image, thereby acquiring feature information of the branch image at different scales. During the downsampling process, the size of each layer of the image is reduced accordingly, which allows the capture of different features of the image from the overall to the local. For example, at a large scale, the overall structure and general outline of the image can be seen, while at a small scale, the local details of the image can be observed. Similarly, downsampling the weight map layer by layer to obtain a weight pyramid ensures that the weight information also has a multi-scale distribution, ensuring that the weights accurately correspond to the image features at different scales.

[0092] In step S322, an image pyramid is constructed based on the Gaussian pyramid of the selected branch image. This image pyramid can further refine the representation of image features. It not only includes image information at different scales provided by the Gaussian pyramid, but also further explores the feature differences and relationships of the image at different scales. The image pyramid constructed by the above method can more comprehensively reflect the features of the image and provide richer and more accurate information for subsequent weighted fusion.

[0093] After constructing the image pyramid and weight pyramid for each branch image through the above steps, it is easier to perform subsequent weighted fusion. This allows information from different branch images at various scales to be integrated according to reasonable weights, which helps to ensure that the final target endoscopic image can clearly and accurately present the information in the endoscopic image, providing doctors with more reliable diagnostic basis.

[0094] In the embodiments, the specific operation methods for constructing Gaussian pyramids from branch images and the specific operation methods for constructing weight pyramids from weight images both adopt the conventional construction method of Gaussian pyramids, and the specific details will not be repeated here.

[0095] Specifically, when the j-th branch image is selected, the i-th layer image data of the image pyramid of the j-th branch image is: the difference between the i-th layer pyramid data and the (i+1)-th layer pyramid data in the Gaussian pyramid of the j-th branch image after upsampling and interpolation. Its specific expression is: ; in, For the image data of the i-th layer of the image pyramid of the j-th branch image, and Let be the data of the i-th and (i+1)-th layers of the Gaussian pyramid in the j-th branch image, respectively, and g be the interpolation kernel. This is the upsampling operation function. This is a convolution operation.

[0096] Preferably, step 33 includes: S331: Select the i-th layer from each image pyramid and each weight pyramid respectively, extract image data from each i-th layer image pyramid to obtain the corresponding i-th layer image data, and extract weight data from each i-th layer weight pyramid to obtain the corresponding i-th layer weight data; wherein, the i-th layer image data in each image pyramid corresponds one-to-one with the i-th layer weight data in the corresponding weight pyramid. S332: Using all the weight data of the i-th layer, perform a weighted summation operation on all the image data of the i-th layer to obtain the fused pyramid layer data of the i-th layer; S333: Traverse the image pyramid and weight pyramid layers of each layer, and obtain the fusion pyramid layer data of each layer in the same way; and obtain the fusion pyramid based on all the fusion pyramid layer data.

[0097] In step S331, by selecting data from the same level in each image pyramid and weight pyramid, the consistency of image data and weight data in scale is ensured, providing a foundation for accurate weighted fusion in the subsequent process. Each level of data represents image features at different scales and their corresponding importance weights. This one-to-one correspondence allows for precise integration of image information based on weights during subsequent fusion.

[0098] In step S332, a weighted summation operation is used to fuse all the image data of the i-th layer according to the corresponding weight data, which can fully take into account the importance of each branch image at this scale and effectively combine the advantageous information of each branch image.

[0099] In step S333, the weighted fusion process described above is repeated to obtain the fusion pyramid layer data for each layer, and the complete fusion pyramid is finally obtained based on all the fusion pyramid layer data. This fusion pyramid integrates the feature information of all branch images at various scales, which can more accurately present the details and features of the endoscopic image, resulting in a significant improvement in the final target endoscopic image in terms of brightness, contrast, reflection suppression, and local details.

[0100] Specifically, the calculation formula for the fused pyramid layer data of the i-th layer is as follows: ; Among them, F i (x,y) represents the fused pyramid layer data of the i-th layer. For the j-th image pyramid, the i-th layer image data is... Let be the weight data of the i-th layer of the j-th weight pyramid, and m be the total number of image pyramids or the total number of weight pyramids.

[0101] The data for each other layer of the fusion pyramid are calculated according to the above formula, and will not be listed here.

[0102] Preferably, step S34 includes: Starting from the top image of the fusion pyramid, upsampling is performed layer by layer and superimposed with the next layer image in the fusion pyramid to obtain the target endoscopic image.

[0103] In the above steps, the top layer image contains the overall features and general outline of the image. As the layers are pushed down, the resolution of the image is continuously increased through upsampling. At the same time, it is superimposed with the next layer image to fuse feature information at different scales. Through the above method, the image resolution can be gradually restored and information at different scales can be integrated to achieve the reconstruction of the final target endoscopic image.

[0104] Specifically, let the number of layers in the fusion pyramid be N, and let the top image of the fusion pyramid be the Nth layer image F. N (x,y), then the Nth layer image F N The image obtained by upsampling (x, y) and superimposing it with the image of the (N-1)th layer in the fusion pyramid is F. N-1 '(x,y)=UP[F N (x,y)])+ F N-1 (x,y), where UP(∙) represents the upsampling operation. This upsampling operation can increase the resolution of the Nth layer image to the same level as the (N-1)th layer image. Then, by superimposing the N-1th layer image, the feature information contained in these two layers can be fully integrated. Next, for the obtained F... N-1 For the (x,y) graph, continue with the same operation, i.e., for F... N-1 Upsample (x,y) and then overlay it with the (N-2)th layer image to obtain F. N-2 '(x,y)=UP[F N-1 '(x,y)] + F N-2 (x, y). This process continues layer by layer, starting with the top-level image, continuously upsampling and overlaying it with the next layer. In this process, each layer restores the image resolution while integrating feature information from different scales. When the first layer is reached, the entire fusion pyramid reconstruction is complete, and the final image obtained is the target endoscopic image.

[0105] When the branch images include the aforementioned first, second, third, and fourth branch images, the reconstructed target endoscopic image, by fusing feature information from each branch image at different scales, exhibits more uniform brightness, avoiding overly bright or dark areas, allowing doctors to more clearly observe the overall image. In terms of contrast, it highlights key features and details in the image, enhancing the distinction between different tissues and making it easier for doctors to identify lesions. Regarding reflection suppression, it effectively reduces the interference of reflective areas in the image, ensuring image clarity and accuracy. Furthermore, it presents richer information in local details, helping doctors to make more accurate diagnoses and analyses, providing a more reliable basis for medical diagnosis.

[0106] In an optional implementation of this embodiment, the complete flow of the multi-branch fusion endoscopic image enhancement method is as follows: Figure 3 As shown, this method effectively balances multiple requirements such as illumination balance, detail enhancement, noise suppression, and reflection suppression, achieving a comprehensive improvement in endoscopic images in terms of brightness, contrast, local detail, and noise suppression.

[0107] Example 2 A multi-branch fusion endoscopic image enhancement system is applied to the multi-branch fusion endoscopic image enhancement method in Embodiment 1, such as... Figure 4 As shown, the system includes: The image noise suppression module is used to acquire the original endoscope image and perform noise suppression processing on the original endoscope image to obtain the RGB image to be enhanced; A multi-dimensional synchronous enhancement module is used to synchronously perform multi-dimensional enhancement processing on the RGB image to be enhanced to obtain multiple branch images; wherein, the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; The multi-branch fusion module is used to fuse all branch images to obtain the target endoscopic image.

[0108] In this embodiment, the image noise suppression module performs noise suppression processing on the acquired raw endoscopic image, which can effectively reduce noise interference in the image and improve the image purity. Then, the multi-dimensional synchronous enhancement module performs multi-dimensional enhancement processing on the RGB image to be enhanced obtained from the noise suppression processing, generating multiple branch images. Each branch image can enhance and optimize the image from different angles, including but not limited to brightness enhancement, contrast adjustment, and other effects. Then, the multi-branch fusion module fuses all these branch images with different enhancement effects, which can give full play to the advantages of different enhancement methods and effectively take into account multiple requirements such as illumination balance and noise suppression.

[0109] The multi-branch fusion endoscopic image enhancement system of this embodiment, based on the simultaneous and fusion of multi-dimensional enhancement processing, can effectively take into account multiple requirements such as illumination balance and noise suppression, ensuring that the quality of the final endoscopic image is significantly improved. It meets the high standards and stability requirements for endoscopic image quality, and can provide clearer and more accurate image evidence for related medical diagnoses, which helps to improve the accuracy and efficiency of diagnosis and detection.

[0110] The functions of each module in the multi-branch fusion endoscopic image enhancement system described in this embodiment are the same as the method steps of the multi-branch fusion endoscopic image enhancement method described in Embodiment 1. Therefore, for details not covered in this embodiment, please refer to Embodiment 1 and... Figures 1 to 3 The specific details will not be repeated here.

[0111] Example 3 This embodiment also provides a multi-branch fusion endoscopic image enhancement device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the method steps in the multi-branch fusion endoscopic image enhancement method of Embodiment 1.

[0112] By using a computer program stored in memory and running on a processor, and through the simultaneous and fusion of multi-dimensional enhancement processing, multiple requirements such as illumination balance and noise suppression can be effectively met, ensuring that the quality of the final endoscopic image is significantly improved. This satisfies the high standards and stability requirements for endoscopic image quality, and provides clearer and more accurate image evidence for related medical diagnoses, thus helping to improve the accuracy and efficiency of diagnosis and testing.

[0113] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting all parts of the computer device through various interfaces and lines.

[0114] Memory can be used to store computer programs and / or models. The processor performs various functions of the computer device by running or executing the computer programs and / or models stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0115] It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by a computer program. These computer programs can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that instructions executable by the processor of the computer or other programmable data processing device generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0116] These computer programs may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0117] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0118] This embodiment also provides a computer storage medium, the computer storage medium including: at least one instruction, which, when executed by a computer, implements the method steps of the multi-branch fusion endoscopic image enhancement method of Embodiment 1.

[0119] By executing a computer storage medium containing at least one instruction, and through the synchronous processing and fusion of multi-dimensional enhancements, multiple requirements such as illumination balance and noise suppression can be effectively met, ensuring that the quality of the final endoscopic image is significantly improved. This satisfies the high standards and stability requirements for endoscopic image quality, and provides clearer and more accurate image evidence for related medical diagnoses, thus helping to improve the accuracy and efficiency of diagnosis and testing.

[0120] Similarly, for details not covered in this embodiment, please refer to Embodiment 1, Embodiment 2, and... Figures 1 to 4 The specific details will not be repeated here.

[0121] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A multi-branch fusion method for enhancing endoscopic images, characterized in that, The method includes: Acquire the original endoscopic image and perform noise suppression processing on the original endoscopic image to obtain the RGB image to be enhanced; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images; wherein, the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; All branch images are fused to obtain the target endoscopic image.

2. The multi-branch fusion endoscopic image enhancement method according to claim 1, characterized in that, The number of branch images must include at least two; The images from all branches are fused to obtain the target endoscopic image, including: Obtain the weight map corresponding to each branch image; Construct an image pyramid for each branch image and a weight pyramid for each weight image; the total number of image pyramids is the same as the total number of weight pyramids, and all image pyramids correspond one-to-one with all weight pyramids, with the same number of layers for each image pyramid and each weight pyramid. Based on all weighted pyramids, a weighted fusion of all image pyramids is performed to obtain a fused pyramid. The fused pyramid is upsampled and reconstructed to obtain the target endoscopic image.

3. The multi-branch fusion endoscopic image enhancement method according to claim 2, characterized in that, Based on all weighted pyramids, a weighted fusion of all image pyramids is performed to obtain a fused pyramid, including: The i-th layer is selected from each image pyramid and each weight pyramid respectively. Image data is extracted from each i-th layer image pyramid to obtain the corresponding i-th layer image data, and weight data is extracted from each i-th layer weight pyramid to obtain the corresponding i-th layer weight data. Among them, the i-th layer image data in each image pyramid corresponds one-to-one with the i-th layer weight data in the corresponding weight pyramid. Using all the weight data of the i-th layer, perform a weighted summation operation on all the image data of the i-th layer to obtain the fused pyramid layer data of the i-th layer; Traverse each layer of the image pyramid and weight pyramid, and obtain the fusion pyramid layer data for each layer using the same method; and obtain the fusion pyramid based on all the fusion pyramid layer data; The specific formula for calculating the fused pyramid layer data of layer i is as follows: ； Among them, F i (x,y) represents the fused pyramid layer data of the i-th layer. For the j-th image pyramid, the i-th layer image data is... Let be the weight data of the i-th layer of the j-th weight pyramid, and m be the total number of image pyramids or the total number of weight pyramids.

4. The multi-branch fusion endoscopic image enhancement method according to claim 2, characterized in that, Obtain the weight map corresponding to each branch image, including: Choose any branch image and calculate the saturation weight, contrast weight, and exposure weight of each pixel in the selected branch image. Based on the calculated saturation weight, contrast weight, and exposure weight of each pixel, the composite weight of each pixel in the selected branch image is obtained. Generate a weight map of the selected branch image based on the composite weights of all pixels in the selected branch image. Traverse each branch image and obtain the weight map of each branch image in the same way; When the j-th branch image is selected, the formula for calculating the composite weight at pixel coordinates (x, y) in the j-th branch image is: ； in, Let be the composite weight at pixel coordinates (x, y) in the j-th branch image. , and These are the saturation weight, contrast weight, and exposure weight at pixel coordinates (x, y) in the j-th branch image, respectively.

5. The multi-branch fusion endoscopic image enhancement method according to claim 2, characterized in that, Construct the image pyramid for each branch image and the weight pyramid for each weight map, including: Choose any branch image and perform layer-by-layer downsampling on the selected branch image to obtain the Gaussian pyramid corresponding to the selected branch image; also perform layer-by-layer downsampling on the weight map corresponding to the selected branch image to obtain the weight pyramid corresponding to the weight map of the selected branch image. Based on the Gaussian pyramid of the selected branch image, an image pyramid of the selected branch image is constructed; Traverse each branch image and construct the image pyramid corresponding to each branch image and the weight pyramid corresponding to each weight image in the same way; When the j-th branch image is selected, the expression for the i-th layer image data of the image pyramid of the j-th branch image is as follows: ； in, For the image data of the i-th layer of the image pyramid of the j-th branch image, and Let be the data of the i-th and (i+1)-th layers of the Gaussian pyramid in the j-th branch image, respectively, and g be the interpolation kernel. This is the upsampling operation function. This is a convolution operation.

6. The multi-branch fusion endoscopic image enhancement method according to claim 1, characterized in that, The branch image specifically includes a first branch image and a second branch image; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images, including: The first branch image is obtained by performing brightness enhancement processing on the RGB image to be enhanced using the multi-scale Retinex enhancement method. The second branch image is obtained by using a histogram equalization method to adjust the contrast of the RGB image to be enhanced.

7. The multi-branch fusion endoscopic image enhancement method according to claim 6, characterized in that, The first branch image is obtained by performing brightness enhancement processing on the RGB image to be enhanced using the multi-scale Retinex enhancement method, including: The RGB image to be enhanced is converted to grayscale to obtain a grayscale image; The grayscale image is enhanced using a multi-scale Retinex enhancement method to obtain a brightness-enhanced image; The three color components of the RGB image to be enhanced are associated and reconstructed with the brightness enhancement image to obtain the three enhanced color components corresponding to the three color components respectively; The three enhanced color components are merged to obtain the first branch image.

8. The multi-branch fusion endoscopic image enhancement method according to claim 6, characterized in that, The second branch image is obtained by performing contrast adjustment processing on the RGB image to be enhanced using a histogram equalization method, including: The RGB image to be enhanced is converted to the LAB color space to obtain the first LAB color image; Extract the L channel component from the first LAB color image, and enhance the extracted L channel component using a histogram equalization method to obtain the first enhanced L channel component. Extract the A-channel and B-channel components from the first LAB color image, and merge the first enhanced L-channel component with the extracted A-channel and B-channel components to obtain the first enhanced LAB color image. The first enhanced color LAB image is converted back to the RGB color space to obtain the second branch image.

9. The multi-branch fusion endoscopic image enhancement method according to claim 6, characterized in that, The branch image also specifically includes a third branch image; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images, and the process also includes: The RGB image to be enhanced is converted to the HSV color space to obtain an HSV color image; The HSV color image is detected using a preset brightness threshold and / or saturation threshold to obtain the highlight area; Based on the highlighted area, a texture synthesis and repair method is used to perform texture synthesis and repair on the area corresponding to the highlighted area in the RGB image to be enhanced, thereby obtaining the third branch image.

10. The multi-branch fusion endoscopic image enhancement method according to claim 6, characterized in that, The branch image also specifically includes a fourth branch image; The RGB image to be enhanced is simultaneously subjected to multi-dimensional enhancement processing to obtain multiple branch images, and the process also includes: The RGB image to be enhanced is converted to the LAB color space to obtain a second LAB color image; Extract the L channel component from the second LAB color image, and decompose the extracted L channel component to obtain the base layer and detail layer; An adaptive gain map of the detail layer is constructed, and the detail layer is modulated using the adaptive gain map to obtain an enhanced detail layer; The base layer and the enhanced detail layer are fused to obtain the second enhanced L-channel component; Extract the A channel component and B channel component from the second LAB color image, and merge the second enhanced L channel component with the extracted A channel component and the B channel component to obtain the second enhanced LAB color image; The second enhanced color LAB image is converted back to the RGB color space to obtain the fourth branch image.

11. The multi-branch fusion endoscopic image enhancement method according to any one of claims 1 to 10, characterized in that, The original endoscopic image is subjected to noise suppression processing to obtain an RGB image to be enhanced, including: The original endoscopic image is converted to the LAB color space to obtain a third LAB color image; Extract the L channel component from the third LAB color image, and denoise the extracted L channel component to obtain the denoised L channel component. Extract the A-channel and B-channel components from the third LAB color image, and merge the denoised L-channel component with the extracted A-channel and B-channel components to obtain the denoised LAB color image. The denoised LAB color image is converted back to the RGB color space to obtain the RGB image to be enhanced.

12. A multi-branch fusion endoscopic image enhancement system, characterized in that, The method for enhancing endoscopic images using multi-branch fusion as described in any one of claims 1 to 11 includes: The image noise suppression module is used to acquire the original endoscope image and perform noise suppression processing on the original endoscope image to obtain the RGB image to be enhanced; A multi-dimensional synchronous enhancement module is used to synchronously perform multi-dimensional enhancement processing on the RGB image to be enhanced to obtain multiple branch images; wherein, the enhancement processing includes at least brightness enhancement processing and contrast adjustment processing; The multi-branch fusion module is used to fuse all branch images to obtain the target endoscopic image.

13. A multi-branch fusion endoscopic image enhancement device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed, implements the method steps of the multi-branch fusion endoscopic image enhancement method as described in any one of claims 1 to 11.

14. A computer storage medium, characterized in that, The computer storage medium includes at least one instruction that, when executed by a computer, implements the method steps of the method for optimizing the display of endoscopic images as described in any one of claims 1 to 11.

Images