Underwater image enhancement method and device and electronic equipment
By acquiring the color balance and local contrast enhancement images of underwater images and performing pyramid algorithm fusion, the problem of low image quality in the prior art is solved, and a higher quality underwater image enhancement effect is achieved.
Patent Information
- Application Number
- CN202411119524.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-06
AI Technical Summary
Existing underwater image enhancement techniques usually focus on a single image attribute adjustment, focusing less on the overall visual quality of the image, resulting in a lower quality of the generated image.
By acquiring the original underwater image, determining its corresponding color balanced image and local contrast enhancement image, and calculating the fusion weight, the image fusion is fusion using the pyramid algorithm to generate the target image to improve the overall visual quality of the image.
It realizes the generation of images with richer details, better contrast and color performance, improves the overall visual quality of underwater images, and solves the problem of lower image quality.
Smart Images

Figure CN119090978A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, and in particular to an underwater image enhancement method, device and electronic equipment. Background Art
[0002] Acquiring underwater images is crucial to the development of many fields, including oceanographic research, underwater archaeology, and sustainable fishery management. However, due to the complexity of the underwater environment, light propagating in water is affected by scattering and absorption, resulting in widespread problems in underwater image quality, such as low contrast, image blur, color distortion, etc., which seriously restricts the further analysis and application of underwater images.
[0003] Underwater image enhancement technology, which aims to improve image quality by improving image contrast and color, has become an important research direction in the fields of computer vision and marine engineering.
[0004] The research on underwater image enhancement technology in related technologies mainly focuses on histogram equalization, gamma correction, image defogging and color correction. However, these methods usually only focus on adjusting a single image attribute, such as improving brightness or restoring color, and pay less attention to the overall visual quality of the image, including detail information and local contrast enhancement, so it is impossible to obtain a more balanced image in terms of color restoration, contrast enhancement and detail expression.
[0005] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0006] The embodiments of the present application provide an underwater image enhancement method, device and electronic device to at least solve the technical problem that the underwater image enhancement technology in the related art usually focuses on the adjustment of a single image attribute and pays less attention to the overall visual quality of the image, resulting in low quality of the generated image.
[0007] According to one aspect of an embodiment of the present application, there is provided an underwater image enhancement method, comprising: acquiring an original underwater image to be processed; determining a first image and a second image corresponding to the original underwater image, wherein the first image is an image after color balancing the original underwater image, and the second image is an image after local contrast enhancement of the first image; determining a fusion weight of the first image and the second image, wherein the fusion weight comprises a first fusion weight and a second fusion weight, the first fusion weight being used to represent the weight after fusion of the weight attribute of the first image, and the second fusion weight being used to represent the weight after fusion of the weight attribute of the second image; performing image fusion on the first image and the second image according to the fusion weight to obtain a target image, wherein the target image is an image after image enhancement of the original underwater image.
[0008] Optionally, determining the first image and the second image corresponding to the original underwater image includes: performing color correction on the RGB channel image of the original underwater image to obtain the first image; and performing spatial transformation on the first image to obtain the second image.
[0009] Optionally, spatially transforming the first image to obtain the second image includes: converting the first image from an RGB color space to a LAB color space to obtain a first color image; enhancing the first color image by a bilateral filter to obtain a second color image, and enhancing the first color image by an adaptive histogram equalization method to obtain a third color image; combining the second color image and the third color image to obtain a fourth color image; and converting the fourth color image from the LAB color space to the RGB color space to obtain the second image.
[0010] Optionally, determining the fusion weight of the first image and the second image includes: determining a first weight attribute, a second weight attribute, a third weight attribute and a fourth weight attribute of the first image, wherein the first weight attribute is used to represent the Laplace contrast of the first image, the second weight attribute is used to represent the local contrast of the first image, the third weight attribute is used to represent the saliency of the first image, and the fourth weight attribute is used to represent the exposure of the first image; determining a fifth weight attribute, a sixth weight attribute, a seventh weight attribute and an eighth weight attribute of the second image, wherein the fifth weight attribute is used to represent the Laplace contrast of the second image, the sixth weight attribute is used to represent the local contrast of the second image, the seventh weight attribute is used to represent the saliency of the second image, and the eighth weight attribute is used to represent the exposure of the second image; and determining the first fusion weight and the second fusion weight based on the first weight attribute, the second weight attribute, the third weight attribute, the fourth weight attribute, the fifth weight attribute, the sixth weight attribute, the seventh weight attribute and the eighth weight attribute.
[0011] Optionally, performing image fusion on the first image and the second image according to the fusion weight includes: performing image fusion on the first image and the second image through a pyramid algorithm, wherein the pyramid algorithm includes a first function and a second function, the first function is used to analyze the fusion weight of the first image and the second image, and the second function is used to analyze the RGB channel images of the first image and the second image; determining a first weight image corresponding to the first image according to the first function, and determining a second weight image corresponding to the second image according to the first function, determining a first channel image corresponding to the first image according to the second function, and determining a second channel image corresponding to the second image according to the second function; generating a target image based on the first weight image, the second weight image, the first channel image, and the second channel image.
[0012] Optionally, determining a first weight image corresponding to the first image according to a first function, determining a second weight image corresponding to the second image according to the first function, determining a first channel image corresponding to the first image according to a second function, and determining a second channel image corresponding to the second image according to a second function, includes: determining the number of levels of the pyramid algorithm during runtime, and determining a first channel of the first image and a second channel of the second image, wherein the first channel is used to represent the RGB channel corresponding to the first image, and the second channel is used to represent the RGB channel corresponding to the second image; analyzing the first image, the second image, the first fusion weight, the second fusion weight, and the number of levels according to the first function to obtain a first weight image and a second weight image; analyzing the first image, the second image, the first channel, the second channel, and the number of levels according to the second function to obtain a first channel image and a second channel image.
[0013] Optionally, generating a target image based on the first weight image, the second weight image, the first channel image and the second channel image includes: determining a fusion result in a target channel in a target level based on the first weight image and the second weight image, wherein the target level is any level in the pyramid algorithm, and the target channel is any channel in the RGB channels in the first channel or the second channel; determining a level image of each level in the pyramid algorithm based on the fusion result; and fusing the level images of all levels to obtain the target image.
[0014] According to another aspect of an embodiment of the present application, an underwater image enhancement device is also provided, including: an acquisition module, used to acquire an original underwater image to be processed; a first determination module, used to determine a first image and a second image corresponding to the original underwater image, wherein the first image is an image after color balancing the original underwater image, and the second image is an image after local contrast enhancement of the first image; a second determination module, used to determine a fusion weight of the first image and the second image, wherein the fusion weight includes a first fusion weight and a second fusion weight, the first fusion weight is used to represent the weight after fusion of the weight attribute of the first image, and the second fusion weight is used to represent the weight after fusion of the weight attribute of the second image; a fusion module, used to perform image fusion on the first image and the second image according to the fusion weight to obtain a target image, wherein the target image is an image after image enhancement of the original underwater image.
[0015] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned underwater image enhancement method.
[0016] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, the non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned underwater image enhancement method by running the computer program.
[0017] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which implement the above-mentioned underwater image enhancement method when executed by a processor.
[0018] In an embodiment of the present application, an original underwater image to be processed is obtained; a first image and a second image corresponding to the original underwater image are determined, wherein the first image is an image after color balancing the original underwater image, and the second image is an image after local contrast enhancement is performed on the first image; a fusion weight of the first image and the second image is determined, wherein the fusion weight includes a first fusion weight and a second fusion weight, the first fusion weight is used to represent the weight after fusion of the weight attribute of the first image, and the second fusion weight is used to represent the weight after fusion of the weight attribute of the second image; the first image and the second image are image-fused according to the fusion weight to obtain a target image, wherein the target image is an image after image enhancement is performed on the original underwater image, thereby achieving the purpose of generating an image with richer details, better contrast and color expression based on multi-scale fusion technology and combining weight attributes of different images, thereby achieving the technical effect of improving the overall visual quality of the underwater image, and further solving the technical problem that the underwater image enhancement technology in the related art usually focuses on a single image attribute adjustment and pays less attention to the overall visual quality of the image, resulting in low quality of the generated image. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0020] Figure 1 is a hardware structure diagram of a computer terminal for implementing an underwater image enhancement method according to an embodiment of the present application;
[0021] Figure 2 is a flow chart of an underwater image enhancement method according to an embodiment of the present application;
[0022] Figure 3 is a schematic diagram of an underwater image enhancement process according to an embodiment of the present application;
[0023] Figure 4 is a schematic diagram of an underwater image visual effect analysis according to an embodiment of the present application;
[0024] Figure 5 It is a structural diagram of an underwater image enhancement device according to an embodiment of the present application. DETAILED DESCRIPTION
[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0027] First, some nouns or terms that appear in the process of explaining the embodiments of the present application are subject to the following explanations:
[0028] RGB (Red Green Blue) is a color space composed of three basic colors: red (R), green (G), and blue (B). In computer graphics and digital image processing, the RGB color space is widely used in monitors, cameras, scanners and other devices to represent and process image colors.
[0029] LAB (Lab Color Space): Consists of lightness (L), a color range from green to red (a-axis), and a color range from blue to yellow (b-axis). The LAB color space is a device-independent color representation that can accurately describe all colors that the human eye can perceive.
[0030] Gaussian pyramid: A method for constructing an image pyramid that generates a series of images with gradually decreasing resolution by continuously performing Gaussian blur and downsampling operations on the image. Gaussian pyramids are usually used for scale space analysis of images and construction of image pyramids.
[0031] Laplacian pyramid: A method for constructing an image pyramid that generates a series of detailed images by upsampling and Gaussian blurring each layer of the image in the Gaussian pyramid. Laplacian pyramids are usually used for image detail extraction and image reconstruction.
[0032] In order to solve the problem of poor underwater image quality in the related art, the present application embodiment provides an underwater image enhancement method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is described below.
[0033] The underwater image enhancement method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing an underwater image enhancement method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0034] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).
[0035] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the underwater image enhancement method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned underwater image enhancement method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0036] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0037] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .
[0038] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.
[0039] In the above-mentioned operating environment, an embodiment of the present application provides an underwater image enhancement method embodiment. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0040] Figure 2 is a flow chart of an underwater image enhancement method according to an embodiment of the present application. Figure 2As shown, the method comprises the following steps:
[0041] Step S202, obtaining an original underwater image to be processed.
[0042] In the above step S202, the original underwater image is obtained as the starting point of image enhancement, wherein the original underwater image often has problems such as color deviation or insufficient contrast due to the particularity of the underwater environment (such as absorption and scattering).
[0043] Step S204, determining a first image and a second image corresponding to the original underwater image, wherein the first image is an image obtained by color balancing the original underwater image, and the second image is an image obtained by locally enhancing the contrast of the first image.
[0044] In the above step S204, the original underwater image is preprocessed, for example, color balancing the original underwater image and correcting the color deviation, so as to obtain a color-balanced first image, wherein the purpose of color balancing is to minimize color distortion and make the color distribution more uniform. Further, spatial transformation is performed on the basis of the first image, and local contrast enhancement is performed in combination with bilateral filter and adaptive histogram equalization technology to enhance the local details of the first image without losing important edge information, so as to obtain a second image.
[0045] Step S206, determining the fusion weight of the first image and the second image, wherein the fusion weight includes a first fusion weight and a second fusion weight, the first fusion weight is used to represent the weight after the weight attribute of the first image is fused, and the second fusion weight is used to represent the weight after the weight attribute of the second image is fused.
[0046] In the above step S206, the calculation of the fusion weight involves multiple weight attributes of the first image and the second image, such as Laplace contrast, local contrast, saliency and exposure. After fusing the multiple weight attributes of the first image and the second image respectively, a first fusion weight corresponding to the first image and a second fusion weight corresponding to the second image can be obtained, wherein the first fusion weight and the second fusion weight reflect the relative importance of the respective weight attributes in the first image and the second image.
[0047] Step S208, performing image fusion on the first image and the second image according to the fusion weight to obtain a target image, wherein the target image is an image obtained by performing image enhancement on the original underwater image.
[0048] In the above step S208, the first image and the second image can be fused according to the calculated first fusion weight and the second fusion weight, so as to obtain the target image after image enhancement. The fusion process can, for example, perform multi-level weighted fusion on each color (red, green, blue) channel through the Liplacian pyramid and the Gaussian pyramid. In the weighted fusion process, the image details of each pyramid are combined according to the corresponding fusion weight, so as to maintain the naturalness of the image while enhancing the details of the contrast.
[0049] Through the above steps S202 to S208, the purpose of generating images with richer details, better contrast and color expression based on multi-scale fusion technology and combining weight attributes of different images is achieved, thereby achieving the technical effect of improving the overall visual quality of underwater images, and further solving the technical problem that the underwater image enhancement technology in related technologies usually focuses on the adjustment of a single image attribute and pays less attention to the overall visual quality of the image, resulting in low quality of the generated image.
[0050] Figure 3 is a schematic diagram of an underwater image enhancement process according to an embodiment of the present application, such as Figure 3 As shown, the underwater image enhancement method can be roughly divided into six steps: the first step is to input the original underwater image and obtain the three color channels and size of the original underwater image, R, G, and B. The second step is to perform image preprocessing. First, perform a simple color balance on the original image to obtain the first image, and then convert the balanced first image from the RGB color space to the LAB color space for subsequent processing. The third step is local contrast enhancement. A bilateral filter and an adaptive histogram equalization are applied to enhance the contrast of the first image. Finally, the image in the LAB space is converted back to the RGB space to obtain the second image. The fourth step is to calculate the fusion weight. The Laplace contrast weight, local contrast weight, saliency weight, and exposure weight of the first image and the second image are calculated respectively, and these four weights are normalized and merged to obtain the final fusion weight of the first image and the second image. The fifth step is to perform the construction of the Gaussian pyramid and the Laplace pyramid step by step, and use the calculated fusion weight to guide the fusion of the image details in the Laplace pyramid, and obtain the fused image through the pyramid reconstruction algorithm. The sixth step is to output the multi-scale fused image, that is, the enhanced target image. The following is combined with Figure 3 The image enhancement process of steps S202 to S208 is described in detail.
[0051] In the above step S204, determining the first image and the second image corresponding to the original underwater image includes: performing color correction on the RGB channel image of the original underwater image to obtain the first image; and performing spatial transformation on the first image to obtain the second image.
[0052] Furthermore, the first image is spatially converted to obtain a second image, including: converting the first image from an RGB color space to a LAB color space to obtain a first color image; performing image enhancement on the first color image through a bilateral filter to obtain a second color image, and performing image enhancement on the first color image through an adaptive histogram equalization method to obtain a third color image; combining the second color image and the third color image to obtain a fourth color image; converting the fourth color image from the LAB color space to the RGB color space to obtain the second image.
[0053] In the embodiment of the present application, the preprocessing of the original underwater image is mainly divided into two parts: one is to color balance the RGB channel image of the original underwater image, and the other is to convert the balanced image from the RGB color space to the LAB color space for subsequent processing. The details are as follows:
[0054] 1. Color Balance
[0055] The key to image color balancing is to correct the color deviation of the original underwater image, which is usually caused by the absorption and scattering of water. The formula involved in this process is as follows:
[0056] I1=simple_color_balance(I0)
[0057] Among them, I0 represents the original underwater image; I1 represents the color-corrected image, that is, the first image mentioned above; the design goal of the function simple_color_balance is to minimize color distortion and make the color distribution more even, laying a good foundation for the subsequent enhancement steps.
[0058] 2. Space Conversion
[0059] First, when performing color space conversion, the color-balanced first image I1 can be converted from the RGB color space to the LAB color space to obtain a converted image, namely the first color image LAB1. The first color image LAB1 consists of three channels: L represents brightness, and A and B represent color opposition data from green to red and from blue to yellow. Through this space conversion, the brightness and color of the first image can be processed independently, which facilitates the enhancement of local contrast and brightness details.
[0060] Secondly, in the LAB color space, the local contrast enhancement of the luminance channel L can be achieved by applying a bilateral filter and an adaptive histogram equalization technique, the purpose of which is to enhance the local contrast of the first image without affecting the A and B channels. The details are as follows:
[0061] The bilateral filter can smooth the color transition area while maintaining the image edge to obtain the second color image L′. The specific formula is as follows:
[0062] L′=bilateralFilter(LAB1)
[0063] Among them, the bilateralFilter function is a bilateral filter, LAB1 is the above-mentioned first color image, and L′ is the second color image processed by the bilateral filter.
[0064] Adaptive histogram equalization technology is a method for processing local image contrast. It adjusts the distribution of image brightness to make it more uniform and improve image details, thereby obtaining the third color image L″. The specific formula is as follows:
[0065] L″=adapthisteq(L′)
[0066] Wherein, adapthisteq is the contrast limiter of the adaptive histogram equalization technology, and L″ is the enhanced third color image.
[0067] Then, the enhanced third color image L″ channel is combined with the unchanged A and B channels to obtain a combined fourth color image LAB2. The specific formula is as follows:
[0068]
[0069] in, and They are the A channel and B channel of the first color image LAB1 respectively.
[0070] Finally, the fourth color image LAB2 is converted back to the RGB color space to obtain a clearer and higher contrast second image I2. The specific formula is as follows:
[0071] I2 = lab_to_rgb(LAB2)
[0072] Among them, lab_to_rgb is the method function for converting from LAB color space to RGB color space.
[0073] In the above step S206, determining the fusion weights of the first image and the second image includes: determining a first weight attribute, a second weight attribute, a third weight attribute and a fourth weight attribute of the first image, wherein the first weight attribute is used to represent the Laplace contrast of the first image, the second weight attribute is used to represent the local contrast of the first image, the third weight attribute is used to represent the saliency of the first image, and the fourth weight attribute is used to represent the exposure of the first image; determining a fifth weight attribute, a sixth weight attribute, a seventh weight attribute and an eighth weight attribute of the second image, wherein the fifth weight attribute is used to represent the Laplace contrast of the second image, the sixth weight attribute is used to represent the local contrast of the second image, the seventh weight attribute is used to represent the saliency of the second image, and the eighth weight attribute is used to represent the exposure of the second image; and determining the first fusion weight and the second fusion weight according to the first weight attribute, the second weight attribute, the third weight attribute, the fourth weight attribute, the fifth weight attribute, the sixth weight attribute, the seventh weight attribute and the eighth weight attribute.
[0074] In the embodiment of the present application, the calculation of the fusion weight involves four main image weight attributes: Laplace contrast (WL), local contrast (WLC), saliency (WS) and exposure (WE), and each weight attribute can generate a corresponding weight map. Among them, the Laplace weight can be calculated by using the Laplace operator; the local contrast weight can be obtained by measuring the brightness contrast of each area in the image and performing Gaussian blur processing on it; the saliency weight can be obtained by a preset saliency detection algorithm; the exposure weight can be calculated based on the exposure degree of the pixel.
[0075] For the first image and the second image generated above, by combining their respective weight attributes, a first fusion weight corresponding to the first image and a second fusion weight corresponding to the second image can be calculated. Specifically:
[0076] Determine the Laplace contrast WL1 (i.e., the first weight attribute), local contrast WLC1 (i.e., the second weight attribute), saliency WS1 (i.e., the third weight attribute), and exposure WE1 (i.e., the fourth weight attribute) of the first image, and the Laplace contrast WL2 (i.e., the fifth weight attribute), local contrast WLC2 (i.e., the sixth weight attribute), saliency WS2 (i.e., the seventh weight attribute), and exposure WE2 (i.e., the eighth weight attribute) of the second image.
[0077] By normalizing and merging the above eight weight attributes, we can obtain the first fusion weight corresponding to the first image and the second fusion weight corresponding to the second image. The specific formula is as follows:
[0078]
[0079]
[0080] Among them, W1 represents the first fusion weight, and W2 represents the second fusion weight.
[0081] In the above step S208, the first image and the second image are fused according to the fusion weight, including: fusing the first image and the second image through a pyramid algorithm, wherein the pyramid algorithm includes a first function and a second function, the first function is used to analyze the fusion weight of the first image and the second image, and the second function is used to analyze the RGB channel images of the first image and the second image; determining a first weight image corresponding to the first image according to the first function, and determining a second weight image corresponding to the second image according to the first function, determining a first channel image corresponding to the first image according to the second function, and determining a second channel image corresponding to the second image according to the second function; generating a target image according to the first weight image, the second weight image, the first channel image, and the second channel image.
[0082] In an embodiment of the present application, a pyramid algorithm may be used to fuse the first image and the second image, wherein the pyramid fusion algorithm is an advanced image processing technique, which is generally used to fuse multiple images into a single image in order to obtain better details, contrast, and color performance in the final image, and is particularly suitable for processing images with complementary information, such as images of the same scene captured under different exposure conditions. The purpose of the pyramid fusion algorithm is to utilize the best parts of the input image while minimizing noise and inconsistency.
[0083] The pyramid algorithm can use Gaussian pyramid and Laplace pyramid to perform multi-level image fusion when performing multi-scale fusion, that is, the pyramid algorithm can include Gaussian pyramid function (that is, the first function mentioned above) and Laplace pyramid function (that is, the second function mentioned above). Among them, the Gaussian pyramid function is used to analyze the fusion weight of the first image and the second image, and the Laplace pyramid function is used to analyze the RGB channel images of the first image and the second image. Therefore, the weight image corresponding to the first image and the second image (such as the first weight image and the second weight image mentioned above) can be obtained through the Gaussian pyramid function, and the channel image corresponding to the first image and the second image (such as the first channel image and the second channel image mentioned above) can be obtained through the Laplace pyramid function.
[0084] Optionally, determining a first weight image corresponding to the first image according to a first function, determining a second weight image corresponding to the second image according to the first function, determining a first channel image corresponding to the first image according to a second function, and determining a second channel image corresponding to the second image according to a second function, includes: determining the number of levels of the pyramid algorithm during runtime, and determining a first channel of the first image and a second channel of the second image, wherein the first channel is used to represent the RGB channel corresponding to the first image, and the second channel is used to represent the RGB channel corresponding to the second image; analyzing the first image, the second image, the first fusion weight, the second fusion weight, and the number of levels according to the first function to obtain a first weight image and a second weight image; analyzing the first image, the second image, the first channel, the second channel, and the number of levels according to the second function to obtain a first channel image and a second channel image.
[0085] In the embodiment of the present application, the specific process of obtaining the weight image and the channel image through the Gaussian pyramid function and the Laplace pyramid function can be shown in Algorithm 1:
[0086]
[0087]
[0088] Among them, Weight1 is the first weight image; Weight2 is the second weight image; the first channel image includes R1, G1, B1; the second channel image includes R2, G2, B2; level indicates the number of levels that will be created for each pyramid. The default setting is 5. The more levels there are, the more levels of detail the pyramid can represent. It can be modified as needed.
[0089] It is easy to know from the above algorithm 1 that the first weight image Weight1 and the second weight image Weight2 can be obtained by analyzing the first image I1, the second image I2, the first fusion weight W1, the second fusion weight W2 and the number of levels according to the Gaussian pyramid function; the first channel image (R1, G1, B1) and the second channel image (R2, G2, B2) can be obtained by analyzing the first image I1, the second image I2, the first channel corresponding to the first image (R, G, B channels of the first image), the second channel corresponding to the second image (R, G, B channels of the second image) and the number of levels according to the Laplace pyramid function.
[0090] In Algorithm 1, the Gaussian pyramid function can smooth and gradually reduce the resolution of the first fusion weight W1 and the second fusion weight W2. The Laplacian pyramid function can extract and create a Laplacian pyramid for each color channel (red, green, blue) in the input first image I1 and the second image I2, wherein the first channel image (R1, G1, B1) is the Laplacian pyramid image of the red, green, and blue channels of the first image I1, respectively, and the second channel image (R2, G2, B2) is the Laplacian pyramid image of the red, green, and blue channels of the second image I2, respectively. The Laplacian pyramid is created by subtracting the upsampled image of the previous level from one level in the Gaussian pyramid, thereby obtaining a series of images containing different scale detail information of the original underwater image.
[0091] Optionally, generating a target image based on the first weight image, the second weight image, the first channel image and the second channel image includes: determining a fusion result in a target channel in a target level based on the first weight image and the second weight image, wherein the target level is any level in the pyramid algorithm, and the target channel is any channel in the RGB channels in the first channel or the second channel; determining a level image of each level in the pyramid algorithm based on the fusion result; and fusing the level images of all levels to obtain the target image.
[0092] In the embodiment of the present application, the weight image is used to fuse each color channel (R, G, B) to obtain an enhanced image, that is, the above-mentioned target image. The specific process can be shown in Algorithm 2:
[0093]
[0094] In the above algorithm 2, the fusion results are R_r{i}, G_g{i} and B_b{i}, where R_r{i} represents the weighted fusion result of the red channel in any level (i.e., the above target level); G_g{i} represents the weighted fusion result of the green channel in any level; B_b{i} represents the weighted fusion result of the blue channel in any level.
[0095] Subsequently, the complete image of each color channel is reconstructed according to the fused Laplacian pyramid layer, where pyramid_reconstruct(R_r), pyramid_reconstruct(G_g), and pyramid_reconstruct(B_b) represent the image reconstruction process for the red, green, and blue channels, respectively. They start from the top layer of the pyramid and reconstruct layer by layer. Each step adds the details of the current layer to the more blurred image reconstructed by the previous step, so that each step can restore more details of the original underwater image until an image of the original underwater image size is reconstructed.
[0096] Once the images of the red, green, and blue channels are reconstructed, these individual color channels can be combined into a complete color image. For example, the step cat(3,uint8(R),uint8(G),uint8(B)) actually merges the three color channels according to the third dimension (color channel dimension), where uint8 ensures that the image data is stored in a standard 8-bit unsigned integer format. After the above operations, the final target image fusion is generated, which combines the best features of the input original underwater image and provides richer details, better contrast and color performance.
[0097] In the embodiment of the present application, a classic original underwater image is used as an experimental image, and the experimental results of the MSR algorithm, the MSRCP algorithm and the LIME algorithm are compared. The visual effect analysis is as follows: Figure 4 As shown. From the visual effect analysis, it can be seen that for underwater images with poor shooting effects and blurred image quality, various underwater image enhancement methods can improve the image quality. Among them, the MSR algorithm greatly retains the color of the original image, but the image is still relatively blurred. For images with foreground objects, the enhancement effect is not good. The MSRCR algorithm can highlight the front object and improve the enhancement effect, but it affects the color of the original image and causes color difference. When enhancing the image, the LIME algorithm highlights the dynamic objects while retaining the original color of the image, which improves the contrast of the image to a certain extent, but the effect is not obvious and there is a slight halo phenomenon, and the contrast of the dark area of the static image is not obvious and the brightness is low. The underwater image enhancement method proposed in this application distinguishes the foreground object and the background on the basis of retaining the original color of the image, performs specific enhancement on the foreground and dynamic objects, corrects the color distortion in the image, improves the image quality, and at the same time increases the contrast and brightness to avoid overexposure, thereby achieving effective enhancement of underwater images.
[0098] In general, the underwater images processed by the proposed algorithms look clearer and more natural to the human eye. However, in order to fully evaluate the application effect of computer vision technology, we also need to analyze the image information from an objective perspective, such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), tone mapping image quality index (TMQI), no-reference image quality evaluation index (NIQE), block contrast quality index (PCQI) and brightness sorting error (LOE), as follows:
[0099] (1) Peak Signal-to-Noise Ratio (PSNR): This metric reveals the level of distortion produced by the image enhancement operation compared to the original ideal illumination image. An increase in the PSNR value indicates less distortion, which represents an improvement in image quality.
[0100] (2) Structural Similarity Index (SSIM): It is used to evaluate the consistency of the structural characteristics of the enhanced image with the original image. The increase in the SSIM value indicates that the structures of the two images are closer.
[0101] (3) Tone Mapping Image Quality Index (TMQI): Based on the principle of SSIM, this index evaluates whether the color of the enhanced image remains consistent with the original image. A high value of TMQI indicates good color consistency, so the image quality is improved.
[0102] (4) No-reference Image Quality Evaluation (NIQE): This metric is based on a model of natural scene statistical properties and evaluates images using a set of quality-aware statistical properties. A lower NIQE value indicates that the image quality is closer to the visual preference of the human eye.
[0103] (5) Block Contrast Quality Index (PCQI): This is an image quality evaluation method that focuses on local areas. By evaluating local contrast, a high PCQI value indicates that better contrast is obtained during image enhancement, improving image quality.
[0104] (6) Luminance Ordering Error (LOE): This method determines the naturalness of an image based on the relative order of the luminance sequence, which is critical for characterizing the direction and changes of lighting. A lower LOE value indicates a smaller luminance error and a more natural image.
[0105] These indicators can objectively measure the quality of the image after enhancement, and have the advantages of simple operation, low cost, easy analysis and implementation. By referring to multiple evaluation results, the quality of the image after enhancement can be analyzed more objectively. The evaluation results are shown in Table 1.
[0106] Table 1 Objective indicator analysis
[0107]
[0108] Through the analysis of objective indicators, it is known that the underwater image enhancement method proposed in the embodiment of the present application is superior to other algorithms in multiple performance indicators.
[0109] In the embodiment of the present application, an underwater image enhancement method based on multi-scale fusion technology is provided, which aims to improve the visibility and quality of underwater images. Through steps such as color balance, local contrast enhancement, and multi-scale fusion technology, the visual effect of underwater images is effectively improved, including color restoration, contrast enhancement, and detail enhancement. Through experimental verification, it is proved that this method has a good enhancement effect for various types of underwater scenes, especially for images with severe color distortion and low contrast. Compared with the current mainstream underwater image enhancement methods (such as MSR, MSRCR and LIME algorithms), the underwater image enhancement method proposed in this application shows excellent performance in visual effects and multiple objective evaluation indicators. At the same time, considering the importance of underwater image enhancement in fields such as oceanographic research, underwater archaeology, and sustainable fishery management, the excellent performance of this method indicates that it can provide effective image processing tools for these fields, helping professionals engaged in these studies to obtain higher quality visual data, thereby improving work efficiency and the quality of research results.
[0110] According to an embodiment of the present application, an underwater image enhancement device is provided. It should be noted that the underwater image enhancement device of the embodiment of the present application can be used to execute the underwater image enhancement method provided by the embodiment of the present application. The underwater image enhancement device provided by the embodiment of the present application is introduced below.
[0111] Figure 5 is a structural diagram of an underwater image enhancement device provided according to an embodiment of the present application. Figure 5 As shown, the device comprises:
[0112] An acquisition module 50 is used to acquire an original underwater image to be processed;
[0113] A first determination module 52 is used to determine a first image and a second image corresponding to the original underwater image, wherein the first image is an image obtained by color balancing the original underwater image, and the second image is an image obtained by locally enhancing the contrast of the first image;
[0114] A second determination module 54 is used to determine a fusion weight of the first image and the second image, wherein the fusion weight includes a first fusion weight and a second fusion weight, the first fusion weight is used to represent the weight after the weight attribute of the first image is fused, and the second fusion weight is used to represent the weight after the weight attribute of the second image is fused;
[0115] The fusion module 56 is used to perform image fusion on the first image and the second image according to the fusion weight to obtain a target image, wherein the target image is an image obtained by performing image enhancement on the original underwater image.
[0116] Through the acquisition module 50, the first determination module 52, the second determination module 54 and the fusion module 56 in the above-mentioned underwater image enhancement device, the purpose of generating images with richer details, better contrast and color expression based on multi-scale fusion technology and combining the weight attributes of different images is achieved, thereby achieving the technical effect of improving the overall visual quality of underwater images, and further solving the technical problem that the underwater image enhancement technology in the related art usually focuses on the adjustment of a single image attribute and pays less attention to the overall visual quality of the image, resulting in low quality of the generated image.
[0117] In the underwater image enhancement device provided in the embodiment of the present application, the first determination module is also used to perform color correction on the RGB channel image of the original underwater image to obtain a first image; and perform spatial transformation on the first image to obtain a second image.
[0118] In the underwater image enhancement device provided in an embodiment of the present application, the first determination module is also used to convert the first image from the RGB color space to the LAB color space to obtain a first color image; perform image enhancement on the first color image through a bilateral filter to obtain a second color image, and perform image enhancement on the first color image through an adaptive histogram equalization method to obtain a third color image; combine the second color image and the third color image to obtain a fourth color image; convert the fourth color image from the LAB color space to the RGB color space to obtain the second image.
[0119] In the underwater image enhancement device provided in an embodiment of the present application, the second determination module is also used to determine a first weight attribute, a second weight attribute, a third weight attribute and a fourth weight attribute of the first image, wherein the first weight attribute is used to represent the Laplace contrast of the first image, the second weight attribute is used to represent the local contrast of the first image, the third weight attribute is used to represent the saliency of the first image, and the fourth weight attribute is used to represent the exposure of the first image; determine a fifth weight attribute, a sixth weight attribute, a seventh weight attribute and an eighth weight attribute of the second image, wherein the fifth weight attribute is used to represent the Laplace contrast of the second image, the sixth weight attribute is used to represent the local contrast of the second image, the seventh weight attribute is used to represent the saliency of the second image, and the eighth weight attribute is used to represent the exposure of the second image; determine a first fusion weight and a second fusion weight based on the first weight attribute, the second weight attribute, the third weight attribute, the fourth weight attribute, the fifth weight attribute, the sixth weight attribute, the seventh weight attribute and the eighth weight attribute.
[0120] In the underwater image enhancement device provided in an embodiment of the present application, the fusion module is also used to perform image fusion on the first image and the second image through a pyramid algorithm, wherein the pyramid algorithm includes a first function and a second function, the first function is used to analyze the fusion weight of the first image and the second image, and the second function is used to analyze the RGB channel images of the first image and the second image; a first weight image corresponding to the first image is determined according to the first function, and a second weight image corresponding to the second image is determined according to the first function, a first channel image corresponding to the first image is determined according to the second function, and a second channel image corresponding to the second image is determined according to the second function; a target image is generated based on the first weight image, the second weight image, the first channel image, and the second channel image.
[0121] In the underwater image enhancement device provided in an embodiment of the present application, the fusion module is also used to determine the number of levels of the pyramid algorithm during operation, and to determine the first channel of the first image and the second channel of the second image, wherein the first channel is used to represent the RGB channel corresponding to the first image, and the second channel is used to represent the RGB channel corresponding to the second image; according to the first function, the first image, the second image, the first fusion weight, the second fusion weight and the number of levels are analyzed to obtain the first weighted image and the second weighted image; according to the second function, the first image, the second image, the first channel, the second channel, and the number of levels are analyzed to obtain the first channel image and the second channel image.
[0122] In the underwater image enhancement device provided in an embodiment of the present application, the fusion module is also used to determine the fusion result in the target channel in the target level based on the first weight image and the second weight image, wherein the target level is any level in the pyramid algorithm, and the target channel is any channel in the RGB channels in the first channel or the second channel; determine the level image of each level in the pyramid algorithm based on the fusion result; and fuse the level images of all levels to obtain the target image.
[0123] An embodiment of the present application also provides an electronic device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above-mentioned underwater image enhancement method.
[0124] It should be noted that the above electronic equipment is used to execute Figure 2 The underwater image enhancement method shown, therefore the relevant explanations and instructions in the above underwater image enhancement method are also applicable to the electronic device and will not be repeated here.
[0125] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned underwater image enhancement method by running the computer program.
[0126] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The underwater image enhancement method shown, therefore the relevant explanations in the above underwater image enhancement method are also applicable to the non-volatile storage medium, and will not be repeated here.
[0127] An embodiment of the present application also provides a computer program product, including computer instructions, which implement the above-mentioned underwater image enhancement method when executed by a processor.
[0128] It should be noted that the above-mentioned computer program product is used to execute Figure 2 The underwater image enhancement method shown, therefore the relevant explanations and instructions in the above underwater image enhancement method are also applicable to the computer program product, and will not be repeated here.
[0129] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0130] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0131] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0132] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0133] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0134] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.
[0135] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A method for underwater image enhancement, It is characterized in that include: Acquire the original underwater image to be processed; Determine a first image and a second image corresponding to the original underwater image, wherein the first image is an image after color balancing the original underwater image, and the second image is an image after local contrast enhancement of the first image; Determining a fusion weight of the first image and the second image, wherein the fusion weight includes a first fusion weight and a second fusion weight, the first fusion weight is used to represent a weight after the weight attribute of the first image is fused, and the second fusion weight is used to represent a weight after the weight attribute of the second image is fused; The first image and the second image are fused according to the fusion weight to obtain a target image, wherein the target image is an image obtained by performing image enhancement on the original underwater image.
2. The method according to claim 1, It is characterized in that Determining a first image and a second image corresponding to the original underwater image includes: Performing color correction on the RGB channel image of the original underwater image to obtain the first image; Performing spatial transformation on the first image to obtain the second image.
3. The method according to claim 2, It is characterized in that Performing spatial transformation on the first image to obtain the second image includes: Convert the first image from the RGB color space to the LAB color space to obtain a first color image; Performing image enhancement on the first color image by using a bilateral filter to obtain a second color image, and performing image enhancement on the first color image by using an adaptive histogram equalization method to obtain a third color image; combining the second color image and the third color image to obtain a fourth color image; The fourth color image is converted from the LAB color space to the RGB color space to obtain the second image.
4. The method according to claim 1, It is characterized in that Determining a fusion weight of the first image and the second image includes: Determining a first weight attribute, a second weight attribute, a third weight attribute, and a fourth weight attribute of the first image, wherein the first weight attribute is used to represent the Laplace contrast of the first image, the second weight attribute is used to represent the local contrast of the first image, the third weight attribute is used to represent the saliency of the first image, and the fourth weight attribute is used to represent the exposure of the first image; determining a fifth weight attribute, a sixth weight attribute, a seventh weight attribute, and an eighth weight attribute of the second image, wherein the fifth weight attribute is used to represent the Laplace contrast of the second image, the sixth weight attribute is used to represent the local contrast of the second image, the seventh weight attribute is used to represent the saliency of the second image, and the eighth weight attribute is used to represent the exposure of the second image; The first fusion weight and the second fusion weight are determined according to the first weight attribute, the second weight attribute, the third weight attribute, the fourth weight attribute, the fifth weight attribute, the sixth weight attribute, the seventh weight attribute, and the eighth weight attribute.
5. The method according to claim 1, It is characterized in that Performing image fusion on the first image and the second image according to the fusion weight includes: Performing image fusion on the first image and the second image by using a pyramid algorithm, wherein the pyramid algorithm includes a first function and a second function, the first function is used to analyze the fusion weight of the first image and the second image, and the second function is used to analyze the RGB channel images of the first image and the second image; Determine a first weight image corresponding to the first image according to the first function, determine a second weight image corresponding to the second image according to the first function, determine a first channel image corresponding to the first image according to the second function, and determine a second channel image corresponding to the second image according to the second function; The target image is generated according to the first weight image, the second weight image, the first channel image, and the second channel image.
6. The method according to claim 5, It is characterized in that Determining a first weight image corresponding to the first image according to the first function, determining a second weight image corresponding to the second image according to the first function, determining a first channel image corresponding to the first image according to the second function, and determining a second channel image corresponding to the second image according to the second function, comprising: Determine the number of levels of the pyramid algorithm when running, and determine a first channel of the first image and a second channel of the second image, wherein the first channel is used to represent an RGB channel corresponding to the first image, and the second channel is used to represent an RGB channel corresponding to the second image; Analyzing the first image, the second image, the first fusion weight, the second fusion weight, and the number of levels according to the first function to obtain the first weighted image and the second weighted image; The first image, the second image, the first channel, the second channel, and the number of levels are analyzed according to the second function to obtain the first channel image and the second channel image.
7. The method according to claim 5, It is characterized in that Generating the target image according to the first weight image, the second weight image, the first channel image and the second channel image comprises: Determine a fusion result in a target channel in a target level according to the first weight image and the second weight image, wherein the target level is any level in the pyramid algorithm, and the target channel is any one of the RGB channels in the first channel or the second channel; Determining a level image of each level in the pyramid algorithm according to the fusion result; The hierarchical images of all levels are fused to obtain the target image.
8. An underwater image enhancement device, It is characterized in that include: An acquisition module, used for acquiring the original underwater image to be processed; A first determination module is used to determine a first image and a second image corresponding to the original underwater image, wherein the first image is an image obtained by color balancing the original underwater image, and the second image is an image obtained by locally enhancing the contrast of the first image; a second determination module, configured to determine a fusion weight of the first image and the second image, wherein the fusion weight includes a first fusion weight and a second fusion weight, the first fusion weight is used to represent a weight after the weight attribute of the first image is fused, and the second fusion weight is used to represent a weight after the weight attribute of the second image is fused; A fusion module is used to perform image fusion on the first image and the second image according to the fusion weight to obtain a target image, wherein the target image is an image obtained by performing image enhancement on the original underwater image.
9. An electronic device, It is characterized in that include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory and is used to execute the underwater image enhancement method described in any one of claims 1 to 7.
10. A non-volatile storage medium, It is characterized in that The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the underwater image enhancement method described in any one of claims 1 to 7 by running the computer program.
11. A computer program product comprising computer instructions, It is characterized in that When the computer instructions are executed by a processor, the underwater image enhancement method described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Underwater image enhancement method based on hierarchical weighted fusion
CN106971379A
Underwater image color correction and contrast enhancement method based on multi-feature fusion
CN111047530A
Underwater scattering image multi-scale fusion restoration method and device
CN113538309A
Underwater image enhancement method and device, electronic equipment and storage medium
CN116258635A
Image restoration method and device, electronic equipment and storage medium
CN117934302A
