Image processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining a plurality of continuous frame images, enabling the continuous frame images to respectively correspond to different exposure states, enabling the exposure states to comprise an overexposure state, an underexposure state and a normal state, carrying out the mirror image edge extension processing and registration processing of the continuous frame images, and enabling the registration of the continuous frame images to be corresponding to the overexposure state, the underexposure state and the normal state; the method comprises the steps of obtaining a plurality of original images, determining a plurality of weight values corresponding to the plurality of original images, the weight values comprising an exposure weight value, a brightness weight value and a saturation weight value, and performing fusion processing on the plurality of original images according to the plurality of weight values to obtain a target image. According to the method, the quality of the fused image is improved.

Description

Technical Field

[0001] The present application belongs to the technical field of image processing, and specifically relates to an image processing method, apparatus, device and storage medium. Background Art

[0002] At present, multi-frame image fusion technology can comprehensively utilize the useful information in multiple frame images, remove or reduce the influence of adverse factors such as noise, and generate a fused image with higher quality and richer information.

[0003] Multi-frame image fusion can be achieved by first correctly matching the image frames in spatial position to align them, then performing information fusion processing on the multiple frames. The aligned image pixels can be weighted averaged. After determining the weight coefficient for each frame, the corresponding pixels of each frame are added together according to the weight to obtain the fused pixel value. Alternatively, pixels can be selected for fusion based on characteristics such as the contrast of the surrounding area. Alternatively, the image can be decomposed into low-frequency and high-frequency sub-bands. The low-frequency portion can be weighted fused, and the high-frequency portion can be selectively fused based on the size of the wavelet coefficients. The fused image can then be obtained through an inverse transform.

[0004] In the above fusion process, fusion algorithms with lower complexity may have image problems such as ghosting, low dynamic range, color loss or inaccuracy, resulting in low quality of the fused image. Summary of the Invention

[0005] The embodiments of the present application relate to an image processing method, apparatus, device and storage medium, which are used to solve the defects of low-complexity fusion algorithms in the prior art, such as ghosting, low dynamic range, color loss or inaccuracy, etc., which lead to low quality of fused images.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0007] Acquire a plurality of continuous frame images, wherein the plurality of continuous frame images respectively correspond to different exposure states, wherein the exposure states include an overexposure state, an underexposure state, and a normal state;

[0008] Perform mirror edge extension and registration processing on multiple continuous frame images to obtain multiple original images;

[0009] Determining a plurality of weight values ​​corresponding to the plurality of original images respectively, the weight values ​​comprising an exposure weight value, a brightness weight value, and a saturation weight value;

[0010] The multiple original images are fused according to the multiple weight values ​​to obtain a target image.

[0011] In a possible implementation, determining a plurality of weight values ​​corresponding to the plurality of original images includes:

[0012] Determining exposure weight values ​​corresponding to the plurality of original images respectively;

[0013] Determining brightness weight values ​​corresponding to the plurality of original images respectively;

[0014] Determine saturation weight values ​​corresponding to the multiple original images respectively.

[0015] In a possible implementation, determining exposure weight values ​​corresponding to the plurality of original images includes:

[0016] Determining exposure times and analog gains corresponding to the plurality of original images respectively;

[0017] Get the preset exposure ratio;

[0018] determining a target gain and a plurality of exposure ratios based on a plurality of exposure times, a plurality of analog gains, and the preset exposure ratio, wherein the exposure ratio represents a ratio between exposure values ​​corresponding to two of the plurality of original images, and the exposure value is a product of the exposure times corresponding to the original images and the corresponding analog gains;

[0019] Exposure weight values ​​corresponding to the plurality of original images are determined according to the preset exposure ratio, the target gain, and the plurality of exposure ratios.

[0020] In a possible implementation, the multiple original images include a reference image and at least one non-reference image, and determining brightness weight values ​​corresponding to the multiple original images includes:

[0021] Determine the average brightness values ​​corresponding to the plurality of color channels in the reference image;

[0022] determining a maximum brightness value among a plurality of brightness average values ​​and a target average value of the sum of the plurality of brightness average values;

[0023] Determining a brightness weight value corresponding to the reference image according to the exposure state of the reference image, the maximum brightness value, and the target average value;

[0024] Determine, according to the maximum brightness value, a brightness weight value corresponding to each of the at least one non-reference images.

[0025] In a possible implementation, for any original image, determining a saturation weight value corresponding to the original image includes:

[0026] Determine a plurality of center points corresponding to the original image;

[0027] For any center point, determine the red value, green value, and blue value corresponding to the center point respectively;

[0028] determining a color maximum and a color minimum among the red value, the green value, and the blue value;

[0029] A saturation weight value corresponding to the original image is determined according to the multiple color maximum values ​​and the multiple color minimum values.

[0030] In a possible implementation, performing fusion processing on the multiple original images according to the multiple weight values ​​to obtain a target image includes:

[0031] Performing block processing on the multiple original images to obtain multiple original image blocks;

[0032] Performing multiple downsampling processes on the multiple original image blocks to obtain multiple target sampled image blocks;

[0033] Determining a plurality of target difference weight values ​​according to the plurality of target sampling image blocks;

[0034] performing multiple fusion processing on the multiple target sample image blocks according to the multiple target difference weight values ​​and the multiple weight values ​​to obtain multiple fused image blocks;

[0035] The multiple fused image blocks are stitched together to obtain a target image.

[0036] In a possible implementation, determining multiple target difference weight values ​​according to the multiple target sample image blocks includes:

[0037] Performing brightness alignment processing on the multiple target sample images to obtain the multiple processed images, wherein the multiple processed images include a reference processed image and at least one non-reference processed image;

[0038] For any non-reference processed image, determining a difference matrix and a similarity matrix according to the reference processed image and the non-reference processed image;

[0039] A difference weight value corresponding to the non-reference processed image is determined according to the difference matrix, the similarity matrix and preset configuration parameters.

[0040] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising:

[0041] An acquisition module is used to acquire a plurality of continuous frame images, wherein the plurality of continuous frame images respectively correspond to different exposure states, wherein the exposure states include an overexposure state, an underexposure state, and a normal state;

[0042] The first processing module is used to perform mirroring and registration processing on a plurality of continuous frame images to obtain a plurality of original images;

[0043] a determination module, configured to determine a plurality of weight values ​​corresponding to the plurality of original images, the weight values ​​comprising an exposure weight value, a brightness weight value, and a saturation weight value;

[0044] The second processing module is used to perform fusion processing on the multiple original images according to the multiple weight values ​​to obtain a target image.

[0045] In a possible implementation, the determination module is specifically configured to:

[0046] Determining exposure weight values ​​corresponding to the plurality of original images respectively;

[0047] Determining brightness weight values ​​corresponding to the plurality of original images respectively;

[0048] Determine saturation weight values ​​corresponding to the multiple original images respectively.

[0049] In a possible implementation, the determination module is specifically configured to:

[0050] Determining exposure times and analog gains corresponding to the plurality of original images respectively;

[0051] Get the preset exposure ratio;

[0052] determining a target gain and a plurality of exposure ratios based on a plurality of exposure times, a plurality of analog gains, and the preset exposure ratio, wherein the exposure ratio represents a ratio between exposure values ​​corresponding to two of the plurality of original images, and the exposure value is a product of the exposure times corresponding to the original images and the corresponding analog gains;

[0053] Exposure weight values ​​corresponding to the plurality of original images are determined according to the preset exposure ratio, the target gain, and the plurality of exposure ratios.

[0054] In a possible implementation, the multiple original images include a reference image and at least one non-reference image, and the determination module is specifically configured to:

[0055] Determine the average brightness values ​​corresponding to the plurality of color channels in the reference image;

[0056] determining a maximum brightness value among a plurality of brightness average values ​​and a target average value of the sum of the plurality of brightness average values;

[0057] Determining a brightness weight value corresponding to the reference image according to the exposure state of the reference image, the maximum brightness value, and the target average value;

[0058] Determine, according to the maximum brightness value, a brightness weight value corresponding to each of the at least one non-reference images.

[0059] In a possible implementation, for any original image, the determination module is specifically configured to:

[0060] Determine a plurality of center points corresponding to the original image;

[0061] For any center point, determine the red value, green value, and blue value corresponding to the center point respectively;

[0062] determining a color maximum and a color minimum among the red value, the green value, and the blue value;

[0063] A saturation weight value corresponding to the original image is determined according to the multiple color maximum values ​​and the multiple color minimum values.

[0064] In a possible implementation, the second processing module is specifically configured to:

[0065] Performing block processing on the multiple original images to obtain multiple original image blocks;

[0066] Performing multiple downsampling processes on the multiple original image blocks to obtain multiple target sampled image blocks;

[0067] Determining a plurality of target difference weight values ​​according to the plurality of target sampling image blocks;

[0068] performing multiple fusion processing on the multiple target sample image blocks according to the multiple target difference weight values ​​and the multiple weight values ​​to obtain multiple fused image blocks;

[0069] The multiple fused image blocks are stitched together to obtain a target image.

[0070] In a possible implementation, the second processing module is specifically configured to:

[0071] Performing brightness alignment processing on the multiple target sample images to obtain the multiple processed images, wherein the multiple processed images include a reference processed image and at least one non-reference processed image;

[0072] For any non-reference processed image, determining a difference matrix and a similarity matrix according to the reference processed image and the non-reference processed image;

[0073] A difference weight value corresponding to the non-reference processed image is determined according to the difference matrix, the similarity matrix and preset configuration parameters.

[0074] In a third aspect, the present application provides a chip having a computer program stored thereon, wherein when the computer program is executed by the chip, the image processing method as described in any one of the first aspects is implemented.

[0075] In a fourth aspect, the present application provides a chip module having a computer program stored thereon, and when the computer program is executed by the chip module, the image processing method as described in any one of the first aspects is implemented.

[0076] In a fifth aspect, an embodiment of the present application provides an electronic device, including: a memory, a processor, and a transceiver;

[0077] The memory stores computer-executable instructions;

[0078] The processor executes the computer-executable instructions stored in the memory to implement the image processing method as described in any one of the first aspects.

[0079] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the image processing method described in any one of the first aspects.

[0080] In a seventh aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the image processing method described in any one of the first aspects.

[0081] The embodiments of the present application provide an image processing method, apparatus, device, and storage medium. In this method, multiple continuous frame images are acquired, each corresponding to a different exposure state, including an overexposure state, an underexposure state, and a normal state. Mirroring and registration processing are performed on the multiple continuous frame images to obtain multiple original images. Multiple weight values ​​corresponding to the multiple original images are determined, including exposure weight values, brightness weight values, and saturation weight values. Based on the multiple weight values, the multiple original images are fused to obtain a target image. In this way, the implementation complexity is relatively low, and the algorithm has good robustness. It can effectively improve the dynamic range, contrast, and color fidelity of the image, avoid ghosting problems in the fused image, and improve the quality of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in this application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0083] Figure 1 A schematic diagram of an image processing process provided in an embodiment of the present application;

[0084] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0085] Figure 3 A flowchart of another image processing method provided in an embodiment of the present application;

[0086] Figure 4 A flowchart of another image processing method provided in an embodiment of the present application;

[0087] Figure 5 A schematic diagram of a segmented modulation curve provided in an embodiment of the present application;

[0088] Figure 6 A flowchart of another image processing method provided in an embodiment of the present application;

[0089] Figure 7 A flowchart of another image processing method provided in an embodiment of the present application;

[0090] Figure 8 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0091] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0092] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0093] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0094] It should be noted that although the terms "first" and "second" are used to describe various information in the embodiments of this application, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from each other. Alternatively, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information.

[0095] It should be understood that the terms "comprise" and "include" indicate the presence of the previously mentioned features, steps, or operations, but do not exclude the presence, occurrence, or addition of one or at least one other feature, step, or operation. The terms "and / or" and the like used in this application may be interpreted as inclusive, or may mean any one or any combination. Alternatively, "A and / or B" means "any of the following: A; B; A and B." In addition, the character " / " in this document generally indicates that the preceding and following objects are in an "or" relationship.

[0096] In related technologies, multi-frame image fusion technology can comprehensively utilize useful information in multiple frame images, remove or reduce the impact of adverse factors such as noise, and generate a fused image with higher quality and richer information.

[0097] Multi-frame image fusion can be achieved by first correctly matching the image frames in spatial position to align them, then performing information fusion processing on the multiple frames. The aligned image pixels can be weighted averaged. After determining the weight coefficient for each frame, the corresponding pixels of each frame are added together according to the weight to obtain the fused pixel value. Alternatively, pixels can be selected for fusion based on characteristics such as the contrast of the surrounding area. Alternatively, the image can be decomposed into low-frequency and high-frequency sub-bands. The low-frequency portion can be weighted fused, and the high-frequency portion can be selectively fused based on the size of the wavelet coefficients. The fused image can then be obtained through an inverse transform.

[0098] In the above fusion process, fusion algorithms with lower complexity may have image problems such as ghosting, low dynamic range, color loss or inaccuracy, resulting in low quality of the fused image.

[0099] To address the aforementioned technical issues, the present invention provides an image processing method that determines exposure, brightness, and saturation weights corresponding to multiple original images and fuses them according to these weights to produce a target image. This method achieves relatively low implementation complexity and robustness, effectively improving the image's dynamic range, contrast, and color fidelity, while avoiding ghosting in the fused image and ultimately enhancing the quality of the fused image.

[0100] Next, combine Figure 1 , the image processing process is illustrated with examples.

[0101] Figure 1 This is a flow chart of an image processing process provided by an embodiment of the present application. Figure 1 , Figure 1 It may include an input module 101 , a mirror edge extension module 102 , a registration module 103 , a fusion weight calculation module 104 and an output module 105 .

[0102] The input module 101 can be used to obtain multiple frame images. For example, the input module obtains three frames of raw images with different exposure times from short to long.

[0103] The mirror edge extension module 102 can be used to pre-process multiple frame images to solve the problem of insufficient information of image edge pixels in subsequent processing.

[0104] The registration module 103 may be configured to select a reference image from multiple frame images and perform registration and alignment using the reference image.

[0105] The fusion weight calculation module 104 may be configured to estimate the fusion weight according to local statistical information of the current image block to obtain a fused image.

[0106] The output module 105 may be configured to output a fused image.

[0107] The technical solutions shown in this application are described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be repeated in different embodiments.

[0108] Figure 2 This is a flow chart of an image processing method provided by an embodiment of the present application. The execution subject of the embodiment of the present application may be a processor. Figure 2 , the method comprising:

[0109] S201: Acquire multiple consecutive frame images.

[0110] Multiple consecutive frame images correspond to different exposure states, including overexposure, underexposure, and normal exposure.

[0111] Multiple consecutive frames can be taken with a preset interval between them. Since the interval is short, the content captured between the consecutive frames is minimally different, with only the exposure changing.

[0112] The exposure status can be determined by adjusting the exposure parameters of the camera during shooting.

[0113] Optionally, a plurality of continuous frame images may be acquired in a preset storage space.

[0114] Optionally, the information to be fused sent by the user may be received, and the information to be fused may be parsed and processed to obtain a plurality of continuous frame images.

[0115] It should be noted that multiple continuous frame images can be obtained according to any feasible implementation method, and the embodiments of the present application are not limited to this.

[0116] S202 , performing mirroring and registration processing on a plurality of continuous frame images to obtain a plurality of original images.

[0117] Mirror edge processing can be used to provide additional information for edge pixels, so that the fusion algorithm can better process the edge part.

[0118] Registration processing can be used to align the relative displacements between multiple frames of images so that multiple consecutive frame images match each other.

[0119] Image preprocessing can be performed on multiple continuous frame images to obtain multiple preprocessed continuous frame images, mirror edge extension processing can be performed on the multiple preprocessed continuous frame images to obtain multiple mirror edge extension continuous frame images, and registration processing can be performed on the multiple mirror edge extension continuous frame images to obtain multiple original images.

[0120] The image preprocessing may include image denoising, image color correction, image bad pixel removal, invalid frame removal, image segmentation and other operations, which are not limited here.

[0121] The registration process can select one of the image frames as the reference frame. You can choose an image with the same background and normal exposure as the reference frame, or select the best quality frame based on the image quality as the reference frame. Using the reference frame as the basis, the remaining frames are registered and aligned. Considering that some scenes may have alignment deviations, especially in motion scenes, there may be large alignment deviations. If a simple fusion algorithm is used, the fusion result may have strong ghosting.

[0122] S203: Determine a plurality of weight values ​​corresponding to the plurality of original images.

[0123] The weight values ​​include exposure weight value, brightness weight value and saturation weight value.

[0124] Multiple weighting algorithms can be obtained, and multiple weighting values ​​corresponding to multiple original images can be determined according to the multiple weighting algorithms.

[0125] Among them, the multiple weight algorithms may include an exposure weight calculation method, a brightness weight calculation method, and a saturation weight calculation method.

[0126] The exposure weight calculation method can be used to dynamically allocate the impact of frame images with different exposure states on the exposure of the fused image through factors such as bit width and exposure time.

[0127] The brightness weight calculation method can be used to obtain the brightness values ​​of the center points of multiple image blocks corresponding to the original image through weighted averaging of the four color channels, and calculate the brightness weight based on the brightness values.

[0128] Saturation weighting can be used to balance color saturation across frames. When fusing images with different exposures, the color saturation of each frame may vary. By calculating saturation weighting, we can adjust the color contribution of the fused frames based on the saturation information of each point in the image.

[0129] Optionally, multiple weight values ​​corresponding to multiple original images can be determined in the following ways: determining exposure weight values ​​corresponding to multiple original images; determining brightness weight values ​​corresponding to multiple original images; and determining saturation weight values ​​corresponding to multiple original images.

[0130] S204: performing fusion processing on the multiple original images according to the multiple weight values ​​to obtain a target image.

[0131] The target image can be used to represent the fused image.

[0132] A fusion algorithm can be obtained, and multiple original images can be fused according to multiple weight values ​​and the fusion algorithm to obtain a fused image. The fused image can then be tone mapped and saturated to obtain a target image.

[0133] The fusion algorithm can obtain the decomposed features through Gaussian pyramid decomposition, and reconstruct the decomposed features through pyramid-like reconstruction to fuse multiple original images.

[0134] Optionally, the fused image can be tone mapped and saturation adjusted in the following manner to obtain a target image: determine the brightness information of the fused image and perform statistical histogram, calculate the minimum brightness value, maximum brightness value and average brightness value of the fused image based on the statistical histogram, calculate the brightness enhancement intensity value based on the minimum brightness value, maximum brightness value and average brightness value, calculate the corresponding mapping curve control point coordinates based on the brightness enhancement intensity value, calculate the Bezier mapping curve based on the mapping curve control point coordinates, perform tone mapping on the fused image based on the Bezier mapping curve to obtain a first image, and adjust the saturation of the first image to obtain the target image.

[0135] Among them, the mapping curve control point coordinates can include four control point coordinates. The first point and the last point among the four control point coordinates use (Ymin, Ymin) and (Ymax, Ymax) respectively, and the coordinates of the remaining two points will be adjusted by setting corresponding control parameters. Ymin can represent the minimum brightness value, and Ymax can represent the maximum brightness value.

[0136] Optionally, the brightness enhancement intensity value can be calculated according to the minimum brightness value, the maximum brightness value, and the average brightness value through the following formula:

[0137] strength=(Ymax - Yavg) / (Ymax - Ymin)

[0138] Among them, strength can be the brightness enhancement intensity value, Ymax can represent the maximum brightness value, Yavg can represent the average brightness value, and Ymin can represent the minimum brightness value.

[0139] Optionally, Ymin can be obtained according to the user-configured minimum brightness ratio parameter. Starting from the leftmost interval of the statistical histogram in terms of the number of pixels, accumulate from left to right. When the accumulated value reaches the number of pixels with the minimum brightness, the corresponding value is the Ymin value.

[0140] Ymax can be obtained according to the user-configured maximum brightness ratio parameter. Starting from the rightmost interval of the statistical histogram in terms of the number of pixels, accumulate from right to left. When the accumulated value reaches the number of pixels with the maximum brightness, the corresponding value is the Ymax value.

[0141] Yavg is the average brightness value obtained by traversing the pixel brightness calculation, accumulating, and then dividing by the number of pixels in the entire image.

[0142] Optionally, the Bezier mapping curve can be calculated according to the mapping curve control point coordinates through the following formula:

[0143] B(t)=p0(bitmax - t) 3 +3p1t(bitmax - t) 2 +3p2t 2 (1 - t)+p3t 3

[0144] Among them, B(t) can represent the Bezier mapping curve, P0, P1, P2, and P3 can represent the four control point coordinates of the mapping curve, bitmax is the maximum bit width of the fused image, t can represent the time parameter or the interpolation parameter, 0 < t < bitmax, and the shape of the Bezier mapping curve can be controlled by changing the coordinate positions of points P1 and P2, and thus the dynamic range change during the mapping process can be controlled.

[0145] Optionally, the saturation of the first image can be adjusted using the following formula to obtain a target image:

[0146] RAW OUT =RAW1+(RAW0-Ymax)*ratio

[0147] Among them, RAW OUT It can represent the target image, RAW0 can represent the fused image, RAW1 can represent the first image, Ymax can represent the maximum brightness value, ratio can represent the intensity of saturation enhancement or reduction, a ratio greater than 0 means enhancing the saturation, and a ratio less than 0 means reducing the saturation.

[0148] The image processing method provided in this embodiment obtains multiple consecutive frame images, each corresponding to different exposure states, including overexposure, underexposure, and normal exposure. The multiple consecutive frame images are mirrored and registered to obtain multiple original images. Multiple weight values ​​corresponding to the multiple original images are determined, including exposure weight, brightness weight, and saturation weight. Based on the multiple weight values, the multiple original images are fused to obtain a target image. This method achieves relatively low implementation complexity and high algorithm robustness, effectively improving the image's dynamic range, contrast, and color fidelity, while avoiding ghosting in the fused image and enhancing the quality of the fused image.

[0149] Next, combine Figure 3 , the process of determining the exposure weight values ​​corresponding to multiple original images is explained.

[0150] Figure 3 This is a flow chart of another image processing method provided in the embodiment of the present application. Based on the above embodiment, please refer to Figure 3 , the method comprising:

[0151] S301: Determine exposure durations and analog gains corresponding to a plurality of original images.

[0152] The exposure duration can be used to indicate the exposure duration when capturing the original image.

[0153] Analog gain can be used to represent the amplification factor of the electrical signal generated by the accumulated photons.

[0154] The metadata corresponding to the multiple original images can be obtained, and the exposure durations and analog gains corresponding to the multiple original images can be determined according to the metadata.

[0155] S302: Obtain a preset exposure ratio.

[0156] The preset exposure ratio may be a preset maximum exposure ratio supported by the current algorithm.

[0157] Algorithm parameters can be obtained, and a preset exposure ratio can be determined based on the algorithm parameters.

[0158] S303: Determine a target gain and multiple exposure ratios according to multiple exposure durations, multiple analog gains, and preset exposure ratios.

[0159] The exposure ratio may be used to represent a ratio between exposure values ​​corresponding to two of the multiple original images. The exposure value is the product of the exposure duration corresponding to the original image and the corresponding analog gain.

[0160] The target gain can be used to make the synthesis result reach the target bit width.

[0161] A first product of the exposure time and the analog gain corresponding to the original image in an overexposed state, a second product of the exposure time and the analog gain corresponding to the original image in an underexposed state, and a third product of the exposure time and the analog gain corresponding to the original image in a normal state can be determined. A first exposure ratio is determined based on the first product and the second product, a second exposure ratio is determined based on the third product and the second product, and a target gain is determined based on the preset exposure ratio and the first exposure ratio.

[0162] Optionally, the first exposure ratio may be determined by the following formula:

[0163]

[0164] Among them, ratio l_s It can express the exposure ratio between the original image in the overexposed state and the original image in the underexposed state, gain l It can represent the analog gain corresponding to the original image in the overexposed state, expo l It can indicate the exposure time of the original image with overexposure status, gain s It can represent the analog gain corresponding to the original image in the underexposed state, expo s It can indicate the exposure time of the original image that is underexposed.

[0165] Optionally, the second exposure ratio may be determined by the following formula:

[0166]

[0167] Among them, ratio m_s It can represent the exposure ratio between the original image in normal state and the original image in underexposure state, gain mIt can represent the analog gain corresponding to the original image with normal exposure state, expo m It can represent the exposure time corresponding to the original image with normal exposure state, gain s It can represent the analog gain corresponding to the original image in the underexposed state, expo s It can indicate the exposure time of the original image that is underexposed.

[0168] Alternatively, the target gain may be determined as follows:

[0169]

[0170] Among them, gain can represent the target gain, EXP max Can indicate preset exposure ratio, ratio l_s It can indicate the exposure ratio between an overexposed original image and an underexposed original image.

[0171] S304: Determine exposure weight values ​​corresponding to the plurality of original images according to the preset exposure ratio, the target gain, and the plurality of exposure ratios.

[0172] The exposure weight value can be used to balance the brightness of images with different exposure levels.

[0173] The exposure weight value corresponding to the original image in the underexposed state can be determined according to the preset exposure ratio, the exposure weight value corresponding to the original image in the normal state can be determined according to the target gain and the second exposure ratio, and the exposure weight value corresponding to the original image in the overexposed state can be determined according to the target gain and the first exposure ratio.

[0174] Optionally, the exposure weight values ​​corresponding to the multiple original images can be determined in the following manner:

[0175] w exp0 =EXP max

[0176] w expref =gain-ratio m_s

[0177] w exp1 =gain-ratio l_s

[0178] Among them, w exp0 It can be used to represent the exposure weight value corresponding to the original image in the underexposed state, w expref It can be used to represent the exposure weight value corresponding to the original image in normal state, w exp1It can be used to represent the exposure weight value corresponding to the original image in the overexposed state, gain can represent the target gain, EXP max Can indicate preset exposure ratio, ratio l_s It can represent the exposure ratio between the original image in the overexposed state and the original image in the underexposed state. m_s It can represent the exposure ratio between the original image in a normal state and the original image in an underexposed state.

[0179] The implementation content of each step in the embodiment of the present application can refer to the description of the corresponding steps or operations in the above method embodiment, and repeated content will not be repeated.

[0180] The image processing method provided in this embodiment determines the exposure durations and analog gains corresponding to multiple original images to obtain a preset exposure ratio. Based on the multiple exposure durations, multiple analog gains, and the preset exposure ratios, a target gain and multiple exposure ratios are determined. The exposure ratio represents the ratio between the exposure values ​​corresponding to two of the multiple original images. The exposure value is the product of the exposure duration and the corresponding analog gain of the original image. Based on the preset exposure ratio, the target gain, and the multiple exposure ratios, exposure weights corresponding to the multiple original images are determined. This method achieves relatively low implementation complexity and high algorithm robustness, thereby improving the quality of the fused image.

[0181] Next, combine Figure 4 , a process of determining brightness weight values ​​corresponding to multiple original images is explained, where the multiple original images include a reference image and at least one non-reference image.

[0182] Figure 4 This is a flow chart of another image processing method provided in the embodiment of the present application. Based on the above embodiment, please refer to Figure 4 , the method comprising:

[0183] S401: Determine average brightness values ​​corresponding to multiple color channels in a reference image.

[0184] The number of blocks corresponding to the reference image may be determined, and based on the number of blocks, the average brightness values ​​corresponding to the plurality of color channels in the reference image may be determined.

[0185] Optionally, when the reference image is a 3×3 image block, the brightness average values ​​corresponding to multiple color channels in the reference image may be determined in the following manner.

[0186]

[0187] Among them, v 0 It can represent the average brightness corresponding to the first color channel, v 1It can represent the average brightness value corresponding to the second color channel, v 2 It can represent the average brightness corresponding to the third color channel, v 3 It can represent the average brightness value corresponding to the fourth color channel. It can represent the brightness value corresponding to the center block in the 3×3 image block in the first color channel, It can represent the brightness value corresponding to the upper left corner image block in the 3×3 image block in the second color channel, It can represent the brightness value corresponding to the lower left corner image block in the 3×3 image block in the second color channel, It can represent the brightness value of the upper right corner image block in the 3×3 image block in the second color channel. It can represent the brightness value of the lower right corner image block in the 3×3 image block in the second color channel. It can represent the brightness value of the image block above the center block in the 3×3 image block in the third color channel. It can represent the brightness value of the image block below the center block in the 3×3 image block in the third color channel. It can represent the brightness value corresponding to the image block to the left of the central block in the 3×3 image blocks in the fourth color channel, and can represent the brightness value corresponding to the image block to the right of the central block in the 3×3 image blocks in the fourth color channel.

[0188] S402: Determine a maximum brightness value among a plurality of brightness average values ​​and a target average value of the sum of the plurality of brightness average values.

[0189] The maximum brightness value is the largest value among multiple brightness average values.

[0190] The target average value is the average of multiple brightness average values.

[0191] S403 : Determine a brightness weight value corresponding to the reference image according to the exposure state, the maximum brightness value, and the target average value of the reference image.

[0192] The overexposure weight and underexposure weight can be determined according to the exposure status, maximum brightness, and target average value of the reference image, and the brightness weight value corresponding to the reference image can be determined according to the overexposure weight and underexposure weight.

[0193] Optionally, the overexposure weight can be determined as follows:

[0194]

[0195] Among them, maskP can represent the overexposure weight, Can represent the target average value, th max Can represent the brightness threshold, v max Can indicate the maximum brightness.

[0196] Optionally, the underexposure weight can be determined as follows:

[0197]

[0198] Among them, maskN can represent the underexposure weight, Can represent the target average value, th max Can represent brightness threshold.

[0199] Optionally, the brightness weight value corresponding to the reference image may be determined according to the overexposure weight and the underexposure weight in the following manner:

[0200] w mask =exp(-(|mask|*sigma mask ))

[0201] mask=min(maskP,maskN)

[0202] Among them, w mask It can represent the brightness weight value corresponding to the reference image, mask can represent the weight value of the reference image, maskP can represent the overexposure weight, maskN can represent the underexposure weight, sigma mask Can represent a preset threshold.

[0203] S404: Determine a brightness weight value corresponding to at least one non-reference image according to the maximum brightness value.

[0204] A brightness weight value corresponding to at least one non-reference image may be determined according to the brightness maximum value and the brightness threshold.

[0205] Optionally, the brightness weight value corresponding to at least one non-reference image may be determined in the following manner:

[0206]

[0207] w mask =exp(-(|th max -mask|*sigma mask ))

[0208] Among them, v max Can represent the maximum brightness, th max Can represent the brightness threshold, sigma mask It can represent a preset threshold, and mask can represent the weight value of the non-reference image.

[0209] Optionally, the brightness weight value can also be controlled by a segmented modulation curve. The segmented modulation curve can be as follows: Figure 5As shown, Figure 5 A schematic diagram of a segmented modulation curve provided in the embodiment of the present application is shown in FIG. Figure 5 ,Through the segmented modulation curve, the brightness weight values ​​of,different exposure images are calculated.

[0210] The segmented modulation curve can include 4 preset points, x0, x1, x2 and x3. According to the 4 preset points, the brightness weight value curve corresponding to the original image in the underexposed state, the brightness weight value curve corresponding to the original image in the normal state, and the brightness weight value curve corresponding to the original image in the overexposed state can be determined.

[0211] The brightness value can be determined by the following formula:

[0212] luma=max(v 0 ,v 1 ,v 2 ,v 3 )

[0213] Among them, luma can represent the brightness value, v 0 It can represent the average brightness corresponding to the first color channel, v 1 It can represent the average brightness value corresponding to the second color channel, v 2 It can represent the average brightness corresponding to the third color channel, v 3 It can represent the average brightness value corresponding to the fourth color channel.

[0214] The corresponding brightness weight value may be determined according to the brightness values ​​of the different exposure images and the brightness weight value curves corresponding to the different exposure states.

[0215] The implementation content of each step in the embodiment of the present application can refer to the description of the corresponding steps or operations in the above method embodiment, and repeated content will not be repeated.

[0216] The image processing method provided in this embodiment determines the average brightness values ​​corresponding to multiple color channels in a reference image, the maximum brightness value among the multiple average brightness values, and a target average value for the sum of the multiple average brightness values. Based on the exposure status, maximum brightness value, and target average value of the reference image, a brightness weight value corresponding to the reference image is determined. Based on the maximum brightness value, a brightness weight value corresponding to at least one non-reference image is determined. This method achieves relatively low implementation complexity and high algorithm robustness, thereby improving the quality of the fused image.

[0217] Next, combine Figure 6 , the process of determining the saturation weight values ​​corresponding to multiple original images is explained.

[0218] Figure 6This is a flow chart of another image processing method provided in the embodiment of the present application. Based on the above embodiment, please refer to Figure 6 , taking any one of the multiple original images as an example, the method includes:

[0219] S601: Determine multiple center points corresponding to the original image.

[0220] The center point can be any pixel in the original image.

[0221] A plurality of pixel points of the original image are obtained, and a plurality of center points corresponding to the original image are determined based on the plurality of pixel points.

[0222] S602: For any center point, determine the red value, green value, and blue value corresponding to the center point.

[0223] The red value, green value, and blue value may be the RGB values ​​corresponding to the point.

[0224] The color type of the center point can be determined, and based on the color type, the red value, green value, and blue value corresponding to the center point can be determined.

[0225] For example, if the center point is of red type, the red, green, and blue values ​​of the center point are:

[0226] R i,j =RAW i,j

[0227]

[0228] Among them, R i,j Can represent red value, G i,j Can represent green value, B i,j Can represent blue value, RAW i,j Can represent the raw value corresponding to the center point, RAW i-1,j It can represent the raw value corresponding to the left side of the center point, RAW i+1,j It can represent the raw value corresponding to the right side of the center point, RAW i,j-1 It can represent the raw value corresponding to the upper side of the center point, RAW i,j+1 It can represent the raw value corresponding to the lower side of the center point, RAW i-1,j-1 It can represent the raw value corresponding to the upper left side of the center point, RAW i-1,j+1 It can represent the raw value corresponding to the lower left side of the center point, RAW i+1,j-1 It can represent the raw value corresponding to the upper right side of the center point, RAW i+1,j+1 It can represent the raw value corresponding to the lower right side of the center point.

[0229] S603: Determine the maximum color value and the minimum color value among the red value, the green value, and the blue value.

[0230] The maximum and minimum color values ​​can be determined by the following formula:

[0231] rgb_max=max(R i,j , G i,j , B i,j )

[0232] rgb_min=min(R i,j , G i,j , B i,j )

[0233] Among them, rgb_max can represent the maximum value of the color, rgb_min can represent the minimum value of the color, R i,j Can represent red value, G i,j Can represent green value, B i,j Can represent blue value.

[0234] S604: Determine a saturation weight value corresponding to the original image according to the maximum color value and the minimum color value.

[0235] A first average value of the color maximum value and the color minimum value may be determined, and a saturation weight value corresponding to the original image may be determined based on the color maximum value and the first average value.

[0236] Alternatively, the saturation weight value corresponding to the original image can be determined by the following formula:

[0237] W _satu =rgb max -(rgb max +rgb_min) / 2

[0238] Among them, W _satu It can represent the saturation weight value, rgb_max can represent the maximum color value, and rgb_min can represent the minimum color value.

[0239] The implementation content of each step in the embodiment of the present application can refer to the description of the corresponding steps or operations in the above method embodiment, and repeated content will not be repeated.

[0240] The image processing method provided in this embodiment determines multiple center points corresponding to the original image; for any center point, determines the red, green, and blue values ​​corresponding to the center point; determines the maximum and minimum color values ​​among the red, green, and blue values; and, based on the multiple maximum and minimum color values, determines the saturation weight value corresponding to the original image. This method has relatively low implementation complexity and good algorithm robustness, thereby improving the quality of the fused image.

[0241] Next, combine Figure 7, the process of determining the brightness weight values ​​corresponding to multiple original images is explained.

[0242] Figure 7 This is a flow chart of another image processing method provided in the embodiment of the present application. Based on the above embodiment, please refer to Figure 7 , the method comprising:

[0243] S701 , performing block processing on a plurality of original images to obtain a plurality of original image blocks.

[0244] For any original image, the block parameters can be obtained, and the original image is divided into blocks according to the block parameters to obtain multiple original image blocks.

[0245] S702 : Perform multiple downsampling processes on the multiple original image blocks to obtain multiple target sampled image blocks.

[0246] The downsampling parameters can be obtained, and for any original image block, the original image block can be downsampled multiple times according to the downsampling parameters to obtain multiple target sampled image blocks.

[0247] Among them, multiple downsampling processes can include three downsampling processes, the first downsampling process can downsample the original image block to a target sampling image block of 13×13 channels, the second downsampling process can downsample the target sampling image block of 13×13 channels to a target sampling image block of 9×9 channels, and the third downsampling process can downsample the target sampling image block of 9×9 channels to a target sampling image block of 5×5 channels.

[0248] S703: Determine multiple target difference weight values ​​according to the multiple target sampling image blocks.

[0249] The target difference weight value can be used to represent the degree of difference between different original images.

[0250] The exposure ratios of the plurality of target sampled image blocks may be adjusted to a target value, and then the block similarity of any target sampled image block may be determined, and the target difference weight value may be determined based on the block similarity.

[0251] Optionally, multiple target difference weight values ​​can be determined based on multiple target sampling image blocks in the following manner: brightness alignment processing is performed on the multiple target sampling images to obtain multiple processed images, and the multiple processed images include a reference processed image and at least one non-reference processed image; for any non-reference processed image, a difference matrix and a similarity matrix are determined based on the reference processed image and the non-reference processed image; based on the difference matrix, the similarity matrix and preset configuration parameters, the difference weight value corresponding to the non-reference processed image is determined.

[0252] Optionally, for any pixel point in the target sample image block, the target difference weight value corresponding to the pixel point can be determined in the following manner:

[0253]

[0254] Among them, d blk It can express the point-by-point similarity, d pix is the point-by-point difference matrix between the target sampled image block and the reference sampled image block. The reference sampled image block can be the sampled image block corresponding to the reference image. sigma mask It can represent a preset threshold, and th0 can represent a first threshold.

[0255] Alternatively, the point-by-point similarity d can be determined as follows: blk :

[0256] d blk =max{diff0,diff1,diff2,diff3}

[0257] diff0=(I 22 *4+(I 02 +I 20 +I 24 +I 42 )*2+(I 00 +I 04 +I 40 +I 44 )) / 16

[0258] diff1=((I 21 +I 23 )*2+(I 01 +I 03 +I 41 +I 43 )) / 8

[0259] diff2=((I 12 +I 32 )*2+(I 10 +I 30 +I 14 +I 34 )) / 8

[0260] diff3=(I 11 +I 13 +I 31 +I 33 ) / 4

[0261] Wherein, I can represent the difference between the reference sample image block and the target sample image block, and the subscript number represents the coordinates of the current point within the 5*5 range with the current point as the center point.

[0262] S704 , performing multiple fusion processes on the multiple target sample image blocks according to the multiple target difference weight values ​​and the multiple weight values ​​to obtain multiple fused image blocks.

[0263] A fusion formula may be obtained, and according to the fusion formula, multiple target difference weight values, and multiple weight values, multiple target sampling image blocks are fused multiple times to obtain multiple fused image blocks.

[0264] The multiple fusion processes may include a first fusion process, a second fusion process, and a third fusion process.

[0265] Optionally, the first fusion process can be performed in the following manner:

[0266]

[0267] Among them, the subscripts ref, 0, and 1 are the image blocks with normal exposure, underexposure, and overexposure, respectively. v can represent the brightness weight value, mask can represent the weight value of the reference image or the weight value of the non-reference image, and w exp Can represent the exposure weight value, w _satu Can represent the saturation weight value, w diff0 is the target difference weight value between the image block with normal exposure and the image block with underexposure, w diff1 It is the target difference weight value between the image block with normal exposure state and the image block with overexposure state.

[0268] The second and third fusion processes are similar to the first fusion process, but the difference weight value of this fusion process will be updated according to the difference weight value of the previous fusion process, and the fusion process will be determined according to the Laplace coefficient.

[0269] Optionally, the Laplace coefficient lap corresponding to the second fusion process level1 :

[0270] lap level1 =gauss level1,44 -gauss level2,22

[0271] Among them, gauss level1,44 It can represent the Gaussian coefficient of the center point of the image after the second downsampling process, gauss level2,22 It can represent the Gaussian coefficient of the center point of the image after the third downsampling process.

[0272] Optionally, the Laplace coefficient lap corresponding to the third fusion processlevel0 :

[0273] lap level0 =gauss level0,66 -gauss level1,44

[0274] Among them, gauss level0,66 It can represent the Gaussian coefficient of the center point of the image after the first downsampling process, gauss level1,44 It can represent the Gaussian coefficient of the center point of the image after the second downsampling process.

[0275] S705: Perform stitching processing on the multiple fused image blocks to obtain a target image.

[0276] The positions corresponding to the multiple fused image blocks are determined, and the multiple fused image blocks are spliced ​​according to the positions corresponding to the multiple fused image blocks to obtain a target image.

[0277] The implementation content of each step in the embodiment of the present application can refer to the description of the corresponding steps or operations in the above method embodiment, and repeated content will not be repeated.

[0278] The image processing method provided in this embodiment divides multiple original images into blocks to obtain multiple original image blocks, performs multiple downsampling processes on these blocks to obtain multiple target sampled image blocks, determines multiple target difference weight values ​​based on these blocks, performs multiple fusion processes on these blocks based on the multiple target difference weight values ​​and the multiple weight values ​​to obtain multiple fused image blocks, and then splices these multiple fused image blocks to obtain the target image. This method achieves relatively low implementation complexity, high algorithm robustness, and improved fused image quality.

[0279] Figure 8 This is a structural diagram of an image processing device provided in an embodiment of the present application. Figure 8 The image processing device 800 includes an acquisition module 801, a first processing module 802, a determination module 803 and a second processing module 804, wherein:

[0280] An acquisition module 801 is configured to acquire a plurality of continuous frame images, each of which corresponds to a different exposure state, including an overexposure state, an underexposure state, and a normal state.

[0281] The first processing module 802 is used to perform mirroring and registration processing on a plurality of consecutive frame images to obtain a plurality of original images;

[0282] A determination module 803 is configured to determine a plurality of weight values ​​corresponding to the plurality of original images, wherein the weight values ​​include an exposure weight value, a brightness weight value, and a saturation weight value;

[0283] The second processing module 804 is configured to perform fusion processing on the multiple original images according to the multiple weight values ​​to obtain a target image.

[0284] In a possible implementation, the determining module 803 is specifically configured to:

[0285] Determining exposure weight values ​​corresponding to the plurality of original images respectively;

[0286] Determining brightness weight values ​​corresponding to the plurality of original images respectively;

[0287] Determine saturation weight values ​​corresponding to the multiple original images respectively.

[0288] In a possible implementation, the determining module 803 is specifically configured to:

[0289] Determining exposure times and analog gains corresponding to the plurality of original images respectively;

[0290] Get the preset exposure ratio;

[0291] determining a target gain and a plurality of exposure ratios based on a plurality of exposure times, a plurality of analog gains, and the preset exposure ratio, wherein the exposure ratio represents a ratio between exposure values ​​corresponding to two of the plurality of original images, and the exposure value is a product of the exposure times corresponding to the original images and the corresponding analog gains;

[0292] Exposure weight values ​​corresponding to the plurality of original images are determined according to the preset exposure ratio, the target gain, and the plurality of exposure ratios.

[0293] In a possible implementation, the multiple original images include a reference image and at least one non-reference image, and the determination module 803 is specifically configured to:

[0294] Determine the average brightness values ​​corresponding to the plurality of color channels in the reference image;

[0295] determining a maximum brightness value among a plurality of brightness average values ​​and a target average value of the sum of the plurality of brightness average values;

[0296] Determining a brightness weight value corresponding to the reference image according to the exposure state of the reference image, the maximum brightness value, and the target average value;

[0297] Determine, according to the maximum brightness value, a brightness weight value corresponding to each of the at least one non-reference images.

[0298] In a possible implementation, for any original image, the determination module 803 is specifically configured to:

[0299] Determine a plurality of center points corresponding to the original image;

[0300] For any center point, determine the red value, green value, and blue value corresponding to the center point respectively;

[0301] determining a color maximum and a color minimum among the red value, the green value, and the blue value;

[0302] A saturation weight value corresponding to the original image is determined according to the multiple color maximum values ​​and the multiple color minimum values.

[0303] In a possible implementation, the second processing module 804 is specifically configured to:

[0304] Performing block processing on the multiple original images to obtain multiple original image blocks;

[0305] Performing multiple downsampling processes on the multiple original image blocks to obtain multiple target sampled image blocks;

[0306] Determining a plurality of target difference weight values ​​according to the plurality of target sampling image blocks;

[0307] performing multiple fusion processing on the multiple target sample image blocks according to the multiple target difference weight values ​​and the multiple weight values ​​to obtain multiple fused image blocks;

[0308] The multiple fused image blocks are stitched together to obtain a target image.

[0309] In a possible implementation, the second processing module 804 is specifically configured to:

[0310] Performing brightness alignment processing on the multiple target sample images to obtain the multiple processed images, wherein the multiple processed images include a reference processed image and at least one non-reference processed image;

[0311] For any non-reference processed image, determining a difference matrix and a similarity matrix according to the reference processed image and the non-reference processed image;

[0312] A difference weight value corresponding to the non-reference processed image is determined according to the difference matrix, the similarity matrix and preset configuration parameters.

[0313] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 9 , the electronic device 900 may include: a memory 901 , a processor 902 , and a transceiver 903 .

[0314] The memory 901 is used to store program instructions;

[0315] The processor 902 is configured to execute program instructions stored in the memory, so as to enable the electronic device 900 to perform any of the above-mentioned image processing methods.

[0316] The transceiver 903 may include a transmitter and / or a receiver. The transmitter may also be referred to as a transmitter, a transmitter, a transmission port, a transmission interface, or similar descriptions, and the receiver may also be referred to as a receiver, a reception port, a reception interface, or similar descriptions. For example, the memory 901, the processor 902, and the transceiver 903 are interconnected via a bus 904.

[0317] The present application also provides a chip having a computer program stored thereon. When the computer program is executed by the chip, the above-mentioned image processing method is implemented. The corresponding content and effects can be referred to the method embodiment part, which will not be described in detail.

[0318] The present application also provides a chip module, on which a computer program is stored. When the computer program is executed by the chip module, the above-mentioned image processing method is implemented. The corresponding content and effects can be referred to the method embodiment part, which will not be repeated here.

[0319] An embodiment of the present application further provides a computer program product, which can be executed by a processor. When the computer program product is executed, the above-mentioned image processing method can be implemented.

[0320] The image processing device, electronic device, computer-readable storage medium and computer program product of the embodiments of the present application can execute the technical solutions shown in the above-mentioned image processing method embodiments. Their implementation principles and beneficial effects are similar and will not be repeated here.

[0321] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-mentioned method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), random access memory (RAM), flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0322] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0323] These computer-executable instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0324] These computer-executable instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0325] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a plurality of continuous frame images, wherein the plurality of continuous frame images respectively correspond to different exposure states, wherein the exposure states include an overexposure state, an underexposure state, and a normal state; Perform mirror edge extension and registration processing on multiple continuous frame images to obtain multiple original images; Determining a plurality of weight values ​​corresponding to the plurality of original images respectively, the weight values ​​comprising an exposure weight value, a brightness weight value, and a saturation weight value; The multiple original images are fused according to the multiple weight values ​​to obtain a target image.

2. The method according to claim 1, characterized in that Determining a plurality of weight values ​​corresponding to the plurality of original images, respectively, includes: Determining exposure weight values ​​corresponding to the plurality of original images respectively; Determining brightness weight values ​​corresponding to the plurality of original images respectively; Determine saturation weight values ​​corresponding to the multiple original images respectively.

3. The method according to claim 2, characterized in that Determining exposure weight values ​​corresponding to the plurality of original images respectively includes: Determining exposure times and analog gains corresponding to the plurality of original images respectively; Get the preset exposure ratio; determining a target gain and a plurality of exposure ratios based on a plurality of exposure times, a plurality of analog gains, and the preset exposure ratio, wherein the exposure ratio represents a ratio between exposure values ​​corresponding to two of the plurality of original images, and the exposure value is a product of the exposure times corresponding to the original images and the corresponding analog gains; Exposure weight values ​​corresponding to the plurality of original images are determined according to the preset exposure ratio, the target gain, and the plurality of exposure ratios.

4. The method according to claim 2, characterized in that The plurality of original images include a reference image and at least one non-reference image, and determining brightness weight values ​​corresponding to the plurality of original images respectively includes: Determine the average brightness values ​​corresponding to the plurality of color channels in the reference image; determining a maximum brightness value among a plurality of brightness average values ​​and a target average value of the sum of the plurality of brightness average values; Determining a brightness weight value corresponding to the reference image according to the exposure state of the reference image, the maximum brightness value, and the target average value; Determine, according to the maximum brightness value, a brightness weight value corresponding to each of the at least one non-reference images.

5. The method according to claim 2, characterized in that For any original image, determining a saturation weight value corresponding to the original image includes: Determine a plurality of center points corresponding to the original image; For any center point, determine the red value, green value, and blue value corresponding to the center point respectively; determining a color maximum and a color minimum among the red value, the green value, and the blue value; A saturation weight value corresponding to the original image is determined according to the multiple color maximum values ​​and the multiple color minimum values.

6. The method according to any one of claims 1 to 5, characterized in that The plurality of original images are fused according to the plurality of weight values ​​to obtain a target image, comprising: Performing block processing on the multiple original images to obtain multiple original image blocks; Performing multiple downsampling processes on the multiple original image blocks to obtain multiple target sampled image blocks; Determining a plurality of target difference weight values ​​according to the plurality of target sampling image blocks; performing multiple fusion processing on the multiple target sample image blocks according to the multiple target difference weight values ​​and the multiple weight values ​​to obtain multiple fused image blocks; The multiple fused image blocks are stitched together to obtain a target image.

7. The method according to claim 6, characterized in that Determining a plurality of target difference weight values ​​according to the plurality of target sample image blocks includes: Performing brightness alignment processing on the multiple target sample images to obtain the multiple processed images, wherein the multiple processed images include a reference processed image and at least one non-reference processed image; For any non-reference processed image, determining a difference matrix and a similarity matrix according to the reference processed image and the non-reference processed image; A difference weight value corresponding to the non-reference processed image is determined according to the difference matrix, the similarity matrix and preset configuration parameters.

8. An image processing device, characterized in that: The device comprises: An acquisition module is used to acquire a plurality of continuous frame images, wherein the plurality of continuous frame images respectively correspond to different exposure states, wherein the exposure states include an overexposure state, an underexposure state, and a normal state; The first processing module is used to perform mirroring and registration processing on a plurality of continuous frame images to obtain a plurality of original images; a determination module, configured to determine a plurality of weight values ​​corresponding to the plurality of original images, the weight values ​​comprising an exposure weight value, a brightness weight value, and a saturation weight value; The second processing module is used to perform fusion processing on the multiple original images according to the multiple weight values ​​to obtain a target image.

9. An electronic device, characterized in that: include: memory, processors, and transceivers; wherein the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images