Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention relates to an image processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a plurality of region-of-interest images corresponding to a to-be-processed image; regions of interest in the plurality of regions of interest images are the same region in the to-be-processed image, and brightness values of pixel points in the to-be-processed image corresponding to pixel points in the plurality of regions of interest images are different; determining a reference image from the plurality of region-of-interest images based on the brightness value of each pixel point in the region-of-interest images; determining a target gain coefficient based on the average brightness value of the plurality of pixel points in the reference image; obtaining an interested target image based on the to-be-processed image and the target gain coefficient; and determining a target image corresponding to the to-be-processed image based on the to-be-processed image and the interested target image. By adopting the method, the accuracy of image processing can be improved.

Description

Technical Field

[0001] This application relates to image processing technology, and in particular, to an image processing method, apparatus, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of image processing technology, tone mapping technology has emerged. Tone mapping technology refers to the technology of adjusting and mapping high-dynamic-range images to the standard dynamic range, and is used to adjust the brightness values of high-dynamic-range images.

[0003] In traditional methods, the global tone mapping method is used to adjust the brightness values of high-dynamic-range images, resulting in low accuracy of image processing. Summary of the Invention

[0004] Embodiments of this application provide an image processing method, apparatus, electronic device, and computer-readable storage medium, which can improve the accuracy of image processing.

[0005] In a first aspect, this application provides an image processing method. The method includes:

[0006] Obtain multiple region-of-interest images corresponding to the image to be processed; the regions of interest in the multiple region-of-interest images are the same region in the image to be processed, and the brightness values of the pixel points in the image to be processed corresponding to the pixel points in the multiple region-of-interest images are different;

[0007] Based on the brightness values of the respective pixel points in the region-of-interest images, determine a reference image from the multiple region-of-interest images;

[0008] Based on the average brightness value of multiple pixel points in the reference image, determine a target gain coefficient;

[0009] Based on the image to be processed and the target gain coefficient, obtain a region-of-interest target image;

[0010] Based on the image to be processed and the region-of-interest target image, determine the target image corresponding to the image to be processed.

[0011] In a second aspect, this application also provides an image processing apparatus. The apparatus includes:

[0012] An obtaining module, configured to obtain multiple region-of-interest images corresponding to the image to be processed; the regions of interest in the multiple region-of-interest images are the same region in the image to be processed, and the brightness values of the pixel points in the image to be processed corresponding to the pixel points in the multiple region-of-interest images are different;

[0013] A selection module, configured to determine a reference image from multiple images of the region of interest based on the luminance values of each pixel point in the images of the region of interest;

[0014] A determination module, configured to determine a target gain coefficient based on the average luminance value of multiple pixel points in the reference image;

[0015] A processing module, configured to obtain an image of the target of interest based on the image to be processed and the target gain coefficient;

[0016] A fusion module, configured to determine the target image corresponding to the image to be processed based on the image to be processed and the image of the target of interest.

[0017] In a third aspect, the present application further provides an electronic device. The electronic device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method in the first aspect are implemented.

[0018] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.

[0019] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method in the first aspect are implemented.

[0020] In the above image processing method, by performing frame splitting on the region of interest in the image to be processed, multiple images of the region of interest with different luminance value ranges are obtained. According to the luminance values of each pixel point in the images of the region of interest, the image of the region of interest with a low overexposure ratio is selected as the reference image. Then, the target gain coefficient corresponding to the region of interest in the image to be processed is determined according to the luminance values of each pixel point in the reference image, that is, the target gain coefficient corresponding to the region of interest is determined according to the reference image with higher accuracy, improving the accuracy of the target gain coefficient. The image to be processed is processed using the target gain coefficient to obtain an image of the target of interest, and the image to be processed and the image of the target of interest processed as above are fused to obtain the target image corresponding to the image to be processed, thereby improving the accuracy of image processing. Description of the Drawings

[0021] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0022] Figure 1 It is a flowchart of an image processing method in an embodiment;

[0023] Figure 2 It is a schematic diagram of the relationship of brightness values in an embodiment;

[0024] Figure 3 It is a flowchart of the reference image determination step in an embodiment;

[0025] Figure 4 It is a flowchart of the target gain coefficient determination step in an embodiment;

[0026] Figure 5 It is a flowchart of the region of interest image determination step in an embodiment;

[0027] Figure 6 It is a flowchart of the target image determination step in an embodiment;

[0028] Figure 7 It is a schematic diagram of the weight curve in an embodiment;

[0029] Figure 8 It is a schematic diagram of the segmented threshold in an embodiment;

[0030] Figure 9 It is a flowchart of the weight curve determination step in an embodiment;

[0031] Figure 10 It is a schematic diagram of the weight curve in another embodiment;

[0032] Figure 11 It is a flowchart of the image processing in an embodiment;

[0033] Figure 12 It is a flowchart of the target frame sequence determination step in an embodiment;

[0034] Figure 13 It is a flowchart of the to-be-processed image determination step in an embodiment;

[0035] Figure 14 It is a flowchart of the to-be-processed image processing process in an embodiment;

[0036] Figure 15 It is a structural block diagram of an image processing apparatus in an embodiment;

[0037] Figure 16 It is a structural block diagram of an electronic device in an embodiment. Specific implementation manners

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0039] In one embodiment, as Figure 1 shown, an image processing method is provided. Taking the application of this method to an electronic device as an example for illustration, the electronic device can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, smart cars, etc., and the portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. It can be understood that this method can also be applied to a server, and can also be applied to a system including an electronic device and a server, and is implemented through the interaction between the electronic device and the server. In this embodiment, the method includes the following steps 102 to step 108, where:

[0040] Step 102, obtaining multiple region-of-interest images corresponding to the image to be processed; the regions of interest in the multiple region-of-interest images are the same region in the image to be processed, and the brightness values of the pixel points corresponding to the pixel points in the image to be processed in the multiple region-of-interest images are different.

[0041] Among them, the image to be processed refers to an image that needs to be processed. The image to be processed can be a high dynamic range (HDR, High Dynamic Range) image, that is, the brightness value range of the pixel points in the image to be processed is large, and at least one of overexposure and underexposure may exist in the image to be processed. The region-of-interest image refers to an image obtained by multiplying the region of interest in the image to be processed by a gain coefficient. It can be understood that the brightness values of the pixel points in the region of interest in the image to be processed are multiplied by the gain coefficient. For example, as Figure 2As shown, the abscissa is the input brightness value, which is the brightness value of the pixel points in the image to be processed, and the ordinate is the output brightness value, which is the brightness value of the corresponding pixel points of the pixel points in the image to be processed in the region of interest image. ev0, ev1, and ev2 are the curves corresponding to the three region of interest images respectively, and the slope represents the gain coefficient. L0, L1, and L2 are the maximum brightness values in the three region of interest images respectively, and are all obtained by multiplying the maximum brightness value in the image to be processed by different gain coefficients. The number of region of interest images can be determined according to the brightness value range of the pixel points in the image to be processed, or can be determined according to the brightness value range of the pixel points in the region of interest in the image to be processed. The larger the brightness value range is, the more the number of region of interest images is, that is, the number of region of interest images is proportional to the brightness value range.

[0042] The region of interest refers to a specific area or part that a user or an algorithm focuses on in an image or a visual scene. The region of interest depends on the specific task and application. For example, in face recognition, the face is the region of interest; in medical image analysis, the lesion area is the region of interest; in an autonomous driving system, the road and traffic signs are the region of interest. The region of interest can be manually marked or automatically detected and recognized through computer vision algorithms. A pixel point is the basic unit that constitutes an image, and it can be understood that the image to be processed is composed of multiple pixel points. The brightness value is a numerical value representing the brightness degree of a pixel point.

[0043] Exemplarily, the electronic device acquires the image to be processed, determines the region of interest of the image to be processed, and determines multiple region of interest images corresponding to the image to be processed based on the region of interest.

[0044] In one embodiment, before step 102, it further includes: the electronic device performs real-time image data acquisition through an image sensor to obtain an initial frame sequence; in the preview mode state, performs a preview processing flow on the initial frame sequence to obtain a reference frame sequence; in the shooting mode state, performs an optimization process on the reference frame sequence to obtain a target frame sequence; performs an alignment process and a restoration process on the multi-target frame sequence to obtain a high-quality frame sequence; and fuses the high-quality frame sequence to obtain the image to be processed. Among them, the optimization process includes but is not limited to BLC (Black Level Correction), BPC (BadPixel Correction), LSC (Lens Shading Correction), AWB (AutoWhite Balance), etc.; the restoration process includes but is not limited to denoising processing and demosaicing processing, etc.

[0045] Step 104: Determine a reference image from multiple region-of-interest images based on the luminance values of each pixel point in the region-of-interest images.

[0046] The reference image refers to one of the multiple region-of-interest images, and the luminance values of the pixel points in this region-of-interest image satisfy a preset condition. The preset condition can be at least one of the overexposure ratio being lower than the overexposure threshold value and the average luminance value being close to the reference average luminance value.

[0047] Exemplarily, for each region-of-interest image, the electronic device determines the overexposure ratio corresponding to the region-of-interest image based on the luminance values of each pixel point in the region-of-interest image, and determines the reference image among the multiple region-of-interest images based on the overexposure ratios corresponding to the multiple region-of-interest images.

[0048] Step 106: Determine a target gain coefficient based on the average luminance value of multiple pixel points in the reference image.

[0049] The average luminance value refers to the average of the luminance values of multiple pixel points in the reference image. The target gain coefficient refers to the multiple by which the luminance values of the pixel points in the region of interest in the image to be processed are magnified or reduced. If the target gain coefficient is greater than one, the luminance values of the pixel points in the region of interest are increased, and the region of interest becomes brighter. If the target gain coefficient is less than one, the luminance values of the pixel points in the region of interest are decreased, and the region of interest becomes darker.

[0050] Exemplarily, the electronic device obtains the luminance value of each pixel point in the reference image, averages the multiple luminance values to obtain the average luminance value corresponding to the reference image, and determines the target gain coefficient based on the average luminance value, the region-of-interest reference value, and the region-of-interest luminance average value.

[0051] Step 108: Obtain a region-of-interest target image based on the image to be processed and the target gain coefficient.

[0052] The region-of-interest target image refers to an image obtained by multiplying the luminance values of the pixel points in the region of interest in the image to be processed by the target gain coefficient.

[0053] Exemplarily, the electronic device obtains the region of interest in the image to be processed, multiplies the luminance value of each pixel point in the region of interest by the target gain coefficient to obtain the target gain luminance value of the pixel point, and obtains the region-of-interest target image based on the target gain luminance values of each pixel point.

[0054] Step 110: Determine the target image corresponding to the image to be processed based on the image to be processed and the region-of-interest target image.

[0055] Exemplarily, the electronic device fuses the image to be processed and the target image of interest to obtain the target image corresponding to the image to be processed.

[0056] In one embodiment, the region of interest is an overexposed region or an underexposed region. The electronic device fuses the region of interest in the target image of interest and the image to be processed to obtain the target image corresponding to the image to be processed.

[0057] In this embodiment, by performing frame splitting on the region of interest in the image to be processed, multiple region-of-interest images with different brightness value ranges are obtained. According to the brightness values of each pixel point in the region-of-interest images, the region-of-interest image with a low overexposure ratio is selected as the reference image. Then, according to the brightness values of each pixel point in the reference image, the target gain coefficient corresponding to the region of interest in the image to be processed is determined, that is, the target gain coefficient corresponding to the region of interest is determined according to the reference image with higher accuracy, improving the accuracy of the target gain coefficient. The image to be processed is processed using the target gain coefficient to obtain the target image of interest, and the image to be processed and the target image of interest are fused to obtain the target image corresponding to the image to be processed, thereby improving the accuracy of image processing.

[0058] In one embodiment, as Figure 3 shown, based on the brightness values of each pixel point in the region-of-interest image, determining the reference image from multiple region-of-interest images includes:

[0059] Step 302, for each region-of-interest image, count the number of pixel points in the region-of-interest image whose brightness values exceed the brightness threshold value to obtain the corresponding statistical quantity of the region-of-interest image.

[0060] Among them, the brightness threshold value is the minimum brightness value of pre-set overexposure. For example, the brightness threshold value is 255. The statistical quantity refers to the number of pixel points in the region-of-interest image whose brightness values exceed the brightness threshold value.

[0061] Exemplarily, the electronic device respectively counts the number of pixel points in each region-of-interest image whose brightness values exceed the brightness threshold value to obtain the corresponding statistical quantity of the region-of-interest image.

[0062] Step 304, count the number of pixel points in the region-of-interest image to obtain the total quantity.

[0063] Among them, the total quantity refers to the total number of pixel points in the region-of-interest image.

[0064] Exemplarily, the electronic device counts the number of pixel points in the region-of-interest image to obtain the total quantity.

[0065] Step 306: Determine the overexposure ratio corresponding to the region of interest (ROI) image based on the statistical quantity and the total quantity.

[0066] The overexposure ratio refers to the proportion of the number of pixel points with luminance values exceeding the luminance threshold value among all pixel points in the ROI image.

[0067] Exemplarily, the electronic device divides the statistical quantity by the total quantity to obtain the overexposure ratio corresponding to the ROI image.

[0068] Step 308: Determine the reference image among multiple ROI images based on the overexposure ratio corresponding to each ROI image.

[0069] Exemplarily, the electronic device compares the overexposure ratios corresponding to multiple ROI images, determines the minimum overexposure ratio, and determines the ROI image corresponding to the minimum overexposure ratio as the reference image.

[0070] In this embodiment, the overexposure ratio corresponding to the ROI image is determined through the statistical quantity and the total quantity. The overexposure ratio reflects the number of pixel points with luminance values exceeding the luminance threshold value in the ROI image. The lower the overexposure ratio, the higher the accuracy of the ROI image. The reference image among multiple ROI images is selected according to the overexposure ratios corresponding to multiple ROI images, thereby improving the accuracy of the reference image.

[0071] In one embodiment, determining the reference image among multiple ROI images based on the overexposure ratio corresponding to each ROI image includes:

[0072] Determine the candidate images as the ROI images with overexposure ratios less than the overexposure threshold value; for each candidate image, average the luminance values of multiple pixel points in the candidate image to obtain the candidate luminance average value corresponding to the candidate image; determine the candidate image corresponding to the candidate luminance average value with the smallest difference from the reference luminance average value as the reference image.

[0073] Among them, the candidate image refers to the ROI image with an overexposure ratio less than the overexposure threshold value. The overexposure threshold value refers to the maximum overexposure ratio corresponding to the candidate image set in advance. For example, the overexposure threshold value is 5%. The candidate luminance average value refers to the average value of the luminance values corresponding to the pixel points in the candidate image. The reference luminance average value refers to the reference luminance value of the ROI set in advance. The reference luminance average value can correspond to the ROI one by one, that is, different ROI correspond to different reference luminance average values. For example, when the ROI is the sky region, the reference luminance average value is 255.

[0074] Exemplarily, the electronic device compares the overexposure ratio corresponding to each region of interest (ROI) image with the overexposure threshold value respectively. If the overexposure ratio is less than the overexposure threshold value, the ROI image is determined as a candidate image, and at least one candidate image is obtained. For each candidate image, the luminance values of multiple pixel points in the candidate image are averaged to obtain the candidate average luminance value corresponding to the candidate image, the reference average luminance value corresponding to the ROI is obtained, the difference value between the candidate average luminance value and the reference average luminance value is calculated, the multiple difference values are compared, the smallest difference value is obtained, and the candidate image corresponding to the smallest difference value is determined as the reference image.

[0075] In this embodiment, candidate images with an overexposure ratio less than the overexposure threshold value are screened out from the ROI images, and then reference images with candidate average luminance values close to the reference average luminance value are screened out from the candidate images. The reference images not only have a low overexposure ratio but also are close to the true luminance value, thereby improving the accuracy of the reference images.

[0076] In one embodiment, as Figure 4 shown, based on the average luminance values of multiple pixel points in the reference image, a target gain coefficient is determined, including:

[0077] Step 402, average the luminance values of multiple pixel points in the reference image to obtain the average luminance value corresponding to the reference image.

[0078] Exemplarily, the electronic device obtains the luminance value of each pixel point in the reference image, and averages the obtained multiple luminance values to obtain the average luminance value corresponding to the reference image.

[0079] Step 404, weight the average luminance value and the ROI reference value to obtain the ROI target value.

[0080] Among them, the ROI reference value refers to the reference value of the preset ROI, and the ROI reference value corresponds to the ROI one by one, that is, different ROIs correspond to different ROI reference values. Weighting refers to the operation process of multiplying different elements by the corresponding weight values and then adding the results.

[0081] Exemplarily, the electronic device obtains the first weight value corresponding to the average luminance value, calculates the second weight value corresponding to the ROI reference value based on the first weight value, multiplies the average luminance value by the first weight value to obtain the first result, multiplies the ROI reference value by the second weight value to obtain the second result, and adds the first result and the second result to obtain the ROI target value.

[0082] In one embodiment, the ROI target value L_tar is:

[0083] $L_{tar} = w\times L+(1 - w)\times L_{ref}$ Equation (1)

[0084] Wherein, $L$ is the average brightness value corresponding to the reference image, $L_{ref}$ is the reference value of the region of interest, $w$ is the weight value corresponding to the average brightness value, and $1 - w$ is the weight value corresponding to the reference value of the region of interest.

[0085] Step 406: Average the brightness values of multiple pixels in the region of interest in the image to be processed to obtain the average brightness value of the region of interest.

[0086] Exemplarily, the electronic device obtains the brightness value of each pixel in the region of interest in the image to be processed, and averages the obtained multiple brightness values to obtain the average brightness value of the region of interest.

[0087] Step 408: Obtain the target gain coefficient based on the ratio between the target value of the region of interest and the average brightness value of the region of interest.

[0088] Exemplarily, the electronic device divides the target value of the region of interest by the average brightness value of the region of interest to obtain the target gain coefficient.

[0089] In one embodiment, the target gain coefficient $gainSkyTar$ is:

[0090] $gainSkyTar = L_{tar} / L_{in}$ Equation (2)

[0091] Wherein, $L_{tar}$ is the target value of the region of interest, and $L_{in}$ is the average brightness value of the region of interest.

[0092] In this embodiment, by weighting the average brightness value and the reference value of the region of interest, the target value of the region of interest is obtained. It can be understood that the reference value of the region of interest is used to correct the average brightness value, and the obtained target value of the region of interest can more accurately reflect the brightness situation of the region of interest after gain, that is, the target value of the region of interest can more accurately reflect the brightness situation of the reference image. Then, the target gain coefficient is determined according to the ratio between the target value of the region of interest and the average brightness value of the region of interest, thereby improving the accuracy of the target gain coefficient.

[0093] In one embodiment, as Figure 5 shown, obtain multiple region-of-interest images corresponding to the image to be processed, including:

[0094] Step 502: Obtain the scene exposure ratio corresponding to the image to be processed.

[0095] Among them, the scene exposure ratio refers to the exposure ratio corresponding to the image to be processed. The exposure ratio represents the relative brightness relationship between two different exposure levels. For example, if the image to be processed is obtained by fusing three frames of images, and each frame of image corresponds to an average brightness, the scene exposure ratio of the image to be processed can be equal to the ratio between the maximum average brightness and the minimum average brightness.

[0096] Exemplarily, the electronic device obtains the scene exposure ratio corresponding to the image to be processed.

[0097] In one embodiment, the electronic device obtains multiple frames of initial images corresponding to the image to be processed, calculates the initial brightness mean value corresponding to each frame of the initial image respectively, compares the initial brightness mean values corresponding to the multiple frames of initial images, obtains the maximum initial brightness mean value and the minimum initial brightness mean value, and determines the scene exposure ratio corresponding to the image to be processed based on the maximum initial brightness mean value and the minimum initial brightness mean value. Among them, the initial image refers to the image that composes the image to be processed.

[0098] Step 504: Determine the number of frame splits corresponding to the image to be processed based on the brightness value range of the image to be processed.

[0099] Among them, the brightness value range refers to the range composed of the minimum brightness value and the maximum brightness value in the image to be processed. The number of frame splits refers to the number of images of the region of interest.

[0100] Exemplarily, the electronic device obtains the brightness value range of the image to be processed. The brightness value range includes a first brightness value and a second brightness value, and the first brightness value is less than the second brightness value. Calculate the relative difference value between the second brightness value and the first brightness value, and determine the number of frame splits corresponding to the image to be processed based on the relative difference value; there is a proportional relationship between the number of frame splits and the relative difference value, that is, the larger the relative difference value, the more the number of frame splits.

[0101] In one embodiment, the electronic device obtains the brightness value range corresponding to the region of interest in the image to be processed, and determines the number of frame splits corresponding to the image to be processed based on the brightness value range corresponding to the region of interest.

[0102] Step 506: Determine multiple gain coefficients based on the scene exposure ratio and the number of frame splits; the number of multiple gain coefficients is equal to the number of frame splits.

[0103] Exemplarily, the electronic device determines multiple gain coefficients according to the scene exposure ratio and the number of frame splits.

[0104] Step 508: Obtain the image of the region of interest to be processed from the image to be processed.

[0105] Among them, the image of the region of interest to be processed refers to the image composed of the regions of interest in the image to be processed.

[0106] Exemplarily, the electronic device performs semantic segmentation on the image to be processed, obtains the region of interest mask, multiplies the image to be processed by the region of interest mask, and obtains the image of the region of interest to be processed.

[0107] Step 510, for each gain coefficient, based on the image of the region of interest to be processed and the gain coefficient, obtain the image of the region of interest.

[0108] Exemplarily, for each gain coefficient, the electronic device multiplies the brightness value of each pixel point in the image of the region of interest to be processed by the gain coefficient, obtains the image of the region of interest corresponding to the gain coefficient, and finally obtains multiple images of the region of interest; the number of images of the region of interest is equal to the number of frame splitting.

[0109] In this embodiment, the number of frame splitting corresponding to the image to be processed is determined by the brightness value range of the image to be processed. The larger the brightness value range, the larger the number of frame splitting, and the more the number of images of the region of interest obtained. Different images of the region of interest correspond to different gain coefficients, that is, images of the region of interest with different scales are obtained, providing sufficient basic data for subsequent determination of the target image of the region of interest.

[0110] In one embodiment, as Figure 6 shown, based on the image to be processed and the target image of the region of interest, determine the target image corresponding to the image to be processed, including:

[0111] Step 602, obtain multiple frame-split images corresponding to the image to be processed; the brightness values of the pixel points corresponding to the pixel points in the multiple frame-split images in the image to be processed are different.

[0112] Among them, the frame-split image refers to the image obtained by multiplying the image to be processed by the gain coefficient. It can be understood that the image obtained by multiplying the brightness value of the pixel point in the image to be processed by the gain coefficient is the frame-split image. The multiple gain coefficients used to convert the image to be processed into multiple frame-split images can be the same as the multiple gain coefficients used to convert the image to be processed into multiple images of the region of interest, or can be different from the multiple gain coefficients used to convert the image to be processed into multiple images of the region of interest. The number of frame-split images can be the same as the number of images of the region of interest, or can be different from the number of images of the region of interest.

[0113] Exemplarily, the electronic device obtains the multiple gain coefficients determined in step 502 and step 504. For each gain coefficient, multiplies the brightness value of each pixel point in the image to be processed by the gain coefficient, and obtains the frame-split image corresponding to the gain coefficient, so as to obtain multiple frame-split images corresponding to the image to be processed.

[0114] Step 604, obtain the segmentation threshold corresponding to the frame-split image.

[0115] Among them, the segmentation threshold refers to the input brightness value at the turning point of the weight curve corresponding to the frame-split image. It can be understood that the weight curve corresponding to the frame-split image consists of at least three curves, and the abscissa value corresponding to the connection of the curves is the segmentation threshold. For example, as Figure 7 shown in the weight curve, P00 and P01 are the segmentation thresholds. A frame-split image corresponds to at least two segmentation thresholds. The segmentation threshold of the frame-split image can be a pre-set threshold. For example, according to the order of the gain coefficients from large to small, multiple groups of segmentation thresholds are set in sequence. Each group of segmentation thresholds includes at least two segmentation thresholds. For example, as Figure 8 shown, it includes the straight lines corresponding to three frame-split images. The slope of each straight line represents the gain coefficient corresponding to the frame-split image. The three frame-split images are ev0, ev1, and ev2 respectively. P00 and P01 are the segmentation thresholds corresponding to ev0, P10, P11, and P12 are the segmentation thresholds corresponding to ev1, and P20, P21, and P22 are the segmentation thresholds corresponding to ev2. P00, P10, and P20 can be the same input brightness value or input brightness values within a certain difference range.

[0116] Exemplarily, the electronic device obtains the segmentation threshold corresponding to the frame-split image based on the gain coefficient corresponding to the frame-split image.

[0117] In one embodiment, obtaining the segmentation threshold corresponding to the frame-split image includes: obtaining the maximum output brightness value; for each frame-split image, based on the maximum output brightness value and the gain coefficient corresponding to the frame-split image, determining the exposure input threshold corresponding to the frame-split image; based on the exposure input thresholds corresponding to multiple frame-split images, respectively determining the segmentation threshold corresponding to each frame-split image. Among them, the maximum output brightness refers to the maximum brightness value at which the pixel points are normally displayed.

[0118] Step 606, based on the segmentation threshold corresponding to the frame-split image, determining the weight curve corresponding to the frame-split image.

[0119] Exemplarily, for each frame-split image, the electronic device determines the weight curve corresponding to the frame-split image according to the segmentation threshold corresponding to the frame-split image.

[0120] Step 608, based on multiple frame-split images, the weight curves corresponding to the frame-split images, and the target image of interest, determining the target image corresponding to the image to be processed.

[0121] Exemplarily, the electronic device determines the non-target image of interest based on multiple frame-split images and the weight curves corresponding to the frame-split images, and obtains the target image corresponding to the image to be processed based on the non-target image of interest and the target image of interest.

[0122] In this embodiment, the image to be processed is converted into frame-split images of multiple scales, the weight curve corresponding to each frame-split image is determined respectively, and multiple frame-split images and the target image of interest are fused at multiple scales to obtain the target image corresponding to the image to be processed, thereby improving the accuracy of the target image.

[0123] In one embodiment, as Figure 9 shown, the segmentation thresholds corresponding to the frame-split image include a first segmentation threshold and a second segmentation threshold, and the first segmentation threshold is less than the second segmentation threshold; based on the segmentation thresholds corresponding to the frame-split image, determining the weight curve corresponding to the frame-split image includes:

[0124] Step 902, when it is determined that the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, the weight corresponding to the input brightness value before the first segmentation threshold is set as the first weight, the weight corresponding to the input brightness value after the second segmentation threshold is set as the second weight, and the weight between the first segmentation threshold and the second segmentation threshold is set as a decreasing curve to obtain the weight curve corresponding to the frame-split image; the first weight is greater than the second weight.

[0125] Among them, the first weight refers to a pre-set weight, and the first weight can be set to 1. The second weight refers to a pre-set weight, and the second weight can be set to 0. The first weight is greater than the second weight, and the first weight and the second weight can be set according to actual needs.

[0126] Exemplarily, the electronic device determines whether the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images. If the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, the weight corresponding to the input brightness value before the first segmentation threshold is set as the first weight, the weight corresponding to the input brightness value after the second segmentation threshold is set as the second weight, and the weight between the first segmentation threshold and the second segmentation threshold is set as a decreasing curve to obtain the weight curve corresponding to the frame-split image. For example, if the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, the weight curve corresponding to the frame-split image can be as Figure 8 shown.

[0127] Step 904, when it is determined that the gain coefficient corresponding to the frame-split image is not the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, the weight corresponding to the input brightness value before the first segmentation threshold is set as the third weight, the weight corresponding to the input brightness value after the second segmentation threshold is set as the fourth weight, and the weight between the first segmentation threshold and the second segmentation threshold is set as an increasing curve to obtain the weight curve corresponding to the frame-split image; the third weight is less than the fourth weight.

[0128] Among them, the third weight refers to a preset weight. The third weight can be the same as the second weight. For example, the third weight is set to 1. The third weight can also be different from the second weight. For example, the third weight is set to 0.9. The fourth weight refers to a preset weight. The fourth weight can be the same as the first weight or different from the second weight. The third weight is less than the fourth weight, and the third weight and the fourth weight can be set according to actual needs.

[0129] Exemplarily, the electronic device determines whether the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images. If the gain coefficient corresponding to the frame-split image is not the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, the weight corresponding to the input luminance value before the first segmentation threshold is set to the third weight, the weight corresponding to the input luminance value after the second segmentation threshold is set to the fourth weight, and the weight between the first segmentation threshold and the second segmentation threshold is set to an increasing curve, thereby obtaining the weight curve corresponding to the frame-split image. For example, if the gain coefficient corresponding to the frame-split image is not the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, the weight curve corresponding to the frame-split image can be as Figure 10 shown.

[0130] In this embodiment, the weight curves corresponding to each frame-split image are determined respectively. Based on the weight curves, appropriate luminance value pixel points can be selected from different frame-split images respectively to participate in the fusion of the target image. Determining the weight curves corresponding to each frame-split image provides accurate basic data for the subsequent participation in the fusion of the target image.

[0131] In one embodiment, based on multiple frame-split images, the weight curves corresponding to the frame-split images, and the target image of interest, determining the target image corresponding to the image to be processed includes:

[0132] For each pixel point to be processed in the image to be processed, when the pixel point to be processed is a pixel point in a non-interested area, based on the target luminance value and the target weight corresponding to the mapped pixel point corresponding to the pixel point to be processed in each frame-split image, determining the fused luminance value corresponding to the pixel point to be processed; when the pixel point to be processed is a pixel point in an interested area, based on the target luminance value of the mapped pixel point corresponding to the pixel point to be processed in the target image of interest, determining the fused luminance value corresponding to the pixel point to be processed;

[0133] Based on the fused luminance values corresponding to each pixel point to be processed, determining the target image corresponding to the image to be processed.

[0134] Among them, the mapped pixel refers to the pixel with the same position as the pixel to be processed in the frame-split image. The target brightness value refers to the brightness value corresponding to the mapped pixel in the frame-split image. The target weight refers to the weight corresponding to the mapped pixel in the weight curve corresponding to the frame-split image, and the target weight is determined by the target input brightness value of the pixel to be processed and the weight curve corresponding to the frame-split image.

[0135] Exemplarily, for each pixel to be processed in the image to be processed, the electronic device determines whether the pixel to be processed is a pixel in the region of interest. If the pixel to be processed is a pixel in the non-region of interest, then based on the target brightness value and the target weight corresponding to the mapped pixel corresponding to the pixel to be processed in each frame-split image, the fused brightness value corresponding to the pixel to be processed is determined; if the pixel to be processed is a pixel in the region of interest, then based on the target brightness value of the mapped pixel corresponding to the pixel to be processed in the target image of interest, the fused brightness value corresponding to the pixel to be processed is determined. Then, according to the fused brightness values corresponding to each pixel to be processed, the target image corresponding to the image to be processed is obtained.

[0136] In this embodiment, according to whether the pixel to be processed is a pixel in the region of interest, different fusion methods are used to determine the fused brightness value of the pixel to be processed, thereby improving the accuracy of the fused brightness value.

[0137] In one embodiment, determining the fused brightness value corresponding to the pixel to be processed based on the target brightness value and the target weight corresponding to the mapped pixel corresponding to the pixel to be processed in each frame-split image includes:

[0138] Obtaining the target input brightness value corresponding to the pixel to be processed; for each frame-split image, determining the target weight of the target input brightness value in the weight curve corresponding to the frame-split image, and determining the target brightness value corresponding to the mapped pixel corresponding to the pixel to be processed in the frame-split image; weighting the target brightness values and target weights corresponding to multiple frame-split images to obtain the fused brightness value corresponding to the pixel to be processed.

[0139] Among them, the target input brightness value refers to the brightness value of the pixel to be processed in the image to be processed.

[0140] Exemplarily, for each pixel to be processed in the image to be processed, the electronic device first obtains the target input brightness value corresponding to the pixel to be processed, secondly determines the target weight of the target input brightness value in the weight curve corresponding to each frame-split image respectively, and determines the target brightness value corresponding to the mapped pixel corresponding to the pixel to be processed in each frame-split image, and thirdly weights the target brightness values and target weights corresponding to multiple frame-split images to obtain the fused brightness value corresponding to the pixel to be processed.

[0141] In one embodiment, the target brightness values and target weights corresponding to multiple unpacked frame images are weighted to obtain the fused brightness value corresponding to the pixel to be processed, including: counting multiple target weights to obtain a target weight statistical value; in the case where the target weight statistical value is not equal to one, normalizing the target weights based on the target weight statistical value to obtain the normalized weights corresponding to the target weights; weighting the target brightness values and the normalized weights corresponding to multiple unpacked frame images to obtain the fused brightness value corresponding to the pixel to be processed.

[0142] In this embodiment, the fusion of multiple unpacked frame images is achieved by weighting the target brightness values and target weights corresponding to the multiple unpacked frame images.

[0143] In an exemplary embodiment, the flowchart of image processing performed by an electronic device such as a mobile phone is as Figure 11 shown. The electronic device uses an image sensor CMOS (Complementary Metal-Oxide-Semiconductor) to collect image data in real time to obtain an initial frame sequence; in the preview mode state, the initial frame sequence is sent to an ISP (Image Signal Processor) chip for preview processing to obtain a reference frame sequence, and the reference frame sequence is stored in a storage unit; in the shooting mode state, the reference frame sequence in the storage unit is sent to a front-end processing module, and the front-end processing module performs optimization processing on the reference frame sequence as Figure 12 shown to obtain a target frame sequence; the optimization processing includes BLC (Black Level Correction), BPC (Bad Pixel Correction), LSC (Lens Shading Correction), and AWB (Auto White Balance).

[0144] The middle-end processing module performs processing on the target frame sequence as Figure 13 shown. The target frame sequence includes digital image 00, digital image 01, and digital image 02. Image restoration processing such as image alignment processing, denoising processing, and demosaicing processing is performed on digital image 00, digital image 01, and digital image 02 to obtain a high-quality frame sequence; the high-quality frame sequence is fused to obtain an image to be processed.

[0145] The back-end processing module performs processing on the image to be processed as Figure 14The processing shown is to obtain the scene exposure ratio corresponding to the image to be processed, determine the number of frames to be split corresponding to the image to be processed based on the brightness value range of the image to be processed, and determine multiple gain coefficients according to the scene exposure ratio and the number of frames to be split. Perform semantic segmentation on the image to be processed to obtain a region of interest mask, multiply the image to be processed by the region of interest mask to obtain an image of the region of interest to be processed. For each gain coefficient, multiply the brightness value of each pixel point in the image of the region of interest to be processed by the gain coefficient to obtain an image of the region of interest corresponding to the gain coefficient, and finally obtain multiple images of the region of interest.

[0146] Count the number of pixel points whose brightness values exceed the brightness threshold value in each image of the region of interest respectively to obtain the statistical quantity corresponding to the image of the region of interest, count the number of pixel points in the image of the region of interest to obtain the total quantity, and divide the statistical quantity by the total quantity to obtain the overexposure ratio corresponding to the image of the region of interest. Compare the overexposure ratio corresponding to each image of the region of interest with the overexposure threshold value respectively. If the overexposure ratio is less than the overexposure threshold value, determine the image of the region of interest as a candidate image to obtain at least one candidate image; for each candidate image, average the brightness values of multiple pixel points in the candidate image to obtain the candidate brightness average value corresponding to the candidate image, obtain the reference brightness average value corresponding to the region of interest, calculate the difference value between the candidate brightness average value and the reference brightness average value, compare multiple difference values to obtain the smallest difference value, and determine the candidate image corresponding to the smallest difference value as the reference image.

[0147] Obtain the brightness value of each pixel point in the reference image, average the obtained multiple brightness values to obtain the average brightness value corresponding to the reference image, obtain the first weight value corresponding to the average brightness value, calculate the second weight value corresponding to the reference value of the region of interest based on the first weight value, multiply the average brightness value by the first weight value to obtain a first result, multiply the reference value of the region of interest by the second weight value to obtain a second result, and add the first result and the second result to obtain the target value of the region of interest; obtain the brightness value of each pixel point in the region of interest in the image to be processed, average the obtained multiple brightness values to obtain the brightness average value of the region of interest; divide the target value of the region of interest by the brightness average value of the region of interest to obtain the target gain coefficient.

[0148] Obtain the region of interest in the image to be processed. For each pixel point in the region of interest, multiply the brightness value of the pixel point by the target gain coefficient to obtain the target gain brightness value of the pixel point, and obtain the target image of the region of interest based on the target gain brightness values of each pixel point.

[0149] For each gain coefficient, multiply the brightness value of each pixel point in the image to be processed by the gain coefficient to obtain the frame-split image corresponding to the gain coefficient, so as to obtain multiple frame-split images corresponding to the image to be processed. Based on the gain coefficient corresponding to the frame-split image, obtain the segmentation threshold corresponding to the frame-split image, and determine the weight curve corresponding to the frame-split image according to the segmentation threshold corresponding to the frame-split image.

[0150] For each pixel point to be processed in the image to be processed, determine whether the pixel point to be processed is a pixel point in the region of interest. If the pixel point to be processed is a pixel point in the non-region of interest, then determine the fused brightness value corresponding to the pixel point to be processed based on the target brightness value and target weight of the mapped pixel point corresponding to the pixel point to be processed in each frame-split image; if the pixel point to be processed is a pixel point in the region of interest, then determine the fused brightness value corresponding to the pixel point to be processed based on the target brightness value of the mapped pixel point corresponding to the pixel point to be processed in the target image of interest. Then, according to the fused brightness values corresponding to each pixel point to be processed, obtain the target image corresponding to the image to be processed. Then perform a series of image processing operations such as sharpening processing and color enhancement processing on the target image to obtain an optimized image, convert the optimized image into YUV (color encoding system) format data, perform image encoding on the YUV format data, and finally display it on the display device of the electronic device.

[0151] In this embodiment, by performing frame-splitting processing on the region of interest in the image to be processed, multiple region-of-interest images with different brightness value ranges are obtained. According to the brightness values of each pixel point in the region-of-interest images, select the region-of-interest image with a low overexposure ratio as the reference image. Then, determine the target gain coefficient corresponding to the region of interest in the image to be processed according to the brightness values of each pixel point in the reference image, that is, determine the target gain coefficient corresponding to the region of interest according to the reference image with higher accuracy, which improves the accuracy of the target gain coefficient. Use the target gain coefficient to process the image to be processed to obtain the target image of interest, and fuse the image to be processed and the target image of interest to obtain the target image corresponding to the image to be processed, thereby improving the accuracy of image processing.

[0152] It should be understood that although each step in the flowcharts involved in the above-described embodiments is sequentially shown as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0153] Based on the same inventive concept, an embodiment of the present application also provides an image processing apparatus for implementing the above-mentioned image processing method. The solution provided by this apparatus for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following image processing apparatus can refer to the limitations on the image processing method in the foregoing, and will not be elaborated here.

[0154] In one embodiment, as Figure 15 shown, an image processing apparatus is provided, including: an acquisition module 1502, a selection module 1504, a determination module 1506, a processing module 1508, and a fusion module 1510, where:

[0155] The acquisition module 1502 is configured to acquire multiple region-of-interest images corresponding to the image to be processed; the region of interest in the multiple region-of-interest images is the same region in the image to be processed, and the brightness values of the pixel points corresponding to the pixel points in the image to be processed in the multiple region-of-interest images are different;

[0156] The selection module 1504 is configured to determine a reference image from the multiple region-of-interest images based on the brightness values of the pixel points in the region-of-interest images;

[0157] The determination module 1506 is configured to determine a target gain coefficient based on the average brightness values of multiple pixel points in the reference image;

[0158] The processing module 1508 is configured to obtain a region-of-interest target image based on the image to be processed and the target gain coefficient;

[0159] The fusion module 1510 is configured to determine a target image corresponding to the image to be processed based on the image to be processed and the region-of-interest target image.

[0160] In one embodiment, the selection module 1504 is further configured to: for each region of interest image, count the number of pixel points in the region of interest image whose brightness value exceeds the brightness threshold value to obtain the corresponding statistical quantity of the region of interest image; count the number of pixel points in the region of interest image to obtain the total quantity; determine the overexposure ratio corresponding to the region of interest image based on the statistical quantity and the total quantity; determine a reference image among the multiple region of interest images based on the overexposure ratio corresponding to each region of interest image.

[0161] In one embodiment, the selection module 1504 is further configured to: determine a region of interest image with an overexposure ratio less than the overexposure threshold as a candidate image; for each candidate image, average the brightness values of multiple pixel points in the candidate image to obtain the corresponding candidate brightness average value of the candidate image; determine the candidate image corresponding to the candidate brightness average value with the smallest difference from the reference brightness average value as the reference image.

[0162] In one embodiment, the determination module 1506 is further configured to: average the brightness values of multiple pixel points in the reference image to obtain the corresponding average brightness value of the reference image; weight the average brightness value and the region of interest reference value to obtain the region of interest target value; average the brightness values of multiple pixel points in the region of interest of the image to be processed to obtain the region of interest brightness average value; obtain the target gain coefficient based on the ratio between the region of interest target value and the region of interest brightness average value.

[0163] In one embodiment, the acquisition module 1502 is further configured to: acquire the scene exposure ratio corresponding to the image to be processed; determine the number of frame splitting of the image to be processed based on the brightness value range of the image to be processed; determine multiple gain coefficients based on the scene exposure ratio and the number of frame splitting; the number of multiple gain coefficients is equal to the number of frame splitting; acquire the region of interest image to be processed from the image to be processed; for each gain coefficient, obtain the region of interest image based on the region of interest image to be processed and the gain coefficient.

[0164] In one embodiment, the fusion module 1510 is further configured to: acquire multiple frame-split images corresponding to the image to be processed; the brightness values of the pixel points corresponding to the pixel points in the image to be processed in the multiple frame-split images are different; acquire the segmentation threshold corresponding to each frame-split image; determine the weight curve corresponding to the frame-split image based on the segmentation threshold corresponding to the frame-split image; determine the target image corresponding to the image to be processed based on the multiple frame-split images, the weight curve corresponding to the frame-split image, and the target image of interest.

[0165] In one embodiment, the fusion module 1510 is further configured to: when it is determined that the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, set the weight corresponding to the input luminance value before the first segmentation threshold to the first weight, set the weight corresponding to the input luminance value after the second segmentation threshold to the second weight, and set the weight between the first segmentation threshold and the second segmentation threshold to a decreasing curve to obtain the weight curve corresponding to the frame-split image; the first weight is greater than the second weight; when it is determined that the gain coefficient corresponding to the frame-split image is not the maximum gain coefficient among the gain coefficients corresponding to multiple frame-split images, set the weight corresponding to the input luminance value before the first segmentation threshold to the third weight, set the weight corresponding to the input luminance value after the second segmentation threshold to the fourth weight, and set the weight between the first segmentation threshold and the second segmentation threshold to an increasing curve to obtain the weight curve corresponding to the frame-split image; the third weight is less than the fourth weight.

[0166] In one embodiment, the fusion module 1510 is further configured to: for each pixel point to be processed in the image to be processed, when the pixel point to be processed is a pixel point in a non-region of interest, determine the fused luminance value corresponding to the pixel point to be processed based on the target luminance value and the target weight corresponding to the mapped pixel point corresponding to the pixel point to be processed in each frame-split image; when the pixel point to be processed is a pixel point in a region of interest, determine the fused luminance value corresponding to the pixel point to be processed based on the target luminance value of the mapped pixel point corresponding to the pixel point to be processed in the target image of interest; determine the target image corresponding to the image to be processed based on the fused luminance values corresponding to the respective pixel points to be processed.

[0167] In one embodiment, the fusion module 1510 is further configured to: obtain the target input luminance value corresponding to the pixel point to be processed; for each frame-split image, determine the target weight of the target input luminance value in the weight curve corresponding to the frame-split image, and determine the target luminance value of the mapped pixel point corresponding to the pixel point to be processed in the frame-split image; perform weighting on the target luminance values and the target weights corresponding to multiple frame-split images to obtain the fused luminance value corresponding to the pixel point to be processed.

[0168] Each module in the above image processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the electronic device in hardware form or be independent of it, or can be stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0169] In one embodiment, an electronic device is provided. The electronic device can be a terminal, and its internal structure diagram can be as Figure 16As shown in the figure. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the electronic device is used to exchange information between the processor and external devices. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements an image processing method. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the electronic device, or an external keyboard, touchpad, or mouse, etc.

[0170] Those skilled in the art can understand that Figure 16 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the electronic device to which the solution of this application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0171] The embodiment of this application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, cause the processors to execute the steps of the image processing method.

[0172] The embodiment of this application also provides a computer program product containing instructions, which, when running on a computer, causes the computer to execute the image processing method.

[0173] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0174] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAM), magnetoresistive random access memories (MRAM), ferroelectric random access memories (FRAM), phase change memories (PCM), graphene memories, etc. Volatile memories can include random access memory (RAM) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., without limitation.

[0175] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0176] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. An image processing method, characterized in that, comprising: obtaining multiple region-of-interest images corresponding to the image to be processed; the regions of interest in the multiple region-of-interest images are the same region in the image to be processed, and the luminance values of the pixel points in the image to be processed corresponding to the pixel points in the multiple region-of-interest images are different; determining a reference image from the multiple region-of-interest images based on the luminance values of the respective pixel points in the region-of-interest images; determining a target gain coefficient based on the average luminance value of multiple pixel points in the reference image; obtaining a region-of-interest target image based on the image to be processed and the target gain coefficient; determining the target image corresponding to the image to be processed based on the image to be processed and the region-of-interest target image.

2. The method according to claim 1, characterized in that, the determining a reference image from the multiple region-of-interest images based on the luminance values of the respective pixel points in the region-of-interest images includes: for each of the region-of-interest images, counting the number of pixel points in the region-of-interest image whose luminance values exceed a luminance threshold value to obtain the statistical quantity corresponding to the region-of-interest image; counting the number of pixel points in the region-of-interest image to obtain the total quantity; determining the overexposure ratio corresponding to the region-of-interest image based on the statistical quantity and the total quantity; determining a reference image from the multiple region-of-interest images based on the overexposure ratio corresponding to each of the region-of-interest images.

3. The method according to claim 2, characterized in that, the determining a reference image from the multiple region-of-interest images based on the overexposure ratio corresponding to each of the region-of-interest images includes: determining the region-of-interest images with an overexposure ratio less than an overexposure threshold value as candidate images; for each of the candidate images, averaging the luminance values of multiple pixel points in the candidate image to obtain the candidate luminance average value corresponding to the candidate image; determining the candidate image corresponding to the candidate luminance average value with the smallest difference from the reference luminance average value as the reference image.

4. The method according to claim 1, characterized in that, the determining a target gain coefficient based on the average luminance value of multiple pixel points in the reference image includes: averaging the luminance values of multiple pixel points in the reference image to obtain the average luminance value corresponding to the reference image; weighting the average luminance value and a region-of-interest reference value to obtain a region-of-interest target value; averaging the luminance values of multiple pixel points in the region of interest in the image to be processed to obtain the region-of-interest luminance average value; obtaining a target gain coefficient based on the ratio between the region-of-interest target value and the region-of-interest luminance average value.

5. The method according to claim 1, characterized in that, the obtaining multiple region-of-interest images corresponding to the image to be processed includes: obtaining the scene exposure ratio corresponding to the image to be processed; determining the number of frame splits corresponding to the image to be processed based on the luminance value range of the image to be processed; Determine a plurality of gain coefficients based on the scene exposure ratio and the number of frame splits; the number of the plurality of gain coefficients is equal to the number of frame splits; Obtain an image of the region of interest to be processed from the image to be processed; For each of the gain coefficients, obtain an image of the region of interest based on the image of the region of interest to be processed and the gain coefficient.

6. The method according to claim 1, wherein, the determining the target image corresponding to the image to be processed based on the image to be processed and the target image of the target of interest includes: Obtain a plurality of frame-split images corresponding to the image to be processed; the brightness values of the pixel points corresponding to the pixel points in the image to be processed in the plurality of frame-split images are different; Obtain the segmentation threshold corresponding to each of the frame-split images; Determine the weight curve corresponding to the frame-split image based on the segmentation threshold corresponding to the frame-split image; Determine the target image corresponding to the image to be processed based on the plurality of frame-split images, the weight curve corresponding to the frame-split image, and the target image of the target of interest.

7. The method according to claim 6, wherein, the segmentation threshold corresponding to the frame-split image includes a first segmentation threshold and a second segmentation threshold, and the first segmentation threshold is less than the second segmentation threshold; the determining the weight curve corresponding to the frame-split image based on the segmentation threshold corresponding to the frame-split image includes: In the case where it is determined that the gain coefficient corresponding to the frame-split image is the maximum gain coefficient among the gain coefficients corresponding to the plurality of frame-split images, set the weight corresponding to the input brightness value before the first segmentation threshold to the first weight, set the weight corresponding to the input brightness value after the second segmentation threshold to the second weight, and set the weight between the first segmentation threshold and the second segmentation threshold to a decreasing curve to obtain the weight curve corresponding to the frame-split image; the first weight is greater than the second weight; In the case where it is determined that the gain coefficient corresponding to the frame-split image is not the maximum gain coefficient among the gain coefficients corresponding to the plurality of frame-split images, set the weight corresponding to the input brightness value before the first segmentation threshold to the third weight, set the weight corresponding to the input brightness value after the second segmentation threshold to the fourth weight, and set the weight between the first segmentation threshold and the second segmentation threshold to an increasing curve to obtain the weight curve corresponding to the frame-split image; the third weight is less than the fourth weight.

8. The method according to claim 6, wherein, the determining the target image corresponding to the image to be processed based on the plurality of frame-split images, the weight curve corresponding to the frame-split image, and the target image of the target of interest includes: For each pixel point to be processed in the image to be processed, in the case where the pixel point to be processed is a pixel point in a non-region of interest, determine the fused brightness value corresponding to the pixel point to be processed based on the target brightness value and the target weight corresponding to the mapped pixel point corresponding to the pixel point to be processed in each of the frame-split images; When the pixel point to be processed is a pixel point in the region of interest, determine the fused brightness value corresponding to the pixel point to be processed based on the target brightness value of the mapped pixel point corresponding to the pixel point to be processed in the target image of the region of interest; Determine the target image corresponding to the image to be processed based on the fused brightness values corresponding to each of the pixel points to be processed.

9. The method according to claim 8, wherein, The determining the fused brightness value corresponding to the pixel point to be processed based on the target brightness value and the target weight value of the mapped pixel point corresponding to the pixel point to be processed in each of the frame-split images includes: Obtain the target input brightness value corresponding to the pixel point to be processed; For each of the frame-split images, determine the target weight value of the target input brightness value in the weight curve corresponding to the frame-split image, and determine the target brightness value of the mapped pixel point corresponding to the pixel point to be processed in the frame-split image; Perform weighting on the target brightness values and the target weight values corresponding to the multiple frame-split images to obtain the fused brightness value corresponding to the pixel point to be processed.

10. An image processing apparatus, wherein, comprising: An acquisition module, configured to acquire multiple region-of-interest images corresponding to an image to be processed; The region of interest in each of the multiple region-of-interest images is the same region in the image to be processed, and the brightness values of the pixel points corresponding to the pixel points in the image to be processed in the multiple region-of-interest images are different; A selection module, configured to determine a reference image from the multiple region-of-interest images based on the brightness values of the pixel points in the region-of-interest images; A determination module, configured to determine a target gain coefficient based on the average brightness value of multiple pixel points in the reference image; A processing module, configured to obtain a target image of the region of interest based on the image to be processed and the target gain coefficient; A fusion module, configured to determine the target image corresponding to the image to be processed based on the image to be processed and the target image of the region of interest.

11. An electronic device, including a memory and a processor, the memory storing a computer program, wherein, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, on which a computer program is stored, wherein, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images