Image processing method, terminal and storage medium

Abstract

The present application discloses an image processing method, a terminal and a storage medium. The method comprises: determining a binary image corresponding to each two LDR image frames based on a pixel difference between each two LDR image frames in at least two LDR image frames used to synthesize an HDR image; wherein the pixels in the binary image represent motion pixels or non-motion pixels; and fusing the at least two LDR image frames based on the binary image corresponding to each two LDR image frames and a fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames to obtain a corresponding HDR image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to an image processing method, a terminal and a storage medium. Background Art

[0002] In the related art, multiple images with different exposure values ​​are usually fused to obtain a high dynamic range (HDR) image. However, when the camera device shakes or there are moving objects in the shooting scene, ghosts or ghosting artifacts may exist in the obtained HDR image. Summary of the invention

[0003] In view of this, the embodiments of the present application hope to provide an image processing method, a terminal and a storage medium to solve the technical problem of ghost images existing in HDR images generated in the related art.

[0004] To achieve the above purpose, the technical solution of this application is implemented as follows:

[0005] The present application provides an image processing method, including:

[0006] Determine a binary image corresponding to each of the two low dynamic range (LDR) images based on a pixel difference between each of the two LDR images in at least two LDR images used to synthesize the HDR image; wherein the pixels in the binary image represent moving pixels or non-moving pixels;

[0007] Based on the binary image corresponding to every two frames of LDR images and the fusion weight corresponding to each pixel point of each frame of LDR image in the at least two frames of LDR images, the at least two frames of LDR images are fused to obtain a corresponding HDR image.

[0008] In the above scheme, the step of determining the binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image comprises at least one of the following:

[0009] Determine a corresponding first binary image based on a pixel difference between each group of corresponding pixel points in each two frames of LDR images and a corresponding first set threshold;

[0010] Based on the inner product of the vectors corresponding to each group of corresponding image blocks in each two frames of LDR images and the corresponding second set threshold, a corresponding second binary image is determined; wherein,

[0011] The vector corresponding to the image block represents the difference between the pixel value of each pixel in the image block and the pixel mean of the image block.

[0012] In the above scheme, when the first binary image corresponding to every two LDR frames and the second binary image corresponding to every two LDR frames are determined, the binary image corresponding to every two LDR frames is determined based on the pixel difference between every two LDR frames in at least two LDR frames used to synthesize the HDR image, and further includes:

[0013] A bitwise OR operation is performed on the corresponding first binary image and the corresponding second binary image to obtain a third binary image corresponding to every two frames of LDR.

[0014] In the above solution, when the corresponding first binary image is determined, the method further includes:

[0015] Determine a corresponding first set threshold value based on the pixel value of the pixel point in the LDR image and a first set data table; the first set data table stores a correspondence between the pixel value and the first set threshold value; the first set threshold value represents the difference between the corresponding pixel value and the pixel value when the human eye can perceive the pixel value change;

[0016] When the corresponding second binary image is determined, the method further includes:

[0017] A corresponding second set threshold is determined based on the vector of the image block corresponding to the pixel point in the LDR image and the second set data table; the second set data table stores the correspondence between the vector and the second set threshold; the second set threshold represents the inner product between the corresponding vector and the vector when the pixel value of the corresponding image block that can be perceived by the human eye changes.

[0018] In the above scheme, the method further includes:

[0019] When the determined binary image indicates that the pixels in the set non-edge area are moving pixels, the moving distance of the terminal is determined based on the sensor data built into the terminal;

[0020] When the moving distance is less than a set distance threshold, error correction processing is performed on the moving pixels in the determined binary image that are in the set non-edge area.

[0021] In the above scheme, the method further includes:

[0022] The fusion weight of each pixel point of each LDR image in the at least two LDR images is determined based on the set curve; wherein,

[0023] The setting curve represents the corresponding relationship between the pixel value of the pixel point and the corresponding fusion weight.

[0024] In the above scheme, the at least two LDR image frames are fused based on the binary image corresponding to each two LDR image frames and the fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames to obtain the corresponding HDR image, including:

[0025] Among the pixel values ​​corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the pixel values ​​of the pixel points represented as moving pixels in the corresponding binary image;

[0026] Based on the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the adjusted at least two LDR images are fused to obtain a corresponding HDR image.

[0027] In the above scheme, the at least two LDR image frames are fused based on the binary image corresponding to each two LDR image frames and the fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames to obtain the corresponding HDR image, including:

[0028] Among the fusion weights corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the fusion weights corresponding to the pixel points represented as motion pixels in the corresponding binary image;

[0029] Based on the adjusted fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the at least two LDR images are fused to obtain a corresponding HDR image.

[0030] In the above scheme, the step of determining the binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image comprises:

[0031] Based on the pixel difference between every two LDR images in at least two LDR images after size reduction, a binary image corresponding to every two LDR images is determined.

[0032] In the above solution, before determining the binary image corresponding to every two LDR frames based on the pixel difference between every two LDR frames in at least two LDR frames used to synthesize the HDR image, the method further includes:

[0033] Based on a first image in at least two LDR image frames, performing brightness registration on a second image in the at least two LDR image frames; wherein the first image is the LDR image with the longest exposure time in the at least two LDR image frames;

[0034] Using an image signal processor of the terminal, image registration is performed on the first image and the second image after brightness registration.

[0035] The present application also provides a terminal, including:

[0036] A first determining unit is used to determine a binary image corresponding to each of the two LDR images based on a pixel difference between each of the two LDR images in at least two LDR images used to synthesize the high dynamic range HDR image; wherein the pixels in the binary image represent moving pixels or non-moving pixels;

[0037] The fusion unit is used to fuse the at least two LDR image frames based on the binary image corresponding to every two LDR image frames and the fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames to obtain a corresponding HDR image.

[0038] The embodiment of the present application further provides a terminal, comprising: a processor and a memory for storing a computer program that can be run on the processor,

[0039] Wherein, the processor is used to execute the steps of the above-mentioned image processing method when running the computer program.

[0040] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned image processing method are implemented.

[0041] In an embodiment of the present application, based on the pixel difference between each two LDR image frames, a binary image corresponding to the two LDR image frames is determined, and based on the determined binary image and the corresponding fusion weight of each pixel point of each LDR image frame in at least two LDR image frames, at least two LDR image frames are fused to obtain a corresponding HDR image. Since the pixels in the corresponding binary image represent motion pixels or non-motion pixels, the pixel points represented as motion pixels in the corresponding binary image can be quickly determined from the corresponding two LDR image frames based on the corresponding binary image, so that the pixel points represented as motion pixels in the at least two LDR image frames are processed, for example, the pixel values ​​of the pixel points represented as motion pixels in the corresponding two LDR image frames are adjusted, or the fusion weights of the pixel points represented as motion pixels in the corresponding two LDR image frames are adjusted, so that when the LDR image is fused, the image information of the pixel points represented as motion pixels can be concealed, so that there is no ghost in the HDR image finally obtained by fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the implementation flow of the image processing method provided in the embodiment of the present application;

[0043] Figure 2 A schematic diagram of a setting curve provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the implementation flow of the image processing method provided in the application embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of the structure of a terminal provided in another embodiment of the present application. DETAILED DESCRIPTION

[0047] The technical solution of the present application is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification.

[0048] Figure 1 The following is a schematic diagram of the implementation process of the image processing method provided in the embodiment of the present application, wherein the execution subject of the process is a terminal such as a mobile phone, a tablet, a digital camera, etc. Figure 1 As shown, the image processing method includes:

[0049] Step 101: Based on the pixel difference between every two LDR images in at least two LDR images used to synthesize the HDR image, determine a binary image corresponding to every two LDR images; wherein the pixels in the binary image represent moving pixels or non-moving pixels.

[0050] The at least two LDR images are LDR images taken in one shooting operation with different exposure times. In practical applications, the terminal can obtain at least two LDR images from a video stream output by a built-in image signal processor (ISP).

[0051] Here, the terminal may obtain at least two LDR images for synthesizing an HDR image, and detect whether the color space of the obtained LDR image is an RGB color space. When the detection result indicates that the color space of the obtained LDR image is an RGB color space, based on the pixel difference between each two LDR images, the binary image corresponding to each two LDR images is determined. When the detection result indicates that the color space of the obtained LDR image is a YUV color space, the color space of the obtained LDR image is converted from the YUV color space to the RGB color space, and based on the pixel difference between each two LDR images, the binary image corresponding to each two LDR images is determined.

[0052] In the binary image, the first value and the second value are used to mark whether the corresponding pixel is a moving pixel or a non-moving pixel. For example, 0 and 1 are used to mark the moving pixel and the non-moving pixel respectively.

[0053] When determining the binary image corresponding to each two frames of LDR images, the terminal can compare the pixel values ​​between the corresponding pixel points of the two frames of LDR images in units of pixels or image blocks, so as to determine whether the pixel points at the same spatial position in the two frames of LDR images are characterized as moving pixels. For example, the terminal compares the pixel difference between each group of corresponding pixel points in the two frames of LDR images pixel by pixel in units of pixels, so as to determine whether the corresponding pixel points are characterized as moving pixels based on the pixel difference; or, the terminal calculates the pixel mean of each image block in each frame of LDR images in units of image blocks, and determines whether the corresponding pixel points are characterized as moving pixels based on the difference between the pixel mean values ​​of the image blocks centered on the corresponding pixel points in the two frames of LDR images.

[0054] In actual application, the terminal can determine a reference image from at least two acquired LDR image frames, compare the reference image with each non-reference image in the at least two LDR image frames one by one, and determine a binary image corresponding to every two LDR image frames.

[0055] In some embodiments, in order to facilitate more accurate determination of whether a pixel point in an LDR image is a moving pixel, before determining a corresponding binary image based on the acquired LDR image, brightness registration and image registration are performed on the acquired LDR image. Before determining a binary image corresponding to each two LDR frames based on a pixel difference between each two LDR frames of at least two LDR frames used to synthesize an HDR image, the method further includes:

[0056] Based on a first image in at least two LDR image frames, performing brightness registration on a second image in the at least two LDR image frames; wherein the first image is the LDR image with the longest exposure time in the at least two LDR image frames;

[0057] The first image and the second image after brightness registration are registered by using the ISP of the terminal.

[0058] Here, the terminal determines a first image with the longest exposure time from at least two LDR image frames, and based on the determined first image, performs brightness alignment on images other than the first image in the at least two LDR image frames; and uses the built-in ISP of the terminal to perform image alignment on the first image and the image after brightness alignment, so that each LDR image in the at least two LDR image frames is aligned based on the same shooting space position.

[0059] In actual application, the terminal can perform a multiplication operation based on the pixel value of the second image and the set exposure ratio to obtain a calculation result, and adjust the pixel value of the second image based on the calculation result so that the pixel value of the adjusted second image is the same as the calculation result, thereby making the brightness of the second image the same as the brightness of the first image. The terminal can also determine a pixel value mapping curve based on the histogram of the first image, and adjust the pixel value of the second image based on the pixel value mapping curve and the histogram of the second image, thereby achieving brightness registration of the second image.

[0060] In this embodiment, after brightness registration and image registration are performed on at least two frames of LDR images, the corresponding binary image is determined, and the pixel difference between each group of corresponding pixel points in each two frames of LDR images can be calculated more quickly and accurately. In addition, since the built-in ISP of the terminal has an image registration function, the built-in ISP is used to perform image registration on at least two frames of LDR images, which can make full use of the hardware resources of the terminal, reduce the time consumed by image registration, and improve image processing efficiency.

[0061] In some embodiments, in order to accurately identify moving pixels, the step of determining a binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image includes at least one of the following:

[0062] Determine a corresponding first binary image based on a pixel difference between each group of corresponding pixel points in each two frames of LDR images and a corresponding first set threshold;

[0063] Based on the inner product of the vector corresponding to the image block where each pixel point in each two frames of LDR image is located and the corresponding second set threshold, the corresponding second binary image is determined; wherein,

[0064] The vector corresponding to the image block represents the difference between the pixel value of each pixel in the image block and the pixel mean of the image block.

[0065] Here, the terminal can compare the pixel values ​​of each group of corresponding pixel points in the two LDR image frames pixel by pixel in units of pixel points, thereby obtaining the pixel difference between each group of corresponding pixel points in the two LDR image frames, and determining the corresponding first binary image based on the pixel difference between each group of corresponding pixel points and the corresponding first set threshold. Alternatively, the terminal can also calculate the pixel mean of each group of corresponding image blocks in the two LDR image frames in units of image blocks, and determine the corresponding first binary image based on the pixel difference between each group of corresponding image blocks and the corresponding first set threshold.

[0066] Wherein, when the pixel difference corresponding to a group of pixel points in two frames of LDR images is greater than or equal to the corresponding first set threshold, the group of pixel points is characterized as a moving pixel in the corresponding first binary image; when the pixel difference corresponding to a group of pixel points is less than the corresponding first set threshold, the pixel points are characterized as a non-moving pixel in the corresponding first binary image, thereby minimizing the error caused by image noise in the process of identifying moving pixels, thereby accurately identifying moving pixels. In this embodiment, different pixel difference values ​​correspond to different first set thresholds.

[0067] The terminal may also determine the vectors corresponding to each group of corresponding image blocks in the two frames of LDR images in units of image blocks, calculate the inner product between the two vectors, and determine the corresponding second binary image based on the inner product between the two vectors and the corresponding second set threshold. Wherein, when the inner product of the two vectors is greater than or equal to the corresponding second set threshold, the central pixel of the corresponding image block is characterized as a motion pixel in the corresponding second binary image; when the inner product of the two vectors is less than the corresponding second set threshold, the central pixel of the corresponding image block is characterized as a non-motion pixel in the corresponding second binary image. When calculating the inner product of the two vectors, the terminal may also normalize the vectors corresponding to the image blocks and calculate the inner product based on the normalized vectors. In this embodiment, the inner products of different vectors correspond to different second set thresholds. Here, based on the inner product of the vectors between each group of corresponding image blocks and the corresponding second set threshold, it is determined whether the central pixel corresponding to the image block is characterized as a motion pixel in the corresponding second binary image. In this process, the continuity difference of the pixel values ​​of the pixels in the LDR image is comprehensively considered, thereby improving the accuracy of the identified motion pixels.

[0068] In some embodiments, when determining the corresponding first binary image, the method also includes: determining a corresponding first set threshold based on the pixel value of the pixel point in the LDR image and a first setting data table; wherein the first setting data table stores the correspondence between the pixel value and the first set threshold; the first set threshold represents the difference between the corresponding pixel value and the pixel value when the human eye can perceive the pixel value change.

[0069] Here, when the terminal compares the pixel difference between each group of corresponding pixels in two LDR image frames pixel by pixel in units of pixels, the terminal can search the first set threshold corresponding to the pixel value from the first set data table based on the pixel value of any pixel in each group of corresponding pixels. When the terminal compares the difference between the pixel means of each group of corresponding image blocks pixel by pixel in units of image blocks, the terminal can search the first set threshold corresponding to the pixel mean from the first set data table based on the pixel mean of any image block in each group of corresponding image blocks.

[0070] Among them, a first setting data table is stored in the terminal. Different first setting thresholds are stored in the first setting data table, which can respectively reflect how much the pixel value change can be perceived by the human eye based on different pixel values. In actual application, the same pixel value can correspond to two first setting thresholds, one of which can reflect how much the pixel value increases based on the pixel value can be perceived by the human eye; the other first setting threshold can reflect how much the pixel value decreases based on the pixel value can be perceived by the human eye.

[0071] In actual application, the terminal can query the first setting data table based on the pixel values ​​of the pixel points in the LDR image with a short exposure time in the two frames of LDR images, and determine the first setting threshold corresponding to the pixel value of the corresponding pixel point based on the first setting thresholds corresponding to different pixel values ​​in the first setting data table. At this time, the corresponding first setting threshold can reflect how much the pixel value increase can be perceived by the human eye based on the corresponding pixel value.

[0072] Of course, the terminal can also query the first setting data table based on the pixel value of the pixel point in the LDR image with a longer exposure time in the two frames of LDR images to obtain the corresponding first setting threshold. At this time, the corresponding first setting threshold can reflect how much the pixel value can be reduced based on the corresponding pixel value and can be perceived by the human eye.

[0073] In some embodiments, when determining the corresponding second binary image, the method also includes: determining a corresponding second set threshold based on the vector of the image block corresponding to the pixel point in the LDR image and a second setting data table; wherein the second setting data table stores the correspondence between the vector and the second set threshold; the second set threshold represents the inner product between the corresponding vector and the vector when the pixel value of the corresponding image block that can be perceived by the human eye changes.

[0074] Here, the terminal may search the second setting threshold value corresponding to the vector from the second setting data table based on the vector corresponding to any image block in each group of corresponding image blocks of the two frames of LDR images.

[0075] Among them, a second setting data table is stored in the terminal. The second setting data table stores second setting thresholds corresponding to different vectors, which can respectively reflect how much the pixel value change of the central pixel point of the corresponding image block can be perceived by the human eye for image blocks centered on different pixels. The different vectors stored in the second setting data table represent the difference between each pixel point and the pixel mean of the image block in the image blocks centered on different pixels. In actual application, each vector in the second setting data table can correspond to two second setting thresholds, one of which can reflect how much the pixel value of the central pixel point of the corresponding image block can be perceived by the human eye; the other second setting threshold can reflect how much the pixel value of the central pixel point of the corresponding image block can be perceived by the human eye.

[0076] In actual application, the terminal can query the second setting data table based on the vector of the image block in the LDR image with a short exposure time in the two frames of LDR images, and determine the second setting threshold corresponding to the corresponding vector based on the second setting thresholds corresponding to different vectors in the second setting data table. At this time, the corresponding second setting threshold can reflect how much the pixel value of the central pixel point of the corresponding image block can be perceived by the human eye.

[0077] Of course, the terminal can also query the second setting data table based on the vector of the image block in the LDR image with a longer exposure time in the two frames of LDR images to obtain the corresponding second setting threshold. At this time, the corresponding second setting threshold can reflect how much the pixel value of the central pixel point of the corresponding image block is reduced and can be perceived by the human eye.

[0078] It should be noted that the size of the image blocks in the two frames of LDR images is the same as the size of the image blocks corresponding to the vectors stored in the second setting data table.

[0079] In some embodiments, in order to more accurately identify moving pixels, when the first binary image corresponding to every two LDR frames and the second binary image corresponding to every two LDR frames are determined, the step of determining the binary image corresponding to every two LDR frames based on the pixel difference between every two LDR frames in at least two LDR frames used to synthesize the HDR image further includes:

[0080] A bitwise OR operation is performed on the corresponding first binary image and the corresponding second binary image to obtain a third binary image corresponding to every two frames of LDR.

[0081] Here, when the first binary image and the second binary image corresponding to the two frames of LDR images are obtained, a bitwise OR operation is performed on the first binary image and the second binary image in units of pixels to obtain a third binary image of the corresponding two frames of LDR images.

[0082] It should be noted that, in the above embodiment, the first binary image or the second binary image corresponding to every two frames of LDR images is taken; in the present embodiment, the third binary image corresponding to the two frames of images is determined based on the first binary image and the second binary image corresponding to every two frames of LDR images, thereby, the moving pixels can be more accurately identified in the third binary image.

[0083] In some embodiments, in the scenario of image fusion of LDR images in a video stream, in order to improve the accuracy of the binary image, when determining the binary image, the method further includes:

[0084] When the determined binary image indicates that the pixels in the set non-edge area are moving pixels, the moving distance of the terminal is determined based on the sensor data built into the terminal;

[0085] When the moving distance is less than a set distance threshold, error correction processing is performed on the moving pixels in the determined binary image that are in the set non-edge area.

[0086] Here, the determined binary image includes a first binary image, a second binary image or a third binary image. Considering that in the process of shooting a video, a moving object usually first appears in the edge area of ​​the image, when a moving object is detected to appear in a non-edge area of ​​the current image, error correction is performed on the pixel points characterized as moving pixels in the current image with reference to the image captured before. In this embodiment, a corresponding binary image sequence is determined based on the LDR image sequence, and based on the determination of the corresponding binary image sequence, error correction processing is performed on the pixels in the non-edge area of ​​the latest binary image in the determined binary image sequence.

[0087] In actual application, when the terminal determines the corresponding binary image sequence based on the LDR image sequence, it determines whether the pixels in the set non-edge area in the latest binary image are characterized as motion pixels. When the pixels in the set non-edge area in the latest binary image are characterized as motion pixels, the terminal's movement distance is determined based on the terminal's built-in sensor data. The built-in sensor includes a gyroscope or an acceleration sensor. When the determined movement distance is less than the set distance threshold, it indicates that the terminal has not moved or the movement amplitude is within the set error range. At this time, error correction processing is performed on the motion pixels in the set non-edge area in the latest binary image in the binary image sequence. The implementation process of the error correction processing is as follows:

[0088] In the latest binary Figure 1 When the first pixel point in the non-edge area is characterized as a moving pixel, the binary image sequence corresponding to the two frames of LDR images is detected except for the latest binary image. Figure 1 Other binary values Figure 2Whether the first pixel point in the non-edge area is characterized as a moving pixel, all the binary values ​​in the corresponding binary image sequence Figure 2 In the case where the first pixel points in the non-edge area are all characterized as non-moving pixels, the latest binary Figure 1 The value corresponding to the first pixel in the image is changed from 1 to 0; all binary values ​​in the corresponding binary image sequence Figure 2 In the case where the first pixel in the non-edge area is partially characterized as a non-moving pixel, the binary value is not modified. Figure 1 The value corresponding to the first pixel in .

[0089] In some embodiments, in order to improve the image processing efficiency, the step of determining the binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image includes:

[0090] Based on the pixel difference between every two LDR images in at least two LDR images after size reduction, a binary image corresponding to every two LDR images is determined.

[0091] Here, after image registration is performed on at least two frames of LDR images, size reduction processing is performed on every two frames of LDR images in the at least two frames of LDR images, and based on the pixel difference between every two frames of LDR images in the at least two frames of LDR images after size reduction, a binary image corresponding to every two frames of LDR images is determined.

[0092] In this embodiment, the LDR image after image registration is reduced in size, so as to determine a binary image based on the LDR image after the size reduction, and determine the fusion weight of each pixel in the LDR image based on the LDR image after the size reduction, which can save time and resources and improve image processing efficiency.

[0093] Step 102: Based on the binary image corresponding to every two LDR image frames and the fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames, the at least two LDR image frames are fused to obtain a corresponding HDR image.

[0094] Here, the terminal determines the fusion weight corresponding to each pixel point of each LDR image in at least two frames of LDR images, and based on the binary image corresponding to each two frames of LDR images and the fusion weight corresponding to each pixel point of each LDR image in at least two frames of LDR images, fuses the at least two frames of LDR images to obtain a corresponding HDR image.

[0095] In actual application, the terminal performs fusion processing on at least two frames of LDR images after brightness registration and image registration.

[0096] It should be noted that, based on the pixel difference between each two LDR images in at least two LDR images after the size reduction, the binary image corresponding to each two LDR images is determined, and after the terminal determines the fusion weight corresponding to each pixel point of each LDR image in at least two LDR images after the size reduction, the at least two LDR images are restored to the size before the size reduction, so as to perform fusion processing on the at least two LDR images of the restored size. Here, determining the fusion weight of the pixel points based on the LDR images after the size reduction can save resources and time and improve the image processing efficiency.

[0097] It should be noted that the binary image mentioned here is the first binary image or the second binary image determined in the previous text; when the corresponding third binary image is determined based on the corresponding first binary image and the corresponding second binary image, the binary image mentioned here is the third binary image.

[0098] Among them, the method for determining the fusion weight corresponding to each pixel point of each frame of LDR image is as follows:

[0099] The terminal can determine a frame of reference image from at least two frames of LDR images, determine a first fusion weight of each pixel in the reference image based on the pixel value of each pixel in the reference image, and determine a second fusion weight of corresponding pixels of each non-reference image in at least two frames of LDR images based on the first fusion weight of each pixel in the reference image. The reference image can be any image in at least two frames of LDR images, and the sum of the first fusion weight and the second fusion weight of each group of corresponding pixels is equal to 1. In actual application, the LDR image with the longest exposure time or the shortest exposure time in at least two frames of LDR images is determined as the reference image.

[0100] For example, in an application scenario where two frames of LDR images are acquired, when the pixel value of a pixel in the first LDR image with the longest exposure time is less than or equal to the third set threshold, the terminal determines the fusion weight of the corresponding pixel in the second LDR image as W0, and determines the fusion weight of the corresponding pixel in the first LDR image as (1-W0); when the pixel value of the pixel in the first LDR image with the longest exposure time is greater than the third set threshold and less than the fourth set threshold, the fusion weight of the corresponding pixel in the first LDR image and the fusion weight of the corresponding pixel in the second LDR image are determined based on the set ratio; when the pixel value of the pixel in the first LDR image with the longest exposure time is greater than or equal to the fourth set pixel threshold, the terminal determines the fusion weight of the corresponding pixel in the second LDR image as W1, and determines the fusion weight of the corresponding pixel in the first LDR image as (1-W1). Wherein, W1 is greater than W0.

[0101] In some embodiments, in order to determine the fusion weight of the pixel point more quickly, the method also includes: determining the fusion weight of each pixel point of each LDR image in the at least two LDR images based on a set curve; wherein the set curve represents the correspondence between the pixel value of the pixel point and the corresponding fusion weight.

[0102] Here, the terminal determines a frame of reference image from at least two frames of LDR images, determines a corresponding setting curve based on the exposure time of the reference image, determines a first fusion weight for each pixel in the reference image based on the determined setting curve, and determines a second fusion weight for corresponding pixels of non-reference images in at least two frames of LDR images based on the first fusion weight of each pixel in the reference image.

[0103] In practical applications, a curve is set to represent the corresponding relationship between the normalized pixel value and the corresponding fusion weight.

[0104] In some examples, in the setting curve, the pixel values ​​in the setting interval and the corresponding fusion weights change linearly, and the pixel values ​​in the non-setting interval correspond to fixed fusion weights. Figure 2 , Figure 2 The corresponding setting curve when the LDR image with the shortest exposure time is used as the reference image is shown.

[0105] like Figure 2 As shown, when the pixel value of a pixel point in the reference image is less than or equal to 0.2 and greater than 0, the first fusion weight of the pixel point in the reference image is 0.2. At this time, the longer the exposure time of the non-reference image in at least two frames of LDR images is, the greater the fusion weight of the corresponding pixel point in the non-reference image is; when the pixel value of the pixel point in the reference image is greater than or equal to 0.8 and less than or equal to 1, the first fusion weight of the pixel point in the reference image is 0.8. At this time, the shorter the exposure time of the non-reference image in at least two frames of LDR images is, the greater the fusion weight of the corresponding pixel point in the non-reference image is.

[0106] It should be noted that when the pixel value of a pixel point of the reference image is greater than 0.2 and less than 0.8, the corresponding fusion weight in each LDR image of at least two LDR images can be determined according to a set ratio.

[0107] After the terminal determines the fusion weight corresponding to each pixel point of each LDR image in at least two LDR images, the terminal may perform fusion processing on the at least two LDR images in the following two ways:

[0108] Mode 1: When fusing the LDR image, the image information of the pixel points characterized as the moving pixel is concealed by adjusting the pixel values ​​of the pixel points characterized as the moving pixel in the LDR image. In some embodiments, the fusing process of the at least two LDR images based on the binary image corresponding to each two LDR images and the fusing weight corresponding to each pixel point of each LDR image in the at least two LDR images to obtain the corresponding HDR image includes:

[0109] Among the pixel values ​​corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the pixel values ​​of the pixel points represented as moving pixels in the corresponding binary image;

[0110] Based on the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the adjusted at least two LDR images are fused to obtain a corresponding HDR image.

[0111] Here, the terminal determines the pixel points represented as motion pixels in the corresponding two LDR images based on the binary images corresponding to each two LDR images, and obtains at least one group of corresponding pixel points represented as motion pixels in the two LDR images; the pixel value of one pixel point in each group of corresponding pixel points in the two LDR images is replaced with the pixel value of another pixel point. This process is equivalent to a cutout operation. The pixel value of one pixel point in each group of corresponding pixel points represented as motion pixels remains unchanged, and the pixel value of another pixel point is modified, so that the pixel value of each group of corresponding pixel points represented as motion pixels is the same. In actual application, a frame of reference image is determined from the LDR image of the synthetic HDR image. When determining each group of corresponding pixel points represented as motion pixels in the reference image and the non-reference image, the pixel value of the corresponding pixel point in the reference image is kept unchanged, and the pixel value of the corresponding pixel point in the non-reference image is modified.

[0112] Based on the adjusted pixel value of each pixel point of each LDR image in the at least two LDR images and the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the at least two LDR images are fused to obtain a corresponding HDR image. In practical applications, before the LDR images are fused, the LDR images with adjusted pixel values ​​may be subjected to image filtering, so that the pixel values ​​of the pixels of the LDR images are smoothly transitioned.

[0113] Before the LDR images are fused, weight filtering may be performed on the fusion weight corresponding to each pixel point of each LDR image in the two LDR images, so that the fusion weight of the pixel points in each LDR image frame smoothly transitions.

[0114] Mode 2: When fusing the LDR image, the image information of the pixel points characterized as the moving pixel is concealed by adjusting the fusing weights of the pixel points in the LDR image that represent the moving pixel. In some embodiments, the fusing process of the at least two LDR images based on the binary image corresponding to each two LDR images and the fusing weight corresponding to each pixel point of each LDR image in the at least two LDR images to obtain the corresponding HDR image includes:

[0115] Among the fusion weights corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the fusion weights corresponding to the pixel points represented as motion pixels in the corresponding binary image;

[0116] Based on the adjusted fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the at least two LDR images are fused to obtain a corresponding HDR image.

[0117] Here, based on the binary image corresponding to each two frames of LDR images, the fusion weight corresponding to the pixel points characterized as motion pixels in the corresponding binary image is adjusted in the determined fusion weight, so that when the LDR image is fused, the image information of the pixel points characterized as motion pixels can be concealed. For example, the terminal determines a frame of reference image from at least two frames of LDR images, determines at least one group of pixel points characterized as motion pixels in the reference image and the non-reference image based on the binary image corresponding to the reference image and the non-reference image in at least two frames of LDR images, resets the fusion weight of the corresponding pixel points in the reference image to 1, and resets the fusion weight of the corresponding pixel points in the corresponding non-reference image to 0. Alternatively, when at least one group of pixel points characterized as motion pixels in the reference image and the non-reference image is determined, when the fusion weight of the corresponding pixel points in the reference image is less than or equal to the set weight threshold, the fusion weight of the corresponding pixel points in the reference image is kept unchanged, and the fusion weight of the corresponding pixel points in the non-reference image is replaced by the pixel value of the corresponding pixel points in the reference image, so that the sum of the fusion weights of each group of corresponding pixel points at the same position in the adjusted at least two frames of LDR images is less than or equal to 1. In practical application, the LDR image with the shortest exposure time among at least two LDR images is determined as the reference image.

[0118] Based on the adjusted fusion weight of each pixel point of each LDR image in the at least two LDR images and the corresponding pixel value, the at least two LDR images are weightedly fused to obtain an HDR image. Wherein, when the at least two LDR images are fused, the at least two LDR images can be weightedly fused based on the fusion weight of the pixel point and the corresponding pixel value to obtain an HDR image; or the at least two LDR images can be weightedly fused using a Laplacian pyramid based on the fusion weight of each pixel point of each LDR image in the at least two LDR images.

[0119] It should be noted that after obtaining the corresponding HDR image, tone mapping can also be performed on the corresponding HDR image to adjust the brightness of the corresponding HDR image, and the tone-mapped HDR image is input to the ISP for noise reduction, color correction, etc. Here, the terminal can perform local tone mapping or global tone mapping on the HDR image.

[0120] In this embodiment, based on the pixel difference between each two LDR images in at least two low dynamic range LDR images used to synthesize the HDR image, a binary image corresponding to each two LDR images is determined, and based on the binary image corresponding to each two LDR images and the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, at least two LDR images are fused to obtain a corresponding HDR image. Since the pixels in the corresponding binary image represent motion pixels or non-motion pixels, the pixels represented as motion pixels in the corresponding binary image can be quickly determined from the corresponding two LDR images based on the corresponding binary image, thereby adjusting the pixel values ​​of the pixels represented as motion pixels in the corresponding two LDR images, or adjusting the fusion weights of the pixels represented as motion pixels in the corresponding two LDR images, so that when the LDR images are fused, the image information of the pixels represented as motion pixels can be concealed, so that there is no ghost in the fused HDR image.

[0121] In addition, since the ISP in the terminal is used to perform image registration on the acquired LDR image in the embodiment of the present application, the image registration efficiency is improved; the binary image is used to quickly determine the pixel points in the LDR image that are characterized as motion pixels, and the fusion weight of each pixel point of the LDR image is quickly determined based on the set curve, thereby the acquired LDR image can be quickly fused into the corresponding HDR image, thereby improving the fusion efficiency of the LDR image. This solution can be applicable to image processing scenarios with high real-time requirements, for example, fusion processing of LDR images in video streams.

[0122] Figure 3 The following is a schematic diagram of the implementation process of the image processing method provided by the application embodiment of the present application. Figure 3As shown, the implementation process of the image processing method is as follows:

[0123] Obtain at least two frames of LDR images output by the ISP for synthesizing an HDR image.

[0124] The color space of at least two frames of LDR images output by the ISP is converted from the YUV color space to the RGB color space to obtain at least two frames of LDR images in the RGB color space.

[0125] In the case that the at least two frames of LDR images acquired have been subjected to Gamma correction, inverse Gamma correction is performed on the at least two frames of LDR images in the RGB color space.

[0126] Perform brightness registration on at least two LDR images after inverse gamma correction.

[0127] Image registration is performed on at least two frames of LDR images after brightness registration.

[0128] Based on the pixel difference between each two LDR images in the at least two LDR images after image registration, a binary image corresponding to each two LDR images is determined. When determining the binary image, a corresponding binary image is determined from a reference image in the at least two LDR images based on the pixel difference between the reference image and the non-reference image in the at least two LDR images.

[0129] The fusion weight of each pixel of each LDR image in at least two LDR images after image registration is determined. The fusion weight of each pixel of the reference image can be determined, and a weight map of the reference image is generated based on the fusion weight of each pixel of the reference image, and a weight map of the non-reference image is generated based on the weight map of the reference image.

[0130] The fusion weight of each pixel in each frame of LDR image is filtered.

[0131] Based on the determined binary image and the filtered fusion weight, at least two LDR images after image registration are fused to obtain a corresponding HDR image.

[0132] Gamma correction for HDR images.

[0133] Perform tone mapping on the gamma-corrected HDR image.

[0134] Convert the color space of the tone-mapped HDR image from RGB to YUV.

[0135] The HDR image in the YUV color space is input into the ISP for noise reduction, color correction and other processing.

[0136] To implement the method of the embodiment of the present application, the embodiment of the present application also provides a terminal, such as Figure 4 As shown, the terminal includes:

[0137] A first determining unit 41 is configured to determine a binary image corresponding to each of the two LDR images based on a pixel difference between each of the two LDR images in at least two LDR images used to synthesize the HDR image; wherein the pixels in the binary image represent moving pixels or non-moving pixels;

[0138] The fusion unit 42 is used to perform fusion processing on the at least two LDR image frames based on the binary image corresponding to every two LDR image frames and the fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames to obtain a corresponding HDR image.

[0139] In some embodiments, the first determining unit 41 is configured to perform at least one of the following:

[0140] The step of determining the binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image comprises at least one of the following:

[0141] Determine a corresponding first binary image based on a pixel difference between each group of corresponding pixel points in each two frames of LDR images and a corresponding first set threshold;

[0142] Based on the inner product of the vectors corresponding to each group of corresponding image blocks in each two frames of LDR images and the corresponding second set threshold, a corresponding second binary image is determined; wherein,

[0143] The vector corresponding to the image block represents the difference between the pixel value of each pixel in the image block and the pixel mean of the image block.

[0144] In some embodiments, when the first binary image corresponding to every two LDR frames and the second binary image corresponding to every two LDR frames are determined, the first determining unit 41 is further used to:

[0145] A bitwise OR operation is performed on the corresponding first binary image and the corresponding second binary image to obtain a third binary image corresponding to every two frames of LDR.

[0146] In some embodiments, when determining the corresponding first binary image, the first determining unit 41 is further configured to:

[0147] Determine a corresponding first set threshold value based on the pixel value of the pixel point in the LDR image and a first set data table; the first set data table stores a correspondence between the pixel value and the first set threshold value; the first set threshold value represents the difference between the corresponding pixel value and the pixel value when the human eye can perceive the pixel value change;

[0148] When determining the corresponding second binary image, the first determining unit 41 is further configured to:

[0149] A corresponding second set threshold is determined based on the vector of the image block corresponding to the pixel point in the LDR image and the second set data table; the second set data table stores the correspondence between the vector and the second set threshold; the second set threshold represents the inner product between the corresponding vector and the vector when the pixel value of the corresponding image block that can be perceived by the human eye changes.

[0150] In some embodiments, the terminal further includes:

[0151] A second determining unit is used to determine the moving distance of the terminal based on the sensor data built into the terminal when the determined binary image represents that the pixels in the set non-edge area are moving pixels;

[0152] The error correction unit is used to perform error correction processing on the moving pixels in the determined binary image that are in the set non-edge area when the moving distance is less than the set distance threshold.

[0153] In some embodiments, the terminal further includes:

[0154] The third determining unit is used to determine the fusion weight of each pixel point of each LDR image in the at least two LDR images based on the set curve; wherein,

[0155] The setting curve represents the corresponding relationship between the pixel value of the pixel point and the corresponding fusion weight.

[0156] In some embodiments, the fusion unit 42 is used to:

[0157] Among the pixel values ​​corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the pixel values ​​of the pixel points represented as moving pixels in the corresponding binary image;

[0158] Based on the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the adjusted at least two LDR images are fused to obtain a corresponding HDR image.

[0159] In some embodiments, the fusion unit 42 is used to:

[0160] Among the fusion weights corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the fusion weights corresponding to the pixel points represented as motion pixels in the corresponding binary image;

[0161] Based on the adjusted fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the at least two LDR images are fused to obtain a corresponding HDR image.

[0162] In some embodiments, the first determining unit 41 is configured to:

[0163] Based on the pixel difference between every two LDR images in at least two LDR images after size reduction, a binary image corresponding to every two LDR images is determined.

[0164] In some embodiments, the terminal further includes:

[0165] A brightness registration unit, configured to perform brightness registration on a second image in at least two LDR image frames based on a first image in the at least two LDR image frames; wherein the first image is the LDR image with the longest exposure time in the at least two LDR image frames;

[0166] The image registration unit is used to perform image registration on the first image and the second image after brightness registration by using an image signal processor of the terminal.

[0167] In actual application, each unit included in the terminal can be implemented by a processor in the terminal, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA).

[0168] It should be noted that: when the terminal provided in the above embodiment performs image processing, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-described processing. In addition, the terminal provided in the above embodiment and the image processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0169] Based on the hardware implementation of the above program modules and in order to implement the method of the embodiment of the present application, the embodiment of the present application also provides a terminal. Figure 5 Schematic diagram of the hardware structure of the terminal in the embodiment of the present application. Figure 5 As shown, the terminal 5 includes:

[0170] Communication interface 51, capable of exchanging information with other devices such as network devices;

[0171] The processor 52 is connected to the communication interface 51 to implement information exchange with other devices and is used to execute the method provided by one or more technical solutions on the terminal side when running the computer program. The computer program is stored in the memory 53.

[0172] Of course, in actual application, the various components in the terminal 5 are coupled together through the bus system 54. It can be understood that the bus system 54 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 54 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 54 .

[0173] The memory 53 in the embodiment of the present application is used to store various types of data to support the operation of the terminal 5. Examples of such data include: any computer program used to operate on the terminal 5.

[0174] It can be understood that the memory 53 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 130 described in the embodiments of the present application is intended to include but is not limited to these and any other suitable types of memories.

[0175] The method disclosed in the above embodiment of the present application can be applied to the processor 52, or implemented by the processor 52. The processor 52 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 52 or the instruction in the form of software. The above processor 52 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 52 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 53, and the processor 52 reads the program in the memory 53 and completes the steps of the above method in combination with its hardware.

[0176] Optionally, when the processor 52 executes the program, it implements the corresponding processes implemented by the terminal in each method of the embodiments of the present application, which will not be described here for the sake of brevity.

[0177] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 53 storing a computer program, and the computer program can be executed by a processor 52 of a terminal to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0179] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0180] In addition, all functional units in the embodiments of the present application may be integrated into one processing module, or each unit may be separately configured as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0181] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0182] It should be noted that the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, characterized in that: include: Determine a binary image corresponding to each of the two LDR images based on a pixel difference between each of the two LDR images in at least two LDR images used to synthesize a high dynamic range HDR image; wherein the pixels in the binary image represent moving pixels or non-moving pixels; Based on the binary image corresponding to every two LDR images and the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the at least two LDR images are fused to obtain a corresponding HDR image; The step of determining the binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image comprises at least one of the following: Determine a corresponding first binary image based on a pixel difference between each group of corresponding pixel points in each two frames of LDR images and a corresponding first set threshold; Based on the inner product of the vectors corresponding to each group of corresponding image blocks in every two frames of LDR images and the corresponding second set threshold, a corresponding second binary image is determined; wherein the vector corresponding to the image block represents the difference between the pixel value of each pixel point in the image block and the pixel mean of the image block.

2. The method according to claim 1, characterized in that: In the case where the first binary image corresponding to every two LDR frames and the second binary image corresponding to every two LDR frames are determined, the method of determining the binary image corresponding to every two LDR frames based on the pixel difference between every two LDR frames in at least two LDR frames used to synthesize the HDR image further includes: A bitwise OR operation is performed on the corresponding first binary image and the corresponding second binary image to obtain a third binary image corresponding to every two frames of LDR.

3. The method according to claim 1, characterized in that When the corresponding first binary image is determined, the method further includes: Determine a corresponding first set threshold value based on the pixel value of the pixel point in the LDR image and a first set data table; the first set data table stores a correspondence between the pixel value and the first set threshold value; the first set threshold value represents the difference between the corresponding pixel value and the pixel value when the human eye can perceive the pixel value change; When the corresponding second binary image is determined, the method further includes: A corresponding second set threshold is determined based on the vector of the image block corresponding to the pixel point in the LDR image and the second set data table; the second set data table stores the correspondence between the vector and the second set threshold; the second set threshold represents the inner product between the corresponding vector and the vector when the pixel value of the corresponding image block that can be perceived by the human eye changes.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: When the determined binary image indicates that the pixels in the set non-edge area are moving pixels, the moving distance of the terminal is determined based on the sensor data built into the terminal; When the moving distance is less than a set distance threshold, error correction processing is performed on the moving pixels in the determined binary image that are in the set non-edge area.

5. The method according to claim 1, characterized in that The method further comprises: The fusion weight of each pixel point of each LDR image in the at least two LDR images is determined based on the set curve; wherein, The setting curve represents the corresponding relationship between the pixel value of the pixel point and the corresponding fusion weight.

6. The method according to claim 1, characterized in that The method of fusing the at least two LDR images based on the binary images corresponding to every two LDR images and the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images to obtain the corresponding HDR image comprises: Among the pixel values ​​corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the pixel values ​​of the pixel points represented as moving pixels in the corresponding binary image; Based on the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the adjusted at least two LDR images are fused to obtain a corresponding HDR image.

7. The method according to claim 1, characterized in that The method of fusing the at least two LDR images based on the binary images corresponding to every two LDR images and the fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images to obtain the corresponding HDR image comprises: Among the fusion weights corresponding to each pixel point of each LDR image in the at least two LDR image frames, adjusting the fusion weights corresponding to the pixel points represented as motion pixels in the corresponding binary image; Based on the adjusted fusion weight corresponding to each pixel point of each LDR image in the at least two LDR images, the at least two LDR images are fused to obtain a corresponding HDR image.

8. The method according to claim 1, characterized in that: The method of determining a binary image corresponding to each two LDR frames based on a pixel difference between each two LDR frames of at least two LDR frames used to synthesize the HDR image comprises: Based on the pixel difference between every two LDR images in at least two LDR images after size reduction, a binary image corresponding to every two LDR images is determined.

9. The method according to any one of claims 1 to 3 and 5 to 8, characterized in that: Before determining the binary image corresponding to each two LDR frames based on the pixel difference between each two LDR frames in at least two LDR frames used to synthesize the HDR image, the method further includes: Based on a first image in at least two LDR image frames, performing brightness registration on a second image in the at least two LDR image frames; wherein the first image is the LDR image with the longest exposure time in the at least two LDR image frames; Using an image signal processor of the terminal, image registration is performed on the first image and the second image after brightness registration.

10. A terminal, characterized in that: include: A first determining unit is used to determine a binary image corresponding to each of the two LDR images based on a pixel difference between each of the two LDR images in at least two LDR images used to synthesize the high dynamic range HDR image; wherein the pixels in the binary image represent moving pixels or non-moving pixels; a fusion unit, configured to perform fusion processing on the at least two LDR image frames based on the binary image corresponding to every two LDR image frames and the fusion weight corresponding to each pixel point of each LDR image frame in the at least two LDR image frames to obtain a corresponding HDR image; The first determination unit is further used to perform at least one of the following: determining a corresponding first binary image based on a pixel difference between each group of corresponding pixel points in each two frames of LDR images and a corresponding first set threshold; determining a corresponding second binary image based on an inner product of vectors corresponding to each group of corresponding image blocks in each two frames of LDR images and a corresponding second set threshold; wherein the vector corresponding to the image block represents the difference between the pixel value of each pixel point in the image block and the pixel mean of the image block.

11. A terminal, characterized in that: include: a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 9.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images