HDR image generation method and device, storage medium and computer equipment

Abstract

The embodiment of the invention discloses an HDR image generation method and device, a storage medium and computer equipment, and relates to the field of image processing. According to the invention, two images with different exposures are obtained and are filtered. Next, the exposure amounts are compared, a long-exposure image and a short-exposure image are determined, and an exposure ratio is calculated. Then, adjusting the pixel value of the long-exposure image, traversing the image by using a rectangular sliding window, calculating the similarity, and judging by applying a threshold value to generate a mask image of the motion area; then, performing expansion operation on the mask image to obtain a weight sequence image; and finally, performing normalization processing on the weight sequence image, and performing weighted averaging on the long exposure image and the short exposure image according to the normalization processing to generate an HDR image. In the whole process, through fine image processing steps, the quality of the HDR image is effectively improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and particularly to a method, apparatus, storage medium, and computer device for generating HDR images. Background Art

[0002] HDR (High Dynamic Range Imaging) technology is a processing technology aimed at improving the brightness and contrast of images. Compared with traditional images, HDR images can provide a wider dynamic range and richer image details, thus more realistically reflecting the visual effects in the actual scene. HDR technology captures and displays details from the darkest to the brightest, greatly expanding the brightness range of the image, making the picture more transparent, clear, and rich in details.

[0003] There are various methods for obtaining HDR images, mainly including the following:

[0004] Enhancing the dynamic range through image signal processing algorithms: Processing the image signal through specific algorithms to enhance the dynamic range of the image, thereby obtaining an HDR image. This method does not require changing the hardware structure, but may be limited by the algorithm performance and computational complexity.

[0005] Synthesizing images with different exposure amounts: This is one of the most commonly used methods for obtaining HDR images. By taking multiple images of the same scene with different exposure amounts, and then using a specific algorithm to synthesize these images into one HDR image. This method can effectively capture and display the bright and dark details in the scene, but may also introduce some problems, such as the image inconsistency problem caused by the time difference as addressed in this application.

[0006] Configuring multiple pixels with different sensitivities in a single pixel of an image sensor: This method integrates multiple sub-pixels with different sensitivities in a single pixel of the image sensor to achieve the capture of different brightness ranges, thereby directly obtaining an HDR image. However, this method has high requirements for hardware and may increase the manufacturing cost and complexity of the image sensor.

[0007] In the method of synthesizing HDR images through different exposure frames, due to the time difference between different exposure frames, relative motion between images will inevitably be introduced during imaging. This relative motion causes the images of different exposure frames not to be completely consistent, thus introducing ghosting and blurring phenomena during the HDR fusion process.

[0008] Ghosting: When there are moving objects in the scene, the position of the moving objects in different exposure frames will change due to the time difference between different exposure frames. If these position changes are not processed during HDR fusion, obvious ghosting will appear in the final HDR image, that is, the outline of the moving object will appear repeatedly in different positions.

[0009] Blurring: In addition to ghosting, relative motion can also cause image blur. Since there are differences in image content between different exposure frames, these differences may be smoothed out during the fusion process, resulting in loss of image details and blurring.

[0010] At present, although there are many HDR image acquisition methods, the image inconsistency problem caused by time difference in the process of synthesizing HDR images through different exposure frames is still a difficult problem to be solved. Existing HDR fusion algorithms often find it difficult to completely eliminate ghosting and blurring, thus affecting the final image quality. Summary of the invention

[0011] The embodiments of the present application provide a method, device, storage medium and computer equipment for generating HDR images, which can solve the problems of ghosting and blurring of HDR images generated in the prior art. The technical solution is as follows:

[0012] In a first aspect, an embodiment of the present application provides a method for generating an HDR image, the method comprising:

[0013] Acquire a first image and a second image captured by an image sensor, wherein the first image and the second image have different exposure amounts;

[0014] Performing filtering on the first image and the second image;

[0015] comparing the exposure amounts of the filtered first image and the filtered second image, taking the image with the larger exposure amount as the long-exposure image, taking the image with the smaller exposure amount as the short-exposure image, and calculating an exposure ratio R between the long-exposure image and the short-exposure image;

[0016] Dividing each pixel value of the long exposure image by the exposure ratio to obtain a converted exposure image;

[0017] Generate a rectangular sliding window of a preset size, and use this rectangular sliding window to slide in the short-exposure image and the converted exposure image to traverse each pixel point; during the traversal, calculate the similarity IM between two rectangular sliding windows with the same pixel coordinates in the short-exposure image and the converted exposure image, then judge the larger value between IM-T1 and 0, divide the larger value by the pixel value of this pixel coordinate in the long-exposure image to obtain an intermediate detection result IM', then judge whether the intermediate detection result IM' is greater than T2, if it is, set the intermediate detection result IM' to T2, otherwise keep it unchanged; generate a motion area mask image according to the motion detection results of each pixel coordinate; T1 is a preset first threshold, and T2 is a preset second threshold;

[0018] Obtain a weight sequence image according to each pixel point in the motion area mask image after performing a dilation operation;

[0019] Normalize each pixel point in the weight sequence image to obtain a normalized image, then use the pixel value of each pixel point in the normalized image as a weight, and perform weighted averaging on the pixel points with the same pixel coordinates in the long-exposure image and the short-exposure image to obtain HDR pixel points, and generate an HDR image according to each generated HDR pixel point.

[0020] In a second aspect, an embodiment of the present application provides a device for generating an HDR image, and the device includes:

[0021] An acquisition unit, configured to acquire a first image and a second image collected by an image sensor, where the first image and the second image have different exposure amounts;

[0022] A filtering unit, configured to perform filtering processing on the first image and the second image;

[0023] A comparison unit, configured to compare the exposure amounts of the filtered first image and the filtered second image, use the image with the larger exposure amount as the long-exposure image, use the image with the smaller exposure amount as the short-exposure image, and calculate the exposure ratio R between the long-exposure image and the short-exposure image;

[0024] A conversion unit, configured to divide each pixel value of the long-exposure image by the exposure ratio to obtain a converted exposure image;

[0025] A detection unit, configured to generate a rectangular sliding window of a preset size, and use the rectangular sliding window to slide in the short-exposure image and the converted exposure image to traverse each pixel point; during the traversal process, calculate the similarity IM between two rectangular sliding windows with the same pixel coordinates in the short-exposure image and the converted exposure image, then determine the larger value between IM-T1 and 0, divide the larger value by the pixel value of this pixel coordinate in the long-exposure image to obtain an intermediate detection result IM', then determine whether the intermediate detection result IM' is greater than T2, if so, set the intermediate detection result IM' to T2, otherwise keep it unchanged; generate a motion area mask image according to the motion detection results of each pixel coordinate; T1 is a preset first threshold, and T2 is a preset second threshold;

[0026] An expansion unit, configured to perform an expansion operation on each pixel point in the motion area mask image to obtain a weight sequence image;

[0027] A weighting unit, configured to perform normalization processing on each pixel point in the weight sequence image to obtain a normalized image, then use the pixel value of each pixel point in the normalized image as a weight, and perform weighted averaging on the pixel points with the same pixel coordinates in the long-exposure image and the short-exposure image to obtain HDR pixel points, and generate an HDR image according to each generated HDR pixel point.

[0028] In a third aspect, an embodiment of the present application provides a computer storage medium, which stores multiple instructions, and the instructions are suitable for being loaded and executed by a processor to perform the above method steps.

[0029] In a fourth aspect, an embodiment of the present application provides a computer device, which may include: a processor and a memory; wherein, the memory stores a computer program, and the computer program is suitable for being loaded and executed by the processor to perform the above method steps.

[0030] The beneficial effects brought by the technical solutions provided by some embodiments of the present application at least include:

[0031] By acquiring images with different exposure levels and performing filtering processing, image noise is reduced, providing a higher-quality image basis for subsequent processing. By comparing the exposure levels and calculating the exposure ratio, the pixel values of the long-exposure image are appropriately adjusted, which helps reduce the ghosting phenomenon caused by exposure differences. At the same time, by traversing the image with a rectangular sliding window, calculating the similarity and performing threshold judgment, the moving areas can be accurately identified and processed, further eliminating ghosting. By performing a dilation operation on the masked image of the moving area, a weighted sequence image is obtained, which helps enhance the edge information in the image and reduce the degradation of image quality caused by motion blur. Finally, by normalizing the weighted sequence image and using the normalized pixel values as weights to perform weighted averaging on the long-exposure and short-exposure images, high-quality HDR pixel points are generated, thus significantly improving the overall quality of the HDR image. In summary, through innovative image processing methods and steps, this technical solution effectively solves the problem of difficult elimination of ghosting and blur phenomena in existing HDR fusion algorithms and significantly improves the quality of HDR images. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0033] Figure 1 is a schematic flowchart of a method for generating an HDR image provided by an embodiment of the present application;

[0034] Figure 2 is a schematic diagram of a dilation operation provided by an embodiment of the present application;

[0035] Figure 3 is a schematic structural diagram of a device for generating an HDR image provided by the present application;

[0036] Figure 4 is a schematic structural diagram of a computer device provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail in conjunction with the drawings.

[0038] Please refer to Figure 2 , which is a schematic flowchart of a method for generating an HDR image provided by an embodiment of the present application. As Figure 2 shown, the method of the embodiment of the present application may include the following steps:

[0039] S101. Obtain a first image and a second image collected by an image sensor, where the first image and the second image have different exposure amounts.

[0040] Among them, the processor obtains two images collected by the image sensor based on the same target in the same scene, namely the first image and the second image. These two images have different exposure amounts. Usually, one is a high-exposure (long-exposure) image, and the other is a low-exposure (short-exposure) image.

[0041] S102. Perform filtering processing on the first image and the second image.

[0042] Among them, the processor performs filtering processing on the two obtained images. The purpose of filtering processing is to remove noise in the images and improve the image quality. Common filtering methods include Gaussian filtering, mean filtering, etc. The processor will select a suitable filter to process the images to reduce the high-frequency noise in the images.

[0043] In some embodiments of the present application, the first image and the second image are filtered by means of convolution.

[0044] Among them, the convolution kernel (also called a filter) is a small matrix that defines how to slide on the image and calculate new pixel values. The size of the convolution kernel is usually a relatively small matrix such as 3×3, 5×5, etc., but it can also be other sizes. Each element in the convolution kernel represents a weight, and these weights are multiplied by the pixel values at the corresponding positions in the image and summed to obtain new pixel values.

[0045] The convolution operation includes the following steps:

[0046] Slide the convolution kernel: The convolution kernel slides from left to right and from top to bottom on the image. Each time it slides, the convolution kernel covers a region on the image. Dot product calculation: For each region covered by the convolution kernel, calculate the dot product of the convolution kernel and the pixel values of the corresponding region. This usually involves multiplying each element in the convolution kernel by the pixel value at the corresponding position in the image and then summing all the products. Update the pixel value: Use the result of the dot product as the pixel value at the corresponding position in the new image. In this way, as the convolution kernel slides on the image, the entire new image can be gradually calculated. In the convolution operation, parameters such as the stride and padding can also be set to adjust the filtering effect. The stride determines the distance at which the convolution kernel slides on the image, and the padding is used to add additional pixel values at the image edges to avoid edge effects.

[0047] By adjusting the weight values in the convolution kernel, different filtering effects can be achieved:

[0048] Edge detection: Using a specific convolution kernel (such as the Sobel operator) can highlight the edge information in the image.

[0049] Blur: Using a mean filter (i.e., a convolution kernel with equal weights for all elements) can smooth the image, reducing noise and details.

[0050] Sharpening: By increasing the weight of the central pixel and decreasing the weights of the surrounding pixels, the image can be made to look sharper.

[0051] For the first image and the second image, the same convolution kernel can be used for filtering. In this way, both images will be affected by the same filtering effect, thus maintaining their consistency. In practical applications, appropriate convolution kernels and filtering parameters can be selected according to needs to achieve the best image processing effect.

[0052] In summary, using convolution to filter images is a flexible and effective image processing method. By adjusting the convolution kernel and filtering parameters, various image processing effects can be achieved to meet different application requirements.

[0053] S103. Compare the exposure amounts of the filtered first image and the filtered second image, take the image with the larger exposure amount as the long-exposure image, take the image with the smaller exposure amount as the short-exposure image, and calculate the exposure ratio R of the long-exposure image to the short-exposure image.

[0054] Among them, the processor compares the exposure amounts of the two filtered images. By calculating the average brightness of the image or other exposure amount metrics, it is determined which image is the long-exposure image (with a large exposure amount) and which is the short-exposure image (with a small exposure amount). At the same time, calculate the exposure ratio R of the long-exposure image to the short-exposure image, that is, the ratio of the exposure amount of the long-exposure image to the exposure amount of the short-exposure image.

[0055] S104. Divide each pixel value of the long-exposure image by the exposure ratio to obtain a converted exposure image.

[0056] Among them, according to the exposure ratio R, the processor divides each pixel value of the long-exposure image by the exposure ratio R to obtain a converted exposure image. The purpose of this step is to adjust the exposure amount of the long-exposure image to a level similar to that of the short-exposure image, preparing for subsequent image fusion.

[0057] S105. Generate a rectangular sliding window of a preset size, and use this rectangular sliding window to slide in the short-exposure image and the converted exposure image to traverse each pixel point; during the traversal, calculate the similarity IM between two rectangular sliding windows with the same pixel coordinates in the short-exposure image and the converted exposure image, then determine the larger value between IM - T1 and 0, divide the larger value by the pixel value at this pixel coordinate in the long-exposure image to obtain an intermediate detection result IM', then determine whether the intermediate detection result IM' is greater than T2. If so, set the intermediate detection result IM' to T2, otherwise keep it unchanged; generate a motion area mask image according to the motion detection results of each pixel coordinate; T1 is a preset first threshold, and T2 is a preset second threshold.

[0058] Among them, the processor generates a rectangular sliding window of a preset size. This sliding window will slide on the short-exposure image and the converted exposure image to traverse each pixel point. During the sliding process, the processor calculates the similarity IM between two rectangular sliding windows with the same pixel coordinates in the short-exposure image and the converted exposure image. The calculation method of the similarity can be based on indicators such as pixel value differences and structural similarities.

[0059] Next, the processor determines the larger value between IM - T1 and 0, and divides it by the pixel value at this pixel coordinate in the long-exposure image to obtain an intermediate detection result IM'. Here, T1 is a preset first threshold used to adjust the sensitivity of the similarity calculation.

[0060] Then, the processor determines whether IM' is greater than T2 (a preset second threshold). If IM' is greater than T2, set it to T2; otherwise, IM' remains unchanged. This process helps to highlight the motion area in the image while suppressing background noise.

[0061] Finally, according to the motion detection results of each pixel coordinate, the processor generates a motion area mask image. This mask image will highlight the motion area in the image and provide guidance for subsequent image fusion.

[0062] In a possible embodiment, calculate the root mean square error of the pixel blocks included in the two rectangular sliding windows, and use the calculation result as the similarity IM.

[0063] Among them, the processor slides the rectangular sliding window on the short-exposure image and the converted exposure image respectively. For each sliding window position, extract all the pixel values within the sliding window to form two pixel blocks (or vectors). For each sliding window position, the processor calculates the root mean square error between the two pixel blocks, and uses the root mean square error as the similarity IM:

[0064] In this example, since a smaller root mean square error indicates that two pixel blocks are more similar, we can directly use the root mean square error or its reciprocal (to avoid division by zero, a very small constant can be added before calculating the reciprocal) as the similarity IM. However, usually, to convert the similarity metric into a more intuitive range (such as 0 to 1), we can normalize the root mean square error.

[0065] There are various ways to perform the normalization process. For example: using the maximum root mean square error as the normalization factor, dividing the current root mean square error by the maximum root mean square error to obtain a value between 0 and 1. Using a fixed threshold, considering the case where the root mean square error is less than the threshold as similar (IM close to 1), and greater than the threshold as dissimilar (IM close to 0).

[0066] S106. Obtain a weight sequence image according to each pixel point in the motion area mask image after performing a dilation operation.

[0067] Among them, the processor performs a dilation operation on the motion area mask image. The purpose of the dilation operation is to expand the range of the motion area, so that the fused HDR image has a better visual effect in the motion area. The dilation operation can be achieved through morphological operations, such as dilating the image using a structuring element. After the dilation operation, the processor generates a weight sequence image according to the dilated image. This weight sequence image will be used to guide the fusion process of the long exposure image and the short exposure image.

[0068] In some embodiments of the present application, obtaining a weight sequence image according to each pixel point in the motion area mask image after performing a dilation operation includes:

[0069] Use a rectangular sliding window to slide in the motion area mask image to traverse each pixel point. During the traversal process, obtain the pixel values of all pixel points included in the rectangular sliding window where each pixel coordinate is located, and use the maximum pixel value as the pixel value of this pixel coordinate.

[0070] Among them, the processor first generates a rectangular sliding window according to a preset size. The size of this sliding window can be adjusted according to actual needs to control the degree of dilation. The processor places the rectangular sliding window at the upper left corner of the motion area mask image, and then gradually slides it to the right until the sliding window completely slides out of the right side of the image. After completing the traversal of one row, the sliding window moves to the starting position of the next row and repeats the above process until the sliding window completely slides out of the bottom of the image. In this way, the sliding window traverses each pixel point in the image.

[0071] When the sliding window slides to each pixel coordinate, the processor obtains the pixel values of all the pixel points included in the sliding window at that coordinate. These pixel values represent the motion intensity of the corresponding area in the motion area mask image. The processor compares the pixel values of all the pixel points within the sliding window and finds the maximum value among them. Then, this maximum value is used as the new pixel value for the current pixel coordinate. This step implements the dilation operation, that is, it enhances the intensity of the motion area, making the motion area more prominent in the weight sequence image. As the sliding window slides and the pixel values are updated, the processor gradually constructs a weight sequence image with the same size as the motion area mask image. Each pixel value in this image represents the result of the dilation operation on the motion intensity of the corresponding coordinate in the motion area mask image. The generated weight sequence image will be used to guide the fusion process of the long-exposure image and the short-exposure image. During fusion, the pixel values in the weight sequence image will be used as weights to perform weighted averaging on the luminance values of the corresponding pixels in the long-exposure image and the short-exposure image, thereby generating an HDR image with a higher dynamic range.

[0072] For example, referring to Figure 2 the schematic diagram of the dilation operation shown, the size of the sliding window is 3×3. Taking the central pixel point of the sliding window as the current pixel coordinate, the pixel values of the 9 pixel points included in the sliding window at this position are obtained. The magnitudes of the 9 pixel values are compared, and it is determined that the maximum value is B. The pixel value of the central pixel point is updated using the maximum value B.

[0073] Through the above steps, the processor can perform the dilation operation on the motion area mask image using the rectangular sliding window and generate a weight sequence image for guiding the HDR image fusion. This process not only enhances the intensity of the motion area but also provides important weight information for subsequent image fusion.

[0074] S107. Normalize each pixel point in the weight sequence image to obtain a normalized image. Then, using the pixel value of each pixel point in the normalized image as a weight, perform weighted averaging on the pixel points with the same pixel coordinates in the long-exposure image and the short-exposure image to obtain HDR pixel points. Generate an HDR image based on each generated HDR pixel point.

[0075] Among them, the processor normalizes each pixel point in the weight sequence image. The purpose of the normalization process is to limit the weight values within a certain range (such as between 0 and 1) for subsequent weighted averaging calculations.

[0076] Then, the processor uses the pixel value of each pixel point in the normalized image as a weight to perform a weighted average calculation on the pixel points with the same pixel coordinates in the long-exposure image and the short-exposure image. The result of the weighted average is the HDR pixel point. The formula for the weighted average is: HDR pixel point = weight * long-exposure image pixel point + (1 - weight) * short-exposure image pixel point.

[0077] Finally, the processor combines each generated HDR pixel point into a final HDR image. This HDR image will have a higher dynamic range and better visual effects, and can display both the bright and dark details in the image simultaneously.

[0078] In some embodiments of the present application, it further includes:

[0079] Performing tone mapping on the generated HDR image to obtain an LDR image, and displaying the LDR image on a display screen

[0080] Among them, tone mapping is a technology for converting a high dynamic range (HDR) image into a low dynamic range (LDR) image. HDR images usually contain a wider range of brightness than LDR images, so tone mapping is required to adjust the brightness and contrast of the HDR image to adapt to the limitations of LDR display devices.

[0081] There are various tone mapping algorithms, and common ones include linear mapping, piecewise mapping, and mapping based on specific visual effects, etc. Linear mapping is the simplest method, but it may not be able to well preserve the details and contrast in the HDR image. Piecewise mapping can divide the HDR image into different intervals according to the brightness range and apply different mapping functions to each interval to better preserve the image details. Mapping based on specific visual effects can adjust the tone and brightness of the HDR image according to specific visual effects (such as movie style, natural style, etc.).

[0082] In practical applications, usually a combination of one or more tone mapping algorithms is selected to achieve the best conversion effect. These algorithms can be implemented in image processing software or hardware and the conversion result can be optimized through parameter adjustment.

[0083] Once the HDR image is converted into an LDR image, it can be displayed on the display screen. Before display, it is necessary to ensure that the display device is compatible with the LDR image format and appropriate color correction and brightness adjustment have been performed. This can be achieved by adjusting the settings of the display device or using professional image calibration tools.

[0084] When displaying an LDR image, it is also necessary to consider whether the image resolution matches the resolution of the display device. If the image resolution is higher than the display device resolution, scaling or cropping may be required to fit the screen size. At the same time, it is also necessary to ensure that the color space of the display device matches the color space of the image to avoid color distortion or deviation.

[0085] In summary, tone mapping an HDR image to obtain an LDR image and displaying the LDR image on a display screen is a process involving multiple steps and considerations. By selecting an appropriate tone mapping algorithm and making appropriate display device settings, it is possible to ensure that the finally displayed LDR image has good visual effects and compatibility.

[0086] The embodiments of the present application specifically include the following beneficial effects:

[0087] By acquiring images with different exposure amounts and performing filtering processing, image noise is reduced, providing a higher-quality image basis for subsequent processing. By comparing the exposure amounts and calculating the exposure ratio, the pixel values of the long-exposure image are appropriately adjusted, which helps to reduce the ghosting phenomenon caused by exposure differences. At the same time, by traversing the image with a rectangular sliding window, calculating the similarity and performing threshold judgment, the moving area can be accurately identified and processed, further eliminating ghosting. By performing a dilation operation on the moving area mask image, a weight sequence image is obtained, which helps to enhance the edge information in the image and reduce the degradation of image quality caused by motion blur. Finally, by normalizing the weight sequence image and using the normalized pixel values as weights to perform weighted averaging on the long-exposure and short-exposure images, high-quality HDR pixel points are generated, thus significantly improving the overall quality of the HDR image. In summary, this technical solution effectively solves the problem of difficult elimination of ghosting and blurring phenomena in existing HDR fusion algorithms through innovative image processing methods and steps, and significantly improves the quality of HDR images.

[0088] The following is an embodiment of the device of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the device embodiment of the present application, please refer to the method embodiment of the present application.

[0089] Please refer to Figure 3 , which shows a schematic structural diagram of a device for generating an HDR image provided by an exemplary embodiment of the present application, hereinafter referred to as device 3. This device 3 can be implemented as all or part of a computer device through software, hardware, or a combination of both. Device 3 includes: an acquisition unit 301, a filtering unit 302, a comparison unit 303, a conversion unit 304, a detection unit 305, a dilation unit 306, and a weighting unit 307.

[0090] An acquisition unit 301, configured to acquire a first image and a second image collected by an image sensor, where the first image and the second image have different exposure amounts;

[0091] A filtering unit 302, configured to perform filtering processing on the first image and the second image;

[0092] A comparison unit 303, configured to compare the exposure amounts of the filtered first image and the filtered second image, use the image with a larger exposure amount as the long-exposure image, use the image with a smaller exposure amount as the short-exposure image, and calculate an exposure ratio R between the long-exposure image and the short-exposure image;

[0093] A conversion unit 304, configured to divide each pixel value of the long-exposure image by the exposure ratio to obtain a converted exposure image;

[0094] A detection unit 305, configured to generate a rectangular sliding window with a preset size, and use the rectangular sliding window to slide in the short-exposure image and the converted exposure image to traverse each pixel point; during the traversal process, calculate a similarity IM between two rectangular sliding windows with the same pixel coordinates in the short-exposure image and the converted exposure image, then determine the larger value between IM-T1 and 0, divide the larger value by the pixel value of the pixel coordinate in the long-exposure image to obtain an intermediate detection result IM', then determine whether the intermediate detection result IM' is greater than T2, if so, set the intermediate detection result IM' to T2, otherwise keep it unchanged; generate a motion region mask image according to the motion detection result of each pixel coordinate; T1 is a preset first threshold, and T2 is a preset second threshold;

[0095] An expansion unit 306, configured to perform an expansion operation according to each pixel point in the motion region mask image to obtain a weight sequence image;

[0096] A weighting unit 307, configured to perform normalization processing on each pixel point in the weight sequence image to obtain a normalized image, then use the pixel value of each pixel point in the normalized image as a weight, perform weighted averaging on the pixel points with the same pixel coordinates in the long-exposure image and the short-exposure image to obtain HDR pixel points, and generate an HDR image according to each generated HDR pixel point.

[0097] In one or more possible embodiments, the preset size is 3×3 or 5×5.

[0098] In one or more possible embodiments, the obtaining a weight sequence image by performing an expansion operation according to each pixel point in the motion region mask image includes:

[0099] Use a rectangular sliding window to slide in the masked image of the motion area to traverse each pixel point. During the traversal, obtain the pixel values of all pixel points included in the rectangular sliding window where each pixel coordinate is located, and use the maximum pixel value among them as the pixel value of this pixel coordinate.

[0100] In one or more possible embodiments, the first image and the second image are filtered by means of convolution.

[0101] In one or more possible embodiments, the first image and the second image are filtered using convolution kernels of the same size.

[0102] In one or more possible embodiments, calculate the root mean square error of the pixel blocks included in two rectangular sliding windows, and use the calculation result as the similarity IM.

[0103] In one or more possible embodiments, it further includes:

[0104] A mapping unit for performing tone mapping on the generated HDR image to obtain an LDR image and displaying the LDR image on a display screen.

[0105] It should be noted that when the device 3 provided in the above embodiment executes the method for generating an HDR image, only the above division of each functional module is used for illustration. In practical applications, the above functions can be assigned to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above functions. In addition, the HDR image generation device provided in the above embodiment and the embodiment of the HDR image generation method belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0106] The serial numbers of the above embodiments of the present application are only for description and do not represent the superiority or inferiority of the embodiments.

[0107] The embodiments of the present application also provide a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor to perform the method steps of the embodiments as described above Figure 1 as shown, and the specific execution process can refer to Figure 1 the specific description of the embodiments as shown, and will not be repeated here.

[0108] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method for generating an HDR image as described in each of the above embodiments.

[0109] Please refer to Figure 4 , which is a schematic structural diagram of a computer device provided by the embodiments of the present application. AsFigure 4 As shown in the figure, the computer device 400 may include: at least one processor 401, at least one network interface 404, a user interface 403, a memory 405, and at least one communication bus 402.

[0110] Among them, the communication bus 402 is used to realize the connection and communication between these components.

[0111] Among them, the user interface 403 may include a display screen and a camera. Optionally, the user interface 403 may further include a standard wired interface and a wireless interface.

[0112] Among them, the network interface 404 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0113] Among them, the processor 401 may include one or more processing cores. The processor 401 connects various parts within the entire computer device 400 through various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 405, and by calling the data stored in the memory 405, the processor 401 executes various functions of the computer device 400 and processes data. Optionally, the processor 401 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 401 may integrate one or a combination of several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, the user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 401 and may be implemented separately by a single chip.

[0114] Among them, the memory 405 may include a Random Access Memory (RAM), or may also include a Read-Only Memory. Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above method embodiments, etc.; the data storage area may store data involved in the above method embodiments. Optionally, the memory 405 may also be at least one storage device located far from the aforementioned processor 401. As Figure 4 shown, the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module, and application programs.

[0115] In Figure 4 the computer device 400 shown, the user interface 403 is mainly used to provide an input interface for the user to obtain user input data; and the processor 401 can be used to call the application programs stored in the memory 405 and specifically execute as Figure 2 shown in the method, and the specific process can be referred to Figure 1 shown, and will not be elaborated here.

[0116] Those of ordinary skill in the art can understand that all or part of the processes of implementing the above method embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory or a random access memory, etc.

[0117] The foregoing disclosure is only for the preferred embodiments of the present application. Of course, the scope of the rights of the present application cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A method for generating an HDR image, characterized in that: include: Acquire a first image and a second image captured by an image sensor, wherein the first image and the second image have different exposure amounts; Performing filtering on the first image and the second image; comparing the exposure amounts of the filtered first image and the filtered second image, taking the image with the larger exposure amount as the long-exposure image, taking the image with the smaller exposure amount as the short-exposure image, and calculating an exposure ratio R between the long-exposure image and the short-exposure image; Dividing each pixel value of the long exposure image by the exposure ratio to obtain a converted exposure image; Generate a rectangular sliding window of a preset size, and use the rectangular sliding window to slide in the short exposure image and the converted exposure image to traverse each pixel point; During the traversal process, the similarity IM between the two rectangular sliding windows of the same pixel coordinates in the short exposure image and the converted exposure image is calculated, and then the larger one between IM-T1 and 0 is determined, and the larger one is divided by the pixel value of the pixel coordinate in the long exposure image to obtain the intermediate detection result IM', and then it is determined whether the intermediate detection result IM' is greater than T2, if so, the intermediate detection result IM' is set to T2, otherwise it remains unchanged; generate a motion area mask image according to the calculated motion detection result of each pixel coordinate; T1 is a preset first threshold, and T2 is a preset second threshold; A weighted sequence image is obtained by performing a dilation operation on each pixel in the motion region mask image; Each pixel in the weight sequence image is normalized to obtain a normalized image, and then the pixel value of each pixel in the normalized image is used as a weight to perform weighted averaging on the pixels with the same pixel coordinates in the long exposure image and the short exposure image to obtain an HDR pixel, and an HDR image is generated according to each generated HDR pixel.

2. The method according to claim 1, characterized in that The preset size is 3×3 or 5×5.

3. The method according to claim 1 or 2, characterized in that: The step of obtaining a weighted sequence image after performing a dilation operation on each pixel in the motion region mask image comprises: A rectangular sliding window is used to slide in the motion area mask image to traverse each pixel point. During the traversal process, the pixel values ​​of all the pixel points contained in the rectangular sliding window where each pixel coordinate is located are obtained, and the largest pixel value is used as the pixel value of the pixel coordinate.

4. The method according to claim 3, characterized in that The first image and the second image are filtered by using a convolution method.

5. The method according to claim 4, characterized in that: The first image and the second image are filtered using convolution kernels of the same size.

6. The method according to claim 1 or 2 or 4 or 5, characterized in that: The root mean square error of the pixel blocks contained in the two rectangular sliding windows is calculated, and the calculation result is used as the similarity IM.

7. The method according to claim 6, characterized in that Also includes: The generated HDR image is tone mapped to obtain an LDR image, and the LDR image is displayed on a display screen.

8. A device for generating an HDR image, characterized in that: include: An acquisition unit, configured to acquire a first image and a second image captured by an image sensor, wherein the first image and the second image have different exposure amounts; A filtering unit, configured to perform filtering processing on the first image and the second image; a comparing unit, configured to compare the exposure amounts of the filtered first image and the filtered second image, use the image with the larger exposure amount as the long-exposure image, use the image with the smaller exposure amount as the short-exposure image, and calculate an exposure ratio R between the long-exposure image and the short-exposure image; a conversion unit, configured to obtain a converted exposure image by dividing each pixel value of the long exposure image by the exposure ratio; A detection unit, configured to generate a rectangular sliding window of a preset size, and use the rectangular sliding window to slide in the short exposure image and the converted exposure image to traverse each pixel point; During the traversal process, the similarity IM between the two rectangular sliding windows of the same pixel coordinates in the short exposure image and the converted exposure image is calculated, and then the larger one between IM-T1 and 0 is determined, and the larger one is divided by the pixel value of the pixel coordinate in the long exposure image to obtain the intermediate detection result IM', and then it is determined whether the intermediate detection result IM' is greater than T2, if so, the intermediate detection result IM' is set to T2, otherwise it remains unchanged; generate a motion area mask image according to the calculated motion detection result of each pixel coordinate; T1 is a preset first threshold, and T2 is a preset second threshold; A dilation unit, used for obtaining a weighted sequence image after dilation operation is performed on each pixel in the motion region mask image; A weighting unit is used to normalize each pixel point in the weight sequence image to obtain a normalized image, then use the pixel value of each pixel point in the normalized image as a weight, perform weighted averaging on the pixel points with the same pixel coordinates in the long exposure image and the short exposure image to obtain an HDR pixel point, and generate an HDR image according to each generated HDR pixel point.

9. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 7.

10. A computer device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 7.

Images