A method, device, product and medium for generating HDR images

Abstract

The present application provides an HDR image generation method, device, product and medium, which are applied to the field of image processing technology to solve the problems of large computational complexity and low efficiency in HDR image imaging, specifically: obtaining at least two LDR image frames with different exposure times; downsampling the at least two LDR image frames at different resolution levels respectively to obtain downsampled images of the at least two LDR image frames at each resolution level; for each resolution level, determining a fusion weight of the downsampled images of the at least two LDR image frames at the resolution level based on a pre-stored fusion weight dictionary, and fusing the downsampled images of the at least two LDR image frames at the resolution level based on the fusion weight to obtain a fused image at the resolution level; and reconstructing multiple fused images of different resolution levels to obtain an HDR image, thereby reducing the computational complexity of HDR image imaging and improving the HDR image imaging efficiency.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, product and medium for generating HDR images. Background Art

[0002] In practical applications, due to the limited dynamic range covered by traditional image sensors, the images captured are mostly low dynamic range (LDR) images. In order to meet the requirements of image brightness and details, high dynamic range (HDR) images have emerged.

[0003] At present, HDR images can be generated based on single-frame LDR images or multi-frame LDR images. Compared with the HDR image generation method based on single-frame LDR images, the HDR image generation method based on multi-frame LDR images has more ideal imaging effect and quality. However, the existing HDR image generation method based on multi-frame LDR images has a large amount of calculation and low efficiency, and cannot be applied in real time on electronic devices with relatively weak computing power. Summary of the Invention

[0004] The embodiments of the present application provide a HDR image generation method, device, product, and medium to solve the problems of high computational complexity and low efficiency in HDR image imaging in the prior art.

[0005] The technical solutions provided in the embodiments of this application are as follows:

[0006] In one aspect, an embodiment of the present application provides a method for generating an HDR image, comprising:

[0007] Acquire at least two LDR images with different exposure times;

[0008] Downsampling the at least two LDR images at different resolution levels is performed respectively to obtain downsampled images of the at least two LDR images at each resolution level;

[0009] For each resolution level, determining fusion weights corresponding to downsampled images of at least two LDR images at the resolution level based on a pre-stored fusion weight dictionary, and fusing the downsampled images of at least two LDR images at the resolution level based on the fusion weights to obtain a fused image at the resolution level;

[0010] Reconstruct multiple fused images of different resolution levels to obtain an HDR image.

[0011] On the other hand, an embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the HDR image generation method provided in the embodiment of the present application is implemented.

[0012] On the other hand, an embodiment of the present application provides a computer program product, which includes program code. When the program code is run on a processor, it implements the HDR image generation method provided by the embodiment of the present application.

[0013] On the other hand, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the HDR image generation method provided by the embodiment of the present application is implemented.

[0014] The beneficial effects of the embodiments of the present application are as follows:

[0015] In an embodiment of the present application, by downsampling at different resolution levels of at least two frames of LDR images and then reconstructing multiple fused images of different resolution levels, it is possible to reduce the amount of calculation in the image fusion process while meeting the imaging quality of HDR images, thereby improving the imaging efficiency of HDR images. Moreover, in the image fusion process, by using a pre-saved fusion weight dictionary to obtain the fusion weight, a large number of real-time calculations can be reduced, further improving the imaging efficiency of HDR images, and thus achieving real-time applications on electronic devices with relatively weak computing power.

[0016] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description or be understood by practicing the present application. The purposes and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A schematic diagram of an overview of the HDR image generation method provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of an overview of the fusion weight dictionary generation method provided in an embodiment of the present application;

[0020] Figure 3a A schematic diagram of the structure of a neural network model provided in an embodiment of the present application;

[0021] Figure 3b A schematic diagram of the structure of the encoder module of the neural network model provided in an embodiment of the present application;

[0022] Figure 3c A schematic diagram of the structure of a decoder module of a neural network model provided in an embodiment of the present application;

[0023] Figure 3d A schematic diagram of the structure of the attention mechanism module of the neural network model provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of an overview of the neural network model training method provided in an embodiment of the present application;

[0025] Figure 5 A schematic diagram of the functional structure of the HDR image generation device provided in an embodiment of the present application;

[0026] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and beneficial effects of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0028] In order to facilitate those skilled in the art to better understand this application, the technical terms involved in this application are briefly introduced below.

[0029] Resolution levels are image layers corresponding to different resolutions in the image pyramid.

[0030] A pixel value combination is a combination of at least two pixel values obtained by permuting and combining pixel values within a set range (eg, 0-255) according to the constraint that at least two pixel values constitute a combination.

[0031] The fusion weight dictionary is a dictionary containing fusion weights corresponding to different pixel value combinations. In an embodiment of the present application, the fusion weight corresponding to each pixel value combination contained in the fusion weight dictionary can be a weight value or a weight ratio. For example, the fusion weight corresponding to the pixel value combination (10, 255) can be (0.2, 0.8), indicating that the weight value of the pixel value 10 is 0.2, and the weight value corresponding to the pixel value 255 is 0.8; for example, the fusion weight corresponding to the pixel value combination (10, 255) can also be (1 / 5, 4 / 5), indicating that the weight ratio of the pixel value 10 is 1 / 5, and the weight ratio of the pixel value 255 is 4 / 5. The weight values of the pixel value 10 and the pixel value 255 can be calculated according to their respective weight ratios. In another embodiment, the fusion weight corresponding to the pixel value combination (10, 255) can also be 0.8, which represents the fusion weight of one of the pixel values 10 and 255. The fusion weight of the other pixel value can be calculated based on the fusion weight in the fusion weight dictionary. For example, the fusion weight of the pixel value 10 is 0.8, and the fusion weight of the pixel value 255 is 1-0.8=0.2.

[0032] The neural network model is a model for determining fusion weights for different pixel value combinations obtained by training an initial neural network model based on multiple sample image combinations including at least two sample image frames and standard weight images corresponding to the multiple sample image combinations. In the embodiments of the present application, the neural network model can be, but is not limited to, a convolutional neural network model, a recurrent neural network model, or the like.

[0033] After introducing the technical terms involved in this application, the design concept of the embodiments of this application is briefly introduced.

[0034] In recent years, significant progress has been made in AI-based research on computer vision, deep learning, machine learning, image processing, and image recognition. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems for simulating and extending human intelligence. AI is a comprehensive discipline encompassing numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, a key branch of AI, specifically enables machines to understand the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robotic navigation and positioning. With the research and advancement of artificial intelligence technology, this technology has been applied in many fields, such as security, urban management, traffic management, building management, park management, facial access, facial attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, autonomous driving, smart medical care, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live streaming, beauty, makeup, medical beauty, smart temperature measurement and other fields.

[0035] Traditional HDR image generation methods also apply artificial intelligence technology, such as single-frame LDR image enhancement algorithms based on neural network models. However, these traditional HDR image generation methods have a large amount of computation and low imaging efficiency, and cannot be applied in real time on electronic devices with relatively weak computing power, such as mobile phones. To this end, in an embodiment of the present application, after obtaining fusion weights of different pixel value combinations through a neural network model in advance and forming a fusion weight dictionary based on the fusion weights of different pixel value combinations, in the HDR image generation process, at least two LDR images to be fused are first downsampled at different resolution levels to obtain the downsampled images of the at least two LDR images at each resolution level. Then, for each resolution level, based on the fusion weight dictionary, the fusion weight corresponding to the downsampled images of the at least two LDR images at that resolution level is determined, and based on the fusion weight, the downsampled images of the at least two LDR images at that resolution level are fused to obtain a fused image at that resolution level. Finally, the fused images of the multiple different resolution levels are reconstructed to obtain an HDR image. In this way, by downsampling at different resolution levels of at least two frames of LDR images and then reconstructing multiple fused images at different resolution levels, it is possible to reduce the amount of calculation in the image fusion process while meeting the imaging quality of HDR images, thereby improving the imaging efficiency of HDR images. Moreover, in the image fusion process, by using a pre-saved fusion weight dictionary to obtain the fusion weight, a large number of real-time calculations can be reduced, further improving the imaging efficiency of HDR images, and thus achieving real-time application on electronic devices with relatively weak computing power.

[0036] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described in detail below.

[0037] The present invention provides an HDR image generation method. The HDR image generation method can be applied to electronic devices with relatively weak computing power, such as mobile phones, notebooks, personal digital assistants (PDAs), smart TVs, etc., and can also be applied to electronic devices with relatively strong computing power, such as computers, virtual machines, servers, etc. Figure 1 As shown, the overview process of the HDR image generation method provided in the embodiment of the present application is as follows:

[0038] Step 110: The electronic device acquires at least two frames of LDR images with different exposure times.

[0039] In practical applications, the electronic device may capture at least two frames of images with different exposure times through a camera, or may obtain at least two frames of LDR images with different exposure times that are pre-stored.

[0040] Step 120: The electronic device performs downsampling processing on the at least two LDR image frames at different resolution levels to obtain downsampled images of the at least two LDR image frames at each resolution level.

[0041] In practical applications, the electronic device may iteratively perform downsampling processing on each of the at least two LDR image frames at different resolution levels on the LDR image, thereby obtaining a downsampled image of the LDR image at each resolution level. For example, the electronic device may iteratively perform Gaussian pyramid downsampling processing on each of the at least two LDR image frames at different resolution levels using the following formula (1), thereby obtaining a downsampled image of the LDR image at each resolution level.

[0042]

[0043] Among them, G l Represents the downsampled image of the lth resolution level, G l-1 represents the downsampled image of the l-1th resolution level, (i, j) represents the pixel points in the downsampled image of the lth resolution level, N represents the number of resolution levels, R l Represents the number of pixel rows in the downsampled image at the lth resolution level, C l Represents the number of pixel columns in the downsampled image of the lth resolution level, w(m,n) represents the window function, m represents the window length of the window function, and n represents the window width of the window function.

[0044] Step 130: For each resolution level, the electronic device determines the fusion weights corresponding to the downsampled images of the at least two LDR images at the resolution level based on a pre-saved fusion weight dictionary, and fuses the downsampled images of the at least two LDR images at the resolution level based on the fusion weights to obtain a fused image at the resolution level.

[0045] In practical applications, the fusion weight dictionary includes fusion weights for different pixel value combinations, and each pixel value combination includes at least two pixel values. Based on this, the electronic device can use, but is not limited to, the following methods when determining, for each resolution level, the fusion weights corresponding to the downsampled images of the at least two LDR images at the resolution level based on the pre-stored fusion weight dictionary:

[0046] First, the electronic device determines each target pixel value combination of the downsampled images of the at least two frames of LDR images at the resolution level; wherein each target pixel value combination includes pixel values of at least two pixel points, and the at least two pixel points respectively belong to the downsampled images of the at least two frames of LDR images at the resolution level and the positions of the at least two pixel points in the downsampled images of the at least two frames of LDR images at the resolution level have a corresponding relationship.

[0047] Then, the electronic device determines the fusion weight of each target pixel value combination based on the fusion weights of different pixel value combinations contained in the fusion weight dictionary.

[0048] Finally, the electronic device determines the fusion weights of the combinations of target pixel values as the fusion weights of the downsampled images of the at least two frames of LDR images at the resolution level.

[0049] Furthermore, the electronic device can determine, for each resolution level, based on a pre-stored fusion weight dictionary, the fusion weights corresponding to the downsampled images of the at least two LDR images at the resolution level, and then fuse the downsampled images of the at least two LDR images at the resolution level based on the fusion weights. Specifically, the electronic device can perform pixel-by-pixel weighted fusion on the downsampled images of the at least two LDR images at the resolution level based on the fusion weights, thereby obtaining a fused image at the resolution level. For example, taking two LDR images as an example, the electronic device can use the following formula (2) for each resolution level to perform pixel-by-pixel weighted fusion on the downsampled images of the two LDR images at the resolution level, thereby obtaining a fused image at the resolution level.

[0050] G l12 (i,j)=G l1 (i,j)*mask+G l2 (i,j)*(1-mask)1≤l12≤N,1≤l1≤N,1≤l2≤N……Formula (2)

[0051] Among them, G l12 Represents the fused image of the lth resolution level, G l1 Represents the downsampled image of a frame of LDR image at the lth resolution level, mask represents the fusion weight of the downsampled image of the frame of LDR image at the lth resolution level, G l2 represents the downsampled image of another frame of LDR image at the lth resolution level, 1-mask represents the fusion weight of the downsampled image of another frame of LDR image at the lth resolution level, (i, j) represents the pixel points in the downsampled image, and N represents the number of resolution levels.

[0052] Step 140: The electronic device reconstructs the fused images of multiple different resolution levels to obtain an HDR image.

[0053] In practical applications, the electronic device may iteratively perform upsampling processing on the fused images of each resolution level in the order of resolution corresponding to each resolution level from low to high, thereby obtaining an HDR image. For example, the electronic device may iteratively perform Laplace upsampling processing on the fused images of each resolution level in the order of resolution corresponding to each resolution level from low to high, thereby obtaining an HDR image.

[0054]

[0055] Among them, L i Characterize HDR image, G i Represents the fused image of the i-th resolution layer, G i+1 Represents the fused image of the i+1th resolution layer, Up represents the upsampling function, Representation convolution, k 5×5 Represents a 5×5 convolution kernel.

[0056] In the embodiment of the present application, in order to reduce the real-time calculation in the process of HDR image generation, the fusion weight dictionary is pre-generated based on the fusion weights of different pixel value combinations. Next, the generation method of the fusion weight dictionary provided in the embodiment of the present application is briefly introduced. Figure 2 As shown, the general process of the method for generating a fusion weight dictionary provided in the embodiment of the present application is as follows:

[0057] Step 210: The electronic device inputs at least two frames of monochrome images with different exposure times corresponding to each pixel value combination into a neural network model to obtain a fusion weight of each pixel value combination; wherein, the at least two frames of monochrome images with different exposure times corresponding to each pixel value combination are monochrome images based on at least two pixel values contained in the pixel value combination, and the exposure times of the at least two pixel value monochrome images are different.

[0058] In actual applications, the electronic device can input at least two frames of monochrome images with different exposure times corresponding to each pixel value combination into the neural network model, obtain the weight image of each pixel value combination, and then determine the fusion weight of each pixel value combination based on the pixel value mean of the weight image of each pixel value combination.

[0059] Step 220: The electronic device assembles fusion weights of multiple pixel value combinations into a fusion weight dictionary.

[0060] In the above implementation process, the fusion weights corresponding to multiple pixel value combinations are obtained through the neural network model, which can make the obtained fusion weights more robust and have a wider range of applications. In addition, after the fusion weights based on the multiple pixel value combinations form a fusion weight dictionary, by pre-storing the fusion weight dictionary in an electronic device (such as a mobile terminal such as a mobile phone), the electronic device can obtain the fusion weight without running the neural network model, thereby reducing the amount of calculation of the electronic device, which is more friendly to electronic devices with relatively weak computing power.

[0061] In the embodiment of the present application, the neural network model used to obtain the fusion weights of each pixel value combination can be a convolutional neural network model using a codec and attention mechanism, see Figure 3a As shown, the neural network model includes: a first convolutional layer, a pooling layer, a first encoding module, a second encoding module, a third encoding module, a first decoding module, a second decoding module, and a second convolutional layer connected in sequence; and a first attention mechanism module is connected between the first encoding module and the second decoding module, and a second attention mechanism module is connected between the second encoding module and the first decoding module; wherein, referring to Figure 3b As shown, each encoding module in the first encoding module, the second encoding module and the third encoding module includes: a first 3×3 Relu convolution layer, a second 3×3 Relu convolution layer and a 2×2 maximum pooling layer connected in sequence; see Figure 3c As shown, each decoding module in the first decoding module and the second decoding module includes: a 2×2 upsampling convolution layer, a first 3×3 Relu convolution layer and a second 3×3 Relu convolution layer connected in sequence; see Figure 3d As shown, each attention mechanism module in the first attention mechanism module and the second attention mechanism module includes: a first 1×1 convolution layer, a BN batch processing layer, a second 1×1 convolution layer and a sigmoid function layer connected in sequence.

[0062] In view of the above neural network model, the present application embodiment proposes a training method for the neural network model, see Figure 4 As shown, the general process of the training method of the neural network model provided in the embodiment of the present application is as follows:

[0063] Step 410: Acquire multiple sample image combinations and determine standard fusion weight images of the multiple sample image combinations; wherein each sample image combination includes at least two frames of sample images.

[0064] In actual applications, the electronic device can first determine, for each sample image combination in multiple sample image combinations, a standard HDR image obtained by fusing at least two frames of sample images included in the sample image combination and each sample pixel value combination corresponding to at least two frames of sample images included in the sample image combination, and then determine the standard fusion weight of each sample pixel value combination based on the pixel values of the pixel points corresponding to each sample pixel value combination in the standard HDR image, and determine the standard fusion weight image of the sample image combination based on the standard fusion weight of each sample pixel value combination.

[0065] Step 420: Training the initial neural network model based on the multiple sample image combinations and the standard fusion weight images of the multiple sample image combinations.

[0066] In practical applications, electronic devices can use a combination of multiple sample images as training images for the initial neural network model, and use a standard fusion weight image of the combination of multiple sample images as a label image in the training process of the initial neural network model, so as to iteratively train the initial neural network model to adjust the various model parameters of the initial neural network model.

[0067] Step 430: When it is determined that the training end condition is met, a neural network model is obtained.

[0068] In actual applications, the electronic device can determine that the training end conditions are met when it is determined that the loss value calculated based on the loss function is less than a set threshold or the number of iterative training reaches a set number, and obtain a neural network model based on the model parameters obtained from the last iterative training.

[0069] Based on the same inventive concept, the present application also provides an HDR image generation device, see Figure 5 As shown, the HDR image generation device 500 provided in the embodiment of the present application includes at least:

[0070] An image acquisition unit 501 is configured to acquire at least two LDR image frames with different exposure times;

[0071] a downsampling unit 502 for performing downsampling processing on the at least two LDR image frames at different resolution levels, to obtain downsampled images of the at least two LDR image frames at each resolution level;

[0072] An image fusion unit 503 is configured to determine, for each resolution level, a fusion weight corresponding to the downsampled images of at least two LDR images at the resolution level based on a pre-stored fusion weight dictionary, and fuse the downsampled images of at least two LDR images at the resolution level based on the fusion weight to obtain a fused image at the resolution level;

[0073] The image reconstruction unit 504 is used to reconstruct multiple fused images of different resolution levels to obtain an HDR image.

[0074] In one possible implementation, the fusion weight dictionary includes fusion weights for different pixel value combinations, and each pixel value combination includes at least two pixel values;

[0075] When determining the fusion weights corresponding to the downsampled images of at least two LDR images at the resolution level based on the pre-stored fusion weight dictionary, the image fusion unit 503 is specifically configured to:

[0076] Determining target pixel value combinations of at least two LDR image frames at the downsampled images of the resolution level; wherein each target pixel value combination includes pixel values of at least two pixels, the at least two pixels respectively belonging to the at least two LDR image frames at the downsampled images of the resolution level, and positions of the at least two pixels in the at least two LDR image frames at the downsampled images of the resolution level have a corresponding relationship;

[0077] Determining the fusion weight of each target pixel value combination based on the fusion weights of different pixel value combinations contained in the fusion weight dictionary;

[0078] The fusion weight of each target pixel value combination is determined as the fusion weight of the downsampled image of at least two frames of LDR images at the resolution level.

[0079] In a possible implementation, when fusing downsampled images of at least two LDR images at a resolution level based on the fusion weight to obtain a fused image at the resolution level, the image fusion unit 503 is specifically configured to:

[0080] Based on the fusion weights, pixel-by-pixel weighted fusion is performed on the downsampled images of at least two frames of LDR images at the resolution level to obtain a fused image at the resolution level.

[0081] In a possible implementation, when downsampling at different resolution levels is performed on at least two LDR images to obtain downsampled images of at least two LDR images at each resolution level, the downsampling unit 502 is specifically configured to:

[0082] For each LDR image of the at least two LDR images, downsampling processing is iteratively performed on the LDR image at different resolution levels to obtain a downsampled image of the LDR image at each resolution level.

[0083] In a possible implementation, when reconstructing a plurality of fused images of different resolution layers to obtain an HDR image, the image reconstruction unit 504 is specifically configured to:

[0084] The fused images of each resolution level are iteratively upsampled in the order of resolution from low to high to obtain an HDR image.

[0085] In one possible implementation, the HDR image generation device 500 provided in the embodiment of the present application further includes:

[0086] The dictionary generation unit 505 is used to input at least two frames of monochrome images with different exposure times corresponding to each pixel value combination into the neural network model, obtain the fusion weight of each pixel value combination, and form a fusion weight dictionary with the fusion weights of multiple pixel value combinations; wherein, the at least two frames of monochrome images with different exposure times corresponding to each pixel value combination are monochrome images based on the at least two pixel values contained in the pixel value combination, and the exposure times of the at least two pixel value monochrome images are different.

[0087] In one possible implementation, when at least two frames of monochrome images with different exposure times corresponding to each pixel value combination are input into the neural network model to obtain the fusion weight of each pixel value combination, the dictionary generation unit 505 is specifically configured to:

[0088] Inputting at least two frames of monochrome images with different exposure times corresponding to each pixel value combination into a neural network model to obtain a weighted image of each pixel value combination;

[0089] The fusion weight of each pixel value combination is determined based on the pixel value mean of the weight image of each pixel value combination.

[0090] It should be noted that since the principle of solving the technical problem by the HDR image generation device 500 provided in the embodiment of the present application is similar to the HDR image generation method provided in the embodiment of the present application, the implementation of the HDR image generation device 500 provided in the embodiment of the present application can refer to the implementation of the HDR image generation method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0091] After introducing the HDR image generation method and apparatus provided in the embodiments of the present application, the electronic device provided in the embodiments of the present application is briefly introduced.

[0092] See Figure 6 As shown, the electronic device 600 provided in the embodiment of the present application includes at least: a processor 601, a memory 602, and a computer program stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program, the HDR image generation method provided in the embodiment of the present application is implemented.

[0093] The electronic device 600 provided in the embodiment of the present application may further include a bus 603 connecting different components (including the processor 601 and the memory 602). The bus 603 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0094] The memory 602 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 6021 and / or a cache memory 6022 , and may further include a read-only memory (ROM) 6023 .

[0095] The memory 602 may also include a program tool 6025 having a set (at least one) of program modules 6024, including but not limited to: an operating subsystem, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0096] The electronic device 600 may also communicate with one or more external devices 604 (e.g., keyboards, remote controls, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 600 (e.g., mobile phones, computers, etc.), and / or any device that enables the electronic device 600 to communicate with one or more other electronic devices 600 (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 605. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 606. Figure 6 As shown, the network adapter 606 communicates with other modules of the electronic device 600 via the bus 603. Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0097] It should be noted that Figure 6 The electronic device 600 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0098] In addition, embodiments of the present application further provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the HDR image generation method provided in embodiments of the present application. Specifically, the computer instructions may be embedded in the processor, so that the processor can implement the HDR image generation method provided in embodiments of the present application by executing the embedded computer instructions.

[0099] In addition, the HDR image generation method provided in the embodiment of the present application can also be implemented as a computer program product, which includes program code. When the program code is run on a processor, it implements the HDR image generation method provided in the embodiment of the present application.

[0100] The computer program product provided in the embodiments of the present application may adopt any combination of one or more readable media, wherein the readable medium may be a readable signal medium or a readable storage medium, and the readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0101] The computer program product provided in the embodiments of the present application may be a CD-ROM and include program code, and may also be run on electronic devices such as mobile phones, PDAs, smart TVs, computers, servers, etc. However, the computer program product provided in the embodiments of the present application is not limited thereto. In the embodiments of the present application, the readable storage medium may be any tangible medium that contains or stores program code, and the program code may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0102] It should be noted that although several units or subunits of the device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, depending on the embodiment of the application, the features and functions of two or more units described above can be embodied in a single unit. Conversely, the features and functions of a single unit described above can be further divided and embodied by multiple units.

[0103] Furthermore, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all illustrated operations must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0104] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0105] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include such modifications and variations.

Claims

1. A method for generating an HDR image, characterized in that: include: Acquire at least two LDR images with different exposure times; performing downsampling processing on the at least two LDR image frames at different resolution levels respectively, to obtain downsampled images of the at least two LDR image frames at each resolution level; For each resolution level, determining, based on a pre-stored fusion weight dictionary, a fusion weight corresponding to a downsampled image of the at least two LDR images at the resolution level, and fusing the downsampled images of the at least two LDR images at the resolution level based on the fusion weight to obtain a fused image at the resolution level; wherein the fusion weight dictionary includes fusion weights for different pixel value combinations, each pixel value combination including at least two pixel values; Reconstruct multiple fused images of different resolution levels to obtain an HDR image.

2. The HDR image generation method according to claim 1, wherein: The fusion weight dictionary includes fusion weights of different pixel value combinations, each pixel value combination includes at least two pixel values; Determining, based on a pre-stored fusion weight dictionary, fusion weights corresponding to downsampled images of the at least two LDR images at the resolution level, including: determining target pixel value combinations of the at least two LDR images at the resolution level, wherein each target pixel value combination includes pixel values of at least two pixels, the at least two pixels respectively belonging to the at least two LDR images at the resolution level, and positions of the at least two pixels in the at least two LDR images at the resolution level have a corresponding relationship; Determining the fusion weight of each target pixel value combination based on the fusion weights of different pixel value combinations included in the fusion weight dictionary; The fusion weights of the respective target pixel value combinations are determined as the fusion weights of the downsampled images of the at least two frames of LDR images at the resolution level.

3. The HDR image generation method according to claim 2, wherein: The method further comprises fusing the downsampled images of the at least two LDR images at the resolution level based on the fusion weight to obtain a fused image at the resolution level, comprising: Based on the fusion weights, pixel-by-pixel weighted fusion is performed on the downsampled images of the at least two frames of LDR images at the resolution level to obtain a fused image at the resolution level.

4. The HDR image generation method according to any one of claims 1 to 3, wherein: Downsampling the at least two LDR images at different resolution levels to obtain downsampled images of the at least two LDR images at each resolution level, comprising: For each LDR image of the at least two LDR images, downsampling processing at different resolution levels is iteratively performed on the LDR image to obtain a downsampled image of the LDR image at each resolution level.

5. The HDR image generation method according to any one of claims 1 to 3, wherein: Reconstruct multiple fused images at different resolution levels to obtain an HDR image, including: The fused images of each resolution level are iteratively upsampled in order from low to high resolutions corresponding to each resolution level to obtain the HDR image.

6. The HDR image generation method according to any one of claims 1 to 5, wherein: The method for generating the fusion weight dictionary includes: At least two frames of monochrome images with different exposure times corresponding to each pixel value combination are input into a neural network model to obtain the fusion weight of each pixel value combination, and the fusion weights of multiple pixel value combinations are combined into a fusion weight dictionary; wherein, the at least two frames of monochrome images with different exposure times corresponding to each pixel value combination are monochrome images based on at least two pixel values contained in the pixel value combination, and the exposure times of the at least two pixel value monochrome images are different.

7. The HDR image generation method according to claim 6, wherein: Inputting at least two frames of monochrome images with different exposure times corresponding to each pixel value combination into a neural network model to obtain a fusion weight of each pixel value combination, including: Inputting at least two frames of monochrome images with different exposure times corresponding to each pixel value combination into the neural network model to obtain a weighted image of each pixel value combination; Based on the pixel value mean of the weight image of each pixel value combination, the fusion weight of each pixel value combination is determined.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the HDR image generation method according to any one of claims 1 to 7 is implemented.

9. A computer program product, characterized in that The computer program product comprises program code, and when the program code is run on a processor, the method for generating an HDR image according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the HDR image generation method according to any one of claims 1 to 7 is implemented.

Images