Application of image overlay on HDR images

Abstract

Embodiments of the invention relate to applying image overlay on HDR images. A computing system for applying image overlay on a high dynamic range (HDR) image is provided. The system receives an HDR image, generates a first gain map and a standard dynamic range (SDR) image based on the received HDR image, renders at least one overlay image to generate an SDR effect image buffer and an effect gain map, overlays the effect gain map and the first gain map to generate a second gain map, and outputs the second gain map to the HDR image buffer. The SDR image and the SDR effect image buffer are superimposed to generate an edited SDR image, an edited HDR image is generated based on the edited SDR image and the second gain map, and an output is generated based on the edited HDR image.

Description

Technical Field

[0001] Various embodiments of this disclosure relate to applying image overlay on HDR images. Background Technology

[0002] Advances in imaging technology have significantly improved visual quality, particularly with the advent of High Dynamic Range (HDR) photography. Compared to Standard Dynamic Range (SDR) photography, HDR photography offers a wider range of brightness and color depth. For example, HDR is characterized by brighter whites, darker blacks, and a potentially larger amount of visible colors, resulting in more vivid and realistic images. This increased color depth and expanded dynamic range make HDR superior in providing a more immersive visual experience, especially when compared to SDR photography, which operates within a more limited color gamut and narrower range of brightness levels.

[0003] With the increasing adoption of HDR-enabled cameras and displays (especially in mobile devices), modern smartphones, tablets, and cameras now typically support HDR photos, bringing a professional-grade viewing experience to the consumer market.

[0004] However, the widespread adoption of HDR photography has also presented challenges to applying image overlay effects designed for SDR photos onto HDR images. Applications and software originally developed for SDR photos are often not optimized to handle the increased color depth and dynamic range of HDR content. For example, when an image designed to be overlaid on an SDR photo is overlaid on an HDR photo, the mismatch between the two formats can lead to severe color distortion. This distortion may manifest as oversaturation or desaturation in certain color spaces, resulting in an unnatural or undesirable photo appearance. Summary of the Invention

[0005] In view of the above, a computational system for applying image overlay to a high dynamic range (HDR) image is provided. The computational system includes a processing circuit system and a memory storing instructions that, when executed, cause the processing circuit system to receive an HDR image. Furthermore, the processing circuit system is caused to generate a first gain map and a standard dynamic range (SDR) image based on the received HDR image. Then, the processing circuit system renders at least one overlay image to generate an SDR effect image buffer and an effect gain map; overlays the effect gain map and the first gain map to generate a second gain map; overlays the SDR image and the SDR effect image buffer to generate an edited SDR image; generates an edited HDR image based on the edited SDR image and the second gain map; and generates an output based on the edited HDR image.

[0006] This summary is provided to introduce some concepts in a simplified form, which will be further described in the following detailed description. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to embodiments that address any or all of the disadvantages pointed out in any part of this disclosure. Attached Figure Description

[0007] Figure 1 A schematic diagram of a computing system according to an example of this disclosure is shown.

[0008] Figure 2 Showing Figure 1 A schematic diagram illustrating the operation of the tone mapping algorithm in the HDR to SDR pipeline of the computing system.

[0009] Figure 3 Showing Figure 1 A schematic diagram of the operation of the gain graph overlay module of the computing system.

[0010] Figure 4 Showing Figure 1 A schematic diagram illustrating the operation of the inverse tone mapping algorithm in the SDR to HDR pipeline of the computing system.

[0011] Figure 5 This is a flowchart of a method for overlaying an SDR image and an SDR effect image buffer according to an example embodiment of the present disclosure.

[0012] Figure 6 This is a flowchart of a method for superimposing an effect gain map and a first gain map according to an example embodiment of the present disclosure.

[0013] Figure 7 This is a flowchart of a method for applying image overlay on an HDR image according to an example embodiment of the present disclosure.

[0014] Figure 8 An example computing environment of this disclosure is shown. Detailed Implementation

[0015] Figure 1A schematic diagram of an example computing system 100 for applying image overlay to a high dynamic range (HDR) image 110 is shown. The example computing system 100 can be implemented using various types of computing devices, including mobile devices, smartphones, personal computers, laptops, computing servers, etc. The example computing system 100 includes a processing circuitry system 102 and a memory 104 storing instructions that, during execution, cause the processing circuitry system 102 to perform various processes described herein to receive an HDR image 110 comprising a plurality of pixels, each pixel having one or more luminance values ​​for each of a plurality of color components. When the HDR image 110 is in a format that encodes a gain map, the processing circuitry system 102 decodes the HDR image 110 into an SDR image 116 and a first gain map 120. When the HDR image 110 is in a format that does not encode a gain map, the processing circuitry system 102 applies at least a tone mapping algorithm 114A to each pixel in the HDR image 110 to generate a first gain map 120 and a standard dynamic range (SDR) image 116 having transformed luminance values ​​for each of the plurality of color components. At least one overlay image is rendered to generate an SDR effect image buffer 126 and an effect gain map 134. The effect gain map 134 and a first gain map 120 are overlaid together to generate a second gain map 144. The SDR image 116 and the SDR effect image buffer 126 are overlaid together to generate an edited SDR image 140. An edited HDR image 148 is encoded based on the edited SDR image 140 and the second gain map 144, and an output is generated based on the edited HDR image 148. The generated edited HDR image 148 (which includes the applied image overlay) can be output for rendering on a display 152 and / or encoded by a video encoder 158 to generate and output a video stream 160 containing the edited HDR image 148. The display 152 can be a display device within the computing system 100 or an external device communicatively coupled to the computing system 100.

[0016] The computing system 100 also includes a camera 106 configured to capture an HDR image 110, which is then transferred to a memory 104 for further processing in an HDR-to-SDR pipeline 114. The video data of the HDR image 110 can be initially processed by an image signal processor before being transferred to the memory 104. Alternatively, the HDR image 110 can be transferred directly to the memory 104 in real time via a high-speed communication interface such as a Universal Serial Bus (USB), Thunderbolt, or High Definition Multimedia Interface (HDMI). For example, the high-speed communication interface can implement wireless technologies such as Wi-Fi, Bluetooth, Wireless HDMI, or cellular networks.

[0017] Alternatively, the HDR image 110 can be imported from various external sources via the image importer 108 and subsequently transferred to the memory 104. For example, the image importer 108 can be embodied as capture hardware configured to capture the HDR image 110 from an external camera and transfer the image data to the memory 104.

[0018] Go to Figure 2 The HDR to SDR pipeline 114 processes the HDR image 110 to generate an SDR image 116, a first gain map 120, and gain map data 122. The HDR image 110 comprises multiple pixels, each pixel having one or more luminance values ​​for each color component. For example, the input HDR image 110 may have a wide color gamut Rec.2020 color space with a color depth of 10 bits. The output SDR image 116 may have a narrow color gamut Rec.709 color space with a color depth of 8 bits. Each pixel of the HDR image 110 is processed by the functions of the HDR to SDR pipeline 114 to generate the SDR image 116, the first gain map 120, and the gain map data 122. The first gain map 120 encodes pixel data in logarithmic space for adjustments to the luminance and contrast of the SDR image 116 to convert the edited SDR image 140 into HDR format. Gain map data 122 can describe the attributes associated with the first gain map 120, including the dynamic range (maximum luminance value and minimum luminance value) and resolution of the original HDR image 110.

[0019] Return to Figure 1 The HDR to SDR pipeline 114 may include a tone mapping algorithm 114A that compresses the range of luminance values ​​in the HDR image 110 to fit the limitations of the display 152, while preserving the visual detail and contrast of the original HDR image 110 to the greatest extent possible within the physical limits of the display 152. Other functions included in the HDR to SDR pipeline 114 may include an electro-optical transfer function and a photo-electrical transfer function. When the HDR image 110 is received from the camera 106, the functions of the HDR to SDR pipeline 114 can be applied to each pixel in the HDR image 110 in real time, so that the generated SDR image 116 is output for rendering on the display 152 without perceptible delay.

[0020] Effects rendering module 124 is configured to render at least one overlay image to generate SDR effects image buffer 126. Examples of overlay images include, but are not limited to, stickers, emojis, virtual props, text effects, face masks, two-dimensional or three-dimensional objects, and particle effects. These overlay images can be rendered onto an empty SDR effects image buffer 126 such that only pixels directly affected by the effect are modified, while pixels without the effect remain completely transparent. When the overlay images are images such as stickers and emojis, the boundaries of these images are smaller than the boundaries of the HDR image 110.

[0021] Subsequently, tone mapping function 128 is applied to SDR-effect image buffer 126 to generate HDR-effect image buffer 130 containing the overlay image. Tone mapping function 128 is configured to determine an adjustment factor for a given pixel in SDR-effect image buffer 126 based on the peak brightness of display 152, and to scale each color component of the given pixel based on the adjustment factor. Examples of tone mapping function 128 that can be used to generate HDR-effect image buffer 130 include: linear functions, logarithmic functions, exponential functions, Reinhard formulas, and cinematic tone mapping operators (such as the Hable tone mapping operator or the Academy Color Coding System (ACES)).

[0022] Gain map generator 132 uses HDR effect image buffer 130 and SDR effect image buffer 126 to calculate and generate effect gain map 134. Effect gain map 134 specifies how pixel values ​​of the visual effect are adjusted when converting from SDR format to HDR format (and vice versa). Similar to the first gain map 120, effect gain map 134 is represented as a scalar function in logarithmic space relative to the maximum and minimum content enhancement values ​​to define the transition in brightness levels between SDR and HDR formats. The minimum content enhancement value defines how much darker the edited HDR image 148 can be relative to the edited SDR image 140. The maximum content enhancement value defines how much brighter the edited HDR image 148 can be relative to the edited SDR image 140.

[0023] Go to Figure 3The gain map overlay module 138 receives the effect gain map 134 and the first gain map 120, and overlays the gain maps 120 and 134 together to encode the second gain map 144. When a given pixel in the effect gain map 134 is opaque, the given pixel in the effect gain map 134 becomes the given pixel in the second gain map 144. When a given pixel in the effect gain map 134 is completely transparent, the corresponding pixel in the first gain map 120 becomes the pixel in the second gain map 144. When a given pixel in the effect gain map 134 is semi-transparent, the gain map overlay module 138 fits the alpha value of the given pixel in the effect gain map 134 to a logarithmic curve, and performs standard linear alpha blending on the given pixel in the effect gain map 134 to generate the given pixel in the second gain map 144.

[0024] Return to Figure 1 The image overlay module 136 overlays the SDR image 116 and the SDR effect image buffer 126 together to generate an edited SDR image 140 with an edited SDR image 140. When a given pixel in the SDR effect image buffer 126 is opaque, that given pixel becomes the given pixel in the edited SDR image 140. When a given pixel in the SDR effect image buffer 126 is completely transparent, the corresponding pixel in the unedited SDR image 116 becomes the given pixel in the edited SDR image 140. When a given pixel in the SDR effect image buffer 126 is semi-transparent, the image overlay module 136 performs standard linear alpha mixing on the given pixel in the SDR effect image buffer 126 to generate the given pixel in the edited SDR image 140.

[0025] Go to Figure 4 The edited SDR image 140, the second gain map 144, and the gain map data 122 are received by the SDR to HDR pipeline 146 and processed to generate the edited HDR image 148. Figure 4 In the example, the edited SDR image 140 and the edited HDR image 148 depict a scene on a train platform, and the rendering effect is a transparent triangle floating on the edge of the train platform. The edited HDR image 148 is generated in a format that encodes a second gain map 144 to render the scene on the train platform within a dynamic range that exceeds the dynamic range of the edited SDR image 140.

[0026] Return to Figure 1The edited HDR image 148 can be output for rendering on display 152 and / or encoded by video encoder 158 to generate and output a formatted encoded video 160 for storage or sharing. Before the user authorizes video encoder 158 to generate the encoded video 160 for sharing, a preview generator 154 can also be executed to generate a preview 156 for rendering on display 152 based on the edited HDR image 148. For example, preview 156 can be rendered on display 152 based on the edited HDR image 148, user input can be received to authorize the generation of the encoded video stream 160, and video encoder 158 generates the encoded video stream 160 for sharing in response to receiving user input.

[0027] Additional or alternative land, such as Figure 1 As indicated by the dashed arrows, the effect rendering module 124 can be configured to render at least one overlay image on the HDR effect image buffer 130. These overlay images can be rendered onto an empty HDR effect image buffer 130 such that only pixels directly affected by the effect are modified, while pixels without the effect remain completely transparent. When the overlay images are images such as stickers and emojis, the boundaries of these images are smaller than the boundaries of the HDR image 110. Subsequently, an inverse tone mapping function 129, which is the inverse function of tone mapping function 128, can be applied to the HDR effect image buffer 130 to generate an SDR effect image buffer 126. The inverse tone mapping function 129 is configured to determine the adjustment factor of a given pixel in the HDR effect image buffer 130 based on the peak brightness of the display 152, and to scale each color component of the given pixel based on the adjustment factor. The generated SDR effect image buffer 126 can then be used by the image overlay module 136 to generate an edited SDR image 140, and also used by the gain map generator 132, together with the HDR effect image buffer 130, to generate an effect gain map 134.

[0028] Figure 5 A flowchart illustrating an example method 200 for overlaying an unedited SDR image and an SDR-effect image buffer is shown. Example method 200 can be derived from... Figure 1The image overlay module 136 executes to overlay the SDR image 116 and the SDR effect image buffer 126 together to generate an edited SDR image 140. Example method 200 includes: at step 202, receiving the SDR image and the SDR effect image buffer. The HDR image can be received in various ways from various sources, including receiving it from a camera configured to capture HDR images or from an image importer configured to import HDR images from various external sources. Example method 200 includes: at step 204, overlaying the SDR image and the SDR effect image buffer together.

[0029] Step 204 may include step 206: determining that a given pixel in the SDR effect image buffer is opaque. For example, step 206 may determine that the alpha value of a given pixel in the SDR effect image buffer is one. In response to determining that a given pixel in the SDR effect image buffer is opaque, at step 208, the given pixel in the SDR effect image buffer becomes a given pixel in the edited SDR image.

[0030] Step 204 may include step 210: determining that a given pixel in the SDR effect image buffer is completely transparent. For example, step 210 may determine that the alpha value of a given pixel in the SDR effect image buffer is zero. In response to determining that a given pixel in the SDR effect image buffer is completely transparent, at step 212, the corresponding pixel in the unedited SDR image becomes the given pixel in the edited SDR image.

[0031] Step 204 may include step 214: determining that a given pixel in the SDR effect image buffer is semi-transparent. For example, step 214 may determine that the alpha value of the given pixel in the SDR effect image buffer is greater than zero and less than one. In response to determining that the given pixel in the SDR effect image buffer is semi-transparent, at step 216, standard linear alpha mixing is performed on the given pixel in the SDR effect image buffer to generate the given pixel of the edited SDR image. At step 220, the edited SDR image is encoded to generate the edited SDR image.

[0032] Figure 6 A flowchart of an example method 300 for superimposing an effect gain map and a first gain map is shown. Example method 300 can be derived from... Figure 1 The image overlay module 136 executes to overlay the first gain map 120 and the effect gain map 134 together to generate a second gain map 144. Example method 300 includes: at step 302, receiving the first gain map and the effect gain map; and at step 304, overlaying the first gain map and the effect gain map together.

[0033] Step 304 may include step 306: determining that a given pixel in the effect gain map is opaque. For example, step 306 may determine that the alpha value of a given pixel in the effect gain map is one. In response to determining that a given pixel in the effect gain map is opaque, at step 308, the given pixel in the effect gain map becomes a given pixel in the second gain map.

[0034] Step 304 may include step 310: determining that a pixel in the effect gain map is completely transparent. For example, step 310 may determine that the alpha value of a pixel in the effect gain map is zero. In response to determining that a pixel in the effect gain map is completely transparent, at step 312, the corresponding pixel in the first gain map becomes a pixel in the second gain map.

[0035] Step 304 may include step 314: determining that a given pixel in the effect gain map is semi-transparent. For example, step 314 may determine that the alpha value of the given pixel in the effect gain map is greater than zero and less than one. In response to determining that the given pixel in the effect gain map is semi-transparent, at step 316, the alpha value of the given pixel in the effect gain map is fitted to a logarithmic curve, and at step 318, standard linear alpha mixing is performed on the given pixel in the effect gain map to generate the given pixel in the second gain map. At step 320, the second gain map is encoded.

[0036] Figure 7 A flowchart illustrating an example method 400 for overlaying visual effects onto an HDR image is shown. Example method 400 can be derived from... Figure 1 The processing circuitry 102 and memory 104 of the computing system 100 are executed. Example method 400 includes: at step 402, receiving an HDR image. Example method 400 further includes: at step 404, generating a first gain map, gain map metadata, and an SDR image based on the received HDR image.

[0037] Example method 400 includes: at step 410, overlaying the SDR image and the SDR effect image buffer to generate an edited SDR image. Example method 400 includes: at step 414, generating an edited HDR image based on the edited SDR image, a second gain map, and gain map metadata. At step 418, method 400 includes generating an output based on the edited HDR image.

[0038] At step 406, method 400 includes rendering at least one overlay image to generate an SDR effect image buffer. At step 408, method 400 includes applying a tone mapping function to the SDR effect image buffer to generate an HDR effect image buffer. At step 412, method 400 includes generating an effect gain map using the SDR effect image buffer and the HDR effect image buffer. At step 416, method 400 includes overlaying the effect gain map and the first gain map to generate a second gain map.

[0039] Additionally or alternatively, method 400 may include: step 420 of rendering at least one overlay image on an HDR effect image buffer; step 422 of applying an inverse tone mapping function to the HDR effect image buffer to generate an SDR effect image buffer; and step 412 of generating an effect gain map using the SDR effect image buffer and the HDR effect image buffer. The SDR effect image buffer generated in step 422 may be used in step 410 to generate an edited SDR image.

[0040] As described in this paper, by converting HDR images to SDR and then applying image overlay, users can retain the high-quality image overlay effects originally designed for SDR content, while ensuring compatibility with the wider color and dynamic range associated with the HDR format. This method allows apps and software originally developed for SDR image processing to be effectively used for HDR content without introducing visual artifacts or distortions (such as oversaturation or desaturation in certain color spaces). Therefore, the system enhances the viewing experience of published HDR images by adding image overlay effects while maintaining the integrity of the original HDR image's visual quality. Furthermore, it can improve the quality of photos created using HDR-enabled cameras to meet high standards of visual fidelity and maintain consistency across various devices and platforms.

[0041] In some embodiments, the methods and processes described herein may be associated with a computing system of one or more computing devices. Specifically, these methods and processes may be implemented as computer applications or services, application programming interfaces (APIs), libraries, and / or other computer program products.

[0042] Figure 8 A non-limiting embodiment of a computing system 500 capable of implementing one or more of the methods and processes described above is schematically illustrated. The computing system 500 is shown in a simplified form. The computing system 500 can embody the above description and... Figure 1The computing system 100 shown in the image. Components of the computing system 500 may be included in one or more personal computers, server computers, tablet computers, home entertainment computers, network computing devices, video game devices, mobile computing devices, mobile communication devices (e.g., smartphones) and / or other computing devices, as well as wearable computing devices (such as smartwatches and head-mounted augmented reality devices).

[0043] The computing system 500 includes a processing circuit system 502, volatile memory 504, and non-volatile storage device 506. The computing system 500 may optionally include a display subsystem 508, an input subsystem 510, a communication subsystem 512, and / or... Figure 8 Other components not shown.

[0044] Processing circuitry systems typically include one or more logic processors, which are physical devices configured to execute instructions. For example, a logic processor may be configured to execute instructions that are part of one or more application programs, programs, routines, libraries, objects, components, data structures, or other logical structures. These instructions may be implemented to perform a task, implement a data type, change the state of one or more components, achieve a technical effect, or otherwise achieve a desired result.

[0045] The logic processor may include one or more physical processors configured to execute software instructions. Additionally or alternatively, the logic processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of the processing circuit system 502 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and / or distributed processing. Components of the processing circuit system may optionally be distributed across two or more separate devices that may be remotely located and / or configured for coordinated processing. For example, aspects of the computing system disclosed herein may be virtualized and executed via remotely accessible networked computing devices configured in a cloud computing configuration. It will be understood that in this case, these virtualized aspects run on different physical logic processors on various different machines. These different physical logic processors on different machines will be understood to be collectively included by the processing circuit system 502.

[0046] The non-volatile storage device 506 includes one or more physical devices configured to store instructions executable by a processing circuitry system to implement the methods and processes described herein. When such methods and processes are implemented, the state of the non-volatile storage device 506 may change, for example, to store different data.

[0047] Non-volatile storage device 506 may include removable and / or built-in physical devices. Non-volatile storage device 506 may include optical memory, semiconductor memory, and / or magnetic memory, or other high-capacity storage device technologies. Non-volatile storage device 506 may include non-volatile, dynamic, static, read / write, read-only, sequential access, location-addressable, file-addressable, and / or content-addressable devices. It should be understood that non-volatile storage device 506 is configured to retain instructions even when power to non-volatile storage device 506 is cut off.

[0048] Volatile memory 504 may include a physical device containing random access memory. Volatile memory 504 is typically used by processing circuitry system 502 to temporarily store information during the processing of software instructions. It should be understood that volatile memory 504 will typically not continue storing instructions when power to it is cut off.

[0049] The processing circuitry 502, the volatile memory 504, and the non-volatile storage device 506 can be integrated into one or more hardware logic components. Such hardware logic components may include field-programmable gate arrays (FPGAs), application-specific integrated circuits (PASICs / ASICs), application-specific standard products (PSSPs / ASSPs), system-on-a-chip (SoCs), and complex programmable logic devices (CPLDs), etc.

[0050] The terms "module," "program," and "engine" can be used to describe an aspect of computing system 500, typically implemented in software by a processor to perform specific functions using portions of volatile memory, the functions involving transformation processing specifically configured for the processor to perform those functions. Therefore, a module, program, or engine can be instantiated by executing instructions stored in non-volatile storage device 506 using portions of volatile memory 504 via processing circuitry system 502. It should be understood that different modules, programs, and / or engines can be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Similarly, the same modules, programs, and / or engines can be instantiated from different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module," "program," and "engine" can encompass single or multiple sets of executable files, data files, libraries, drivers, scripts, database records, etc.

[0051] Display subsystem 508 (when included) can be used to present a visual representation of data stored by non-volatile storage device 506. The visual representation may take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data stored by the non-volatile storage device, and thus change the state of the non-volatile storage device, the state of display subsystem 508 can also be changed in the same way to visually represent changes in the underlying data. Display subsystem 508 may include one or more display devices utilizing virtually any type of technology. Such display devices may be combined with processing circuitry system 502, volatile memory 504, and / or non-volatile storage device 506 in a shared housing, or such display devices may be peripheral display devices.

[0052] The input subsystem 510 (when included) may include one or more user input devices, such as a keyboard, mouse, touchscreen, camera, or microphone, or interface with such user input devices.

[0053] Communication subsystem 512 (when included) may be configured to communicatively couple the various computing devices described herein to each other and to other devices. Communication subsystem 512 may include wired and / or wireless communication devices compatible with one or more different communication protocols. As a non-limiting example, the communication subsystem may be configured to communicate via wired or wireless local area networks or wide area networks, broadband cellular networks, etc. In some embodiments, the communication subsystem may allow computing system 500 to send messages to and / or receive messages from other devices via a network such as the Internet.

[0054] The following paragraphs provide additional description of the subject matter of this disclosure. One aspect provides a computational system for applying image overlay on a high dynamic range (HDR) image, the computational system including a processing circuitry and a memory storing instructions that, when executed, cause the processing circuitry to: receive an HDR image; generate a first gain map and a standard dynamic range (SDR) image based on the received HDR image; render at least one overlay image to generate an SDR effect image buffer and an effect gain map; overlay the effect gain map and the first gain map to generate a second gain map; overlay the SDR image and the SDR effect image buffer to generate an edited SDR image; generate an edited HDR image based on the edited SDR image and the second gain map; and generate an output based on the edited HDR image.

[0055] In this regard, additionally or alternatively, the processing circuitry may also render the at least one overlay image on the SDR effect image buffer, apply a tone mapping function to the SDR effect image buffer to generate an HDR effect image buffer, and generate the effect gain map using the SDR effect image buffer and the HDR effect image buffer.

[0056] In this regard, additionally or alternatively, the edited HDR image can be output for rendering on a display, and the tone mapping function can be configured to: determine an adjustment factor for a given pixel in the SDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

[0057] In this regard, additionally or alternatively, when the SDR image and the SDR effect image buffer are superimposed: when a given pixel of the SDR effect image buffer is opaque, the given pixel of the SDR effect image buffer can become a given pixel of the edited SDR image; when a given pixel of the SDR effect image buffer is completely transparent, the corresponding pixel of the SDR image can become a given pixel of the edited SDR image; and when a given pixel of the SDR effect image buffer is semi-transparent, standard linear alpha mixing can be performed on the given pixel of the SDR effect image buffer to generate the given pixel of the edited SDR image.

[0058] In this regard, additionally or alternatively, the processing circuitry may also render the at least one overlay image on the HDR effect image buffer, apply an inverse tone mapping function to the HDR effect image buffer to generate the SDR effect image buffer, and generate the effect gain map using the SDR effect image buffer and the HDR effect image buffer.

[0059] In this regard, additionally or alternatively, the edited HDR image can be output for rendering on a display, and the inverse tone mapping function can be configured to: determine an adjustment factor for a given pixel in the HDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

[0060] In this regard, additionally or alternatively, when the effect gain map and the first gain map are superimposed: when a given pixel of the effect gain map is opaque, the given pixel of the effect gain map can become a given pixel of the second gain map; when a given pixel of the effect gain map is completely transparent, the corresponding pixel of the first gain map can become a given pixel of the second gain map; and when a given pixel of the effect gain map is semi-transparent, the alpha value of the given pixel of the effect gain map can be fitted to a logarithmic curve, and a standard linear alpha mixing can also be performed on the given pixel of the effect gain map to generate the given pixel of the second gain map.

[0061] In this regard, additionally or alternatively, the at least one overlay image may include at least one of the following: stickers, emojis, virtual props, text effects, face masks, objects, or particle effects.

[0062] In this regard, additionally or alternatively, a tone mapping algorithm can be applied to each pixel in the HDR image to further generate gain metadata.

[0063] In this regard, additionally or alternatively, the gain metadata may describe the dynamic range and resolution of the HDR image.

[0064] On the other hand, a computational method for applying image overlay on a high dynamic range (HDR) image is provided, the computational method comprising: receiving an HDR image; generating a first gain map and a standard dynamic range (SDR) image based on the received HDR image; rendering at least one overlay image to generate an SDR effect image buffer and an effect gain map; overlaying the effect gain map and the first gain map to generate a second gain map; overlaying the SDR image and the SDR effect image buffer to generate an edited SDR image; generating an edited HDR image based on the edited SDR image and the second gain map; and generating an output based on the edited HDR image.

[0065] In this regard, additionally or alternatively, the calculation method may further include: rendering the at least one overlay image on the SDR effect image buffer, applying a tone mapping function to the SDR effect image buffer to generate an HDR effect image buffer, and generating the effect gain map using the SDR effect image buffer and the HDR effect image buffer.

[0066] In this regard, additionally or alternatively, the edited HDR image can be output for rendering on a display, and the tone mapping function can be configured to: determine an adjustment factor for a given pixel in the SDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

[0067] In this regard, additionally or alternatively, when the SDR image and the SDR effect image buffer are superimposed: when a given pixel of the SDR effect image buffer is opaque, the given pixel of the SDR effect image buffer can become a given pixel of the edited SDR image; when a given pixel of the SDR effect image buffer is completely transparent, the corresponding pixel of the SDR image can become a given pixel of the edited SDR image; and when a given pixel of the SDR effect image buffer is semi-transparent, standard linear alpha mixing can be performed on the given pixel of the SDR effect image buffer to generate the given pixel of the edited SDR image.

[0068] In this regard, additionally or alternatively, the calculation method may further include: rendering the at least one overlay image on an HDR effect image buffer, applying an inverse tone mapping function to the HDR effect image buffer to generate the SDR effect image buffer, and generating the effect gain map using the SDR effect image buffer and the HDR effect image buffer.

[0069] In this regard, additionally or alternatively, the edited HDR image can be output for rendering on a display, and the inverse tone mapping function can be configured to: determine an adjustment factor for a given pixel in the HDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

[0070] In this regard, additionally or alternatively, when the effect gain map and the first gain map are superimposed: when a given pixel of the effect gain map is opaque, the given pixel of the effect gain map can become a given pixel of the second gain map; when a given pixel of the effect gain map is completely transparent, the corresponding pixel of the first gain map can become a given pixel of the second gain map; and when a given pixel of the effect gain map is semi-transparent, the alpha value of the given pixel of the effect gain map can be fitted to a logarithmic curve, and a standard linear alpha mixing can also be performed on the given pixel of the effect gain map to generate the given pixel of the second gain map.

[0071] In this regard, additionally or alternatively, the at least one overlay image includes at least one of the following: stickers, emojis, virtual props, text effects, face masks, objects, or particle effects.

[0072] In this regard, additionally or alternatively, a tone mapping algorithm can be applied to each pixel in the HDR image to further generate gain metadata.

[0073] On the other hand, a computing system for applying image overlay on a high dynamic range (HDR) image is provided. The computing system includes a processing circuitry and a memory storing instructions that, when executed, cause the processing circuitry to: receive the HDR image; generate a first gain map and a standard dynamic range (SDR) image based on the received HDR image; render at least one overlay image to generate an SDR effect image buffer; generate an effect gain map using a tone mapping function; overlay the effect gain map and the first gain map to generate a second gain map; overlay the SDR image and the SDR effect image buffer to generate an edited SDR image; generate an edited HDR image based on the edited SDR image and the second gain map; and output the edited HDR image for rendering on a display. The tone mapping function is configured to: determine an adjustment factor for a given pixel of the SDR effect image buffer based on the peak brightness of the display; and scale each color component of the given pixel based on the adjustment factor.

[0074] It should be understood that the configurations and / or methods described herein are exemplary in nature, and these specific embodiments or examples should not be considered limiting, as various variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. Therefore, the various actions illustrated and / or described may be performed in the illustrated and / or described order, in a different order, in parallel, or omitted. Similarly, the order of the above processes may also be changed.

[0075] The subject matter of this disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations disclosed herein, as well as any and all equivalents thereof.

Claims

1. A computing system for applying image overlay on high dynamic range (HDR) images, the computing system comprising: A processing circuit system and a memory for storing instructions, which, when executed, cause the processing circuit system to: Receive HDR images; A first gain map and a standard dynamic range SDR image are generated based on the received HDR image; Render at least one overlay image to generate an SDR effect image buffer and an effect gain map; The effect gain map and the first gain map are superimposed to generate a second gain map; The SDR image and the SDR effect image buffer are superimposed to generate the edited SDR image; An edited HDR image is generated based on the edited SDR image and the second gain map; as well as Output is generated based on the edited HDR image.

2. The computing system of claim 1, wherein the processing circuitry is further configured as follows: Render the at least one overlay image on the SDR effect image buffer; Apply a tone mapping function to the SDR-effect image buffer to generate an HDR-effect image buffer; and The effect gain map is generated using the SDR effect image buffer and the HDR effect image buffer.

3. The computing system according to claim 2, wherein Output the edited HDR image for rendering on the display; and The tone mapping function is configured to: determine an adjustment factor for a given pixel in the SDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

4. The computing system according to claim 1, wherein when the SDR image and the SDR effect image buffer are superimposed together, When a given pixel in the SDR effect image buffer is opaque, the given pixel in the SDR effect image buffer becomes the given pixel in the edited SDR image; When a given pixel in the SDR effect image buffer is completely transparent, the corresponding pixel in the SDR image becomes the given pixel in the edited SDR image; and When a given pixel in the SDR effect image buffer is semi-transparent, standard linear alpha mixing is performed on the given pixel in the SDR effect image buffer to generate the given pixel of the edited SDR image.

5. The computing system of claim 1, wherein the processing circuitry system further comprises: Render the at least one overlay image on the HDR effect image buffer; The inverse tone mapping function is applied to the HDR effect image buffer to generate the SDR effect image buffer; and The effect gain map is generated using the SDR effect image buffer and the HDR effect image buffer.

6. The computing system according to claim 5, wherein Output the edited HDR image for rendering on the display; and The inverse tone mapping function is configured to: determine an adjustment factor for a given pixel in the HDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

7. The computing system of claim 1, wherein when the effect gain map and the first gain map are superimposed, When a given pixel in the effect gain map is opaque, the given pixel in the effect gain map becomes a given pixel in the second gain map; When a given pixel in the effect gain map is completely transparent, the corresponding pixel in the first gain map becomes the given pixel in the second gain map; and When a given pixel in the effect gain map is semi-transparent, the alpha value of the given pixel in the effect gain map is fitted to a logarithmic curve, and a standard linear alpha blend is also performed on the given pixel in the effect gain map to generate the given pixel in the second gain map.

8. The computing system of claim 1, wherein the at least one overlaid image comprises at least one of the following: stickers, emojis, virtual props, text effects, face masks, objects, or particle effects.

9. The computing system according to claim 1, wherein A tone mapping algorithm is applied to each pixel in the HDR image to further generate gain image data.

10. The computing system of claim 9, wherein the gain image data describes the dynamic range and resolution of the HDR image.

11. A calculation method for applying image overlay on high dynamic range (HDR) images, the calculation method comprising: Receive HDR images; A first gain map and a standard dynamic range SDR image are generated based on the received HDR image; Render at least one overlay image to generate an SDR effect image buffer and an effect gain map; The effect gain map and the first gain map are superimposed to generate a second gain map; The SDR image and the SDR effect image buffer are superimposed to generate the edited SDR image; An edited HDR image is generated based on the edited SDR image and the second gain map; as well as Output is generated based on the edited HDR image.

12. The calculation method according to claim 11, further comprising: Render the at least one overlay image on the SDR effect image buffer; Apply a tone mapping function to the SDR-effect image buffer to generate an HDR-effect image buffer; and The effect gain map is generated using the SDR effect image buffer and the HDR effect image buffer.

13. The calculation method according to claim 12, wherein Output the edited HDR image for rendering on the display; and The tone mapping function is configured to: determine an adjustment factor for a given pixel in the SDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

14. The calculation method according to claim 11, wherein when the SDR image and the SDR effect image buffer are superimposed together, When a given pixel in the SDR effect image buffer is opaque, the given pixel in the SDR effect image buffer becomes the given pixel in the edited SDR image; When a given pixel in the SDR effect image buffer is completely transparent, the corresponding pixel in the SDR image becomes the given pixel in the edited SDR image; and When a given pixel in the SDR effect image buffer is semi-transparent, standard linear alpha mixing is performed on the given pixel in the SDR effect image buffer to generate the given pixel of the edited SDR image.

15. The calculation method according to claim 11, further comprising: Render the at least one overlay image on the HDR effect image buffer; The inverse tone mapping function is applied to the HDR effect image buffer to generate the SDR effect image buffer; and The effect gain map is generated using the SDR effect image buffer and the HDR effect image buffer.

16. The calculation method according to claim 15, wherein... Output the edited HDR image for rendering on the display; and The inverse tone mapping function is configured to: determine an adjustment factor for a given pixel in the HDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

17. The calculation method according to claim 11, wherein when the effect gain map and the first gain map are superimposed, When a given pixel in the effect gain map is opaque, the given pixel in the effect gain map becomes a given pixel in the second gain map; When a given pixel in the effect gain map is completely transparent, the corresponding pixel in the first gain map becomes the given pixel in the second gain map; and When a given pixel in the effect gain map is semi-transparent, the alpha value of the given pixel in the effect gain map is fitted to a logarithmic curve, and a standard linear alpha blend is also performed on the given pixel in the effect gain map to generate the given pixel in the second gain map.

18. The calculation method of claim 11, wherein the at least one overlaid image comprises at least one of the following: stickers, emojis, virtual props, text effects, face masks, objects, or particle effects.

19. The calculation method according to claim 11, wherein A tone mapping algorithm is applied to each pixel in the HDR image to further generate gain image data.

20. A computing system for applying image overlay on high dynamic range (HDR) images, the computing system comprising: A processing circuit system and a memory for storing instructions, which, when executed, cause the processing circuit system to: Receive the HDR image; A first gain map and a standard dynamic range SDR image are generated based on the received HDR image; Render at least one overlay image to generate an SDR-effect image buffer; Use tone mapping functions to generate effect gain maps; The effect gain map and the first gain map are superimposed to generate a second gain map; The SDR image and the SDR effect image buffer are superimposed to generate the edited SDR image; An edited HDR image is generated based on the edited SDR image and the second gain map; as well as The edited HDR image is output for rendering on the display, wherein The tone mapping function is configured to: determine an adjustment factor for a given pixel in the SDR effect image buffer based on the peak brightness of the display, and scale each color component of the given pixel based on the adjustment factor.

Images