A hybrid display processing method, device, equipment and medium

Abstract

The application relates to the field of high dynamic range videos, in particular to a hybrid display processing method and device, equipment and medium. A code stream obtained includes enhancement layer content for constructing an enhanced area image, base layer content for constructing a non-enhanced area image and non-enhanced area presentation intention indication metadata. The non-enhanced area presentation intention indication metadata is an independent control parameter, is read and takes effect when the enhanced area is active and in a hybrid display mode, and the value of the non-enhanced area presentation intention indication metadata corresponds to a specific processing mode applied to the non-enhanced area image. Due to the existence of the non-enhanced area presentation intention indication metadata, the processing logic can be switched from a unified default mode to a specific processing mode matched with the value according to different values, and finally a hybrid display picture is output, so that the hybrid display picture can be differentially constructed according to a preset intention, and the same code stream can present consistent transition degrees of the enhanced area and the non-enhanced area on different application end devices.

Description

Technical Field

[0001] This application relates to the field of high dynamic range video, and in particular to a processing method, apparatus, device, and medium for hybrid display. Background Technology

[0002] In mobile live streaming scenarios, users default to watching a basic video stream, such as 1080P SDR (Standard Dynamic Range). When a user is interested in a specific area of ​​the screen, they can zoom in on that area using interactive commands like two-finger zoom. At this point, the center of the zoomed-in image switches to a higher-resolution (e.g., 4K) HDR (High Dynamic Range Vivid) Vivid enhancement layer, while the surrounding areas retain the basic content. When the user no longer focuses on that area, they can exit zooming using two-finger pinch-out commands, and the image reverts from the mixed HDR Vivid and SDR display back to the full-screen basic display. This interaction mode is called DRZ (Dynamic Range / Resolution Zoom).

[0003] DRZ hybrid display refers to the simultaneous presence of regions of interest (ROIs) and non-ROIs in the same output image. The enhanced region is obtained by applying HDR Vivid global / local tone mapping to the enhanced layer content, used to present higher spatial resolution and / or higher dynamic range; the non-enhanced region is obtained by converting the base layer content to the same target display domain. The two contents are then combined into the output. In practical applications, highlighting differences and maintaining consistent brightness are two reasonable but different presentation goals. Highlighting differences aims to allow users to clearly identify the enhanced areas, while maintaining consistent brightness aims to ensure a smooth transition between the enhanced region and surrounding areas in terms of brightness, white point, and skin tone, avoiding punch-hole mapping or broken boundary lighting. These two goals are suitable for different application scenarios and may need to be switched between different times within the same standard, the same reference software, or even the same live broadcast, and the displayed results after switching should be consistent across different terminals.

[0004] However, in the above-mentioned implementation, the processing of the non-enhanced area is performed at the decoding end according to fixed default parameters. Since the default processing parameters of the non-enhanced area are inconsistent for different terminals, the same bitstream presents inconsistent transition degrees between the enhanced and non-enhanced areas on different devices, resulting in a decrease in the overall visual consistency of the mixed display screen.

[0005] Therefore, how to ensure that the same bitstream exhibits a consistent transition between enhanced and non-enhanced regions on different devices is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this application is to provide a mixed display processing method, apparatus, device, and medium that can solve the above-mentioned technical problems.

[0007] Firstly, a method for processing mixed displays is provided, including: Obtain the non-enhanced area rendering intent indication metadata, enhancement layer content, and base layer content from the bitstream; the non-enhanced area rendering intent indication metadata is used to indicate the rendering intent of the non-enhanced area image in the mixed display; The enhanced region image and the non-enhanced region image are determined based on the content of the enhancement layer and the content of the base layer; When the enhanced area is active and is currently in a mixed display, the non-enhanced area image is subjected to corresponding non-enhanced area processing according to the value of the non-enhanced area presentation intent indication metadata, and the processed non-enhanced area image is obtained. The enhanced area image is combined with the processed non-enhanced area image to output a mixed display screen.

[0008] In a preferred embodiment, this application can be further configured such that the values ​​of the non-enhanced area presentation intent indication metadata include: a first value, a second value, and a third value; Based on the value of the non-enhanced region rendering intent indication metadata, corresponding non-enhanced region processing is applied to the non-enhanced region image to obtain the processed non-enhanced region image, including: When the value is the first value, difference highlighting processing is used as the non-enhanced region processing, and difference highlighting processing is performed on the non-enhanced region image to obtain the processed first non-enhanced region image. When the value is the second value, brightness and color consistency processing is used as the non-enhanced region processing, and brightness and color consistency processing is performed on the non-enhanced region image to obtain the processed second non-enhanced region image. When the value is the third value, interpolation is used as the non-enhanced region processing. The non-enhanced region image is interpolated to obtain the processed third non-enhanced region image.

[0009] In a preferred embodiment, this application can be further configured to: employ brightness-color consistency processing as the non-enhanced region processing, perform brightness-color consistency processing on the non-enhanced region image, and obtain a processed second non-enhanced region image, including: Calculate the ratio of the first brightness of the enhanced area image to the second brightness of the non-enhanced area image, and limit the brightness ratio based on preset minimum and maximum brightness gains to obtain the brightness gain corresponding to the non-enhanced area image; The brightness of the non-enhanced area image is enhanced using the brightness gain to complete the brightness-color consistency processing, resulting in the processed second non-enhanced area image. or, The non-enhanced area image is mapped using a preset standard tone mapping curve, so that the non-enhanced area image and the enhanced area image have the same peak brightness, resulting in a processed second non-enhanced area image.

[0010] In a preferred embodiment, this application can be further configured to: employ interpolation processing as the non-enhanced region processing, perform interpolation processing on the non-enhanced region image to obtain a processed third non-enhanced region image, including: Determine the interpolation weights; based on the interpolation weights, perform interpolation processing on the first image after difference highlighting processing and the second image after brightness and color consistency processing to obtain the processed third non-enhanced region image.

[0011] In a preferred embodiment, this application may be further configured such that the interpolation weight is the area ratio of the enhanced area in the hybrid display screen, or a smooth step function value determined based on the normalized progress of the scaling factor relative to the starting and ending scaling factors.

[0012] In a preferred embodiment, this application can be further configured to: combine the enhanced region image with the processed non-enhanced region image to output a mixed display screen, including: When the value is the first value, the enhanced area image and the processed non-enhanced area image are combined using a hard-cutting method, or the enhanced area image and the processed non-enhanced area image are combined, and the combined image is subjected to spatial domain transition or temporal domain transition to output a mixed display screen; When the value is the second value, if the difference between the enhanced area image and the processed non-enhanced area image is less than a preset difference threshold, or if smoothing processing has been performed, then the enhanced area image and the processed non-enhanced area image are combined, and the spatial domain transition or temporal domain transition is turned off, and a mixed display screen is output.

[0013] In a preferred embodiment, this application may be further configured to include: Obtain the position information of the enhancement area in the bitstream in the output screen; Determining the enhanced region image and the non-enhanced region image based on the content of the enhancement layer and the content of the base layer includes: Based on the position information of the enhancement area in the output screen and the current viewport, the enhancement layer content is projected to the enhancement area range in the target display domain, and the basic layer content is projected to the non-enhanced area range in the target display domain, so as to determine the spatial position of the two images in the output screen. Perform HDR display adaptation processing on the enhancement layer content projected onto the enhancement area to obtain the enhancement area image corresponding to the enhancement layer content; The base layer content projected onto the non-enhanced area is transformed to obtain the non-enhanced area image corresponding to the base layer content. The enhanced area image and the non-enhanced area image are both located in the same target display domain, so that they can be spatially combined into an output image.

[0014] Secondly, a hybrid display processing device is provided, comprising: The acquisition module is used to acquire non-enhanced area rendering intent indication metadata, enhancement layer content, and base layer content in the bitstream; the non-enhanced area rendering intent indication metadata is used to indicate the rendering intent of the non-enhanced area image in the mixed display; The determination module is used to determine the enhanced region image and the non-enhanced region image based on the content of the enhancement layer and the content of the base layer; The processing module is used to apply corresponding non-enhanced area processing to the non-enhanced area image when the enhanced area is active and is currently in a mixed display, based on the value of the non-enhanced area presentation intent indication metadata, to obtain the processed non-enhanced area image. The combined output module is used to combine the enhanced area image with the processed non-enhanced area image to output a mixed display screen.

[0015] Thirdly, an electronic device is provided, the electronic device including a memory and a processor, the memory storing a computer program, the processor executing the method of any one of the first aspects when running the computer program.

[0016] Fourthly, a computer-readable storage medium is provided, wherein at least one piece of program code is stored therein, the program code being loaded and executed by a processor to implement the method as described in any of the first aspects.

[0017] Fifthly, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the method as described in any of the first aspects.

[0018] In summary, the method provided in this application has the following beneficial technical effects: In this application, the acquired bitstream includes enhancement layer content for constructing the enhanced area image and basic layer content for constructing the non-enhanced area image, as well as non-enhanced area rendering intent indication metadata. This non-enhanced area rendering intent indication metadata, as an independent control parameter, is read and takes effect when the enhancement area is active and the system is in hybrid display mode. Its value directly determines the specific processing method applied to the non-enhanced area image subsequently. Due to the existence of this non-enhanced area rendering intent indication metadata, the processing logic can be switched from a unified default mode to a specific processing method matching that value, ultimately outputting a hybrid display screen. This ensures that the hybrid display screen can be constructed differently according to a preset intent, allowing the same bitstream to present a consistent transition between enhanced and non-enhanced areas on different application devices. Furthermore, this application also provides an apparatus, device, and medium, all of which possess the aforementioned beneficial technical effects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a DRZ hybrid display in a mobile live streaming scenario provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a mixed display processing method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a hybrid display processing device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0022] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the permission and consent of the object, the permission and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the consent of the individual. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the permission and consent of the object.

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Furthermore, the terms "and / or" in this document are merely descriptions of the relationships between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0025] DRZ hybrid display refers to the simultaneous presence of Region of Interest (ROI) and Non-ROI in the same output image. The enhanced region is obtained by applying HDR Vivid global / local tone mapping to the enhanced layer content, used to present higher spatial resolution and / or higher dynamic range; the non-enhanced region is obtained by converting the base layer content to the same target display domain. The two contents are then combined into the output. In practice, there are two reasonable but different presentation goals for hybrid display. The first is to highlight the differences: users should be able to clearly identify which area is enhanced by spatial resolution and dynamic range; the second is to ensure consistent brightness and smooth transitions: the enhanced region should be continuous with the surrounding background in terms of brightness, white point, and skin tone, avoiding hole-punch textures or broken edge lighting. These two approaches are suitable for different application scenarios and may be needed at different times within the same standard, reference software, or even the same live stream. It is necessary to implement content-based switching of display modes, and the switching results should be interoperable. In the above scheme, the bitstream does not express which non-enhanced visual experience the production / application end wants; the decoding end uses the same set of non-enhanced region ROI (Region of Interest) processing.

[0026] The core theme of this application is to set the presentation intent of the non-enhanced area as metadata that follows the content, so that the decoding end can express the presentation intent of the non-enhanced area as needed during mixed display, while the enhanced area always follows the enhancement layer display adaptation path.

[0027] This application belongs to the field of high dynamic range (HDR) video, and specifically relates to a method for indicating, transmitting and differentiating the presentation intent of non-enhanced areas (non-ROI) in a dynamic range / resolution zoom (DRZ) hybrid display scenario.

[0028] In one embodiment, please refer to Figure 1 , Figure 1 A flowchart illustrating a DRZ hybrid display in a mobile live streaming scenario is provided, including: the user defaults to watching a 1080P SDR base layer video stream; when the user is interested in a certain area of ​​the screen and zooms in on that area using two fingers, the center of the zoomed-in screen switches to 4K HDR Vivid enhancement layer content, while the surrounding areas retain the base layer content; when the user exits zooming, the screen reverts from hybrid display to full-screen base layer display. In this scenario, the production end writes the presentation intent of the non-enhanced areas into HDR dynamic metadata (e.g., non_roi_presentation_mode with values ​​of 0 / 1 / 2), and encapsulates it in the bitstream along with the enhancement layer content and the base layer content; after receiving the bitstream, the decoding end, since the user is interested in the enhancement area, the enhancement area is active and currently in a hybrid display state, executes the method of this application, thereby applying differential processing to the non-enhanced areas, so that the enhancement and non-enhanced areas are presented according to the production intent.

[0029] If a user is watching a 1080P SDR base layer video stream by default and is not interested in a particular area of ​​the screen, the production end writes the non-enhanced area presentation intention into HDR dynamic metadata (e.g., non_roi_presentation_mode with a value of 0 / 1 / 2), and encapsulates it in the bitstream along with the enhanced layer content and the base layer content. After the decoding end receives this bitstream, since the user is not interested in the enhanced area, the current enhanced area state is inactive, and the base layer video screen is displayed directly.

[0030] Specifically, embodiments of this application provide a mixed display processing method, executed by an electronic device, such as... Figure 2 As shown, the method includes: S110. Obtain the non-enhanced area rendering intent indication metadata, enhancement layer content, and base layer content in the bitstream; the non-enhanced area rendering intent indication metadata is used to indicate the rendering intent of the non-enhanced area image in the mixed display.

[0031] The bitstream refers to a bitstream carrying enhancement layer content, base layer content, and dynamic metadata, including non-enhanced area presentation intent indication metadata and other metadata.

[0032] The presentation intent can be represented by two unsigned integers.

[0033] Enhancement layers represent high spatial resolution and high dynamic range enhanced video content, such as a 4K HDR Vivid enhancement layer, used to present magnified enhanced areas. Base layers represent default video stream content, such as a 1080p SDR base layer, used to present non-enhanced areas.

[0034] Specifically, the production team adds a non-enhanced area rendering intent indication metadata to the HDR dynamic metadata. This information is only valid when the enhanced area is active and the current display is in a blended state. It's understood that DRZ mode allows this functionality, but in DRZ scenarios, the display isn't always in a blended state; it only occurs when there's a part of interest and the enhanced area is active. However, even if only the enhanced area is active, there are still scenes where pure HDR display fills the entire screen when zoomed in significantly. Therefore, in this application, it needs to be valid only when the enhanced area is active and the current display is in a blended state.

[0035] The production team sets the appropriate processing method for the current non-enhanced area based on actual needs, such as determining the processing method based on director / application configuration / content analysis, and writing the corresponding value of the non-enhanced area presentation intent indication metadata into the dynamic metadata.

[0036] Additionally, other relevant optional metadata can be configured as needed. For example, the intent is to set corresponding metadata indicating the transition to avoid hard spatial boundaries and hard temporal transitions during highlighting. The metadata for various modes is related to the execution algorithm specified by the application, and the specific embodiments of this application are not limited thereto.

[0037] The decoder decapsulates the bitstream, separating the base layer content, enhancement layer content, and HDR dynamic metadata to obtain base layer content frames and enhancement layer content frames, both of which are raw layer content that has not yet been adapted for display. Simultaneously, it parses the HDR dynamic metadata, extracting the non-enhanced area presentation intent indication metadata field (e.g., non_roi_presentation_mode and its value). This metadata, along with the two layer contents, will serve as input for subsequently determining the non-enhanced area differential processing to be applied to the two images. It should be noted that this metadata is generated and written into the bitstream by the production end before content distribution; the decoder is only responsible for parsing and consuming it.

[0038] The metadata non_roi_presentation_mode, which indicates the presentation intent of the non-enhanced region, can be defined as follows: It can be represented by two unsigned integers: When the value is 0: it indicates that the difference strategy is applied, and the non-enhanced area retains the inherent appearance of the base layer, highlighting the difference from the enhanced area; When the value is 1: it means that the non-enhanced area is aligned with the bright color appearance of the enhanced area, reducing the breakage of the boundary light and shadow; When the value is 2: it means that the value is interpolated between 0 and 1 according to the scaling factor / viewport coverage. All other values ​​are retained.

[0039] S120. Determine the enhanced area image and the non-enhanced area image based on the content of the enhancement layer and the content of the base layer.

[0040] The enhancement layer content refers to high dynamic range enhanced video content that has not yet been adapted for display (such as a 4K HDRVivid enhancement layer). The base layer content refers to standard dynamic range base video content that has not yet been adapted for display (such as a 1080PSDR base layer). The enhanced area image refers to the image obtained by adapting the enhancement layer content to HDR Vivid and projecting it onto the current viewport. The non-enhanced area image refers to the image obtained by transforming the base layer content through color gamut / transfer function (which may include brightness compensation) and projecting it onto the current viewport. The target display domain is a common target display domain that represents the unified display domain before the two content streams are combined.

[0041] This step is performed after obtaining the content of both layers and before applying differential processing to the non-enhanced area. Specifically, the application (i.e., the terminal device that needs to display the image; tone mapping and other processing are performed on the display device according to the target device's brightness) processes the enhancement layer content and the base layer content separately and projects them onto the same target display domain, thus obtaining the enhanced area image and the non-enhanced area image: the enhanced area image is obtained from the enhancement layer content through HDR Vivid adaptation (tone mapping according to the target device's brightness), and it always follows the enhancement layer adaptation path and is not affected by the non-enhanced area processing; the non-enhanced area image is obtained from the base layer content through color gamut / transfer function conversion (including brightness compensation). Because the two images are in the same target display domain, their brightness, color gamut, and coordinate reference are consistent, which allows them to be spatially combined into a single output image.

[0042] In one possible implementation of this application embodiment, the location information of the enhancement region in the bitstream in the output image is obtained; the enhancement region image and the non-enhanced region image are determined based on the enhancement layer content and the base layer content, including: S120a. Obtain the position information of the enhancement area in the bitstream in the output screen.

[0043] The location information of the enhanced region in the output screen refers to the parameters used to describe the spatial position and size of the enhanced region (ROI) in the output screen.

[0044] S120b: Based on the position information of the enhancement area in the output screen and the current viewport, the enhancement layer content is projected to the enhancement area range in the target display domain, and the base layer content is projected to the non-enhanced area range in the target display domain, so as to determine the spatial position of the two images in the output screen.

[0045] Here, the current viewport represents the range and magnification of the content being viewed, determined by the user's two-finger zoom, and is used to determine the sampling range and magnification of the layer content to the target display domain. The target display domain represents the unified display domain before combination. The enhancement area range represents the rectangle or region occupied by the enhancement area within the target display domain. The non-enhanced area range represents the remaining area excluding the enhancement area range. In some embodiments, using the enhancement area location information and the current viewport as parameters, a mapping matrix from the enhanced layer content to the enhancement area range and from the base layer content to the non-enhanced area range is calculated to determine the spatial positions of the two images in the output screen.

[0046] S120c: Perform HDR display adaptation processing on the enhancement layer content projected onto the enhancement area to obtain the enhancement area image corresponding to the enhancement layer content.

[0047] HDR adaptation processing refers to performing HDR tone mapping and adaptation on the enhancement layer content according to the target brightness (e.g., 500 nits). The enhancement area image is the adapted image that maintains the HDR display adaptation path of the enhancement layer and is not subject to subsequent non-enhanced area differentiation processing. In this process, only the enhancement area is mapped; the non-enhanced area is not mapped.

[0048] S120d: The basic layer content projected onto the non-enhanced area is transformed to obtain a non-enhanced area image corresponding to the basic layer content. The enhanced area image and the non-enhanced area image are both in the same target display domain so that they can be spatially combined into an output image.

[0049] Here, "conversion" refers to at least one of color gamut conversion and transfer function conversion performed on the base layer content, and may include brightness compensation of the base layer content to obtain the non-enhanced area image. The embodiments of this application align the two images within the same target display domain, supporting subsequent non-enhanced area differentiation processing while ensuring spatial composability with the enhanced area image.

[0050] The purpose of this step is to obtain enhanced area images and non-enhanced area images that are in the same target display domain from the two layers of content. In this embodiment, the two contents are adapted separately at the display end according to the brightness of the target device. This ensures the image quality stability of the enhanced area following the HDR Vivid adaptation path of the enhanced layer, and allows the non-enhanced area to be processed differently as an independent input. The two images can be combined in the same target display domain.

[0051] S130. When the enhanced area is active and currently in a mixed display, apply corresponding non-enhanced area processing to the non-enhanced area image according to the value of the non-enhanced area presentation intent indication metadata to obtain the processed non-enhanced area image.

[0052] The value of the metadata indicating the presentation intent of the non-enhanced area refers to the specific value of the metadata field. The magnitude of this value and its corresponding meaning can be customized by the user, and are not limited in this embodiment. In one possible implementation, the value may include: a first value, a second value, and a third value, corresponding to presentation intents such as highlighting differences, consistent brightness, and interpolation, respectively.

[0053] In this embodiment, difference highlighting processing, brightness and color consistency processing, or interpolation processing are applied to the non-enhanced area according to the presentation intent; the processed non-enhanced area image refers to the non-enhanced area image that has been processed and is used to combine with the enhanced area image for output.

[0054] Specifically, the decoder reads the value of the non-enhanced region rendering intent indicator metadata and selects the corresponding non-enhanced region processing method accordingly. It then applies the appropriate processing to the non-enhanced region image to obtain the processed non-enhanced region image. This process can be represented by the unified formula O_nROI=P_nROI(I_nROI;mode,θ), where P_nROI(·) represents the processing operation on the non-enhanced region, mode is the value of the non-enhanced region rendering intent indicator metadata, θ is the corresponding set of optional parameters, I_nROI is the non-enhanced region image, and O_nROI is the processed non-enhanced region image.

[0055] S140. Combine the enhanced area image with the processed non-enhanced area image to output a mixed display screen.

[0056] Among them, the hybrid display screen refers to the final screen output by the display device, which is formed by splicing the enhanced area image and the processed non-enhanced area image according to the enhanced area geometry.

[0057] Specifically, based on the position and size of the enhanced area in the output screen, the application places the enhanced area image within the enhanced area range of the target display domain and the processed non-enhanced area image within the non-enhanced area range, thereby combining them into a mixed display screen and outputting it to the display device. The combination method can be selected from hard cut or additional spatial domain blending, temporal domain fade-in and fade-out transition methods, etc.

[0058] As can be seen, in this embodiment, the acquired bitstream includes enhancement layer content for constructing the enhanced area image, basic layer content for constructing the non-enhanced area image, and non-enhanced area rendering intent indication metadata. This non-enhanced area rendering intent indication metadata, as an independent control parameter, is read and takes effect when the enhancement area is active and the system is in hybrid display mode. Its value directly determines the specific processing method applied to the non-enhanced area image subsequently. Due to the existence of this non-enhanced area rendering intent indication metadata, the processing logic can be switched from a unified default mode to a specific processing method matching that value, ultimately outputting a hybrid display screen. This ensures that the hybrid display screen can be constructed differently according to a preset intent, allowing the same bitstream to present a consistent transition between enhanced and non-enhanced areas on different application devices.

[0059] In one possible implementation of this application embodiment, the value of the non-enhanced area presentation intent indication metadata includes a first value, a second value, and a third value; according to the value, corresponding non-enhanced area processing is applied to the non-enhanced area image to obtain a processed non-enhanced area image, including: when the value is the first value, difference highlighting processing is used as non-enhanced area processing, and difference highlighting processing is performed on the non-enhanced area image to obtain a processed first non-enhanced area image; when the value is the second value, brightness and color consistency processing is used as non-enhanced area processing, and brightness and color consistency processing is performed on the non-enhanced area image to obtain a processed second non-enhanced area image; when the value is the third value, interpolation processing is used as non-enhanced area processing, and interpolation processing is performed on the non-enhanced area image to obtain a processed third non-enhanced area image.

[0060] The first value corresponds to the intention to highlight differences, for example, mode=0, which indicates that the difference between the non-enhanced area and the enhanced area should be maintained or enhanced, corresponding to difference highlighting processing (T_diff). The second value corresponds to the intention to maintain brightness consistency, for example, mode=1, which indicates that the non-enhanced area should be aligned with the brightness appearance of the enhanced area, corresponding to brightness consistency processing (T_cons). The third value corresponds to the intention to interpolate, for example, mode=2, which indicates that weighted interpolation should be performed between difference highlighting and brightness consistency, corresponding to interpolation processing.

[0061] As can be seen, in this embodiment, when the metadata value is the first value, difference highlighting processing is used to preserve or enhance the visual difference between the non-enhanced area and the enhanced area; when the value is the second value, brightness and color consistency processing is used to adjust the brightness and color performance of the non-enhanced area to make it more uniform in appearance with the enhanced area, thereby eliminating the visual disjointedness that may occur when the two areas are placed side by side; when the value is the third value, interpolation processing is used to generate a compromise image state between the two extreme effects of difference highlighting processing and brightness and color consistency processing, providing a transition effect for mixed display.

[0062] When the value is the first value, difference highlighting is used as the non-enhanced area processing. Difference highlighting is performed on the non-enhanced area image to obtain the processed first non-enhanced area image. For example, the reference white for SDR and HDR interoperability is maintained, the non-enhanced area is darkened and desaturated, and the tone mapping curve used in the enhanced area is prohibited from being applied to the non-enhanced area, so that the viewer can stably identify the boundary of the enhanced area.

[0063] When the value is the second value, i.e., mode is 1, brightness and color consistency processing is used as the non-enhanced area processing. To perform brightness and color consistency processing on the non-enhanced area image, an operation to align the non-enhanced area with the appearance of the enhanced area needs to be applied to it to obtain the processed second non-enhanced area image.

[0064] In this application embodiment, the processing of brightness and color consistency is no longer limited, and may include setting the gain according to the ratio of the brightness of the enhanced area to the non-enhanced area, or performing tone mapping on the non-enhanced area. Users can choose according to their actual needs.

[0065] To set the gain, the ratio of the first brightness of the enhanced area image to the second brightness of the non-enhanced area image is calculated, and the brightness ratio is limited based on the preset minimum and maximum brightness gains to obtain the brightness gain corresponding to the non-enhanced area image. The brightness gain is then used to perform gain processing on the brightness of the non-enhanced area image to complete the brightness-color consistency processing, resulting in the processed second non-enhanced area image.

[0066] The first luminance, denoted as L_ROI, is the luminance value of the enhanced region image in the linear optical domain or PQ domain. The calculation method for the luminance signal is unrestricted; peak luminance, etc., can be used. The second luminance, denoted as L_nROI, is the luminance value of the non-enhanced region image in the linear optical domain or PQ domain. The calculation method for the luminance signal is unrestricted; peak luminance, etc., can be used.

[0067] The brightness ratio refers to the ratio of L_ROI to L_nROI, used to represent the relative difference in brightness between the enhanced and non-enhanced areas. The preset minimum brightness gain, denoted as g_min, represents the lower limit of the brightness gain allowed to be applied to the non-enhanced area, used to avoid alignment failure due to excessively low gain. Users can set its value based on practical experience. The preset maximum brightness gain (denoted as g_max) represents the upper limit of the brightness gain allowed to be applied to the non-enhanced area, used to avoid overexposure or distortion in the non-enhanced area due to excessive gain. Users can set its value based on practical experience.

[0068] In this embodiment, the gain can be set according to the ratio of the brightness of the enhanced area to that of the non-enhanced area: g_nROI=Clip(L_ROI / L_nROI,g_min,g_max), where g_nROI represents the calculated brightness gain.

[0069] The calculated luminance gain is multiplied by the luminance of the non-enhanced area to align the luminance. Chroma can remain unchanged, aligning only the luminance; alternatively, the saturation of the non-enhanced area can be brought closer to the enhanced area to further align the visual perception.

[0070] It is understandable that other methods can be used to calculate the brightness gain, such as estimating the gain based on brightness histogram matching, which is not limited here.

[0071] Tone mapping is performed on the non-enhanced area. In this embodiment, a preset standard tone mapping curve is used to map the non-enhanced area image so that the non-enhanced area image and the enhanced area image have the same peak brightness, resulting in a processed second non-enhanced area image.

[0072] It is understandable that the enhanced area undergoes a standard global tone mapping during the process of determining the enhanced area image based on the enhanced layer content; however, the non-enhanced area does not undergo mapping. In this embodiment, the non-enhanced area image can also be mapped using a preset standard tone mapping curve, so that the non-enhanced area image and the enhanced area image have the same peak brightness and display style, resulting in a processed second non-enhanced area image. The mapping direction and mapping curve of the enhanced area and the non-enhanced area can be different. For example, when the target device brightness is 500 nits, the enhanced area HDR content brightness is 1000 nits, and the non-enhanced area brightness is 203 nits, the non-enhanced area needs to perform up mapping, and the enhanced area needs to perform down mapping, so that the non-enhanced area aligns with the bright color appearance of the enhanced area, thereby reducing boundary light and shadow breaks.

[0073] As can be seen, in this embodiment, during brightness and color consistency processing, the ratio of brightness between the enhanced area image and the non-enhanced area image is calculated, and this ratio is limited using preset minimum and maximum brightness gains to obtain a brightness gain. Finally, this gain is applied to the brightness of the non-enhanced area image, causing its brightness level to approach that of the enhanced area. A preset standard tone mapping curve is then used to remap the non-enhanced area image, aligning its peak brightness and display style with the enhanced area image.

[0074] When the value is the third option, i.e., mode is 2, it indicates that interpolation can be performed between highlighting and brightness-consistent processing based on the scaling factor / viewport coverage. The interpolation weights are determined; based on these weights, interpolation is performed on the first image after highlighting and the second image after brightness-consistent processing to obtain the processed third non-enhanced area image. The interpolation weights are either the proportion of the enhanced area in the mixed display or a smooth step function value determined based on the normalized progress of the scaling factor relative to the initial and final scaling factors.

[0075] The interpolation weight (denoted as w) represents the weight used for interpolation between difference enhancement processing and brightness consistency processing, with a value range of [0,1]. In one implementation, the proportion ρ∈[0,1] of the enhanced area within the viewport to the entire output image is statistically calculated, and ρ is directly used as the interpolation weight w. Alternatively, the current scaling factor is normalized relative to the starting and ending scaling factors to a progress u∈[0,1], and the smooth step function SmoothStep(u) is used as the interpolation weight w. It is understood that other methods can also be used to determine the interpolation weight, such as the production end explicitly carrying the weight value in the metadata; this is not limited here.

[0076] Furthermore, for the third value, difference highlighting processing can be used as the non-enhanced area processing, applying difference highlighting processing to the non-enhanced area image; brightness and color consistency processing can be used as the non-enhanced area processing, applying brightness and color consistency processing to the non-enhanced area image; interpolation processing is then performed on the two images obtained above to obtain the processed third non-enhanced area image. O_nROI=(1-w)·T_diff(I_nROI)+w·T_cons(I_nROI), where T_diff(I_nROI) represents the image obtained through difference highlighting processing, and T_cons(I_nROI) represents the image obtained through brightness and color consistency processing. At this point, at low magnification and when the enhanced area occupies only a small portion of the image, difference highlighting is preferred; when it is nearly full, brightness and color consistency is preferred.

[0077] It is understandable that other methods can be used to implement interpolation, such as generating two processing results in parallel for the non-enhanced region and then mixing them according to weights; this is not limited here.

[0078] Through the above interpolation process, a natural transition can be achieved between highlighting differences and maintaining consistent brightness, allowing users to gradually zoom in from a low magnification to near full coverage, so that the visual perception of the non-enhanced area smoothly transitions from highlighting differences to maintaining consistent brightness.

[0079] In one possible implementation of this application embodiment, different strategies may be used when combining outputs.

[0080] If differential processing (mode 0) is used in the reprocessing of non-enhanced regions, a significant appearance difference is usually intentionally maintained on both sides of the boundary. If a hard cut is used, the seam may serve as a guide; if the seam is unacceptable, a spatial / temporal transition can be enabled to achieve a smooth transition.

[0081] If the non-enhanced region reprocessing adopts consistent / smoothing processing (mode 1), since its goal is to reduce differences, when the differences are no longer perceptible, or when the reprocessing itself includes smoothing, additional spatial / temporal transitions can be turned off.

[0082] Specifically, in this embodiment of the application, the enhanced area image is combined with the processed non-enhanced area image to output a mixed display screen, including: When the value is the first value, the enhanced area image and the processed non-enhanced area image are combined using a hard-cutting method, or the enhanced area image and the processed non-enhanced area image are combined, and the combined image is subjected to spatial domain transition or temporal domain transition to output a mixed display screen.

[0083] Among them, the hard cut method means that the grouping method directly splices the boundary between the enhanced region and the non-enhanced region without applying additional smoothing.

[0084] Spatial transition refers to a transition method that weights and blends the pixels of the two paths according to their spatial distance at the seam. Temporal transition refers to a transition method that fades in and out of the two images according to the time frame rate during switching. The seam of the combined image represents the boundary area between the enhanced area and the non-enhanced area.

[0085] Specifically, since the first value is intended to maintain a large visual difference between the enhanced and non-enhanced areas, the two images can be combined by default using a hard cut method, so that the seam itself becomes a visual cue to guide the viewer to identify the enhanced area; if the seam is unacceptable, a spatial or temporal transition can be enabled at the seam to achieve a smooth transition, depending on the implementation strategy. This strategy can be preset on the application side or random, and this application does not limit it.

[0086] When the value is the second value, if the difference between the enhanced area image and the processed non-enhanced area image is less than the preset difference threshold, or if smoothing has been performed, the enhanced area image and the processed non-enhanced area image will be combined, and the spatial domain transition or temporal domain transition will be turned off to output a mixed display screen.

[0087] The preset difference threshold represents the brightness / chromaticity difference threshold used to determine whether the difference between two images is negligible. This threshold can be set by the user based on experience. The non-enhanced area processing corresponding to the second value includes smoothing, which represents brightness and color consistency processing. Smoothing / alignment operations (such as standard curve remapping) are already included in the pre-processing.

[0088] Specifically, since the goal of the second value is to reduce the difference between the enhanced and non-enhanced areas, when the difference between the two is less than the preset difference threshold, or when the brightness and color consistency processing itself already includes smoothing, additional spatial / temporal transitions are unnecessary. The decoding end directly combines the two images and turns off the spatial or temporal transitions to output a mixed display screen, thereby saving computational overhead and avoiding blurring introduced by unnecessary transitions.

[0089] In some embodiments, the combined output under the second value can be achieved in a variety of ways: Optionally, the difference determination to close the transition method is adopted: calculate the difference measure between the enhanced area image and the processed non-enhanced area image in the seam area, and if it is less than the preset difference threshold, then directly combine and close the spatial / temporal transition.

[0090] As can be seen, in this embodiment, when the metadata value is the first value, a hard-cutting method is used to combine the enhanced area image and the processed non-enhanced area image, or a spatial or temporal transition is applied to the combined image. The hard-cutting method directly and without delay stitches the two regions together, which can clearly delineate the boundary of the enhanced area, thereby preserving and presenting the image quality contrast emphasized by the difference highlighting processing to the greatest extent. On the basis of hard cutting, the spatial or temporal transition can be superimposed to appropriately soften the abrupt change at the boundary while keeping the enhanced area prominent, reducing visual abruptness. When the metadata value is the second value, if the difference between the enhanced area image and the processed non-enhanced area image is less than a preset difference threshold, or if smoothing processing has been performed, it indicates that the two areas have reached sufficient similarity or fusion in appearance. At this time, the spatial or temporal transition is turned off when combining the images. On the one hand, this avoids applying unnecessary transition processing on the boundary of the already consistent area, and on the other hand, it simplifies the processing flow of the combined output.

[0091] Based on any of the above embodiments, this application embodiment, taking a mobile live streaming DRZ scenario as an example, includes: On the production side: The live streaming platform server determines the presentation intent of the current non-enhanced area. For example, during the time when the main broadcast focuses on a certain area, if the director wants the audience to clearly identify the enhanced area, then the presentation intent of the non-enhanced area is determined to be "distinction highlighting". The non-enhanced area presentation intent indication metadata non_roi_presentation_mode is set to the first value (corresponding to mode=0) and written into the HDR dynamic metadata, which is distributed along with the enhanced layer content (4KHDRVivid) and the basic layer content (1080PSDR) in the bitstream; this metadata only takes effect when mixed display is active.

[0092] Decoding end: The viewer terminal device receives the bitstream, decapsulates it to obtain the basic layer content frame and the enhancement layer content frame, and parses the HDR dynamic metadata to obtain the non-enhanced area presentation intent indication metadata value (the first value in this example), the position information of the enhancement area in the output screen, and (when the mixed display is active) confirm that the current display is in a mixed display.

[0093] Application side: Based on the enhancement area location information and the current viewport (two-finger zoom state), the enhancement layer content is projected onto the enhancement area range in the target display domain, and the base layer content is projected onto the non-enhanced area range; HDR display adaptation is performed on the enhancement layer content projected onto the enhancement area range (downmapped to the target 500nit brightness) to obtain the enhancement area image (preserving the enhancement layer adaptation path); color gamut / transfer function conversion (including brightness compensation) is performed on the base layer content projected onto the non-enhanced area range to obtain the non-enhanced area image.

[0094] The metadata value is read as the first value. The difference highlighting process T_diff is used to highlight the non-enhanced area image, maintain the reference white for SDR and HDR interoperability, darken and desaturate the non-enhanced area, and prohibit the tone mapping curve used in the enhanced area from being applied to the non-enhanced area, thus obtaining the processed first non-enhanced area image.

[0095] Combined output: The enhanced area image is combined with the processed first non-enhanced area image using a hard-cutting method, so that the viewer can stably identify the boundary of the enhanced area and output a mixed display image.

[0096] At another time, if the production end sets the metadata to the second value (same brightness and color, corresponding to mode=1), the application end calculates the brightness gain g_nROI=Clip(L_ROI / L_nROI,g_min,g_max) and aligns the gain (or remaps using the standard tone mapping curve), and turns off the extra transition because the difference is already small; If the production end sets the metadata to the third value (interpolation, corresponding to mode=2), the application end determines the weight w according to the SmoothStep(u) of the enhanced area ratio ρ or the normalized scaling progress u, and interpolates T_diff(I_nROI) and T_cons(I_nROI) to obtain the third non-enhanced area image, so as to achieve a natural transition from highlighting differences to consistent brightness as the interaction progresses.

[0097] Through the above process, the presentation intent of the non-enhanced area within the mixed display is made into bitstream semantics. The presentation intent is distinguished from the metadata semantics. The bitstream can accurately express how the non-enhanced area within the mixed display should be presented without affecting the enhanced area content. Only the non-enhanced area content is reprocessed, taking into account multiple presentation intents and corresponding decoding end operation methods. The combined output transition operation is not mandatory. Whether to smoothly transition is selected according to different modes / conditions. This allows the decoding end to rely on the corresponding metadata parameters to realize the reproduction of the production intent, and the enhanced area image quality path remains stable and unaffected.

[0098] The following describes a hybrid display processing apparatus provided in an embodiment of this application. The apparatus described below corresponds to the method described above. The apparatus of this embodiment is installed in an electronic device. Figure 3 , Figure 3 This is a structural block diagram of an apparatus according to one embodiment of this application, comprising: The acquisition module 210 is used to acquire the non-enhanced area rendering intent indication metadata, enhancement layer content, and base layer content in the bitstream; the non-enhanced area rendering intent indication metadata is used to indicate the rendering intent of the non-enhanced area image in the mixed display; The determination module 220 is used to determine the enhanced area image and the non-enhanced area image based on the enhanced layer content and the base layer content; The processing module 230 is used to apply corresponding non-enhanced area processing to the non-enhanced area image according to the value of the non-enhanced area presentation intent indication metadata when the enhanced area is active and is currently in a mixed display, so as to obtain the processed non-enhanced area image. The combined output module 240 is used to combine the enhanced area image with the processed non-enhanced area image to output a mixed display screen.

[0099] In one feasible approach, the values ​​of the non-enhanced area rendering intent indicator metadata include: a first value, a second value, and a third value; Processing module 230 is used for: When the value is the first value, difference highlighting is used as the non-enhanced region processing. Difference highlighting is performed on the non-enhanced region image to obtain the processed first non-enhanced region image. When the value is the second value, brightness and color consistency processing is used for non-enhanced areas. Brightness and color consistency processing is performed on the non-enhanced area image to obtain the processed second non-enhanced area image. When the value is the third value, interpolation is used as the non-enhanced region processing. The non-enhanced region image is interpolated to obtain the processed third non-enhanced region image.

[0100] In one possible implementation, the processing module 230 is used for: Calculate the ratio of the first brightness of the enhanced area image to the second brightness of the non-enhanced area image, and limit the brightness ratio based on the preset minimum brightness gain and maximum brightness gain to obtain the brightness gain corresponding to the non-enhanced area image. The brightness of the non-enhanced area image is increased by using brightness gain to achieve brightness-color consistency processing, resulting in the processed second non-enhanced area image. or, The non-enhanced area image is mapped using a preset standard tone mapping curve, so that the non-enhanced area image and the enhanced area image have the same peak brightness, resulting in the processed second non-enhanced area image.

[0101] In one possible implementation, the processing module 230 is used for: Determine the interpolation weights; based on the interpolation weights, perform interpolation processing on the first image after difference highlighting processing and the second image after brightness and color consistency processing to obtain the processed third non-enhanced region image.

[0102] In one feasible approach, the interpolation weight is either the area ratio of the enhanced region in the mixed display or a smooth step function value determined based on the normalized progress of the scaling factor relative to the initial and final scaling factors.

[0103] In one possible implementation, the combined output module 240 is used for: When the value is the first value, the enhanced area image and the processed non-enhanced area image are combined using a hard-cutting method, or the enhanced area image and the processed non-enhanced area image are combined, and the combined image is subjected to spatial domain transition or temporal domain transition to output a mixed display screen; When the value is the second value, if the difference between the enhanced area image and the processed non-enhanced area image is less than the preset difference threshold, or if smoothing has been performed, the enhanced area image and the processed non-enhanced area image will be combined, and the spatial domain transition or temporal domain transition will be turned off to output a mixed display screen.

[0104] In one possible implementation, module 210 is also used for: Obtain the position information of the enhancement area in the bitstream in the output screen; The determination module 220 is used to: project the enhancement layer content to the enhancement area range in the target display field and the base layer content to the non-enhanced area range in the target display field according to the position information of the enhancement area in the output screen and the current viewport, so as to determine the spatial position of the two images in the output screen. Perform HDR display adaptation processing on the enhancement layer content projected onto the enhancement area to obtain the enhancement area image corresponding to the enhancement layer content; The base layer content projected onto the non-enhanced area is transformed to obtain the non-enhanced area image corresponding to the base layer content. In this design, both the enhanced and non-enhanced images are located within the same target display domain, allowing them to be spatially combined into a single output image.

[0105] Figure 4 A structural diagram of an electronic device provided in an embodiment of this application, such as... Figure 4 As shown, the electronic device includes: a memory 60 for storing a computer program; and a processor 61 for executing the computer program to implement the steps of the method as described in the above embodiments.

[0106] The electronic devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0107] The processor 61 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 61 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 61 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 61 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0108] The memory 60 may include one or more computer-readable storage media, which may be non-transitory. The memory 60 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 60 is used to store at least the following computer program 601, which, after being loaded and executed by the processor 61, is capable of implementing the relevant steps of the method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 60 may also include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. The operating system 602 may include Windows, Unix, Linux, etc.

[0109] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64, a power supply 65, and a communication bus 66.

[0110] Those skilled in the art will understand that Figure 4 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0111] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, magnetic disks, or optical disks, and other media capable of storing program code.

[0112] Based on this, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described above.

[0113] Based on this, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described method. It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0114] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for processing hybrid displays, characterized in that, include: Obtain metadata indicating the rendering intent of non-enhanced areas, enhancement layer content, and base layer content from the bitstream; The non-enhanced area rendering intent indicator metadata is used to indicate the rendering intent of the non-enhanced area image in the mixed display; The enhanced region image and the non-enhanced region image are determined based on the content of the enhancement layer and the content of the base layer; When the enhanced area is active and is currently in a mixed display, the non-enhanced area image is subjected to corresponding non-enhanced area processing according to the value of the non-enhanced area presentation intent indication metadata, and the processed non-enhanced area image is obtained. The enhanced area image is combined with the processed non-enhanced area image to output a mixed display screen.

2. The method according to claim 1, characterized in that, The values ​​of the non-enhanced area presentation intent indication metadata include: a first value, a second value, and a third value; Based on the value of the non-enhanced region rendering intent indication metadata, corresponding non-enhanced region processing is applied to the non-enhanced region image to obtain the processed non-enhanced region image, including: When the value is the first value, difference highlighting processing is used as the non-enhanced region processing, and difference highlighting processing is performed on the non-enhanced region image to obtain the processed first non-enhanced region image. When the value is the second value, brightness and color consistency processing is used as the non-enhanced region processing, and brightness and color consistency processing is performed on the non-enhanced region image to obtain the processed second non-enhanced region image. When the value is the third value, interpolation is used as the non-enhanced region processing. The non-enhanced region image is interpolated to obtain the processed third non-enhanced region image.

3. The method according to claim 2, characterized in that, Brightness and color consistency processing is applied to the non-enhanced region image to obtain a processed second non-enhanced region image, including: Calculate the ratio of the first brightness of the enhanced area image to the second brightness of the non-enhanced area image, and limit the brightness ratio based on preset minimum and maximum brightness gains to obtain the brightness gain corresponding to the non-enhanced area image; The brightness of the non-enhanced area image is enhanced using the brightness gain to complete the brightness-color consistency processing, resulting in the processed second non-enhanced area image. or, The non-enhanced area image is mapped using a preset standard tone mapping curve, so that the non-enhanced area image and the enhanced area image have the same peak brightness, resulting in a processed second non-enhanced area image.

4. The method according to claim 2, characterized in that, Interpolation processing is used as the processing for the non-enhanced region. Interpolation processing is performed on the non-enhanced region image to obtain a processed third non-enhanced region image, including: Determine the interpolation weights; based on the interpolation weights, perform interpolation processing on the first image after difference highlighting processing and the second image after brightness and color consistency processing to obtain the processed third non-enhanced region image.

5. The method according to claim 4, characterized in that, The interpolation weight is the area ratio of the enhanced area in the hybrid display image, or a smooth step function value determined based on the normalized progress of the scaling factor relative to the starting and ending scaling factors.

6. The method according to claim 2, characterized in that, The enhanced area image is combined with the processed non-enhanced area image to output a mixed display screen, including: When the value is the first value, the enhanced area image and the processed non-enhanced area image are combined using a hard-cutting method, or the enhanced area image and the processed non-enhanced area image are combined, and the combined image is subjected to spatial domain transition or temporal domain transition to output a mixed display screen; When the value is the second value, if the difference between the enhanced area image and the processed non-enhanced area image is less than a preset difference threshold, or if smoothing processing has been performed, then the enhanced area image and the processed non-enhanced area image are combined, and the spatial domain transition or temporal domain transition is turned off, and a mixed display screen is output.

7. The method according to claim 1, characterized in that, Also includes: Obtain the position information of the enhancement area in the bitstream in the output screen; Determining the enhanced region image and the non-enhanced region image based on the content of the enhancement layer and the content of the base layer includes: Based on the position information of the enhancement area in the output screen and the current viewport, the enhancement layer content is projected to the enhancement area range in the target display domain, and the basic layer content is projected to the non-enhanced area range in the target display domain, so as to determine the spatial position of the two images in the output screen. Perform HDR display adaptation processing on the enhancement layer content projected onto the enhancement area to obtain the enhancement area image corresponding to the enhancement layer content; The base layer content projected onto the non-enhanced area is transformed to obtain the non-enhanced area image corresponding to the base layer content. The enhanced area image and the non-enhanced area image are both located in the same target display domain, so that they can be spatially combined into an output image.

8. A hybrid display processing apparatus, characterized in that, include: The acquisition module is used to acquire metadata indicating the presentation intent of non-enhanced areas, enhancement layer content, and basic layer content in the bitstream; The non-enhanced area rendering intent indicator metadata is used to indicate the rendering intent of the non-enhanced area image in the mixed display; The determination module is used to determine the enhanced region image and the non-enhanced region image based on the content of the enhancement layer and the content of the base layer; The processing module is used to apply corresponding non-enhanced area processing to the non-enhanced area image when the enhanced area is active and is currently in a mixed display, based on the value of the non-enhanced area presentation intent indication metadata, to obtain the processed non-enhanced area image. The combined output module is used to combine the enhanced area image with the processed non-enhanced area image to output a mixed display screen.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the method of any one of claims 1 to 7 when running the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the method as described in any one of claims 1 to 7.

Images