A near-eye display device and image display method

Abstract

The application discloses a near-eye display device and an image display method. The image display method is applied to a display system, and the display system comprises a signal output device and the near-eye display device. The image display method sends display configuration information to the signal output device through the near-eye display device. The signal output device obtains video source data, decodes the video source data to obtain a first image frame and dynamic metadata corresponding to the first image frame. The signal output device performs dynamic tone mapping on the first image frame according to the display configuration information and the dynamic metadata. The signal output device performs perceptual quantization non-linear encoding on the first image frame to obtain a second image frame, and transmits the second image frame to the near-eye display device. The near-eye display device performs gamma conversion on the second image frame and displays the second image frame. The application can adaptively process a video signal in combination with the display capability of the near-eye display device, thereby improving the display effect.

Description

Technical Field

[0001] This application relates to the field of display technology, specifically to a near-eye display device and an image display method. Background Technology

[0002] With the development of augmented reality (AR) display technology, AR glasses are gradually being applied to video playback and immersive viewing scenarios. Their high brightness and wide color gamut display capabilities provide the hardware foundation for presenting high-quality images. However, due to limitations in the internal space, power consumption, and heat dissipation of the glasses, existing AR glasses are usually unable to complete complex high dynamic range image processing on their own devices.

[0003] Currently, most AR glasses use an SDR video playback link, which directly stretches a video signal with a brightness of about 100 nits and an sRGB / BT.709 color gamut to a high-brightness, high-color-gamut screen. This method does not fully utilize the actual display capabilities of AR glasses, and is prone to causing abnormalities such as brightness level distortion and color shift, as well as image quality problems such as contour lines and color level breaks, thus affecting the overall display effect.

[0004] Therefore, the technology still needs to be improved and enhanced. Summary of the Invention

[0005] This application provides a near-eye display device and an image display method, which can combine the display capabilities of the near-eye display device to adapt and process video signals, thereby improving the overall display effect.

[0006] This application provides an image display method applied to a display system, which includes a signal output device and a near-eye display device. The image display method includes: The near-eye display device sends display configuration information to the signal output device. The display configuration information includes the peak brightness information of the near-eye display device. The signal output device acquires video source data, decodes the video source data, and obtains the first image frame and the dynamic metadata corresponding to the first image frame; The signal output device performs dynamic tone mapping on the first image frame based on the display configuration information and dynamic metadata. The signal output device performs perceptual quantization nonlinear encoding on the first image frame to obtain the second image frame, and transmits the second image frame to the near-eye display device; The near-eye display device performs gamma conversion on the second image frame and displays it.

[0007] In some embodiments of the image display method, the step of acquiring video source data includes: The signal output device receives streaming media data, decapsulates and decodes the streaming media data to obtain the video source data.

[0008] In some embodiments of the image display method, the steps of the signal output device acquiring video source data, decoding the video source data, and obtaining a first image frame and dynamic metadata corresponding to the first image frame include: The signal output device acquires video source data, decodes the video source data, and obtains the basic image layer and the enhanced image layer; The signal output device superimposes the base image layer and the enhanced image layer to obtain the first image frame; The signal output device obtains the dynamic metadata corresponding to the first image frame from the video source data.

[0009] In some embodiments of the image display method, dynamic metadata includes a brightness mapping reference curve; The steps of the signal output device performing dynamic tone mapping on the first image frame based on display configuration information and dynamic metadata include: The signal output device dynamically adjusts the brightness mapping reference curve in the dynamic metadata based on the peak brightness information to obtain the first brightness mapping curve. The signal output device performs pixel-by-pixel brightness conversion on the first image frame according to the first brightness mapping curve.

[0010] In some embodiments of the image display method, dynamic metadata includes brightness mapping parameters; The steps of the signal output device performing dynamic tone mapping on the first image frame based on display configuration information and dynamic metadata include: The signal output device generates a second brightness mapping curve corresponding to the near-eye display device based on the peak brightness information and the brightness mapping parameters in the dynamic metadata; The signal output device performs pixel-by-pixel brightness conversion on the first image frame according to the second brightness mapping curve.

[0011] In some embodiments of the image display method, the display configuration information further includes target color gamut information of the near-eye display device; Image display methods also include: The signal output device acquires the source color gamut information of the video source data; The signal output device performs color gamut mapping on the first image frame data based on the target color gamut information and the source color gamut information.

[0012] In some embodiments of the image display method, the step of the signal output device acquiring source color gamut information of video source data includes: The signal output device obtains the source color gamut information from the static metadata of the video source data.

[0013] In some embodiments of the image display method, the step of the signal output device acquiring source color gamut information of video source data includes: The signal output device determines the standard preset color gamut as the source color gamut information based on the encoding format of the video source data.

[0014] In some embodiments of the image display method, the step of the signal output device performing color gamut mapping on the first image frame data according to the target color gamut information and the source color gamut information includes: The signal output device generates a color gamut mapping matrix based on the target color gamut information and the source color gamut information; The signal output device performs color gamut conversion on the first image frame data according to the color gamut mapping matrix.

[0015] This application also provides an image display method, which is applied to a signal output device. The image display method includes: Obtain the display configuration information output by the near-eye display device, which includes the peak brightness information of the near-eye display device; The video source data is acquired and decoded to obtain the first image frame and the dynamic metadata corresponding to the first image frame. Dynamic tone mapping is performed on the first image frame based on the display configuration information and dynamic metadata; The first image frame is subjected to perceptual quantization nonlinear encoding to obtain the second image frame, and the second image frame is transmitted to the near-eye display device, so that the near-eye display device performs gamma conversion on the second image frame and displays it.

[0016] In some embodiments of the image display method, the step of acquiring video source data includes: It receives streaming media data, decapsulates and decodes the streaming media data to obtain the video source data.

[0017] In some embodiments of the image display method, the steps of acquiring video source data, decoding the video source data, and obtaining a first image frame and dynamic metadata corresponding to the first image frame include: The video source data is acquired and decoded to obtain the base image layer and the enhanced image layer. The first image frame is obtained by overlaying the base image layer and the enhancement image layer; Obtain the dynamic metadata corresponding to the first image frame from the video source data.

[0018] In some embodiments of the image display method, dynamic metadata includes a brightness mapping reference curve; The steps for performing dynamic tone mapping on the first image frame based on display configuration information and dynamic metadata include: Based on the peak brightness information, the brightness mapping reference curve in the dynamic metadata is dynamically mapped and adjusted to obtain the first brightness mapping curve; Perform brightness conversion pixel by pixel on the first image frame according to the first brightness mapping curve.

[0019] In some embodiments of the image display method, dynamic metadata includes brightness mapping parameters; The steps for performing dynamic tone mapping on the first image frame based on display configuration information and dynamic metadata include: Based on the peak brightness information and the brightness mapping parameters in the dynamic metadata, a second brightness mapping curve corresponding to the near-eye display device is generated. The brightness conversion of the first image frame is performed pixel by pixel according to the second brightness mapping curve.

[0020] In some embodiments of the image display method, the display configuration information further includes target color gamut information of the near-eye display device; Image display methods also include: Obtain the source color gamut information of the video source data; Color gamut mapping is performed on the first image frame data based on the target color gamut information and the source color gamut information.

[0021] In some embodiments of the image display method, the step of obtaining source color gamut information of video source data includes: Obtain source color gamut information from the static metadata of the video source data.

[0022] In some embodiments of the image display method, the step of obtaining source color gamut information of video source data includes: Based on the encoding format of the video source data, the standard preset color gamut is determined as the source color gamut information.

[0023] In some embodiments of the image display method, the step of color gamut mapping of the first image frame data based on the target color gamut information and the source color gamut information includes: Generate a color gamut mapping matrix based on the target color gamut information and the source color gamut information; The color gamut of the first image frame data is converted according to the color gamut mapping matrix.

[0024] This application embodiment also provides a near-eye display device, the near-eye display device comprising: Display module; The image quality processing module is connected to the display module. The image quality processing module is used to perform gamma conversion on the second image frame in the above image display method in order to control the display module to display.

[0025] The near-eye display device and image display method provided in this application report the peak brightness information of the near-eye display device to the signal output device, enabling the signal output device to perform dynamic tone mapping according to the actual capabilities of the display device, thereby alleviating the brightness distortion problem caused by directly stretching the video signal. Attached Figure Description

[0026] The technical solution and other beneficial effects of this application will become apparent from the following detailed description of specific embodiments in conjunction with the accompanying drawings.

[0027] Figure 1 This is a structural block diagram of the display system provided in an embodiment of this application.

[0028] Figure 2 This is a schematic flowchart of the first embodiment of the image display method provided in this application.

[0029] Figure 3 This is a schematic flowchart of a second embodiment of the image display method provided in this application.

[0030] Figure 4 This is a schematic flowchart of a third embodiment of the image display method provided in this application.

[0031] Figure 5 This is a schematic flowchart of the fourth embodiment of the image display method provided in this application.

[0032] Figure 6 This is a first structural block diagram of a near-eye display device provided in an embodiment of this application.

[0033] Figure 7 This is a second structural block diagram of a near-eye display device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0035] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Features thus defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0036] This application provides a near-eye display device and an image display method. The image display method reports the peak brightness information of the near-eye display device to a signal output device, enabling the signal output device to perform dynamic tone mapping based on the actual capabilities of the display device, thereby alleviating the brightness distortion problem caused by directly stretching the video signal. The near-eye display device and the image display method are described below.

[0037] Please see Figure 1 This application provides an image display method applied to a display system, which includes a signal output device 100 and a near-eye display device 200. The signal output device 100 can be a smartphone, tablet, laptop, TV box, or other electronic device with video decoding and transmission capabilities. The near-eye display device 200 includes, but is not limited to, head-mounted near-eye display devices such as virtual reality (VR) headsets, augmented reality (AR) glasses, mixed reality (MR) headsets, and smart glasses.

[0038] Please see Figure 2 The image display method in this embodiment includes steps S101 to S105, as follows: S101, the near-eye display device 200 sends display configuration information to the signal output device 100, the display configuration information including the peak brightness information of the near-eye display device 200.

[0039] After establishing a communication connection with the signal output device 100, the near-eye display device 200 reads its own stored display capability parameters, generates display configuration information, and sends it to the signal output device 100. The display configuration information includes the peak brightness information of the near-eye display device 200.

[0040] S102, the signal output device 100 acquires video source data, decodes the video source data, and obtains the first image frame and the dynamic metadata corresponding to the first image frame.

[0041] The signal output device 100 can receive streaming media data via wired or wireless networks, and can also read video source data from local storage media.

[0042] In one embodiment, the signal output device 100 receives streaming media data via a network, decapsulates and decodes the streaming media data to obtain video source data. Specifically, the signal output device 100 can obtain streaming media data streams from a web server via at least one of the following protocols: HTTP, HLS (HTTP Live Streaming), or DASH (Dynamic Adaptive Streaming over HTTP). Streaming media data is typically transmitted in segments, each segment containing several seconds of video data. After receiving these segments, the signal output device 100 first performs a decapsulation operation, extracting the video encoded stream (such as HEVC / H.265 encoded stream, AVC / H.264 encoded stream, etc.) from the streaming media container format (such as MP4, TS, etc.). Then, the signal output device 100 decodes the video encoded stream to obtain the video source data.

[0043] For Dolby Vision streaming content acquired via the network, a single-layer Dolby Vision format is typically used. A single-layer Dolby Vision format contains only the Base Layer (BL) and Reference Processing Unit (RPU), but not the Enhancement Layer (EL). After decoding, the signal output device 100 directly obtains the first image frame and the corresponding RPU, without needing to overlay the Base Layer and Enhancement Layer. For HDR10+ streaming content acquired via the network, the signal output device 100 decodes the first image frame and RPU containing luminance mapping parameters.

[0044] In this embodiment, by supporting online playback of streaming media data, users can directly watch video content provided by online video platforms on the near-eye display device 200 without having to download video files in advance, thereby improving the convenience of use and the accessibility of content.

[0045] In another embodiment, the signal output device 100 reads a video source file from a local storage medium. The local storage medium can be the signal output device 100's built-in memory, an external storage device, or an optical disc medium. The signal output device 100 reads the video file through a file system interface, then decapsulates the video file, and then decodes the video encoded stream to obtain the video source data.

[0046] For Dolby Vision format video files read locally, a dual-layer Dolby Vision format can be used. Specifically, step S102 includes: the signal output device 100 acquires video source data, decodes the video source data to obtain a base image layer and an enhanced image layer; the signal output device 100 superimposes the base image layer and the enhanced image layer to obtain a first image frame; and the signal output device 100 acquires the dynamic metadata corresponding to the first image frame from the video source data.

[0047] The Dolby Vision video stream includes a base image layer, an enhancement image layer, and associated metadata. During decoding, the signal output device 100 decodes the video source data to obtain the base image layer and the enhancement image layer, respectively. The base image layer and the enhancement image layer are then superimposed to generate a complete first image frame (i.e., a complete HDR image frame). Simultaneously, the signal output device 100 extracts dynamic metadata corresponding to the first image frame from the video source data. The dynamic metadata contains brightness distribution information for that frame or scene, such as minimum brightness, average brightness, and maximum brightness levels. This metadata can be created by professional colorists during content creation, helping to maintain the creative intent of the film to a greater extent on the playback device. In this embodiment, by performing the superposition operation of the base layer and the enhancement layer on the signal output device 100 side, the near-eye display device 200 side does not need to process the complex dual-layer Dolby Vision data structure.

[0048] For HDR10+ format video files read locally, the signal output device 100 directly obtains the first image frame and dynamic metadata after decoding, without the need for double-layer overlay operation.

[0049] In this embodiment, video source data is obtained through local file reading, allowing users to play high-quality local HD video content and enjoy high-quality dual-layer Dolby Vision picture quality. Simultaneously, the superposition calculation of the base layer and enhancement layer is performed on the signal output device 100 side, reducing complex calculations on the near-eye display device 200 side, thereby helping to reduce the power consumption of the near-eye display device 200.

[0050] S103, Signal output device 100 performs dynamic tone mapping on the first image frame based on display configuration information and dynamic metadata.

[0051] After obtaining the display configuration information (especially peak brightness information) of the near-eye display device 200 and the dynamic metadata corresponding to the first image frame, the signal output device 100 performs dynamic tone mapping on the first image frame based on these two types of information.

[0052] Dynamic tone mapping is the process of mapping the brightness values ​​in a high dynamic range (HDR) image to the brightness range that the target display device can display. Traditional static tone mapping uses a fixed mapping curve, which cannot be adjusted according to changes in image content, easily leading to loss of detail in bright areas or insufficient gradation in dark areas. Dynamic tone mapping, on the other hand, dynamically adjusts the mapping curve according to the brightness distribution characteristics of each frame or each scene, so that each frame can achieve better brightness presentation on the target display device.

[0053] For video source data encoded in Dolby Vision format, the dynamic metadata includes a luminance mapping reference curve. Step S103 includes: the signal output device 100 dynamically adjusts the luminance mapping reference curve in the dynamic metadata according to the peak luminance information to obtain a first luminance mapping curve; the signal output device 100 performs luminance conversion pixel by pixel on the first image frame according to the first luminance mapping curve.

[0054] The brightness mapping reference curve can be a mapping curve targeting the peak brightness of a reference monitor (such as 1000 nits or 4000 nits). When the peak brightness of the target display device (i.e., the near-eye display device 200) differs from that of the reference monitor, the reference curve needs to be adjusted. For example, if the peak brightness of the reference monitor is 1000 nits, while the peak brightness of the near-eye display device 200 is 600 nits, the output range of the reference curve needs to be compressed from 0~1000 nits to 0~600 nits. This compression is not a simple linear scaling, but requires non-linear adjustment based on the visual characteristics of the human eye to maintain the perceptual consistency of brightness levels. The signal output device 100 dynamically adjusts the key nodes of the brightness mapping reference curve (such as dark inflection points, midtone slopes, and highlight roll-off points) based on the peak brightness information of the near-eye display device 200 to generate a first brightness mapping curve adapted to the near-eye display device 200. Next, the signal output device 100 iterates through each pixel of the first image frame, and based on the original brightness value of that pixel, finds the corresponding mapped brightness value on the first brightness mapping curve to complete the brightness conversion. Pixel-by-pixel processing ensures that every detail in each frame can obtain accurate brightness mapping.

[0055] In this embodiment, dynamic adjustment is performed based on the brightness mapping reference curve in the dynamic metadata, which can largely preserve the creative intent of the content creator, while adapting to the actual display capability of the near-eye display device 200 and alleviating the problem of brightness level distortion caused by simple stretching.

[0056] For HDR10+ format video source data, the dynamic metadata includes luminance mapping parameters, rather than a complete luminance mapping baseline curve. Step S103 specifically includes: the signal output device 100 generates a second luminance mapping curve corresponding to the near-eye display device 200 based on the peak luminance information and the luminance mapping parameters in the dynamic metadata; the signal output device 100 performs pixel-by-pixel luminance conversion on the first image frame based on the second luminance mapping curve.

[0057] The signal output device 100 constructs a basic brightness mapping relationship based on the brightness mapping parameters in the dynamic metadata, and then adapts and adjusts this basic mapping relationship according to the peak brightness information of the near-eye display device 200 to generate a second brightness mapping curve. After generating the second brightness mapping curve, the signal output device 100 performs brightness conversion pixel by pixel on the first image frame according to the curve. In this embodiment, the brightness mapping information is transmitted in a parameterized manner, which is beneficial to improving the accuracy of dynamic tone mapping.

[0058] S104, the signal output device 100 performs perceptual quantization nonlinear encoding on the first image frame to obtain the second image frame, and transmits the second image frame to the near-eye display device 200.

[0059] After completing dynamic tone mapping, the signal output device 100 performs perceptual quantization nonlinear encoding on the processed first image frame to obtain a second image frame, and then transmits the second image frame to the near-eye display device 200 via a wired or wireless connection. Perceptual quantization nonlinear encoding can be a nonlinear transfer function based on the characteristics of the human visual system. In this embodiment, perceptual quantization nonlinear encoding converts the linear brightness value after brightness mapping into a nonlinear encoded value to maintain the perceptual uniformity of brightness during transmission.

[0060] S105, the near-eye display device 200 performs gamma conversion on the second image frame and displays it.

[0061] The second image frame output by signal output device 100 uses perceptual quantization nonlinear encoding. However, near-eye display device 200 typically uses a gamma curve as its electro-optical conversion function. To correctly reproduce the image content on near-eye display device 200, the perceptual quantization nonlinear encoded signal needs to be converted into the gamma-encoded signal used by display module 220. Therefore, after receiving the second image frame transmitted from signal output device 100, near-eye display device 200 performs gamma conversion on the second image frame.

[0062] The image display method of this embodiment reports the peak brightness information of the near-eye display device 200 to the signal output device 100, enabling the signal output device 100 to perform dynamic tone mapping according to the actual capabilities of the display device, thereby alleviating the brightness distortion problem caused by directly stretching the video signal.

[0063] Please see Figure 3 In some embodiments, the display configuration information also includes target color gamut information of the near-eye display device 200. The target color gamut information can be the color gamut range that the near-eye display device 200 can cover, such as the DCI-P3 color gamut, BT.2020 color gamut, or sRGB / BT.709 color gamut. The image display method further includes steps S201 and S202, as follows: S201, Signal output device 100 acquires source color gamut information of video source data.

[0064] Source color gamut information refers to the color gamut range used when the video source data was produced. There are two ways to obtain source color gamut information.

[0065] In one embodiment, step S201 includes: the signal output device 100 obtaining source color gamut information from the static metadata of the video source data. For Dolby Vision video streams, static metadata is carried during encoding. Static metadata is a set of parameters describing the inherent attributes of the entire video content, which is fixed during the encoding stage and does not change with scene variations. The static metadata includes source color gamut information, such as primary color coordinates and white point coordinates. The signal output device 100 obtains the source color gamut information by parsing this static metadata.

[0066] In another embodiment, step S201 includes: the signal output device 100 determines the standard preset color gamut as the source color gamut information according to the encoding format of the video source data.

[0067] Different video encoding formats typically have a fixed correspondence with specific color gamut standards. For example, if the video source data uses the BT.709 encoding format, the source color gamut can be determined to be sRGB / BT.709; if it uses the BT.2020 encoding format, the source color gamut can be determined to be BT.2020. Furthermore, for HDR video formats such as HDR10 and HDR10+, regardless of their specific encoding implementation, their color space is based on the ITU-R BT.2020 standard; therefore, the BT.2020 color gamut is also determined as the source color gamut information.

[0068] S202, Signal output device 100 performs color gamut mapping on the first image frame data based on the target color gamut information and the source color gamut information.

[0069] Color gamut mapping is the process of converting colors from a source color gamut to a target color gamut. Different color gamuts cover different color ranges on a chromaticity diagram. For example, the BT.2020 color gamut covers a wider range of colors than the DCI-P3 color gamut, which in turn covers a wider range than the sRGB / BT.709 color gamut. Color gamut mapping is required when the color gamut of the video source data is inconsistent with that of the near-eye display device 200. For example, HDR10+ videos are typically produced using the BT.2020 color gamut, while the near-eye display device 200 may only cover the DCI-P3 color gamut. In this case, colors in the BT.2020 color gamut need to be mapped to the DCI-P3 color gamut.

[0070] It should be noted that the color gamut mapping in this embodiment is a static mapping process for Dolby Vision and HDR10+ format video source data. Dynamic tone mapping adjusts the mapping curve frame by frame based on the dynamic metadata of each frame or scene to adapt to the brightness distribution characteristics of different images; while the source color gamut information and target color gamut information on which color gamut mapping is based are static parameters. The source color gamut information comes from the static metadata of the video source data or is inferred from the encoding format. Once the static parameters are determined, they remain unchanged throughout the playback of the video content and will not change with changes in scenes or frames. Therefore, the color gamut mapping in this embodiment only needs to be calculated once and can be applied to all image frames without repeated calculations frame by frame.

[0071] Specifically, step 202 includes: the signal output device 100 generating a color gamut mapping matrix based on the target color gamut information and the source color gamut information; and the signal output device 100 performing color gamut conversion on the first image frame data based on the color gamut mapping matrix. The color gamut mapping matrix can be a 3×3 matrix used to convert the RGB values ​​of the source color gamut to the RGB values ​​of the target color gamut. In this embodiment, through color gamut mapping processing, the colors in the video source data can be converted to the target color gamut of the near-eye display device 200, alleviating the problems of color shift and saturation distortion caused by color gamut mismatch.

[0072] This embodiment of the application performs complex processing such as dynamic tone mapping and color gamut mapping on the signal output device 100 side, and the near-eye display device 200 only needs to complete a simple gamma conversion. This achieves the display of high dynamic range video signals such as Dolby Vision without increasing the space and power consumption of the near-eye display device 200.

[0073] It should be noted that, in some embodiments, the color gamut mapping process can also be completed on the near-eye display device 200 side.

[0074] Specifically, after completing dynamic tone mapping and perceptual quantization nonlinear encoding, the signal output device 100 transmits the generated second image frame and the source color gamut information of the video source data to the near-eye display device 200. The source color gamut information can be transmitted through an extended transmission protocol or metadata channel, for example, by embedding the primary color coordinates and white point coordinates of the source color gamut in the auxiliary data field of the video stream transmission. After receiving the second image frame and the source color gamut information, the near-eye display device 200 performs color gamut conversion and gamma conversion processing on the second image frame.

[0075] The near-eye display device 200 pre-stores its target color gamut information. Based on the received source color gamut information and the locally stored target color gamut information, the near-eye display device 200 generates a color gamut mapping matrix. The method for generating the color gamut mapping matrix is ​​the same as that used on the signal output device 100 side. Then, the near-eye display device 200 performs color gamut conversion on each pixel in the received second image frame, mapping the pixel value from the source color gamut to the target color gamut.

[0076] In this embodiment, when the processing capacity of the signal output device 100 is limited, the color gamut mapping task can be offloaded to the near-eye display device 200, and the near-eye display device 200 can perform the color gamut mapping, thereby achieving flexible allocation of system load.

[0077] Please see Figure 4 This application also provides an image display method, which is applied to a signal output device 100. The image display method includes the following steps S301 to S304.

[0078] S301. Obtain the display configuration information output by the near-eye display device 200. The display configuration information includes the peak brightness information of the near-eye display device 200.

[0079] The signal output device 100 receives display configuration information sent by the near-eye display device 200 via a wired or wireless communication connection. The display configuration information includes the peak brightness information of the near-eye display device 200.

[0080] S302. Obtain video source data, decode the video source data, and obtain the first image frame and the dynamic metadata corresponding to the first image frame.

[0081] The signal output device 100 can receive streaming media data via wired or wireless networks, and can also read video source data from local storage media.

[0082] In one embodiment, the signal output device 100 receives streaming media data via a network, decapsulates and decodes the streaming media data to obtain video source data. For Dolby Vision format streaming media content obtained via the network, a single-layer Dolby Vision format is typically used. A single-layer Dolby Vision format only includes a base layer (BL) and a reference processing unit (RPU), and does not include an enhancement layer (EL). After decoding, the signal output device 100 directly obtains the first image frame and the corresponding RPU, without needing to perform the overlay operation of the base layer and enhancement layer. For HDR10+ format streaming media content obtained via the network, the signal output device 100 obtains the first image frame and RPU containing luminance mapping parameters after decoding.

[0083] In this embodiment, by supporting online playback of streaming media data, users can directly watch video content provided by online video platforms on the near-eye display device 200 without having to download video files in advance, thereby improving the convenience of use and the accessibility of content.

[0084] In another embodiment, the signal output device 100 reads a video source file from a local storage medium. The local storage medium can be the signal output device 100's built-in memory, an external storage device, or an optical disc medium. The signal output device 100 reads the video file through a file system interface, then decapsulates the video file, and then decodes the video encoded stream to obtain the video source data.

[0085] For Dolby Vision format video files read locally, a dual-layer Dolby Vision format can be used. Specifically, step S302 includes: acquiring video source data, decoding the video source data to obtain a base image layer and an enhanced image layer; superimposing the base image layer and the enhanced image layer to obtain a first image frame; and acquiring the dynamic metadata corresponding to the first image frame from the video source data.

[0086] The Dolby Vision video stream includes a base image layer, an enhancement image layer, and associated metadata. During decoding, the signal output device 100 decodes the video source data to obtain the base image layer and the enhancement image layer, respectively. The base image layer and the enhancement image layer are then superimposed to generate a complete first image frame (i.e., a complete HDR image frame). Simultaneously, the signal output device 100 extracts the dynamic metadata corresponding to the first image frame from the video source data. In this embodiment, by performing the superposition operation of the base layer and the enhancement layer on the signal output device 100 side, the near-eye display device 200 does not need to process the complex dual-layer Dolby Vision data structure.

[0087] S303. Perform dynamic tone mapping on the first image frame based on the display configuration information and dynamic metadata.

[0088] After obtaining the display configuration information (especially peak brightness information) of the near-eye display device 200 and the dynamic metadata corresponding to the first image frame, the signal output device 100 performs dynamic tone mapping on the first image frame based on these two types of information.

[0089] Dynamic tone mapping is the process of mapping the brightness values ​​in a high dynamic range (HDR) image to the brightness range that the target display device can display. Traditional static tone mapping uses a fixed mapping curve, which cannot be adjusted according to changes in image content, easily leading to loss of detail in bright areas or insufficient gradation in dark areas. Dynamic tone mapping, on the other hand, dynamically adjusts the mapping curve according to the brightness distribution characteristics of each frame or each scene, so that each frame can achieve better brightness presentation on the target display device.

[0090] For Dolby Vision format encoded video source data, dynamic metadata includes a luminance mapping reference curve. Step S303 includes: dynamically mapping and adjusting the luminance mapping reference curve in the dynamic metadata according to the peak luminance information to obtain a first luminance mapping curve; and performing luminance conversion pixel by pixel on the first image frame according to the first luminance mapping curve.

[0091] The brightness mapping reference curve can be a mapping curve targeting the peak brightness of a reference monitor (such as 1000 nits or 4000 nits). When the peak brightness of the target display device (i.e., the near-eye display device 200) differs from that of the reference monitor, the reference curve needs to be adjusted. For example, if the peak brightness of the reference monitor is 1000 nits, while the peak brightness of the near-eye display device 200 is 600 nits, the output range of the reference curve needs to be compressed from 0~1000 nits to 0~600 nits. This compression is not a simple linear scaling, but requires non-linear adjustment based on the visual characteristics of the human eye to maintain the perceptual consistency of brightness levels. The signal output device 100 dynamically adjusts the key nodes of the brightness mapping reference curve (such as dark inflection points, midtone slopes, and highlight roll-off points) based on the peak brightness information of the near-eye display device 200 to generate a first brightness mapping curve adapted to the near-eye display device 200. Next, the signal output device 100 iterates through each pixel of the first image frame, and based on the original brightness value of that pixel, finds the corresponding mapped brightness value on the first brightness mapping curve to complete the brightness conversion. Pixel-by-pixel processing ensures that every detail in each frame can obtain accurate brightness mapping.

[0092] In this embodiment, dynamic adjustment is performed based on the brightness mapping reference curve in the dynamic metadata, which can largely preserve the creative intent of the content creator, while adapting to the actual display capability of the near-eye display device 200 and alleviating the problem of brightness level distortion caused by simple stretching.

[0093] For HDR10+ format video source data, the dynamic metadata includes luminance mapping parameters, rather than a complete luminance mapping baseline curve. Step S303 specifically includes: generating a second luminance mapping curve corresponding to the near-eye display device 200 based on the peak luminance information and the luminance mapping parameters in the dynamic metadata; and performing luminance conversion pixel by pixel on the first image frame based on the second luminance mapping curve.

[0094] The signal output device 100 constructs a basic brightness mapping relationship based on the brightness mapping parameters in the dynamic metadata, and then adapts and adjusts this basic mapping relationship according to the peak brightness information of the near-eye display device 200 to generate a second brightness mapping curve. After generating the second brightness mapping curve, the signal output device 100 performs brightness conversion pixel by pixel on the first image frame according to the curve. In this embodiment, the brightness mapping information is transmitted in a parameterized manner, which is beneficial to improving the accuracy of dynamic tone mapping.

[0095] S304. Perform perceptual quantization nonlinear encoding on the first image frame to obtain a second image frame, and transmit the second image frame to the near-eye display device 200, so that the near-eye display device 200 performs gamma conversion on the second image frame and displays it.

[0096] The signal output device 100 performs perceptual quantization nonlinear encoding on the first image frame after dynamic tone mapping to generate a second image frame. Then, the signal output device 100 transmits the second image frame to the near-eye display device 200 via a wired or wireless connection. Upon receiving the second image frame, the near-eye display device 200 performs gamma conversion, converting the perceptual quantization non-encoded signal into the required gamma-encoded signal for display.

[0097] The image display method in this embodiment obtains the peak brightness information of the near-eye display device 200 and can perform dynamic tone mapping according to the actual capabilities of the display device, thereby alleviating the brightness distortion problem caused by directly stretching the video signal.

[0098] Please see Figure 5 In some embodiments, the display configuration information also includes target color gamut information of the near-eye display device 200. The target color gamut information can be the color gamut range that the near-eye display device 200 can cover, such as the DCI-P3 color gamut, BT.2020 color gamut, or sRGB / BT.709 color gamut. The image display method further includes steps S401 and S402, as follows: S401. Obtain the source color gamut information of the video source data.

[0099] Source color gamut information refers to the color gamut range used when the video source data was produced. There are two ways to obtain source color gamut information.

[0100] In one embodiment, step S401 includes: obtaining source color gamut information from the static metadata of the video source data. Dolby Vision video streams carry static metadata during encoding. Static metadata is a set of parameters describing the inherent attributes of the entire video content; it is fixed during the encoding stage and does not change with scene variations. The static metadata includes source color gamut information, such as primary color coordinates and white point coordinates. The signal output device 100 can obtain the source color gamut information by parsing this static metadata.

[0101] In another embodiment, step S401 includes: determining the standard preset color gamut as the source color gamut information according to the encoding format of the video source data.

[0102] Different video encoding formats typically have a fixed correspondence with specific color gamut standards. For example, if the video source data uses the BT.709 encoding format, the source color gamut can be determined to be sRGB / BT.709; if it uses the BT.2020 encoding format, the source color gamut can be determined to be BT.2020. Furthermore, for HDR video formats such as HDR10 and HDR10+, regardless of their specific encoding implementation, their color space is based on the ITU-R BT.2020 standard; therefore, the BT.2020 color gamut is also determined as the source color gamut information.

[0103] S402. Perform color gamut mapping on the first image frame data based on the target color gamut information and the source color gamut information.

[0104] Color gamut mapping is the process of converting colors from a source color gamut to a target color gamut. Different color gamuts cover different color ranges on a chromaticity diagram. For example, the BT.2020 color gamut covers a wider range of colors than the DCI-P3 color gamut, which in turn covers a wider range than the sRGB / BT.709 color gamut. Color gamut mapping is required when the color gamut of the video source data is inconsistent with that of the near-eye display device 200. For example, HDR10+ videos are typically produced using the BT.2020 color gamut, while the near-eye display device 200 may only cover the DCI-P3 color gamut. In this case, colors in the BT.2020 color gamut need to be mapped to the DCI-P3 color gamut.

[0105] Specifically, step 402 includes: generating a color gamut mapping matrix based on the target color gamut information and the source color gamut information; and performing color gamut conversion on the first image frame data based on the color gamut mapping matrix. The color gamut mapping matrix can be a 3×3 matrix used to convert the RGB values ​​of the source color gamut to the RGB values ​​of the target color gamut. In this embodiment, through color gamut mapping processing, the colors in the video source data can be converted to the target color gamut of the near-eye display device 200, alleviating the problems of color shift and saturation distortion caused by color gamut mismatch.

[0106] In this embodiment, the signal output device 100 performs complex processing such as dynamic tone mapping and color gamut mapping so that the subsequent near-eye display device 200 only needs to complete a simple gamma conversion. This achieves the display of high dynamic range video signals such as Dolby Vision without increasing the space and power consumption of the near-eye display device 200.

[0107] Please see Figure 6 This application provides a near-eye display device 200, which includes an image quality processing module 210 and a display module 220, with the image quality processing module 210 connected to the display module 220.

[0108] The display module 220 can be a micro-OLED (Micro-OLED) display, a Micro-LED display, an LCoS (Liquid Crystal on Silicon) panel, or a DLP (Digital Light Processing) chip, etc. The image processing module 210 can be an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), or other chips or circuits with image processing capabilities.

[0109] The image quality processing module 210 is connected to the display module 220. The image quality processing module 210 is used to perform gamma conversion on the second image frame output by the signal output device 100 in order to control the display module 220 to display the image. In this embodiment, the second image frame is the image frame after dynamic tone mapping and color gamut mapping processing by the signal output device 100. That is, the signal output device 100 has completed all the complex processing such as dynamic tone mapping, color gamut mapping and perceptual quantization nonlinear coding. The second image frame received by the near-eye display device 200 is a perceptual quantization nonlinear coding signal that has completed image quality adaptation.

[0110] After receiving the second image frame transmitted from the signal output device 100, the image quality processing module 210 converts the perceptual quantization nonlinear coding value of each pixel in the second image frame into a gamma domain value according to the conversion lookup table or conversion algorithm, and then drives the display module 220 to emit light according to the gamma domain value.

[0111] In this embodiment, since complex processing such as dynamic tone mapping and color gamut mapping is completed on the signal output device 100 side, the image quality processing module 210 of the near-eye display device 200 only needs to complete a simple gamma conversion operation. Therefore, the near-eye display device 200 achieves the display of high dynamic range video signals such as Dolby Vision without increasing additional space and power consumption, effectively alleviating the physical limitations of the near-eye display device 200 in terms of space and power consumption, while also achieving high dynamic range video content image quality reproduction.

[0112] In some embodiments, color gamut mapping can also be performed on the near-eye display device 200 side. In this case, the second image frame output by the near-eye display device 200 side can be an image frame that has undergone dynamic tone mapping and perceptual quantization nonlinear coding processing, but has not undergone color gamut mapping processing.

[0113] Specifically, after completing dynamic tone mapping and perceptual quantization nonlinear encoding, the signal output device 100 transmits the generated second image frame and the source color gamut information of the video source data to the near-eye display device 200. After receiving the second image frame and the source color gamut information, the near-eye display device 200 performs color gamut mapping and gamma conversion by the image quality processing module 210.

[0114] Please see Figure 7 The near-eye display device 200 has its target color gamut information for its display module 220 pre-stored in its storage module 230. The image quality processing module 210 generates a color gamut mapping matrix based on the received source color gamut information and the locally stored target color gamut information. Then, the image quality processing module 210 performs color gamut conversion on each pixel in the received second image frame, mapping the pixel value from the source color gamut to the target color gamut.

[0115] It should be noted that the processing order of the gamma conversion and color gamut mapping processes in the near-eye display device 200 can be set according to actual needs. Alternatively, the image quality processing module 210 can first perform color gamut mapping on the second image frame to obtain a perceptually quantized nonlinear coded image frame after color gamut conversion, and then perform gamma conversion on this image frame. After gamma conversion, the image quality processing module 210 outputs the pixel values ​​in the gamma domain to the display module 220, which then emits light according to the driving current or voltage corresponding to the gamma domain values. Alternatively, the image quality processing module 210 can first perform gamma conversion on the second image frame, converting the perceptually quantized nonlinear coded values ​​into gamma domain values, then perform color gamut mapping on the pixel values ​​in the gamma domain, and finally drive the display module 220 to display the image.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0117] The near-eye display device and image display method provided in the embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the technical solutions and core ideas of this application. Those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An image display method, characterized in that, The image display method is applied to a display system, the display system including a signal output device and a near-eye display device, and the image display method includes: The near-eye display device sends display configuration information to the signal output device, the display configuration information including the peak brightness information of the near-eye display device; The signal output device acquires video source data, decodes the video source data, and obtains a first image frame and dynamic metadata corresponding to the first image frame. The signal output device performs dynamic tone mapping on the first image frame according to the display configuration information and the dynamic metadata; The signal output device performs perceptual quantization nonlinear encoding on the first image frame to obtain a second image frame, and transmits the second image frame to the near-eye display device; The near-eye display device performs gamma conversion on the second image frame and displays it.

2. The image display method according to claim 1, characterized in that, The steps for obtaining the video source data include: The signal output device receives streaming media data, decapsulates and decodes the streaming media data to obtain the video source data.

3. The image display method according to claim 1, characterized in that, The steps of acquiring video source data by the signal output device, decoding the video source data, and obtaining a first image frame and dynamic metadata corresponding to the first image frame include: The signal output device acquires video source data, decodes the video source data, and obtains a basic image layer and an enhanced image layer. The signal output device superimposes the base image layer and the enhanced image layer to obtain the first image frame; The signal output device obtains the dynamic metadata corresponding to the first image frame from the video source data.

4. The image display method according to claim 1, characterized in that, The dynamic metadata includes a brightness mapping baseline curve; The step of the signal output device performing dynamic tone mapping on the first image frame based on the display configuration information and the dynamic metadata includes: The signal output device dynamically adjusts the brightness mapping reference curve in the dynamic metadata according to the peak brightness information to obtain a first brightness mapping curve. The signal output device performs pixel-by-pixel brightness conversion on the first image frame according to the first brightness mapping curve.

5. The image display method according to claim 1, characterized in that, The dynamic metadata includes brightness mapping parameters; The step of the signal output device performing dynamic tone mapping on the first image frame based on the display configuration information and the dynamic metadata includes: The signal output device generates a second brightness mapping curve corresponding to the near-eye display device based on the peak brightness information and the brightness mapping parameters in the dynamic metadata; The signal output device performs pixel-by-pixel brightness conversion on the first image frame according to the second brightness mapping curve.

6. The image display method according to claim 1, characterized in that, The display configuration information also includes the target color gamut information of the near-eye display device; The image display method further includes: The signal output device acquires the source color gamut information of the video source data; The signal output device performs color gamut mapping on the first image frame data based on the target color gamut information and the source color gamut information.

7. The image display method according to claim 6, characterized in that, The steps for the signal output device to acquire the source color gamut information of the video source data include: The signal output device obtains the source color gamut information from the static metadata of the video source data.

8. The image display method according to claim 6, characterized in that, The steps for the signal output device to acquire the source color gamut information of the video source data include: The signal output device determines the standard preset color gamut as the source color gamut information according to the encoding format of the video source data.

9. The image display method according to claim 6, characterized in that, The step of the signal output device performing color gamut mapping on the first image frame data based on the target color gamut information and the source color gamut information includes: The signal output device generates a color gamut mapping matrix based on the target color gamut information and the source color gamut information; The signal output device performs color gamut conversion on the first image frame data according to the color gamut mapping matrix.

10. An image display method, characterized in that, The image display method is applied to a signal output device, and the image display method includes: Obtain display configuration information output by the near-eye display device, wherein the display configuration information includes the peak brightness information of the near-eye display device; Acquire video source data, decode the video source data to obtain a first image frame and dynamic metadata corresponding to the first image frame; Dynamic tone mapping is performed on the first image frame based on the display configuration information and the dynamic metadata; The first image frame is subjected to perceptual quantization nonlinear encoding to obtain a second image frame, and the second image frame is transmitted to the near-eye display device, so that the near-eye display device performs gamma conversion on the second image frame and displays it.

11. The image display method according to claim 10, characterized in that, The steps for obtaining the video source data include: Receive streaming media data, decapsulate and decode the streaming media data to obtain the video source data.

12. The image display method according to claim 10, characterized in that, The steps of acquiring video source data, decoding the video source data, and obtaining a first image frame and dynamic metadata corresponding to the first image frame include: Acquire video source data, decode the video source data to obtain a base image layer and an enhanced image layer; The first image frame is obtained by superimposing the base image layer and the enhanced image layer; Obtain the dynamic metadata corresponding to the first image frame from the video source data.

13. The image display method according to claim 10, characterized in that, The dynamic metadata includes a brightness mapping baseline curve; The step of performing dynamic tone mapping on the first image frame based on the display configuration information and the dynamic metadata includes: Based on the peak brightness information, the brightness mapping reference curve in the dynamic metadata is dynamically mapped and adjusted to obtain the first brightness mapping curve; The brightness of the first image frame is converted pixel by pixel according to the first brightness mapping curve.

14. The image display method according to claim 10, characterized in that, The dynamic metadata includes brightness mapping parameters; The step of performing dynamic tone mapping on the first image frame based on the display configuration information and the dynamic metadata includes: Based on the peak brightness information and the brightness mapping parameters in the dynamic metadata, a second brightness mapping curve corresponding to the near-eye display device is generated. The first image frame is subjected to pixel-by-pixel brightness conversion based on the second brightness mapping curve.

15. The image display method according to claim 10, characterized in that, The display configuration information also includes the target color gamut information of the near-eye display device; The image display method further includes: Obtain the source color gamut information of the video source data; Color gamut mapping is performed on the first image frame data based on the target color gamut information and the source color gamut information.

16. The image display method according to claim 15, characterized in that, The step of obtaining the source color gamut information of the video source data includes: The source color gamut information is obtained from the static metadata of the video source data.

17. The image display method according to claim 15, characterized in that, The step of obtaining the source color gamut information of the video source data includes: Based on the encoding format of the video source data, the standard preset color gamut is determined as the source color gamut information.

18. The image display method according to claim 15, characterized in that, The step of performing color gamut mapping on the first image frame data based on the target color gamut information and the source color gamut information includes: A color gamut mapping matrix is ​​generated based on the target color gamut information and the source color gamut information; The first image frame data is subjected to color gamut conversion based on the color gamut mapping matrix.

19. A near-eye display device, characterized in that, The near-eye display device includes: Display module; An image quality processing module is connected to the display module and is used to perform gamma conversion on the second image frame in the image display method according to any one of claims 10 to 18 in order to control the display module to display.

Images