Image color conversion method, video coding device, and computer storage medium

Abstract

The application provides an image color conversion method, a video coding device and a computer storage medium. The image color conversion method comprises the following steps: acquiring a code stream encoded according to a security monitoring digital video and audio coding standard; analyzing video display metadata in the code stream; and performing color conversion on decoded image data based on the video display metadata. The application can improve the color conversion efficiency and display efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image color conversion method, a video encoding and decoding device, and a computer storage medium. Background Technology

[0002] Video image data is relatively large, so it is usually necessary to compress the video pixel data (RGB, YUV, etc.). The compressed data is called a bitstream, which is transmitted to the user's end via wired or wireless network for decoding and viewing. The entire image processing flow includes color conversion at the encoding end, encoding, decoding, and color conversion at the decoding end.

[0003] During the long-term research and development process, the inventors of this application discovered that current image color conversion methods still have certain limitations, which also affect display efficiency to some extent. Summary of the Invention

[0004] To address the aforementioned technical problems, this application proposes an image color conversion method, a video encoding / decoding device, and a computer storage medium.

[0005] To address the aforementioned technical problems, this application proposes an image color conversion method, which includes:

[0006] Acquire the bitstream encoded using the digital audio and video encoding / decoding technology standard for security monitoring;

[0007] Parse the video display metadata in the bitstream;

[0008] The decoded image data is color-converted based on the video display metadata.

[0009] To address the aforementioned technical problems, this application proposes another image color conversion method, which includes:

[0010] Acquire image data;

[0011] The image data is color-converted to obtain the image data to be encoded;

[0012] The image data to be encoded is encoded into a bitstream using the security and monitoring digital audio and video encoding and decoding technology standard, and the video display metadata used in the color conversion process of the image data is encoded into the bitstream.

[0013] To address the aforementioned technical problems, this application also proposes a video encoding and decoding apparatus, which includes a memory and a processor coupled to the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the image color conversion method described above.

[0014] To address the aforementioned technical problems, this application also proposes a computer storage medium for storing program data, which, when executed by a computer, is used to implement the aforementioned image color conversion method.

[0015] Compared with the prior art, the beneficial effects of this application are: acquiring a bitstream encoded with the security monitoring digital audio-visual codec technology standard; parsing the video display metadata in the bitstream; and performing color conversion on the decoded image data based on the video display metadata. This defines the color characteristics of the source data, ensuring correct display on different devices. It also makes up for the lack of color space management in the security monitoring digital audio-visual codec technology standard, improving the dynamic range performance of video / images and thus improving display efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0017] Figure 1 This is a schematic diagram of an embodiment of the color conversion process provided in this application;

[0018] Figure 2 This is a schematic diagram of another embodiment of the color conversion process provided in this application;

[0019] Figure 3 This is a flowchart illustrating one embodiment of the image color conversion method provided in this application;

[0020] Figure 4 This is a flowchart illustrating another embodiment of the image color conversion method provided in this application;

[0021] Figure 5 This is a flowchart illustrating another embodiment of the image color conversion method provided in this application;

[0022] Figure 6 This is a flowchart illustrating another embodiment of the image color conversion method provided in this application;

[0023] Figure 7 This is a schematic diagram of the structure of an embodiment of the video encoding and decoding apparatus provided in this application;

[0024] Figure 8 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0026] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] Please see Figure 1 , Figure 1 This is a schematic diagram of an embodiment of image processing provided in this application. For example... Figure 1 As shown, the overall process of image processing is as follows: the encoding end starts from the acquisition of light information and ends at the end of the bit stream, while the decoding end starts from the beginning of the bit stream and ends at the display of non-linear RGB.

[0028] Furthermore, for technologies related to high bit depth (e.g., 10-bit) HDR streams, the image processing procedure is similar to the above process, with the addition of a tone mapping process after the electro-optical conversion, such as... Figure 2 As shown.

[0029] Tone mapping maps the colors and brightness of high dynamic range (HDR) content to low dynamic range (LDR) displays to ensure that the colors are not distorted, while inverse tone mapping is the opposite process.

[0030] Color mapping methods are divided into global, local, and temporal domains. The mapping process is divided into luminance mapping and color mapping. Luminance mapping requires information such as the maximum display brightness of the display device, the minimum display brightness of the display device, the maximum brightness of the displayed content, and the maximum average brightness of the displayed content. Color mapping requires information such as the x-coordinates of the three primary colors of the display device, the y-coordinates of the three primary colors of the display device, the x-coordinates of the standard white light of the display device, and the y-coordinates of the standard white light of the display device.

[0031] However, the Security and Surveillance Digital Audio-Visual Codec (SVAC) standard does not specify any color space, making it impossible to accurately describe the color characteristics of images / videos. This results in inaccurate display in actual products and affects the decoding and display effect.

[0032] Based on this, this application proposes an image color conversion method. This method transmits video display metadata through the bitstream encoded by the digital audio-visual codec technology standard for security monitoring. The standard makes relevant provisions for color space, accurately describes the color characteristics of the image or video through the bitstream, and enables the decoding end to accurately display the corresponding image or video based on the color characteristics of the image or video in the bitstream, thereby improving the image or video display effect of the decoding end.

[0033] Please refer to the details. Figure 3 , Figure 3 This is a schematic flowchart of the first embodiment of the image color conversion method provided in this application. It is understood that the image color conversion method of this application can be applied to image processing, or can be applied to image processing in general.

[0034] The image color conversion method of this application is applied to a video encoding and decoding device, wherein the video encoding and decoding device can be a server, a terminal device, or a system in which the server and the terminal device cooperate with each other. Accordingly, the various parts of the video encoding and decoding device, such as various units, sub-units, templates, and sub-templates, can all be set in the server, all in the terminal device, or separately in the server and the terminal device.

[0035] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software templates, such as software or software templates used to provide distributed servers, or as a single software program or software template; no specific limitations are made here.

[0036] like Figure 3 As shown, the overall implementation process of the image color conversion method provided in this application is as follows:

[0037] Step S11: Obtain the bitstream encoded using the security monitoring digital audio and video codec technology standard.

[0038] In this embodiment, after the encoding end encodes the image or video using the security monitoring digital audio-visual codec technology standard, the decoding end can obtain the bitstream encoded by the security monitoring digital audio-visual codec technology standard from the encoding end.

[0039] The encoding end can be a device capable of encoding images or videos according to the digital audio-visual codec standard for security monitoring, such as an image acquisition device. The decoding end is unrestricted, as long as it can obtain the bitstream encoded according to the digital audio-visual codec standard for security monitoring from the encoding end. In one implementation, the encoding end and the decoding end can be the same device; however, they can also be different devices.

[0040] Step S12: Parse the video display metadata in the bitstream.

[0041] This application transmits video display metadata via a bitstream encoded using the security monitoring digital audio-visual codec technology standard. Thus, at the decoding end, the bitstream encoded using the security monitoring digital audio-visual codec technology standard can be obtained, and the video display metadata in the bitstream can be parsed so that the decoded image data or video data can be color-converted using the video display metadata in the bitstream.

[0042] Among them, video display metadata is used to specify the color space of an image or video, so that the display characteristics of an image or video can be described through video display metadata.

[0043] Optionally, the video display metadata may include, but is not limited to, at least one of sequence display information and target display device and content metadata extension information.

[0044] The sequence display information may include, but is not limited to, at least one of the following: video format, sample range, color information description, color three primary colors, photoelectric transfer characteristics, color signal conversion matrix, horizontal display size, and vertical display size.

[0045] The sequence display information can be specified at the sequence level in the video. Of course, in other embodiments, the sequence display information can also be specified at other levels such as the image level in the video.

[0046] Sequence display information can be transmitted in supplemental enhancement payload, extended data, and sequence parameter sets, meaning the decoding end can parse the video sequence display information from the supplemental enhancement payload, extended data, and sequence parameter sets in the bitstream.

[0047] In one implementation, the sequence display information can be parsed from the supplementary enhanced load of a first preset load type in the bitstream. The first preset load type can be set according to actual conditions and is not limited here.

[0048] The following two examples will describe the sequence display information in the bitstream by specifying the sequence display information at the sequence level in the video and transmitting the sequence display information through supplementary enhanced payload.

[0049] In a specific example, a new payload type (payload type 27 in the table below) is added to the supplemental enhanced payload syntax. This new payload type uses the sequence of transmitted information to mark the video's color space attributes. The specific syntax and semantics are as follows (the bolded parts are the modifications):

[0050]

[0051]

[0052] Here, `sei_payload` refers to the function used to parse the supplementary and enhanced payload. `sei_payload(PayloadType, PayloadSize)` determines the content type of the SEI (i.e., supplementary and enhanced payload) based on `PayloadType` (payload type), calls the corresponding processing logic, and ensures that the data is byte-aligned.

[0053] As shown above, when the load type is 3, the processing logic corresponding to gmv_extension (i.e., encoding anti-jitter extension) is called; when the load type is 8, the processing logic corresponding to roi_parameters (i.e., region of interest parameters) is called; when the load type is 16, the processing logic corresponding to user_data (i.e., user data) is called; when the load type is 17, the processing logic corresponding to temporal_scalability_info (temporal domain hierarchical information) is called; when the load type is 18, the processing logic corresponding to air_parameter (i.e., adaptive intra-frame refresh parameters) is called; when the load type is 19, the processing logic corresponding to seq_privacy_parameters (i.e., sequence-level privacy protection regions) is called; and when the load type is 20... When the load type is 25, the processing logic corresponding to pic_privacy_parameters (i.e., image-level privacy protection area) is called; when the load type is 26, the processing logic corresponding to layer_info (i.e., layer information) is called; when the load type is 26, the processing logic corresponding to library_display_info (i.e., display knowledge image display information) is called; when the load type is 27, the processing logic corresponding to seq_display_info (i.e., sequence display information) is called; when the load type is 127, the processing logic corresponding to version_info (i.e., version information) is called; when the load type is any other than the above load types, the processing logic corresponding to reserved_sei_message (i.e., retain supplementary enhancement information) is called.

[0054] `byte_aligned` refers to bit alignment. `if(!byte_aligned())` checks if the current bitstream is not aligned to a byte boundary. If the current bitstream is not aligned to a byte boundary, `bit_equal_to_one` (i.e., write a binary bit 1); `while(!byte_aligned())` (loops to check if it is still not aligned to a byte boundary), if not aligned, `bit_equal_to_zero` (continues to fill with 0s).

[0055] The sequence display information payload definition is shown in the table below. As mentioned above, in this example, the sequence display information is specified at the sequence level in the video, so that seq_display_info (i.e., sequence display information) can be transmitted in the access units of random access point images and non-display knowledge images.

[0056]

[0057]

[0058] In another specific example, a new payload type (i.e., the payload type of else in the table below) is added to the supplementary enhanced payload syntax. The color space attributes of the video are marked by the sequence of information transmitted in the new payload type. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0059]

[0060] Slightly different from the previous example, this example does not add a new load type of 27. Instead, it calls the processing logic for the sequence display information through the other load types (i.e., load types other than 3, 8, 16, 17, 18, 19, 20, 25, 26, and 127).

[0061] The sequence display information payload definition for this example is shown in the table below. As described above, in this example, the sequence display information is specified at the sequence level in the video, so that seq_display_info (i.e., sequence display information) can be transmitted in the access units of random access point images and non-display knowledge images.

[0062]

[0063]

[0064] In another implementation, sequence display information can be parsed from the extended data of the first preset extended data type in the bitstream. The first preset extended data type can be set according to the actual situation and is not restricted here.

[0065] The following two examples will describe the sequence display information in the bitstream by specifying the sequence display information at the sequence level in the video and by displaying the sequence display information through extended data transmission.

[0066] In a specific example, an extended data type (i.e., the 0xe5 extended data type in the table below) is added to the extended data. The color space attributes of the video are marked by the sequence display information transmitted in the newly added extended data type. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0067]

[0068] `surveillance_extension_rbsp` is a function used to parse extended data. `surveillance_extension_stop_byte` refers to the stop byte for monitoring extended information. `while(next_bits(8)!=0x80){}` means that while parsing extended data, `surveillance_extension_stop_byte` means that the extended data is continuously parsed until the stop flag 0x80 is detected.

[0069] next_bits(8) refers to the next 8 bits of the returned bit stream. During the parsing of extended data, if the following n bits of the returned bitstream are equal to hexadecimal 0x04, then `time_extension()` is called to handle absolute time information extension; if the following n bits of the returned bitstream are equal to hexadecimal 0x10, then `gis_extension()` is called to handle geographic information extension; if the following n bits of the returned bitstream are equal to hexadecimal 0x12, then `osd_extension()` is called to handle screen display information extension; if the following n bits of the returned bitstream are equal to hexadecimal 0xe1, then `analysis_extension()` is called to handle video structured information extension; if the following n bits of the returned bitstream are equal to hexadecimal 0xe3, then `ftm_extension()` is called to handle feature tensor map information extension; if the following n bits of the returned bitstream are equal to hexadecimal 0xe5, then `dsp_extension()` is called to handle sequence display information; if none of the above conditions are matched, `reserved_extension()` is executed to handle reserved or undefined extension types.

[0070] The extended data definition for sequence display information is shown in the table below.

[0071]

[0072]

[0073] The extended data type `extension_id` can be an 8-bit unsigned integer. The table number `extension_id` for sequence display information can be equal to 0xe5.

[0074] The extension length (extension_length) can be an 8-bit unsigned integer, which represents the length of the current extension syntax element following extension_length, in bytes.

[0075] In another specific example, an extended data type (i.e., the extended data type of else in the table below) is added to the extended data. The color space attributes of the video are marked by the sequence display information transmitted in the newly added extended data type. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0076]

[0077] Slightly different from the previous example, this example does not add a new extended data type 0xe5. Instead, it calls the processing logic for the sequence display information through the other extended data types (i.e., the extended data types other than 0x04, 0x10, 0x12, 0xe1, and 0xe3).

[0078] The extended data definition for the sequence display information in this example is shown in the table below.

[0079]

[0080]

[0081] The extended data type `extension_id` can be an 8-bit unsigned integer. The table number `extension_id` for sequence display information can be equal to 0xe5.

[0082] The extension length (extension_length) can be an 8-bit unsigned integer, which represents the length of the current extension syntax element following extension_length, in bytes.

[0083] In another implementation, sequence display information can be parsed from the sequence parameter set in the bitstream.

[0084] The following example describes the sequence display information in the bitstream by specifying the sequence display information at the sequence level in the video and transmitting the sequence display information through a sequence parameter set.

[0085] In a specific example, a syntactic representation (i.e., vui_parameters_present_flag in the following table) is added to the sequence parameter set, and the color space attribute of a video is marked by the newly added syntactic representation. Specific syntax and semantic provisions are as follows (the bolded part is the modified part):

[0086]

[0087] seq_parameter_set_rbsp() is a parsing function for a Sequence Parameter Set (SPS), which is used to define global coding parameters of a video sequence.

[0088] NumOfLayers represents the number of layers, and indicates the layered coding structure (e.g., spatial scalability) of a video. for(i=0;i<NumOfLayers;i++) refers to circularly processing patch parameters of each coding layer.

[0089] stable_patch_flag is a consistency flag for tile partitioning.

[0090] uniform_patch_flag is a uniform tile size flag.

[0091] patch_width_minus1 and patch_height_minus1 are tile width minus one and tile height minus one respectively.

[0092] vui_parameters_present_flag is video usability information (VUI), which is used to define display parameters of a video (e.g., aspect ratio, frame rate, color space).

[0093] reserved_bits are reserved bits for future expansion.

[0094] rbsp_trailing_bits indicates the end of a raw byte sequence payload byte stream.

[0095] When the video availability information presence flag vui_parameters_present_flag is equal to 1, it indicates that the availability information parameter vui_parameters() syntax structure exists. When vui_parameters_present_flag is equal to 0, it indicates that the vui_parameters() syntax structure does not exist.

[0096] Availability information parameters can be defined as shown in the following table.

[0097]

[0098] In the examples of the sequence display information above, the video format `video_format` can be a 3-bit unsigned integer, which describes the format of the video before it was encoded according to this file, as shown in the table below. If no sequence display extension appears in the bitstream, the video format can be assumed to be "unspecified video format".

[0099]

[0100] PAL stands for Phase Alternating Line.

[0101] NTSC stands for National Television Standards Committee.

[0102] SECAM stands for Séquentiel couleuràmémoire (Secam system).

[0103] MAC stands for Multiplexed Analogue Components.

[0104] The sample range (sample_range) can be a binary variable that describes the range of luminance and chrominance signal samples. If sample_range does not exist in the bitstream, its value is '0'.

[0105] The color information description `colour_description` can be a binary variable. A value of '1' indicates that the bit stream contains `colour_primaries`, `transfer_characteristics`, and `matrix_coefficients`; a value of '0' indicates that the bit stream does not contain `colour_primaries`, `transfer_characteristics`, and `matrix_coefficients`.

[0106] The color primary colors (colour_primaries) can be 8-bit unsigned integers. They specify the chromaticity coordinates of the three primary colors in the source image. Specific rules are shown in the table below.

[0107]

[0108]

[0109] If there is no sequence display information in the bitstream, or the value of colour_description is '0', it is assumed that the chroma is implicitly defined by the application itself.

[0110] The photoelectric transfer characteristics (transfer_characteristics) can be an 8-bit unsigned integer. It describes the photoelectric transfer characteristics of the source image, as specifically defined in the table below. In the table, Lc is the linear input signal, proportional to the light intensity.

[0111]

[0112]

[0113] If there is no sequence display information in the bit stream, or the value of colour_description is '0', it is assumed that the photoelectric transfer characteristics are implicitly defined by the application itself.

[0114] The color signal conversion matrix matrix_coefficients can be an 8-bit unsigned integer, which specifies the conversion matrix used when converting from the three primary colors (red, green, and blue) to luminance and chrominance signals. The specific specifications are shown in the table below.

[0115]

[0116] In the table above:

[0117] E' Y It is an analog quantity with a value between 0 and 1;

[0118] E' CB and E' CR It is an analog quantity with a value between -0.5 and 0.5;

[0119] E' R 、E' G and E' B It is an analog quantity with a value between 0 and 1;

[0120] Y, Cb and Cr with E' Y 、E' CB and E' CR The relationship is as follows:

[0121] If the value of sample_range is '0':

[0122]

[0123] If the value of sample_range is '1':

[0124]

[0125] BitDetph represents the precision of the encoded sample. For example:

[0126] When BitDepth = 8 and sample_range is '0':

[0127]

[0128] The value range of Y is 16 to 235, and the value ranges of Cb and Cr are 16 to 240.

[0129] When BitDepth = 8 and sample_range is '1':

[0130]

[0131] The values ​​of Y, Cb, and Cr all range from 0 to 255.

[0132] The decoding process can limit the sample values ​​of the output Y, Cb, and Cr to 0–2 BitDepth⁻¹. If there is no sequence display extension in the bitstream, or if the value of colour_description is '0', it is assumed that the transformation matrix is ​​implicitly defined by the application itself. Some applications may have multiple different video signals, and these different video signals may have different color gradations, transfer characteristics, and / or transformation matrices. In this case, it is recommended that the application first convert these different parameter sets to a unified parameter set.

[0133] The horizontal display size `display_horizontal_size` can be a 14-bit unsigned integer. The vertical display size `display_vertical_size` can also be a 14-bit unsigned integer. Together, they define a rectangle. If the size of this rectangle is smaller than the size of the encoded image, it is advisable to display only a portion of the encoded image; if the size of the rectangle is larger than the size of the encoded image, it is advisable to display only a portion of the reconstructed image on the display device. The unit of `display_horizontal_size` should be the number of samples per row of the encoded image. The unit of `display_vertical_size` should be the number of rows of the encoded image. `display_horizontal_size` and `display_vertical_size` have no effect on the decoding process. They can be used by the display process.

[0134] In addition, the target display device and content metadata extension information may include, but is not limited to, at least one of the following: display device three primary color X coordinates, display device three primary color Y coordinates, display device standard white light X coordinates, display device standard white light Y coordinates, display device maximum display brightness, display device minimum display brightness, display content maximum brightness, and display content maximum image average brightness.

[0135] The target display device and content metadata extension information can be specified at the sequence level in the video. Of course, in other embodiments, the target display device and content metadata extension information can also be specified at other levels, such as the image level, in the video.

[0136] The target display device and content metadata extension information can be transmitted in supplemental enhancement load or extended data, that is, the decoding end can parse the video sequence display information from the supplemental enhancement load or extended data in the bitstream.

[0137] In one implementation, the target display device and content metadata extension information can be parsed from the supplementary enhanced load of a second preset load type in the bitstream. The second preset load type can be set according to actual conditions and is not limited here.

[0138] The following two examples will illustrate the use of supplementary enhanced payload transmission of target display device and content metadata extension information to describe the target display device and content metadata extension information in the bitstream.

[0139] In a specific example, a new payload type (payload type 28 in the table below) is added to the supplementary enhanced payload syntax. This new payload type uses the target display device and content metadata extension information transmitted to mark the display capabilities and video content characteristics of the target display device. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0140]

[0141] Here, `sei_payload` refers to the function used to parse the supplementary and enhanced payload. `sei_payload(PayloadType, PayloadSize)` determines the content type of the SEI (i.e., supplementary and enhanced payload) based on `PayloadType` (payload type), calls the corresponding processing logic, and ensures that the data is byte-aligned.

[0142] As shown above, when the load type is 3, the processing logic corresponding to `gmv_extension` (i.e., encoding stabilization extension) is called; when the load type is 8, the processing logic corresponding to `roi_parameters` (i.e., region of interest parameters) is called; when the load type is 16, the processing logic corresponding to `user_data` (i.e., user data) is called; when the load type is 17, the processing logic corresponding to `temporal_scalability_info` (temporal domain hierarchical information) is called; when the load type is 18, the processing logic corresponding to `air_parameter` (i.e., adaptive intra-frame refresh parameters) is called; when the load type is 19, the processing logic corresponding to `seq_privacy_parameters` (i.e., sequence-level privacy protection regions) is called; and when the load type is 20, the processing logic corresponding to `pic_privacy_p` is called. The processing logic for arameters (i.e., image-level privacy protection areas) is called; when the load type is 25, the processing logic for layer_info (i.e., layer information) is called; when the load type is 26, the processing logic for version_info (i.e., display knowledge image display information) is called; when the load type is 28, the processing logic for mastering_display_and_content_metadata_extension (i.e., target display device and content metadata extension information) is called; when the load type is 127, the processing logic for version_info (i.e., version information) is called; and when the load type is any other than the above load types, the processing logic for reserved_sei_message (i.e., reserved supplementary enhancement information) is called.

[0143] The `if(!byte_aligned())` method checks if the current bitstream is not aligned to a byte boundary. If it is, `bit_equal_to_one` (writes a binary bit 1); `while(!byte_aligned())` (loops to check if it is still not aligned to a byte boundary), and if not, `bit_equal_to_zero` (continues to fill with 0s).

[0144] The target device display and content metadata extension definitions are shown in the table below. `mastering_display_and_content_metadata_extension()` can only appear in access units of random access point images and non-display knowledge images.

[0145]

[0146] The x-coordinates (display_primaries_x[c]) and y-coordinates (display_primaries_y[c]) of the display device's three primary colors can be 16-bit unsigned integers, representing the normalized x- and y-coordinates of the display device's three primary colors, respectively. These coordinates conform to ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000. Values ​​of c of 0, 1, and 2 correspond to green, blue, and red, respectively.

[0147] The standard white light X-coordinate (white_point_x) and Y-coordinate (white_point_y) of the display device can be 16-bit unsigned integers, representing the normalized chromaticity x-coordinate and y-coordinate of the standard white light of the display device, respectively. These coordinates conform to the specifications in ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000.

[0148] The maximum display brightness (max_display_mastering_luminance) of a display device can be a 16-bit unsigned integer, representing the maximum display brightness of the device. (In 1 cd / m²) 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0149] The minimum display brightness of a display device, min_display_mastering_luminance, can be a 16-bit unsigned integer representing the minimum display brightness of the device, and it can be expressed in increments of 0.0001 cd / m². 2 The units range from 0.0001 cd / m 2 Up to 6.5535 cd / m 2 .

[0150] Optionally, the value of max_display_mastering_luminance can be greater than the value of min_display_mastering_luminance.

[0151] The maximum brightness of the displayed content (max_content_light_level) can be a 16-bit unsigned integer, representing the maximum brightness of the displayed content, expressed in units of 1 cd / m². 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0152] The value of `max_content_light_level` is the maximum value of the `PictureMaxLightLevel` of all displayed images for a given content. The maximum brightness of the displayed images, `PictureMaxLightLevel`, is calculated as follows:

[0153] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0154] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0155] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0156] The PictureMaxLightLevel of the displayed image is equal to the maximum value of maxRGB for all pixels within the effective display area.

[0157] `max_picture_average_light_level` can be a 16-bit unsigned integer representing the maximum average brightness of the displayed content. (In 1 cd / m²) 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0158] The value of `max_picture_average_light_level` is the maximum value of the average brightness (`PictureAverageLightLevel`) of all displayed images for a given content. The average brightness (`PictureAverageLightLevel`) is calculated as follows:

[0159] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0160] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0161] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0162] —The PictureAverageLightLevel of the displayed image is equal to the average of the maxRGB values ​​of all pixels within the effective display area.

[0163] In another specific example, a new payload type (i.e., the payload type in the else section of the table below) is added to the supplementary enhanced payload syntax. This new payload type uses the target display device and content metadata extension information transmitted to mark the display capabilities and video content characteristics of the target display device. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0164]

[0165]

[0166] Slightly different from the previous example, this example does not add a new load type of 28. Instead, it calls the target device's display and content metadata processing logic through the other load types (i.e., load types other than 3, 8, 16, 17, 18, 19, 20, 25, 26, and 127).

[0167] The target device display and content metadata extension definitions for this example are shown in the table below. `mastering_display_and_content_metadata_extension()` can only appear in access units of random access point images and non-display knowledge images.

[0168]

[0169] The x-coordinates (display_primaries_x[c]) and y-coordinates (display_primaries_y[c]) of the display device's three primary colors can be 16-bit unsigned integers, representing the normalized x- and y-coordinates of the display device's three primary colors, respectively. These coordinates conform to ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000. Values ​​of c of 0, 1, and 2 correspond to green, blue, and red, respectively.

[0170] The standard white light X-coordinate (white_point_x) and Y-coordinate (white_point_y) of the display device can be 16-bit unsigned integers, representing the normalized chromaticity x-coordinate and y-coordinate of the standard white light of the display device, respectively. These coordinates conform to the specifications in ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000.

[0171] The maximum display brightness of a display device, max_display_mastering_luminance, can be a 16-bit unsigned integer representing the maximum display brightness of the device, and it can be expressed in terms of 1 cd / m². 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0172] The minimum display brightness of a display device, min_display_mastering_luminance, can be a 16-bit unsigned integer representing the minimum display brightness of the device, and it can be expressed in increments of 0.0001 cd / m². 2 The units range from 0.0001 cd / m 2 Up to 6.5535 cd / m 2 .

[0173] Optionally, the value of max_display_mastering_luminance can be greater than the value of min_display_mastering_luminance.

[0174] The maximum brightness of the displayed content (max_content_light_level) can be a 16-bit unsigned integer, representing the maximum brightness of the displayed content, expressed in units of 1 cd / m². 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0175] The value of `max_content_light_level` is the maximum value of the `PictureMaxLightLevel` of all displayed images for a given content. The maximum brightness of the displayed images, `PictureMaxLightLevel`, is calculated as follows:

[0176] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0177] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0178] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0179] —The PictureMaxLightLevel of the displayed image is equal to the maximum value of maxRGB for all pixels within the effective display area.

[0180] `max_picture_average_light_level` can be a 16-bit unsigned integer representing the maximum average brightness of the displayed content. (In 1 cd / m²) 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0181] The value of `max_picture_average_light_level` is the maximum value of the average brightness (`PictureAverageLightLevel`) of all displayed images for a given content. The average brightness (`PictureAverageLightLevel`) is calculated as follows:

[0182] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0183] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0184] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0185] —The PictureAverageLightLevel of the displayed image is equal to the average of the maxRGB values ​​of all pixels within the effective display area.

[0186] In another implementation, the target display device and content metadata extension information can be parsed from the extended data of the second preset extended data type in the bitstream. The second preset extended data type can be set according to actual conditions and is not limited here.

[0187] The following two examples will illustrate the use of extended data transmission target display device and content metadata extension information to describe the target display device and content metadata extension information in the bitstream.

[0188] In a specific example, an extended data type (i.e., the 0xe7 extended data type in the table below) is added to the extended data. The target display device and content metadata extended information transmitted in the newly added extended data type are used to mark the display capabilities and video content characteristics of the target display device. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0189]

[0190] surveillance_extension_rbsp is a function used to parse extended data. while(next_bits(8)!=0x80){}surveillance_extension_stop_byte means that while parsing extended data, the extended data is continuously parsed until the stop flag 0x80 is detected.

[0191] `next_bits(8)` refers to the next n bits of the returned bit stream. During the parsing of extended data, if the next n bits of the returned bit stream equal hexadecimal 0x04, then `time_extension()` is called to handle absolute time information extension; if the next n bits of the returned bit stream equal hexadecimal 0x10, then `gis_extension()` is called to handle geographic information extension; if the next n bits of the returned bit stream equal hexadecimal 0x12, then `osd_extension()` is called to handle screen display information extension; if the next n bits of the returned bit stream equal hexadecimal 0xe1, then `analysis_extension()` is called. `ion()` handles video structured information extensions; if the next n bits of the returned bitstream are equal to hexadecimal 0xe3, then `ftm_extension()` is called to handle feature tensor map information extensions; if the next n bits of the returned bitstream are equal to hexadecimal 0xe7, then `mastering_display_and_content_metadata_extension()` is called to handle target device display and content metadata extensions; if none of the above conditions are matched, `reserved_extension()` is executed to handle reserved or undefined extension types.

[0192] The target device display and content metadata extension definitions are shown in the table below.

[0193]

[0194] The video extension label extension_id can be an 8-bit unsigned integer, which can identify the target display device and the table number of the content metadata extension, extension_id should be equal to 0xe7.

[0195] The X-coordinates (display_primaries_x[c]) and Y-coordinates (display_primaries_y[c]) of the display device's three primary colors can be 16-bit unsigned integers, representing the normalized x- and y-coordinates of the display device's three primary colors, respectively. These coordinates conform to ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000. Values ​​of c of 0, 1, and 2 correspond to green, blue, and red, respectively.

[0196] The standard white light X-coordinate (white_point_x) and Y-coordinate (white_point_y) of the display device can be 16-bit unsigned integers, representing the normalized chromaticity x-coordinate and y-coordinate of the standard white light of the display device, respectively. These coordinates conform to the specifications in ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000.

[0197] The maximum display brightness (max_display_mastering_luminance) of a display device can be a 16-bit unsigned integer, representing the maximum display brightness of the device. (In 1 cd / m²) 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0198] The minimum display brightness of a display device, min_display_mastering_luminance, can be a 16-bit unsigned integer representing the minimum display brightness of the device, and it can be expressed in increments of 0.0001 cd / m². 2 The units range from 0.0001 cd / m 2 Up to 6.5535 cd / m 2 .

[0199] Optionally, the value of max_display_mastering_luminance can be greater than the value of min_display_mastering_luminance.

[0200] The maximum brightness of the displayed content (max_content_light_level) can be a 16-bit unsigned integer, representing the maximum brightness of the displayed content, expressed in units of 1 cd / m². 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0201] The value of `max_content_light_level` is the maximum value of the `PictureMaxLightLevel` of all displayed images for a given content. The maximum brightness of the displayed images, `PictureMaxLightLevel`, is calculated as follows:

[0202] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0203] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0204] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0205] —The PictureMaxLightLevel of the displayed image is equal to the maximum value of maxRGB for all pixels within the effective display area.

[0206] `max_picture_average_light_level` can be a 16-bit unsigned integer representing the maximum average brightness of the displayed content. (In 1 cd / m²) 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0207] The value of `max_picture_average_light_level` is the maximum value of the average brightness (`PictureAverageLightLevel`) of all displayed images for a given content. The average brightness (`PictureAverageLightLevel`) is calculated as follows:

[0208] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0209] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0210] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0211] —The PictureAverageLightLevel of the displayed image is equal to the average of the maxRGB values ​​of all pixels within the effective display area.

[0212] In another specific example, an extended data type (i.e., the extended data type of else in the table below) is added to the extended data. The target display device and content metadata extended information transmitted in the newly added extended data type are used to mark the display capabilities and video content characteristics of the target display device. The specific syntax and semantics are as follows (the bolded parts are the modified parts):

[0213]

[0214] Slightly different from the previous example, this example does not add a new extended data type for 0xe7. Instead, it calls the processing logic for the target device's display and content metadata extended information through other extended data types (i.e., extended data types other than 0x04, 0x10, 0x12, 0xe1, and 0xe3).

[0215] The target device display and content metadata extension definitions for this example are shown in the table below.

[0216]

[0217] The video extension label extension_id can be an 8-bit unsigned integer, which can identify the target display device and the table number of the content metadata extension, extension_id should be equal to 0xe7.

[0218] The X-coordinates (display_primaries_x[c]) and Y-coordinates (display_primaries_y[c]) of the display device's three primary colors can be 16-bit unsigned integers, representing the normalized x- and y-coordinates of the display device's three primary colors, respectively. These coordinates conform to ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000. Values ​​of c of 0, 1, and 2 correspond to green, blue, and red, respectively.

[0219] The standard white light X-coordinate (white_point_x) and Y-coordinate (white_point_y) of the display device can be 16-bit unsigned integers, representing the normalized chromaticity x-coordinate and y-coordinate of the standard white light of the display device, respectively. These coordinates conform to the specifications in ISO / CIE 11664 (all parts), with units of 0.00002 and a range from 0 to 50000.

[0220] The maximum display brightness (max_display_mastering_luminance) of a display device can be a 16-bit unsigned integer, representing the maximum display brightness of the device. (In 1 cd / m²) 2 The unit is 1 cd / m 2Up to 65535cd / m 2 .

[0221] The minimum display brightness of a display device, min_display_mastering_luminance, can be a 16-bit unsigned integer representing the minimum display brightness of the device, and it can be expressed in increments of 0.0001 cd / m². 2 The units range from 0.0001 cd / m 2 Up to 6.5535 cd / m 2 .

[0222] Optionally, the value of max_display_mastering_luminance can be greater than the value of min_display_mastering_luminance.

[0223] The maximum brightness of the displayed content (max_content_light_level) can be a 16-bit unsigned integer, representing the maximum brightness of the displayed content, expressed in units of 1 cd / m². 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0224] The value of `max_content_light_level` is the maximum value of the `PictureMaxLightLevel` of all displayed images for a given content. The maximum brightness of the displayed images, `PictureMaxLightLevel`, is calculated as follows:

[0225] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0226] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0227] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0228] —The PictureMaxLightLevel of the displayed image is equal to the maximum value of maxRGB for all pixels within the effective display area.

[0229] `max_picture_average_light_level` can be a 16-bit unsigned integer representing the maximum average brightness of the displayed content. (In 1 cd / m²) 2 The unit is 1 cd / m 2 Up to 65535cd / m 2 .

[0230] The value of `max_picture_average_light_level` is the maximum value of the average brightness (`PictureAverageLightLevel`) of all displayed images for a given content. The average brightness (`PictureAverageLightLevel`) is calculated as follows:

[0231] —Calculate the maximum value (maxRGB) of the R, G, and B components of each pixel within the effective display area of ​​the displayed image. The effective display area is a rectangular region defined by display_horizontal_size and display_vertical_size.

[0232] The non-linear (R',G',B') values ​​of the pixels are converted to linear (R,G,B) values ​​and calibrated to 1 cd / m 2 Values ​​in units;

[0233] The maximum value of the pixel's R, G, and B components, maxRGB, is calculated from the (R, G, B) values ​​after pixel calibration.

[0234] The PictureAverageLightLevel of the displayed image is equal to the average of the maxRGB values ​​of all pixels within the effective display area.

[0235] In one application scenario, two payload types can be added to the supplementary and enhanced payload syntax to mark the video's color space attributes and the target display device's display capabilities and video content characteristics, respectively. The specific syntax and semantics are as follows (the modified parts are in red):

[0236]

[0237]

[0238] Here, `sei_payload` refers to the function used to parse the supplementary and enhanced payload. `sei_payload(PayloadType, PayloadSize)` determines the content type of the SEI (i.e., supplementary and enhanced payload) based on `PayloadType` (payload type), calls the corresponding processing logic, and ensures that the data is byte-aligned.

[0239] As shown above, when the load type is 3, the processing logic corresponding to `gmv_extension` (i.e., encoding stabilization extension) is called; when the load type is 8, the processing logic corresponding to `roi_parameters` (i.e., region of interest parameters) is called; when the load type is 16, the processing logic corresponding to `user_data` (i.e., user data) is called; when the load type is 17, the processing logic corresponding to `temporal_scalability_info` (temporal domain hierarchical information) is called; when the load type is 18, the processing logic corresponding to `air_parameter` (i.e., adaptive intra-frame refresh parameters) is called; when the load type is 19, the processing logic corresponding to `seq_privacy_parameters` (i.e., sequence-level privacy protection regions) is called; and when the load type is 20, the processing logic corresponding to `pic_privacy_parameters` (i.e., image-level privacy protection regions) is called. The corresponding processing logic is as follows: When the load type is 25, the processing logic corresponding to layer_info (i.e., layer information) is called; when the load type is 26, the processing logic corresponding to version_info (i.e., display knowledge image display information) is called; when the load type is 27, the processing logic corresponding to seq_display_info (i.e., sequence display information) is called; when the load type is 28, the processing logic corresponding to mastering_display_and_content_metadata_extension (i.e., target display device and content metadata extension information) is called; when the load type is 127, the processing logic corresponding to version_info (i.e., version information) is called; when the load type is any other than the above load types, the processing logic corresponding to reserved_sei_message (i.e., reserved supplementary enhancement information) is called.

[0240] The `if(!byte_aligned())` method checks if the current bitstream is not aligned to a byte boundary. If it is, `bit_equal_to_one` (writes a binary bit 1); `while(!byte_aligned())` (loops to check if it is still not aligned to a byte boundary), and if not, `bit_equal_to_zero` (continues to fill with 0s).

[0241] The sequence display information payload is defined as shown in the table below. `seq_display_info()` should only appear in access cells of the random access point image and non-display knowledge image.

[0242]

[0243] The target device display and content metadata extension definitions are shown in the table below. `mastering_display_and_content_metadata_extension()` should only appear in access units of random access point images and non-display knowledge images.

[0244]

[0245] In this application scenario, the syntax of target device display and content metadata as well as sequence display information can be found in the examples above, and will not be elaborated here.

[0246] Step S13: Perform color conversion on the decoded image data based on video display metadata.

[0247] After parsing the video display metadata from the bitstream through the above steps, color conversion can be performed on the decoded image data based on the parsed video display metadata. It is understood that if the image color conversion method of this application is applied to video processing, the decoded image data in step S13 can be understood as the decoded video data.

[0248] Preferably, in some scenarios, after decoding, the video stream needs to be displayed on different display devices. If color gamut identification information is missing, the display devices may not be able to correctly map the video's color space, resulting in color distortion. Therefore, it can be displayed according to the color gamut standard conforming to the color characteristics of the target display device. Thus, step S13 may include: converting the decoded video data into XYZ data based on the video display metadata; and converting the XYZ data into non-linear RGB data conforming to the video production and transmission standard of the display device, which is used for display on the display device.

[0249] Optionally, the step of converting the XYZ data into nonlinear RGB data conforming to the video production and transmission standard of the display device may include: converting the XYZ data into linear RGB conforming to the video production and transmission standard of the display device to obtain third RGB data; and converting the third RGB data into nonlinear RGB data according to the photoelectric transfer function specified in the video production and transmission standard of the display device.

[0250] Among them, such as Figure 4As shown, the step of converting the decoded video data into XYZ data based on the video display metadata may include: converting the decoded video data into RGB data; then electro-optically converting the converted RGB data into linear RGB data; and then converting the linear RGB data into XYZ data.

[0251] In this process, the color characteristics of the source video can be used to process the decoded video data to obtain XYZ data. In other words, the video display metadata parsed in step S12 can be used in this process.

[0252] The video display metadata parsed in step S12 may include sequence display information. As mentioned above, the sequence display information may include, but is not limited to, sample range, primary colors, photoelectric transfer characteristics, and color signal conversion matrix. The step of processing the decoded video data using the color characteristics of the source video to obtain XYZ data may include: converting the decoded video data into RGB according to the sample range and color signal conversion matrix to obtain first RGB data; converting the first RGB data into linear RGB according to the electro-optic transfer function specified by the photoelectric transfer characteristics to obtain second RGB data; and converting the second RGB data into the XYZ data based on the primary colors.

[0253] In one example, assuming the source video conforms to the video production and transmission standard BT.601 and the target display device's color characteristics conform to the video production and transmission standard BT.709, the conversion process can be as follows: Figure 4 As shown:

[0254] (1) The SVAC3 stream is decoded into YUV. The color characteristics of the YUV at this time are obtained based on the decoded sample_range, color_primaries, transfer_characteristics, and matrix_coefficients.

[0255] (2) Convert YUV to RGB based on sample_range and matrix_coefficients;

[0256] (3) Convert RGB to linear RGB according to the electro-optical transfer function specified by transfer_characteristics;

[0257] (4) Based on color_primaries, convert linear RGB to XYZ, which is independent of color characteristics;

[0258] (5) Since the color characteristics conformed to the standard BT.709 for display devices, convert XYZ to linear RGB that conforms to BT.709;

[0259] (6) Convert linear RGB to nonlinear RGB according to the photoelectric transfer function (gamma correction) specified in BT.709;

[0260] (7) At this time, the display device can correctly display the SVAC3 stream data from BT.601.

[0261] In certain situations, such as when the photoelectric conversion function (EOTF) at the encoding end is inconsistent with that at the decoding end, or when the dynamic range of the display device is insufficient, tone mapping can be performed during the processing of the decoded video data to obtain XYZ data. This allows the video data to adapt to the brightness and color gamut of the display device through tone mapping or inverse tone mapping. Tone mapping or inverse tone mapping can be performed using the target device's display and content metadata. Specifically, converting the second RGB data into the XYZ data based on the three primary colors includes: performing tone mapping or inverse tone mapping on the second RGB data based on the target display device and extended content metadata information to obtain fourth RGB data adapted to the brightness and color gamut of the display device; and converting the fourth RGB data into the XYZ data based on the three primary colors.

[0262] In one example, the source video conforms to the BT.2020 standard, and the target display device's color characteristics conform to BT.709. The conversion process is as follows: Figure 5 As shown:

[0263] (1) The SVAC3 bitstream is decoded into YUV, and the color characteristics of the YUV at this time are obtained based on the decoded sample_range, color_primaries, transfer_characteristics, and matrix_coefficients.

[0264] (2) Convert YUV to non-linear RGB based on sample_range and matrix_coefficients;

[0265] (3) Convert non-linear RGB to linear RGB according to transfer_characteristics;

[0266] (4) Perform hue mapping on linear RGB to adapt it to the brightness and color range of the display device;

[0267] (5) Based on color_primaries, convert linear RGB to XYZ, which is independent of color characteristics;

[0268] (6) Since the color characteristics conformed to the standard BT.709 for display devices, convert XYZ to linear RGB that conforms to BT.709;

[0269] (7) Convert linear RGB to nonlinear RGB according to the electro-optical transfer function specified in BT.709;

[0270] (8) At this point, the SVAC3 stream from BT.2020 can be displayed on the display device while maintaining the detail of the source data in both bright and shadow areas.

[0271] The tone mapping process is as follows:

[0272] (1) Convert the linear RGB to YUV according to the BT.2020 standard, as shown below:

[0273] Y′=0.2627R′+0.6780G′+0.0593B′

[0274] Convert the Y-luminance to a perceptual linear space:

[0275]

[0276] Where L HDR `max_display_mastering_luminance` is the maximum display brightness of the mastering device, measured in 1000 cd / m². 2 .

[0277] (2) Apply the inflection point function to Y in the perceptual domain, as shown in the following equation:

[0278]

[0279] Convert back to the gamma domain, as shown in the following figure:

[0280]

[0281] Where ρ SDR The maximum brightness of the target display device, in units of 100 cd / m². 2 .

[0282] (3) Convert YUV back to RGB.

[0283] In this embodiment, a bitstream encoded with the digital audio-visual codec technology standard for security monitoring is acquired; video display metadata in the bitstream is parsed; and color conversion is performed on the decoded image data based on the video display metadata. This defines the color characteristics of the source video, ensuring correct display on different devices. It also compensates for the lack of color space management in the digital audio-visual codec technology standard for security monitoring, improving the dynamic range of the video. This is especially important in high-contrast scenes (such as strong light and shadow in monitoring) and better meets industry needs.

[0284] Corresponding to the image color conversion method of the above embodiments, this application provides another image color conversion method. Specifically, as shown in the figure... Figure 6 As shown, the image color conversion method of the first embodiment proposed in this application specifically includes the following steps. It should be noted that the step numbers are for simplification only and are not intended to limit the execution order of the steps. The execution order of each step in this embodiment can be arbitrarily changed without departing from the technical concept of this application. Furthermore, the relevant content of the image color conversion method of this embodiment can be combined with the image color conversion methods of other embodiments.

[0285] S21: Acquire image data.

[0286] like Figure 1 and Figure 2 As shown, image data can be acquired through an image sensor.

[0287] In step S21, the light signal from the outside world is focused by the optical device and reaches the image sensor. The image sensor converts the incident photons into corresponding electrons, accumulates the charge of the converted electrons, and then converts them into corresponding voltage signals. The analog-to-digital converter then converts the voltage signals into linear digital signals according to the corresponding video production and transmission standards. At this time, linear RGB data conforming to standard A is obtained, which is the acquired image data.

[0288] It is understood that the image color conversion method of this embodiment can be applied to image processing or video processing. If the application scenario of this image color conversion method is video processing, then step S21 can acquire image data in real time so that the acquired image data can be color converted and encoded in the subsequent process to obtain the video bitstream.

[0289] S22: Perform color conversion on the image data to obtain the image data to be encoded.

[0290] Because the human visual perception system does not work in a linear manner, the acquired image data can be color-converted to obtain the image data to be encoded.

[0291] Among them, such as Figure 1 As shown, step S22 may include: converting linear RGB to nonlinear RGB using the photoelectric transfer function specified in the corresponding video production and transmission standard; converting the nonlinear RGB to YUV data according to the corresponding video production and transmission standard; and then sampling the chromaticity information of the YUV format to facilitate image space compression.

[0292] As shown in the previous embodiment, Figure 2 As shown, in some cases, color conversion of image data may also involve tone mapping or inverse tone mapping.

[0293] S23: Encode the image data to be encoded into a bitstream, and encode the video display metadata used in the color conversion process of the image data into the bitstream.

[0294] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0295] To implement the above image color conversion method, this application also proposes a video encoding and decoding device, please refer to [link / reference needed]. Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the video encoding and decoding device provided in this application.

[0296] The video encoding / decoding device 500 of this embodiment includes a processor 51, a memory 52, an input / output device 53, and a bus 54.

[0297] The processor 51, memory 52, and input / output device 53 are respectively connected to the bus 54. The memory 52 stores program data, and the processor 51 is used to execute the program data to implement the image color conversion method described in the above embodiment.

[0298] In this embodiment, processor 51 can also be referred to as a CPU (Central Processing Unit). Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor, or processor 51 can be any conventional processor.

[0299] This application also provides a computer storage medium; please refer to the following: Figure 8 , Figure 8 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 600 stores a computer program 61, which, when executed by a processor, is used to implement the image color conversion method of the above embodiment.

[0300] When the embodiments of this application are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0301] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. An image color conversion method, characterized in that, The method includes: Acquire the bitstream encoded using the digital audio and video encoding / decoding technology standard for security monitoring; Parse the video display metadata in the bitstream; The decoded image data is color-converted based on the video display metadata.

2. The image color conversion method according to claim 1, characterized in that, The video display metadata includes sequence display information, which includes at least one of the following: video format, sample range, color information description, primary colors, photoelectric transfer characteristics, color signal conversion matrix, horizontal display size, and vertical display size; and / or, The video display metadata includes target display device and content metadata extension information. The target display device and content metadata extension information includes at least one of the following: display device three primary color X coordinates, display device three primary color Y coordinates, display device standard white light X coordinates, display device standard white light Y coordinates, display device maximum display brightness, display device minimum display brightness, display content maximum brightness, and display content maximum image average brightness.

3. The image color conversion method according to claim 1, characterized in that, The video display metadata includes sequence display information; parsing the video display metadata in the bitstream includes: The sequence display information is parsed from the augmented payload supplemented from the bitstream; or, The sequence display information is parsed from the extended data of the bitstream; or, The sequence display information is parsed from the sequence parameter set in the bitstream.

4. The image color conversion method according to claim 3, characterized in that, The step of parsing the sequence display information from the supplementary enhancement load in the bitstream includes: parsing the sequence display information from the supplementary enhancement load of a first preset load type in the bitstream; or, The step of parsing the sequence display information from the extended data in the bitstream includes: parsing the sequence display information from the extended data of a first preset extended data type in the bitstream.

5. The image color conversion method according to claim 1, characterized in that, The video display metadata includes the target display device and extended information of the content metadata; The parsing of video display metadata in the bitstream includes: The target display device and content metadata extension information are parsed from the supplementary enhanced load of the bitstream; or, The target display device and content metadata extension information are parsed from the extended data in the bitstream.

6. The image color conversion method according to claim 5, characterized in that, The step of parsing the target display device and content metadata extension information from the supplementary enhancement load in the bitstream includes: parsing the target display device and content metadata extension information from the supplementary enhancement load of the second preset load type in the bitstream; or, The step of parsing the target display device and content metadata extension information from the extended data in the bitstream includes: parsing the target display device and content metadata extension information from the extended data of the second preset extended data type in the bitstream.

7. The image color conversion method according to claim 1, characterized in that, The step of color conversion of the decoded image data based on the video display metadata includes: Based on the video display metadata, the decoded image data is converted into XYZ data; The XYZ data is converted into non-linear RGB data that conforms to the video production and transmission standards of the display device.

8. The image color conversion method according to claim 7, characterized in that, The video display metadata includes sample range, primary colors, photoelectric transfer characteristics, and color signal conversion matrix. The process of converting the decoded image data into XYZ data based on the video display metadata includes: The decoded image data is converted into RGB based on the sample range and the color signal conversion matrix to obtain the first RGB data; The first RGB data is converted into linear RGB data according to the electro-optic transfer function specified by the photoelectric transfer characteristics to obtain the second RGB data; Based on the three primary colors of color, the second RGB data is converted into the XYZ data; The process of converting the XYZ data into non-linear RGB data conforming to the video production and transmission standards of the display device includes: The XYZ data is converted into linear RGB that conforms to the video production and transmission standards of the display device to obtain the third RGB data; The third RGB data is converted into nonlinear RGB data according to the photoelectric transfer function specified in the video production and transmission standard of the display device.

9. The image color conversion method according to claim 8, characterized in that, The video display metadata includes the target display device and extended information of the content metadata; The conversion of the second RGB data into the XYZ data based on the three primary colors includes: Based on the target display device and content metadata extension information, the second RGB data is hue-mapped or inversely hue-mapped to obtain fourth RGB data adapted to the brightness and color gamut of the target display device; Based on the three primary colors of color, the fourth RGB data is converted into the XYZ data.

10. An image color conversion method, characterized in that, The image color conversion method includes: Acquire image data; The image data is color-converted to obtain the image data to be encoded; The image data to be encoded is encoded into a bitstream using the security and monitoring digital audio and video encoding and decoding technology standard, and the video display metadata used in the color conversion process of the image data is encoded into the bitstream.

11. A video encoding / decoding apparatus, characterized in that, The video encoding / decoding device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the image color conversion method as described in any one of claims 1 to 10.

12. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which, when executed by the computer, is used to implement the image color conversion method as described in any one of claims 1 to 10.

Images