An image acquisition method, an image display method and an apparatus

Abstract

An image acquisition method, an image display method and device are provided to reduce the difference between the image displayed by the display end and the image displayed by the acquisition end. The acquisition end transmits data for expressing the brightness mapping relationship between the pre-compression image and the post-compression image to the display end during encoding, so as to ensure the consistency of the images of the acquisition end and the display end. In addition, the screen-related information for expressing the brightness of the acquisition end is transmitted to the display end, so that the display end refers to the screen-related information when adjusting the backlight and the pixel value, thereby improving the consistency of the images of the acquisition end and the display end.

Description

[0001] This application claims priority to Chinese Patent Application No. 202210056484.6, filed on January 18, 2022, entitled “An Image Acquisition Method, Image Display Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of image processing technology, and in particular to an image acquisition method, an image display method, and an apparatus. Background Technology

[0003] Existing terminal devices, such as mobile phones and tablets, compress and encode image data after image acquisition without considering information from other devices. The display end automatically adjusts the screen brightness according to changes in ambient light. Regardless of the brightness, the data sent for display remains unchanged, which may lead to a significant difference in display effect between the display end and the acquisition end. Summary of the Invention

[0004] This application provides an image acquisition method, an image display method, and an apparatus to reduce the difference between the image displayed on the display end and the image displayed on the acquisition end.

[0005] In a first aspect, embodiments of this application provide an image acquisition method applied to an acquisition terminal device, comprising: acquiring a first image to obtain image data of the first image; compressing the first image according to the image data of the first image to obtain image data of a second image, wherein the number of bits occupied by the first image is higher than the number of bits occupied by the second image; obtaining transmission data of the second image according to the image data of the first image and the image data of the second image; wherein the transmission data of the second image includes encoded data of the second image and first brightness-related data, the first brightness-related data being used to characterize the brightness mapping relationship between the first image and the second image; and transmitting the transmission data of the second image.

[0006] In current solutions, while the acquisition device retains a significant amount of highlight detail during image capture, it fails to fully record the relative brightness relationships of the real scene, thus weakening these relationships. This embodiment, however, transmits image details from the acquisition device to the display device via first brightness-related data. The display device can then adjust the pixel values ​​of the received image based on this first brightness-related data to obtain the image to be displayed, making it closer to the image captured by the acquisition device.

[0007] In one possible design, the transmitted data further includes at least one tone mapping curve, which is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device.

[0008] In the above scheme, during the encoding stage, the acquisition device generates a tone mapping curve for other possible screen brightness levels based on the screen information used by the acquisition device, in order to maintain the dynamic range seen by the acquisition device.

[0009] In one possible design, the transmitted data further includes screen-related information, which includes at least one of the following: the screen backlight parameters of the acquisition device, or the mapping relationship between the backlight brightness under the screen backlight parameters of the acquisition device and the backlight brightness under at least one set screen backlight parameter, or the adjustment ratio of the backlight brightness under at least one set screen backlight parameter relative to the backlight brightness under the screen backlight parameters of the acquisition device.

[0010] In the above design, the acquisition end transmits screen-related information from the acquisition end to the display end, which can be used to guide the backlight boost ratio during display. The goal is to maximize the consistency between the display effect on the display device and the display effect on the acquisition end.

[0011] In one possible design, obtaining the first brightness-related data based on the first image and the second image includes: normalizing the image data of the first image and the image data of the second image according to the bit depth of the first image and the second image respectively, to obtain normalized data of the first image and normalized data of the second image; and obtaining the first brightness-related data based on the normalized data of the first image and the normalized data of the second image.

[0012] In the above design, normalization can reduce the amount of encoded data and transmitted data, thereby improving encoding and transmission efficiency.

[0013] In one possible design, the first image has N bits, the second image has M bits, the normalized data of the first image ranges from 0 to X, and the normalized data of the second image ranges from 0 to Y; where Y / X = 2. M-N .

[0014] In one possible design, obtaining the brightness-related data based on the normalized data of the first image and the normalized data of the second image includes: generating the first brightness-related data based on the ratio of the normalized data of corresponding pixels in the first image and the second image.

[0015] In one possible design, generating the first brightness-related data based on the ratio of normalized data of corresponding pixels in the first image and the second image includes: obtaining the ratio of normalized data of corresponding pixels in the first image and the second image; when the ratio of normalized data of the first pixel in the first image to the normalized data of the first pixel in the second image is less than 1, correcting the ratio of normalized data of the first pixel to obtain the first brightness-related data; the first pixel is a pixel in the first image or the second image.

[0016] In one possible design, obtaining the first brightness-related data based on the normalized data of the first image and the normalized data of the second image includes: generating second brightness-related data based on the ratio of the normalized data of corresponding pixels in the first image and the second image; identifying that the first image includes a set bright object, and adjusting the brightness mapping relationship of the pixels corresponding to the bright object in the second brightness-related data according to the photoelectric conversion curve to obtain the first brightness-related data.

[0017] Because the human eye is non-linear, during compression processing, such as compressing a 12-bit image into an 8-bit image, an S-curve can be used to further compress the 8-bit image, enhancing image details. However, using an S-curve may cause the brightness values ​​of some pixels in the second image to be higher than the corresponding pixels in the first image. Therefore, this application, through the above design, can further adjust the determined brightness-related data (ratios or adjustment values) to ensure that the brightness values ​​of pixels in the second image are no longer higher than the corresponding pixels in the first image, thus improving the accuracy of the image displayed on the display device.

[0018] In one possible design, generating the second brightness-related data based on the ratio of normalized data of corresponding pixels in the first image and the second image includes: obtaining the ratio of normalized data of corresponding pixels in the first image and the second image; when the ratio of normalized data of the first pixel in the first image to the normalized data of the first pixel in the second image is less than 1, correcting the ratio of normalized data of the first pixel to obtain the second brightness-related data; the first pixel is a pixel in the first image or the second image.

[0019] In one possible design, the photoelectric conversion curve includes at least one of the following: a perceptual quantization (PQ) conversion curve, a hybrid log-gamma (HLG) conversion curve, or a scene brightness fidelity (SLF) conversion curve.

[0020] In one possible design, the first brightness-related data is carried in the metadata of the second image.

[0021] In one possible design, the first brightness-related data, screen-related data, and at least one tone mapping curve are encoded according to the encoding format of the second image's encoded data.

[0022] In one possible design, the transmission data of the second image is contained in a JPEG file encoded using the Joint Picture Experts Group (JEP) JPEG format. The JPEG file includes a first JPEG image frame that carries the encoded data of the second image, and the first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

[0023] In the above design, the first brightness-related data is encoded to the display end through the APP field. The display end decodes the first brightness-related data during the decoding of the first JPEG image frame, and then adjusts the pixel values ​​of the received image according to the first brightness-related data to obtain the image to be displayed, so that the image to be displayed is closer to the image captured by the acquisition device.

[0024] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0025] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type identifier takes a first value to indicate that the data type of the first brightness-related data is metadata.

[0026] In one possible design, the transmission data of the second image also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame.

[0027] In one possible design, the transmission data of the second image is contained in a JPEG file encoded in JPEG format, the JPEG file including a first JPEG image frame and a second JPEG image frame, the first JPEG image frame carrying the encoded data of the second image, and the second JPEG image frame carrying the first brightness-related data.

[0028] In the above design, the first luminance-related data is encoded using independent JPEG image frames, providing a simple and effective encoding method. The first luminance-related data and the image data of the second image are encoded independently, allowing for independent decoding. This improves decoding efficiency when decoding different frames in parallel.

[0029] In one possible design, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file.

[0030] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. The file type identifier of the first JPEG image frame is a third value, which is used to indicate that the APP field of the first JPEG image frame carries the encoded data of the second image. The file type identifier of the second JPEG image frame is a fourth value, which is used to indicate that the APP field of the second JPEG image frame carries the first brightness-related data.

[0031] The third and fourth values ​​are different.

[0032] In one possible design, the APP field of the second JPEG image frame includes a data type identifier, which takes the value of a second value to indicate that the data type of the first brightness-related data is an image frame type.

[0033] The first and second values ​​are different.

[0034] In one possible design, the transmission data of the second image also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame or in the APP field of the second JPEG image frame.

[0035] In one possible design, the transmitted data further includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled-down image of the first image compared to the first image; the high-frequency texture enhancement data is used to describe the increase in high-frequency texture information in the magnified image of the first image compared to the first image or to describe the decrease in high-frequency texture information in the scaled-down image of the first image compared to the first image.

[0036] In the above design, the scaling data is sent to the display end, so that when the display end needs to zoom in or out of the displayed image, the details of the displayed image can be further increased to enhance the displayed image.

[0037] In one possible design, the scaling enhancement data is carried in the APP field of the first JPEG image frame.

[0038] In one possible design, when the scaling display data includes the tone mapping curve for scaling the display, the tone mapping curve for scaling the display is carried in the APP field of the first JPEG image frame; or,

[0039] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; or...

[0040] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data.

[0041] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file.

[0042] In one possible design, the APP field of the first JPEG image frame also includes a scaling enhancement method identifier;

[0043] When the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to superimpose the scaling enhancement data on the image after enhancing the second image according to the first brightness-related data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly superimpose the high-frequency texture enhancement data on the image after enhancing the second image according to the first brightness-related data; or, when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image after enhancing the second image according to the first brightness-related data based on the tone mapping curve used for scaling display.

[0044] In this embodiment, the first and second values ​​are different. The third and fourth values ​​are different. The fifth, sixth, and seventh values ​​are all different. In some scenarios, the first, second, third, fourth, fifth, sixth, and seventh values ​​may have repeated values, such as the first and third values ​​being the same, the fourth and fifth values ​​being the same, etc. This application does not specifically limit this.

[0045] Secondly, embodiments of this application provide an image display method applied to a display device, comprising: receiving transmission data of a second image, the transmission data of the second image including encoded data of the second image and first brightness-related data, the first brightness-related data being used to characterize the brightness mapping relationship between the second image and a first image, the first image occupying a higher number of bits than the second image; the second image being an image obtained by compression based on the first image; determining a first tone mapping curve; performing tone mapping processing on the second image according to the first tone mapping curve to obtain a third image; and adjusting the pixel values ​​of the pixels of the third image according to the brightness-related data to obtain an image to be displayed.

[0046] In one possible design, the transmitted data further includes at least one tone mapping curve, which is determined based on the screen backlight information of the acquisition device, the second image, and at least one set screen display parameter; determining the first tone mapping curve includes: determining the first tone mapping curve from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device match the set screen display parameters corresponding to the first tone mapping curve.

[0047] In the above scheme, the display end generates a TM curve adapted to the display device based on the TM curve transmitted by the acquisition end, and applies it to the image so that the image effect seen by the user under various brightness screens is as consistent as possible with the effect of the acquisition end on the reference screen.

[0048] In one possible design, the transmitted data for receiving the second image includes:

[0049] The received bitstream includes a JPEG file encoded using the Joint Picture Experts Group (JETG) JPEG format. The JPEG file includes a first JPEG image frame, which carries the encoded data of the second image, and the first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

[0050] Decode the second image from the first JPEG image frame and decode the first brightness-related data from the APP field of the first JPEG image frame.

[0051] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type takes a first value indicating that the data type of the first brightness-related data is metadata. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0052] When the data type identifier decoded from the APP field of the first JPEG image frame is a first value, the first brightness-related data is decoded from the APP field of the first JPEG image frame.

[0053] In one possible design, the transmitted data also includes at least one tone mapping curve, which is carried in the application identifier (APP) field of the first JPEG image frame;

[0054] The method further includes:

[0055] Decode the at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame.

[0056] In one possible design, the transmitted data for receiving the second image includes:

[0057] Receive a bitstream, the bitstream including a JPEG file encoded in JPEG format, the JPEG file including a first JPEG image frame and a second JPEG image frame;

[0058] Decode the second image from the first JPEG image frame and decode the first brightness-related data from the second JPEG image frame.

[0059] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type is a second value indicating that the data type of the first brightness-related data is an image frame type. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0060] When the data type identifier decoded from the APP field of the first JPEG image frame is a second value, the first brightness-related data is decoded from the second JPEG image frame.

[0061] In one possible design, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file;

[0062] Decoding the second image from the first JPEG image frame and decoding the first luminance-related data from the second JPEG image frame includes:

[0063] Decode the second image from the first JPEG image frame and obtain the first location identifier;

[0064] The starting position of the second JPEG image frame is determined from the JPEG file based on the first position identifier, and the first brightness-related data is decoded from the second JPEG image frame based on the starting position.

[0065] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data.

[0066] Decoding the image data of the second image from the first JPEG image frame includes:

[0067] When the file type identifier decoded from the APP field of the first JPEG image frame is a third value, the image data of the second image is decoded from the first JPEG image frame;

[0068] Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0069] When the file type identifier decoded from the APP field of the second JPEG image frame is a fourth value, the first brightness-related data is decoded from the second JPEG image frame.

[0070] In one possible design, the transmitted data further includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame or the APP field of the second JPEG image frame;

[0071] The method further includes:

[0072] Decode the at least one tone mapping curve from the APP field of the first JPEG image frame or the APP field of the second JPEG image frame.

[0073] In one possible design, the transmitted data further includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image; the high-frequency texture enhancement data includes high-frequency texture information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image and / or high-frequency texture information describing the decrease in high-frequency texture values ​​in the scaled image of the first image compared to the first image.

[0074] In one possible design, the method further includes:

[0075] In response to a user's zoom-in or zoom-out operation on the image to be displayed, the zoom display data is decoded from the JPEG file, and the image to be displayed is enhanced based on the zoom display data to obtain an enhanced image to be displayed.

[0076] As an example, the scaling display data includes scaling enhancement data, which includes information describing the increase in pixel values ​​in the enlarged image compared to the first image. When a user performs a zoom operation on the image to be displayed, the display device enhances the image to be displayed based on this scaling enhancement data. For example, the scaling enhancement data may be overlaid on top of the image to be displayed.

[0077] As an example, the scaling display data includes scaling enhancement data, which includes information describing the reduction in pixel values ​​in the scaled-down image compared to the first image. When a user performs a zoom-in operation on the image to be displayed, the display device adjusts the image to be displayed based on this scaling enhancement data. For example, the scaling enhancement data may be subtracted from the image to be displayed.

[0078] As an example, the scaling display data includes high-frequency texture enhancement data, which includes high-frequency texture information describing the increased texture content of the magnified image compared to the first image. When a user performs a magnification operation on the image to be displayed, the display device enhances the image to be displayed based on the high-frequency texture enhancement data. For example, the high-frequency texture enhancement data is enhanced according to a set enhancement coefficient, and the enhanced high-frequency texture enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed.

[0079] As an example, the scaling display data includes high-frequency texture enhancement data, which includes high-frequency texture information describing the reduction in texture information between the scaled-down image and the first image. When a user performs a scaling-down operation on the image to be displayed, the display device adjusts the image to be displayed based on the high-frequency texture enhancement data. For example, the high-frequency texture enhancement data may be negatively enhanced according to a set enhancement coefficient, and the negatively enhanced high-frequency texture enhancement data may be superimposed on the image to be displayed to obtain the enhanced image to be displayed.

[0080] In one possible design, the scaling display data is carried in the APP field of the first JPEG image frame, and decoding the scaling display data from the JPEG file includes:

[0081] The tone mapping curve used for scaling display is decoded from the APP field of the first JPEG image frame.

[0082] In one possible design, when the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; decoding the scaling display data from the JPEG file includes:

[0083] Decode the tone mapping curve used for scaling display from the APP field of the first JPEG image frame;

[0084] or,

[0085] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; decoding the scaling display data from the JPEG file includes:

[0086] Decode the scaling enhancement data from the third JPEG image frame;

[0087] or,

[0088] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data; decoding the scaling display data from the JPEG file includes:

[0089] Decode the high-frequency texture enhancement data from the fourth JPEG image frame.

[0090] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file;

[0091] Decoding the scaling enhancement data from the third JPEG image frame includes:

[0092] The start position of the third JPEG image frame is determined from the JPEG file based on the second position identifier, and the scaling enhancement data is decoded from the third JPEG image frame based on the start position; or,

[0093] Decoding the high-frequency texture enhancement data from the fourth JPEG image frame includes:

[0094] The start position of the fourth JPEG image frame is determined from the JPEG file based on the third position identifier, and the scaling enhancement data is decoded from the fourth JPEG image frame based on the start position.

[0095] In one possible design, the APP field of the first JPEG image frame further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0096] In one possible design, enhancing the image to be displayed based on the scaling display data to obtain an enhanced image to be displayed includes:

[0097] When the scaling enhancement method is identified as the fifth value, the scaling enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed; or...

[0098] When the scaling enhancement method is identified as the sixth value, the high-frequency texture enhancement data is enhanced according to the set enhancement coefficient, and the enhanced high-frequency texture enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed.

[0099] When the scaling enhancement mode is identified as the seventh value, the brightness of the image to be displayed is adjusted based on the tone mapping curve used for scaling the display.

[0100] In one possible design, adjusting the pixel values ​​of the pixels in the third image according to the first brightness-related data to obtain the image to be displayed includes: determining the screen backlight of the display device before adjustment and determining the screen backlight of the display device after adjustment; adjusting the pixel values ​​of the pixels in the third image according to the screen backlight of the display device before adjustment and the screen backlight of the display device after adjustment to obtain a fourth image; and performing a weighted fusion process on the fourth image and the third image to obtain the image to be displayed; wherein the weights of the pixels in the third image and the fourth image are determined according to the first brightness-related data.

[0101] In some embodiments, the display device can determine the required increase in screen backlight based on the decoded screen-related information. For example, the screen-related information may include the screen backlight parameters of the acquisition device, or the mapping relationship between the backlight brightness under the screen backlight parameters of the acquisition device and the backlight brightness under at least one set screen backlight parameter, or the adjustment ratio of the backlight brightness under at least one set screen backlight parameter relative to the backlight brightness under the screen backlight parameters of the acquisition device.

[0102] In one possible design, the pixel values ​​of each pixel in the fourth image satisfy the following condition:

[0103] pixel1 i =pow(BL0 / BL1,1 / gamma)*pixel0 i ;

[0104] Among them, pixel1 i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i BL0 represents the pixel value of the i-th pixel in the third image, BL1 represents the screen backlight of the display device before adjustment, gamma represents the gamma value of the display device, and pow(xExp, nExp) represents nExp raised to the power of xExp.

[0105] In one possible design, the pixel values ​​of each pixel in the image to be displayed satisfy the following condition:

[0106] weight i =hdrlayer i / 2 N ;

[0107] pixel i =pixel0 i weight i +pixel1i *(1-weight i );

[0108] Where N represents the number of bits occupied by the second image, hdrlayer i This represents the first brightness-related data corresponding to the i-th pixel, pixel1. i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i Represents the pixel value of the i-th pixel in the third image, pixel. i This represents the pixel value of each pixel in the image to be displayed.

[0109] In one possible design, the method further includes: adjusting the screen backlight of the display device.

[0110] In one possible design, the adjusted screen backlight of the display device satisfies the following conditions:

[0111] BL1 = BL0 * (k1 * AVG (hdrlayer) i )+b1); or,

[0112] BL1 = BL0 * (k2 * max (hdrlayer) i )+b2);

[0113] Wherein, BL1 represents the adjusted screen backlight of the display device, BL0 represents the original screen backlight of the display device, and hdrlayer i This represents the brightness-related data corresponding to the i-th pixel of the display screen of the display device, AVG(hdrlayer) i ) represents the average value of the brightness-related data corresponding to the pixels of the display screen of the display device, max(hdrlayer) i ) represents the maximum value of the brightness-related data corresponding to the pixels of the display screen of the display device, where k1 and k2 are both positive numbers, and b1 and b2 are both positive numbers less than 1.

[0114] Thirdly, embodiments of this application provide an image acquisition device, applied to an acquisition terminal device, comprising:

[0115] The acquisition module is used to acquire the first image to obtain the image data of the first image;

[0116] The processing module is configured to compress the first image based on the first image data to obtain image data of the second image, wherein the number of bits occupied by the first image is higher than the number of bits occupied by the second image; and to obtain transmission data of the second image based on the image data of the first image and the image data of the second image; wherein the transmission data of the second image includes the encoded data of the second image and first brightness-related data, wherein the first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image.

[0117] A sending module is used to send the transmitted data.

[0118] In one possible design, the transmitted data further includes at least one tone mapping curve, which is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device.

[0119] In one possible design, the processing module is specifically configured to normalize the image data of the first image and the image data of the second image according to the number of bits of the first image and the second image, respectively, to obtain normalized data of the first image and normalized data of the second image; and to obtain the first brightness-related data based on the normalized data of the first image and the normalized data of the second image.

[0120] In one possible design, the first image has N bits, the second image has M bits, the normalized data of the first image ranges from 0 to X, and the normalized data of the second image ranges from 0 to Y; where Y / X = 2. M-N .

[0121] In one possible design, the processing module is specifically used to generate the first brightness-related data based on the ratio of the normalized data of the corresponding pixels in the first image and the second image.

[0122] In one possible design, the processing module is specifically used to obtain the ratio of normalized data of corresponding pixels in the first image and the second image; when the ratio of normalized data of the first pixel in the first image to normalized data of the first pixel in the second image is less than 1, the ratio of normalized data of the first pixel is corrected to obtain the first brightness-related data; the first pixel is a pixel in the first image or the second image.

[0123] In one possible design, the processing module is specifically used to generate second brightness-related data based on the ratio of normalized data of corresponding pixels in the first image and the second image; identify that the first image includes a set bright object, and adjust the brightness mapping relationship of the pixels corresponding to the bright object in the second brightness-related data according to the photoelectric conversion curve to obtain the first brightness-related data.

[0124] In one possible design, the processing module is specifically used to obtain the ratio of normalized data of corresponding pixels in the first image and the second image; when the ratio of normalized data of the first pixel in the first image to normalized data of the first pixel in the second image is less than 1, the ratio of normalized data of the first pixel is corrected to obtain the second brightness-related data; the first pixel is a pixel in the first image or the second image.

[0125] In one possible design, the photoelectric conversion curve includes at least one of the following: a perceptual quantization (PQ) conversion curve, a hybrid log-gamma (HLG) conversion curve, or a scene brightness fidelity (SLF) conversion curve.

[0126] In one possible design, the first brightness-related data is carried in the metadata of the second image.

[0127] In one possible design, the first brightness-related data, screen-related data, and at least one tone mapping curve are encoded according to the encoding format of the second image's encoded data.

[0128] In one possible design, the transmission data of the second image is contained in a JPEG file encoded using the Joint Picture Experts Group (JEP) JPEG format. The JPEG file includes a first JPEG image frame that carries the encoded data of the second image, and the first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

[0129] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0130] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier. When the data type identifier takes a first value, it indicates that the data type of the first brightness-related data is metadata.

[0131] In one possible design, the transmission data of the second image also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame.

[0132] In one possible design, the transmission data of the second image is contained in a JPEG file encoded in JPEG format, the JPEG file including a first JPEG image frame and a second JPEG image frame, the first JPEG image frame carrying the encoded data of the second image, and the second JPEG image frame carrying the first brightness-related data.

[0133] In one possible design, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file.

[0134] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data.

[0135] In one possible design, the APP field of the second JPEG image frame includes a data type identifier, and when the data type identifier takes a second value, it indicates that the data type of the first brightness-related data is an image frame type.

[0136] In one possible design, the transmission data of the second image also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame or in the APP field of the second JPEG image frame.

[0137] In one possible design, the transmitted data further includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image; the high-frequency texture enhancement data includes high-frequency texture information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image and / or high-frequency texture information describing the decrease in high-frequency texture values ​​in the scaled image of the first image compared to the first image.

[0138] In one possible design, the scaling display data is carried in the APP field of the first image frame.

[0139] In one possible design, when the scaling display data includes the tone mapping curve for scaling the display, the tone mapping curve for scaling the display is carried in the APP field of the first JPEG image frame; or,

[0140] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; or...

[0141] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data.

[0142] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file.

[0143] In one possible design, the APP field of the first JPEG image frame also includes a scaling enhancement method identifier;

[0144] When the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to superimpose the scaling enhancement data on the image after enhancing the second image according to the first brightness-related data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly superimpose the high-frequency texture enhancement data on the image after enhancing the second image according to the first brightness-related data; or, when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image after enhancing the second image according to the first brightness-related data based on the tone mapping curve used for scaling display.

[0145] Fourthly, embodiments of this application provide an image display device, applied to a display terminal device, comprising:

[0146] The decoding module is used to receive the transmission data of the second image, which includes the encoded data of the second image and the first brightness-related data. The first brightness-related data is used to characterize the brightness mapping relationship between the second image and the first image. The first image occupies more bits than the second image. The second image is an image obtained by compression based on the first image. The first tone mapping curve is determined.

[0147] The processing module is used to perform tone mapping processing on the second image according to the first tone mapping curve to obtain a third image; and to adjust the pixel values ​​of the pixels of the third image according to the brightness-related data to obtain the image to be displayed.

[0148] In one possible design, the transmitted data further includes at least one tone mapping curve, which is determined based on the screen backlight information of the acquisition device, the second image, and at least one set screen display parameter.

[0149] The decoding module is specifically used to determine a first tone mapping curve from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve.

[0150] In one possible design, a receiving module is also included for receiving a bitstream, the bitstream including a JPEG file encoded using the Joint Picture Experts Group (JETG) JPEG format, the JPEG file including a first JPEG image frame, the first JPEG image frame carrying encoded data of the second image, and the first brightness-related data carried in the application identifier (APP) field of the first JPEG image frame;

[0151] The decoding module is specifically used to decode the second image from the JPEG image frame and to decode the first brightness-related data from the APP field of the JPEG image frame.

[0152] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type is a first value indicating that the data type of the first brightness-related data is metadata. The decoding module is specifically configured to: decode the first brightness-related data from the APP field of the first JPEG image frame when the data type identifier decoded from the APP field of the first JPEG image frame is the first value.

[0153] In one possible design, the transmitted data also includes at least one tone mapping curve, which is carried in the application identifier (APP) field of the first JPEG image frame;

[0154] The decoding module is further configured to decode the at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame.

[0155] In one possible design, a receiving module is also included for receiving a bitstream, the bitstream including a JPEG file encoded in JPEG format, the JPEG file including a first JPEG image frame and a second JPEG image frame;

[0156] The decoding module is specifically used to decode the second image from the first JPEG image frame and to decode the first brightness-related data from the second JPEG image frame.

[0157] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type is a second value indicating that the data type of the first brightness-related data is an image frame type. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0158] When the data type identifier decoded from the APP field of the first JPEG image frame is a second value, the first brightness-related data is decoded from the second JPEG image frame.

[0159] In one possible design, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file;

[0160] The decoding module is specifically used to decode the second image from the first JPEG image frame and obtain the first position identifier; determine the start position of the second JPEG image frame from the JPEG file according to the first position identifier, and decode the first brightness-related data from the second JPEG image frame according to the start position.

[0161] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame each includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data. The decoding module is specifically used to: decode the image data of the second image from the first JPEG image frame when the file type identifier decoded from the APP field of the first JPEG image frame is a third value; and decode the first brightness-related data from the second JPEG image frame when the file type identifier decoded from the APP field of the second JPEG image frame is a fourth value.

[0162] In one possible design, the transmitted data further includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame or the APP field of the second JPEG image frame;

[0163] The decoding module is also used to decode the at least one tone mapping curve from the APP field of the first JPEG image frame or the APP field of the second JPEG image frame.

[0164] In one possible design, the transmitted data further includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the magnified image of the first image compared to the first image; the high-frequency texture enhancement data includes high-frequency texture information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image and / or high-frequency texture information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image;

[0165] The decoding module is further configured to decode the scaling display data from the JPEG file. The processing module is further configured to respond to a user's zoom-in or zoom-out operation on the image to be displayed, and to perform enhancement processing on the image to be displayed according to the scaling display data to obtain an enhanced image to be displayed.

[0166] In one possible design, the scaling display data is carried in the APP field of the first JPEG image frame, and the decoding module is specifically used for:

[0167] The tone mapping curve used for scaling display is decoded from the APP field of the first JPEG image frame.

[0168] In one possible design, when the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; the decoding module is specifically used to decode the tone mapping curve for scaling the display from the APP field of the first JPEG image frame.

[0169] In one possible design, when the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame carrying the scaling enhancement data; the decoding module is specifically used to decode the scaling enhancement data from the third JPEG image frame.

[0170] In one possible design, when the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame carrying the high-frequency texture enhancement data; the decoding module is specifically used to decode the high-frequency texture enhancement data from the fourth JPEG image frame.

[0171] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file;

[0172] The decoding module is specifically used to determine the start position of the third JPEG image frame from the JPEG file based on the second position identifier, and to decode the scaling enhancement data from the third JPEG image frame based on the start position; or,

[0173] The decoding module is specifically used to determine the starting position of the fourth JPEG image frame from the JPEG file based on the third position identifier, and to decode the scaling enhancement data from the fourth JPEG image frame based on the starting position.

[0174] In one possible design, the APP field of the first JPEG image frame further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0175] The processing module is further configured to:

[0176] When the scaling enhancement method is identified as the fifth value, the scaling enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed; or...

[0177] When the scaling enhancement method is identified as the sixth value, the high-frequency texture enhancement data is enhanced according to the set enhancement coefficient, and the enhanced high-frequency texture enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed.

[0178] When the scaling enhancement mode is identified as the seventh value, the brightness of the image to be displayed is adjusted based on the tone mapping curve used for scaling the display.

[0179] In one possible design, the processing module is specifically configured to determine the screen backlight of the display device before adjustment and the screen backlight of the display device after adjustment; adjust the pixel values ​​of the pixels of the third image according to the screen backlight of the display device before adjustment and the screen backlight of the display device after adjustment to obtain a fourth image; perform weighted fusion processing on the fourth image and the third image to obtain the image to be displayed; wherein the weights of the pixels of the third image and the fourth image are determined according to the first brightness-related data.

[0180] In one possible design, the pixel values ​​of each pixel in the fourth image satisfy the following condition:

[0181] pixel1 i =pow(BL0 / BL1,1 / gamma)*pixel0 i ;

[0182] Among them, pixel1 i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i BL0 represents the pixel value of the i-th pixel in the third image, BL1 represents the screen backlight of the display device before adjustment, gamma represents the gamma value of the display device, and pow(xExp, nExp) represents nExp raised to the power of xExp.

[0183] In one possible design, the pixel values ​​of each pixel in the image to be displayed satisfy the following condition:

[0184] weight i =hdrlayer i / 2 N ;

[0185] pixel i =pixel0 i weight i +pixel1 i *(1-weight i );

[0186] Where N represents the number of bits occupied by the second image, hdrlayer i This represents the first brightness-related data corresponding to the i-th pixel, pixel1. i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i Represents the pixel value of the i-th pixel in the third image, pixel.i This represents the pixel value of each pixel in the image to be displayed.

[0187] In one possible design, the processing module is also used to adjust the screen backlight of the display device.

[0188] In one possible design, the adjusted screen backlight of the display device satisfies the following conditions:

[0189] BL1 = BL0 * (k1 * AVG (hdrlayer) i )+b1); or,

[0190] BL1 = BL0 * (k2 * max (hdrlayer) i )+b2);

[0191] Wherein, BL1 represents the adjusted screen backlight of the display device, BL0 represents the original screen backlight of the display device, and hdrlayer i This represents the brightness-related data corresponding to the i-th pixel of the display screen of the display device, AVG(hdrlayer) i ) represents the average value of the brightness-related data corresponding to the pixels of the display screen of the display device, max(hdrlayer) i ) represents the maximum value of the brightness-related data corresponding to the pixels of the display screen of the display device, where k1 and k2 are both positive numbers, and b1 and b2 are both positive numbers less than 1.

[0192] Fifthly, embodiments of this application provide an image encoding method applied to an encoding apparatus, comprising: encoding image data of a second image and first brightness-related data into a bitstream, the bitstream including a Joint Image Experts Group (JEEG) JPEG file, the JPEG file carrying the image data of the second image and the first brightness-related data; the image data of the second image being obtained by compressing the image data of a first image, the first image occupying a higher number of bits than the second image; the first brightness-related data being used to characterize the brightness mapping relationship between the first image and the second image; and transmitting the bitstream.

[0193] In current solutions, while the acquisition device retains a significant amount of highlight detail during image capture, it fails to fully record the relative brightness relationships of the real scene, thus weakening these relationships. This embodiment, however, encodes image details captured during capture and sends them to the display device using first brightness-related data. The display device can then adjust the pixel values ​​of the received image based on this first brightness-related data to obtain the image to be displayed, making it closer to the image captured by the acquisition device.

[0194] In one possible design, the JPEG file includes a first JPEG image frame, which carries image data of the second image, and the first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

[0195] In the above design, the first brightness-related data and the image data of the second image are uniformly encoded, and the first brightness-related data is reused in the APP field of the image frame. Thus, the display device decodes the first brightness-related data in the process of decoding the image data of the second image, which is simple and effective.

[0196] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0197] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier. When the data type identifier takes a first value, it indicates that the data type of the first brightness-related data is metadata.

[0198] In one possible design, the method further includes: encoding at least one tone mapping curve into the APP field of the first JPEG image frame.

[0199] In one possible design, the JPEG file includes a first JPEG image frame and a second JPEG image frame, wherein the first JPEG image frame carries the encoded data of the second image and the second JPEG image frame carries the first luminance-related data.

[0200] In the above design, the first luminance-related data is encoded using independent JPEG image frames, providing a simple and effective encoding method. The first luminance-related data and the image data of the second image are encoded independently, allowing for independent decoding. This improves decoding efficiency when decoding different frames in parallel.

[0201] In one possible design, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file.

[0202] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. The file type identifier of the first JPEG image frame is a third value, which is used to indicate that the APP field of the first JPEG image frame carries the image data of the second image. The file type identifier of the second JPEG image frame is a fourth value, which is used to indicate that the APP field of the second JPEG image frame carries the first brightness-related data.

[0203] In one possible design, the APP field of the second JPEG image frame includes a data type identifier, and when the data type identifier takes a second value, it indicates that the data type of the first brightness-related data is an image frame type.

[0204] In one possible design, the method further includes:

[0205] At least one tone mapping curve is incorporated into the APP field of the second JPEG image frame.

[0206] In one possible design, the at least one tone mapping curve is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device.

[0207] In one possible design, the method further includes:

[0208] The scaling display data is encoded into the APP field of the first JPEG image frame;

[0209] The transmitted data also includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image; the high-frequency texture enhancement data includes information describing the increase in high-frequency texture in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture in the scaled image of the first image compared to the first image.

[0210] In one possible design, the JPEG file further includes a third JPEG image frame, and the method further includes:

[0211] The scaling display data is incorporated into the JPEG file, and the transmitted data also includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the magnified image of the first image compared to the first image; the high-frequency texture enhancement data includes information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture values ​​in the magnified image of the first image compared to the first image.

[0212] When the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; or,

[0213] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; or...

[0214] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the scaling enhancement data.

[0215] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file.

[0216] In one possible design, the APP field of the first JPEG image frame also includes a scaling enhancement method identifier;

[0217] When the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to superimpose the scaling enhancement data on the image after enhancing the second image according to the first brightness-related data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly superimpose the high-frequency texture enhancement data on the image after enhancing the second image according to the first brightness-related data; or, when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image after enhancing the second image according to the first brightness-related data based on the tone mapping curve used for scaling display.

[0218] Sixthly, embodiments of this application provide an image decoding method, the specific effects of which can be found in the relevant description of the fifth aspect, and will not be repeated here. The method includes: receiving a bitstream, the bitstream including a JPEG file encoded in JPEG format by the Joint Picture Experts Group; decoding image data of a second image and first brightness-related data from the JPEG file; wherein the image data of the second image is obtained by compressing the image data of a first image, and the first image occupies a higher number of bits than the second image; the first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image. The first brightness-related data is used to perform mid-enhancement processing on the image data of the second image.

[0219] In one possible design, the image data of the second image is enhanced based on the first brightness-related data to obtain the image to be displayed.

[0220] In one possible design, the JPEG file includes a first JPEG image frame, the first JPEG image frame carrying image data of the second image, and the first brightness-related data carried in the application identifier APP field of the first JPEG image frame;

[0221] Decoding the image data of the second image and the first brightness-related data from the JPEG file includes:

[0222] The first JPEG image frame is decoded to obtain the image data of the second image, and the first brightness-related data is obtained from the APP field of the first JPEG image frame.

[0223] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0224] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type takes a first value indicating that the data type of the first brightness-related data is metadata. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0225] When the data type identifier decoded from the APP field of the first JPEG image frame is a first value, the first brightness-related data is decoded from the APP field of the first JPEG image frame.

[0226] In one possible design, the method further includes:

[0227] Decode at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame;

[0228] The image data of the second image is enhanced based on the first brightness-related data to obtain the image to be displayed, including:

[0229] A first tone mapping curve is determined from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve;

[0230] The third image is obtained by performing tone mapping processing on the second image based on the first tone mapping curve;

[0231] The pixel values ​​of the pixels in the third image are adjusted based on the first brightness-related data to obtain the image to be displayed.

[0232] In one possible design, the JPEG file includes a first JPEG image frame and a second JPEG image frame, wherein the first JPEG image frame carries image data of the second image and the second JPEG image frame carries the first brightness-related data;

[0233] Decoding the image data of the second image and the first brightness-related data from the JPEG file includes:

[0234] Decode the image data of the second image from the first JPEG image frame, and decode the first brightness-related data from the second JPEG image frame.

[0235] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type is a second value indicating that the data type of the first brightness-related data is an image frame type. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0236] When the data type identifier decoded from the APP field of the first JPEG image frame is a second value, the first brightness-related data is decoded from the second JPEG image frame.

[0237] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data.

[0238] Decoding the image data of the second image from the first JPEG image frame includes:

[0239] When the file type identifier decoded from the APP field of the first JPEG image frame is a third value, the image data of the second image is decoded from the first JPEG image frame.

[0240] Decoding the first brightness-related data from the APP field of the first JPEG image frame includes:

[0241] When the file type identifier decoded from the APP field of the second JPEG image frame is a fourth value, the first brightness-related data is decoded from the second JPEG image frame.

[0242] In one possible design, the method further includes:

[0243] Decode at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame or from the APP field of the second JPEG image frame;

[0244] The image data of the second image is enhanced based on the first brightness-related data to obtain the image to be displayed, including:

[0245] A first tone mapping curve is determined from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve;

[0246] The third image is obtained by performing tone mapping processing on the second image based on the first tone mapping curve;

[0247] The pixel values ​​of the pixels in the third image are adjusted based on the first brightness-related data to obtain the image to be displayed.

[0248] In one possible design, the JPEG file also encodes scaling display data; the method further includes:

[0249] In response to a user's zoom-in or zoom-out operation on the image to be displayed, the zoom display data is decoded from the JPEG file, and the image to be displayed is enhanced based on the zoom display data to obtain an enhanced image to be displayed;

[0250] The transmitted data further includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display. The scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image. The high-frequency texture enhancement data includes information describing the increase in high-frequency texture in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture in the scaled image of the first image compared to the first image.

[0251] In one possible design, the scaling display data is carried in the APP field of the first JPEG image frame, and decoding the scaling display data from the JPEG file includes:

[0252] The zoom display data is decoded from the APP field of the first JPEG image frame.

[0253] In one possible design, when the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; decoding the scaling display data from the JPEG file includes:

[0254] Decode the tone mapping curve used for scaling display from the APP field of the first JPEG image frame;

[0255] or,

[0256] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; decoding the scaling display data from the JPEG file includes:

[0257] Decode the scaling enhancement data from the third JPEG image frame;

[0258] or,

[0259] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data; decoding the scaling display data from the JPEG file includes:

[0260] Decode the high-frequency texture enhancement data from the fourth JPEG image frame.

[0261] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file;

[0262] Decoding the scaling enhancement data from the third JPEG image frame includes:

[0263] The start position of the third JPEG image frame is determined from the JPEG file based on the second position identifier, and the scaling enhancement data is decoded from the third JPEG image frame based on the start position; or,

[0264] Decoding the high-frequency texture enhancement data from the fourth JPEG image frame includes:

[0265] The start position of the fourth JPEG image frame is determined from the JPEG file based on the third position identifier, and the scaling enhancement data is decoded from the fourth JPEG image frame based on the start position.

[0266] In one possible design, the APP field of the first JPEG image frame further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0267] Seventhly, embodiments of this application provide an encoding device, including:

[0268] An encoding module is used to encode the image data of the second image and the first brightness-related data into a bitstream. The bitstream includes a Joint Image Experts Group (JEEG) JPEG file, which carries the image data of the second image and the first brightness-related data. The image data of the second image is obtained by compressing the image data of the first image, and the first image occupies more bits than the second image. The first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image.

[0269] A sending module is used to send the bitstream.

[0270] In one possible design, the JPEG file includes a first JPEG image frame, which carries image data of the second image, and the first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

[0271] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0272] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier. When the data type identifier takes a first value, it indicates that the data type of the first brightness-related data is metadata.

[0273] In one possible design, the encoding module is further configured to:

[0274] At least one tone mapping curve is encoded into the APP field of the first JPEG image frame.

[0275] In one possible design, the JPEG file includes a first JPEG image frame and a second JPEG image frame, wherein the first JPEG image frame carries the encoded data of the second image and the second JPEG image frame carries the first luminance-related data.

[0276] In one possible design, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file.

[0277] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. The file type identifier of the first JPEG image frame is a third value, which is used to indicate that the APP field of the first JPEG image frame carries the encoded data of the second image. The file type identifier of the second JPEG image frame is a fourth value, which is used to indicate that the APP field of the second JPEG image frame carries the first brightness-related data.

[0278] In one possible design, the APP field of the second JPEG image frame includes a data type identifier, and when the data type identifier takes a second value, it indicates that the data type of the first brightness-related data is an image frame type.

[0279] In one possible design, the encoding module is further configured to:

[0280] At least one tone mapping curve is incorporated into the APP field of the second JPEG image frame.

[0281] In one possible design, the at least one tone mapping curve is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device.

[0282] In one possible design, the encoding module is further configured to:

[0283] The scaling display data is encoded into the APP field of the first JPEG image frame;

[0284] The transmitted data also includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image; the high-frequency texture enhancement data includes information describing the increase in high-frequency texture in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture in the scaled image of the first image compared to the first image.

[0285] In one possible design, the JPEG file further includes a third JPEG image frame, and the encoding module is further configured to:

[0286] The scaling display data is incorporated into the JPEG file, and the transmitted data also includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the magnified image of the first image compared to the first image; the high-frequency texture enhancement data includes information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture values ​​in the magnified image of the first image compared to the first image.

[0287] When the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; or,

[0288] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; or...

[0289] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the scaling enhancement data.

[0290] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file.

[0291] In one possible design, the APP field of the first JPEG image frame also includes a scaling enhancement method identifier;

[0292] When the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to superimpose the scaling enhancement data on the image after enhancing the second image according to the first brightness-related data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly superimpose the high-frequency texture enhancement data on the image after enhancing the second image according to the first brightness-related data; or, when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image after enhancing the second image according to the first brightness-related data based on the tone mapping curve used for scaling display.

[0293] Eighthly, embodiments of this application provide a decoding apparatus, including:

[0294] A receiving module is used to receive a bitstream, the bitstream including JPEG files encoded in the Joint Picture Experts Group (JPT) JPEG format;

[0295] The decoding module is used to decode the image data of the second image and the first brightness-related data from the JPEG file;

[0296] The image data of the second image is obtained by compressing the image data of the first image, with the first image occupying a higher number of bits than the second image. The first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image. The first brightness-related data is used to enhance the image data of the second image.

[0297] In one possible design, the decoding device also includes

[0298] The processing module is used to enhance the image data of the second image based on the first brightness-related data to obtain the image to be displayed.

[0299] In one possible design, the JPEG file includes a first JPEG image frame, the first JPEG image frame carrying image data of the second image, and the first brightness-related data carried in the application identifier APP field of the first JPEG image frame;

[0300] The decoding module is specifically used to decode the first JPEG image frame to obtain the image data of the second image, and to obtain the first brightness-related data from the APP field of the first JPEG image frame.

[0301] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0302] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type is a first value indicating that the data type of the first brightness-related data is metadata. The decoding module is specifically used to decode the first brightness-related data from the APP field of the first JPEG image frame when the data type identifier decoded from the APP field of the first JPEG image frame is the first value.

[0303] In one possible design, the decoding module is further configured to decode at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame;

[0304] The processing module is specifically used for:

[0305] A first tone mapping curve is determined from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve;

[0306] The third image is obtained by performing tone mapping processing on the second image based on the first tone mapping curve;

[0307] The pixel values ​​of the pixels in the third image are adjusted based on the first brightness-related data to obtain the image to be displayed.

[0308] In one possible design, the JPEG file includes a first JPEG image frame and a second JPEG image frame, wherein the first JPEG image frame carries image data of the second image and the second JPEG image frame carries the first brightness-related data;

[0309] The decoding module is specifically used to decode the image data of the second image from the first JPEG image frame, and to decode the first brightness-related data from the second JPEG image frame.

[0310] In one possible design, the APP field of the first JPEG image frame further includes a data type identifier, wherein the data type is a second value indicating that the data type of the first brightness-related data is an image frame type. Specifically, the decoding module is used to decode the first brightness-related data from the second JPEG image frame when the data type identifier decoded from the APP field of the first JPEG image frame is the second value.

[0311] In one possible design, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data.

[0312] The decoding module is specifically used to decode the image data of the second image from the first JPEG image frame when the file type identifier decoded from the APP field of the first JPEG image frame is a third value; and to decode the first brightness-related data from the second JPEG image frame when the file type identifier decoded from the APP field of the second JPEG image frame is a fourth value.

[0313] In one possible design, the decoding module is further configured to decode at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame or from the APP field of the second JPEG image frame.

[0314] The processing module is specifically used for:

[0315] A first tone mapping curve is determined from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve;

[0316] The third image is obtained by performing tone mapping processing on the second image based on the first tone mapping curve;

[0317] The pixel values ​​of the pixels in the third image are adjusted based on the first brightness-related data to obtain the image to be displayed.

[0318] In one possible design, the JPEG file also encodes scaling display data; the decoding module is further configured to decode the scaling display data from the JPEG file in response to a user's zoom-in or zoom-out operation on the image to be displayed.

[0319] The processing module is further configured to perform enhancement processing on the image to be displayed based on the scaling display data to obtain an enhanced image to be displayed;

[0320] The transmitted data further includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display. The scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image. The high-frequency texture enhancement data includes information describing the increase in high-frequency texture in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture in the scaled image of the first image compared to the first image.

[0321] In one possible design, the scaling display data is carried in the APP field of the first JPEG image frame, and the decoding module is further configured to decode the scaling display data from the APP field of the first JPEG image frame.

[0322] In one possible design, when the scaling display data includes a tone mapping curve for scaling display, the tone mapping curve for scaling display is encoded in the APP field of the first JPEG image frame; the decoding module is specifically configured to decode the tone mapping curve for scaling display from the APP field of the first JPEG image frame.

[0323] or,

[0324] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; the decoding module is specifically used to decode the scaling enhancement data from the third JPEG image frame;

[0325] or,

[0326] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data; the decoding module is specifically used to decode the high-frequency texture enhancement data from the fourth JPEG image frame.

[0327] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file;

[0328] The decoding module is specifically configured to determine the start position of the third JPEG image frame from the JPEG file based on the second position identifier, and to decode the scaling enhancement data from the third JPEG image frame based on the start position; or,

[0329] The decoding module is specifically configured to determine the starting position of the fourth JPEG image frame from the JPEG file based on the third position identifier, and to decode the scaling enhancement data from the fourth JPEG image frame based on the starting position.

[0330] In one possible design, the APP field of the first JPEG image frame further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0331] Ninthly, embodiments of this application provide a data acquisition device for implementing the methods described in the first or fifth aspects.

[0332] In a tenth aspect, embodiments of this application provide a data acquisition device comprising: a non-volatile memory and a processor coupled to each other, wherein the processor invokes program code stored in the memory to execute the method described in the first aspect or any design of the first aspect, or to execute the method described in the fifth aspect or any design of the fifth aspect. The data acquisition device may further include a sensor for acquiring images.

[0333] In one aspect, embodiments of this application provide a display device for implementing the methods described in the second or sixth aspect.

[0334] In a twelfth aspect, embodiments of this application provide a display device comprising: a non-volatile memory and a processor coupled to each other, wherein the processor calls program code stored in the memory to execute the method described in the second aspect or any design of the second aspect, or to execute the method described in the sixth aspect or any design of the sixth aspect.

[0335] In a thirteenth aspect, embodiments of this application provide an image processing system, including the acquisition end device described in the fifth or sixth aspect, and the display end device described in the eleventh or twelfth aspect.

[0336] In a fourteenth aspect, embodiments of this application provide a computer-readable storage medium storing program code, wherein the program code includes instructions for performing some or all of the steps of any method of the first aspect, or for performing some or all of the steps of any method of the second aspect, or for performing some or all of the steps of any method of the fifth aspect, or for performing some or all of the steps of any method of the sixth aspect.

[0337] In the fifteenth aspect, embodiments of this application provide a computer program product that, when the computer program product is run on a computer, causes the computer to perform part or all of the steps of any method of the first aspect, or causes the computer to perform part or all of the steps of any method of the second aspect, or causes the computer to perform part or all of the steps of any method of the fifth aspect, or causes the computer to perform part or all of the steps of any method of the sixth aspect.

[0338] It should be understood that the beneficial effects of aspects three to fifteen of this application can be found in the relevant descriptions of aspects one to two, and will not be repeated here.

[0339] Based on the implementations provided in the above aspects, this application can be further combined to provide more implementations. Attached Figure Description

[0340] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0341] Figure 1A A schematic block diagram of an image display system provided in an embodiment of this application;

[0342] Figure 1B A schematic block diagram of another image display system provided in the embodiments of this application;

[0343] Figure 2This is a schematic flowchart of the image processing method provided in the embodiments of this application;

[0344] Figure 3 This is a schematic diagram of the image acquisition method executed by the acquisition terminal device provided in the embodiments of this application;

[0345] Figure 4 This is a schematic flowchart of the image encoding method provided in the embodiments of this application;

[0346] Figure 5A This is a schematic flowchart of an image display method executed by a display device provided in an embodiment of this application.

[0347] Figure 5B A schematic diagram of the screen backlight adjustment process of the display device provided in the embodiments of this application;

[0348] Figure 6 This is a schematic flowchart of an image decoding method provided in an embodiment of this application;

[0349] Figure 7 This is a schematic diagram of an image acquisition device according to an embodiment of this application;

[0350] Figure 8 This is a schematic diagram of an image display device according to an embodiment of this application;

[0351] Figure 9 This is a schematic diagram of an encoding device in an embodiment of this application;

[0352] Figure 10 This is a schematic diagram of a decoding device according to an embodiment of this application;

[0353] Figure 11 This is a schematic diagram of the data acquisition device structure in an embodiment of this application;

[0354] Figure 12 This is a schematic diagram of the display device structure in an embodiment of this application. Detailed Implementation

[0355] The terms "first," "second," etc., in the specification, embodiments, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as including a series of steps or units. A method, system, product, or apparatus is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or apparatuses.

[0356] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0357] To facilitate understanding of the technical solutions of the embodiments of this application, relevant technical terms and physical concepts will be introduced first.

[0358] (1) A pixel is the basic unit that makes up an image. The color of a pixel is usually described by several (for example, three) relatively independent attributes. The combined effect of these independent attributes naturally constitutes a spatial coordinate, that is, a color space. In the embodiments of this application, the independent attributes that make up a pixel are called the color components of each pixel or the pixel value of each pixel. For example, R component, G component, B component or Y component.

[0359] (2) Metadata: Records key information about images in a video, scene, or frame. Static metadata: Metadata that remains unchanged throughout the sequence. Dynamic metadata: Metadata that changes as the image or scene changes.

[0360] (3) Image bit depth: refers to the number of bits required to display the colors in an image. For example, if each color (R, G, B) is represented by one byte (8 bits), each image can have 16.7 million colors; if each color (R, G, B) is represented by two bytes (16 bits), each image can have 10 to the power of 12 colors; if it is a grayscale image, each pixel is represented by one byte (8 bits), and an image can have 256 gray levels; if each pixel is represented by two bytes (16 bits), an image can have 65,536 gray levels.

[0361] (4) Dynamic backlight technology (content adaptive backlight control, CABC) is an existing backlight adjustment technology that controls the global backlight according to the display environment and display content to save power. It is currently mainly used in mobile devices such as mobile phones.

[0362] In edge-lit local dimming, all backlight sources for the display are located at the edge of the screen facing the center. The edge backlight is divided into multiple zones, providing backlight dimming control over these areas of the screen. The more zones are divided, the finer the backlight control over the displayed content.

[0363] In full-array local dimming, the display panel typically has a series of individual backlight areas to illuminate the screen, rather than using any type of edge-lit backlight. Each backlight area is independently controlled, dividing the display screen into many small, individual "areas," and local dimming is performed on these small areas. For the content appearing on the screen at the same time, different numbers of bright and dark areas can be identified, and local dimming is applied to each area based on its varying brightness to improve contrast. The more and smaller the divided areas, the better the dimming control of the displayed content, and the higher the contrast.

[0364] (5) Histogram Information: Histogram information is a histogram used to represent the brightness distribution in a digital image, depicting the number of pixels for each brightness value in the image. Observing this histogram can help understand how the brightness distribution needs to be adjusted. In this histogram, the left side of the horizontal axis represents pure black, darker areas, while the right side represents brighter, pure white areas. Therefore, the data in the histogram of a darker image is mostly concentrated on the left and in the middle; while the opposite is true for an image that is generally bright with only a few shadows.

[0365] (6) Backlight brightness: In this application, it refers to the brightness of the screen light source. Backlight brightness usually refers to the brightness of the light source that illuminates the screen (such as an LCD), while self-emissive screens usually do not use the term "backlight". For ease of expression, the "backlight brightness" mentioned in this application includes the self-emissive brightness of a self-emissive screen.

[0366] Screen backlight parameters refer to the maximum brightness displayed on the screen or the maximum display brightness supported by the screen, such as 600 nits. Screen backlight parameters can be a maximum display brightness for the entire screen, or a maximum display brightness corresponding to different areas of the screen.

[0367] (7) Tone Mapping Curve:

[0368] The acquisition device uses light signals from a natural scene to generate images or videos containing that scene. To facilitate image or video transmission, the light signals need to be converted into electrical signals, and image information for each pixel is recorded using fixed-range brightness or chromaticity values ​​(e.g., grayscale values ​​between 0 and 255). The curve that converts light signals into electrical signals is called the photoelectric transfer function. Common photoelectric transfer functions include the PQ (perceptual quantizer) photoelectric transfer function, the HLG (hybrid log-gamma) photoelectric transfer function, and the scene luminance fidelity (SLF) photoelectric transfer function. The display device can reproduce the natural scene by displaying the image or video. The display device can determine the brightness of an object at the time of capture based on the photoelectric transfer function and the brightness or grayscale values ​​of each pixel in the image; that is, the electronic device can convert YUV or RGB information into brightness in nits. However, the brightness of the object may exceed the display capability of the display device. Because the brightness information acquired by the acquisition device does not match the brightness display capability of the display device, there are situations where a high-brightness image is displayed on a display device with low brightness display capability, and vice versa. Both situations require tone mapping of the image acquired by the acquisition device to ensure the image matches the display device's capabilities. For example, a high dynamic range image can be tone-mapped onto a display device with low dynamic range display capability based on a tone mapping curve.

[0369] (7) Metadata: Used to describe key information or features needed during video or image processing. Static metadata is metadata that remains unchanged in the video. Dynamic metadata is metadata that changes as a video or image changes.

[0370] (8) Description method of bitstream syntax:

[0371] Bitstream syntax is described similarly to the C programming language. Bitstream syntax elements are represented in bold, and each element is described by its name (a group of lowercase English letters separated by underscores), syntax, and semantics. Values ​​of syntax elements in the syntax table and text are represented in regular font.

[0372] In some cases, additional variable values ​​derived from grammar elements can be applied to the grammar table. Such variables are named in the grammar table or text using lowercase underscores, or a mixture of lowercase and uppercase letters. Variables beginning with uppercase letters are used to decode the current and related grammar structures, and can also be used to decode subsequent grammar structures. Variables beginning with lowercase letters are only used within the section in which they are located.

[0373] The relationship between the mnemonics for syntax element values ​​and variable values ​​and their values ​​is explained in the text. In some cases, the two are used interchangeably.

[0374] When the length of a bit string is a multiple of 4, it can be represented using hexadecimal notation. The hexadecimal prefix is ​​"0x", for example, "0x1a" represents the bit string "0001 1010".

[0375] In a conditional statement, 0 represents FALSE, and a non-zero value represents TRUE.

[0376] The syntax table describes all supersets that conform to the bitstream syntax of this document; additional syntax restrictions are described in the relevant clauses.

[0377] See Figure 1A and Figure 1B The diagram shown is a schematic block diagram of an exemplary image display system provided in an embodiment of this application. The image display system may include a data acquisition device 100 and a display device 200. For example, the data acquisition device 100 can acquire images, which may be HDR images or standard dynamic range (SDR) images. The display device 200 can display the images acquired by the data acquisition device 100.

[0378] The acquisition device 100 and the display device 200 can communicate via a link 102, allowing the display device 200 to receive image data from the acquisition device 100 via the link 102. The link 102 may include one or more communication media or devices. The one or more communication media may include wireless and / or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. Optionally, the one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (e.g., the Internet). The one or more communication media may include a router, switch, base station, or other devices facilitating communication from the acquisition device 100 to the display device 200. In an optional case, the link 102 may be a Bluetooth wireless link.

[0379] For example, the acquisition device 100 includes an image source 11 and a communication interface 12. The communication interface 12 can also be called a transmission interface. Alternatively, the acquisition device 100 may also include an image processor 13. In specific implementations, the image processor 13, the image source 11, and the communication interface 12 may be hardware components in the acquisition device 100, or they may be software programs in the acquisition device 100, or the communication interface 12 may be a combination of a hardware module and a software program.

[0380] Image source 11 may include or be any type of image capture device for, for example, capturing real-world images, and / or any type of image or commentary (for screen content encoding, some text on the screen is also considered part of the image or picture to be encoded) generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-animated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). For example, image source 11 may be a camera for capturing images or a memory for storing images. Image source 11 may also include any type of (internal or external) interface for storing previously captured or generated images and / or acquiring or receiving images. When image source 11 is a camera, image source 11 may be, for example, a local or integrated camera in the acquisition device. When image source 11 is a memory, image source 11 may be local or, for example, an integrated memory in the acquisition device. When the image source 11 includes an interface, the interface may be, for example, an external interface for receiving images from an external video source. The external video source may be, for example, an external image capture device, such as a camera, external storage, or an external image generation device. The external image generation device may be, for example, an external computer graphics processor, a computer, or a server.

[0381] An image can be viewed as a two-dimensional array or matrix of pixels. The pixels in the array can also be called sampling points. In one alternative case, to represent color, each pixel includes three color components. For example, in RGB format or color space, an image includes corresponding red, green, and blue sampling arrays. However, in video encoding, each pixel is typically represented in a luma / chroma format or color space. For example, for a YUV format image, this includes a luma component indicated by Y (sometimes also indicated by L) and two chroma components indicated by U and V. The luma component Y represents the brightness or grayscale level intensity (e.g., both are the same in a grayscale image), while the two chroma components U and V represent chroma or color information components. Accordingly, a YUV format image includes a luma sampling array of luma sample values ​​(Y) and two chroma sampling arrays of chroma values ​​(U and V). RGB format images can be converted or transformed to YUV format, and vice versa; this process is also called color conversion or color format conversion. If the image is black and white, it may only include a luma sampling array. In this embodiment of the application, the image transmitted from the image source 11 to the image processor can also be referred to as the original image data.

[0382] Image processor 13 is used to perform image processing, such as brightness mapping, tone mapping, color format conversion (e.g., from RGB to YUV format), color gamut conversion, saturation adjustment, color correction, resolution adjustment, or noise reduction.

[0383] Communication interface 12 can be used to receive image data that has undergone image processing, and can transmit the image data to display device 200 for further image processing via link 102, or to memory for storage. For example, communication interface 12 can be used to encapsulate the image data into a suitable format, such as data packets, for transmission over link 102.

[0384] In some embodiments, the image processor also has an encoding function for encoding the processed image data.

[0385] In other embodiments, the acquisition device 100 may also include an encoder 14. The encoder is used to encode the image data processed by the image processor 13, see [link to relevant documentation]. Figure 1B As shown.

[0386] The display device 200 includes a communication interface 21, an image processor 22, and a display device 23. These are described below:

[0387] Communication interface 21 can be used to receive image-processed image data from acquisition device 100 or any other source, such as a storage device. Specific examples of communication interfaces 12 and 21 can be found in the foregoing description of the interfaces, and will not be repeated here. Communication interface 21 can be used to transmit or receive image-processed image data via link 102 between acquisition device 100 and display device 200 or any other type of network. Communication interface 21 can, for example, be used to decapsulate data packets transmitted by communication interface 12 to obtain the image-processed image data.

[0388] Both communication interface 21 and communication interface 12 can be configured as unidirectional or bidirectional communication interfaces, and can be used, for example, to send and receive messages to establish connections, acknowledge and exchange any other information related to the communication link and / or, for example, image data processed by image processing and / or data transmission. For example, communication interface 21 and communication interface 12 can be any type of interface according to any proprietary or standardized interface protocol, such as High Definition Multimedia Interface (HDMI), Mobile Industry Processor Interface (MIPI), MIPI-standardized Display Serial Interface (DSI), Video Electronics Standards Association (VESA) standardized Embedded Display Port (eDP), Display Port (DP), or V-By-One interface (a digital interface standard developed for image transmission), as well as various wired or wireless interfaces, optical interfaces, etc.

[0389] Image processor 22 is used to perform tone mapping processing on image data that has undergone image processing, so as to obtain tone-mapped image data. The processing performed by image processor 22 may also include: super-resolution, color format conversion (e.g., from YUV format to RGB format), noise reduction, color gamut conversion, saturation adjustment, brightness mapping, upsampling, downsampling, and image sharpening, etc., and may also be used to transmit tone-mapped image data to display device 23. It should be understood that image processor 13 and image processor 22 can be general-purpose central processing units (CPUs), systems on chips (SOCs), processors integrated on SOCs, separate processor chips, or controllers, etc.; image processor 22 and image processor 13 can also be dedicated processing devices, such as application-specific integrated circuits (ASICs), image signal processors (ISPs), field-programmable gate arrays (FPGAs) or digital signal processors (DSPs), dedicated video or graphics processors, graphics processing units (GPUs), and neural-network processing units (NPUs), etc. Image processor 13 and image processor 22 can also be processor groups composed of multiple processors, which are coupled to each other through one or more buses.

[0390] Display device 23 is used to receive tone-mapped image data to display the image to a user or viewer. Display device 23 can be or may include any type of display for presenting a reconstructed image, such as an integrated or external display or monitor. For example, the display may include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display. In an optional case, display device 23 itself has image processing capabilities, and tone mapping of the image can also be implemented at display device 23. Alternatively, display device 23 may not be located in display end device 200; it is understood that display end device 200 does not have display functionality.

[0391] In some embodiments, the image processor 23 also has an encoding function for encoding the processed image data.

[0392] In other embodiments, the display device 200 may also include a decoder 24. The decoder 24 is used to receive and decode the bitstream, see [link to documentation]. Figure 1B As shown.

[0393] In one alternative scenario, an image processing device can simultaneously function as both the acquisition device 100 and the display device 200. For example, a smartphone may have a camera, an image processor, and a display screen, where the camera corresponds to the acquisition device 100, the image processor corresponds to the display device 200, and the display screen corresponds to the display device 23. Similarly, a smart TV may have a webcam, an image processor, and a display screen, where the webcam corresponds to the acquisition device 100, the image processor corresponds to the display device 200, and the display screen corresponds to the display device 23.

[0394] In one possible scenario, the acquisition device 100 needs to transmit a video stream consisting of multiple consecutive images to the display device 200. Before transmitting the video stream to the display device 200, the acquisition device 100 can encode each image in the video stream into a bitstream and transmit it to the display device 200 via link 102. In this scenario, after receiving the encoded bitstream, the display device 200 decodes the individual images in the video stream from the bitstream and then performs further processing on each image. As an example, in this scenario, the acquisition device 100 can be called an encoder or encoding device, and the display device 200 can be called a decoder or decoding device.

[0395] In one optional embodiment, the display device includes a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on top of the operating system layer. The hardware layer includes hardware such as a CPU, a memory management unit (MMU), and memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as Linux, Unix, Android, iOS, or Windows. The application layer includes applications such as browsers, address books, word processing software, and instant messaging software. Furthermore, this application embodiment does not specifically limit the specific structure of the execution subject of the method provided in this application embodiment, as long as it can perform image processing by running relevant code. For example, the execution subject of the method provided in this application embodiment can be an electronic device, or a functional module in an electronic device capable of calling and executing programs, such as a processor in an electronic device.

[0396] It should be understood that the acquisition device 100 and the display device 200 may include any type of handheld or stationary device, such as a laptop or laptop computer, mobile phone, smartphone, tablet or tablet computer, camera, desktop computer, set-top box, television, camera, in-vehicle device, display device, digital media player, video game console, video streaming device (e.g., content service server or content distribution server), broadcast receiver device, broadcast transmitter device, etc., and may or may not use any type of operating system.

[0397] Currently, when capturing images, acquisition devices use multiple frames with different exposures for fusion or employ high dynamic range sensors to record the high dynamic range of real-world scenes, resulting in images with higher bit depths. However, to preserve highlight details, image processing is often used to save lower bit depth (e.g., 8-bit) JPG images or videos, requiring highlight area compression. The image data from the acquisition device is compressed and encoded before being distributed. The display device's screen brightness typically reaches 600-1200 nits, but it adaptively adjusts its brightness according to ambient light. When the ambient light is below 1000 nits, it typically only uses around 200 nits. Regardless of the brightness level, the decoded image data is directly sent to the display device, leading to a significant difference in display quality between the acquisition and display devices.

[0398] Based on this, embodiments of this application provide an image acquisition method, an image display method, and an apparatus to reduce the difference between the image from the acquisition device and the image displayed by the display device.

[0399] See Figure 2 The diagram shows a schematic flow chart of an image processing method provided in an embodiment of this application. Steps 201-204 of this method can be applied to an acquisition device. For example, steps 201-204 of this method can be implemented by one or more processors. These one or more processors may include... Figure 1A or Figure 1B The image processor in the acquisition device 100 shown. Or, in other words, Figure 2 Steps 201-204 shown can be implemented at the encoding end. Steps 205-207 of this method can be applied to a display device. For example, steps 205-207 of this method can be implemented by one or more processors. These one or more processors may include... Figure 1A or Figure 1B The image processor in the display device 200 shown. Or, in other words, Figure 2 Steps 205-207 shown are implemented at the decoding end.

[0400] 201. The acquisition device acquires the first image to obtain the image data of the first image. It should be noted that the first image can be an image captured by an image sensor, or an image obtained by fusing multiple frames of images with different exposures acquired by the image sensor, or it can be a video frame image from a captured video stream.

[0401] In one possible design, the image data format of the first image may include one of the following: RAW format, RGB format or YUV format, HSV format, Lab format, CMY format or YCbCr format.

[0402] For example, the image frame format acquired by the image sensor in the acquisition device can be RAW format. Depending on the sensor design, the RAW format will vary. For instance, RAW formats can include Bayer RGGB, RYYB, RCCC, RCCB, RGBW, CMYW, and many others. In some embodiments, the acquisition device may also include an ISP to convert RAW images of various formats into RGB format. The ISP can also convert RAW format to the first of the following formats: YUV, HSV, Lab, CMY, or YCbCr.

[0403] It should be noted that the first image captured by the acquisition device uses a high number of bits to represent the relative brightness relationships of the scene as much as possible, and no data compression is required. For example, the first image may use 12 bits.

[0404] 202. The acquisition device compresses the first image based on the first image data to obtain the image data of the second image, wherein the number of bits occupied by the first image is higher than the number of bits occupied by the second image.

[0405] In this embodiment, the compression process involves compressing high-bit-count image data into low-bit-count image data. For ease of distinction from the first image, the compressed image is referred to as the second image.

[0406] 203. The acquisition device obtains the transmission data of the second image based on the image data of the first image and the image data of the second image.

[0407] For example, brightness-related data can be obtained first based on the image data of the first image and the image data of the second image. To distinguish it from subsequent brightness-related data, the obtained brightness-related data is referred to here as first brightness-related data. The transmission data of the second image includes the encoded data of the second image and the first brightness-related data. The first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image. In this embodiment, the brightness-related data can also be called brightness enhancement relationship data, brightness enhancement data, lightmap data, HDR layer data, or HDR layer data, etc. This embodiment does not specifically limit the terminology.

[0408] In one possible example, the first brightness-related data can be determined based on the ratio of the brightness value of a pixel in the first image to the brightness value of the corresponding pixel in the second image. Taking the first pixel as an example, the first brightness-related data can be the ratio of the brightness value of the first pixel in the first image to the brightness value of the first pixel in the second image, or a value after adjustment of the ratio. The specific adjustment method will be described in detail later and will not be repeated here. In another example, the first brightness-related data can also be the change in the brightness value of a pixel in the second image relative to the brightness value of the corresponding pixel in the first image, or a value after adjustment of the change value. The specific adjustment method will be described in detail later and will not be repeated here. Taking the first pixel as an example, the first brightness-related data can be the change in the brightness value of the first pixel in the second image relative to the brightness value of the first pixel in the second image.

[0409] In another possible example, the image can be divided into image blocks, and the first brightness-related data of the image block can be determined based on the brightness values ​​of corresponding image blocks in the two images. The brightness value of the image block can be the ratio or variation between the average brightness (or maximum brightness value or minimum brightness value) of the pixels included in the image block. For example, both the first image and the second image are divided into Km*Kn image blocks. Km and Kn are both positive integers. Taking the ratio as an example, the first brightness-related data of the i-th image block can be the ratio of the average brightness of the pixels included in the i-th image block in the first image to the average brightness of the pixels included in the i-th image block in the second image, or the ratio of the maximum brightness of the pixels included in the i-th image block in the first image to the maximum brightness of the pixels included in the i-th image block in the second image, or the ratio of the minimum brightness of the pixels included in the i-th image block in the first image to the minimum brightness of the pixels included in the i-th image block in the second image. The value of i is greater than 0 and less than or equal to Km*Kn.

[0410] In some possible embodiments, in order to reduce the number of bits occupied by the first luminance-related data during the encoding and transmission process, the image data of the first image and the image data of the second image can be normalized first, and then the normalized data can be combined to obtain the first luminance-related data.

[0411] For example, the image data of the first image and the image data of the second image can be normalized by combining the bit depth of the first image and the bit depth of the second image. For ease of distinction, the normalized data obtained after normalizing the first image is called the first normalized data, and the normalized data obtained after normalizing the second image is called the second normalized data. Further, first brightness-related data is obtained based on the first and second normalized data. Taking the ratio of the brightness value of a pixel in the first image to the brightness value of the corresponding pixel in the second image as an example, after normalization, the ratio of the brightness value of a pixel in the first normalized data to the brightness value of the corresponding pixel in the second normalized data can be used as the brightness-related data of that pixel.

[0412] For example, by normalizing the image data of the second image with a low bit depth to [0, X], the range of normalized data for the first image can be further determined based on the bit depth of the first and second images. For instance, if the first image has N bits and the second image has M bits, and the range of normalized data for the first image is [0, X], then the range of normalized data for the second image can be determined to be [0, X]; where Y / X = 2. M-NFor example, if the first image has 8 bits, its image data can be normalized to [0, 1.0]. If the second image has 12 bits, its image data can be normalized to [0, 16.0].

[0413] In some embodiments, the second image needs to be encoded before data transmission. Specifically, the original image can be converted into a media data format that is easily recognizable by the human eye and occupies less storage space. Examples of image media data formats include: .ipg, .jpg, .bmp, .tga, .png, and .gif. Examples of video frame media formats include: MPEG, AVI, nAVI, ASF, MOV, WMV, 3GP, RM, RMVB, FLV / F4V, H.264, and H.265. For example, when encoding the first luminance-related data, the encoding format of the second image's encoded data can be used. For instance, if the second image's encoded data uses .jpg format, the second image is encoded as a .jpg image, and the first luminance-related data can also be encoded as a .jpg image. Then, the .jpg image encoded by the second image and the .jpg image encoded by the first luminance-related data are linked together. Alternatively, the image data of the second image and the first luminance-related data can be encoded into a single .jpg image.

[0414] 204. The data acquisition device sends the transmitted data.

[0415] The display device is used to perform steps 205-207.

[0416] 205. The display device receives the transmission data of the second image sent by the acquisition device. The transmission data includes the encoded data of the second image and the first brightness-related data.

[0417] 206. The display device determines the tone mapping curve and performs tone mapping processing on the second image according to the first tone mapping curve to obtain the third image.

[0418] 207. The display device adjusts the pixel values ​​of the third image based on the brightness-related data to obtain the image to be displayed.

[0419] In current solutions, while the acquisition device retains a significant amount of highlight detail during image capture, it fails to fully record the relative brightness relationships of the real scene, thus weakening these relationships. This embodiment, however, transmits image details from the acquisition device to the display device via first brightness-related data. The display device can then adjust the pixel values ​​of the received image based on this first brightness-related data to obtain the image to be displayed, making it closer to the image captured by the acquisition device.

[0420] In some scenarios, the scene captured by the acquisition device may include bright objects, such as the sun or jewelry. High-bit-depth images acquired by the acquisition device may also contain overexposed areas. Although the loss of bright area information due to bit width and ISP processing can be addressed using first brightness correlation data, the human eye's response is non-linear, while the pixel-level or block-level ratio in the first brightness correlation data is linear. To improve accuracy, the brightness correlation data of the pixels or image blocks corresponding to the bright objects can be adjusted using photoelectric conversion curves, ensuring that the adjusted first brightness correlation data satisfies the conversion relationship of the photoelectric conversion curve. For ease of distinction, the adjustment of the brightness correlation data in the area containing the bright objects is referred to as the first adjustment of the brightness correlation data.

[0421] In some embodiments, since the human eye is non-linear, when performing compression processing, such as compressing a 12-bit image into an 8-bit image, an S-curve can be further used to compress the 8-bit image to enhance image details. However, using an S-curve may cause the brightness values ​​of some pixels in the second image to be higher than the corresponding pixel values ​​in the first image. Therefore, the determined brightness-related data (ratio or adjustment value) can be further adjusted so that the brightness values ​​of pixels in the second image are no longer higher than the corresponding pixel values ​​in the first image. For ease of distinction, this adjustment is referred to as the second adjustment of brightness-related data. It should be noted that the brightness-related data of the highlighted object area can be adjusted first, i.e., the first adjustment of brightness-related data, and then this adjustment can be performed, i.e., the second adjustment of brightness-related data. Alternatively, this adjustment can be performed first, i.e., the second adjustment of brightness-related data, and then the brightness-related data of the highlighted object area can be adjusted, i.e., the first adjustment of brightness-related data. This application embodiment does not specifically limit this approach.

[0422] In some embodiments, the scene captured by the acquisition device may not include bright objects. In this case, the adjustment of the brightness-related data of the area where the bright object is located may not be performed. In this case, after determining the ratio or adjustment value, the first brightness-related data of each pixel or each image block can be obtained after a second adjustment of the brightness-related data.

[0423] The following describes the method flow executed by the acquisition device in this embodiment of the application, taking a scenario with highlighted objects as an example. See Figure 3 As shown. Figure 3 Taking pixel-level ratios as an example, the brightness-related data between China and Israel are used.

[0424] 301. The image sensor in the acquisition device acquires a first image with a high bit depth to obtain the image data of the first image. Taking a first image with a bit depth of 12 bits as an example.

[0425] 302. The ISP in the acquisition device compresses the first image to obtain the image data of the second image. For example, the second image has 8 bits.

[0426] For example, when the ISP of the acquisition device compresses the first image, it can use a compression curve to compress the first image to obtain the image data of the second image.

[0427] In some embodiments, after compressing the 12-bit first image into an 8-bit image, an S-curve can be used to enhance the details of the 8-bit image to obtain a second image. For example, a compression curve for compressing the 12-bit first image into a second image can be obtained.

[0428] 303a, the acquisition device normalizes the image data of the first image to obtain normalized data of the first image. For example, normalizing the image data of the second image to 0-16.0.

[0429] 303b, the acquisition device normalizes the image data of the second image to obtain normalized data of the second image. For example, normalizing the image data of the first image to 0-1.0.

[0430] 304. The acquisition device identifies bright objects in the first image, such as the sun or jewelry. When identifying bright objects, a defined highlight classification network can be used to identify them from the first image, such as a neural network. This application embodiment does not specifically limit the highlight classification network used to identify bright objects.

[0431] 305. The acquisition device performs ratio processing on the normalized data of the first image and the normalized data of the second image to obtain the brightness-related data of each pixel. For ease of distinction, this ratio is referred to as the second brightness-related data.

[0432] For example, the second brightness-related data satisfies the conditions shown in formula (1).

[0433]

[0434] Where hdrlayeri represents the second brightness-related data of the i-th pixel. This represents the normalized data of the pixel value of the i-th pixel in the first image. This represents the normalized pixel value of the i-th pixel in the second image. I is a positive integer less than or equal to the number of pixels in either the first or second image.

[0435] 306. The acquisition device corrects the second brightness-related data to obtain the first brightness-related data. For example, it corrects the bright object area, which is the first adjustment of the brightness-related data. Another example is to perform a second adjustment on the first adjusted brightness-related data based on a compression curve. For instance, if the compression curve is processed into an S-curve, the second adjustment can be performed.

[0436] Before outputting the second image, the acquisition device further encodes it. In some embodiments, in addition to encoding the first luminance-related data and the image data of the second image, at least one tone mapping (TM) curve and / or screen-related information may also be encoded. In this case, the transmitted data may include at least one tone mapping curve and / or screen-related information in addition to the first luminance-related data and the encoded data of the second image. In one possible example, the TM curve can be generated by combining the screen backlight parameters of the acquisition device and the histogram information of the second image. In some scenarios, histogram statistics can be performed on the second image in blocks to generate TM curves corresponding to different blocks. For example, the second image is divided into M*N image blocks. Histogram statistics are performed on each image block to generate TM curves corresponding to different blocks. It should be noted that the method of generating TM curves is not limited in the embodiments of this application. For example, histogram equalization methods or other methods may be used, and the embodiments of this application do not specifically limit this. In another possible example, the relationship between the screen parameters of the acquisition device and the set screen parameters can be generated by configuration. For example, if the screen parameter of the acquisition device is 1000 nits, corresponding to screen parameters of 500 nits, 200 nits, and 10 nits, then, for instance, the same scene can be displayed on four object screens with parameters of 1000 nits, 500 nits, 200 nits, and 10 nits respectively. Colorists can then use adjustment algorithms or software to adjust the 500 nits, 200 nits, and 10 nits screens to make their display effect as close as possible to that of the 1000 nits screen. This yields tone mapping curves for the 1000 nits screen and the corresponding 500 nits, 200 nits, and 10 nits screens. In some embodiments, other processing methods can also be used to obtain tone mapping curves between the screen parameters of different acquisition devices and multiple screens with set screen parameters.

[0437] In some embodiments, screen-related information may be the screen backlight parameters of the acquisition device, or the mapping relationship between the backlight brightness under the screen backlight parameters of the acquisition device and the backlight brightness under at least one set screen backlight parameter, or the adjustment ratio of the backlight brightness under at least one set screen backlight parameter relative to the backlight brightness under the screen backlight parameters of the acquisition device.

[0438] For example, taking a JPG image as the encoding format for the second image, when encoding the first brightness-related data, the image data of the second image, at least one tone mapping (TM) curve, and / or screen-related information, the image data of the second image can be encoded into a JPG image, and then the first brightness-related data can be encoded into another JPG image. The encoded data of the two JPG images can then be concatenated. At least one tone mapping (TM) curve and / or screen-related information can be carried as metadata in the APP field of the JPG image. This application does not specifically limit the specific encoding method.

[0439] For example, the encoded data of the second image can be obtained using the Joint Photographic Experts Group (JPEG) format.

[0440] See Figure 4 The diagram shown is a schematic flowchart of an image encoding method provided in an embodiment of this application. The method utilizes an encoding device.

[0441] 401. The image data of the second image and the first brightness-related data are encoded into a bitstream, the bitstream including a Joint Panel of Image Experts (JPI) JPEG file, the JPEG file carrying the image data of the second image and the first brightness-related data; the image data of the second image is obtained by compressing the image data of the first image, the first image occupying more bits than the second image; the first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image;

[0442] 402, send the bitstream.

[0443] The first luminance-related data and the second image data can be encoded in the same image frame or in different image frames. Encoding in different image frames can be understood as encoding the second image data into one JPEG image frame and the first luminance-related data into another JPEG image frame. For example, one or more JPEG image frames are concatenated with a JPEG image frame containing the encoded data of the second image.

[0444] In some embodiments, the aforementioned at least one tone mapping (TM) curve and / or screen-related information may be encoded in the same JPEG image frame as the image data of the second image, or may be encoded separately in another JPEG image frame, independent of the JPEG image frame encoding the image data of the second image.

[0445] The following is an introduction to the format of JPEG image frames. JPEG image frames are the most commonly used image file format, with the file extension .jpg or .jpeg. Taking JFIF (JPEG file interchangeformat) as an example, a JPEG image frame can include a Start of Image (SOI) field, an image frame header, a payload, and an End of Frame (EOI) field.

[0446] The SOI field records markers indicating the start of the original image. The image frame header may record information such as the encoding scheme (JPEG), image size, quantization table, Huffman table, etc. The payload records the actual encoded original image. The EOI field records markers indicating the end of the original image.

[0447] The image frame header may include one or more of the following fields: application identifier (APP), start of frame (SOF), defined Huffman table (DHT), defined restart interval (DRI), and start of scan (SOS). The APP field may include APP0, APP1, ..., APPn. n = 0 to 15 (optional).

[0448] (1) The mark code of the SOI field occupies 2 bytes and is generally a fixed value of 0xFFD8.

[0449] (2) APP0 field. The marker code occupies 2 bytes and is generally a fixed value of 0xFFE0. The APP0 field can include 9 specific fields:

[0450] ① Data length, occupying 2 bytes, used to represent the total length of fields ① to ⑨ (9 fields). That is, excluding the marker code, but including this field.

[0451] ②Identifier, occupying 5 bytes, is generally a fixed value of 0x4A46494600, i.e., the string "JFIF0".

[0452] ③ Version number, occupying 2 bytes, is usually a fixed value of 0x0102, which represents the JFIF version number 1.2. Other values ​​may represent other versions.

[0453] ④ The density units for X and Y occupy 1 byte and have three possible values: 0 indicates no unit; 1 indicates dots per inch; and 2 indicates dots per centimeter.

[0454] ⑤ X-direction pixel density, occupying 2 bytes.

[0455] ⑥Y-direction pixel density, occupying 2 bytes.

[0456] ⑦ Number of horizontal pixels in the thumbnail, occupying 1 byte.

[0457] ⑧ Thumbnail vertical pixel count, occupying 1 byte.

[0458] ⑨ Thumbnail RGB bitmap, occupying a length that may be a multiple of 3, representing thumbnail RGB bitmap data. This tag segment can contain a miniature version of the image, stored as 24 bits of RGB pixels. If there is no miniature image (which is more common), the values ​​of field ⑦ "Number of horizontal pixels in thumbnail" and field ⑧ "Number of vertical pixels in thumbnail" are both 0.

[0459] (3)APPn, Application, application identifier n, where n = 1 to 15 (optional).

[0460] The marker code occupies 2 bytes and is generally a fixed value from 0xFFE1 to 0xFFF.

[0461] APPn can contain two specific fields:

[0462] ① Data length 2 bytes ① to ② The total length of the two fields, excluding the marker code but including this field.

[0463] ②Detailed information, typically 2 bytes in length.

[0464] (4) DQT: The marker code occupies 2 bytes and is usually a fixed value of 0xFFDB.

[0465] DQT can contain 9 specific fields:

[0466] ① Data length, occupying 2 bytes, is used to represent the total length of field ① and multiple fields ②, that is, excluding the marker code, but including this field.

[0467] ② The quantization table occupies 2 bytes of data. The quantization table includes the following a) and b).

[0468] a) Precision and quantization table ID, generally occupying 1 byte. The high 4 bits represent the precision, with a value of 0 indicating 8 bits and a value of 1 indicating 16 bits. The low 4 bits: quantization table ID, with a value range of 0 to 3.

[0469] b) Table entries typically occupy (64 × (precision + 1)) bytes. For example, an 8-bit precision quantization table has an entry length of 64 × (0 + 1) = 64 bytes. In this marker segment, field ② can appear repeatedly, indicating multiple quantization tables.

[0470] (5) SOF0, the marker code is 2 bytes, usually a fixed value of 0xFFC0.

[0471] SOF0 can contain 9 specific fields:

[0472] ① Data length, occupying 2 bytes, represents the total length of the six fields ① to ⑥, excluding the marker code, but including this field.

[0473] ② Precision, occupying 1 byte, represents the number of bits for each data sample. Usually 8 bits.

[0474] ③ Image height, occupying 2 bytes, is used to represent the image height (unit: pixels).

[0475] ④ Image width, occupying 2 bytes, is used to represent the image width (unit: pixels).

[0476] ⑤ Number of color components, occupying 1 byte. A value of 1 indicates a grayscale image; a value of 3 indicates YCrCb or YIQ; and a value of 4 indicates CMYK. JFIF uses YCrCb, so the number of color components here is 3.

[0477] ⑥ Color component information, occupying 3 bytes (usually 9 bytes) of the number of color components. Color component information includes the following:

[0478] a) Color component ID, occupying 1 byte.

[0479] b) Horizontal / vertical sampling factor, occupying 1 byte. Generally, the high 4 bits represent the horizontal sampling factor, and the low 4 bits represent the vertical sampling factor.

[0480] c) Quantization table, occupying 1 byte, is the ID of the quantization table used by the current component. In this marker segment, field ⑥ should appear repeatedly, indicating the number of times each color component (field ⑤) will appear.

[0481] (6) The DHT tag code occupies 2 bytes and is usually a fixed value of 0xFFC4.

[0482] A DHT field can contain two specific fields:

[0483] ① Data length, occupying 2 bytes, represents the total length of field ① and multiple fields ②, that is, excluding the marker code, but including this field.

[0484] ② Huffman tables, with a data length of 2 bytes. Huffman tables include the following:

[0485] a) Table ID and table type, occupying 1 byte. The high 4 bits represent the type, with a value of 0 indicating DC and a value of 1 indicating AC. The low 4 bits represent the Huffman table ID.

[0486] b) The number of codewords with different bit lengths generally occupies 16 bytes.

[0487] c) Encoded content, the sum of the number of codewords of 16 different bit lengths (bytes).

[0488] In this tag section, field ② may appear repeatedly or only once.

[0489] (7) The DRI tag code occupies 2 bytes and is usually a fixed value of 0xFFDD.

[0490] DRI can contain two specific fields:

[0491] ① The data length is 2 bytes, usually a fixed value of 0x0004, which represents the total length of the two fields ① and ②, that is, excluding the marker code, but including this field.

[0492] ② The restart interval within a Minimum Coded Unit (MCU) block. It occupies 2 bytes. For example, if its value is n, it means there is a reset (RST) mark every n MCU blocks. The first mark is RST0, the second is RST1, and so on, repeating from RST0 after RST7. If there is no such mark segment, or the interval value is 0, it means there is no restart interval or RST mark. In JPEG, the smallest group of interval-coded data units is called the Minimum Coded Unit (MCU).

[0493] (8) SOS, the marker code occupies 2 bytes and is usually a fixed value of 0xFFDA.

[0494] SOS can contain two specific fields:

[0495] ① Data length, occupying 2 bytes, represents the total length of fields ① to ④, excluding the marker code but including this field.

[0496] ② Number of color components, occupying 1 byte. A value of 1 indicates a grayscale image. A value of 3 indicates YCrCb or YIQ; a value of 4 indicates CMYK.

[0497] ③ Color component information, including:

[0498] a) Color component ID, occupying 1 byte.

[0499] b) DC / AC coefficient table number, occupying 1 byte. The high 4 bits represent the Huffman tree number used for the DC component, and the low 4 bits represent the Huffman tree number used for the AC component.

[0500] ④ Compress image data

[0501] a) Select the first byte of spectrum with a fixed value of 0x00.

[0502] b) The spectrum selection ends with a fixed value of 0x3F (1 byte).

[0503] c) The spectrum selection 1 byte is always 00 in basic JPEG.

[0504] In this marked section, field ③ should appear repeatedly, the number of times corresponding to the number of color components (field ②) (usually 3 times). Following this section is the actual image information. The image information automatically ends upon encountering a marker code, typically indicated by an EOI marker.

[0505] The following is an example description of the encoding and transmission method of the transmission data of the second image.

[0506] Example 1:

[0507] Taking the encoding of the first brightness-related data and the image data of the second image into a single JPEG image frame as an example.

[0508] For example, the first luminance-related data is carried in the application identifier (APP) field of the JPEG image frame. This can be understood as the transmission data of the second image including the first luminance-related data and the image data of the second image. The image data of the second image is contained in a JPEG image frame encoded using the Joint Picture Experts Group (JPI) JPEG format, and the first luminance-related data is carried in the application identifier (APP) field of that JPEG image frame, such as the APPn field.

[0509] In some embodiments, the first brightness-related data may be carried in the application identifier (APP) field of the JPEG image frame as metadata.

[0510] As an example, the metadata used to carry the first brightness-related data is called highlight enhancement metadata. For example, highlight enhancement metadata can be represented by the syntax element `enhance_metadata()`. Of course, it can also be represented by other syntax elements, and this application embodiment does not specifically limit this.

[0511] For example, the HDR image format corresponding to the JPEG image frame can be seen in Table 1-1. Table 1-1 uses an 8-bit SDR image as an example for the second image. The first image is a 12-bit image. The transmission data of the second image is encoded in the JPEG image frame shown in Table 1-1. The first luminance-related data is carried in the APPn field in the form of metadata. The tag code of the APPn field can represent the HDR basic SDR image tag. It can be understood that the information carried by the fields other than the APPn field is used to decode the second image.

[0512] Table 1-1

[0513]

[0514] For example, the bitstream syntax of the APPn field can be described in Table 1-2. Of course, the bitstream syntax of the APPn field may include more or fewer bitstream syntaxes than those in Table 1-2, and this embodiment of the application does not limit this.

[0515] Table 1-2

[0516]

[0517] Here, `start_code` is a 16-bit unsigned integer used to represent the JPEG APP segment identifier. The descriptor for the `start_code` of APP0 is 0xFFE0, and the descriptor for the `start_code` of APPn is 0xFFEn.

[0518] Length is a 16-bit unsigned integer. It is used to represent the length of the entire APP segment.

[0519] The ITUT35_tag is a 6-character ASCII character representing a segment type tag. Its typical ASCII value is 0x49, 0x54, 0x55, 0x54, 0x33, or 0x35, identifying "ITUT35".

[0520] `itu_t_t35_country_code` is an 8-bit unsigned integer. It is the code in ITU T35 that identifies the country of the organization to which the custom content block belongs. In this application's embodiment, the HDR image format uses 0x26.

[0521] `itu_t_t35_country_code_extension_byte` is an 8-bit unsigned integer. It is the extension code in ITU T35 that identifies the country of the organization to which the custom content block belongs. It generally does not appear in HDR image formats.

[0522] The terminal_provide_code is a 16-bit unsigned integer. It is the code in ITU T35 that identifies the organization to which the custom content block belongs. In the HDR image format of this application embodiment, 0x04 can be used.

[0523] The terminal_provide_oriented_code is a 16-bit unsigned integer. It is the encoding within the organization to which the custom content block belongs, as identified in ITU T35. In the HDR image format of this application embodiment, 0x07 can be used.

[0524] `enhence_metadata_size` is a 16-bit unsigned positive number. It represents the total size of the highlight enhancement metadata. The highlight enhancement metadata is used to carry the first brightness-related data.

[0525] enhence_metadata() represents highlight enhancement metadata, used to carry first brightness-related data.

[0526] u(n) is represented using an n-bit unsigned integer, obtained by conversion from n bits.

[0527] Example 2:

[0528] Taking the encoding of the first brightness-related data and the image data of the second image into two JPEG image frames as an example.

[0529] The transmission data of the second image is encoded in a JPEG file. The JPEG file includes a first JPEG image frame and a second JPEG image frame. The first JPEG image frame encodes the image data of the second image. The second JPEG image frame encodes first luminance-related data. It can be understood that the transmission data of the second image includes both the first luminance-related data and the image data of the second image. The image data of the second image is contained within the first JPEG image frame encoded using the Joint Panel of Photographic Experts (JPP) JPEG format, and the first luminance-related data is carried within the first JPEG image frame encoded using the JPP JPEG format.

[0530] In some embodiments, the APP field of the first JPEG image frame may further include a position identifier indicating the starting position of the second JPEG image frame in the JPEG file. This embodiment refers to this identifier as a first position identifier. Exemplarily, the position identifier can be represented by the syntax element frame_offset[i]. frame_offset[i] indicates the starting position of the current JPEG image frame in the JPEG file. For example, frame_offset[i] can be represented as the offset of the current JPEG image frame (i.e., the i-th image frame in the JPEG file) from the starting position in the JPEG file.

[0531] Furthermore, when decoding, the display device (or the decoding device in the display device) can decode the image data of the second image from the first JPEG image frame and obtain the first position identifier; then, it can determine the starting position of the second JPEG image frame from the JPEG file according to the first position identifier, and decode the first brightness-related data from the second JPEG image frame according to the starting position.

[0532] In some embodiments, the APP field of a JPEG image frame may also include a file type identifier to indicate the file type of the current JPEG image frame. For example, the file type may include SDR raw image data, first brightness-related data (also known as highlight enhancement data), etc. In some scenarios, where the transmitted image data also includes other types of data, and these other types of data are carried in independent image frames, different values ​​of the file type identifier can also be used to indicate this. For example, the file type identifier can be represented by the syntax element `file_type`, or other syntax elements can be used; this application does not specifically limit this.

[0533] Specifically, the APP field of the first JPEG image frame and the second JPEG image frame each includes a file type identifier. The file type identifier of the first JPEG image frame takes a third value, which indicates that the APP field of the first JPEG image frame carries the image data of the second image (which can be understood as the original SDR image data). The file type identifier of the second JPEG image frame takes a fourth value, which indicates that the APP field of the second JPEG image frame carries the first brightness-related data (i.e., highlight enhancement data). The values ​​of the third and fourth values ​​are different.

[0534] For example, a JPEG file includes a first JPEG image frame and a second JPEG image frame. The format of the JPEG file can be seen in Table 2-1. Table 2-1 uses an 8-bit SDR image as an example for the second image. The first image is a 12-bit image. The data of the second image is encoded in the first JPEG image frame shown in Table 2-1. The first luminance-related data is encoded in the second JPEG image frame shown in Table 2-1.

[0535] Table 2-1

[0536]

[0537]

[0538] For example, the bitstream syntax of the APPn field of the first JPEG image frame can be described in Table 2-2. Of course, the bitstream syntax of the APPn field may include more or fewer bitstream syntaxes than those in Table 2-2, and this embodiment of the application does not limit this.

[0539] Table 2-2

[0540]

[0541] The explanations of the grammatical elements in Table 2-2 that are the same as those in Table 1-2 can be found in the relevant descriptions in Table 1-2, and will not be repeated here.

[0542] `frame_number` is an 8-bit unsigned integer. It represents the number of JPEG image frames contained in a JPEG file.

[0543] `File_type` occupies a 16-bit unsigned integer. It represents the type of the current JPEG image frame (also known as the current JPEG independent frame), and can be understood as the data type of the data included in the APPn field of the current JPEG image frame; it can be called a file type identifier. For example, different values ​​represent different data types. For instance, a third value for `File_type` indicates that the current JPEG image frame carries the image data of the second image, i.e., the SDR raw image data. A fourth value for `File_type` indicates that the current JPEG image frame encodes the first luminance-related data. For example, the third value is 1, the fourth value is 2, and vice versa. Of course, other values ​​can also be used, and this application embodiment does not specifically limit them. It should be noted that if the transmitted data of the second image includes other data besides image data and the first luminance-related data, other values ​​can be defined to express the data type of other data.

[0544] `frame_offset[]` is a 32-bit unsigned integer. It indicates the starting position of the current JPEG image frame in the JPEG file. For example, it can be the offset of the current JPEG image frame from the beginning position in the JPEG file.

[0545] For example, the bitstream syntax of the APPn field of the second JPEG image frame can be described in Table 2-3. Of course, the bitstream syntax of the APPn field may include more or fewer bitstream syntaxes than those in Table 2-3, and this embodiment of the application does not limit this.

[0546] Table 2-3

[0547]

[0548] The explanations of the grammatical elements in Table 2-3 that are the same as those in Tables 1-2 and 2-2 can be found in the relevant descriptions in Tables 1-2 and 2-2, and will not be repeated here.

[0549] The enhancement info size represents the total size of the enhancement information, which is a 16-bit unsigned integer.

[0550] `enhence_Info` represents highlight enhancement information. When `enhence_type` in an image frame indicates that the data is in image frame format, the `APP` field of that image frame includes this highlight enhancement information. It should be noted that the data encoded in fields other than the `APP` field in the image frame, along with the highlight enhancement information included in the `APP` field, together constitute the first brightness-related data. For example, the highlight enhancement information can be used to indicate key information or rules for processing the second image using the first brightness-related data.

[0551] In some possible scenarios, the option to use either Example 1 or Example 2 can be supported.

[0552] The following example 3 illustrates the choice between using the methods in Example 1 or Example 2.

[0553] Example 3:

[0554] For example, an identifier indicating the data format of the first luminance-related data can be added to the bitstream syntax. This identifier, called the data type identifier, can be represented by the syntax element `enhance_type`. `enhance_type` is a 16-bit unsigned integer representing the data format of the first luminance-related data carried in the JPEG image frame. Different values ​​can represent different data formats. When the data type identifier is set to the first value, it indicates that the data type of the first luminance-related data is metadata; when it is set to the second value, it indicates that the data type of the first luminance-related data is image frame type. For example, a value of 1 indicates image frame format, and a value of 2 indicates metadata format. The first and second values ​​are different.

[0555] In some scenarios, the first value can be the same as the third value, the second value can be the same as the fourth value, or the second value can be the same as the third value, and the first value can be the same as the fourth value. This application does not impose any limitations on this.

[0556] For example, a description of the bitstream syntax of the APPn field of the JPEG image frame encoding the data of the second image can be found in Table 3-1.

[0557] Table 3-1

[0558]

[0559] Taking image data, which includes first brightness-related data and second image data, as an example: When the first brightness-related data is in metadata format, frame_number == 1, enhancement_type == 2. When the first brightness-related data is in image frame format, frame_number == 2, enhancement_type == 1.

[0560] Example 4:

[0561] In some scenarios, the transmitted data includes not only first luminance-related data and second image data, but also at least one tone mapping curve. The tone mapping curve can include a global tone mapping (GTM) curve, or a combination of a global tone mapping curve and a local tone mapping (LTM) curve. When using local tone mapping, the image can be divided into M*N regions, each corresponding to a tone mapping curve. The acquisition end can transmit the parameters of at least one tone mapping curve to the display end in the form of metadata.

[0562] Thus, the display can decode SDR image data from the image data of the second image. When generating an HDR image based on the parameters of the SDR image data and the local tone mapping curve, LTM technology can be used in reverse to recover hue / brightness. In various embodiments, LTM technology can be performed in any tone space, such as RGB or YUV color space.

[0563] The tone mapping curve can be carried in the JPEG image frame where the image data of the second image is located, or in the JPEG image frame where the first brightness-related data is located, or it can be carried in a separate JPEG image frame.

[0564] The following example uses a JPEG image frame containing the tone mapping curve of the second image's image data as an example. The second image's image data and the first luminance-related data are jointly encoded in a single JPEG image frame. The format of a JPEG image frame is shown in Table 1-1.

[0565] In some embodiments, the tone mapping curve is also carried in the APPn field of the JPEG image frame in the form of metadata. The APPn field may include an identifier indicating GTM or GTM+LTM. For example, this identifier can be called a tone indicator identifier, or it can be named something else; this application embodiment does not specifically limit this. For example, the tone indicator identifier can be represented by the syntax element meta_type, or other syntax elements can be used to represent the tone indicator identifier; this application embodiment does not specifically limit this. Different values ​​of meta_type are used to indicate the type of different tone mapping curve parameters. For example, a meta_type value of 1 indicates GTM, a meta_type value of 2 indicates GTM+LTM, and vice versa. As another example, a meta_type value of 0 indicates GTM, a meta_type value of 1 indicates GTM+LTM, and vice versa.

[0566] Metadata for tone mapping curves can include GTM metadata, or both GTM and LTM metadata.

[0567] As an example, the bitstream syntax of the APPn field in a JPEG image frame can be described in Table 4-1. Of course, the bitstream syntax of the APPn field may include more or fewer bitstream syntaxes than those in Table 4-1, and this embodiment of the application does not limit this.

[0568] Table 4-1

[0569]

[0570]

[0571] The explanations of the grammatical elements in Table 4-1 that are the same as those in Tables 1-2, 2-2 and 2-3 can be found in the relevant descriptions in Tables 1-2, 2-2 and 2-3, and will not be repeated here.

[0572] Wherein, `HDR_metedata size` represents the data length of GTM metadata, expressed as a 16-bit unsigned integer. `HDR metadata()` represents GTM metadata. `HDR metadata both size` represents the data length of GTM metadata + LTM metadata, expressed as a 16-bit unsigned integer. `HDR_metedata_enhence` represents LTM metadata.

[0573] `meta_type` is a 16-bit unsigned integer. It represents the data type of the tone mapping curve parameters carried by the APP field in the current JPEG image frame. Different values ​​of `meta_type` are used to represent different data types of tone mapping curve parameters. For example, a value of 1 for `meta_type` represents global tone mapping (GTM) curve data, and a value of 2 represents GTM + local tone mapping (LTM) curve data.

[0574] The following example uses a JPEG image frame containing the tone mapping curve in the image data of the second image. The image data of the second image and the first luminance-related data are independently encoded in two JPEG image frames. The format of the JPEG image frames is shown in Table 2-1. The parameters of the tone mapping curve are carried in the APPn field of the first JPEG image frame.

[0575] As an example, the bitstream syntax of the APPn field of the first JPEG image frame can be described in Table 4-2. The bitstream syntax of the APPn field of the second JPEG image frame can be described in Table 2-3. Of course, the bitstream syntax of the APPn field of the first JPEG image frame may include more or fewer bitstream syntaxes than those in Table 4-2, and this embodiment of the application does not limit this.

[0576] Table 4-2

[0577]

[0578]

[0579] Example 5:

[0580] In some scenarios, the transmitted data may include augmented data for scaling the display. For example, augmented data for scaling the display may be carried in the same JPEG image frame as the image data of the second image. Alternatively, augmented data for scaling the display may be encoded in one or more separate JPEG image frames. In the following description, augmented data for scaling the display will be referred to as scaling display data.

[0581] The scaling display data may include scaling enhancement data and high-frequency texture enhancement data, and may also include tone mapping curves for scaling the display.

[0582] The scaling enhancement data describes the differences in pixels between the first image and the first image after scaling (enlarging and / or shrinking) the first image. The scaling enhancement data includes information describing the increase in pixel values ​​in the first image after scaling (enlarging) compared to the first image, and / or information describing the decrease in pixel values ​​in the first image after shrinking compared to the first image.

[0583] For example, detailed information could include using a preset algorithm (bilinear interpolation, bicubic spline interpolation, etc.) to upscale a low-resolution image to the same resolution as a high-resolution image, and then calculating the RGB or YUV pixel difference value for each pixel point by point. Alternatively, detailed information could include using a preset algorithm (bilinear interpolation, bicubic spline interpolation, etc.) to upscale a low-resolution image to the same resolution as a high-resolution image, and then selecting the interpolation algorithm that minimizes the difference from the high-resolution image, including information about that algorithm.

[0584] The high-frequency texture enhancement data may include the high-frequency texture differences between the first image and the image in a scaled (enlarged and / or reduced) version of the first image. The high-frequency texture enhancement data is used to describe the increased high-frequency texture information in the enlarged version of the first image compared to the first image, or to describe the decreased high-frequency texture information in the reduced version of the first image compared to the first image.

[0585] In some embodiments, scaling enhancement data and high-frequency texture enhancement data can be carried independently in different JPEG image frames. Alternatively, they can be carried in the APPn field of the JPEG image frame used to carry image data for the second image. They can also be carried as metadata in the APPn field of the JPEG image frame used to carry image data for the second image. For example, this metadata could be called scaling display enhancement metadata, which is used to carry scaling display data.

[0586] In some embodiments, the tone mapping curve used for scaling the display can be carried in the APPn field of the JPEG image frame used to carry image data of the second image.

[0587] For example, scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling displays can be indicated by defining an identifier that indicates the data type of the scaling display data. For instance, this identifier can be represented by the syntax element HDR_type. Of course, other syntax elements can also be used, and this embodiment of the application does not specifically limit their application.

[0588] HDR_type can be a 16-bit unsigned integer. It represents the data type of the scaling display data contained in the current JPEG image. In some embodiments, the data type of the scaling display data in this application may include scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves used for scaling display.

[0589] As an example, a value of 1 indicates that the JPEG file includes scaling enhancement data, a value of 2 indicates that it includes high-frequency texture enhancement data, and a value of 3 indicates that it includes tone mapping curves for scaling display.

[0590] HDR_type is also used to indicate the scaling enhancement method used. This HDR_type can be called the scaling enhancement method identifier, or other names can be used. This application embodiment does not specifically limit this.

[0591] When the scaling enhancement method is identified as the fifth value, it indicates that the scaling enhancement method is to overlay scaling enhancement data on the image after the second image has been enhanced based on the first brightness-related data; when the scaling enhancement method is identified as the sixth value, it indicates that the scaling enhancement method is to weightedly overlay high-frequency texture enhancement data on the image after the second image has been enhanced based on the first brightness-related data; or, when the scaling enhancement method is identified as the seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image after the second image has been enhanced based on the first brightness-related data based on the tone mapping curve used for scaling display.

[0592] In the embodiments of this application, the first value and the second value are different. The third value and the fourth value are different. The fifth value, the sixth value, and the seventh value are all different. In some scenarios, the first value, the second value, the third value, the fourth value, the fifth value, the sixth value, and the seventh value may have repeated values, for example, the first value and the third value are the same, the fourth value and the fifth value are the same, etc. The embodiments of this application do not specifically limit this.

[0593] In some embodiments, when the transmitted data includes scaling enhancement data and high-frequency texture enhancement data, the scaling enhancement data and high-frequency texture enhancement data can be encoded in the same JPEG image frame or in different JPEG image frames.

[0594] For example, a JPEG file includes a first JPEG image frame and a third JPEG image frame. The first JPEG image frame carries the image data of the second image and the tone mapping curve for scaling display. The third JPEG image frame carries the scaling display data (scaling enhancement data and / or high-frequency texture enhancement data).

[0595] For example, the corresponding HDR image formulas for the JPEG file can be found in Table 5-1. Table 5-1 uses an 8-bit SDR image as an example for the second image. The first image is a 12-bit image. The data for the second image is encoded in the first JPEG image frame shown in Table 5-1. The scaling display data is encoded in the third JPEG image frame shown in Table 5-1.

[0596] Table 5-1

[0597]

[0598] As an example, the bitstream syntax of the APPn field of the first JPEG image frame can be described in Table 5-2. The bitstream syntax of the APPn field of the third JPEG image frame can be described in Table 5-3. Of course, the bitstream syntax of the APPn field of the first JPEG image frame may include more or fewer bitstream syntaxes than those in Table 5-2, and the bitstream syntax of the APPn field of the third JPEG image frame may include more or fewer bitstream syntaxes than those in Table 5-3. This application embodiment does not limit this.

[0599] Table 5-2

[0600]

[0601]

[0602] HDR_scale_metedata_size can be a 16-bit unsigned integer. It represents the total size of the tone map curves used for scaling the display. HDR_scale_metedata() represents the tone map curves used for scaling the display.

[0603] Table 5-3

[0604]

[0605] Example 6:

[0606] In some scenarios, the transmitted data may include image data other than that of the second image; the specific data included can be determined based on the actual situation. Besides the image data of the second image, the transmitted data may include one or more of the following: first luminance-related data, at least one tone mapping curve, and scaling display data. The specific data included can be indicated using different syntax elements. Scaling display data may include scaling enhancement data, high-frequency texture enhancement data, and tone mapping curves used for scaling display.

[0607] As an example, consider a transmission of data including first luminance-related data, scaling enhancement data, and high-frequency texture enhancement data. The first luminance-related data, scaling enhancement data, and high-frequency texture enhancement data are all independently encoded in different JPEG image frames. The JPEG file includes a first JPEG image frame, a second JPEG image frame, a third JPEG image frame, and a fourth image frame. The first JPEG image frame carries the image data of the second image and the tone mapping curve for scaling display; the second JPEG image frame carries the first luminance-related data; the third JPEG image frame carries the scaling enhancement data; and the fourth image frame carries the high-frequency texture enhancement data.

[0608] For example, the corresponding HDR image radicals for JPEG files can be found in Table 6-1.

[0609] Table 6-1

[0610]

[0611] As an example, the bitstream syntax of the APPn field of the first JPEG image frame can be described in Table 6-2. Of course, the bitstream syntax of the APPn field of the first JPEG image frame may include more or fewer bitstream syntaxes than those in Table 6-2, and this application embodiment does not limit this.

[0612] Table 6-2

[0613]

[0614] As an example, the bitstream syntax of the APPn field of the second JPEG image frame can be described in Table 6-3. Of course, the bitstream syntax of the APPn field of the second JPEG image frame may include more or fewer bitstream syntaxes than those in Table 6-3, and this embodiment of the application does not limit this.

[0615] Table 6-3

[0616]

[0617] In this context, `meta_type == 0` indicates the default value, meaning no data is transmitted. Similarly, `HDR_type == 0` also indicates the default value, meaning no data is transmitted.

[0618] As an example, the bitstream syntax of the APPn field of the third JPEG image frame can be described in Table 6-4. Of course, the bitstream syntax of the APPn field of the third JPEG image frame may include more or fewer bitstream syntaxes than those in Table 6-4, and this embodiment of the application does not limit this.

[0619] Table 6-4

[0620]

[0621]

[0622] In this context, `meta_type == 0` indicates the default value, meaning no data is transmitted. `enhance_type == 0` also indicates the default value, meaning no data is transmitted.

[0623] As an example, the bitstream syntax of the APPn field of the fourth image frame can be described in Table 6-5. Of course, the bitstream syntax of the APPn field of the fourth image frame may include more or fewer bitstream syntaxes than those in Table 6-5, and this embodiment of the application does not limit this.

[0624] Table 6-5

[0625]

[0626] In this context, `meta_type == 0` indicates the default value, meaning no data is transmitted. `enhance_type == 0` also indicates the default value, meaning no data is transmitted.

[0627] In the above scheme, during the acquisition and processing phase, the acquisition device generates a TM curve for other possible screen brightness levels based on the screen information used by the acquisition device, in order to maintain the dynamic range observed by the acquisition device. Simultaneously, it records screen-related information from the acquisition device to guide the backlight boost ratio during display, aiming to maximize the consistency between the display effect of the display device and the display effect of the acquisition device.

[0628] The following describes the method and process for a display device to acquire an image to be displayed. (See also...) Figure 5A As shown.

[0629] 501, The display device decodes the image data of the second image and the first brightness-related data from the transmission data of the second image.

[0630] In some embodiments, the transmitted data may also include at least one tone mapping curve and / or screen-related information, in which case the display device may further decode at least one tone mapping curve and / or screen-related information from the transmitted data.

[0631] For example, the display device receives a bitstream and obtains image data of the second image and first brightness-related data from the bitstream.

[0632] In some scenarios, the transmitted data also includes other data, in which case the display device also decodes the other data from the bitstream.

[0633] In one possible implementation, the bitstream includes a JPEG file. The JPEG file includes a first JPEG image frame, which carries encoded data of a second image, and first luminance-related data carried in the application identifier (APP) field of the first JPEG image frame. The display device can then decode the second image from the JPEG image frame and decode the first luminance-related data from the APP field of the JPEG image frame. For example, the bitstream syntax during decoding can be found in Tables 1-2 and 1-3, or in Tables 3-1 and 1-3.

[0634] In another possible implementation, the bitstream includes a JPEG file comprising a first JPEG image frame and a second JPEG image frame. The display device decodes the second image from the first JPEG image frame and decodes the first luminance-related data from the second JPEG image frame. For example, the bitstream syntax during decoding can be found in Tables 3-1 and 2-3.

[0635] In another possible implementation, the transmitted data further includes scaling enhancement data. The scaling display data includes at least one of scaling enhancement data, high-frequency texture enhancement data, or a tone mapping curve for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image, or information describing the decrease in pixel values ​​in the scaled-down image of the first image compared to the first image; the high-frequency texture enhancement data describes the increase in high-frequency texture information in the magnified image of the first image compared to the first image, or describes the decrease in high-frequency texture information in the scaled-down image of the first image compared to the first image.

[0636] In one scenario, the scaling enhancement data is carried in the APP field of the first JPEG image frame. The display device can also decode the scaling enhancement data from the APP field of the first JPEG image frame.

[0637] In another scenario, when the scaling display data includes the tone mapping curve for scaling display, the tone mapping curve for scaling display is carried in the APP field of the first JPEG image frame; or, when the scaling display data includes the scaling enhancement data, the JPEG file further includes a third JPEG image frame, which carries the scaling enhancement data; or, when the scaling display data includes the high-frequency texture enhancement data, the JPEG file further includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data.

[0638] 502, the display device obtains a first tone mapping curve from at least one tone mapping curve in the metadata, and this first tone mapping curve matches the screen display parameters of the display device. For ease of description, the matched tone mapping curve is referred to as the first tone mapping curve in this embodiment. For example, at least one tone mapping curve includes three curves, namely: tone mapping curve 1 corresponding to 1000nit and 500nit screens, tone mapping curve 2 corresponding to 1000nit and 200nit screens, and tone mapping curve 3 corresponding to 1000nit and 100nit screens. For example, if the screen display parameter of the display device is 500nit, then the display device matches tone mapping curve 1 from the three tone mapping curves.

[0639] For example, at least one tone mapping curve is carried in the application identifier (APP) field of the JPEG image frame. The display device can then decode at least one tone mapping curve from the metadata of the application identifier (APP) field of the JPEG image frame.

[0640] For example, if the screen display parameter of the display device is 600 nits, and there is no 600 nits tone mapping curve, the 600 nits tone mapping curve can be obtained by interpolating the 500 nits tone mapping curve.

[0641] In the above scheme, the display end generates a TM curve adapted to the display device based on the TM curve transmitted by the acquisition end, and applies it to the image so that the image effect seen by the user under various brightness screens is as consistent as possible with the effect of the acquisition end on the reference screen.

[0642] 503. Apply the first tone mapping curve to the second image. For example, the image data of the third image can be obtained by applying the following formula (2) to the second image. For example, the image data includes the R component, G component, and B component of the pixel.

[0643]

[0644] outR = inR * gain;

[0645] outG = inG * gain;

[0646] outB = inB * gain; Formula (2)

[0647] Where inY represents the backlight brightness of the display, LUT represents the first tone mapping curve, and LUT[inY] represents the backlight brightness mapped by the backlight brightness of the display in the first tone mapping curve. Gain represents the pixel value adjustment gain. inR represents the R component of the pixel before adjustment, outR represents the R component of the pixel after adjustment, inG represents the G component of the pixel before adjustment, outG represents the G component of the pixel after adjustment, inB represents the B component of the pixel before adjustment, and outB represents the B component of the pixel after adjustment.

[0648] 504, Enhance the screen backlight of the display device. In one possible implementation, the display device can determine the required enhancement of the screen backlight based on decoded screen-related information. For example, the screen-related information may include the screen backlight parameters of the acquisition device, or a mapping relationship between the backlight brightness under the acquisition device's screen backlight parameters and the backlight brightness under at least one set screen backlight parameter, or an adjustment ratio of the backlight brightness under at least one set screen backlight parameter relative to the backlight brightness under the acquisition device's screen backlight parameters. Enhancing the screen backlight of the display device as described here is also for the purpose of increasing image brightness.

[0649] The display device can determine the backlight brightness enhancement ratio based on relevant screen information. Then, based on the current backlight brightness and the enhancement ratio, it determines the enhanced backlight brightness. The current backlight brightness of the display device is represented by BL0, and the enhanced backlight brightness is represented by BL1.

[0650] For example, the improved backlight brightness and the original backlight brightness can satisfy the conditions shown in the following formula (3).

[0651] BL1=U*BL0 formula (3)

[0652] In one example, the display device can determine the boost ratio U based on the screen backlight parameters of both the acquisition device and the display device. For example, U = screen backlight parameters of the acquisition device / screen backlight parameters of the display device.

[0653] For example, U = a * screen backlight parameters of the acquisition device / screen backlight parameters of the display device + q. a and q can be determined empirically. The values ​​of a and q can be configured in the display device.

[0654] In another example, the display device can determine the boost ratio U based on the mapping relationship between the backlight brightness under the screen backlight parameters of the acquisition device and the backlight brightness under at least one set screen backlight parameter. A set screen backlight parameter 1 that matches the screen backlight parameters of the display device is found from the at least one set screen backlight parameter. Then, the ratio of the backlight brightness under the screen backlight parameters of the acquisition device to the backlight brightness of the set screen backlight parameter 1 is obtained as U.

[0655] In another example, the display device can determine U based on the adjustment ratio of the backlight brightness under at least one set screen backlight parameter relative to the backlight brightness under the screen backlight parameter of the acquisition device. For example, a set screen backlight parameter 1 that matches the screen backlight parameter of the display device can be found from at least one set screen backlight parameter, and then the adjustment ratio of the backlight brightness of the set screen backlight parameter 1 relative to the backlight brightness under the screen backlight parameter of the acquisition device can be obtained as U.

[0656] In another possible implementation, the display device can determine the enhanced backlight brightness (also known as screen backlight) based on the first brightness-related data and the current backlight brightness of the display device.

[0657] For example, the improved screen backlight brightness satisfies the conditions shown in any of the following formulas (4) to (7).

[0658] BL1 = BL0 * (k1 * AVG (hdrlayer) i )+b1) Formula (4)

[0659] BL1 = BL0 * (k2 * max (hdrlayer) i Formula (5) + b2)

[0660] BL1 = BL0 * k1 * AVG(lhdrlayer) i )+b1 formula (6)

[0661] BL1 = BL0 * k2 * max(lhdrlayer) i Formula (7) + b2

[0662] Wherein, BL1 represents the adjusted screen backlight of the display device, BL0 represents the original screen backlight (also known as backlight brightness) of the display device, hdrlayer i This represents the brightness enhancement data corresponding to the i-th pixel of the display screen of the display device, AVG(hdrlayer) i ) represents the average brightness enhancement data corresponding to the pixels of the display screen of the display device, max(hdrlayer) i ) represents the maximum value of the brightness enhancement data corresponding to the pixels included in the display screen of the display device, where k1 and k2 are both positive numbers, and b1 and b2 are both positive numbers less than 1.

[0663] In some embodiments, different areas of the screen may use different backlights. The screen backlight of the Lth area can then be represented by any one of the following formulas (8)-(11).

[0664]

[0665]

[0666]

[0667]

[0668] Among them, BL1 L BL0 represents the adjusted screen backlight of the Lth region of the display device. L This refers to the screen backlight (also known as backlight brightness) of the Lth region of the display device before adjustment. This represents the brightness enhancement data corresponding to the i-th pixel in the L-th region of the display screen of the display device, AVG. The maximum value represents the average brightness enhancement data of the pixels in the Lth region of the display screen of the display device. This represents the maximum value of the brightness enhancement data corresponding to the pixels in the Lth region of the display screen of the display device, where k3 and k4 are both positive numbers, and b3 and b4 are both positive numbers less than 1.

[0669] In some embodiments, for a Full Array Local Dimming screen, the backlight brightness can be adjusted for each image block. The screen backlight of the Lth region can then be represented by the following formula (12).

[0670]

[0671] Among them, BL1 L BL0 represents the adjusted screen backlight of the Lth region of the display device. L This refers to the screen backlight (also known as backlight brightness) of the Lth region of the display device before adjustment. This represents the brightness enhancement data corresponding to the i-th pixel in the L-th region of the display screen of the display device, AVG. This represents the average brightness enhancement data corresponding to the pixels in the Lth region of the display screen of the display device, AVG(hdrlayer) i ) represents the average brightness enhancement data corresponding to the pixels of the display screen of the display device, where k5 is a positive number and b5 is a positive number less than 1. In some scenarios, the above AVG can also be replaced by max.

[0672] In some embodiments, for an Edge Local Dimming screen, the backlight brightness can be adjusted for each image block. The screen backlight of the Lth region can then be represented by the following formula (13).

[0673]

[0674] Among them, BL1 L BL0 represents the adjusted screen backlight of the Lth region of the display device. L This refers to the screen backlight (also known as backlight brightness) of the Lth region of the display device before adjustment. This represents the brightness enhancement data corresponding to the i-th pixel in the L-th edge region of the display screen of the display device, AVG. This represents the average brightness enhancement data of the pixels included in the Lth edge region of the display screen of the display device, AVG(hdrlayer) i ) represents the average value of the brightness enhancement data corresponding to the pixels of the display screen of the display device, where k6 is a positive number and b6 are both positive numbers less than 1.

[0675] It should be noted that brightness enhancement data can be at the pixel level or at the image patch level. Therefore, the above... This refers to the brightness enhancement data of the i-th image block within the L-th region. The division of the image block can be the same as or different from the division of the screen region.

[0676] 505, Adjust the pixel values ​​of the third image.

[0677] In one specific embodiment, the pixel values ​​of the pixels of the third image can be adjusted according to the screen backlight of the display device before and after adjustment to obtain the fourth image.

[0678] In some embodiments, the pixel values ​​of each pixel in the fourth image satisfy the condition shown in the following formula (14):

[0679] pixel1 i =pow(BL0 / BL1,1 / gamma)*pixel0 i Formula (14)

[0680] Among them, pixel1 i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i BL0 represents the pixel value of the i-th pixel in the third image, BL1 represents the screen backlight of the display device before adjustment, gamma represents the gamma value of the display device, and pow(xExp, nExp) represents nExp raised to the power of xExp.

[0681] It should be noted that in some scenarios, BL0 and BL1 can be at the region level. In this case, BL0 in formula (14) can be understood as the current backlight brightness (before adjustment) of the region where the i-th pixel is located. BL1 can be understood as the adjusted backlight brightness of the region where the i-th pixel is located. In some scenarios, BL0 and BL1 can be at the screen level, in which case the backlight brightness before adjustment for each pixel is the same, and the backlight brightness after adjustment for each pixel is also the same.

[0682] 506. The third and fourth images are weighted and fused to obtain the image to be displayed.

[0683] For example, the pixel weights of the third image and the fourth image are determined based on the first brightness enhancement data.

[0684] For example, the pixel values ​​of each pixel in the image to be displayed satisfy the condition shown in the following formula (15):

[0685] weight i =hdrlayer i / (2 N -1);

[0686] pixel i =pixel0 i weight i +pixel1 i *(1-weight i ) Formula (15)

[0687] Where N represents the number of bits occupied by the second image, hdrlayer i This represents the first brightness-related data corresponding to the i-th pixel, pixel1. i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i Represents the pixel value of the i-th pixel in the third image, pixel. i This represents the pixel value of each pixel in the image to be displayed.

[0688] In current solutions, the acquisition device only encodes image data during encoding, without recording the screen information used for effect debugging or considering the screen information that the display device might use. If the screen used by the acquisition device and the screen used by the display device are inconsistent, the effect seen on the display device will not reflect the intended message, resulting in a discrepancy. The display device directly sends image data without considering the difference between the current screen brightness and the brightness of the screen used by the acquisition device, failing to perform appropriate adaptation, leading to suboptimal results. In this embodiment, the display device enhances the screen brightness based on brightness-related data and metadata transmitted from the acquisition device, while simultaneously adjusting image pixel values ​​to maintain a constant perceived brightness. Furthermore, weights are determined based on the brightness-related data, and the images before and after pixel value adjustment are weighted to present the relative brightness relationships of the real scene as accurately as possible.

[0689] During the display phase, brightness-related data can be combined with local dimming screens to adjust the screen backlight, increasing the brightness of bright areas and decreasing the brightness of dark areas. This keeps the perceived brightness constant and does not affect details while presenting the relative brightness relationship of the real scene as accurately as possible.

[0690] For example, see Figure 5B The diagram shows the screen backlight adjustment process of a display device. The display device determines the image to be displayed. For details on the determination process, please refer to [link to relevant documentation]. Figure 5A The description of the illustrated embodiment will not be repeated here. The display device controls the screen backlight of the display device's screen based on the first brightness-related data. For example, local backlight adjustment can be performed according to the different types of display screens. The adjusted backlight parameters of the display screen can be determined using any of the possible methods of the preceding formulas (3)-(13).

[0691] In some embodiments, the transmitted data also includes scaling display data. When the user performs a zoom-in or zoom-out operation on the image to be displayed, the display end can perform enhancement processing on the image to be displayed based on the decoded scaling display data.

[0692] As an example, the scaling display data includes scaling enhancement data, which includes information describing the increase in pixel values ​​in the enlarged image compared to the first image. When a user performs a zoom operation on the image to be displayed, the display device enhances the image to be displayed based on this scaling enhancement data. For example, the scaling enhancement data may be overlaid on top of the image to be displayed.

[0693] As an example, the scaling display data includes scaling enhancement data, which includes information describing the reduction in pixel values ​​in the scaled-down image compared to the first image. When a user performs a zoom-in operation on the image to be displayed, the display device adjusts the image to be displayed based on this scaling enhancement data. For example, the scaling enhancement data may be subtracted from the image to be displayed.

[0694] As an example, the scaling display data includes high-frequency texture enhancement data, which includes high-frequency texture information describing the increased texture content of the magnified image compared to the first image. When a user performs a magnification operation on the image to be displayed, the display device enhances the image to be displayed based on the high-frequency texture enhancement data. For example, the high-frequency texture enhancement data is enhanced according to a set enhancement coefficient, and the enhanced high-frequency texture enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed.

[0695] As an example, the scaling display data includes high-frequency texture enhancement data, which includes high-frequency texture information describing the reduction in texture information between the scaled-down image and the first image. When a user performs a scaling-down operation on the image to be displayed, the display device adjusts the image to be displayed based on the high-frequency texture enhancement data. For example, the high-frequency texture enhancement data may be negatively enhanced according to a set enhancement coefficient, and the negatively enhanced high-frequency texture enhancement data may be superimposed on the image to be displayed to obtain the enhanced image to be displayed.

[0696] For example, the scaling display data is carried in a JPEG image frame containing encoded data for carrying a second image. The display device can also decode the scaling display data from the JPEG file. In response to a user's zoom-in or zoom-out operation on the image to be displayed, the display device enhances the image to be displayed based on the scaling display data to obtain an enhanced image to be displayed.

[0697] For example, scaling display data is carried in the third JPEG image frame of the JPEG file. The display device decodes the scaling enhancement data from the third JPEG image frame. In response to a user's zoom-in or zoom-out operation on the image to be displayed, the image to be displayed is enhanced according to the scaling enhancement data to obtain the enhanced image to be displayed.

[0698] In some embodiments, the APP field of the first JPEG image frame (SDR image data) further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0699] The enhanced image to be displayed is obtained by performing enhancement processing on the image to be displayed based on the scaling enhancement data, including:

[0700] When the scaling enhancement method is identified as the fifth value, the scaling enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed; or...

[0701] When the scaling enhancement method is identified as the sixth value, the high-frequency texture enhancement data is enhanced according to the set enhancement coefficient, and the enhanced high-frequency texture enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed; or,

[0702] When the scaling increase method is identified as the seventh value, the brightness value of the image to be displayed is adjusted according to the scaling tone mapping curve to obtain the adjusted image to be displayed.

[0703] For example, the scaling enhancement method identifier is represented by the element HDR_type.

[0704] When HDR_type equals 1, the data obtained from decoding the JPEG frame is JPEG_scale1[] (i.e., scaling enhancement data). The HDR scaling map HDR_scale[] obtained from the path is then the final scaling display data HDR_scale_display[] is:

[0705] HDR_scale_display[]=HDR_scale[]+JPEG_scale1[].

[0706] In this embodiment, the HDR scale image HDR_scale[] obtained from the path can be an HDR image obtained by processing an SDR image. For example, it can be an image obtained by processing a second image based on first brightness enhancement data.

[0707] When HDR_type equals 2, the data obtained from decoding the JPEG frame is JPEG_scale2[] (i.e., high-frequency texture enhancement data). The HDR scaling map HDR_scale[] obtained from the path is then the final scaled display data HDR_scale_display[] is:

[0708] HDR_scale_display[]=HDR_scale[]-JPEG_scale2[]+A*JPEG_scale2[].

[0709] In some embodiments, A can be determined based on a preset value and the backlight of the current display device. A can also be carried in metadata by the sending device and sent to the display device.

[0710] When HDR_type equals 3, the data obtained by decoding the JPEG image is JPEG_scale_lut[], and the HDR scaling image obtained from the path is HDR_scale_display[], so the final scaling display data HDR_scale_display3[] is:

[0711] HDR_scale_display3.R[]=HDR_scale_display.R*JPEG_scale_lut[HDR_scale_display.MAXRGB[]]

[0712] HDR_scale_display3.G[]=HDR_scale_display.G*JPEG_scale_lut[HDR_scale_display.MAXRGB[]]

[0713] HDR_scale_display3.B[]=HDR_scale_display.B*JPEG_scale_lut[HDR_scale_display.MAXRGB[]]

[0714] Wherein, HDR_scale_display represents the image frame data of the image to be displayed. HDR_scale_display.MAXRGB[] represents the image frame data after processing with HDR_scale_display. Each pixel value in the image frame of HDR_scale_display.MAXRGB[] is equal to the maximum component value among the RGB component values ​​of each pixel in HDR_scale_display.

[0715] JPEG_scale_lut[] represents the tone mapping curve corresponding to the magnified version of the first image. The JPEG_scale_lut[] can be used to query the luminance value of each pixel in the processed image frame data after mapping, or to query the factor by which the luminance is magnified or reduced compared to the luminance of each pixel in the processed image frame data.

[0716] The following is a detailed description of the image decoding method provided in the embodiments of this application.

[0717] See Figure 6 The diagram shown is a flowchart illustrating an image decoding method provided in an embodiment of this application.

[0718] 601, Receive bitstream, the bitstream including JPEG files encoded in JPEG format by the Joint Picture Experts Group.

[0719] 602, Decode the image data of the second image and the first brightness-related data from the JPEG file; wherein, the image data of the second image is obtained by compressing the image data of the first image, and the number of bits occupied by the first image is higher than the number of bits occupied by the second image; the first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image, and the first brightness-related data is used to perform mid-enhancement processing on the image data of the second image.

[0720] In some embodiments, the processor in the acquisition device can enhance the image data of the second image based on the first brightness-related data to obtain the image to be displayed.

[0721] In some embodiments, the JPEG file includes a first JPEG image frame, the first JPEG image frame carrying image data of a second image, and first brightness-related data carried in the application identifier (APP) field of the first JPEG image frame. Further, decoding the image data of the second image and the first brightness-related data from the JPEG file can be achieved in the following manner:

[0722] The first JPEG image frame is decoded to obtain the image data of the second image, and the first brightness-related data is obtained from the APP field of the first JPEG image frame.

[0723] In some embodiments, the encoding end also sends at least one tone mapping curve to the decoding end. The at least one tone mapping curve is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device. Therefore, the decoding device also decodes at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame.

[0724] In some embodiments, when the processor in the acquisition device enhances the image data of the second image based on the first brightness-related data, it can determine the first tone mapping curve from at least one tone mapping curve based on the screen display parameters of the display device, and the screen display parameters of the display device match the set screen display parameters corresponding to the first tone mapping curve; perform tone mapping processing on the second image based on the first tone mapping curve to obtain the third image; and adjust the pixel values ​​of the pixels of the third image based on the first brightness-related data to obtain the image to be displayed.

[0725] In some embodiments, the image data of the second image and the first brightness-related data are encoded separately. The JPEG file includes a first JPEG image frame and a second JPEG image frame, wherein the first JPEG image frame carries the image data of the second image, and the second JPEG image frame carries the first brightness-related data.

[0726] Furthermore, the decoding device decodes the image data of the second image from the first JPEG image frame, and decodes the first brightness-related data from the second JPEG image frame.

[0727] In some embodiments, at least one tone mapping curve may be carried in the APP field of the second JPEG image frame or the application identifier APP field of the first JPEG image frame. The decoding device may decode at least one tone mapping curve from the application identifier APP field of the first JPEG image frame or from the APP field of the second JPEG image frame.

[0728] In some embodiments, the JPEG file further encodes scaling display data. The decoding device also decodes the scaling display data from the JPEG file. The scaling display data includes at least one of scaling enhancement data, high-frequency texture enhancement data, or a tone mapping curve for scaling display; the scaling enhancement data describes the increase in pixel values ​​in the enlarged image of the first image compared to the first image, or the decrease in pixel values ​​in the scaled image of the first image compared to the first image; the high-frequency texture enhancement data describes the increase in high-frequency texture information in the enlarged image of the first image compared to the first image, or the decrease in high-frequency texture information in the scaled image of the first image compared to the first image.

[0729] In some embodiments, the scaling display data is carried in the APP field of the first JPEG image frame, and decoding the scaling display data from the JPEG file includes:

[0730] The zoom display data is decoded from the APP field of the first JPEG image frame.

[0731] In some embodiments, when the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of a first JPEG image frame. A decoding device can decode the tone mapping curve for scaling the display from the APP field of the first JPEG image frame.

[0732] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; the decoding device can decode the scaling enhancement data from the third JPEG image frame.

[0733] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data; the decoding device can decode the high-frequency texture enhancement data from the fourth JPEG image frame.

[0734] In some embodiments, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file; the decoding device can determine the start position of the third JPEG image frame from the JPEG file based on the second position identifier, and decode the scaling enhancement data from the third JPEG image frame based on the start position; or, the decoding device can determine the start position of the fourth JPEG image frame from the JPEG file based on the third position identifier, and decode the scaling enhancement data from the fourth JPEG image frame based on the start position.

[0735] In some embodiments, the APP field of the first JPEG image frame further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0736] In some embodiments, when the scaling enhancement mode is identified as a fifth value, scaling enhancement data is decoded from the JPEG file. When the scaling enhancement mode is identified as a sixth value, high-frequency texture enhancement data is decoded from the JPEG file. When the scaling enhancement mode is identified as a seventh value, tone mapping curves for scaling display are decoded from the JPEG file.

[0737] Scaling enhancement data, high-frequency texture enhancement data, and tone mapping curves used for scaling displays are used to enhance the enlarged image to be displayed when performing operations to scale the large image to be displayed.

[0738] Based on the same inventive concept as the method described above, this application provides an image acquisition device applied to an acquisition terminal device, see [link to relevant documentation]. Figure 7 As shown.

[0739] Acquisition module 701 is used to acquire the first image to obtain the image data of the first image;

[0740] The processing module 702 is configured to compress the first image according to the first image data to obtain image data of the second image, wherein the number of bits occupied by the first image is higher than the number of bits occupied by the second image; and to obtain transmission data of the second image according to the image data of the first image and the image data of the second image; wherein the transmission data of the second image includes the encoded data of the second image and the first brightness-related data, wherein the first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image.

[0741] The sending module 703 is used to send the transmitted data.

[0742] In one possible implementation, the transmitted data further includes at least one tone mapping curve, which is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device.

[0743] In one possible implementation, the processing module 702 is specifically configured to normalize the image data of the first image and the image data of the second image according to the bit depth of the first image and the second image, respectively, to obtain normalized data of the first image and normalized data of the second image; and to obtain the first brightness-related data based on the normalized data of the first image and the normalized data of the second image.

[0744] In one possible implementation, the first image has N bits, the second image has M bits, the normalized data of the first image ranges from 0 to X, and the normalized data of the second image ranges from 0 to Y; wherein Y / X = 2. M-N .

[0745] In one possible implementation, the processing module 702 is specifically used to generate the first brightness-related data based on the ratio of the normalized data of the corresponding pixels of the first image and the second image.

[0746] In one possible implementation, the processing module 702 is specifically used to obtain the ratio of normalized data of corresponding pixels in the first image and the second image; when the ratio of normalized data of the first pixel in the first image to normalized data of the first pixel in the second image is less than 1, the ratio of normalized data of the first pixel is corrected to obtain the first brightness-related data; the first pixel is a pixel in the first image or the second image.

[0747] In one possible implementation, the processing module 702 is specifically used to generate second brightness-related data based on the ratio of normalized data of corresponding pixels in the first image and the second image; identify that the first image includes a set bright object, and adjust the brightness mapping relationship of the pixels corresponding to the bright object in the second brightness-related data according to the photoelectric conversion curve to obtain the first brightness-related data.

[0748] In one possible implementation, the processing module 702 is specifically used to obtain the ratio of normalized data of corresponding pixels in the first image and the second image; when the ratio of normalized data of the first pixel in the first image to normalized data of the first pixel in the second image is less than 1, the ratio of normalized data of the first pixel is corrected to obtain the second brightness-related data; the first pixel is a pixel in the first image or the second image.

[0749] In one possible implementation, the photoelectric conversion curve includes at least one of the following: a perceptual quantization (PQ) conversion curve, a hybrid log-gamma (HLG) conversion curve, or a scene brightness fidelity (SLF) conversion curve.

[0750] In one possible implementation, the first brightness-related data is carried in the metadata of the second image.

[0751] In one possible implementation, the first brightness-related data, screen-related data, and at least one tone mapping curve are encoded according to the encoding format of the second image's encoded data.

[0752] This application also provides an image display device, applied to a display terminal device, see [link to relevant documentation]. Figure 8 As shown, the device includes:

[0753] Decoding module 801 is used to receive transmission data of the second image, the transmission data of the second image includes the encoded data of the second image and the first brightness-related data, the first brightness-related data is used to characterize the brightness mapping relationship between the second image and the first image, the first image occupies more bits than the second image; the second image is an image obtained by compression based on the first image; and a first tone mapping curve is determined.

[0754] The processing module 802 is used to perform tone mapping processing on the second image according to the first tone mapping curve to obtain a third image; and to adjust the pixel values ​​of the pixels of the third image according to the brightness-related data to obtain the image to be displayed.

[0755] In one possible design, the transmitted data further includes at least one tone mapping curve, which is determined based on the screen backlight information of the acquisition device, the second image, and at least one set screen display parameter.

[0756] The decoding module 801 is specifically used to determine a first tone mapping curve from the at least one tone mapping curve according to the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve.

[0757] In one possible design, the processing module 802 is specifically used to determine the screen backlight of the display device before adjustment and the screen backlight of the display device after adjustment; adjust the pixel values ​​of the pixels of the third image according to the screen backlight of the display device before adjustment and the screen backlight of the display device after adjustment to obtain a fourth image; perform weighted fusion processing on the fourth image and the third image to obtain the image to be displayed; wherein, the weights of the pixels of the third image and the fourth image are determined according to the first brightness enhancement data.

[0758] In one possible design, the pixel values ​​of each pixel in the fourth image satisfy the following condition:

[0759] pixel1 i =pow(BL0 / BL1,1 / gamma)*pixel0 i ;

[0760] Among them, pixel1 i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i BL0 represents the pixel value of the i-th pixel in the third image, BL1 represents the screen backlight of the display device before adjustment, gamma represents the gamma value of the display device, and pow(xExp, nExp) represents nExp raised to the power of xExp.

[0761] In one possible design, the pixel values ​​of each pixel in the image to be displayed satisfy the following condition:

[0762] weight i =hdrlayer i / 2 N ;

[0763] pixel i =pixel0 i weight i+pixel1 i *(1-weight i );

[0764] Where N represents the number of bits occupied by the second image, hdrlayer i This represents the first brightness-related data corresponding to the i-th pixel, pixel1. i Pixel0 represents the pixel value of the i-th pixel in the fourth image. i Represents the pixel value of the i-th pixel in the third image, pixel. i This represents the pixel value of each pixel in the image to be displayed.

[0765] In one possible design, the processing unit is also used to adjust the screen backlight of the display device.

[0766] In one possible design, the adjusted screen backlight of the display device satisfies the following conditions:

[0767] BL1 = BL0 * (k1 * AVG (hdrlayer) i )+b1); or,

[0768] BL1 = BL0 * (k2 * max (hdrlayer) i )+b2);

[0769] Wherein, BL1 represents the adjusted screen backlight of the display device, BL0 represents the original screen backlight of the display device, and hdrlayer i This represents the brightness enhancement data corresponding to the i-th pixel of the display screen of the display device, AVG(hdrlayer) i ) represents the average brightness enhancement data corresponding to the pixels of the display screen of the display device, max(hdrlayer) i ) represents the maximum value of the brightness enhancement data corresponding to the pixels included in the display screen of the display device, where k1 and k2 are both positive numbers, and b1 and b2 are both positive numbers less than 1.

[0770] See Figure 9 The diagram shown is a schematic representation of an encoding device according to an embodiment of this application. The encoding device includes:

[0771] Encoding module 901 is used to encode the image data of the second image and the first brightness-related data into a bitstream. The bitstream includes a Joint Image Experts Group (JEEG) JPEG file, which carries the image data of the second image and the first brightness-related data. The image data of the second image is obtained by compressing the image data of the first image, and the first image occupies more bits than the second image. The first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image.

[0772] The transmitting module 902 is used to transmit the bitstream.

[0773] In one possible implementation, the JPEG file includes a first JPEG image frame, which carries image data of a second image, and first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

[0774] In one possible implementation, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0775] In one possible implementation, the APP field of the first JPEG image frame also includes a data type identifier. When the data type identifier is set to a first value, it indicates that the data type of the first brightness-related data is metadata.

[0776] In one possible implementation, the encoding module 901 is further used for:

[0777] At least one tone mapping curve is encoded into the APP field of the first JPEG image frame.

[0778] In one possible implementation, the JPEG file includes a first JPEG image frame and a second JPEG image frame, the first JPEG image frame carrying encoded data of the second image, and the second JPEG image frame carrying first luminance-related data.

[0779] In one possible implementation, the APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file.

[0780] In one possible implementation, the APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. The file type identifier of the first JPEG image frame takes a third value to indicate that the APP field of the first JPEG image frame carries the encoded data of the second image. The file type identifier of the second JPEG image frame takes a fourth value to indicate that the APP field of the second JPEG image frame carries the first brightness-related data.

[0781] In one possible implementation, the APP field of the second JPEG image frame includes a data type identifier. When the data type identifier is a second value, it indicates that the data type of the first brightness-related data is an image frame type.

[0782] In one possible implementation, the encoding module 901 is further used for:

[0783] At least one tone mapping curve is incorporated into the APP field of the second JPEG image frame.

[0784] In one possible implementation, at least one tone mapping curve is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter, which is different from the screen backlight parameters of the acquisition device.

[0785] In one possible implementation, the encoding module 901 is further used for:

[0786] Encode the scaling display data into the APP field of the first JPEG image frame;

[0787] The scaling display data includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data is used to describe the increase in pixel values ​​in the magnified image of the first image compared to the first image, or to describe the decrease in pixel values ​​in the magnified image of the first image compared to the first image; the high-frequency texture enhancement data is used to describe the increase in high-frequency texture information in the magnified image of the first image compared to the first image, or to describe the decrease in high-frequency texture information in the magnified image of the first image compared to the first image.

[0788] In one possible implementation, the JPEG file also includes a third JPEG image frame, and the encoding module is further used for:

[0789] The scaling display data is compiled into the JPEG file. The scaling display data includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display. The scaling enhancement data is used to describe the increase in pixel values ​​in the magnified image of the first image compared to the first image, or to describe the decrease in pixel values ​​in the magnified image of the first image compared to the first image. The high-frequency texture enhancement data is used to describe the increase in high-frequency texture information in the magnified image of the first image compared to the first image, or to describe the decrease in high-frequency texture information in the magnified image of the first image compared to the first image.

[0790] When the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; or,

[0791] When the scaling display data includes scaling enhancement data, the JPEG file also includes a third JPEG image frame that carries the scaling enhancement data; or...

[0792] When the scaling display data includes high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame that carries the scaling enhancement data.

[0793] In one possible implementation, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file.

[0794] In one possible implementation, the APP field of the first JPEG image frame also includes a scaling enhancement method identifier;

[0795] When the scaling enhancement method is identified as the fifth value, it indicates that the scaling enhancement method is to overlay scaling enhancement data on the image after the second image has been enhanced based on the first brightness-related data; when the scaling enhancement method is identified as the sixth value, it indicates that the scaling enhancement method is to weightedly overlay high-frequency texture enhancement data on the image after the second image has been enhanced based on the first brightness-related data; or, when the scaling enhancement method is identified as the seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image after the second image has been enhanced based on the first brightness-related data based on the tone mapping curve used for scaling display.

[0796] See Figure 10 The diagram shown is a schematic diagram of a decoding device provided in an embodiment of this application.

[0797] Receiver module 1001 is used to receive a bitstream, the bitstream including JPEG files encoded in JPEG format by the Joint Picture Experts Group;

[0798] Decoding module 1002 is used to decode the image data of the second image and the first brightness-related data from the JPEG file;

[0799] The image data of the second image is obtained by compressing the image data of the first image, and the first image occupies more bits than the second image; the first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image.

[0800] In one possible implementation, the decoding device further includes:

[0801] Processing module ( Figure 10 (As shown in the image), it is used to enhance the image data of the second image based on the first brightness-related data to obtain the image to be displayed.

[0802] In one possible design, the JPEG file includes a first JPEG image frame, the first JPEG image frame carrying image data of the second image, and the first brightness-related data carried in the application identifier APP field of the first JPEG image frame;

[0803] The decoding module 1002 is specifically used to decode the first JPEG image frame to obtain the image data of the second image, and to obtain the first brightness-related data from the APP field of the first JPEG image frame.

[0804] In one possible design, the first brightness-related data is carried as metadata in the APP field of the first JPEG image frame.

[0805] In one possible design, the decoding module 1002 is further configured to decode at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame.

[0806] In one possible design, the processing module is specifically configured to: determine a first tone mapping curve from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device match the set screen display parameters corresponding to the first tone mapping curve; perform tone mapping processing on the second image based on the first tone mapping curve to obtain a third image; and adjust the pixel values ​​of the pixels of the third image based on the first brightness-related data to obtain the image to be displayed.

[0807] In one possible design, the JPEG file includes a first JPEG image frame and a second JPEG image frame, wherein the first JPEG image frame carries image data of the second image and the second JPEG image frame carries the first brightness-related data;

[0808] The decoding module 1002 is specifically used to decode the image data of the second image from the first JPEG image frame, and to decode the first brightness-related data from the second JPEG image frame.

[0809] In one possible design, the decoding module 1002 is further configured to decode at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame or from the APP field of the second JPEG image frame.

[0810] In one possible design, the processing module is specifically configured to: determine a first tone mapping curve from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device match the set screen display parameters corresponding to the first tone mapping curve; perform tone mapping processing on the second image based on the first tone mapping curve to obtain a third image; and adjust the pixel values ​​of the pixels of the third image based on the first brightness-related data to obtain the image to be displayed.

[0811] In one possible design, the JPEG file also encodes scaling display data; the decoding module 1002 is further configured to decode the scaling display data from the JPEG file.

[0812] The scaling display data includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data is used to describe the increase in pixel values ​​in the magnified image of the first image compared to the first image, or to describe the decrease in pixel values ​​in the magnified image of the first image compared to the first image; the high-frequency texture enhancement data is used to describe the increase in high-frequency texture information in the magnified image of the first image compared to the first image, or to describe the decrease in high-frequency texture information in the magnified image of the first image compared to the first image.

[0813] The processing module, in response to the user's zoom-in or zoom-out operation on the image to be displayed, performs enhancement processing on the image to be displayed according to the zoom display data to obtain an enhanced image to be displayed.

[0814] In one possible design, the scaling display data is carried in the APP field of the first JPEG image frame, and the decoding module is further configured to decode the scaling display data from the APP field of the first JPEG image frame.

[0815] In one possible design, when the scaling display data includes a tone mapping curve for scaling display, the tone mapping curve for scaling display is encoded in the APP field of the first JPEG image frame; the decoding module 1002 is specifically used to decode the tone mapping curve for scaling display from the APP field of the first JPEG image frame.

[0816] or,

[0817] When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; the decoding module 1002 is specifically used to decode the scaling enhancement data from the third JPEG image frame;

[0818] or,

[0819] When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data; the decoding module 1002 is specifically used to decode the high-frequency texture enhancement data from the fourth JPEG image frame.

[0820] In one possible design, the APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file;

[0821] The decoding module 1002 is specifically used for:

[0822] The start position of the third JPEG image frame is determined from the JPEG file based on the second position identifier, and the scaling enhancement data is decoded from the third JPEG image frame based on the start position; or,

[0823] The decoding module 1002 is specifically configured to: determine the starting position of the fourth JPEG image frame from the JPEG file based on the third position identifier, and decode the scaling enhancement data from the fourth JPEG image frame based on the starting position.

[0824] In one possible design, the APP field of the first JPEG image frame further includes a scaling enhancement method identifier; when the scaling enhancement method identifier is a fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; when the scaling enhancement method identifier is a sixth value, it indicates that the scaling enhancement method is to weightedly overlay the high-frequency texture enhancement data; when the scaling enhancement method identifier is a seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

[0825] This application also provides a data acquisition device, such as... Figure 11As shown, the data acquisition device 1100 may include a communication interface 1110 and a processor 1120. Optionally, the data acquisition device 1100 may also include a memory 1130. The memory 1130 may be located inside the data acquisition device or externally. Figure 7 The acquisition module 701 and processing module 702 shown can both be implemented by the processor 1120. In some embodiments, the above... Figure 9 The function of the encoding module 901 in the encoding device shown can be implemented by the processor 1120, and the function of the transmitting module 902 can be implemented by the communication interface 1110.

[0826] In some embodiments, the acquisition device 1100 further includes an image sensor 1140 for acquiring images. In some embodiments, the acquisition device 1100 also includes a display screen 1150. Of course, the display screen 1150 may also be disposed outside the acquisition device 1100.

[0827] In one possible implementation, processor 1120 is used to implement Figures 2-6 The acquisition terminal described herein executes any method and outputs transmitted data through the communication interface 1110.

[0828] In the implementation process, each step of the processing flow can be completed through the integrated logic circuits in the hardware of the processor 1120 or through software instructions. Figures 2-6 The method executed by the acquisition end (or encoding device) is described below for brevity. The program code executed by the processor 1120 to implement the above method can be stored in the memory 1130. The memory 1130 and the processor 1120 are coupled.

[0829] Any communication interface involved in the embodiments of this application can be a circuit, a bus, a transceiver, or any other device that can be used for information exchange. For example, the communication interface 1110 in the acquisition terminal device 1100. Exemplarily, the other device can be a device connected to the acquisition terminal device 1100, such as a display terminal device.

[0830] Processor 1120 may operate in conjunction with memory 1130. Memory 1130 may be non-volatile memory, such as hard disk drive (HDD) or solid-state drive (SSD), or it may be volatile memory, such as random-access memory (RAM). Memory 1130 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited to this.

[0831] This application embodiment does not limit the specific connection medium between the communication interface 1110, processor 1120, and memory 1130. This application embodiment... Figure 11 The memory 1130, processor 1120, and communication interface 1110 are connected via a bus, and the bus is in... Figure 11 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 11 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0832] This application also provides a display device, such as... Figure 12 As shown, the display device 1200 may include a communication interface 1210 and a processor 1220. Optionally, the display device 1200 may also include a memory 1230. The memory 1230 may be located inside the display device or externally. Figure 8 The decoding module 801 and processing module 802 shown can both be implemented by the processor 1220. In some embodiments, the display device may further include a display screen 1240 for displaying the image to be displayed. In some embodiments, the above... Figure 10 The function of the receiving module 1001 in the decoding device shown can be implemented by the communication interface 1210, and the function of the decoding module 1002 can be implemented by the processor 1220.

[0833] In one possible implementation, the processor 1220 is used to acquire transmission data through the communication interface 1210 and implement... Figures 2-6 Any method performed by the display device (or decoding device) described herein.

[0834] In the implementation process, each step of the processing flow can be completed through the integrated logic circuits in the processor 1220 or through software instructions. Figures 2-6 The method executed on the display terminal. For simplicity, it will not be described in detail here. The program code executed by the processor 1220 to implement the above method can be stored in the memory 1230. The memory 1230 and the processor 1220 are coupled.

[0835] Any communication interface involved in the embodiments of this application can be a circuit, a bus, a transceiver, or any other device that can be used for information exchange. For example, the communication interface 1210 in the display device 1200. Exemplarily, the other device can be a device connected to the display device 1200, such as a data acquisition device.

[0836] The processor 1220 may operate in conjunction with the memory 1230. The memory 1230 may be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or it may be volatile memory, such as random-access memory (RAM). The memory 1230 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto.

[0837] This application embodiment does not limit the specific connection medium between the communication interface 1210, processor 1220, and memory 1230. This application embodiment... Figure 12 The memory 1230, processor 1220, and communication interface 1210 are connected via a bus, and the bus is in... Figure 12 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0838] The processors involved in the embodiments of this application can be general-purpose processors, digital signal processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0839] The coupling in the embodiments of this application is an indirect coupling or communication connection between devices, modules, or modules, which can be electrical, mechanical, or other forms, and is used for information interaction between devices, modules, or modules.

[0840] Based on the above embodiments, this application also provides a computer storage medium storing software programs. When these software programs are read and executed by one or more processors, they can implement the methods provided in any one or more of the above embodiments. The computer storage medium may include various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory, random access memory, magnetic disk, or optical disk.

[0841] Based on the above embodiments, this application also provides a chip, which includes a processor for implementing the functions involved in any one or more of the above embodiments, such as for implementing... Figures 2-6 The method executed by the acquisition terminal, or used to implement Figures 2-6 The method executed by the display device. Optionally, the chip further includes a memory for storing program instructions and data necessary for the processor to execute. The chip may be composed of individual chips or may include chips and other discrete components.

[0842] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0843] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0844] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0845] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0846] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. An image acquisition method, characterized in that, Applied to data acquisition devices, including: Acquire the first image to obtain the image data of the first image; The image data of the second image is obtained by compressing the first image based on the image data of the first image, wherein the number of bits occupied by the first image is higher than the number of bits occupied by the second image; The transmission data of the second image is obtained based on the image data of the first image and the image data of the second image; The transmission data of the second image includes the encoded data of the second image, at least one tone mapping curve, and first brightness-related data. The first brightness-related data is used to characterize the brightness mapping relationship between the first image and the second image. The at least one tone mapping curve is determined based on the screen backlight parameters of the acquisition device, the image data of the second image, and at least one set screen backlight parameter. The at least one set screen backlight parameter is different from the screen backlight parameters of the acquisition device. Send the transmission data of the second image.

2. The method as described in claim 1, characterized in that, The first brightness-related data is obtained based on the first image and the second image, including: Based on the number of bits in the first image and the second image, the image data of the first image and the image data of the second image are normalized respectively to obtain the normalized data of the first image and the normalized data of the second image. The first brightness-related data is obtained based on the normalized data of the first image and the normalized data of the second image.

3. The method as described in claim 2, characterized in that, The first image has N bits, the second image has M bits, the normalized data of the first image ranges from 0 to X, and the normalized data of the second image ranges from 0 to Y; where Y / X = 2. M-N .

4. The method as described in claim 2, characterized in that, The step of obtaining the brightness-related data based on the normalized data of the first image and the normalized data of the second image includes: The first brightness-related data is generated based on the ratio of the normalized data of the corresponding pixels in the first image to that in the second image.

5. The method as described in claim 4, characterized in that, The first brightness-related data is generated based on the ratio of the normalized data of corresponding pixels in the first image and the second image, including: Obtain the ratio of the normalized data of corresponding pixels in the first image to that in the second image; When the ratio of the normalized data of the first pixel of the first image to the normalized data of the first pixel of the second image is less than 1, the ratio of the normalized data of the first pixel is corrected to obtain the first brightness-related data. The first pixel is a pixel in either the first image or the second image.

6. The method as described in claim 2, characterized in that, The step of obtaining the first brightness-related data based on the normalized data of the first image and the normalized data of the second image includes: Second brightness-related data is generated based on the ratio of the normalized data of corresponding pixels in the first image to that in the second image. The identification of the first image includes setting a bright object, and adjusting the brightness mapping relationship of the pixels corresponding to the bright object in the second brightness-related data according to the photoelectric conversion curve to obtain the first brightness-related data.

7. The method as described in claim 6, characterized in that, The second brightness-related data is generated based on the ratio of the normalized data of corresponding pixels in the first image and the second image, including: Obtain the ratio of the normalized data of corresponding pixels in the first image to that in the second image; When the ratio of the normalized data of the first pixel of the first image to the normalized data of the first pixel of the second image is less than 1, the ratio of the normalized data of the first pixel is corrected to obtain the second brightness-related data. The first pixel is a pixel in either the first image or the second image.

8. The method as described in claim 6, characterized in that, The photoelectric conversion curve includes at least one of the following: perceptual quantization (PQ) conversion curve, hybrid log-gamma (HLG) conversion curve, or scene brightness fidelity (SLF) conversion curve.

9. The method according to any one of claims 1-8, characterized in that, The transmission data of the second image is contained in a JPEG file encoded using the Joint Picture Experts Group (JEP) JPEG format. The JPEG file includes a first JPEG image frame, which carries the encoded data of the second image. The first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame.

10. The method as described in claim 9, characterized in that, The first brightness-related data is carried in the APP field of the first JPEG image frame in the form of metadata.

11. The method as described in claim 10, characterized in that, The APP field of the first JPEG image frame also includes a data type identifier. When the data type identifier is set to a first value, it indicates that the data type of the first brightness-related data is metadata.

12. The method as described in claim 9, characterized in that, The transmission data of the second image also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame.

13. The method according to any one of claims 1-8, characterized in that, The transmission data of the second image is contained in a JPEG file encoded in JPEG format. The JPEG file includes a first JPEG image frame and a second JPEG image frame. The first JPEG image frame carries the encoded data of the second image, and the second JPEG image frame carries the first brightness-related data.

14. The method as described in claim 13, characterized in that, The APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file.

15. The method as described in claim 13, characterized in that, The APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data.

16. The method as described in claim 15, characterized in that, The APP field of the second JPEG image frame includes a data type identifier. When the data type identifier is a second value, it indicates that the data type of the first brightness-related data is an image frame type.

17. The method as described in claim 13, characterized in that, The transmission data of the second image also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame or in the APP field of the second JPEG image frame.

18. The method as described in claim 13, characterized in that, The transmitted data also includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display. The scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled-down image of the first image compared to the first image. The high-frequency texture enhancement data includes high-frequency texture information describing the increase in high-frequency texture values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture values ​​in the scaled-down image of the first image compared to the first image.

19. The method as described in claim 18, characterized in that, The scaling display data is carried in the APP field of the first image frame.

20. The method as described in claim 18, characterized in that, When the scaling display data includes the tone mapping curve for scaling display, the tone mapping curve for scaling display is carried in the APP field of the first JPEG image frame; or, When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; or, When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data.

21. The method as described in claim 20, characterized in that, The APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file.

22. The method according to any one of claims 18-21, characterized in that, The APP field of the first JPEG image frame also includes a scaling enhancement method identifier; When the scaling enhancement method is identified as the fifth value, it indicates that the scaling enhancement method is to superimpose the scaling enhancement data on the image after enhancing the second image based on the first brightness-related data. When the scaling enhancement method is identified as the sixth value, it indicates that the scaling enhancement method is to add the high-frequency texture enhancement data to the image after enhancing the second image based on the first brightness-related data; Alternatively, when the scaling enhancement method is identified as the seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data for the second image according to the tone mapping curve used for scaling display.

23. An image display method, characterized in that, Applications in display devices, including: The system receives transmission data of a second image, which includes encoded data of the second image, at least one tone mapping curve, and first luminance-related data. The first luminance-related data characterizes the luminance mapping relationship between the second image and the first image. The first image occupies more bits than the second image. The second image is a compressed image obtained based on the first image. The at least one tone mapping curve is determined based on the screen backlight information of the acquisition device, the second image, and at least one set screen display parameter. A first tone mapping curve is determined from the at least one tone mapping curve based on the screen display parameters of the display device, wherein the screen display parameters of the display device are matched with the set screen display parameters corresponding to the first tone mapping curve; The third image is obtained by performing tone mapping processing on the second image based on the first tone mapping curve; The pixel values ​​of the pixels in the third image are adjusted based on the first brightness-related data to obtain the image to be displayed.

24. The method as described in claim 23, characterized in that, The transmitted data for receiving the second image includes: The received bitstream includes a JPEG file encoded using the Joint Picture Experts Group (JETG) JPEG format. The JPEG file includes a first JPEG image frame, which carries the encoded data of the second image, and the first brightness-related data is carried in the application identifier (APP) field of the first JPEG image frame. Decode the second image from the first JPEG image frame and decode the first brightness-related data from the APP field of the first JPEG image frame.

25. The method as described in claim 24, characterized in that, The APP field of the first JPEG image frame further includes a data type identifier, wherein the data type identifier takes a first value, indicating that the data type of the first brightness-related data is metadata. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes: When the data type identifier decoded from the APP field of the first JPEG image frame is a first value, the first brightness-related data is decoded from the APP field of the first JPEG image frame.

26. The method as described in claim 24, characterized in that, The transmitted data also includes at least one tone mapping curve, which is carried in the application identifier (APP) field of the first JPEG image frame; The method further includes: Decode the at least one tone mapping curve from the application identifier (APP) field of the first JPEG image frame.

27. The method as described in claim 23, characterized in that, The transmitted data for receiving the second image includes: Receive a bitstream, the bitstream including a JPEG file encoded in JPEG format, the JPEG file including a first JPEG image frame and a second JPEG image frame; Decode the second image from the first JPEG image frame and decode the first brightness-related data from the second JPEG image frame.

28. The method as described in claim 27, characterized in that, The APP field of the first JPEG image frame includes a first position identifier, which is used to indicate the starting position of the second JPEG image frame in the JPEG file; Decoding the second image from the first JPEG image frame and decoding the first luminance-related data from the second JPEG image frame includes: Decode the second image from the first JPEG image frame and obtain the first location identifier; The starting position of the second JPEG image frame is determined from the JPEG file based on the first position identifier, and the first brightness-related data is decoded from the second JPEG image frame based on the starting position.

29. The method as described in claim 28, characterized in that, The APP field of the first JPEG image frame further includes a data type identifier, the value of which is a second value indicating that the data type of the first brightness-related data is an image frame type. Decoding the first brightness-related data from the APP field of the first JPEG image frame includes: When the data type identifier decoded from the APP field of the first JPEG image frame is a second value, the first brightness-related data is decoded from the second JPEG image frame.

30. The method as described in claim 28, characterized in that, The APP field of the first JPEG image frame and the second JPEG image frame respectively includes a file type identifier. When the file type identifier of the first JPEG image frame is a third value, it indicates that the APP field of the first JPEG image frame carries the encoded data of the second image. When the file type identifier of the second JPEG image frame is a fourth value, it indicates that the APP field of the second JPEG image frame carries the first brightness-related data. Decoding the image data of the second image from the first JPEG image frame includes: When the file type identifier decoded from the APP field of the first JPEG image frame is a third value, the image data of the second image is decoded from the first JPEG image frame; Decoding the first brightness-related data from the APP field of the first JPEG image frame includes: When the file type identifier decoded from the APP field of the second JPEG image frame is a fourth value, the first brightness-related data is decoded from the second JPEG image frame.

31. The method according to any one of claims 27-30, characterized in that, The transmitted data also includes at least one tone mapping curve, which is carried in the APP field of the first JPEG image frame or the APP field of the second JPEG image frame; The method further includes: Decode the at least one tone mapping curve from the APP field of the first JPEG image frame or the APP field of the second JPEG image frame.

32. The method according to any one of claims 24-30, characterized in that, The transmitted data also includes scaling display data, which includes at least one of scaling enhancement data, high-frequency texture enhancement data, or tone mapping curves for scaling display; the scaling enhancement data includes information describing the increase in pixel values ​​in the magnified image of the first image compared to the first image and / or information describing the decrease in pixel values ​​in the scaled image of the first image compared to the first image; the high-frequency texture enhancement data includes information describing the increase in high-frequency texture in the magnified image of the first image compared to the first image and / or information describing the decrease in high-frequency texture in the scaled image of the first image compared to the first image. The method further includes: In response to a user's zoom-in or zoom-out operation on the image to be displayed, the zoom display data is decoded from the JPEG file, and the image to be displayed is enhanced based on the zoom display data to obtain an enhanced image to be displayed.

33. The method as described in claim 32, characterized in that, The scaling display data is carried in the APP field of the first JPEG image frame. Decoding the scaling display data from the JPEG file includes: The tone mapping curve used for scaling display is decoded from the APP field of the first JPEG image frame.

34. The method as described in claim 32, characterized in that, When the scaling display data includes a tone mapping curve for scaling the display, the tone mapping curve for scaling the display is encoded in the APP field of the first JPEG image frame; Decoding the zoomed display data from the JPEG file includes: Decode the tone mapping curve used for scaling display from the APP field of the first JPEG image frame; or, When the scaling display data includes the scaling enhancement data, the JPEG file also includes a third JPEG image frame, which carries the scaling enhancement data; decoding the scaling display data from the JPEG file includes: Decode the scaling enhancement data from the third JPEG image frame; or, When the scaling display data includes the high-frequency texture enhancement data, the JPEG file also includes a fourth JPEG image frame, which carries the high-frequency texture enhancement data; decoding the scaling display data from the JPEG file includes: Decode the high-frequency texture enhancement data from the fourth JPEG image frame.

35. The method as described in claim 34, characterized in that, The APP field of the first JPEG image frame further includes a second position identifier and / or a third position identifier, wherein the second position identifier is used to indicate the start position of the third JPEG image frame in the JPEG file, and the third position identifier is used to indicate the start position of the fourth JPEG image frame in the JPEG file; Decoding the scaling enhancement data from the third JPEG image frame includes: The start position of the third JPEG image frame is determined from the JPEG file based on the second position identifier, and the scaling enhancement data is decoded from the third JPEG image frame based on the start position; or, Decoding the high-frequency texture enhancement data from the fourth JPEG image frame includes: The start position of the fourth JPEG image frame is determined from the JPEG file based on the third position identifier, and the scaling enhancement data is decoded from the fourth JPEG image frame based on the start position.

36. The method according to any one of claims 29-30 and 33-35, characterized in that, The APP field of the first JPEG image frame also includes a scaling enhancement method identifier; when the scaling enhancement method identifier is the fifth value, it indicates that the scaling enhancement method is to overlay the scaling enhancement data; When the scaling enhancement method identifier is the sixth value, it indicates that the scaling enhancement method is a weighted superposition of the high-frequency texture enhancement data; When the scaling enhancement method is identified as the seventh value, it indicates that the scaling enhancement method is to adjust the brightness of the image enhanced based on the first brightness-related data of the second image according to the tone mapping curve used for scaling display. The enhanced image to be displayed is obtained by enhancing the image based on the scaling display data, including: When the scaling enhancement method is identified as the fifth value, the scaling enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed; or... When the scaling enhancement method is identified as the sixth value, the high-frequency texture enhancement data is enhanced according to the set enhancement coefficient, and the enhanced high-frequency texture enhancement data is superimposed on the image to be displayed to obtain the enhanced image to be displayed. When the scaling enhancement mode is identified as the seventh value, the brightness of the image to be displayed is adjusted based on the tone mapping curve used for scaling the display.

37. The method according to any one of claims 23-30 and 33-35, characterized in that, The image to be displayed is obtained by adjusting the pixel values ​​of the pixels in the third image based on the first brightness-related data, including: Determine the screen backlight of the display device before adjustment and determine the screen backlight of the display device after adjustment; The pixel values ​​of the third image are adjusted based on the screen backlight of the display device before adjustment and the screen backlight of the display device after adjustment to obtain the fourth image; The fourth image and the third image are weighted and fused to obtain the image to be displayed; The pixel weights of the third and fourth images are determined based on the first brightness-related data.

38. The method as described in claim 37, characterized in that, The pixel values ​​of each pixel in the fourth image satisfy the following conditions: ； in, This represents the pixel value of the i-th pixel in the fourth image. This represents the pixel value of the i-th pixel in the third image. This refers to the screen backlight of the display device before adjustment. This indicates the adjusted screen backlight of the display device. This indicates the gamma value of the display device. Indicated by base Power of 1.

39. The method as described in claim 37, characterized in that, The pixel values ​​of each pixel in the image to be displayed satisfy the following conditions: ； ； Where N represents the number of bits occupied by the second image. This represents the first brightness-related data corresponding to the i-th pixel. This represents the pixel value of the i-th pixel in the fourth image. This represents the pixel value of the i-th pixel in the third image. This represents the pixel value of each pixel in the image to be displayed.

40. The method according to any one of claims 23-30, 33-35, and 38-39, characterized in that, The method further includes: Adjust the screen backlight of the display device.

41. The method as described in claim 37, characterized in that, The adjusted screen backlight of the display device meets the following conditions: ;or, ； in, This indicates the adjusted screen backlight of the display device. This refers to the screen backlight of the display device before adjustment. This represents the brightness-related data corresponding to the i-th pixel of the display screen of the display device. This represents the average value of the brightness-related data corresponding to the pixels of the display screen of the display device. This represents the maximum value of the brightness-related data corresponding to the pixels on the display screen of the display device. All are positive numbers. All are positive numbers less than 1.

42. A data acquisition device, characterized in that, include: A non-volatile memory and a processor are coupled together, the processor calling program code stored in the memory to perform the method as described in any one of claims 1-22.

43. A display terminal device, characterized in that, include: A non-volatile memory and a processor are coupled together, the processor calling program code stored in the memory to perform the method as described in any one of claims 23-41.

44. An image processing system, characterized in that, It includes the acquisition device as described in claim 42 and the display device as described in claim 43.

45. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, the program code including instructions for a processor to execute the method as described in any one of claims 1-41.

Images