Encoding method and apparatus, decoding method and apparatus, and device, storage medium and program product

Abstract

The present application belongs to the technical field of image compression. Disclosed are an encoding method and apparatus, a decoding method and apparatus, and a device, a storage medium and a program product. The encoding method comprises: extracting data of a target component in a first image, so as to obtain first component data, wherein a color space model of the first image comprises data of a plurality of color components, and the target component is at least one of the plurality of color components (701); acquiring a second image having the same content as the first image, wherein the sampling rate of the target component in the second image is less than the sampling rate of the target component in the first image (702); and encoding the first component data and the second image into a bitstream (703). The present application ensures that all original data in a first image can be encoded and transmitted to a receiving end, such that the encoded data is more complete and comprehensive.

Description

Coding and decoding method, device, equipment, storage medium and program product

[0001] The present application claims priority to the Chinese patent application No. 202410904499.2, filed on July 5, 2024, and entitled "Coding and decoding method, device, equipment, storage medium and program product", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of image compression technology, and in particular, to a coding and decoding method, device, equipment, storage medium and program product. BACKGROUND

[0003] Screen sharing is a core function in video conferencing, which is usually used to share the screen content of a browser window, an application, a power point (PPT), a word document, an excel table, etc., so that the members in the video conference can see the same content.

[0004] In the related art, screen sharing is usually achieved by using screen capture. That is, during the process of video conferencing, the screen content to be shared is captured and down-sampled to obtain a screen image in YUV420 format, and then the screen image in YUV420 format is encoded into a code stream, and the code stream is transmitted to a destination. The destination receives the code stream and decodes the code stream to obtain a reconstructed screen image, and then displays the reconstructed screen image.

[0005] However, coding and decoding the screen image in YUV420 format can cause the problem of poor image quality in screen sharing, such as image blurring or blurring of text in an image containing text, loss of details, etc. SUMMARY

[0006] The present application provides a coding and decoding method, device, equipment, storage medium and program product, which can code and decode the transmission of screen images in YUV422 format and YUV44 format, and improve the display effect of screen images in the screen projection / screen sharing scenario. The technical solution is as follows:

[0007] In a first aspect, a coding method is provided, and the method comprises:

[0008] extracting data of a target component in the first image to obtain first component data, wherein a color space model of the first image comprises data of a plurality of color components, and the target component is at least one of the plurality of color components; obtaining a second image having the same content as the first image, wherein a sampling rate of the target component in the second image is less than a sampling rate of the target component in the first image; and encoding the first component data and the second image into a bitstream.

[0009] In consideration of the fact that, when encoding the first image, the data of at least one color component in the first image is usually down-sampled to realize encoding compression of the image, but the loss of data of the color component will result in impaired image quality. Therefore, to reduce the impairment of the image quality of the first image to be encoded in the image compression process, the present application, when encoding the first image, follows the encoding mode in the related art, first down-samples the target component in the first image to obtain a second image having the same content as the first image and a sampling rate of the target component being less than that of the first image, and then encodes the second image into a bitstream. On this basis, the present application also encodes the first component data corresponding to the original target component in the first image into the bitstream. In this way, all original data in the first image can be encoded and transmitted to the receiving end, and the encoded data is more complete and comprehensive.

[0010] Further, the receiving end can parse the reconstructed first component data and the reconstructed second image from the bitstream after receiving the bitstream, and then reconstruct the first image according to the reconstructed first component data and the reconstructed second image. In this way, the quality of the reconstructed image can be greatly improved by reconstructing the first image in combination with the first component data of the first image.

[0011] As can be seen, the present application can effectively solve the problem of image blurring caused by loss of chroma information under the condition of existing software and hardware devices, that is, the efficiency of encoding a screen image is improved without increasing additional hardware costs.

[0012] In a possible implementation manner, the color space model of the first image is a YUV model of luminance, chrominance and saturation, the target component is a chrominance component, and the chrominance component comprises a U component and / or a V component. The obtaining of the second image having the same content as the first image comprises: performing down-sampling processing on the data of the chrominance component in the first image to obtain the second image.

[0013] Therefore, when the first image is encoded, the encoding process of the second image in YUV420 format is not changed, and on this basis, the original UV44 data in the first image is also encoded and transmitted, so that the code stream transmitted to the receiving end includes not only the second image in YUV420 format, but also the original UV44 data in the first image. In this way, the receiving end can parse the reconstructed second image in YUV420 format and the reconstructed UV44 data from the code stream, and further reconstruct the first image in YUV444 format.

[0014] That is, the present application can be compatible with YUV444 format or YUV422 format image coding based on the existing YUV420 format image coding and decoding scheme, so that the quality loss of the reconstructed image is small, and the quality of the reconstructed image is improved.

[0015] In a possible implementation, the encoding the first component data and the second image into a code stream comprises: encoding the second image into the code stream; and encoding the first component data into a target field in the code stream, the target field being a field in the code stream for carrying auxiliary information.

[0016] In a possible implementation, the target field comprises a supplemental enhancement information (SEI) field.

[0017] Therefore, the present application can transmit the first component data and the second image through two code streams, or transmit the first component data and the second image through one code stream. In the case of transmitting the first component data and the second image through one code stream, the first component data can be written into the target field of the second image encoding code stream.

[0018] Encoding the first component data comprises the following two cases:

[0019] The first case is to independently encode the first component data according to the encoding process of the second image, and write the encoded data into the target field in the code stream.

[0020] The second case is to obtain a reconstructed second image based on the code stream; obtain second component data, the second component data being obtained by upsampling the data of the target component in the reconstructed second image; and encode residual data between the second component data and the first component data into the target field in the code stream.

[0021] That is, the target field carries residual data corresponding to the first component data.

[0022] In the encoding of the first component data, if the first component data is calculated by using inter-frame prediction to obtain residual data, a previous frame of the first image is taken as a reference frame, and the residual between the data of the target component in the reference frame and the first component data is encoded into the target field in the code stream. If the first component data is calculated by using intra-frame prediction to obtain residual data, the first component data is independently encoded by using the first encoding method, and the residual between the data of the target component in the reconstructed first image and the first component data is encoded into the target field in the code stream by taking the reconstructed first image as a reference frame; or the second encoding method is used for encoding, and the residual between the second component data and the first component data is encoded into the target field in the code stream.

[0023] In a possible implementation, the first image is in YUV444 format, the second image is in YUV420 format, and the first component data includes UV44 data; or the first image is in YUV444 format, the second image is in YUV422 format, and the first component data includes UV44 data; or the first image is in YUV422 format, the second image is in YUV420 format, and the first component data includes UV22 data.

[0024] In a possible implementation, the first image is determined based on screen content in a screen sharing process.

[0025] For example, the first image is obtained by taking a screenshot, recording a screen, or taking a photo of the screen content in the screen sharing process. The application does not limit the manner of obtaining the first image in the screen sharing scenario.

[0026] In a second aspect, a decoding method is provided, and the method includes:

[0027] reconstructing the first image based on the reconstructed second image and the reconstructed first component data.

[0028] Therefore, after receiving the code stream, the receiving end can parse the reconstructed first component data and the reconstructed second image from the code stream, and then reconstruct the first image by combining the reconstructed first component data and the reconstructed second image. Since the sampling rate of each channel data in the reconstructed first image is consistent with that of the first image, the quality loss of the reconstructed first image is small, thereby improving the quality of the reconstructed image.

[0029] In other words, the present application can effectively solve the problem of image blurring caused by loss of chrominance information under the condition of existing software and hardware devices, that is, improve the efficiency of encoding screen images without increasing additional hardware costs.

[0030] In a possible implementation, the color space model of the first image is a brightness color saturation YUV model, and the target component is a chrominance component, which includes a U component and / or a V component.

[0031] Taking a screen image in the YUV444 format as an example, after receiving the code stream encoded by the encoding method provided in the present application, the receiving end can parse the reconstructed second image in the YUV420 format and the reconstructed UV44 data from the code stream, and then merge the Y channel data in the second image with the reconstructed UV44 data to reconstruct the first image in the YUV444 format.

[0032] Therefore, the present application can be compatible with the YUV444 format or the YUV422 format image coding on the basis of the existing YUV420 format image coding and decoding scheme, so that the quality loss of the reconstructed image is small, and the quality of the reconstructed image is improved.

[0033] In a possible implementation, the obtaining of the reconstructed first component data based on the code stream includes: obtaining the reconstructed first component data based on a target field in the code stream, the target field being a field in the code stream for carrying auxiliary information.

[0034] In a possible implementation, the target field includes a supplemental enhancement information (SEI) field.

[0035] Therefore, the present application can send the first component data and the second image through two code streams, or send the first component data and the second image through one code stream. In the case of sending the first component data and the second image through one code stream, the first component data can be written into the target field of the second image encoding code stream.

[0036] In a possible implementation, the obtaining of the reconstructed first component data based on the target field in the code stream includes:

[0037] obtaining reconstructed second component data, the reconstructed second component data being obtained by up-sampling data of the target component in the reconstructed second image; obtaining reconstructed residual data based on the target field in the code stream, the residual data being a residual between the first component data and the reconstructed second component data; and obtaining the reconstructed first component data based on the residual data and the reconstructed second component data.

[0038] That is, the target field carries residual data corresponding to the first component data, and if the residual data is determined based on data of the target component in the reconstructed second image as reference information, then when the reconstructed first component data is obtained, data of the target component in the reconstructed second image also needs to be referred to.

[0039] In a possible implementation, the reconstructing the first image based on the reconstructed second image and the reconstructed first component data comprises:

[0040] extracting data corresponding to other color components of the reconstructed second image except the target component to obtain third component data; and performing component data merging on the third component data and the reconstructed first component data to obtain the reconstructed first image.

[0041] In this case, the second image and the first image are both YUV format images, the first component data can be data of a chroma component, and the third component data can be data of a luma component. For example, the first image is in YUV444 format, the second image is in YUV420 format, the first component data is UV data, and the third component data is Y data.

[0042] In a possible implementation, the reconstructed second image is in YUV420 format, the reconstructed first component data includes UV44 data, and the reconstructed first image is in YUV444 format; or the reconstructed second image is in YUV422 format, the reconstructed first component data includes UV44 data, and the reconstructed first image is in YUV444 format; or the reconstructed second image is in YUV420 format, the reconstructed first component data includes UV22 data, and the reconstructed first image is in YUV422 format.

[0043] In a possible implementation, the first image is determined based on screen content in a screen sharing process.

[0044] For example, the screen content in the screen sharing process is captured, recorded, photographed, or the like, to obtain the first image. The manner of obtaining the first image in the screen sharing scenario is not limited in the present application.

[0045] In a third aspect, an encoding apparatus is provided, which has the function of implementing the encoding method in the first aspect. The encoding apparatus comprises at least one module for implementing the encoding method provided in the first aspect.

[0046] In a fourth aspect, a decoding apparatus is provided, which has the function of implementing the decoding method in the second aspect. The decoding apparatus comprises at least one module for implementing the decoding method provided in the second aspect.

[0047] In a fifth aspect, an encoding device is provided, which comprises a processor and a memory. The memory is configured to store a computer program for executing the encoding method provided in the first aspect. The processor is configured to execute the computer program stored in the memory, so as to implement the encoding method in the first aspect.

[0048] In a possible implementation, the encoding device can further comprise a communication bus for establishing a connection between the processor and the memory.

[0049] In a sixth aspect, a decoding device is provided, which comprises a processor and a memory. The memory is configured to store a computer program for executing the decoding method provided in the second aspect. The processor is configured to execute the computer program stored in the memory, so as to implement the decoding method in the second aspect.

[0050] In a possible implementation, the decoding device can further comprise a communication bus for establishing a connection between the processor and the memory.

[0051] In a seventh aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program runs on a computer or a processor, the computer or the processor executes the steps of the encoding method in the first aspect, or executes the steps of the decoding method in the second aspect.

[0052] In an eighth aspect, a computer program product is provided, which comprises computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the steps of the encoding method in the first aspect, or executes the steps of the decoding method in the second aspect. Alternatively, a computer program is provided. When the computer program runs on a computer or a processor, the computer or the processor executes the steps of the encoding method in the first aspect, or executes the steps of the decoding method in the second aspect.

[0053] In a ninth aspect, a coding system is provided, which includes an encoding device and a decoding device. The encoding device is configured to implement the steps of the encoding method of the first aspect. The decoding device is configured to implement the steps of the decoding method of the second aspect.

[0054] In a tenth aspect, an encoded bitstream is provided, which is generated according to the encoding method of the first aspect.

[0055] In an eleventh aspect, an encoded bitstream is provided, which includes first component data and a second image.

[0056] The first component data is data of a target component in the first image, the color space model of the first image includes data of a plurality of color components, and the target component is at least one of the plurality of color components. The second image has the same content as the first image, and a sampling rate of a target component in the second image is less than a sampling rate of the target component in the first image.

[0057] In a twelfth aspect, a computer-readable storage medium is provided, which stores a bitstream generated according to the encoding method of the first aspect.

[0058] In a thirteenth aspect, an apparatus for storing a bitstream is provided, which includes a receiver and at least one storage medium. The receiver is configured to receive a bitstream generated according to the encoding method of the first aspect. The at least one storage medium is configured to store the bitstream.

[0059] In a fourteenth aspect, an apparatus for transmitting a bitstream is provided, which includes a receiver and a transmitter. The receiver is configured to receive a bitstream generated according to the encoding method of the first aspect. The transmitter is configured to transmit the bitstream to an end-side device via a transmission medium.

[0060] In a fifteenth aspect, an apparatus for transmitting a bitstream is provided, which includes a transmitter and at least one storage medium. The at least one storage medium is configured to store a bitstream generated according to the encoding method of the first aspect. The transmitter is configured to obtain the bitstream from the storage medium and transmit the bitstream to an end-side device via a transmission medium.

[0061] In a sixteenth aspect, a system for distributing a bitstream is provided, which includes at least one storage medium configured to store a bitstream generated according to the encoding method of the first aspect, and a streaming device configured to obtain a target bitstream from the at least one storage medium and transmit the target bitstream to an end-side device. The streaming device includes a content server or a content distribution server.

[0062] The technical effects obtained by the third aspect to the sixteenth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect and the second aspect, and will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0063] FIG. 1 is a flow diagram of a screen sharing method according to an embodiment of the present application;

[0064] FIG. 2 is a flow diagram of an H.264 encoding method according to an embodiment of the present application;

[0065] FIG. 3 is a schematic diagram of an implementation environment according to an embodiment of the present application;

[0066] FIG. 4 is a schematic diagram of another implementation environment according to an embodiment of the present application;

[0067] FIG. 5 is a structural diagram of a first video module according to an embodiment of the present application;

[0068] FIG. 6 is a structural diagram of a second video module according to an embodiment of the present application;

[0069] FIG. 7 is a flow diagram of an encoding method according to an embodiment of the present application;

[0070] FIG. 8 is a flow diagram of a decoding method according to an embodiment of the present application;

[0071] FIG. 9 is a flow diagram of an encoding and decoding method when sharing a screen image in a video conference according to an embodiment of the present application;

[0072] FIG. 10 is a flow diagram of an encoding method of a YUV444 image according to an embodiment of the present application;

[0073] FIG. 11 is a flow diagram of another encoding method of a YUV444 image according to an embodiment of the present application;

[0074] FIG. 12 is a structural diagram of an encoding apparatus according to an embodiment of the present application;

[0075] FIG. 13 is a structural diagram of a decoding apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in further detail below with reference to the drawings.

[0077] To facilitate understanding, before the encoding and decoding method according to the embodiments of the present application is explained and described, the terms, application background, and implementation environment related to the embodiments of the present application are introduced.

[0078] First, the terms related to the embodiments of the present application are introduced.

[0079] 1. H.26x encoding

[0080] H.26x encoding is a commonly used video encoding format and a block-oriented, motion-compensated video encoding standard. For example, advanced video coding (AVC, also known as H.264), high efficiency video coding (HEVC, also known as H.265).

[0081] 2. Quantization parameter

[0082] The quantization parameter (QP) value is used to reflect the spatial detail compression of an image. The smaller the QP value, the higher the quantization precision, and under the condition of constant resolution and frame rate, the higher the image quality and the larger the code rate.

[0083] 3. Multipoint control unit (MCU)

[0084] MCU, also known as multipoint controller, is a key device of a multipoint video conference system. The role of MCU is equivalent to that of a "switch" in a video conference system, which can separate the information streams from each conference site, extract audio, video, data, and signaling information, and then send the information and signaling of each conference site to the same processing module to complete the corresponding processes such as audio mixing or switching, video mixing or switching, data broadcasting and routing selection, timing and conference control, and finally recombine the required information of each conference site and send it to the corresponding terminal system device. That is, MCU can be responsible for the access of all video conference terminals and the exchange, forwarding and processing of conference audio and video code streams.

[0085] 4. RGB format

[0086] RGB is a commonly used image format. R stands for red, G stands for green, and B stands for blue. These three colors can display a wide range of colors when superimposed in different amounts. For example, in 8-bit RGB format, (0, 0, 0) represents black and (255, 255, 255) represents white.

[0087] 5. YUV format

[0088] YUV is a commonly used image format. "Y" in YUV represents luminance (Luma), i.e. the gray value; "U" and "V" represent chrominance (Chroma), which is used to specify the color of a pixel, and functions to describe the color and saturation, and the higher the UV value, the more saturated the color of the pixel. YUV usually refers to YCbCr, in which Y refers to the luminance component, Cb refers to the blue chrominance component, and Cr refers to the red chrominance component. For example, YUV in the H.264 encoding standard refers to YCbCr, i.e. U in YUV is equivalent to Cb, and the term V is equivalent to Cr.

[0089] Since the sensitivity of the human eye to Y is higher than that to U and V, in order to reduce the bandwidth, more luminance information can be saved and less chrominance information can be saved in image encoding. That is, the chrominance information to which the human eye is relatively insensitive can be compressed and sampled by reducing the sampling rate of U and V, so that a relatively small image file is obtained for transmission and display while the image quality is not significantly reduced.

[0090] Among them, the commonly used YUV formats include YUV444, YUV422 and YUV420. YUV444 represents full sampling, i.e. 4 Y, 4 U and 4 V values are sampled for every 4 points; YUV422 represents 2:1 horizontal sampling and full vertical sampling, i.e. 4 Y, 2 U and 2 V values are sampled for every 4 points; YUV420 represents 2:1 horizontal sampling and 2:1 vertical sampling, i.e. 4 Y, 1 U and 1 V values are sampled for every 4 points.

[0091] It should be noted that in the embodiments of the present application, the YUV format is referred to as a luminance color saturation format, the Y component is referred to as a luminance component, and the UV components are collectively referred to as color saturation, and the U component and the V component are collectively referred to as chrominance components.

[0092] 6、1080P and 4K

[0093] 1080P can also be referred to as 1080P resolution, and 1080P is two million pixels, and the commonly used resolution format is 1920*1080. 4K can also be referred to as 4K resolution, and 4K resolution is more than 8 million pixels, which is 4 times that of 1080P, and the commonly used resolution format is 3840*2160.

[0094] 7、Image up-sampling

[0095] Image up-sampling is to change an image from low resolution to high resolution to realize image enlargement. For an image with a resolution of H*W, S times up-sampling is performed on the image, and an image with a resolution of (SH)*(SW) is obtained.

[0096] 8、Image down-sampling

[0097] Image downsampling, also known as image reduction sampling, is the process of reducing the resolution of an image from high resolution to low resolution, thereby shrinking the image. For an image with a resolution of H*W, performing an S-fold downsampling on the image will result in an image with a resolution of (H / S)*(W / S).

[0098] 9. Supplemental Enhancement Information (SEI)

[0099] The SEI field in H.264 / H.265 encoding is a standard mechanism for embedding additional information into the video bitstream. During video encoding, the SEI field allows the addition of unnecessary auxiliary information to the video data stream. This auxiliary information does not directly affect the video decoding process, but it can be used to provide additional contextual information or enhance the user experience of the video content.

[0100] Taking the H.264 video coding standard as an example, the H.264 video coding standard divides the entire system framework into two layers: the video coding layer (VCL) and the network abstraction layer (NAL). The VCL layer is responsible for processing the content of the video data, including algorithms for compression and decompression, to ensure the effective transmission and storage of the video data. The main task of the NAL layer is to format the video data and provide necessary header information. The header information primarily ensures that the data can adapt to the requirements of various channels and storage media during transmission, such as different network bandwidths, transmission protocols, and storage formats.

[0101] The NAL unit (also known as the NAL module) is the basic syntax structure of the NAL layer, used to encapsulate the raw byte sequence payload (RBSP) from the VCL. Each NAL unit consists of a one-byte header and a series of RBSP data. The NAL header contains a NAL unit type field, which identifies the data type carried by the NAL unit; the RBSP data is passed from the VCL layer and contains the main content of the video data.

[0102] The H.264 / AVC standard has 17 different NAL unit types, each corresponding to different data types, such as video keyframes and video decoder configuration parameters. When the NAL unit type field value is 6, this NAL unit contains SEI information, which can include various types of data, such as timecode, scene descriptions, and user-defined data.

[0103] In some implementations, the SEI field has a specific format, including data type (payload type), data size (payload size), universally unique identifier (UUID), and valid data content (payload content). Specifically, when the data type is 5, this field is used to process unregistered user data, and this is handled by the function user_data_unregistered().

[0104] Therefore, the SEI field has the characteristics of independence and flexibility. Independence means that the SEI information is independent of the video decoding process. Even if the SEI information is lost, it will not affect the video decoding and playback. Flexibility means that the SEI field can be used to transmit various types of information, such as encoder parameters, copyright information, camera parameters, editing events, custom messages, etc.

[0105] In summary, when encoding and decoding video data, SEI information can be embedded in the video bitstream during encoding; during decoding, specific business logic can be executed based on this SEI information, such as obtaining copyright information or user-defined data.

[0106] 10. Screen content coding (SCC)

[0107] In recent years, screen content video, including computer-generated text, graphics, and animations, has become increasingly popular. This screen content video exhibits different colors and other signal characteristics compared to natural video. To address the need for efficient encoding of screen content video, many SCC (Screen Computing and Coding) tools have been developed, achieving significant progress in encoding efficiency.

[0108] HEVC-SCC is an encoding standard that includes SCC functionality. It has made many optimizations for screen content encoding, including encoding tools such as intra block copy (IBC), palette mode, adaptive color transformation, and adaptive motion vector resolution, which have improved the compression efficiency of screen content and the display effect of screen images to a certain extent.

[0109] Since HEVC-SCC coding is an extension of HEVC coding, HEVC-SCC coding naturally inherits all the features of HEVC coding, including the SEI field.

[0110] Secondly, the related background of the embodiments of the present application is introduced.

[0111] Screen projection / screen sharing is a core function in video conferencing, and screen sharing is usually used to share and display screen content such as browser windows, applications, PPTs, documents, and tables, so that members in the video conference can see the same content.

[0112] In the related art, screen sharing is usually implemented in the way of screen capture. That is, referring to FIG. 1, a video source image of a sending end is captured to obtain a screen image of screen content to be shared. Then, the sending end encodes the screen image to obtain a code stream, and transmits the code stream to a receiving end. After receiving the code stream, the receiving end decodes the code stream to obtain a reconstructed screen image, and then displays the reconstructed screen image.

[0113] In a possible implementation, for local screen projection, a code stream can be transmitted in a wireless connection mode of wireless fidelity (WiFi) or a wired connection mode of high definition multimedia interface (HDMI); for remote screen sharing, a video conference supports real-time synchronous interactive communication between points, between a point and multiple points, and between multiple points, which is mainly controlled by an MCU.

[0114] In a possible implementation, for screen sharing, the format of a video source image is usually RGB format, which needs to be converted into YUV format according to a related algorithm to obtain a screen image to be encoded. For the receiving end, the format supported by image display is usually RGB format, and if the format of a decoded and reconstructed image is YUV format, the image needs to be converted into RGB format before being displayed.

[0115] In some embodiments, in terms of video coding, the coding manner mainly includes a traditional video coding technology for camera-captured images, such as H.264 / H.265 coding, and a coding technology for screen image content, such as HEVC-SCC coding. For camera-captured images, there are sensor noise, continuous color tones, and complex textures in the images. For screen images, the images usually have no noise, discrete color tones, delicate lines, and sharp edges.

[0116] Taking the traditional video coding technology H.264 coding as an example, as shown in FIG. 2, when encoding the current frame image F n , the current frame image F n is first divided into a plurality of pixel blocks, for example, the H.264 coding uses a 16*16 size region as a macro block by default, that is, the current frame is divided into a plurality of macro blocks with a size of 16*16, and based on the 16*16 size macro block, smaller sub blocks can be further divided, and the sub block size can be 16*8, 8*8, 4*4, 8*4, 4*8, etc. Then, the plurality of pixel blocks are encoded respectively to realize the encoding of the current frame image. In FIG. 2, the reference frame is only exemplified by the previous frame image (denoted as F n-1 ) of the current frame image F n , and the reconstructed frame image of the current frame image F n is denoted as F n ’.

[0117] Among them, the H.264 coding mainly uses the following four compression methods to compress the current frame image.

[0118] (1) Intra-frame prediction compression.

[0119] The intra-frame prediction compression is also called spatial compression, which is a video coding technology based on the correlation between the pixels inside the image, and reduces the amount of data to be encoded by predicting the value of the current pixel block. That is, the intra-frame prediction compression is based on the correlation between the pixels inside the image, and reduces the amount of data when encoding the current macro block by predicting the pixel value of the current macro block, thereby solving the spatial data redundancy problem of a single frame image and realizing efficient compression.

[0120] Among them, the implementation process of the intra-frame prediction compression can be: first, the current block is predicted in multiple modes (such as vertical prediction, horizontal prediction and average value prediction), to obtain the corresponding prediction block, then the current block and the prediction block are subtracted to calculate the residual (i.e. difference), and then the residual data is encoded and transmitted.

[0121] (2) Inter-frame prediction compression.

[0122] The inter-frame prediction compression is also called time compression, which is a video coding technology based on the correlation between the video image frames (i.e. time correlation), and reduces the amount of data to be encoded by predicting the pixel value of the current block. That is, the inter-frame prediction utilizes the similarity between adjacent frames to reduce the amount of data to be encoded by predicting the pixel value of the current block, thereby solving the time redundancy problem between video frames and realizing efficient compression.

[0123] Since the content between adjacent frames in a video is often very similar, it is possible to reduce the amount of data that needs to be transmitted or stored by exploiting this similarity. The principle of inter-frame prediction compression is to find a reference block from an adjacent reference frame that has the least difference with a current block in the current frame through motion estimation (ME), and then encode and transmit the motion vector (MV) of this reference block and the residual data (i.e., the difference between the current block and the reference block).

[0124] Correspondingly, when decoding, the receiving end will reconstruct the image of the current frame according to the received motion vector and residual data in combination with the reference frame.

[0125] As an example, the reference frame can be the previous coded reconstructed frame of the current frame F n , i.e., F n-1 frame, and the reference frame can also be multiple coded reconstructed frames, which is not limited by the embodiments of the present application.

[0126] (3) Integer discrete cosine transform (DCT).

[0127] Integer DCT is a technique for lossy data compression of image blocks, which converts the image from the spatial domain to the frequency domain and combines quantization and encoding techniques to achieve efficient data compression while maintaining acceptable image quality.

[0128] When encoding, DCT transform is first applied to each macroblock to convert it from the spatial domain to the frequency domain. In the frequency domain, the energy of the image is usually concentrated in the low-frequency part, while the high-frequency part contains the details and noise of the image, and DCT transform makes this energy distribution more obvious, providing convenience for subsequent compression steps. Then, the DCT coefficients after the transform are quantized. Quantization is a lossy process that rounds or truncates the DCT coefficients according to a certain quantization table, thereby reducing the amount of data to be stored or transmitted. The size of the quantization step determines the trade-off between compression ratio and image quality, the larger the quantization step, the higher the compression ratio, but the greater the loss of image quality; conversely, the smaller the quantization step, the higher the image quality, but the compression ratio is also reduced accordingly. Further, the quantized DCT coefficients are further encoded, for example, using Huffman coding or other entropy coding techniques, to further reduce the data volume. Finally, the encoded data can be stored or transmitted.

[0129] During the decoding process, these encoded data will be decoded, dequantized, and applied with an inverse DCT transform (also known as IDCT) to recover an approximate representation of the image. Since quantization is lossy, there will be some quality loss between the decoded image and the original image, but in most cases, this quality loss is acceptable to the human eye.

[0130] (4) Context-based adaptive binary arithmetic coding (CABAC) compression.

[0131] CABAC is a context-based adaptive binary arithmetic coding technique that uses the principles of arithmetic coding to encode input data (such as syntax elements in video coding) into binary form. Unlike traditional variable-length coding (such as Huffman coding), arithmetic coding does not map each symbol to a fixed-length or variable-length code word, but rather encodes the entire message as a real number in the interval [0, 1).

[0132] Both CABAC and Huffman coding make use of the statistical properties of the data for encoding. In Huffman coding, data with high frequency of occurrence (high-frequency data) is assigned shorter code words, while data with low frequency of occurrence (low-frequency data) is assigned longer code words. CABAC also adopts a similar strategy by assigning a probability model to each syntax element and using arithmetic coding to generate code words. High-frequency syntax elements are usually assigned shorter code words, while low-frequency syntax elements are assigned longer code words.

[0133] In addition, CABAC, as an adaptive coding technique, dynamically updates the probability model of each syntax element during encoding to better match the actual data distribution. This adaptability makes CABAC generally superior to fixed coding table techniques such as Huffman coding in terms of coding efficiency. Moreover, CABAC also considers context information. When encoding a certain syntax element, it considers the values and types of previously encoded syntax elements to select a probability model that is more suitable for the current context. This context-relatedness enables CABAC to more effectively compress data.

[0134] In summary, traditional video coding techniques (such as H.264 / H.265 encoding) usually use a hybrid coding structure of inter-prediction and intra-prediction, as well as transform and quantization. However, when using this coding structure to encode screen images, there are cases of text blurring, detail loss, etc., resulting in poor coding effect.

[0135] Moreover, in the above-mentioned H.264 encoding process, since the format of the screen image is usually RGB format or YUV444 format, but most of the existing hardware devices only support YUV420 format encoding and decoding and transmission, therefore, before encoding the screen image, the format of the screen image needs to be converted into YUV420 format. However, after converting the screen image from YUV444 format to YUV420 format, the sampling rate of U and V components becomes one fourth of the original, and the loss of chrominance information will cause loss of image quality. Especially in the auxiliary stream projection scene of video conference, the loss of chrominance information will cause color abnormal phenomena such as fuzzy edges of text and black edges.

[0136] Based on this, the embodiment of the present application provides a coding method, which can realize the coding of YUV444 format image under the condition of existing software and hardware devices, and is compatible with the coding of YUV420 format image, so as to ensure that the shared YUV444 format screen image can be normally received and displayed at the receiving end, effectively solve the problem of image blur caused by the loss of chrominance information, and improve the display effect of screen image in the projection / screen sharing scene.

[0137] Finally, the implementation environment of the embodiment of the present application is introduced.

[0138] Please refer to FIG. 3, which is a schematic diagram of an implementation environment provided by the embodiment of the present application. The implementation environment includes a source device 10, a destination device 20, a link 30 and a storage device 40. The source device 10 can generate encoded video, i.e. a bitstream. Therefore, the source device 10 can also be referred to as an encoding device. The destination device 20 can decode the bitstream generated by the source device 10. Therefore, the destination device 20 can also be referred to as a decoding device. The link 30 can receive the encoded video generated by the source device 10, and can transmit the encoded video to the destination device 20. The storage device 40 can receive the encoded video generated by the source device 10, and can store the encoded video. In this case, the destination device 20 can directly obtain the encoded video from the storage device 40. Alternatively, the storage device 40 can correspond to a file server or another intermediate storage device that can save the encoded video generated by the source device 10, in which case the destination device 20 can stream or download the encoded video stored by the storage device 40.

[0139] Source device 10 and destination device 20 each can include one or more processors and a memory coupled to the one or more processors, the memory can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, any other medium that can be used to store desired program code in the form of instructions or data structures, and that can be accessed by a computer, etc. Source device 10 and destination device 20 each can include, for example, a phone, a smartphone, a personal digital assistant (PDA), a wearable device, a pocket PC (PPC), a tablet PC, a smart car, a smart television, a smart speaker, a desktop computer, a mobile computing device, a notebook (e.g., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a so-called "smart" phone, a television, a camera, a display device, a digital media player, a video gaming console, an in-car computer, or the like.

[0140] Link 30 can include one or more media or devices capable of communicating encoded video from source device 10 to destination device 20. In one possible implementation, link 30 can include one or more communication media that enable source device 10 to transmit encoded video directly to destination device 20 in real-time. In embodiments of the present disclosure, source device 10 can modulate the encoded video based on a communication standard, which can be a wireless communication protocol, and transmit the modulated video to destination device 20. The one or more communication media can include wireless and / or wired communication media, such as the one or more communication media can include one or more physical transmission lines, a radio frequency (RF) spectrum, or other like mediums. The one or more communication media can form part of a packet-based network, such as a local area network, a wide-area network, or a global network such as the Internet, and can include routers, switches, base stations, or other equipment that facilitate communication from source device 10 to destination device 20. Embodiments of the present disclosure are not limited to a particular type of communication media.

[0141] In one possible implementation, the storage device 40 can store the encoded video received from the source device 10, and the destination device 20 can access the encoded video directly from the storage device 40. In such a case, the storage device 40 can include any of a variety of distributed or locally accessed data storage media such as a hard drive, Blu-ray discs, digital versatile discs (DVDs), compact disc read-only memories (CD-ROMs), flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing the coded video stream, and the like.

[0142] In one possible implementation, the storage device 40 can correspond to a file server, or another intermediate storage device that can hold the coded video generated by the source device 10, from which the destination device 20 can access the stored images via streaming or download. The file server can be any type of server capable of storing encoded video and transmitting that encoded video to the destination device 20. In one possible implementation, the file server can include a web server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive, among others. The destination device 20 can access the encoded images stored by the storage device 40 through any standard data connection, including an Internet connection. The standard data connection can include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, or the like), or a combination of both that is suitable for accessing the encoded video stored on the file server. The transmission of the encoded video from the storage device 40 can be a streaming transmission, a download transmission, or a combination of both.

[0143] The implementation environment shown in FIG. 3 is merely one possible implementation, and the techniques of embodiments of the present application can be applied to other devices that can encode video and devices that can decode a coded video stream in addition to the source device 10 that can encode images and the destination device 20 that can decode encoded video shown in FIG. 2, which is not limited in the present application.

[0144] In the example environment shown in FIG. 3, source device 10 includes a data source 120, an encoder 100, and an output interface 140. In some embodiments, output interface 140 can include a modulator / demodulator (modem) and / or a transmitter, which can also be known as a transmitter. Data source 120 can include a video capture device (e.g., a video camera, etc.), a storage containing previously captured video, a feed interface to receive video from a video content provider, and / or a computer graphics system to generate video, or a combination of such sources of video.

[0145] Data source 120 can send video to encoder 100, which can encode the video received by data source 120 to produce encoded video. The encoder can send the encoded video to output interface. In some embodiments, source device 10 sends the encoded video directly to destination device 20 via output interface 140. In other embodiments, the encoded video can also be stored onto storage device 40 for later access by destination device 20 and use in decoding and / or display.

[0146] In the example environment shown in FIG. 3, destination device 20 includes an input interface 240, a decoder 200, and a display device 220. In some embodiments, input interface 240 includes a receiver and / or a modem. Input interface 240 can receive the encoded video from link 30 and / or from storage device 40, and then send the encoded video to decoder 200, which can decode the received encoded video to produce decoded video. The decoder can send the decoded video to display device 220. Display device 220 can be integrated with destination device 20 or can be external to destination device 20. In general, display device 220 displays the decoded video. Display device 220 can be any of a variety of types of display devices such as, for example, a liquid crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or another type of display device.

[0147] Although not shown in FIG. 3, in some aspects, encoder 100 and decoder 200 can each be integrated with an encoder and a decoder, respectively, and can include appropriate multiplexer-demultiplexer (MUX-DEMUX) units or other hardware and software, for encoding both audio and video in a common data stream or separate data streams. In some embodiments, if applicable, the MUX-DEMUX units can conform to the ITU H.223 multiplexer protocol, or other protocols, such as the user datagram protocol (UDP).

[0148] Encoder 100 and decoder 200 each can be any of the following: one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic circuitry, hardware, or any combinations thereof. If the techniques of this disclosure are implemented partially in software, a device can store instructions for the software in a suitable, non- volatile computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be considered processing circuitry. Each of encoder 100 and decoder 200 can be included in one or more encoders or decoders, respectively, any of which can be integrated as part of a combined encoder / decoder (CODEC) in a respective device.

[0149] Embodiments of the disclosure can generally refer to encoder 100 as "signaling" or "sending" certain information to another device, such as decoder 200. The term "signaling" or "sending" can generally refer to the communication of syntax elements and / or other data used for decoding compressed video. This communication can occur in real-time or near real-time. Alternatively, this communication can occur after a period of time, such as when syntax elements are stored in an encoded bitstream to a computer-readable storage medium at the time of encoding, which a decoding device can then retrieve at any time after the syntax elements are stored to this medium.

[0150] FIG. 4 is a schematic diagram of another implementation environment provided by the embodiment of the present application, which includes a sending terminal, a receiving terminal and an MCU. The sending terminal includes a first control module, and a first video module, a first display module and a first network transmission module connected to the first control module respectively. The receiving terminal includes a second control module, and a second video module, a second display module and a second network transmission module connected to the second control module respectively.

[0151] The local conference terminal (i.e. the sending terminal) can capture the screen content to be shared by the first video module to obtain a screen image, and then encode the screen image to obtain a first code stream. The first video module sends the first code stream to the MCU through the first network transmission module. The MCU sends the first code stream to the receiving terminal. After receiving the first code stream, the second network transmission module of the receiving terminal sends the first code stream to the second video module. The second video module can analyze the first code stream to obtain a reconstructed screen image, and send the reconstructed screen image to the second control module. The second control module controls the second display module to display the reconstructed screen image.

[0152] The MCU is mainly used for real-time synchronous interactive communication between the sending terminal and the receiving terminal. In some embodiments, if the decoding capability of the receiving terminal is insufficient, the MCU can also transcode the code stream into a format that can be decoded by the receiving terminal based on the decoding capability of the receiving terminal.

[0153] In a possible implementation, the sending terminal further includes a first audio module, which can be connected to a microphone. The receiving terminal further includes a second audio module, which can be connected to a loudspeaker. In this case, the microphone of the sending terminal can process the collected sound through the first audio module to obtain an audio file, and then send the audio file to the MCU through the first network transmission module. The MCU receives the audio file and sends the audio file to the second network transmission module of the receiving terminal. The second network transmission module sends the audio file to the second audio module. The second audio module receives the audio file and controls the loudspeaker to play the audio file.

[0154] In a possible implementation, the first control module of the sending terminal can also be connected with a camera, and the camera is configured to capture a user participating in the screen sharing. In this case, the camera of the sending terminal can send the captured picture to the first control module, the first control module receives the captured picture and sends the captured picture to the first video module, the first video module receives the captured picture and encodes the captured picture to obtain a second code stream, and the first network transmission module sends the second code stream to the MCU. The MCU sends the second code stream to the receiving terminal, the second network transmission module of the receiving terminal receives the second code stream and sends the second code stream to the second video module, the second video module analyzes the second code stream to obtain a reconstructed captured picture, and then the second video module sends the reconstructed captured picture to the second control module, and the second control module controls the second display module to display the reconstructed captured picture.

[0155] That is, in the case where the first control module is connected with the camera, when the screen sharing is performed, not only the screen content can be shared, but also the real-time video of the user can be realized.

[0156] Referring to FIG. 5, the first video module of the sending terminal includes a data acquisition sub-module, an image encoding sub-module, a component encoding sub-module, and a code stream merging sub-module. When the encoding method provided in the present application is executed, after the first image is obtained by taking a screenshot of the screen content that needs to be shared, the data acquisition sub-module extracts the data of the target component in the first image to obtain first component data, and a second image with the same content as the first image is obtained, and the sampling rate of the target component in the second image is less than the sampling rate of the target component in the first image. Then, the second image is encoded into a code stream by the image encoding sub-module, the first component data is encoded into a code stream by the component encoding sub-module, and then the code stream of the second image and the code stream of the first component data are merged by the code stream processing sub-module, and are sent to the receiving terminal.

[0157] Referring to FIG. 6, the second video module of the receiving terminal includes an image decoding sub-module, a component decoding sub-module, and an image reconstruction sub-module. When the decoding method provided in the present application is executed, the reconstructed second image is parsed from the code stream by the image decoding sub-module, and the reconstructed first component data is parsed from the code stream by the component decoding sub-module. Then, the reconstructed first image is generated based on the reconstructed second image and the reconstructed first component data.

[0158] The above-mentioned sending terminal and receiving terminal can be any kind of electronic device with a screen sharing function, for example, the electronic device can be a PC (personal computer), a mobile phone, a PPC (pocket PC), a tablet computer, a television, etc.

[0159] The structure of the receiving end and the sending end in FIGS. 4, 5 and 6 is only an example. In actual application, the sending end can also be the receiving end, and the receiving end can also be the sending end. The embodiments of the present application do not limit this. If the sending end can also be the receiving end, and the receiving end can also be the sending end, in this case, the structure of the sending end and the receiving end is the same, that is, the sending end and the receiving end both include the first control module, the first video module, the first display module, the first network transmission module, the second control module, the second video module, the second display module and the second network transmission module.

[0160] It should be noted that the application scenarios and implementation environments described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of application scenarios and implementation environments, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0161] FIG. 7 is a flowchart of an encoding method provided by an embodiment of the present application. Please refer to FIG. 7, the method includes the following steps.

[0162] Step 701: Extract data of a target component in a first image to obtain first component data; wherein the color space model of the first image includes data of multiple color components, and the target component is at least one of the multiple color components.

[0163] In some embodiments, the color space model of the first image can be YUV, HSV, etc. Wherein YUV uses one luminance component (Y) and two chrominance components (U, V) to describe the color of a pixel. "Y" represents brightness, that is, the gray value; "U" and "V" represent chrominance, which are used to describe color and saturation. HSV is a color space model based on hue (H), saturation (S) and value (V). "H" represents hue, which determines the color type, such as red, blue, etc. "S" represents saturation, which determines the purity of the color. The higher the saturation, the purer the color. "V" represents the brightness of the color.

[0164] In a possible implementation, the color space model of the first image in step 701 is a YUV model, and the target component is a chrominance component, which includes a U component and / or a V component.

[0165] As an example, the first image is in YUV444 format, and the first component data is UV44 data; or the first image is in YUV422 format, and the first component data is UV22 data. The embodiments of the present application do not limit this.

[0166] In some embodiments, when the encoding method provided in the embodiments of the present application is applied to encoding the content of a screen shared / presented in a video conference, the first image is determined based on the screen content in the screen sharing process, for example, the screen content in the screen sharing process is screenshot, screen recording, photographing, etc. to obtain the first image, and the embodiments of the present application do not limit the way of obtaining the first image in the screen sharing scenario.

[0167] In actual applications, the image obtained by screenshot of the screen content in the screen sharing process can also be in RGB format, in which case the format of the image needs to be converted to YUV444 format according to the related algorithm.

[0168] It should be noted that the embodiments of the present application do not limit the resolution of the first image. For example, the resolution of the first image can be 180P, 360P, 540P, 720P, 1080P (i.e. 1920*1080), 4K, etc.

[0169] Step 702: obtaining a second image with the same content as the first image, the sampling rate of the target component in the second image being less than the sampling rate of the target component in the first image.

[0170] In a possible implementation, the implementation process of step 702 can be: determining the data of the target component in the first image, and then performing down-sampling processing on the data of the target component to obtain the second image.

[0171] As described above, the color space model of the first image is YUV model, and the target component is chrominance component, at this time, the process of obtaining the second image can be: performing down-sampling processing on the data of the chrominance component in the first image to obtain the second image.

[0172] As an example, the first image is in YUV444 format, and the second image obtained after down-sampling processing is in YUV420 format; or, the first image is in YUV444 format, and the second image obtained after down-sampling processing is in YUV422 format; or, the first image is in YUV422 format, and the second image obtained after down-sampling processing is in YUV420 format.

[0173] Step 703: encoding the first component data and the second image into a code stream.

[0174] In a possible implementation, the implementation process of step 703 can be: encoding the first component data into a code stream to obtain an encoded first code stream; and encoding the second image into a code stream to obtain an encoded second code stream.

[0175] That is, the first component data and the second image are encoded respectively to obtain two code streams. The encoding manner can be the traditional video encoding technology H.264 / H.265 encoding or HEVC-SCC encoding, and the embodiments of the present application do not limit this.

[0176] In another possible implementation, the implementation process of step 703 can be: encoding the second image into a code stream, and encoding the first component data into a target field in the code stream. The target field is a field in the code stream for carrying auxiliary information.

[0177] In a possible implementation, the target field includes an SEI field. That is, when the second image is encoded by using an encoding protocol such as H.264 encoding, H.265 encoding, or HEVC-SCC encoding, the first component data can be written into the SEI field in the code stream.

[0178] In some embodiments, encoding the first component data includes the following two cases:

[0179] In the first case, the first component data is independently encoded according to the encoding process of the second image, and the encoded data is written into the target field in the code stream.

[0180] In the second case, the reconstructed second image is obtained based on the code stream, and then the second component data is obtained from the reconstructed second image. The second component data is obtained by upsampling the data of the target component in the reconstructed second image. Furthermore, the residual data between the second component data and the first component data is encoded into the target field in the code stream.

[0181] It should be noted that, in either the first case or the second case, when the first component data is encoded, if the first component data uses inter-frame prediction to calculate residual data, a previous frame of the first image is used as a reference frame, and the residual between the data of the target component in the reference frame and the first component data is encoded into the target field in the code stream. If the first component data uses intra-frame prediction to calculate residual data, when the first component data is independently encoded according to the first case, the reconstructed first image is used as a reference frame, and the residual between the data of the target component in the reconstructed first image and the first component data is encoded into the target field in the code stream. When the first component data is encoded according to the second case, the reference information is the second component data, that is, the residual between the second component data and the first component data is encoded into the target field in the code stream. The embodiments of the present application do not limit this.

[0182] In summary, considering that the data of at least one color component in the first image is usually down-sampled when the first image is encoded, so as to realize the encoding compression of the image, but the loss of the data of the color component will result in the image quality being damaged. Therefore, in order to reduce the damage of the image quality of the first image to be encoded in the image compression process, the embodiment of the present application uses the encoding mode in the related art when encoding the first image, first performs down-sampling processing on the target component in the first image to obtain a second image with the same content as the first image and the sampling rate of the target component being less than the first image, and then encodes the second image into a code stream. On this basis, the embodiment of the present application also encodes the first component data corresponding to the original target component in the first image into the code stream. In this way, it is ensured that all original data in the first image can be encoded and transmitted to the receiving end, and the encoded data is more complete and comprehensive.

[0183] Further, after the receiving end receives the code stream, the reconstructed first component data and the reconstructed second image can be parsed from the code stream, and then the first image is reconstructed according to the reconstructed first component data and the reconstructed second image. In this way, the quality of the reconstructed image can be greatly improved by combining the first component data of the first image to reconstruct the first image.

[0184] Taking the first image as a screen image in YUV444 format as an example, as described above, the related image coding and decoding schemes are all down-sampling processing on the first image to obtain a second image in YUV420 format, and then encoding and transmitting the second image in YUV420 format. However, the method provided by the embodiment of the present application does not change the encoding process of the second image in YUV420 format when encoding the first image, but on this basis, the original UV44 data in the first image is also encoded and transmitted, so as to ensure that the code stream transmitted to the receiving end includes not only the second image in YUV420 format, but also the original UV44 data in the first image. In this way, the receiving end can parse the reconstructed second image in YUV420 format and the reconstructed UV44 data from the code stream after receiving the code stream, and then reconstruct the first image in YUV444 format.

[0185] As can be seen, the embodiment of the present application can effectively solve the problem of image blurring caused by the loss of chrominance information under the condition of existing software and hardware devices, that is, the efficiency of encoding the screen image is improved without increasing the additional hardware cost.

[0186] FIG. 8 is a flowchart of a decoding method provided by an embodiment of the present application. Please refer to FIG. 8, the method includes the following steps.

[0187] Step 801: obtaining a reconstructed second image based on a code stream; wherein a sampling rate of a target component in the second image is less than a sampling rate of the target component in the first image, a color space model of the first image comprises data of a plurality of color components, the target component is at least one of the plurality of color components, and the second image has the same content as the first image.

[0188] In other words, the second image is obtained by down-sampling the data of the target component in the first image and is sent to the receiving end after being encoded. Based on this, the receiving end can parse the reconstructed second image from the code stream corresponding to the second image after receiving the code stream.

[0189] In some embodiments, the color space model of the first image is a YUV model, and the target component is a chroma component, which includes a U component and / or a V component.

[0190] As an example, when the first image is in YUV444 format and the target component is a chroma component, the reconstructed second image can be in YUV422 format or YUV420 format; when the first image is in YUV422 format and the target component is a chroma component, the reconstructed second image can be in YUV420 format.

[0191] In some embodiments, when the encoding method provided by the embodiments of the present application is applied to the encoding of the content of the screen projection / sharing in the video conference, the first image is determined based on the screen content in the screen sharing process, for example, the screen content in the screen sharing process is captured, recorded, photographed, etc. to obtain the first image, and the embodiments of the present application do not limit the way of obtaining the first image in the screen sharing scenario.

[0192] Step 802: obtaining reconstructed first component data based on the code stream, the first component data being data of the target component in the first image.

[0193] As an example, when the first image is in YUV444 format and the reconstructed second image is in YUV422 format, the reconstructed first component data is UV44 data; when the first image is in YUV444 format and the reconstructed second image is in YUV420 format, the reconstructed first component data is UV44 data; when the first image is in YUV422 format and the reconstructed second image is in YUV420 format, the reconstructed first component data is UV22 data.

[0194] It should be noted that the code streams in step 802 and step 801 can be two different code streams or the same code stream. The embodiments of the present application do not limit this.

[0195] When the code stream of step 802 and the code stream of step 801 are different code streams, the receiving end receives two code streams, which are assumed to be a first code stream and a second code stream. In this case, the receiving end can parse the first component data to be reconstructed and the second image to be reconstructed from the two code streams according to the identification information of the two code streams. For example, if the identification information of the first code stream indicates that the code stream is an encoded code stream of component data, the first code stream can be parsed to obtain the first component data to be reconstructed; if the identification information of the second code stream indicates that the code stream is an encoded code stream of an image, the second code stream can be parsed to obtain the second image to be reconstructed. Embodiments of the present application do not limit the identification information corresponding to the two code streams, and are intended to illustrate that the second image and the first component data can be transmitted to the receiving end through the two code streams.

[0196] When the code stream of step 802 and the code stream of step 801 are the same code stream, the code stream simultaneously carries the encoded data of the second image and the encoded data of the first component data. In this case, one decoding operation can be performed on the code stream to parse the first component data to be reconstructed and the second image to be reconstructed.

[0197] In a possible implementation, the implementation process of obtaining the first component data to be reconstructed based on the code stream in step 802 can be: obtaining the first component data to be reconstructed based on a target field in the code stream, the target field being a field in the code stream for carrying auxiliary information.

[0198] In a possible implementation, the target field includes an SEI field. That is, when the second image is encoded by using an encoding protocol such as H.264 encoding, H.265 encoding, HEVC-SCC encoding, or the like, the first component data can be written into the SEI field in the code stream.

[0199] In a possible implementation, the target field carries residual data corresponding to the first component data. Referring to the related description in the encoding method, if the residual data is determined based on the data of the target component in the second image to be reconstructed as reference information, the data of the target component in the second image to be reconstructed also needs to be referred to when the first component data to be reconstructed is obtained.

[0200] Based on this, the implementation process of obtaining the first component data to be reconstructed based on the target field in the code stream can be: obtaining the second component data to be reconstructed, which is obtained by upsampling the data of the target component in the second image to be reconstructed; obtaining the residual data based on the target field in the code stream, the residual data being the residual between the first component data and the second component data to be reconstructed; and obtaining the first component data to be reconstructed based on the residual data and the second component data to be reconstructed.

[0201] Step 803: reconstructing the first image based on the second image to be reconstructed and the first component data to be reconstructed.

[0202] In a possible implementation, the implementation of step 803 can be: extracting data corresponding to the color components other than the target component in the reconstructed second image to obtain third component data; and performing component data merging on the third component data and the reconstructed first component data to obtain the reconstructed first image.

[0203] As an example, assuming that the first image and the second image are both in YUV format, and the target component is a chroma component, the reconstructed second image is in YUV420 format, and the reconstructed first component data is reconstructed UV44 data, at this time, the data of the Y channel in the reconstructed second image and the reconstructed UV44 data can be merged to obtain the reconstructed first image.

[0204] In summary, after receiving the bitstream, the receiving end can parse the reconstructed first component data and the reconstructed second image from the bitstream, and then reconstruct the first image by combining the reconstructed first component data and the reconstructed second image. Since the sampling rate of each channel data in the reconstructed first image is consistent with that of the first image, the quality loss of the reconstructed first image is small, thereby improving the quality of the reconstructed image.

[0205] Taking a screen image in YUV444 format as an example, after receiving the bitstream encoded according to the encoding method shown in the embodiments of the present application, the receiving end can parse the reconstructed second image in YUV420 format and the reconstructed UV44 data from the bitstream, and then merge the Y channel data in the second image with the reconstructed UV44 data to reconstruct the first image in YUV444 format.

[0206] As can be seen, the embodiments of the present application can effectively solve the problem of image blurring caused by loss of chroma information under the condition of existing software and hardware devices, that is, improve the encoding efficiency of screen images without increasing additional hardware costs.

[0207] For ease of understanding, the encoding and decoding method provided by the embodiments of the present application will be explained and described exemplarily below taking the screen projection / screen sharing content in a video conference as an example.

[0208] Whether it is local screen projection (for example, local mobile phone / PC screen projection to a local large screen) or remote screen sharing (for example, local mobile phone / PC screen sharing transmitted to a remote display through a network), the overall encoding and decoding process is shown in FIG. 9. Referring to FIG. 9, the sending end collects the screen image to be shared, which can be a natural image captured by a camera, or a screenshot containing text, pictures, or a mixture of text and pictures, and the embodiments of the present application do not limit this.

[0209] It should be noted that if the screen image is in RGB format, it needs to be converted into YUV444 format, so as to obtain a YUV444 format image. In this example, for the convenience of description, the YUV444 format image is simply referred to as a YUV444 image, and the YUV420 format image is simply referred to as a YUV420 image.

[0210] Referring to FIG. 10, in a possible implementation, the process of encoding the YUV444 image by the sending end according to the encoding method provided in the embodiments of the present application can be as follows: first, performing down-sampling processing on the YUV444 image to obtain a YUV420 image; meanwhile, extracting the data of the U channel and the V channel from the YUV444 image to obtain original UV data. Then, encoding the YUV420 image by using an existing encoding protocol. For example, encoding the YUV420 image according to the H.264 encoding protocol to obtain a code stream of the YUV420 image. In the process of encoding the YUV420 image, a reconstructed YUV420 image is also generated. Performing 2 times up-sampling processing on the UV channel of the reconstructed YUV420 image to obtain UV44 data (i.e., an image including only the UV two channels, and the resolution is the same as that of the Y channel), and encoding the original UV44 data by taking the UV44 data as a reference to obtain UV44 encoded data. Finally, writing the UV44 encoded data into an SEI field in the code stream corresponding to the YUV420 image to obtain a final output code stream, and sending the code stream to the receiving end.

[0211] It should be noted that in the above implementation, the encoding method of the UV44 data is similar to the encoding process of the three channel data in the YUV420 image, except that the intra-frame and inter-frame prediction references the UV44 data obtained by the up-sampling processing, other steps such as residual calculation, transformation and quantization, residual inverse transformation, filtering, and entropy encoding remain unchanged.

[0212] Referring to FIG. 11, in another possible implementation, the process of encoding the YUV444 image by the sending end according to the encoding method provided in the embodiments of the present application can be as follows: first, performing down-sampling processing on the YUV444 image to obtain a YUV420 image; meanwhile, extracting the data of the U channel and the V channel from the YUV444 image to obtain original UV data. Then, encoding the YUV420 image by using an existing encoding protocol. For example, encoding the YUV420 image according to the H.264 encoding protocol to obtain a code stream of the YUV420 image. Similarly, encoding the original UV44 data according to the encoding method of the three channel data in the YUV420 image to obtain UV44 encoded data. Finally, writing the UV44 encoded data into an SEI field in the code stream of the YUV420 image to obtain a final output code stream, and sending the code stream to the receiving end.

[0213] It should be noted that, in this implementation, since the original UV data is independently encoded and does not refer to the UV data in the YUV420 image, the UV data can also be directly decoded when decoding without referring to the UV data in the reconstructed YUV420 image.

[0214] When the receiving end receives the code stream, if the SEI field in the code stream does not carry the UV44 encoded data, the reconstructed YUV420 image is directly decoded based on the code stream, so as to realize the compatibility of the YUV420 format. If the SEI field in the code stream carries the UV44 encoded data, the reconstructed UV44 data is decoded from the code stream, and then the reconstructed UV44 data is merged with the Y channel data in the reconstructed YUV420 image to obtain the reconstructed YUV444 image, so as to realize the coding and decoding of the YUV444 image.

[0215] In summary, when the YUV444 image is encoded and transmitted, the encoding process of the YUV420 image is not changed, and on this basis, the original UV44 data in the YUV444 image is also encoded and transmitted, so as to ensure that the code stream transmitted to the receiving end includes the data of the YUV420 image and the UV44 data in the YUV444 image. The receiving end can parse the reconstructed YUV420 image and the reconstructed UV44 data from the code stream after receiving the code stream, and then reconstruct the YUV444 image. As can be seen, the embodiment of the application can effectively solve the problem of image blur caused by loss of chrominance information under the condition of existing software and hardware devices, that is, the efficiency of encoding the screen image is improved without increasing the additional hardware cost.

[0216] FIG. 12 is a structural schematic diagram of an encoding device provided by an embodiment of the application. The encoding device can be realized by software, hardware or a combination of both to become part or all of an encoding end device or a sending end device. Referring to FIG. 12, the encoding device includes a component extraction module 1201, an image acquisition module 1202 and an encoding module 1203.

[0217] The component extraction module 1201 is configured to extract data of a target component in a first image to obtain first component data, wherein a color space model of the first image includes data of a plurality of color components, and the target component is at least one of the plurality of color components.

[0218] The image acquisition module 1202 is configured to acquire a second image having the same content as the first image, and a sampling rate of the target component in the second image is less than a sampling rate of the target component in the first image.

[0219] The encoding module 1203 is configured to encode the first component data and the second image into a code stream.

[0220] In a possible implementation, the color space model of the first image is a brightness color saturation YUV model, the target component is a chroma component, and the chroma component includes a U component and / or a V component; the image acquisition module 1202 is specifically configured to:

[0221] perform down-sampling processing on the data of the chroma component in the first image to obtain a second image.

[0222] In a possible implementation, the encoding module 1203 includes:

[0223] a first encoding unit, configured to encode the second image into a code stream;

[0224] a second encoding unit, configured to encode the first component data into a target field in the code stream, the target field being a field in the code stream for carrying auxiliary information.

[0225] In a possible implementation, the second encoding unit is specifically configured to:

[0226] obtain a reconstructed second image based on the code stream;

[0227] obtain second component data by performing up-sampling processing on the data of the target component in the reconstructed second image;

[0228] encode residual data between the second component data and the first component data into the target field in the code stream.

[0229] In a possible implementation, the target field includes a supplemental enhancement information SEI field.

[0230] In a possible implementation, the first image is in YUV444 format, the second image is in YUV420 format, and the first component data includes UV44 data; or the first image is in YUV444 format, the second image is in YUV422 format, and the first component data includes UV44 data; or the first image is in YUV422 format, the second image is in YUV420 format, and the first component data includes UV22 data.

[0231] In a possible implementation, the first image is determined based on screen content in a screen sharing process.

[0232] In the embodiments of the present application, considering that the data of at least one color component in the first image is usually down-sampled when the first image is encoded, so as to realize the encoding compression of the image, but the loss of the data of the color component will result in the impairment of the image quality. Therefore, in order to reduce the impairment of the image quality of the first image to be encoded in the image compression process, in the encoding of the first image, the encoding manner in the related art is used, the target component in the first image is first down-sampled to obtain a second image with the same content as the first image and the sampling rate of the target component being less than that of the first image, and then the second image is encoded into a code stream. On this basis, the first component data corresponding to the original target component in the first image is also encoded into the code stream. In this way, all the original data in the first image can be encoded and transmitted to the receiving end, and the encoded data is more complete and comprehensive.

[0233] Further, after receiving the code stream, the receiving end can parse the reconstructed first component data and the reconstructed second image from the code stream, and then reconstruct the first image according to the reconstructed first component data and the reconstructed second image. In this way, the quality of the reconstructed image can be greatly improved by reconstructing the first image in combination with the first component data of the first image.

[0234] As can be seen, the embodiments of the present application can effectively solve the problem of image blurring caused by the loss of chroma information under the condition of existing software and hardware devices, that is, the efficiency of encoding the screen image is improved without increasing the additional hardware cost.

[0235] It should be noted that the encoding apparatus provided in the above embodiments is only used as an example for illustrating the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to the needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the above described functions. In addition, the encoding apparatus and the encoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0236] FIG. 13 is a structural schematic diagram of a decoding apparatus provided in the embodiments of the present application. The decoding apparatus can be realized by software, hardware or a combination of both to become part or all of the decoding end device or the receiving device. Referring to FIG. 13, the decoding apparatus includes a first decoding module 1301, a second decoding module 1302 and an image reconstruction module 1303.

[0237] The first decoding module 1301 is configured to obtain a reconstructed second image based on the bitstream; wherein a sampling rate of a target component in the second image is less than a sampling rate of the target component in the first image, a color space model of the first image comprises data of a plurality of color components, the target component is at least one of the plurality of color components, and the second image has the same content as the first image.

[0238] The second decoding module 1302 is configured to obtain reconstructed first component data based on the bitstream, the first component data being data of the target component in the first image.

[0239] The image reconstruction module 1303 is configured to reconstruct the first image based on the reconstructed second image and the reconstructed first component data.

[0240] In a possible implementation, the color space model of the first image is a YUV model, and the target component is a chroma component, the chroma component comprising a U component and / or a V component.

[0241] In a possible implementation, the second decoding module 1302 is specifically configured to:

[0242] obtain the reconstructed first component data based on a target field in the bitstream, the target field being a field in the bitstream for carrying auxiliary information.

[0243] In a possible implementation, the target field comprises a supplemental enhancement information (SEI) field.

[0244] In a possible implementation, the second decoding module 1302 is specifically configured to:

[0245] obtain reconstructed second component data by up-sampling data of the target component in the reconstructed second image;

[0246] obtain reconstructed residual data based on a target field in the bitstream, the residual data being a residual between the first component data and the reconstructed second component data;

[0247] obtain the reconstructed first component data based on the residual data and the reconstructed second component data.

[0248] In a possible implementation, the image reconstruction module 1303 is specifically configured to:

[0249] extract data corresponding to color components other than the target component in the reconstructed second image to obtain third component data;

[0250] perform component data merging on the third component data and the reconstructed first component data to obtain the reconstructed first image.

[0251] In a possible implementation, the reconstructed second image is in YUV420 format, the reconstructed first component data comprises UV44 data, and the reconstructed first image is in YUV444 format; or, the reconstructed second image is in YUV422 format, the reconstructed first component data comprises UV44 data, and the reconstructed first image is in YUV444 format; or, the reconstructed second image is in YUV420 format, the reconstructed first component data comprises UV22 data, and the reconstructed first image is in YUV422 format.

[0252] In a possible implementation, the first image is determined based on screen content in a screen sharing process.

[0253] In the embodiments of the present application, the decoding apparatus can parse the reconstructed first component data and the reconstructed second image from the bitstream, and then reconstruct the first image by combining the reconstructed first component data and the reconstructed second image. Since the sampling rate of each channel data in the reconstructed first image is consistent with that of the first image, the reconstructed first image has less loss of image quality, thereby improving the quality of the reconstructed image.

[0254] It can be seen that the embodiments of the present application can effectively solve the problem of image blurring caused by loss of chrominance information under the condition of existing software and hardware devices, that is, improve the efficiency of encoding screen images without increasing additional hardware costs.

[0255] It should be noted that the decoding apparatus provided in the above embodiments is only used as an example to illustrate the division of the above functional modules when decoding the first image based on the bitstream. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the decoding apparatus and the decoding method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0256] The embodiments of the present application also provide a computer readable storage medium storing a computer program, when the computer program runs on a computer or a processor, the computer or the processor executes the steps of the encoding method shown in the above embodiments, or executes the steps of the decoding method shown in the above embodiments.

[0257] The embodiments of the present application further provide a computer program product, which contains computer instructions, and when the computer instructions are executed by a computer or a processor, the computer or the processor executes the steps of the encoding method shown in the above embodiments or the steps of the decoding method shown in the above embodiments. In other words, a computer program is provided, and when the computer program is run on a computer or a processor, the computer or the processor executes the steps of the encoding method shown in the above embodiments or the steps of the decoding method shown in the above embodiments.

[0258] The embodiments of the present application further provide a codec system, which includes an encoding device and a decoding device, the encoding device is used to implement the steps of the encoding method shown in the above embodiments, and the decoding device is used to implement the steps of the decoding method shown in the above embodiments.

[0259] The embodiments of the present application further provide an encoded code stream, which is generated according to the encoding method shown in the above embodiments.

[0260] In a possible implementation, the code stream includes first component data and a second image; the first component data is data of a target component in a first image, a color space model of the first image includes data of a plurality of color components, and the target component is at least one of the plurality of color components; the second image has the same content as the first image, and a sampling rate of a target component in the second image is less than a sampling rate of the target component in the first image.

[0261] The embodiments of the present application further provide a computer readable storage medium, which stores a code stream generated according to the encoding method shown in the above embodiments.

[0262] The embodiments of the present application further provide a device for storing a code stream, which includes a receiver and at least one storage medium, the receiver is used to receive a code stream generated according to the encoding method shown in the above embodiments, and the at least one storage medium is used to store the code stream.

[0263] The embodiments of the present application further provide a device for transmitting a code stream, which includes a transmitter and a receiver, the receiver is used to receive a code stream generated according to the encoding method shown in the above embodiments, and the transmitter is used to transmit the code stream to an end-side device through a transmission medium.

[0264] The embodiments of the present application further provide a device for transmitting a code stream, which includes a transmitter and at least one storage medium, the at least one storage medium is used to store a code stream generated according to the encoding method shown in the above embodiments, and the transmitter is used to acquire the code stream from the storage medium and transmit the code stream to an end-side device through a transmission medium.

[0265] The application further provides a system for distributing a code stream, the system comprising at least one storage medium for storing a code stream generated by the encoding method according to the above-mentioned embodiments, and a streaming media device for obtaining a target code stream from the at least one storage medium and sending the target code stream to an end-side device, wherein the streaming media device comprises a content server or a content distribution server.

[0266] In the above-mentioned embodiments, the implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented by using software, the implementation can be achieved entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the entire or partial processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium accessible by a computer, or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital versatile disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc. It is worth noting that the computer-readable storage medium mentioned in the embodiments of the present application can be a non-volatile storage medium, in other words, can be a non-transitory storage medium.

[0267] It should be understood that the "multiple" mentioned herein refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" herein only describes the relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist together, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, "first", "second" and the like are used to distinguish the same items or similar items with basically the same function and effect. The skilled in the art can understand that "first", "second" and the like do not limit the quantity and execution order, and "first", "second" and the like do not necessarily mean different.

[0268] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0269] The above describes the embodiments provided by the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An encoding method characterized by comprising: The method comprises: extracting data of a target component in a first image to obtain first component data, wherein a color space model of the first image comprises data of a plurality of color components, and the target component is at least one of the plurality of color components; obtaining a second image with the same content as the first image, wherein a sampling rate of the target component in the second image is less than a sampling rate of the target component in the first image; encoding the first component data and the second image into a code stream.

2. The method of claim 1, wherein, The color space model of the first image is a YUV model, the target component is a chroma component, and the chroma component comprises a U component and / or a V component. The obtaining of the second image with the same content as the first image comprises: performing down-sampling processing on data of the chroma component in the first image to obtain the second image.

3. The method of claim 1 or 2, wherein, The encoding of the first component data and the second image into the code stream comprises: encoding the second image into the code stream; encoding the first component data into a target field in the code stream, wherein the target field is a field in the code stream for carrying auxiliary information.

4. The method of claim 3, wherein, The encoding of the first component data into the target field in the code stream comprises: obtaining a reconstructed second image based on the code stream; obtaining second component data by performing up-sampling processing on data of the target component in the reconstructed second image; encoding residual data between the second component data and the first component data into the target field in the code stream.

5. The method of claim 3 or 4, wherein, The target field comprises a SEI field.

6. The method according to any one of claims 1 to 5, wherein, The first image is determined based on screen content in a screen sharing process.

7. A decoding method, comprising: The method comprises: obtaining a reconstructed second image based on a code stream, wherein a sampling rate of a target component in the second image is less than a sampling rate of the target component in a first image, a color space model of the first image comprises data of a plurality of color components, the target component is at least one of the plurality of color components, and the second image has the same content as the first image; obtaining reconstructed first component data based on the code stream, wherein the first component data is data of the target component in the first image; reconstructing the first image based on the reconstructed second image and the reconstructed first component data.

8. The method of claim 7, wherein, The color space model of the first image is a YUV model, the target component is a chroma component, and the chroma component comprises a U component and / or a V component.

9. The method of claim 7 or 8, wherein, The obtaining of the reconstructed first component data based on the code stream comprises: obtaining the reconstructed first component data based on a target field in the code stream, wherein the target field is a field in the code stream for carrying auxiliary information.

10. The method of claim 9, wherein, The target field comprises a SEI field.

11. The method of claim 9 or 10, wherein, The obtaining of the reconstructed first component data based on the target field in the code stream comprises: obtaining reconstructed second component data by performing up-sampling processing on data of the target component in the reconstructed second image; reconstructed residual data based on the target field in the code stream, the residual data being a residual between the first component data and the reconstructed second component data; reconstructing the first component data based on the residual data and the reconstructed second component data.

12. The method according to any one of claims 7 to 11, characterized in that, reconstructing the first image based on the reconstructed second image and the reconstructed first component data, comprises: extracting data corresponding to other color components of the reconstructed second image except the target component to obtain third component data; performing component data merging on the third component data and the reconstructed first component data to obtain the reconstructed first image.

13. The method according to any one of claims 7 to 12, wherein, The first image is determined based on screen content in a screen sharing process.

14. An encoding apparatus, comprising: The apparatus comprises: a component extraction module configured to extract data of a target component in a first image to obtain first component data, wherein a color space model of the first image comprises data of a plurality of color components, and the target component is at least one of the plurality of color components; an image acquisition module configured to acquire a second image having the same content as the first image, wherein a sampling rate of the target component in the second image is less than a sampling rate of the target component in the first image; an encoding module configured to encode the first component data and the second image into a code stream.

15. A decoding apparatus, characterized by comprising: The apparatus comprises: a first decoding module configured to obtain a reconstructed second image based on a code stream, wherein a sampling rate of a target component in the second image is less than a sampling rate of the target component in a first image, a color space model of the first image comprises data of a plurality of color components, the target component is at least one of the plurality of color components, and the second image has the same content as the first image; a second decoding module configured to obtain reconstructed first component data based on the code stream, the first component data being data of the target component in the first image; an image reconstruction module configured to reconstruct the first image based on the reconstructed second image and the reconstructed first component data.

16. An encoding device, comprising: The encoding device comprises a memory and a processor; the memory is configured to store a computer program; the processor is configured to execute the computer program to implement steps of the encoding method in any one of claims 1-6.

17. A decoding device, comprising: The decoding device comprises a memory and a processor; the memory is configured to store a computer program; the processor is configured to execute the computer program to implement steps of the decoding method in any one of claims 7-13.

18. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, when the computer program runs on a computer or a processor, causes the computer or the processor to execute steps of the encoding method in any one of claims 1-6, or execute steps of the decoding method in any one of claims 7-13.

19. A computer program product, characterised in that, The computer program product contains computer instructions, when the computer instructions are executed by a computer or a processor, causes steps of the encoding method in any one of claims 1-6 to be executed, or steps of the decoding method in any one of claims 7-13 to be executed.

20. A coded bitstream, comprising: The code stream is generated according to the method in any one of claims 1-6.

21. A coded bitstream, comprising: The code stream comprises first component data and a second image. The first component data is data of a target component in the first image, the color space model of the first image comprises data of a plurality of color components, and the target component is at least one of the plurality of color components; the second image has the same content as the first image, and a sampling rate of a target component in the second image is less than a sampling rate of the target component in the first image.

22. A computer-readable storage medium, characterized in that, The computer readable storage medium stores the code stream in any one of claims 1-6.

23. An apparatus for storing a bitstream, characterized by The apparatus comprises a receiver and at least one storage medium, The receiver is configured to receive the code stream in any one of claims 1-6; The at least one storage medium is configured to store the code stream.

24. An apparatus for transmitting a code stream, the apparatus comprising: The apparatus comprises a transmitter and a receiver, The receiver is configured to receive the code stream in any one of claims 1-6; The transmitter is configured to transmit the code stream to an end-side device via a transmission medium.

25. An apparatus for transmitting a code stream, the apparatus comprising: The apparatus comprises a transmitter and at least one storage medium, The at least one storage medium is configured to store the code stream in any one of claims 1-6; The transmitter is configured to obtain the code stream from the storage medium and transmit the code stream to an end-side device via a transmission medium.

26. A system for distributing a code stream, the system comprising: The system comprises: At least one storage medium configured to store at least one code stream in any one of claims 1-6; A streaming media device configured to obtain a target code stream from the at least one storage medium and transmit the target code stream to an end-side device, wherein the streaming media device comprises a content server or a content distribution server.

Images