Multi-format video processing method and apparatus, terminal device, and medium

Abstract

The application discloses a multi-format video processing method and device, terminal equipment and medium, comprising: obtaining a to-be-transmitted video in a first pixel format and extracting a first video image; performing down-sampling on the first video image to generate a second video image in a second pixel format; performing up-sampling on the second video image to generate a third video image in the first pixel format; generating a corresponding first recombined image and a second recombined image according to the first video image and the corresponding third video image; encoding the second video image, the first recombined image and the second recombined image and transmitting them to a video receiving end so that the video receiving end can generate a video in one or more pixel formats. Therefore, the application can make the transmitted video meet the performance and demand of different video receiving ends without causing great pressure on the encoding end and the transmission end.

Description

Technical Field

[0001] This invention relates to the field of video encoding and transmission technology, and in particular to a multi-format video processing method, apparatus, terminal equipment, and medium. Background Technology

[0002] In the field of high-quality image and video processing, YUV format images and videos are commonly used. However, with technological advancements, the limitations of the YUV420 format in color sampling make it difficult to meet the demands of applications requiring higher color depth, saturation, and detail fidelity. Currently, higher sampling rate video formats, such as YUV422 and YUV444, are preferred in video encoding. Compared to YUV420, YUV422 and YUV444 formats offer higher chroma sampling rates, enabling them to more accurately capture the original color information in images and videos. With the development of digital video and image applications, the widespread adoption of the YUV444 format has become a trend.

[0003] However, during application, limitations in video transmission and encoding were discovered. First, different video receivers have different decoding characteristics and performance; some receivers do not support YUV422 and YUV444 formats. To comprehensively consider the performance and needs of different video receivers, existing technologies generally use a simultaneous transmission mode, transmitting the three encoded video data lines corresponding to YUV422, YUV420, and YUV444 formats together to the receiver. This allows the receiver to select the appropriate video format based on its performance and requirements.

[0004] While this approach can meet the performance and requirements of different video receivers, the encoding complexity of YUV422 is 1.3 times that of YUV420, and YUV444 is twice that of YUV420. This places a huge burden on the video encoding end. Furthermore, multiple video streams need to be sent during transmission, which requires sacrificing a significant amount of bandwidth resources from the transmitting end. Therefore, how to ensure that the transmitted video meets the performance and requirements of different video receivers without placing excessive pressure on the encoding and transmitting ends is a problem that urgently needs to be solved. Summary of the Invention

[0005] This invention provides a multi-format video processing method, apparatus, terminal device, and medium that can enable the transmitted video to meet the performance and requirements of different video receiving ends without putting excessive pressure on the encoding and transmission ends.

[0006] An embodiment of the present invention provides a multi-format video processing method, comprising:

[0007] Obtain the video to be transmitted in the first pixel format, and extract all video frames of the video to be transmitted to obtain the first video image;

[0008] The first video image is downsampled to generate a second video image in a second pixel format; wherein the first pixel format is different from the second pixel format.

[0009] The second video image is encoded, reconstructed, and upsampled to generate a third video image in the first pixel format;

[0010] Based on the first video image and the corresponding third video image, generate the first chromaticity residual image corresponding to the U component and the second chromaticity residual image corresponding to the V component.

[0011] The first chromaticity residual image and the second chromaticity residual image are reconstructed pixel by pixel to generate a first reconstructed image and a second reconstructed image;

[0012] The second video image, the first reconstructed image, and the second reconstructed image are encoded, and the generated encoded data is transmitted to the video receiving end, so that the video receiving end can generate a video in one or more pixel formats based on the second video image, the first reconstructed image, and the second reconstructed image.

[0013] Furthermore, the step of generating a first chromaticity residual image corresponding to the U component and a second chromaticity residual image corresponding to the V component based on the first video image and the corresponding third video image includes:

[0014] The first residual pixel value of each pixel U component of the first video image and the corresponding third video image is calculated using the following formula, and a first chromaticity residual image is generated based on each of the first residual pixel values.

[0015] ;

[0016] Where u' is the first residual pixel value under the U component of each pixel, u1 is the first chromaticity value under the U component of each pixel of the third video image, u ori The first chromaticity value of each pixel U component in the first video image;

[0017] The second residual pixel value of each pixel V component of the first video image and the corresponding third video image is calculated using the following formula, and a second chromaticity residual image is generated based on each second residual pixel value.

[0018] ;

[0019] Wherein, v' is the second residual pixel value under each pixel V component, v1 is the second chromaticity value under each pixel V component of the third video image, and vori is the second chromaticity value under each pixel V component of the first video image.

[0020] Further, the first chroma residual image and the second chroma residual image are pixel-recombined to generate a first recombined image and a second recombined image, including:

[0021] Based on the pixel ordinates of the first chromaticity residual image, a first image to be reconstructed is formed by combining a number of image pixels with even ordinates in the first chromaticity residual image, and a second image to be reconstructed is formed by combining a number of image pixels with odd ordinates.

[0022] Based on the pixel ordinates of the second chromaticity residual image, a number of image pixels with even-numbered ordinates in the second chromaticity residual image are used as the third image to be reconstructed, and a number of image pixels with odd-numbered ordinates are used to form the fourth image to be reconstructed.

[0023] The first reconstructed image is generated by cross-recombining each column of image pixels of the first image to be reconstructed with each column of image pixels of the third image to be reconstructed.

[0024] The second reconstructed image is generated by cross-recombining each column of image pixels of the second image to be reconstructed with each column of image pixels of the fourth image to be reconstructed.

[0025] Furthermore, the step of encoding the second video image, the first reconstructed image, and the second reconstructed image, and transmitting the generated encoded data to the video receiving end, includes:

[0026] The first reconstructed image and the second reconstructed image are encoded using a color encoder to generate a first bitstream;

[0027] The second video image is encoded using a video encoder to generate a second bitstream;

[0028] The first and second bitstreams are spliced ​​together to generate the encoded data, which is then transmitted to the video receiving end.

[0029] Furthermore, the video receiving end generates a video in one or more pixel formats based on the second video image, the first reconstructed image, and the second reconstructed image, including:

[0030] The video receiver generates video in one or more pixel formats by performing one or more of the following decoding operations:

[0031] Decode the second video image to obtain the video in the second pixel format;

[0032] The second video image, the first reconstructed image, and the second reconstructed image are decoded and reconstructed to generate a video in the first pixel format.

[0033] And decode the first reconstructed image and the second reconstructed image and reconstruct them to generate a video in the third pixel format.

[0034] Another embodiment of the present invention provides a multi-format video processing apparatus, comprising:

[0035] The first video image acquisition module is used to acquire the video to be transmitted in the first pixel format and extract all video frames of the video to be transmitted to obtain the first video image;

[0036] The second video image generation module is used to downsample the first video image to generate a second video image in a second pixel format; wherein the first pixel format is different from the second pixel format.

[0037] The third video image generation module is used to encode and reconstruct the second video image and upsample it to generate a third video image in the first pixel format;

[0038] The chromaticity residual image generation module is used to generate a first chromaticity residual image corresponding to the U component and a second chromaticity residual image corresponding to the V component based on the first video image and the corresponding third video image.

[0039] The reconstructed image generation module is used to perform pixel recombination on the first chromaticity residual image and the second chromaticity residual image to generate a first reconstructed image and a second reconstructed image;

[0040] A video image encoding module is used to encode the second video image, the first reconstructed image, and the second reconstructed image, and transmit the generated encoded data to a video receiving end, so that the video receiving end can generate a video in one or more pixel formats based on the second video image, the first reconstructed image, and the second reconstructed image.

[0041] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement a multi-format video processing method as described in the above-described embodiments of the invention.

[0042] Another embodiment of the present invention provides a medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the medium is located to perform a multi-format video processing method as described in the above-described embodiments of the invention.

[0043] The following benefits can be obtained by implementing the present invention:

[0044] This invention provides a multi-format video processing method, apparatus, terminal device, and medium. It involves downsampling and then upsampling a first-pixel format video to obtain a second video image in a low-sampling-rate second-pixel format and a third video image in a first-pixel format with incomplete information. Chromaticity residuals are calculated using the first and third video images, and then a first and second reconstructed image are generated based on the chromaticity residual images. During video encoding, the low-sampling-rate second video image, several first reconstructed images, and several second reconstructed images are encoded, and the encoded data is transmitted to a video receiving end. This allows the video receiving end to generate video in one or more pixel formats based on the second video image, the first reconstructed images, and the second reconstructed images. Therefore, this invention enables the transmitted video to meet the performance and requirements of different video receiving ends. Furthermore, during video encoding, it eliminates the need to encode all pixel formats, reducing encoding complexity. Moreover, only one encoded data line is transmitted during video transmission, minimizing the consumption of significant bandwidth resources at the transmitting end. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating a multi-format video processing method provided in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of chromaticity sampling in YUV format.

[0047] Figure 3 This is a schematic diagram of the process for generating a first reconstructed image and a second reconstructed image according to an embodiment of the present invention.

[0048] Figure 4 This is a schematic diagram illustrating the arrangement of the first and second reconstructed images provided in an embodiment of the present invention.

[0049] Figure 5 This is a schematic diagram of the video receiving end generating video in various pixel formats according to an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] See Figure 1 This is a flowchart illustrating a multi-format video processing method according to an embodiment of the present invention, comprising:

[0052] S1. Obtain the video to be transmitted in the first pixel format, and extract all video frames of the video to be transmitted to obtain the first video image;

[0053] In one embodiment of the present invention, the first pixel format is YUV444 format. See also Figure 2 This is a schematic diagram of chroma sampling in YUV format. YUV format refers to a pixel format that represents luminance and chroma components separately. Common formats include YUV420, YUV422, and YUV444. Compared to YUV420, YUV444 has a higher chroma sampling rate. In YUV444 format video images, each luminance sample is accompanied by a corresponding chroma sample, which can more accurately capture the original color information in images and videos, improving image and video quality and fidelity. It can be understood that acquiring all video frames of the video to be transmitted in YUV444 format yields several first video images, where the first video images are in YUV444 format.

[0054] S2. Downsample the first video image to generate a second video image in a second pixel format; wherein the first pixel format is different from the second pixel format.

[0055] In one embodiment of the present invention, the second pixel format is YUV420 format. It is understood that the first video image in YUV444 format can be downsampled to obtain a second video image in YUV420 format with a lower chroma sampling rate. YUV420 format is the most widely applicable pixel format. It is understood that YUV420 format is the pixel format that most video receivers can decode and use.

[0056] S3. Encode and reconstruct the second video image and upsample it to generate a third video image in the first pixel format;

[0057] In one embodiment of the present invention, a second video image in YUV420 format is encoded, reconstructed, and upsampled to generate a third video image in YUV444 format. It can be understood that the third video image is obtained by upsampling the YUV420 image after encoding loss. Therefore, compared to the corresponding first video image, the third video image is an incomplete image lacking much information.

[0058] It should be noted that step S3 simulates the upsampling operation of the YUV420 format video at the video encoding end, just like the upsampling operation at the video receiving end. Therefore, the upsampling method used in step S3 must be consistent with the upsampling method used at the video decoding end. That is, if the video encoding end uses a bilateral filtering upsampling scheme, then each video receiving end needs to use the bilateral filtering upsampling scheme; otherwise, abnormal video display problems will occur when the video receiving end performs upsampling.

[0059] S4. Based on the first video image and the corresponding third video image, generate the first chromaticity residual image corresponding to the U component and the second chromaticity residual image corresponding to the V component.

[0060] In one embodiment of the present invention, since the third video image is an incomplete image lacking a lot of information, by comparing the first video image with the third video image, the specific information missing in the third video image can be obtained, namely the first chromaticity residual image corresponding to the U component and the second chromaticity residual image corresponding to the V component.

[0061] Preferably, the generation of the first chromaticity residual image corresponding to the U component and the second chromaticity residual image corresponding to the V component is achieved through the following steps:

[0062] S41. Calculate the first residual pixel value of each pixel U component of the first video image and the corresponding third video image using the following formula, and generate a first chromaticity residual image based on each first residual pixel value.

[0063] ;

[0064] Where u' is the first residual pixel value under the U component of each pixel, u1 is the first chromaticity value under the U component of each pixel of the third video image, u ori The first chromaticity value of each pixel U component in the first video image;

[0065] S42. Calculate the second residual pixel value of each pixel V component of the first video image and the corresponding third video image using the following formula, and generate a second chromaticity residual image based on each second residual pixel value.

[0066] ;

[0067] Wherein, v' is the second residual pixel value under each pixel V component, v1 is the second chromaticity value under each pixel V component of the third video image, v ori It represents the second chromaticity value of each pixel V component in the first video image.

[0068] S5. Perform pixel recombination on the first chromaticity residual image and the second chromaticity residual image to generate a first recombined image and a second recombined image;

[0069] In one embodiment of the present invention, the first chromaticity residual image and the second chromaticity residual image are recombined into pixels according to the UV residual characteristics of each pixel format to generate a corresponding first recombined image and a second recombined image.

[0070] Preferred, see Figure 3 as well as Figure 4 The present invention provides a flowchart and an arrangement diagram for generating a first reconstructed image and a second reconstructed image according to an embodiment of the present invention, including:

[0071] S51. Based on the pixel ordinates of the first chromaticity residual image, a first image to be reconstructed is formed by combining a number of image pixels with even ordinates in the first chromaticity residual image, and a second image to be reconstructed is formed by combining a number of image pixels with odd ordinates.

[0072] S52. Based on the pixel ordinates of the second chromaticity residual image, a number of image pixels with even-numbered ordinates in the second chromaticity residual image are used as the third image to be reconstructed, and a number of image pixels with odd-numbered ordinates are used to form the fourth image to be reconstructed.

[0073] S53. Cross-reconstruct each column of image pixels of the first image to be reconstructed with each column of image pixels of the third image to be reconstructed to generate the first reconstructed image;

[0074] S54. Cross-recombine each column of image pixels of the second image to be recombined with each column of image pixels of the fourth image to be recombined to generate the second recombined image.

[0075] It is understandable that step S5 generates the corresponding first and second reconstructed images based on the characteristics of the UV residuals of each pixel format, using the first chromaticity residual image corresponding to the U component and the second chromaticity residual image corresponding to the V component of the first and third video images. Essentially, this also involves column-wise reconstructing of the pixels in the first and second chromaticity residual images to generate the corresponding first and second reconstructed images. It is also understandable that if the video receiver receives a YUV420 format video but lacks the corresponding first and second reconstructed images, then after upsampling, the video receiver will only obtain lower-quality YUV422 and YUV444 format videos.

[0076] S6. Encode the second video image, the first reconstructed image, and the second reconstructed image, and transmit the generated encoded data to the video receiving end, so that the video receiving end can generate a video in one or more pixel formats based on the second video image, the first reconstructed image, and the second reconstructed image.

[0077] In one embodiment of the present invention, the second video image in YUV420 format, the first reconstructed image, and the second reconstructed image are encoded, and the generated encoded data is transmitted to a video receiving end. After receiving the encoded data, the video receiving end can generate a video in the corresponding pixel format from the second video image, the first reconstructed image, and the second reconstructed image according to its own performance and needs.

[0078] Preferably, the second video image, the first reconstructed image, and the second reconstructed image are encoded, and the generated encoded data is transmitted to the video receiving end, including:

[0079] S61. Encode the first reconstructed image and the second reconstructed image using a color encoder to generate a first bitstream;

[0080] S62. Encode the second video image using a video encoder to generate a second bitstream;

[0081] S63. The first bitstream and the second bitstream are spliced ​​together to generate the encoded data and transmitted to the video receiving end.

[0082] It is understandable that the image to be transmitted, composed of the second video image, is encoded by a video encoder, while the first reconstructed image and the second reconstructed image are encoded by a chroma encoder. Finally, the encoded bitstreams are spliced ​​together and transmitted over the network to various video receiving ends.

[0083] Preferred, see Figure 5The video receiving end generates a video in one or more pixel formats by performing one or more of the following decoding operations:

[0084] Decode the second video image to obtain the video in the second pixel format;

[0085] The second video image, the first reconstructed image, and the second reconstructed image are decoded and reconstructed to generate a video in the first pixel format.

[0086] And decode the first reconstructed image and the second reconstructed image and reconstruct them to generate a video in the third pixel format.

[0087] In one embodiment of the present invention, the third pixel format is YUV422. It is understood that different video receivers can choose different decoding methods based on their own decoding capabilities, network bandwidth, or image quality requirements. Specifically, if the video receiver has poor downlink bandwidth or weak decoding capabilities, it can choose to decode only the second video image to obtain a YUV420 format video, or it can choose to decode and reassemble the second video image and the first reconstructed image, that is, reconstruct the YUV420 format video image with the first reconstructed image to obtain a YUV422 format video. If the video receiver has sufficient performance, it can choose to decode and reconstruct the second video image, the first reconstructed image, and the second reconstructed image, that is, decode and reconstruct the YUV420 format video image, the first reconstructed image, and the second reconstructed image to obtain a YUV444 format video. Of course, the video receiver can also switch the video format according to the needs of video quality. However, it should be noted again that the upsampling scheme used by the video receiver when switching to YUV422 or YUV444 format must be consistent with the upsampling scheme used by the video encoder for YUV420 format video images.

[0088] This invention provides a multi-format video processing method. It involves downsampling and then upsampling a first-pixel format video to obtain a second video image in a low-sampling-rate second-pixel format and a third video image in a first-pixel format with incomplete information. Chromaticity residuals are calculated using the first and third video images, and then a first and second reconstructed image are generated based on the chromaticity residual images. During video encoding, the low-sampling-rate second video image, several first reconstructed images, and several second reconstructed images are encoded, and the encoded data is transmitted to the video receiving end. This allows the video receiving end to generate video in one or more pixel formats based on the second video image, the first reconstructed images, and the second reconstructed images. Therefore, this invention enables the transmitted video to meet the performance and requirements of different video receiving ends. Furthermore, during video encoding, it eliminates the need to encode all pixel formats, reducing encoding complexity. Moreover, only one encoded data line is transmitted during video transmission, minimizing the consumption of significant bandwidth resources at the transmitting end.

[0089] See Figure 2 This is a schematic diagram of the structure of a multi-format video processing device provided in an embodiment of the present invention, comprising:

[0090] The first video image acquisition module is used to acquire the video to be transmitted in the first pixel format and extract all video frames of the video to be transmitted to obtain the first video image;

[0091] The second video image generation module is used to downsample the first video image to generate a second video image in a second pixel format; wherein the first pixel format is different from the second pixel format.

[0092] The third video image generation module is used to encode and reconstruct the second video image and upsample it to generate a third video image in the first pixel format;

[0093] The chromaticity residual image generation module is used to generate a first chromaticity residual image corresponding to the U component and a second chromaticity residual image corresponding to the V component based on the first video image and the corresponding third video image.

[0094] The reconstructed image generation module is used to perform pixel recombination on the first chromaticity residual image and the second chromaticity residual image to generate a first reconstructed image and a second reconstructed image;

[0095] A video image encoding module is used to encode the second video image, the first reconstructed image, and the second reconstructed image, and transmit the generated encoded data to a video receiving end, so that the video receiving end can generate a video in one or more pixel formats based on the second video image, the first reconstructed image, and the second reconstructed image.

[0096] This invention provides a multi-format video processing apparatus. It downsamples and then upsamples a first-pixel format video to obtain a second video image in a low-sampling-rate second-pixel format and a third video image in a first-pixel format with incomplete information. Chromaticity residuals are calculated using the first and third video images, and then a first reconstructed image and a second reconstructed image are generated based on the chromaticity residual images. During video encoding, the low-sampling-rate second video image, several first reconstructed images, and several second reconstructed images are encoded, and the encoded data is transmitted to a video receiving end. This allows the video receiving end to generate video in one or more pixel formats based on the second video image, the first reconstructed images, and the second reconstructed images. Therefore, this invention enables the transmitted video to meet the performance and requirements of different video receiving ends. Furthermore, during video encoding, it eliminates the need to encode all pixel formats, reducing encoding complexity. Moreover, only one encoded data line is transmitted during video transmission, minimizing the consumption of significant bandwidth resources at the transmitting end.

[0097] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0098] Those skilled in the art will clearly understand that, for convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0099] One embodiment of the present invention provides a terminal device, which may be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The terminal device may include, but is not limited to, a processor and a memory.

[0100] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0101] The memory can be used to store the computer program. The processor implements various functions of the terminal device by running or executing the computer program stored in the memory and calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0102] One embodiment of the present invention provides a medium, which is a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0103] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A multi-format video processing method, characterized by, The method comprises the following steps: acquiring a video in a first pixel format to be transmitted, and extracting all video frames of the video to be transmitted to obtain a first video image; down-sampling the first video image to generate a second video image in a second pixel format; wherein the first pixel format is YUV444 format, and the second pixel format is YUV420 format; encoding and reconstructing the second video image and up-sampling to generate a third video image in the first pixel format; generating a first chroma residual image corresponding to the first video image in U component and a second chroma residual image corresponding to the third video image in V component according to the first video image and the corresponding third video image; according to the pixel vertical coordinates of the first chroma residual image, grouping a plurality of image pixels with even pixel vertical coordinates in the first chroma residual image to form a first image to be reorganized, and grouping a plurality of image pixels with odd pixel vertical coordinates to form a second image to be reorganized; according to the pixel vertical coordinates of the second chroma residual image, grouping a plurality of image pixels with even pixel vertical coordinates in the second chroma residual image to form a third image to be reorganized, and grouping a plurality of image pixels with odd pixel vertical coordinates to form a fourth image to be reorganized; cross-reorganizing each column of image pixels of the first image to be reorganized with each column of image pixels of the third image to be reorganized to generate a first reorganized image; cross-reorganizing each column of image pixels of the second image to be reorganized with each column of image pixels of the fourth image to be reorganized to generate a second reorganized image; encoding the second video image, the first reorganized image and the second reorganized image, and transmitting the generated encoded data to a video receiving end, so that the video receiving end performs one or more decoding operations according to the decoding capability of the device itself, the network bandwidth or the quality requirement to generate a video in one or more pixel formats: decoding the second video image to obtain a video in the second pixel format; decoding and reorganizing the second video image, the first reorganized image and the second reorganized image to obtain a video in the first pixel format; and decoding and reorganizing the second video image and the first reorganized image to obtain a video in a third pixel format; wherein the third pixel format is YUV422 format.

2. The multi-format video processing method of claim 1, wherein, The method further comprises the following steps: calculating first residual pixel values in U component of each pixel of the first video image and the corresponding third video image according to the following formula, and generating a first chroma residual image according to the first residual pixel values; ； Wherein, the u' is the first residual pixel value under each pixel U component, u1 is the first chroma value under each pixel U component of the third video image, u ori is the first chroma value under each pixel U component of the first video image; calculating second residual pixel values in V component of each pixel of the first video image and the corresponding third video image according to the following formula, and generating a second chroma residual image according to the second residual pixel values; ； Wherein, the v' is the second residual pixel value under each pixel V component, v1 is the second chroma value under each pixel V component of the third video image, v ori is the second chroma value under each pixel V component of the first video image.

3. The multi-format video processing method of claim 2, wherein, The method further comprises the following steps: encoding the first reorganized image and the second reorganized image through a chroma encoder to generate a first code stream; encoding the second video image through a video encoder to generate a second code stream; splicing the first code stream and the second code stream to generate the encoded data and transmit the encoded data to a video receiving end.

4. A multi-format video processing apparatus, characterized by comprising: The method comprises: a first video image acquisition module configured to acquire a to-be-transmitted video in a first pixel format and extract all video frames of the to-be-transmitted video to obtain a first video image; a second video image generation module configured to down-sample the first video image to generate a second video image in a second pixel format; wherein the first pixel format is YUV444 format and the second pixel format is YUV420 format; a third video image generation module configured to perform encoding reconstruction and up-sampling on the second video image to generate a third video image in the first pixel format; a chroma residual image generation module configured to generate a first chroma residual image in U component and a second chroma residual image in V component according to the first video image and the corresponding third video image; a reorganized image generation module configured to group image pixels with even pixel vertical coordinates in the first chroma residual image as a first to-be-reorganized image and group image pixels with odd pixel vertical coordinates in the first chroma residual image as a second to-be-reorganized image according to the pixel vertical coordinates of the first chroma residual image; group image pixels with even pixel vertical coordinates in the second chroma residual image as a third to-be-reorganized image and group image pixels with odd pixel vertical coordinates in the second chroma residual image as a fourth to-be-reorganized image according to the pixel vertical coordinates of the second chroma residual image; and cross-reorganize each column of image pixels of the first to-be-reorganized image with each column of image pixels of the third to-be-reorganized image to generate a first reorganized image, and cross-reorganize each column of image pixels of the second to-be-reorganized image with each column of image pixels of the fourth to-be-reorganized image to generate a second reorganized image; a video image encoding module configured to encode the second video image, the first reorganized image, and the second reorganized image, and transmit the generated encoded data to a video receiving end, so that the video receiving end performs one or more decoding operations according to the decoding capability of the video receiving end, network bandwidth, or picture quality requirement to generate a video in one or more pixel formats: decoding the second video image to obtain a video in the second pixel format; decoding and reorganizing the second video image, the first reorganized image, and the second reorganized image to obtain a video in the first pixel format; and decoding and reorganizing the second video image and the first reorganized image to obtain a video in a third pixel format; wherein the third pixel format is YUV422 format.

5. A terminal device, characterized by, The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and the processor implements a multi-format video processing method according to any one of claims 1 to 3 when executing the computer program.

6. A medium characterized by, Computer program product comprising a stored computer program which, when executed by a device in which the medium is located, controls the device to perform a multi-format video processing method as claimed in any one of claims 1 to 3.

Images