A method, device and apparatus for processing a video image

Abstract

This invention discloses a method, apparatus, and device for processing video images. The method includes: sequentially receiving video images; for each first pixel block in the video image, based on a first dithering matrix corresponding to the first pixel block, dispersing the energy of a target pixel located at the center of the first pixel block to pixels at other locations in the first pixel block to obtain a first target video image; for each second pixel block in the first target video image, based on a second dithering matrix corresponding to the second pixel block, dispersing the energy of pixels in a third pixel block to pixels corresponding to the second pixel block to obtain a second target video image; wherein the third pixel block is located in the first target video image of the previous frame, and a target video stream is generated based on the obtained second target video image. By first performing spatial dithering processing on the video image and then temporal dithering processing, the display effect of the video image can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method, apparatus, and device for processing video images. Background Technology

[0002] In the field of image processing, image dithering algorithms reduce color quantization errors by introducing controllable noise, especially when converting high color depth images (such as 24-bit true color) to low color depth images (such as 8-bit indexed color or black-and-white binary images), effectively avoiding color banding artifacts. Its core purpose is to visually simulate more colors or smooth transitions.

[0003] Error diffusion is an advanced implementation of image dithering algorithms. When converting from high resolution to low resolution, the quantization error of an image pixel is diffused to neighboring pixels through the image dithering algorithm, thereby preserving more image details and visually presenting more image color depth and image display.

[0004] In layman's terms, image dithering algorithms can improve the visual quality of an image by simulating more colors or grayscale levels when the color or grayscale levels are limited. For example, ... Figure 1a and Figure 1b As shown, Figure 1a The image in the image is a 128-bit grayscale image, which would normally require 8 bits of depth, or 256 grayscale levels. However, through an image dithering algorithm, the same visual effect can be simulated using only one bit of depth. Figure 1b As shown.

[0005] Existing image dithering algorithms typically involve spatially dithering the energy of pixels in a static image. However, since video images are refreshed frame by frame over time, visual errors are not merely spatial. Therefore, using existing image dithering algorithms to process video images can negatively impact the display quality. Summary of the Invention

[0006] This invention provides a method, device, and apparatus for processing video images, which solves the problem of poor display effect of video images when using existing image dithering algorithms to process video images in the prior art.

[0007] In a first aspect, this application provides a method for processing video images, the method comprising:

[0008] Receive video images of each frame in the video stream sequentially;

[0009] For each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixels at other positions in the first pixel block to obtain the first target video image of the current frame;

[0010] For each second pixel block in the first target video image of the current frame, based on the second dither matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame.

[0011] A target video stream is generated based on the second target video image obtained in each frame in sequence.

[0012] In one possible implementation, the step of dispersing the energy of the target pixel located at the center of the first pixel block to pixels at other locations in the first pixel block based on the first jitter matrix corresponding to the first pixel block, to obtain the first target video image of the current frame, includes:

[0013] The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value;

[0014] The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

[0015] In one possible implementation, the step of distributing the energy of pixels in the third pixel block to the corresponding pixels in the second pixel block based on the second dithering matrix corresponding to the second pixel block to obtain the second target video image of the current frame includes:

[0016] The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value;

[0017] Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

[0018] In one possible implementation, obtaining the second target video image of the current frame based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block includes:

[0019] For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value;

[0020] The image formed by the obtained third pixel value is used as the second target video image of the current frame.

[0021] In one possible implementation, generating the target video stream based on each sequentially obtained frame of the second target video image includes:

[0022] For each frame of the second target video image, a frame synchronization signal, a line synchronization signal, and a data validity flag signal are added to the video signal corresponding to the second target video image to generate the target video stream.

[0023] In one possible implementation, the first jitter matrix corresponding to the first pixel block is implemented through a register transfer stage, dispersing the energy of the target pixel located at the center of the first pixel block to pixels at other locations within the first pixel block, thereby obtaining the first target video image of the current frame; and / or

[0024] The register transfer stage is used to implement the second dithering matrix corresponding to the second pixel block, which distributes the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block, thereby obtaining the second target video image of the current frame.

[0025] Secondly, this application also provides a video image processing device, including at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor performs the following steps:

[0026] Receive video images of each frame in the video stream sequentially;

[0027] For each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixels at other positions in the first pixel block to obtain the first target video image of the current frame;

[0028] For each second pixel block in the first target video image of the current frame, based on the second dither matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame.

[0029] A target video stream is generated based on the second target video image obtained in each frame in sequence.

[0030] In one possible implementation, the processor is specifically used for:

[0031] The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value;

[0032] The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

[0033] In one possible implementation, the processor is specifically used for:

[0034] The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value;

[0035] Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

[0036] In one possible implementation, the processor is specifically used for:

[0037] For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value;

[0038] The image formed by the obtained third pixel value is used as the second target video image of the current frame.

[0039] In one possible implementation, the processor is specifically used for:

[0040] For each frame of the second target video image, a frame synchronization signal, a line synchronization signal, and a data validity flag signal are added to the video signal corresponding to the second target video image to generate the target video stream.

[0041] In one possible implementation, the first jitter matrix corresponding to the first pixel block is implemented through a register transfer stage, dispersing the energy of the target pixel located at the center of the first pixel block to pixels at other locations within the first pixel block, thereby obtaining the first target video image of the current frame; and / or

[0042] The register transfer stage is used to implement the second dithering matrix corresponding to the second pixel block, which distributes the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block, thereby obtaining the second target video image of the current frame.

[0043] Thirdly, this application also provides a video image processing apparatus, comprising:

[0044] The spatial dithering module is used to sequentially receive video images of each frame in the video stream; for each first pixel block in the video image of the current frame, based on the first dithering matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixel points at other positions in the first pixel block to obtain the first target video image of the current frame.

[0045] The time jitter module is used to distribute the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block for each second pixel block in the first target video image of the current frame, based on the second jitter matrix corresponding to the second pixel block, to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame. A target video stream is generated based on the second target video images of each frame obtained in sequence.

[0046] In one possible implementation, the spatial jitter module is specifically used for:

[0047] The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value;

[0048] The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

[0049] In one possible implementation, the time jitter module is specifically used for:

[0050] The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value;

[0051] Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

[0052] In one possible implementation, the time jitter module is specifically used for:

[0053] For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value;

[0054] The image formed by the obtained third pixel value is used as the second target video image of the current frame.

[0055] In one possible implementation, the time jitter module is specifically used for:

[0056] For each frame of the second target video image, a frame synchronization signal, a line synchronization signal, and a data validity flag signal are added to the video signal corresponding to the second target video image to generate the target video stream.

[0057] In one possible implementation, the first jitter matrix corresponding to the first pixel block is implemented through a register transfer stage, dispersing the energy of the target pixel located at the center of the first pixel block to pixels at other locations within the first pixel block, thereby obtaining the first target video image of the current frame; and / or

[0058] The register transfer stage is used to implement the second dithering matrix corresponding to the second pixel block, which distributes the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block, thereby obtaining the second target video image of the current frame.

[0059] The beneficial effects of this invention are as follows:

[0060] This application provides a video image processing method, apparatus, and device. The method includes: firstly, sequentially receiving video images of each frame in a video stream; then, for each first pixel block in the current frame's video image, dispersing the energy of a target pixel located at the center of the first pixel block to pixels at other positions in the first pixel block based on a first dithering matrix corresponding to the first pixel block, to obtain a first target video image of the current frame; then, for each second pixel block in the current frame's first target video image, dispersing the energy of pixels in a third pixel block to pixels corresponding to the second pixel block based on a second dithering matrix corresponding to the second pixel block, to obtain a second target video image of the current frame, wherein the third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame; finally, generating a target video stream based on the second target video images of each frame obtained sequentially. First, based on the first dithering matrix, the energy of the target pixel at the center of the first pixel block in the video image is distributed to other pixels in the first pixel block to obtain the first target video image, which is spatial dithering processing of the video image. Then, based on the second dithering matrix, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block to obtain the second target video image. The third pixel block is located in the first target video image of the previous frame, and the second pixel block is located in the first target video image of the current frame, which is temporal dithering processing of the video image, thereby improving the display effect of the video image. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1a A schematic diagram of an 8-bit depth image provided in an embodiment of this application;

[0063] Figure 1b A schematic diagram of an image with a 1-bit depth provided in an embodiment of this application;

[0064] Figure 2 A flowchart illustrating a video image processing method provided in an embodiment of this application;

[0065] Figure 3 A schematic diagram of pixels in a pixel block in a video image provided in an embodiment of this application;

[0066] Figure 4 A schematic diagram illustrating pixel values ​​of a pixel block in a video image, provided as an embodiment of this application;

[0067] Figure 5 A schematic diagram of the first pixel value of a pixel block in an intermediate video image provided in an embodiment of this application;

[0068] Figure 6 A schematic diagram of the first pixel value of a pixel block in another intermediate video image provided in an embodiment of this application;

[0069] Figure 7 A schematic diagram illustrating the third pixel value of a pixel block in a target video image, provided as an embodiment of this application;

[0070] Figure 8 This is a schematic diagram of a video stream display format provided in an embodiment of this application;

[0071] Figure 9 A schematic diagram of the structure of a video image processing device provided in an embodiment of this application;

[0072] Figure 10 A schematic diagram of the structure of a video image processing apparatus provided in an embodiment of this application;

[0073] Figure 11 This is a schematic diagram of the structure of a spatial jitter module provided in an embodiment of this application;

[0074] Figure 12 This is a schematic diagram of the structure of a time jitter module provided in an embodiment of this application. Detailed Implementation

[0075] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0076] The image jitter algorithms provided in related technologies are usually designed for static images; there is currently no suitable image jitter algorithm for video images.

[0077] This application provides a video image processing method, apparatus, and device for jittering video images to improve their display quality. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0078] like Figure 2The diagram shown is a flowchart illustrating a video image processing method provided in an embodiment of this application. Specifically, the method includes the following steps:

[0079] S201. Receive video images of each frame in the video stream in sequence;

[0080] S202. For each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixel points at other positions in the first pixel block to obtain the first target video image of the current frame.

[0081] S203. For each second pixel block in the first target video image of the current frame, based on the second dithering matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame.

[0082] S204 generates a target video stream based on the second target video image obtained in each frame in sequence.

[0083] This application provides a video image processing method. First, it sequentially receives video images from each frame of a video stream. Then, for each first pixel block in the current frame's video image, based on a first dithering matrix corresponding to that first pixel block, it distributes the energy of the target pixel located at the center of the first pixel block to pixels at other positions within the first pixel block, obtaining a first target video image for the current frame. Next, for each second pixel block in the first target video image of the current frame, based on a second dithering matrix corresponding to that second pixel block, it distributes the energy of pixels in a third pixel block to pixels corresponding to that second pixel block, obtaining another first target video image for the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame. Finally, it generates a target video stream based on the second target video images obtained sequentially for each frame. By performing spatial dithering processing on the video image first, followed by temporal dithering processing, the display effect of the video image can be improved.

[0084] In specific implementations, obtaining the first target video image involves spatial jitter processing, and obtaining the second target video image from the first target video image involves temporal jitter processing. Since video images are refreshed frame by frame over time, visual errors are not only spatial errors but also temporal visual errors caused by frame refreshes. The video image processing method provided in this application performs both spatial and temporal jitter processing on the video image, which improves the visual display effect compared to performing only spatial jitter processing.

[0085] It should be noted that, in the embodiments of this application, the energy of a pixel is used to describe the uniformity of the pixel grayscale distribution in the image, measure the randomness contained in the image, and characterize the complexity of the image.

[0086] In one embodiment, before performing spatial jitter processing on the video image of the current frame, the video image is first divided into pixel blocks, for example, the video image is divided into multiple 3×3 pixel blocks, and then for each pixel block, a first jitter matrix and a second jitter matrix are preset. The first jitter matrix and the second jitter matrix can be obtained based on a large number of experiments and the experience of designers.

[0087] In the first jitter matrix, the ratio corresponding to the target pixel in the first pixel block of the video image can be less than or equal to 1. Similarly, the ratio corresponding to other pixels in the first pixel block besides the target pixel can also be less than 1. This is merely an example, and the embodiments of this application do not limit this. Similarly, the ratio values ​​in the second jitter matrix are not limited in this embodiment.

[0088] The first jitter matrix corresponds to the first pixel block of the video image. For example, if the first pixel block is a 3×3 pixel block, then the first jitter matrix is ​​a 3×3 matrix. Or, if the first pixel block is a 5×5 pixel block, then the first jitter matrix is ​​a 5×5 matrix.

[0089] The second jitter matrix corresponds to the second pixel block of the first target video image. For example, if the second pixel block is a 3×3 pixel block, then the second jitter matrix is ​​a 3×3 matrix. Or, if the second pixel block is a 5×5 pixel block, then the second jitter matrix is ​​a 5×5 matrix.

[0090] It should be noted that the size of the first pixel block and the size of the second pixel block can be the same or different.

[0091] After receiving video images from each frame of the video stream, the video images can be buffered for subsequent spatial jitter processing. During spatial jitter processing, for each first pixel block in the current frame's video image, a first jitter matrix corresponding to that first pixel block is first determined. Then, the pixel value of each pixel in the first pixel block is multiplied by the corresponding proportional value in the determined first jitter matrix to obtain the first pixel value. The image composed of the obtained first pixel values ​​is used as the first target video image of the current frame.

[0092] In one embodiment, a first correspondence between the position of the first pixel block and the first dithering matrix can be preset. In practice, the position of the first pixel block is first determined, and then the first dithering matrix corresponding to the position is determined from the first correspondence. The first dithering matrix is ​​used as the dithering matrix corresponding to the first pixel block.

[0093] like Figure 3 The diagram shown is a schematic representation of the pixels in a video image according to an embodiment of this application. Figure 3 In this example, the video image is divided into 12 pixel blocks, each containing 9 pixels, meaning each pixel block is a 3×3 pixel block. (For...) Figure 3 For each first pixel block in the image, a 3×3 first dithering matrix is ​​preset. For example, the first dithering matrix corresponding to the first pixel block in the upper left corner is... .

[0094] Figure 3 In the image, the first pixel block in the upper left corner includes pixel 1, pixel 2, pixel 3, pixel 4, pixel 5, pixel 6, pixel 7, pixel 8, and pixel 9. Pixel 5 is the target pixel located at the center of the first pixel block, while pixel 1, pixel 2, pixel 3, pixel 4, pixel 6, pixel 7, pixel 8, and pixel 9 are pixels located at other positions in the first pixel block.

[0095] like Figure 4 The diagram shown is a schematic representation of pixel values ​​in a video image according to an embodiment of this application. Figure 4 In the image, pixel 1 corresponds to pixel value a11, pixel 2 corresponds to pixel value a12, pixel 3 corresponds to pixel value a13, pixel 4 corresponds to pixel value a21, pixel 5 corresponds to pixel value a22, pixel 6 corresponds to pixel value a23, pixel 7 corresponds to pixel value a31, pixel 8 corresponds to pixel value a32, and pixel 9 corresponds to pixel value a33.

[0096] Reference Figure 3 and Figure 4In the first pixel block, the pixel value a11 of pixel 1 corresponds to the ratio value b11 in the first jitter matrix; the pixel value a12 of pixel 2 in the first pixel block corresponds to the ratio value b12 in the first jitter matrix; the pixel value a13 of pixel 3 in the first pixel block corresponds to the ratio value b13 in the first jitter matrix; the pixel value a21 of pixel 4 in the first pixel block corresponds to the ratio value b21 in the first jitter matrix; the pixel value a22 of pixel 5 in the first pixel block corresponds to the ratio value b22 in the first jitter matrix; the pixel value a23 of pixel 6 in the first pixel block corresponds to the ratio value b23 in the first jitter matrix; the pixel value a31 of pixel 7 in the first pixel block corresponds to the ratio value b31 in the first jitter matrix; the pixel value a32 of pixel 8 in the first pixel block corresponds to the ratio value b32 in the first jitter matrix; and the pixel value a33 of pixel 9 in the first pixel block corresponds to the ratio value b33 in the first jitter matrix.

[0097] like Figure 5 The diagram shown is a schematic representation of the first pixel value of a pixel block in a first target video image according to an embodiment of this application. (Refer to...) Figure 5 The pixel value a11 of pixel 1 is multiplied by the ratio value b11 in the first dithering matrix to obtain the first pixel value c11, and this first pixel value c11 is used as the pixel value at pixel 1; the pixel value a12 is multiplied by the ratio value b12 in the first dithering matrix to obtain the first pixel value c12, and this first pixel value c12 is used as the pixel value at pixel 2; the pixel value a13 is multiplied by the ratio value b13 in the first dithering matrix to obtain the first pixel value c13, and this first pixel value c13 is used as the pixel value at pixel 3; the pixel value a21 is multiplied by the ratio value b21 in the first dithering matrix to obtain the first pixel value c21, and this first pixel value c21 is used as the pixel value at pixel 4; the pixel value a22 is multiplied by the ratio value b22 in the first dithering matrix... The first pixel value c22 is obtained and used as the pixel value at pixel 5. The pixel value a23 is multiplied by the ratio value b23 in the first dithering matrix to obtain the first pixel value c23, which is used as the pixel value at pixel 6. The pixel value a31 is multiplied by the ratio value b31 in the first dithering matrix to obtain the first pixel value c31, which is used as the pixel value at pixel 7. The pixel value a32 is multiplied by the ratio value b32 in the first dithering matrix to obtain the first pixel value c32, which is used as the pixel value at pixel 8. The pixel value a33 is multiplied by the ratio value b33 in the first dithering matrix to obtain the first pixel value c33, which is used as the pixel value at pixel 9.

[0098] After updating all pixels in all pixel blocks of the video image to the first pixel value using the above method, the resulting image is the first target video image.

[0099] After obtaining the first target video image, the spatial jitter processing of the video image ends, and the temporal jitter processing begins. Specifically, for each second pixel block in the first target video image of the current frame, based on the second jitter matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame.

[0100] In this embodiment, temporal jitter processing is performed on the first target video image of the current frame based on the second jitter matrix corresponding to the second pixel block in the first target video image of the current frame and the pixel values ​​corresponding to the pixels in the third pixel block of the first target video image of the previous frame. Since the first target video image of the previous frame is used when performing temporal jitter processing on the first target video image of the current frame, the received first target video image can be buffered first, and then temporal jitter processing can be performed on the first target video image of the current frame based on the second jitter matrix and the buffered first target video image of the previous frame.

[0101] It should be noted that if the first video image of the current frame is the first video image of the first frame, then there is no first video image of the previous frame. In this case, no jitter processing is performed on the first video image of the current frame, that is, no time jitter processing is performed on the first video image of the first frame.

[0102] In one embodiment, for each second pixel block in the first target video image of the current frame, a second dithering matrix corresponding to the second pixel block is first determined. Then, the pixel value of each pixel in the third pixel block is multiplied by the corresponding proportional value in the second dithering matrix to obtain a second pixel value. For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain a third pixel value. The image composed of the obtained third pixel values ​​is used as the second target video image of the current frame.

[0103] In one embodiment, a second correspondence between the position of the second pixel block and the second dithering matrix can be preset. In practice, the position of the second pixel block is first determined, and then the second dithering matrix corresponding to the position is determined from the second correspondence. The determined second dithering matrix is ​​used as the dithering matrix corresponding to the second pixel block.

[0104] In this embodiment, the first target video image of the previous frame is the video image of the frame preceding the first target video image of the current frame. In a specific implementation, after the first target video images are generated sequentially, the sequentially generated first target video images are cached. Then, the position of the second pixel block in the first target video image of the current frame is determined. A third pixel block with the same position as the second pixel block is determined from the cached first target video image of the previous frame. Based on the determined position of the second pixel block, a second dithering matrix corresponding to the position is determined from a second correspondence. The determined second dithering matrix is ​​used as the dithering matrix corresponding to the second pixel block.

[0105] After determining the third pixel block and the second dithering matrix, the pixel value of each pixel in the third pixel block is multiplied by the corresponding proportional value in the second dithering matrix to obtain the second pixel value. For each second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value.

[0106] like Figure 6 The diagram shown is a schematic representation of a pixel block in a first target video image provided in an embodiment of this application. For example, Figure 6 The first target video image in the current frame is the first target video image from the previous frame. Figure 5 The first target video image in the current frame is the first target video image. Figure 5 and Figure 6 Each of the blocks contains 12 3×3 pixel blocks.

[0107] For example, with Figure 5 The second jitter matrix corresponding to the second pixel block in the upper left corner of the first target video image in the current frame is: The first row of the second pixel block, from left to right, consists of pixel 1, pixel 2, and pixel 3; the second row, from left to right, consists of pixel 4, pixel 5, and pixel 6; and the third row, from left to right, consists of pixel 7, pixel 8, and pixel 9. Pixel 1 in the second pixel block corresponds to the ratio d11 in the second jitter matrix; Pixel 2 in the second pixel block corresponds to the ratio d12 in the second jitter matrix; Pixel 3 in the second pixel block corresponds to the ratio d13 in the second jitter matrix; Pixel 4 in the second pixel block corresponds to the ratio d21 in the second jitter matrix; Pixel 5 in the second pixel block corresponds to the ratio d22 in the second jitter matrix; Pixel 6 in the second pixel block corresponds to the ratio d23 in the second jitter matrix; Pixel 7 in the second pixel block corresponds to the ratio d31 in the second jitter matrix; Pixel 8 in the second pixel block corresponds to the ratio d32 in the second jitter matrix; and Pixel 9 in the second pixel block corresponds to the ratio d33 in the second jitter matrix.

[0108] Since there is a one-to-one correspondence between the pixels in the second pixel block and the pixels in the third pixel block, pixel 1 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix, that is, the pixel value c14 of pixel 1 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix; pixel 2 in the third pixel block corresponds to the ratio value d12 in the second dithering matrix, that is, the pixel value c15 of pixel 2 in the third pixel block corresponds to the ratio value d12 in the second dithering matrix; pixel 3 in the third pixel block corresponds to the ratio value d13 in the second dithering matrix, that is, the pixel value c16 of pixel 3 in the third pixel block corresponds to the ratio value d13 in the second dithering matrix; pixel 4 in the third pixel block corresponds to the ratio value d21 in the second dithering matrix, that is, the pixel value c24 of pixel 4 in the third pixel block corresponds to the ratio value d21 in the second dithering matrix; pixel 5 in the third pixel block corresponds to the ratio value d21 in the second dithering matrix; pixel 6 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix; pixel 7 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix; pixel 8 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix; pixel 9 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix; pixel 10 in the third pixel block corresponds to the ratio value d11 in the second dithering matrix; pixel 11 in the third pixel block corresponds to the ratio value d12 in the second dithering matrix; pixel 11 in the third pixel block corresponds to the ratio value d12 in the second dithering matrix; pixel 11 in the third pixel block corresponds to the ratio value d13 in the second dithering matrix; pixel 11 in the third pixel block corresponds to the ratio value The proportional value d22 in the motion matrix corresponds to the pixel value c25 of pixel 5 in the third pixel block, which corresponds to the proportional value d22 in the second jitter matrix; the pixel value c26 of pixel 6 in the third pixel block corresponds to the proportional value d23 in the second jitter matrix; the pixel value c34 of pixel 7 in the third pixel block corresponds to the proportional value d31 in the second jitter matrix; the pixel value c35 of pixel 8 in the third pixel block corresponds to the proportional value d32 in the second jitter matrix; the pixel value c36 of pixel 9 in the third pixel block corresponds to the proportional value d33 in the second jitter matrix.

[0109] Multiply the pixel value of each pixel in the third pixel block by the corresponding proportional value in the second dithering matrix to obtain the second pixel value, that is... Figure 6 Multiplying the pixel value c14 of pixel 1 in the third pixel block by the ratio d11 of the second dithering matrix yields the second pixel value f11. Figure 6 Multiplying the pixel value c15 of pixel 2 in the third pixel block by the ratio d12 in the second dithering matrix yields the second pixel value f12. Figure 6Multiplying the pixel value c16 of pixel 3 in the third pixel block by the ratio d13 in the second dithering matrix yields the second pixel value f13. Multiplying the pixel value c24 of pixel 4 in the third pixel block by the ratio d21 in the second dithering matrix yields the second pixel value f21. Multiplying the pixel value c25 of pixel 5 in the third pixel block by the ratio d22 in the second dithering matrix yields the second pixel value f22. Multiplying the pixel value c26 of pixel 6 in the third pixel block by the ratio d22 in the second dithering matrix yields the second pixel value f22. Multiply the value d23 in the dithering matrix to get the second pixel value f23. Multiply the pixel value c34 of pixel 7 in the third pixel block with the ratio value d31 in the second dithering matrix to get the second pixel value f31. Multiply the pixel value c35 of pixel 8 in the third pixel block with the ratio value d32 in the second dithering matrix to get the second pixel value f32. Multiply the pixel value c36 of pixel 9 in the third pixel block with the ratio value d33 in the second dithering matrix to get the second pixel value f33.

[0110] Superimpose each obtained second pixel value onto Figure 5 Specifically, in the second pixel block at the top left corner, the pixel value of the corresponding pixel is added to the pixel value of pixel 1 in the second pixel block to obtain the third pixel value g11. Figure 7 As shown, the third pixel value g11 is used as the pixel value of pixel 1 in the second pixel block; the second pixel value f12 is added to the pixel value c12 of pixel 2 in the second pixel block to obtain the third pixel value g12, as follows. Figure 7 As shown, the third pixel value g12 is used as the pixel value of pixel 2 in the second pixel block; the second pixel value f13 is added to the pixel value c13 of pixel 3 in the second pixel block to obtain the third pixel value g13, as follows. Figure 7 As shown, the third pixel value g13 is used as the pixel value of pixel 3 in the second pixel block; the second pixel value f21 is added to the pixel value c21 of pixel 4 in the second pixel block to obtain the third pixel value g21, as follows. Figure 7 As shown, the third pixel value g21 is used as the pixel value of pixel 4 in the second pixel block; the second pixel value f22 is added to the pixel value c22 of pixel 5 in the second pixel block to obtain the third pixel value g22, as shown. Figure 7 As shown, the third pixel value g22 is used as the pixel value of pixel 5 in the second pixel block; the second pixel value f23 is added to the pixel value c23 of pixel 6 in the second pixel block to obtain the third pixel value g23, as shown. Figure 7 As shown, the third pixel value g23 is used as the pixel value of pixel 6 in the second pixel block; the second pixel value f31 is added to the pixel value c31 of pixel 7 in the second pixel block to obtain the third pixel value g31, as follows. Figure 7As shown, the third pixel value g31 is used as the pixel value of pixel 7 in the second pixel block; the second pixel value f32 is added to the pixel value c32 of pixel 8 in the second pixel block to obtain the third pixel value g32, as follows. Figure 7 As shown, the third pixel value g32 is used as the pixel value of pixel 8 in the second pixel block; the second pixel value f33 is added to the pixel value c33 of pixel 9 in the second pixel block to obtain the third pixel value g33, as follows. Figure 7 As shown, the third pixel value g33 is used as the pixel value of pixel 9 in the second pixel block.

[0111] After updating all pixels in all pixel blocks of the first target video image in the current frame to the third pixel value using the above method, the resulting image is the second target video image of the current frame.

[0112] like Figure 8 The diagram shown is a schematic representation of a video stream display format provided in an embodiment of this application. Figure 8 In this context, VSYNC is the frame synchronization signal, HSYNC is the line synchronization signal, VDEN is the data valid flag signal, VD is the video data signal (RGB or YUV), and VCLK is the clock signal. The two rising edges of the frame synchronization signal VSYNC represent one frame of video signal, which includes multiple lines of video signal. The two rising edges of the line synchronization signal HSYNC represent one line of video signal.

[0113] In this embodiment of the application, both spatial jitter processing and temporal jitter processing of the video image are jitter processing of the video data signal in the video signal. After the jitter processing is completed, in order to ensure the output format of the video, that is, to ensure that the video can be displayed normally after output, it is necessary to add frame synchronization signal, line synchronization signal and data validity flag signal to the video data signal corresponding to the second target video image of each frame.

[0114] That is, after obtaining the second target video image of each frame, a frame synchronization signal, a line synchronization signal, and a data validity flag signal are added to the video data signal corresponding to the second target video image to ensure that the video can be displayed normally after output.

[0115] In one embodiment, the energy of the target pixel located at the center of the first pixel block is distributed to pixels at other locations in the first pixel block based on a first jitter matrix corresponding to the first pixel block using RTL (Register-Transfer Level), thereby obtaining the first target video image of the current frame; and / or

[0116] By implementing RTL based on the second jitter matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block, thus obtaining the second target video image of the current frame.

[0117] In this embodiment, spatial jitter processing of video images is achieved through RTL, and / or temporal jitter processing of video images is achieved through RTL. RTL is an abstract description in digital circuit design. Using RTL to achieve jitter processing of video images is fast and has low power consumption. It can be applied to real-time processing of high frame rate video streams. It is a hardware algorithm processing method. Compared with various processor core-based software processing methods, it has advantages such as high real-time performance and low resource requirements, and does not require external DDR-SDRAM cache.

[0118] Based on the same inventive concept, this application also provides a video image processing device. The implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0119] like Figure 9 The diagram shown is a structural schematic of a video image processing device according to an embodiment of this application. The device includes at least one processor 901 and at least one memory 902, wherein the memory stores a computer program. When the program is executed by the processor, the processor performs the following steps:

[0120] Receive video images of each frame in the video stream sequentially;

[0121] For each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixels at other positions in the first pixel block to obtain the first target video image of the current frame;

[0122] For each second pixel block in the first target video image of the current frame, based on the second dither matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame.

[0123] Each frame of the second target video image obtained in sequence is used as the target video stream.

[0124] In one embodiment, processor 901 is specifically used for:

[0125] The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value;

[0126] The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

[0127] In one embodiment, processor 901 is specifically used for:

[0128] The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value;

[0129] Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

[0130] In one embodiment, processor 901 is specifically used for:

[0131] For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value;

[0132] The image formed by the obtained third pixel value is used as the second target video image of the current frame.

[0133] In one embodiment, processor 901 is specifically used for:

[0134] For each frame of the second target video image, a frame synchronization signal, a line synchronization signal, and a data validity flag signal are added to the video signal corresponding to the second target video image to generate the target video stream.

[0135] In one embodiment, after obtaining the intermediate video image and before obtaining the target video image, the processor 901 is further configured to:

[0136] Each frame of the first target video image is cached.

[0137] In one embodiment, the first dithering matrix corresponding to the first pixel block is implemented using RTL to distribute the energy of the target pixel located at the center of the first pixel block to pixels at other locations in the first pixel block, thereby obtaining the first target video image of the current frame; and / or

[0138] The second target video image of the current frame is obtained by distributing the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block using the second dither matrix based on the second pixel block using RTL.

[0139] Based on the same inventive concept, this application also provides a video image processing apparatus. The implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be described again.

[0140] like Figure 10 The diagram shown is a structural schematic of a video image processing apparatus provided in an embodiment of this application. The apparatus includes:

[0141] The spatial jitter module 1001 is used to sequentially receive video images of each frame in the video stream; for each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixel points at other positions in the first pixel block to obtain the first target video image of the current frame.

[0142] The time jitter module 1002 is used to distribute the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block for each second pixel block in the first target video image of the current frame, based on the second jitter matrix corresponding to the second pixel block, to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame. A target video stream is generated based on the second target video images of each frame obtained in sequence.

[0143] In one embodiment, the spatial jitter module 1001 is specifically used for:

[0144] The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value;

[0145] The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

[0146] In this embodiment of the application, the first target video images generated sequentially constitute an intermediate video stream.

[0147] In one embodiment, the time jitter module 1002 is specifically used for:

[0148] The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value;

[0149] Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

[0150] In one embodiment, the time jitter module 1002 is specifically used for:

[0151] For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value;

[0152] The image formed by the obtained third pixel value is used as the second target video image of the current frame.

[0153] In one embodiment, the time jitter module 1002 includes a frame buffer submodule;

[0154] The frame buffer submodule is used to buffer each frame of the first target video image obtained.

[0155] In one embodiment, the first dithering matrix corresponding to the first pixel block is implemented using RTL to distribute the energy of the target pixel located at the center of the first pixel block to pixels at other locations in the first pixel block, thereby obtaining the first target video image of the current frame; and / or

[0156] The second target video image of the current frame is obtained by distributing the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block using the second dither matrix based on the second pixel block using RTL.

[0157] In specific implementation, such as Figure 11 As shown, the spatial jitter module 1001 includes a data buffer submodule, a first jitter matrix lookup submodule, a spatial jitter processing submodule, and a first output control submodule. The data buffer submodule buffers the input video stream, that is, it buffers the video images of each frame in the video stream for subsequent processing. The first jitter matrix lookup submodule determines each first jitter matrix corresponding to each first pixel block in the video image using a pre-set first jitter matrix. The spatial jitter processing submodule, based on the determined first jitter matrix, distributes the energy of the pixel to surrounding pixels. The first output control submodule outputs the data from the spatial jitter processing submodule according to the format of the video stream.

[0158] like Figure 12As shown, the time jitter module 1002 includes a frame buffer submodule, a second jitter matrix lookup submodule, a pixel overlay calculation submodule, and a second output control submodule. The frame buffer submodule buffers the first target video image using a frame buffer, i.e., it buffers the video data of each frame in the intermediate video stream for processing by lower-level modules. In a specific implementation, this can be achieved using SRAM. The second jitter matrix lookup submodule determines the second jitter matrix corresponding to the second pixel block in the first target video image using a pre-set second jitter matrix. The pixel overlay calculation submodule, based on the second jitter matrix, distributes the energy from the first target video image in the previous frame to the first target video image in the current frame, obtaining the second target video image. The second output control submodule places the frame synchronization signal, line synchronization signal, and data validity flag signal into the video signal corresponding to the second target video image to generate the target video stream, ensuring the output format of the target video stream.

[0159] The video image processing method, device, and apparatus provided in this application spread the quantization error of the previous frame of the video image to the current frame of the video image, that is, retain the energy of some pixels of Frame(n) in Frame(n+1). In this way, even if the gray level or color depth of the image is reduced in the continuous visual effect, more image details can be retained, which saves a lot of bandwidth in the transmission of video images and also makes the quality of video images visually lossless or enhanced.

[0160] The present application has been described above with reference to block diagrams and / or flowcharts illustrating methods, apparatus (systems), and / or computer program products according to embodiments of the present application. It should be understood that a block of a block diagram and / or flowchart, as well as combinations of blocks of block diagrams and / or flowcharts, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, and / or other programmable data processing means to produce a machine such that the instructions, executable via the computer processor and / or other programmable data processing means, create methods for implementing the functions / actions specified in the blocks of the block diagrams and / or flowcharts.

[0161] Accordingly, this application can also be implemented using hardware and / or software (including firmware, resident software, microcode, etc.). Furthermore, this application can take the form of a computer program product on a computer-usable or computer-readable storage medium, having computer-usable or computer-readable program code implemented in the medium for use by or in conjunction with an instruction execution system. In the context of this application, a computer-usable or computer-readable medium can be any medium that can contain, store, communicate, transmit, or deliver a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0162] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for processing video images, characterized in that, The method includes: Receive video images of each frame in the video stream sequentially; For each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixels at other positions in the first pixel block to obtain the first target video image of the current frame; For each second pixel block in the first target video image of the current frame, based on the second dither matrix corresponding to the second pixel block, the energy of the pixels in the third pixel block is distributed to the corresponding pixels in the second pixel block to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame. A target video stream is generated based on the second target video image obtained in each frame in sequence.

2. The method according to claim 1, characterized in that, The step of dispersing the energy of the target pixel located at the center of the first pixel block to pixels at other locations in the first pixel block based on the first jitter matrix corresponding to the first pixel block, to obtain the first target video image of the current frame, includes: The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value; The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

3. The method according to claim 1, characterized in that, The step of distributing the energy of pixels in the third pixel block to the corresponding pixels in the second pixel block based on the second jitter matrix corresponding to the second pixel block to obtain the second target video image of the current frame includes: The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value; Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

4. The method according to claim 3, characterized in that, The step of obtaining the second target video image of the current frame based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block includes: For each obtained second pixel value, the second pixel value is superimposed on the pixel value of the corresponding pixel in the second pixel block to obtain the third pixel value; The image formed by the obtained third pixel value is used as the second target video image of the current frame.

5. The method according to claim 1, characterized in that, The generation of the target video stream based on the second target video image obtained sequentially for each frame includes: For each frame of the second target video image, a frame synchronization signal, a line synchronization signal, and a data validity flag signal are added to the video signal corresponding to the second target video image to generate the target video stream.

6. The method according to any one of claims 1 to 5, characterized in that, The first target video image of the current frame is obtained by implementing the first dithering matrix corresponding to the first pixel block through the register transfer stage, which distributes the energy of the target pixel located at the center position in the first pixel block to the pixels at other positions in the first pixel block. and / or The register transfer stage is used to implement the second dithering matrix corresponding to the second pixel block, which distributes the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block, thereby obtaining the second target video image of the current frame.

7. A video image processing device, characterized in that, It includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, causes the processor to perform the method as described in any one of claims 1 to 6.

8. A video image processing apparatus, characterized in that, include: The spatial dithering module is used to sequentially receive video images from each frame of the video stream. For each first pixel block in the video image of the current frame, based on the first jitter matrix corresponding to the first pixel block, the energy of the target pixel point located at the center position in the first pixel block is distributed to the pixels at other positions in the first pixel block to obtain the first target video image of the current frame; The time jitter module is used to distribute the energy of the pixels in the third pixel block to the corresponding pixels in the second pixel block for each second pixel block in the first target video image of the current frame, based on the second jitter matrix corresponding to the second pixel block, to obtain the second target video image of the current frame. The third pixel block is located in the first target video image of the previous frame, and the position of the third pixel block in the first target video image of the previous frame is the same as the position of the second pixel block in the first target video image of the current frame. A target video stream is generated based on the second target video images of each frame obtained in sequence.

9. The apparatus according to claim 8, characterized in that, The spatial jitter module is specifically used for: The pixel value of each pixel in the first pixel block is multiplied by the corresponding ratio value in the first dithering matrix to obtain the first pixel value; The image formed by the obtained first pixel values ​​is used as the first target video image of the current frame.

10. The apparatus according to claim 8, characterized in that, The time jitter module is specifically used for: The pixel value of each pixel in the third pixel block is multiplied by the corresponding ratio value in the second dithering matrix to obtain the second pixel value; Based on the obtained second pixel value and the pixel value of the corresponding pixel in the second pixel block, the second target video image of the current frame is obtained.

Images