Video encoding device, video decoding device, video encoding method, and video decoding method

Abstract

In order to prevent signal degradation in a video encoding scheme in which bit truncation is performed, this video encoding device comprises: a preprocessing means that executes preprocessing including processing for applying thinning of pictures in an input video and processing including color conversion of pixels; and an encoding means that executes encoding processing of a video signal subjected to the preprocessing. The preprocessing means executes the color conversion immediately before the encoding processing.

Description

Video Encoding Device, Video Decoding Device, Video Encoding Method, and Video Decoding Method

[0001] The present disclosure relates to a video encoding device, a video decoding device, a video encoding method, and a video decoding method.

[0002] In order to efficiently transmit or record video, a video encoding device that generates an encoded representation (hereinafter referred to as a bitstream) by encoding an input video, and a video decoding device that decodes the bitstream to generate a decoded video are used.

[0003] As standards for video encoding methods, there are H.265 / HEVC (High-Efficiency Video Coding) (see Non-Patent Document 1), H.266 / VVC (Versatile Video Coding) (see Non-Patent Document 2), etc., which are standardized by ITU-T SG16 and ISO / IEC / SC29.

[0004] In these video encoding methods, video data is managed in a hierarchical structure and encoded and decoded. The hierarchical structure consists of, for example, pictures that make up the video data, slices (or tiles) obtained by dividing a picture, coding tree units (CTUs) obtained by dividing a slice, and coding units (CUs) obtained by dividing a coding tree unit.

[0005] The input image of the CU to be processed is usually prediction-encoded based on a prediction image generated based on a decoded image that was encoded and decoded before the CU to be processed. That is, the prediction error image obtained by subtracting the prediction image from the input image is encoded and decoded. As prediction encoding, there are intra prediction (intra prediction) that uses a decoded image included in a picture at the same display time as the CU to be processed, and inter prediction (inter prediction) that uses a decoded image included in a picture at a different display time from the CU to be processed.

[0006] A CU encoded based on intra prediction is called an intra CU. A CU encoded based on inter prediction is called an inter CU.

[0007] A picture that can only contain an intraCU is called an I-picture. A picture that can contain not only an intraCU but also an interCU that uses a decoded image contained in one picture with a different display time than the target CU is called a P-picture. A picture that can contain not only an intraCU and an interCU that uses a decoded image contained in one picture with a different display time than the target CU, but also an interCU that uses decoded images contained in two pictures with different display times than the target CU is called a B-picture.

[0008] The coding of the prediction error image is performed based on frequency conversion, quantization, and entropy coding. The decoding of the prediction error image is performed based on entropy decoding, inverse quantization, and inverse frequency conversion.

[0009] Recommendation ITU-T H.264 "High efficiency video coding", Telecommunication Standardization Sector of ITU, April 2015 Recommendation ITU-T H.266 "Versatile video coding", Telecommunication Standardization Sector of ITU, August 2020

[0010] Development of encoding schemes for machines used in machine recognition is underway. For example, ISO / IEC / SC29 is working on standardizing a video encoding scheme for machines called Video for Coding Machines (VCM).

[0011] The performance of a video encoding scheme for machines is generally evaluated by the processing performance performed by the machine when decoding video from a bitstream of a similar amount of data. Video encoding devices that perform video encoding for machines may apply pre-processing that has not been used in schemes intended for human viewing, in order to improve the performance of the encoding scheme.

[0012] The following technologies are considered elemental technologies for preprocessing to improve the performance of encoding schemes: • Time sampling (time resampling) • Spatial sampling (spatial resampling) • Targeting (retargeting) • Bit depth truncation (bit depth shifting)

[0013] Time sampling (time resampling) is a technique in which the video encoder reduces the frame rate in the time direction, and the video decoder upsamples the frame rate to restore it to its original value. In other words, it is a technique in which the video encoder performs frame decimation (decimation of pictures in the video), and the video decoder restores the frames (pictures) through frame interpolation.

[0014] Spatial sampling (or spatial resampling) is a technique in which a video encoder reduces the spatial resolution of an image, and a video decoder increases the resolution (returning it to the original resolution). In other words, it is a technique in which the video encoder shrinks the picture, and the video decoder enlarges the picture.

[0015] Specifically, in spatial sampling, a reduction process is applied to the input video on a picture-by-picture basis so that the size (width and height) of the picture to be processed is reduced. In other words, the video encoder performs spatial resampling on the picture to be processed. By reducing the number of pixels to be encoded, it becomes easier to reduce the amount of data in the bitstream. During decoding, as a corresponding post-processing step, an enlargement process to the original size is applied to the decoded video. In other words, in spatial resampling, the video decoder performs spatial reconstruction on the picture to be processed.

[0016] Targeting (retargeting) is the process of replacing pixel values ​​in areas of the input video other than the region of interest (ROI), i.e., non-focused areas, with arbitrary pixel values. The region of interest is the area in the video that should be given importance (important area). The video encoder also reduces the region of interest, which is of relatively low importance. The video decoder then enlarges (restores) the reduced region of interest.

[0017] Bit depth truncation (bit depth shifting) reduces the number of bits used to represent an image, for example, the Y component, by one bit. Therefore, the number of bits in the Y component (luminance component) of a video signal in YUV space is reduced by one bit. Bit depth truncation is equivalent to reducing the luminance by half.

[0018] When transmitting video signals, video encoders generally do not encode RGB signals directly. To improve compression efficiency, video encoders convert RGB signals to signals in a different color space before encoding them.

[0019] Another color space, for example, is used. YUV 4:2:2 and YUV 4:2:0 formats can be used as signal formats for the YUV color space. Hereafter, the conversion of color space signals will be referred to as color conversion.

[0020] Figure 10 is a block diagram showing an example of a video decoder 220 suitable for a video encoding scheme for machines. In other words, it is a block diagram showing an example of the configuration of a video decoder 220 that performs post-processing. Note that the video encoder performs pre-processing (pre-processing).

[0021] Note that the arrows in Figure 10 simply indicate the direction of signal (data) flow, but do not exclude bidirectional flow. This is also true for other block diagrams.

[0022] The video decoder 220 includes a demultiplexer 201, an internal video decoder (internal decoder) 202, a spatial resampling unit 203, a retargeting unit 204, a temporal resampling unit 205, and a bit depth shift unit 206.

[0023] The demultiplexer 201 demultiplexes the bitstream input to the video decoder 220 to obtain a video bitstream and an auxiliary information bitstream. The demultiplexer 201 supplies the video bitstream to the internal video decoder 202. The demultiplexer 201 supplies the auxiliary information contained in the auxiliary information bitstream to the spatial resampling unit 203, the retargeting unit 204, the temporal resampling unit 205, and the bit depth shift unit 206.

[0024] The internal video decoder 202 decodes the video bitstream and supplies the decoded picture (decoded video frame) to the spatial resampling unit 203.

[0025] The spatial resampling unit 203 applies spatial resampling to the decoded picture. The spatial resampling performed by the spatial resampling unit 203 is, for example, picture enlargement. The retargeting unit 204 performs retargeting processing on the output of the spatial resampling unit 203. The temporal resampling unit 205 performs resampling on the output of the retargeting unit 204. The resampling performed by the temporal resampling unit 205 is, for example, frame interpolation.

[0026] The bit depth shift unit 206 restores the video from the output of the time resampling unit 205. The bit depth shift unit 206 performs, for example, a 1-bit left shift of the Y component.

[0027] Figure 11 is a block diagram showing an example of the configuration of a video decoder 220, including a detailed configuration example of the time resampling unit 205. In Figure 11, YUV means a signal in YUV4:2:0 format that has not undergone bit shifting (one bit to the right shift) (the Y component is not halved). YUV' means a signal in YUV4:2:0 format that has been bit shifted (at least the Y component has been truncated). Hereinafter, a YUV4:2:0 format signal that has not undergone bit shifting will be called a YUV signal, and a bit-shifted YUV4:2:0 format signal will be called a YUV' signal.

[0028] The time resampling unit 205 performs time resampling on signals in the RGB space.

[0029] For conversion between the RGB color space and the YUV color space (YCbCr color space), formulas such as those defined in ITU-R Recommendation BT. 709 can be used. Furthermore, the same approach as in BT. 709 can be applied to BT. 2020, for example. Figure 11 illustrates the conversion formulas from a YUV4:2:0 signal to an RGB signal and from an RGB signal to a YUV4:2:0 signal as defined in BT. 709.

[0030] In the video decoder 220, the time resampling unit 205 includes a first conversion unit 2051 that converts a YUV' signal to an RGB signal and a second conversion unit 2052 that converts an RGB signal to a YUV' signal.

[0031] When using Recommendation BT. 709, the first conversion unit 2051 converts the YUV' signal to an RGB signal based on the conversion formula illustrated in Figure 11. After time resampling for frame interpolation is performed, the second conversion unit 2052 converts the RGB signal to a YUV' signal based on the conversion formula illustrated in Figure 11.

[0032] In a video encoder, the internal video encoder encodes a video signal that has undergone bit truncation (bit removal). Therefore, in the video decoder 220, the time resampling unit 205 performs color conversion (conversion from YUV color space to RGB space, and conversion from RGB space to YUV color space) on the video signal containing the bit-truncate Y component. As a result, signal degradation occurs.

[0033] The present invention aims to provide a video encoding device, a video decoding device, a video encoding method, a video decoding method, a video encoding program, and a video decoding program that can prevent signal degradation in a video encoding scheme in which bit truncation is performed.

[0034] The video encoding method based on this disclosure is a video encoding method that performs preprocessing including a process that applies picture decimation to an input video and a process that includes pixel color conversion, and then performs encoding processing of the preprocessed video signal, wherein the color conversion is performed immediately before the encoding processing.

[0035] The video decoding method according to this disclosure decodes a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and the number of bits of the luminance component has been reduced by at least one bit; performs a first restoration process to restore the number of bits of the luminance component in the decoded video signal; performs a second restoration means process to restore the decimated pictures; the second restoration process includes a color space conversion process that converts the color space of the decoded video signal, and the first restoration process is performed before the second restoration means process.

[0036] A video decoding method in another aspect of the present disclosure decodes a video bitstream that includes encoded data based on a video signal in which picture decimation is applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; performs a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and performs a second restoration means process to restore the decimated pictures. The second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and performs the color space conversion process using a conversion formula that includes coefficients corresponding to a predetermined conversion formula.

[0037] The video encoding apparatus according to this disclosure includes a preprocessing means that performs preprocessing including a process that applies picture decimation to the input video and a process that includes color conversion of pixels, and an encoding means that performs encoding processing of the preprocessed video signal, wherein the preprocessing means performs color conversion immediately before the encoding processing.

[0038] The video decoding device according to this disclosure includes decoding means for decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and at least the number of bits of the luminance component has been reduced; a first restoration means for restoring the number of bits of the luminance component in the decoded video signal; and a second restoration means for restoring the decimated pictures, wherein the second restoration means includes a color space conversion means for converting the color space of the decoded video signal, and the first restoration means and the second restoration means are arranged such that processing by the first restoration means is performed before processing by the second restoration means.

[0039] Other embodiments of the video decoding apparatus based on this disclosure include decoding means for decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video has been applied to reduce the number of bits of the luminance component by at least through a color space conversion process using a predetermined conversion formula; first restoration means for restoring the number of bits of the luminance component in the decoded video signal; and second restoration means for restoring the decimated pictures, wherein the second restoration means includes color space conversion means for performing a color space conversion of the decoded video signal, and the color space conversion means performs the color space conversion process using a conversion formula that includes coefficients corresponding to a predetermined conversion formula.

[0040] The video encoding program based on this disclosure causes a computer to perform preprocessing, which includes processing to apply picture decimation to the input video and processing to perform color conversion of pixels, and to perform encoding processing of the preprocessed video signal, in which the color conversion is performed immediately before the encoding processing in the preprocessing.

[0041] The video decoding program based on this disclosure causes a computer to perform the following processes: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and at least the number of bits of the luminance component has been reduced; a first restoration process that restores the number of bits of the luminance component in the decoded video signal; and a second restoration means process that restores the decimated pictures. The second restoration process includes a color space conversion process that converts the color space of the decoded video signal, and causes the computer to perform the first restoration process before the second restoration means process.

[0042] Other embodiments of the video decoding program based on this disclosure cause a computer to perform the following processes: decoding a video bitstream including encoded data based on a video signal in which picture decimation in the video is applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and a second restoration means process to restore the decimated pictures. The second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and causes the computer to perform the color space conversion process using a conversion formula that includes coefficients corresponding to a predetermined conversion formula.

[0043] According to the present invention, signal degradation can be prevented in a video encoding method in which bit truncation is performed.

[0044] It is a block diagram showing an example of the configuration of a video encoder and a video decoder including a detailed configuration example of a time sampling unit and a time resampling unit. It is a flowchart showing the operation of the video encoder. It is a flowchart showing the operation of the video decoder. It is a block diagram showing an example of the configuration of a video encoder and a video decoder including a detailed configuration example of a time sampling unit and a time resampling unit. It is a flowchart showing the operation of the video decoder. It is a block diagram showing an example of the configuration of an information processing system. It is a block diagram showing the main part of a video encoding device. It is a block diagram showing the main part of a video decoding device. It is a block diagram showing the main part of a video decoding device of another aspect. It is a block diagram showing an example of a video decoder. It is a block diagram showing an example of the configuration of a video decoder including a detailed configuration example of a time resampling unit.

[0045] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0046] Embodiment 1. FIG. 1 is a block diagram showing an example of the configuration of a video encoder 100 and a video decoder 200 according to the first embodiment including a detailed configuration example of a time sampling unit and a time resampling unit.

[0047] The video encoder 100 includes a preprocessing unit (time sampling unit 101, targeting unit 102, spatial sampling unit 103, bit depth truncation unit 104), an internal video encoder (internal encoder) 105, and a multiplexer 106.

[0048] The time sampling unit 101 performs time resampling on the input video (input picture). The time sampling performed by the time sampling unit 101 is an inter-frame thinning process.

[0049] In the video encoder 100, the time sampling unit 101 includes a first conversion unit 1011 that converts a YUV signal into an RGB signal and a second conversion unit 1012 that converts an RGB signal into a YUV signal.

[0050] When using Recommendation BT. 709, the first conversion unit 1011 converts the YUV signal to an RGB signal based on the conversion formula illustrated in Figure 1. After time sampling with respect to frame decimation is performed, the second conversion unit 1012 converts the RGB signal to a YUV signal based on the conversion formula illustrated in Figure 1.

[0051] The targeting unit 102 performs targeting processing on the input video that has been downsampled. The spatial sampling unit 103 applies spatial sampling to the output of the targeting unit 102. The spatial sampling performed by the spatial sampling unit 103 is, for example, picture reduction.

[0052] The bit depth truncation unit 104 applies bit depth truncation (truncation) to the output of the spatial sampling unit 103. The bit depth truncation performed by the bit depth truncation unit 104 is, for example, a one-bit right shift of the pixel value (typically the Y component).

[0053] The internal video encoder 105 encodes the picture supplied from the bit depth trimming unit 104 using a predetermined video encoding scheme. As the predetermined video encoding scheme, for example, an encoding scheme based on H.266 / VVC can be used. As an example, the predetermined video encoding scheme can be VVC with End2End added.

[0054] The multiplexer 106 outputs a bitstream obtained by multiplexing the video bitstream supplied from the internal video encoder 105 and the auxiliary information bitstream as a bitstream.

[0055] The video decoder 200 includes a demultiplexer 201, an internal video decoder 202, a spatial resampling unit 203, a retargeting unit 204, a temporal resampling unit 205, and a bit depth shift unit 206.

[0056] The configuration and function of each block in the video decoder 200 are the same as those shown in Figures 10 and 11. However, in this embodiment, the execution order of each process constituting the post-processing differs from the execution order in the video decoder 220 shown in Figure 11.

[0057] In other words, time resampling by the time resampling unit 205 is performed at least after the bit depth shifting process by the bit depth shifting unit 206. Therefore, the number of bits in the video signal is increased before color conversion (conversion from YUV color space to RGB space, and conversion from RGB space to YUV color space) is performed. That is, the number of bits truncated by the video encoder 100 is restored to the original number of bits.

[0058] Next, the operation of the video encoder 100 of the first embodiment will be explained with reference to the flowchart in Figure 2.

[0059] The time sampling unit 101 performs time sampling on the input RGB video signal (step S101). Furthermore, the time sampling unit 101 converts the time-sampled video signal into a YUV signal. The targeting unit 102 performs targeting processing on the output of the time sampling unit 101 (step S102). The spatial sampling unit 103 performs spatial sampling on the output of the targeting unit 102 (step S103). The bit depth truncation unit 104 right-shifts the pixel values ​​of the video signal output by the spatial sampling unit 103 (step S104).

[0060] In addition, the targeting unit 102 and the spatial sampling unit 103 handle YUV video signals, and the time sampling unit 101 performs the YUV conversion. However, if the targeting unit 102 and the spatial sampling unit 103 handle RGB video signals, the conversion to YUV may be performed immediately before the bit depth truncation unit 104.

[0061] The internal video encoder 105 encodes the video signal output by the bit depth trimming unit 104 (step S105).

[0062] The multiplexer 106 multiplexes the encoded data (video bitstream) output by the bit depth truncation unit 104 and the auxiliary information bitstream, and outputs a bitstream. The auxiliary information bitstream includes auxiliary information as processing parameters output by the time sampling unit 101, the targeting unit 102, the spatial sampling unit 103, and the bit depth truncation unit 104.

[0063] Next, the operation of the video decoder 200 of the first embodiment will be explained with reference to the flowchart in Figure 3.

[0064] The demultiplexer 201 demultiplexes the received bitstream and obtains the video bitstream and the auxiliary information bitstream (step S201). The demultiplexer 201 supplies the video bitstream to the internal video decoder 202. The demultiplexer 201 supplies the auxiliary information contained in the auxiliary information bitstream to the bit depth shift unit 206, the spatial resampling unit 203, the retargeting unit 204, and the temporal resampling unit 205.

[0065] The internal video decoder 202 decodes the encoded data contained in the video bitstream to obtain a video signal (decoded video signal) (step S202).

[0066] The bit depth shift unit 206 shifts the pixel values ​​(typically the Y component) of the decoded video signal, which is the output of the time resampling unit 205, to the left (step S203). The spatial resampling unit 203 performs spatial resampling on the output of the bit depth shift unit 206 (step S204). The retargeting unit 204 applies retargeting processing to the spatial resampling unit 203 (step S205).

[0067] In the time resampling unit 205, the first conversion unit 2051 performs time resampling (step S206).

[0068] The video signal resulting from the processing of the time resampling unit 205 is output as output video from the video decoder 200.

[0069] In this embodiment, the video decoder 200 performs time resampling on the video signal whose bit depth has been restored, thereby reducing signal degradation caused by the execution of time resampling. As a result, the quality of the decoded video is improved compared to the video decoded by the video decoder 220 shown in Figure 10.

[0070] Furthermore, in the video decoder 200, it is sufficient that bit depth shifting by the bit depth shifting unit 206 is performed before time resampling by the time resampling unit 205 is performed. Therefore, bit depth shifting may be performed immediately after spatial resampling by the spatial resampling unit 203. Alternatively, bit depth shifting may be performed immediately after retargeting processing by the retargeting unit 204.

[0071] In other words, in the configuration illustrated in Figure 1, the bit depth shift unit 206 may be provided after the spatial resampling unit 203, or after the retargeting unit 204.

[0072] Embodiment 2. Figure 4 is a block diagram showing an example of the configuration of the video encoder 100 and video decoder 210 of the second embodiment, including a detailed configuration example of the time sampling unit and the time resampling unit. The overall configuration of the video encoder 100 and video decoder 210 is the same as the configuration of the video encoder 100 and video decoder 200 shown in Figure 1. The fact that the overall configuration is the same means that the roles of each block are the same.

[0073] The video encoder 100 includes a time sampling unit 101, a targeting unit 102, a spatial sampling unit 103, a bit depth truncation unit 104, an internal video encoder 105 (not shown in Figure 4), and a multiplexer 106 (not shown in Figure 4).

[0074] The video decoder 200 includes a demultiplexer 201 (not shown in Figure 4), an internal video decoder 202 (not shown in Figure 4), a spatial resampling unit 203, a retargeting unit 204, a temporal resampling unit 205B, and a bit depth shift unit 206. The configuration and functions of the demultiplexer 201, internal video decoder 202, spatial resampling unit 203, retargeting unit 204, and bit depth shift unit 206 are the same as those shown in Figures 10 and 11.

[0075] The time resampling unit 205B includes a first conversion unit 2053 that converts a YUV' signal to an RGB signal and a second conversion unit 2054 that converts an RGB signal to a YUV' signal.

[0076] The processing performed by the first conversion unit 2053 and the second conversion unit 2054 differs from that performed by the first conversion unit 2051 and the second conversion unit 2052 in the first embodiment.

[0077] Figure 4 illustrates the conversion formulas from a YUV4:2:0 signal to an RGB signal and from an RGB signal to a YUV4:2:0 signal. The conversion formulas used by the first conversion unit 2053 and the second conversion unit 2054 differ from those used by the first conversion unit 2051 and the second conversion unit 2052 in the first embodiment.

[0078] The conversion formulas used by the first conversion unit 2053 and the second conversion unit 2054 are modified versions of the formula defined in Recommendation BT. 709. Specifically, among the elements (coefficients) of the matrix that constitute the conversion formula used by the first conversion unit 2053, the value multiplied by the Y component is different from the value in the formula defined in Recommendation BT. 709 (the conversion formula used by the first conversion unit 1011 in the video encoder 100). More specifically, the value multiplied by the Y component is twice the value in the formula defined in Recommendation BT. 709 (corresponding to a 1-bit left shift).

[0079] Furthermore, among the elements (coefficients) of the matrix that constitute the transformation formula used by the second transformation unit 2054, the value multiplied by the Y component differs from the value in the formula defined in Recommendation BT. 709 (the transformation formula used by the second transformation unit 1012 in the video encoder 100). More specifically, the value multiplied by the Y component is half the value in the formula defined in Recommendation BT. 709 (corresponding to a 1-bit right shift).

[0080] In other words, in the time resampling unit 205B, the first conversion unit 2053 and the second conversion unit 2054 perform color conversion using coefficients corresponding to the fact that the Y component was truncated (shifted 1 bit to the right) in the video encoder 100.

[0081] Next, the operation of the video decoder 210 of the second embodiment will be explained with reference to the flowchart in Figure 5.

[0082] The processes in steps S201, S202, S204, and S205 are the same as those in the first embodiment.

[0083] Next, in the time resampling unit 205B, the first conversion unit 2053 converts the YUV' signal to an RGB signal using the conversion formula described above (step S206B). After time resampling is performed, the second conversion unit 2054 converts the RGB signal to a YUV' signal using the conversion formula described above (step S206B).

[0084] Then, the bit depth shift unit 206 shifts the pixel values ​​of the decoded video signal (typically the Y component), which is the output of the time resampling unit 205B, to the left (step S203).

[0085] The video signal resulting from the bit depth shifting unit 206 is output as output video from the video decoder 210.

[0086] In this embodiment, the video decoder 210 performs color conversion using a coefficient corresponding to the truncation of the Y component in the video encoder 100, and then performs time resampling. This reduces the impact of the truncation of the Y component. As a result, signal degradation caused by the execution of time resampling is reduced. Therefore, the quality of the decoded video is improved compared to the video decoded by the video decoder 220 shown in Figure 10.

[0087] While the above embodiment can be implemented using hardware, it can also be realized using a computer program.

[0088] The information processing system shown in Figure 6 comprises a processor 701 such as a CPU (Central Processing Unit), a program memory 702, a storage medium 703 for storing video data, and a storage medium 704 for storing bitstreams. Multiple processors 701 may be provided. The storage mediums 703 and 704 may be separate storage mediums or may be storage areas consisting of the same storage medium. Magnetic storage media such as hard disks can be used as storage media. Furthermore, the information processing system may also comprise multiple processors 701.

[0089] In the information processing system, the program memory 702 stores programs (video encoding programs or video decoding programs) for realizing the functions of each block shown in the above embodiment.

[0090] The processor 701 then performs processing according to the program stored in the program memory 702, thereby realizing the functions of the video encoder 100 or video decoders 200, 210 shown in the above embodiment.

[0091] For example, the functions of the video encoder 100 are realized when the processor 701 executes processing according to a video encoding program that realizes the functions of each block in the video encoder 100 shown in Figure 1.

[0092] Furthermore, for example, the functions of the video decoder are realized when the processor 701 executes processing according to a video decoding program that realizes the functions of each block in the video decoders 200 and 210 shown in Figures 1 and 4.

[0093] Furthermore, at least the program memory 702 is a non-transitory computer-readable medium. However, the program may be stored in various types of transient computer-readable medium. The program is supplied to the transient computer-readable medium, for example, via a wired communication channel or a wireless communication channel, i.e., via electrical signals, optical signals or electromagnetic waves.

[0094] Figure 7 is a block diagram showing the main parts of a video encoding device. The video encoding device 10 shown in Figure 7 (implemented by a video encoder 100 in this embodiment) comprises a preprocessing means 11 (implemented by a time sampling unit 101, a targeting unit 102, a spatial sampling unit 103, and a bit depth truncation unit 104 in this embodiment) that performs preprocessing including a process that applies picture decimation to the input video and a process that includes color conversion of pixels, and an encoding means 12 (implemented by an internal video encoder 105 in this embodiment) that performs encoding processing of the preprocessed video signal, wherein the preprocessing means 11 performs color conversion immediately before the encoding processing.

[0095] Figure 8 is a block diagram showing the main parts of the video decoding device. The video decoding device 20 shown in Figure 8 (implemented by a video encoder 100 in this embodiment) includes a decoding means 22 (implemented by an internal video decoder 202 in this embodiment) that decodes a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and at least the number of bits of the luminance component has been reduced, a first restoration means 23 (implemented by a bit depth shift unit 206 in this embodiment) that restores the number of bits of the luminance component in the decoded video signal, and a second restoration means 24 (implemented by a time resampling unit 205 in this embodiment) that restores the decimated pictures. The second restoration means 24 includes a color space conversion means 241 (implemented by a first conversion unit 2051 and a second conversion unit 2052 in this embodiment) that performs a color space conversion of the decoded video signal, and the first restoration means 23 and the second restoration means 24 are arranged such that the processing by the first restoration means 23 is performed before the processing by the second restoration means 24.

[0096] Figure 9 is a block diagram showing the main parts of another embodiment of a video decoding device. The video decoding device 21 shown in Figure 9 (implemented by a video decoder 210 in this embodiment) includes a decoding means 22 (implemented by an internal video decoder 202 in this embodiment) that decodes a video bitstream that includes encoded data based on a video signal from which picture decimation in the video has been applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; a first restoration means 23 (implemented by a bit depth shift unit 206 in this embodiment) that restores the number of bits of the luminance component in the decoded video signal; and a second restoration means 25 (implemented by a time resampling unit 205B in this embodiment) that restores the decimated pictures. The second restoration means 25 includes a color space conversion means 251 (implemented by a first conversion unit 2053 and a second conversion unit 2054 in this embodiment) that performs a color space conversion of the decoded video signal. The color space conversion means 251 performs a color space conversion process using a conversion formula that includes coefficients corresponding to a predetermined conversion formula.

[0097] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0098] (Note 1) A video encoding method that performs preprocessing including a process that applies picture decimation to an input video and a process that includes pixel color conversion, and then performs encoding processing of the video signal that has undergone the preprocessing, wherein the color conversion is performed immediately before the encoding processing.

[0099] (Note 2) The color conversion is the video encoding method described in Note 1, which is a conversion to the YUV space.

[0100] (Note 3) A video decoding method comprising: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and the number of bits of the luminance component has been reduced by at least one bit; performing a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and performing a second restoration means process to restore the decimated pictures, wherein the second restoration process includes a color space conversion process that converts the color space of the decoded video signal, and the first restoration process is performed before the second restoration means process.

[0101] (Note 4) A video decoding method comprising: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video has been applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; performing a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and performing a second restoration means process to restore the decimated pictures, wherein the second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and the color space conversion process is performed using a conversion formula that includes coefficients corresponding to the predetermined conversion formula.

[0102] (Note 5) The video decoding method described in Note 4, wherein the predetermined conversion formula is a formula defined in ITU-R Recommendation BT. 709 or BT. 2020.

[0103] (Note 6) A video encoding device comprising: a preprocessing means that performs preprocessing including a process that applies picture decimation to the input video and a process that includes color conversion of pixels; and an encoding means that performs encoding processing of the video signal that has been preprocessed, wherein the preprocessing means performs the color conversion immediately before the encoding processing.

[0104] (Note 7) The color conversion is a conversion to the YUV space, as described in Note 6 of the video encoding device.

[0105] (Note 8) A video decoding device comprising: a decoding means for decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and the number of bits of the luminance component has been reduced; a first restoration means for restoring the number of bits of the luminance component in the decoded video signal; and a second restoration means for restoring the decimated pictures, wherein the second restoration means includes a color space conversion means for converting the color space of the decoded video signal, and the first restoration means and the second restoration means are arranged such that the processing by the first restoration means is performed before the processing by the second restoration means.

[0106] (Note 9) A video decoding device comprising: a decoding means for decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video has been applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; a first restoration means for restoring the number of bits of the luminance component in the decoded video signal; and a second restoration means for restoring the decimated pictures, wherein the second restoration means includes a color space conversion means for performing a color space conversion of the decoded video signal, and the color space conversion means performs the color space conversion process using a conversion formula that includes coefficients corresponding to the predetermined conversion formula.

[0107] (Note 10) The video decoding device described in Note 9, wherein the predetermined conversion formula is a formula defined in ITU-R Recommendation BT. 709 or BT. 2020.

[0108] (Note 11) A video encoding program that causes a computer to perform preprocessing including a process that applies picture decimation to the input video and a process that includes color conversion of pixels, to perform encoding processing of the video signal that has undergone the preprocessing, and to perform the color conversion immediately before the encoding processing in the preprocessing.

[0109] (Note 12) The color conversion is a video encoding program as described in Note 11, which is a conversion to the YUV space.

[0110] (Note 13) A video decoding program that causes a computer to perform the following: a process of decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video has been applied and the number of bits of the luminance component has been reduced by at least one bit; a first restoration process of restoring the number of bits of the luminance component in the decoded video signal; and a second restoration means process of restoring the decimated pictures, wherein the second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and the computer to perform the first restoration process before the second restoration means process.

[0111] (Note 14) A video decoding program that causes a computer to perform the following: a process of decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video is applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and a second restoration means process to restore the decimated pictures, wherein the second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and the computer to perform the color space conversion process using a conversion formula that includes coefficients corresponding to the predetermined conversion formula.

[0112] (Note 15) The video decoding program described in Note 14, wherein the predetermined conversion formula is a formula defined in ITU-R Recommendation BT. 709 or BT. 2020.

[0113] Appendices 2, 5, 7, 10, 12, and 15 above allow for the dependency of some or all of the configurations described above to various hardware, software, various recording means for recording software, or systems, provided that they do not deviate from the embodiments described above.

[0114] Although the present invention has been described above with reference to embodiments and examples, the present invention is not limited to the above embodiments and examples. Various modifications to the structure and details of the present invention can be made, as can be understood by those skilled in the art within the scope of the present invention.

[0115] This application claims priority based on Japanese Patent Application No. 2024-174908, filed on 4 October 2024, and incorporates all of its disclosures herein.

[0116] 10 Video encoding device 11 Preprocessing means 12 Encoding means 20, 21 Video decoding device 22 Decoding means 23 First restoration means 24, 25 Second restoration means 241, 251 Color space conversion means 100 Video encoder 101 Time sampling unit 102 Targeting unit 103 Spatial sampling unit 104 Bit depth truncation unit 105 Internal video encoder 106 Multiplexer 1011 First conversion unit 1012 Second conversion unit 200, 210, 220 Video decoder 201 Demultiplexer 202 Internal video decoder 203 Spatial resampling unit 204 Retargeting unit 205, 205B Time resampling unit 206 Bit depth shift unit 2051, 2053 First conversion unit 2052, 2054 Second conversion unit 701 Processor 702 Program memory 703, 704 Storage medium

Claims

1. A video encoding method that performs preprocessing including a process that applies picture decimation to an input video and a process that includes pixel color conversion, and then performs encoding processing of the video signal that has undergone the preprocessing, wherein the color conversion is performed immediately before the encoding processing.

2. The video encoding method according to claim 1, wherein the color conversion is a conversion to the YUV space.

3. A video decoding method comprising: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and the number of bits of the luminance component has been reduced by at least one bit; performing a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and performing a second restoration means process to restore the decimated pictures, wherein the second restoration process includes a color space conversion process that converts the color space of the decoded video signal, and the first restoration process is performed before the second restoration means process.

4. A video decoding method comprising: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video has been applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; performing a first restoration process to restore the number of bits of the luminance component in the decoded video signal; and performing a second restoration means process to restore the decimated pictures, wherein the second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and the color space conversion process is performed using a conversion formula that includes coefficients corresponding to the predetermined conversion formula.

5. The video decoding method according to claim 4, wherein the predetermined conversion formula is a formula defined in ITU-R Recommendation BT. 709 or BT. 2020.

6. A video encoding device comprising: preprocessing means that performs preprocessing including a process that applies picture decimation to the input video and a process that includes color conversion of pixels; and encoding means that performs encoding processing of the video signal that has undergone the preprocessing, wherein the preprocessing means performs the color conversion immediately before the encoding processing.

7. The video encoding device according to claim 6, wherein the color conversion is a conversion to the YUV space.

8. A video decoding apparatus comprising: a decoding means for decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and the number of bits of the luminance component has been reduced; a first restoration means for restoring the number of bits of the luminance component in the decoded video signal; and a second restoration means for restoring the decimated pictures, wherein the second restoration means includes a color space conversion means for converting the color space of the decoded video signal, and the first restoration means and the second restoration means are arranged such that the processing by the first restoration means is performed before the processing by the second restoration means.

9. A video decoding device comprising: a decoding means for decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video has been applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of at least the luminance component; a first restoration means for restoring the number of bits of the luminance component in the decoded video signal; and a second restoration means for restoring the decimated pictures, wherein the second restoration means includes a color space conversion means for performing a color space conversion of the decoded video signal, and the color space conversion means performs the color space conversion process using a conversion formula that includes coefficients corresponding to the predetermined conversion formula.

10. The video decoding device according to claim 9, wherein the predetermined conversion formula is a formula defined in ITU-R Recommendation BT. 709 or BT. 2020.

11. A video encoding program that causes a computer to perform preprocessing including a process that applies picture decimation to the input video and a process that includes color conversion of pixels, to perform encoding processing of the video signal that has undergone the preprocessing, and to perform the color conversion immediately before the encoding processing in the preprocessing.

12. The video encoding program according to claim 11, wherein the color conversion is a conversion to the YUV space.

13. A video decoding program that causes a computer to perform the following processes: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation has been applied and the number of bits of the luminance component has been reduced by at least one bit; a first restoration process that restores the number of bits of the luminance component in the decoded video signal; and a second restoration means process that restores the decimated pictures, wherein the second restoration process includes a color space conversion process that converts the color space of the decoded video signal, and the computer to perform the first restoration process before the second restoration means process.

14. A video decoding program that causes a computer to perform the following processes: decoding a video bitstream that includes encoded data based on a video signal in which picture decimation in the video is applied after a color space conversion process using a predetermined conversion formula, thereby reducing the number of bits of the luminance component by at least one bit; a first restoration process that restores the number of bits of the luminance component in the decoded video signal; and a second restoration means process that restores the decimated pictures, wherein the second restoration process includes a color space conversion process that performs a conversion of the color space of the decoded video signal, and the computer to perform the color space conversion process using a conversion formula that includes coefficients corresponding to the predetermined conversion formula.

15. The video decoding program according to claim 14, wherein the predetermined conversion formula is a formula defined in ITU-R Recommendation BT. 709 or BT. 2020.

Images