Video comparison and evaluation device and program therefor

Abstract

To provide a video comparison and evaluation device capable of performing a comparison between videos by adaptively changing an evaluation reference in accordance with complexity of videos.SOLUTION: A video comparison and evaluation device 1 comprises: a reduction conversion section 11 for generating a reduced reference video by reducing a reference video into a resolution of an evaluation video; a feature amount extraction section 14 for extracting a feature amount to be an index of complexity in a space direction and a time direction from the reduced reference video; an inter-pixel differential calculation section 13 for calculating a differential of pixel values between pixels on a frame or field basis from the reduced reference video and the evaluation video; a quality evaluation value calculation section 15 for calculating a quality evaluation value from the differential of the pixel values; and a quality evaluation value determination section 16 for determining an abnormality of the quality evaluation value by more mitigating a reference for determining the abnormality of the quality evaluation value as a degree of complexity based on the feature amount is larger.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a video comparison evaluation apparatus and a program thereof.

Background Art

[0002] In recent years, the production of 4K (UHDTV: Ultra High Definition Television) content, which far exceeds the image quality of 2K (HDTV: High Definition Television), has been increasing. Most of the 4K content is down-converted or transcoded into 2K content and used for 2K broadcasting. The content used for this 2K broadcasting is not simply a subsampled version of 4K content, but is processed to remove quality degradation, scratches, block noise, etc. Since this processed 2K content is to be used for broadcasting, it is evaluated through careful trial screenings by humans. As a result, in addition to scratches, block noise, etc., excessive quality changes (such as color changes) are judged, and by making corrections, 2K content for broadcasting is produced. Conventionally, for videos with the same resolution, as comparison evaluation methods, the peak signal-to-noise ratio (PSNR: Peak Signal to Noise Ratio) using the difference between pixels (noise power) and the ITU-T J.144 international standard method (see Non-Patent Document 1), which is a comparison evaluation method closer to human subjective evaluation, etc. have been used.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When 2K content obtained by down-converting or transcoding 4K content is processed for broadcasting, it is possible to compare the processed 2K content with the 2K content simply thinned out from the 4K content at the same resolution using the conventional method.

[0005] However, the evaluation using the conventional method and the evaluation by human trial shooting may differ depending on the spatial complexity and temporal complexity of the content. For example, when the spatial complexity or temporal complexity is large, even if it is within the allowable range in human evaluation, it may be determined as a quality abnormality by the conventional method. In this case, it becomes necessary to further perform an evaluation by human trial shooting, which hinders the automation of quality evaluation determination. Therefore, an evaluation method close to the evaluation by human trial shooting has been desired. Therefore, an object of the present invention is to provide a video comparison evaluation apparatus and a program thereof that can adaptively change an evaluation criterion according to the complexity of video and perform comparison between videos.

Means for Solving the Problems

[0006] In order to solve the above problems, a video comparison evaluation apparatus according to the present invention is a video comparison evaluation apparatus that compares and evaluates two videos of the same content with different resolutions, with the video having a higher resolution as a reference video and the video having a lower resolution as an evaluation video, and includes a reduction conversion unit, a feature amount extraction unit, a pixel - to - pixel difference calculation unit, a quality evaluation value calculation unit, and a quality evaluation value determination unit.

[0007] In such a configuration, the video comparison evaluation apparatus generates a reduced reference video by reducing the reference video to the resolution of the evaluation video by the reduction conversion unit. Then, the video comparison evaluation apparatus extracts feature amounts serving as indicators of spatial and temporal complexities from the reduced reference video by the feature amount extraction unit. In addition, the video comparison and evaluation device calculates the difference in pixel values between pixels in units of frames or fields of images from the reduced reference video and the evaluation video by the inter-pixel difference calculation unit. Then, the video comparison and evaluation device calculates a quality evaluation value from the difference in pixel values by the quality evaluation value calculation unit. Then, the video comparison and evaluation device relaxes the criterion for determining an abnormality in the quality evaluation value as the degree of complexity based on the feature amount increases, and determines an abnormality in the quality evaluation value by the quality evaluation value determination unit. In this way, the video comparison and evaluation device can change the criterion for determining an abnormality in the quality evaluation value according to the degree of complexity of the video. As a result, if the video is complex in the spatial and temporal directions, the video comparison and evaluation device can relax the determination of an abnormality in the quality evaluation value.

[0008] In addition, in order to solve the above problems, the video comparison and evaluation device according to the present invention is a video comparison and evaluation device that compares and evaluates one of two videos with the same content as a reference video and the other as an evaluation video, and includes a feature amount extraction unit, an inter-pixel difference calculation unit, a quality evaluation value calculation unit, and a quality evaluation value determination unit.

[0009] In such a configuration, the video comparison and evaluation device extracts a feature amount that is an index of complexity in the spatial and temporal directions from the reference video by the feature amount extraction unit. In addition, the video comparison and evaluation device calculates the difference in pixel values between pixels in units of frames or fields of images from the reference video and the evaluation video by the inter-pixel difference calculation unit. Then, the video comparison and evaluation device calculates a quality evaluation value from the difference in pixel values by the quality evaluation value calculation unit. Then, the video comparison and evaluation device relaxes the criterion for determining an abnormality in the quality evaluation value as the degree of complexity based on the feature amount increases, and determines an abnormality in the quality evaluation value by the quality evaluation value determination unit. In this way, the video comparison and evaluation device can change the criterion for determining an abnormality in the quality evaluation value according to the degree of complexity of the video. As a result, when the video is complex in the spatial and temporal directions, the video comparison evaluation device can relax the abnormality determination of the quality evaluation value. Note that the video comparison evaluation device can be operated by a program for causing a computer to function as each of the above-described units.

Effect of the Invention

[0010] According to the present invention, it is possible to adaptively change the evaluation criteria according to the complexity of the video and perform a comparison between videos, and it is possible to realize an evaluation method close to the evaluation by human test shooting.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2A

Figure 2B

Figure 3

Figure 4

Figure 5A

Figure 5B

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 11

Figure 12

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [Configuration of Video Comparison Evaluation Device] First, with reference to FIG. 1, the configuration of a video comparison evaluation device 1 according to an embodiment of the present invention will be described.

[0013] The video comparison evaluation device 1 compares and evaluates two videos of the same content with different resolutions, taking the video with the higher resolution as the reference video and the video with the lower resolution as the evaluation video. Here, the reference video is a 4K video (hereinafter, 4K reference video), and the evaluation video is a 2K video (hereinafter, 2K evaluation video). It is assumed that the reference video is stored in the storage device A1 in a predetermined file format, and the evaluation video is stored in the storage device A2 in a predetermined file format. In addition, the evaluation video (2K evaluation video) is generated from a reference video (4K reference video) by the video conversion device B1. Note that the video conversion device B1 is a device that converts the resolution, such as a general down converter or transcoder.

[0014] As shown in FIG. 1, the video comparison evaluation device 1 includes a reference video decoding unit 10, a reduction conversion unit 11, an evaluation video decoding unit 12, a pixel - to - pixel difference calculation unit 13, a feature amount extraction unit 14, a quality evaluation value calculation unit 15, a quality evaluation value determination unit 16, an interference evaluation value calculation unit 17, and an interference evaluation value determination unit 18.

[0015] The reference video decoding unit 10 decodes (deciphers) a 4K reference video in a predetermined file format stored in the storage device A1. For example, the reference video decoding unit 10 decodes a 4K video in the MP4 file format and decodes it into a video composed of frames with a horizontal pixel number of 3840×a vertical pixel number of 2160 (4K[2160p]). The reference video decoding unit 10 outputs the decoded 4K reference video to the reduction conversion unit 11 frame by frame.

[0016] The reduction conversion unit 11 reduces the reference video to the resolution of the evaluation video to generate a reduced reference video. Here, the reduction conversion unit 11 predicts pixel values corresponding to the resolution of the 2K evaluation video from the pixel values of the 4K reference video and generates a reduced reference video obtained by reducing the reference video. Hereinafter, the reduced reference video is referred to as the 2K reference video. The reduction conversion unit 11 predicts the pixel values of the 2K reference video from the pixel values of the surrounding pixels for each pixel of the 4K reference video. Thereby, the reduction conversion unit 11 can accurately predict the pixel values of the 2K reference video from the 4K reference video. Here, the reduction conversion unit 11 predicts the pixel values of the 2K reference video by performing the following two - step process.

[0017] As the first step, the reduction conversion unit 11 predicts a 2K reference video of 2K[1080p] from a 4K reference video of 4K[2160p]. A specific example will be described with reference to FIG. 2A. In FIG. 2A, the ● mark indicates the pixel P of the 4K reference video, and the × mark indicates the pixel C of the 2K reference video. As shown in FIG. 2A, for each block BL of four pixels (P1 to P4) of the 4K reference video, the reduction conversion unit 11 obtains the average of the pixel values of the four pixels within the block BL, and sets it as the pixel value of the pixel C of the 2K reference video corresponding to the central position of the four pixels (P1 to P4).

[0018] As the second step, the reduction conversion unit 11 performs filtering processing on the 2K reference video predicted in the first step using filter coefficients, thereby expanding the pixel range to be referred to and further improving the accuracy. A specific example will be described with reference to FIG. 2B. In FIG. 2B, the × mark indicates the pixel of the 2K reference video predicted in the first step. Also, the pixel marked with 〇 indicates the pixel of the top field, and the pixel marked with □ indicates the pixel of the bottom field. For the pixel C of the bottom field, the reduction conversion unit 11 B uses a predetermined α filter to calculate the pixel value X of the pixel C from its own pixel value x0 and the pixel values x1 to x8 of the surrounding pixels of the pixel C according to the following formula (1). B B

[0019]

Equation

[0020] For the pixel C of the top field, the reduction conversion unit 11 T uses a predetermined α filter to calculate the pixel value Y of the pixel C from its own pixel value x7 and the pixel values x4, x0, x5 to x11 of the surrounding pixels of the pixel C according to the following formula (2). T T

[0021]

Equation

[0022] Note that the filter coefficient α in Formula (1) and Formula (2) preferably has a value of about 0.6 according to the experimental results. This is because the value of PSNR calculated based on the difference calculated by the pixel difference calculation unit 13 described later is maximized (less deterioration) at α = 0.6 through experiments, and the result shows that it decreases as α approaches 0.0 or 1.0 from 0.6. That is, it is preferable to use the filter coefficient α as the value at the deterioration branch point. As a result, the downscaling conversion unit 11 can generate a 2K reference video with high prediction accuracy. Returning to FIG. 1, the description of the configuration of the video comparison evaluation apparatus 1 will be continued.

[0023] The downscaling conversion unit 11 outputs the generated 2K reference video in units of frame or field images to the pixel difference calculation unit 13, the feature amount extraction unit 14, and the quality evaluation value calculation unit 15. Hereinafter, a frame or field of the 2K reference video will be referred to as a 2K reference image.

[0024] The evaluation video decoding unit 12 decodes (deciphers) a 2K evaluation video in a predetermined file format stored in the storage device A2. For example, the evaluation video decoding unit 12 decodes a 2K video in the MP4 file format and decodes it into a video composed of frames (2K [1080p]) with a horizontal pixel number of 1920 × a vertical pixel number of 1080. The evaluation video decoding unit 12 outputs the decoded 2K evaluation video to the pixel difference calculation unit 13 in units of frame or field images. Hereinafter, a frame or field of the 2K evaluation video will be referred to as a 2K evaluation image.

[0025] The pixel difference calculation unit 13 calculates the difference between pixel values between pixels in units of frame or field images from the downscaled reference video and the evaluation video. Here, the pixel - to - pixel difference calculation unit 13 inputs the reduced reference video (2K reference video) generated by the reduction conversion unit 11 in units of frame or field images (2K reference images), and inputs the evaluation video (2K evaluation video) decoded by the evaluation video decoding unit 12 in units of frame or field images (2K evaluation images). Note that for the two inputs, if one is a frame, the other is also a frame, and if one is a field, the other is also a field. The pixel - to - pixel difference calculation unit 13 calculates the difference in pixel values between the same pixel positions in the input 2K reference image and 2K evaluation image. The pixel - to - pixel difference calculation unit 13 outputs the difference value of pixel values for each pixel to the quality evaluation value calculation unit 15 and the interference evaluation value calculation unit 17.

[0026] The feature amount extraction unit 14 extracts feature amounts that are indicators of complexity in the spatial and temporal directions from the reduced reference video (2K reference video) generated by the reduction conversion unit 11. The feature amount extraction unit 14 extracts a feature amount (spatial feature amount) that is an indicator of spatial complexity and a feature amount (temporal feature amount) that is an indicator of temporal complexity from the 2K reference video. Here, the feature amount extraction unit 14 includes an edge detection unit 140 and a zero - crossing detection unit 141.

[0027] The edge detection unit 140 detects edges (contours) that are indicators of spatial complexity for each frame or field (2K reference image) of the 2K reference video, and extracts the number of edges as a spatial feature amount, which is one of the feature amounts. The edge detection unit 140 detects pixels with a pixel value difference of a predetermined threshold or more between adjacent pixels in the 2K reference image as edges.

[0028] A specific example will be described with reference to FIG. 3. The edge detection unit 140 divides the 2K reference image F S into blocks B of a predetermined size, and detects edges for each block B. For example, the edge detection unit 140 divides the 2K reference image F SIt is divided into a total of 256 small - area blocks B with 16 horizontal divisions and 16 vertical divisions. Here, for the 2K reference image F S at time t, the pixel value of the coordinates (x, y) (1 ≤ x ≤ X, 1 ≤ y ≤ Y) within the block B at the divided horizontal position h (1 ≤ h ≤ H) and vertical position v (1 ≤ v ≤ V) is denoted as S(t, x, y, v, h). Also, let the predetermined threshold values be Th1 and Th2. Here, when |S(t, x, y, v, h)-S(t, x - 1, y, v, h)|≥Th1, EH(t, x, y, v, h)=1; otherwise, EH(t, x, y, v, h)=0. Also, when |S(t, x, y, v, h)-S(t, x, y - 1, v, h)|≥Th2, EV(t, x, y, v, h)=1; otherwise, EH(t, x, y, v, h)=0.

[0029] Thus, in EH(t, x, y, v, h), "1" is set when an edge is detected in the horizontal direction at the corresponding pixel, and "0" is set when no edge is detected. Also, in EV(t, x, y, v, h), "1" is set when an edge is detected in the vertical direction at the corresponding pixel, and "0" is set when no edge is detected. And the edge detection unit 140 calculates the number of edges EB(t, v, h) within the block B at time t, horizontal position h, and vertical position v according to the following formula (3).

[0030]

Equation

[0031] The edge detection unit 140 calculates the number of edges for all blocks B at time t. Here, the edge detection unit 140 obtains the average number of edges per block B by dividing the sum of the number of edges of all blocks B by the number of blocks B (V×H), and uses it as a feature amount (spatial feature amount) that is an index of the complexity in the spatial direction. Returning to FIG. 1, the configuration of the video comparison evaluation apparatus 1 will be further described. The edge detection unit 140 outputs the detected spatial feature amount to the quality evaluation value determination unit 16.

[0032] The zero-crossing detection unit 141 detects zero-crossings indicating fluctuations in pixel values in the time direction for each pixel from the time-series frames or fields (2K reference images) of the 2K reference video, and extracts the number of zero-crossings as a time feature amount, which is one of the feature amounts. The zero-crossing detection unit 141 detects, as zero-crossings, pixels showing a rapid change equal to or greater than a predetermined threshold value between two adjacent 2K reference images in the time direction.

[0033] A specific example will be described with reference to FIGS. 4, 5A, and 5B. As shown in FIG. 4, the zero-crossing detection unit 141 detects zero-crossings for each block B of a predetermined size in the 2K reference images F continuous in time series. Note that the block B has the same size as the block used by the edge detection unit 140. S

[0034] First, the zero-crossing detection unit 141 accumulates, within the block B, pixels whose differential values (difference values) of pixel values in the time direction vary by a predetermined threshold value or more. Here, let the cumulative number of pixels whose pixel values have varied by the threshold value or more at a certain time t, horizontal position h, and vertical position v (see FIG. 3) in the block B be B(t, h, v). Then, the zero-crossing detection unit 141 calculates the inter-frame acceleration by second-order differentiating (|2B(t, h, v) - B(t - 1, h, v) - B(t + 1, h, v)|) in the time direction the cumulative number B(t, h, v) of the blocks B at the same position continuous in the time direction.

[0035] As shown in FIG. 5A, the zero-crossing detection unit 141 accumulates the number of blocks B (the number of zero-crossings) in which the inter-frame acceleration changes from “+” to “-” or from “-” to “+”. Note that, as shown in FIG. 5B, there may be a case where no zero-crossing exists in a video with little motion. Here, the zero-crossing detection unit 141 obtains the average number of zero-crossings per block B by dividing the sum of the number of all zero-crossings at a certain time by the number of blocks B (V×H), and uses it as a feature amount (temporal feature amount) that is an index of the complexity in the temporal direction. Returning to FIG. 1, the description of the configuration of the video comparison evaluation apparatus 1 will be continued. The zero-crossing detection unit 141 outputs the detected temporal feature amount to the quality evaluation value determination unit 16.

[0036] The quality evaluation value calculation unit 15 calculates a quality evaluation value that is a measure of the quality of the 2K evaluation image from the 2K reference image generated by the reduction conversion unit 11 and the difference in pixel values for each pixel between the 2K reference image calculated by the pixel difference calculation unit 13 and the 2K evaluation image. A general image comparison evaluation value may be used as the quality evaluation value. For example, the quality evaluation value calculation unit 15 can calculate the quality evaluation value using DSCQS (Double-stimulus continuous quality-scale) according to the KDDI method defined in ITU-T J.144 (see Non-Patent Document 1), PSNR (Peak Signal to Noise Ratio), etc.

[0037] When PSNR is used as the quality evaluation value, the quality evaluation value calculation unit 15 calculates PSNR from the difference value for each pixel between the 2K reference image calculated by the pixel difference calculation unit 13 and the 2K evaluation image without using the 2K reference image generated by the reduction conversion unit 11. Here, the quality evaluation value calculation unit 15 uses DSCQS, which gives an evaluation result closer to human subjective evaluation, as the quality evaluation value. The quality evaluation value calculation unit 15 outputs the calculated quality evaluation value to the quality evaluation value determination unit 16.

[0038] The quality evaluation value determination unit 16 determines an abnormality of the quality evaluation value by relaxing the criterion for determining an abnormality of the quality evaluation value as the degree of complexity based on the feature amount extracted by the feature amount extraction unit 14 increases. That is, the quality evaluation value determination unit 16 changes a threshold value for determining whether the quality evaluation value corresponds to a quality abnormality according to the feature amount. Generally, a video that includes many edges (contours) in the spatial direction and has complex (irregular and high-speed) movement in the temporal direction has a high load on down-conversion and transcoding in the video conversion device B1, and even in normal operation, the DSCQS shows a high value.

[0039] Therefore, when the DSCQS is a significantly high value based on the spatial feature amount and the temporal feature amount of the 2K reference image extracted by the feature amount extraction unit 14, the quality evaluation value determination unit 16 relaxes (increases) the threshold value for determining a quality abnormality. Specifically, the quality evaluation value determination unit 16 calculates the following threshold value (DSCQS_Th) as a threshold value for determining the DSCQS.

[0040]

Equation

[0041] Here, TH0 is a fixed value “16” (TH0 = 16). When CE is the average number of edges, which is the spatial feature amount input from the edge detection unit 140 to E, CE = 0 when E ≤ THE = 330, and CE = 0.057 × E - 19 when E > 330 = THE. When CZ is the average number of zero crossings, which is the temporal feature amount input from the zero crossing detection unit 141 to Z, CZ = 0 when Z ≤ THZ = 800, and CZ = 0.006 × Z - 4 when Z > THZ = 800. Note that THE is a predetermined threshold value for determining whether a 2K reference image contains many edges, and THZ is a predetermined threshold value for determining whether a 2K reference image contains many zero crossings.

[0042] The quality evaluation value determination unit 16 determines whether the quality evaluation value calculated by the quality evaluation value calculation unit 15 is a value indicating a quality abnormality based on the threshold value calculated by the formula (4). Here, when the quality evaluation value (DSCQS) exceeds the threshold value (DSCQS_Th), the quality evaluation value determination unit 16 determines that the frame or field is of abnormal quality, and displays the occurrence of the abnormality to the outside by an error message, an alarm, or the like.

[0043] Here, with reference to FIG. 6, the control of the determination by the quality evaluation value determination unit 16 will be described. FIG. 6(a) shows the time on the horizontal axis and the CE (average number of edges) and CZ (average number of zero crossings) on the vertical axis. Here, FIG. 6(a) shows a state where CE and CZ exceed their respective threshold values THE and THZ as time elapses. FIGS. 6(b) and (c) show the time on the horizontal axis and the quality evaluation value (DSCQS) on the vertical axis.

[0044] FIG. 6(b) shows a state where DSCQS increases as time elapses, but DSCQS is less than the threshold value (DSCQS_Th) at all time points. In this case, the quality evaluation value determination unit 16 determines that there is no abnormality in quality even if the average number of edges (CE) is large and the image is complex in the spatial direction, and the average number of zero crossings (CZ) is large and the image is complex in the time direction.

[0045] FIG. 6(c) shows a state where both DSCQS and DSCQS_Th increase as time elapses, and DSCQS becomes equal to or greater than DSCQS_Th at a certain time t1. In this case, the quality evaluation value determination unit 16 determines that an abnormality has occurred in quality after time t1. In this way, the quality evaluation value determination unit 16 can vary the determination criteria for quality evaluation according to the feature amount of the frame or field. Returning to FIG. 1, the description of the configuration of the video comparison evaluation apparatus 1 will be continued.

[0046] The interference evaluation value calculation unit 17 calculates an evaluation value for each local area including the pixel from the difference in pixel values for each pixel between the 2K reference image and the 2K evaluation image calculated by the pixel difference calculation unit 13, and calculates the deviation value of the evaluation value as an interference evaluation value that serves as a measure for interfering with the quality. Even when the quality evaluation value determination unit 16 does not determine a quality abnormality, the interference evaluation value calculation unit 17 calculates an evaluation value (interference evaluation value) for determining pixel errors such as locally occurring scratches and block noise.

[0047] Examples of pixel errors to be determined are shown in FIGS. 7 and 8. FIG. 7(b) shows an image obtained by pseudo-inserting a scratch S (2×2 pixels) into the 4K reference video in FIG. 7(a) and converting (down-converting / transcoding) it into a 2K evaluation video. FIG. 8(b) shows an image obtained by pseudo-inserting block noise N into the 4K reference video in FIG. 8(a) and converting (down-converting / transcoding) it into a 2K evaluation video. The interference evaluation value calculation unit 17 targets scratches and block noise that occur after conversion, such as in FIGS. 7(b) and 8(b), as the determination targets for pixel errors. Needless to say, the 4K reference video input to the video comparison evaluation device 1 is a clean video without scratches or block noise inserted.

[0048] Here, the interference evaluation value calculation unit 17 statistically analyzes an evaluation value (for example, here, the mean square error [MSE]) with a local area of 8×8 pixel size as a unit from a single pixel, and sets the deviation value of the evaluation value per local area as the interference evaluation value. This interference evaluation value serves as an indicator showing how far the evaluation value of the local area is from the value within the frame or field.

[0049] Here, with reference to FIG. 9, the calculation method of the interference evaluation value of the interference evaluation value calculation unit 17 will be described. As shown in FIG. 9, the interference evaluation value calculation unit 17 has a 2K reference image F S and a 2K evaluation image F EThe difference value for each pixel with respect to [object] is input. Here, the 2K reference image F S and the 2K evaluation image F E shall have an image size of X×Y pixels, and the size of the local region R shall be N×N pixels. Also, let the difference value be d(x,y) (1≦x≦X - N + 1, 1≦y≦Y - N + 1). First, the interference evaluation value calculation unit 17 calculates the mean squared error (MSE) in the local region R according to the following formula (5) by shifting one pixel and one line in the horizontal direction and the vertical direction respectively.

[0050]

Equation

[0051] Then, the interference evaluation value calculation unit 17 averages the MSE in units of local regions calculated by shifting one pixel and one line in the horizontal direction and the vertical direction respectively according to the following formula (6) to calculate the average MSE.

[0052]

Equation

[0053] Furthermore, the interference evaluation value calculation unit 17 calculates the variance V shown in the following formula (7) from the MSE in units of local regions shown in formula (5) and the average MSE shown in formula (6).

[0054]

Equation

[0055] Then, the interference evaluation value calculation unit 17 calculates the deviation value D according to the following formula (8) for each coordinate (x,y).

[0056]

Equation

[0057] As a result, the interference evaluation value calculation unit 17 can calculate the interference evaluation value (deviation value D) for each pixel. Returning to FIG. 1, the description of the configuration of the video comparison evaluation apparatus 1 will be continued.

[0058] The interference evaluation value calculation unit 17 calculates the interference evaluation value according to the type of pixel error. For example, when the pixel error is a scratch, it often continuously occurs at the same position over a plurality of frames or fields with a pixel count of less than approximately 4×4 pixels. Therefore, the interference evaluation value calculation unit 17 sets N described in FIG. 9 to less than 4 and calculates the interference evaluation value for scratch detection. Also, when the pixel error is block noise, since it occurs with a certain area of approximately 4×4 pixels or more, the interference evaluation value calculation unit 17 sets N described in FIG. 9 to 4 or more and calculates the interference evaluation value for block noise detection. The interference evaluation value calculation unit 17 outputs the calculated interference evaluation value for each pixel to the interference evaluation value determination unit 18.

[0059] The interference evaluation value determination unit 18 determines whether a pixel error has occurred from the interference evaluation value for each pixel calculated by the interference evaluation value calculation unit 17. Here, the interference evaluation value determination unit 18 compares the interference evaluation value for scratch detection calculated by the interference evaluation value calculation unit 17 with a predetermined threshold value (first threshold value) for each pixel, and when the number of consecutive frames or fields that become equal to or greater than the threshold value at the same pixel position is equal to or greater than a predetermined threshold value (second threshold value), it determines that a scratch has occurred as a pixel error.

[0060] Also, the interference evaluation value determination unit 18 compares the interference evaluation value for block noise detection calculated by the interference evaluation value calculation unit 17 with a predetermined threshold value (first threshold value) for each pixel, and when the number of pixels that become equal to or greater than the threshold value within the same frame or the same field is equal to or greater than a predetermined threshold value (third threshold value), it determines that block noise has occurred as a pixel error. In this way, since the interference evaluation value determination unit 18 uses the deviation value as the interference evaluation value, a fixed threshold value may be used regardless of the differences in the characteristics of the video. The interference evaluation value determination unit 18 determines the interference evaluation value, and when detecting the occurrence of a pixel error, it displays the occurrence of the pixel error to the outside by an error message, an alarm, or the like.

[0061] Here, with reference to FIGS. 10A and 10B, the control of the determination by the interference evaluation value determination unit 18 will be described. FIG. 10A(a) shows the transition of the average PSNR and the minimum PSNR with time on the horizontal axis and PSNR (dB) on the vertical axis. The average PSNR is obtained by converting the average MSE calculated by Equation (6) into PSNR with decibels (dB) as the unit of the peak signal-to-noise ratio. The minimum PSNR is obtained by converting the minimum value of the MSE for each pixel calculated by Equation (5) into PSNR with decibels (dB) as the unit of the peak signal-to-noise ratio. FIG. 10A(b) shows the deviation value on the horizontal axis and time on the vertical axis, and shows the transition of the deviation value of the minimum PSNR in FIG. 10A(a). As shown in FIG. 10A(a), there are cases (A1, A2, A3) where the minimum PSNR partially drops significantly and falls below the threshold ThP (for example, 20 dB). At this time, as shown in FIG. 10A(b), the interference evaluation value determination unit 18 determines A1 and A2 as scratches and block noise from the pixels where the deviation value jumps significantly and becomes equal to or greater than the threshold ThD. For A3 that is below the threshold ThP, since the deviation value is less than the threshold ThD, the interference evaluation value determination unit 18 determines it as normal.

[0062] FIG. 10B is the same graph as FIG. 10A, but the video to be evaluated is different. As shown in FIG. 10B(a), the average PSNR cannot be said to be a good value compared to FIG. 10A(a), and furthermore, the minimum PSNR is below the threshold ThP almost throughout all frames. However, since the deviation value is much lower than the threshold ThD as shown in FIG. 10B(b), the interference evaluation value determination unit 18 does not determine it as a pixel error. In this way, for a video that contains many edges in the spatial direction and involves complex motion in the temporal direction, even when the average PSNR or the minimum PSNR is low, the interference evaluation value determination unit 18 can make a normal determination without misdetecting pixel errors.

[0063] With the configuration described above, the video comparison evaluation apparatus 1 can adaptively change the criteria for quality evaluation according to the characteristics of the video and compare and evaluate the video. In addition, the video comparison evaluation apparatus 1 can accurately detect pixel errors even for a video that changes complexly in the spatial direction or the temporal direction. In this way, the video comparison evaluation apparatus 1 can perform evaluation at the frame or field unit and evaluation at the pixel unit together. This video comparison evaluation apparatus 1 can be operated by a program for causing a computer (not shown) to function as each of the above-described units.

[0064] [Operation of Video Comparison Evaluation Apparatus] Next, with reference to FIG. 11 (appropriately refer to FIG. 1 for the configuration), the operation of the video comparison evaluation apparatus 1 according to the embodiment of the present invention will be described.

[0065] In step S1, the reference video decoding unit 10 decodes a 4K reference video in a predetermined file format stored in the storage device A1. In step S2, the downscaling conversion unit 11 predicts, for each pixel in the 4K reference video decoded in step S1, a pixel value corresponding to the resolution of the evaluation video (2K evaluation video) from the pixel values of the surrounding pixels, and generates a 2K reference video. Here, the downscaling conversion unit 11 generates a 2K reference video by using the pixel value average of 2×2 pixels of the 4K reference video as the pixel value of the pixel of the 2K reference video and further performing filtering processing using a filter coefficient. In step S3, the evaluation video decoding unit 12 decodes a 2K evaluation video in a predetermined file format stored in the storage device A2. In step S4, the pixel difference calculation unit 13 calculates the difference between pixels in the 2K reference image and the 2K evaluation image, which are the respective frames or fields in the 2K reference video generated in step S2 and the 2K evaluation video decoded in step S3.

[0066] In step S5, the feature extraction unit 14 extracts features in units of frames or fields (2K reference images) in the 2K reference video generated in step S2. Here, the feature extraction unit 14 detects edges (contours) for each block of a predetermined size in the 2K reference image by the edge detection unit 140, obtains the average number of edges, and uses it as a feature amount (spatial feature amount) that is an index of the complexity in the spatial direction. Also, the feature extraction unit 14 detects zero crossings for each block of a predetermined size in the 2K reference images consecutive in time series by the zero crossing detection unit 141, obtains the average number of zero crossings, and uses it as a feature amount (temporal feature amount) that is an index of the complexity in the temporal direction.

[0067] In step S6, the quality evaluation value calculation unit 15 calculates, for each frame or field, a quality evaluation value that is a measure of the quality of the 2K evaluation video from the 2K reference images constituting the 2K reference video generated in step S2 and the difference in pixel values for each pixel between the 2K reference image and the 2K evaluation image calculated in step S4. Here, the quality evaluation value calculation unit 15 calculates DSCQS, PSNR, etc. as the quality evaluation value.

[0068] In step S7, the quality evaluation value determination unit 16 varies a threshold that is a criterion for performing quality evaluation based on the feature amounts (spatial feature amount and temporal feature amount) extracted in step S5, and determines whether the quality evaluation value calculated in step S6 corresponds to a quality abnormality. Here, the quality evaluation value determination unit 16 relaxes the threshold for determining a quality abnormality when the 2K reference video contains many edges in the spatial direction and complex motion in the temporal direction in units of frames or fields according to the feature amounts. Then, when the quality evaluation value determination unit 16 determines that the quality evaluation value corresponds to a quality abnormality based on the fluctuation threshold value, it displays the quality abnormality to the outside by an error message, an alarm, etc. (not shown as a step).

[0069] In step S8, the interference evaluation value calculation unit 17 calculates an interference evaluation value, which is a measure for interfering with the quality of the 2K evaluation video, for each frame or field from the difference in pixel values for each pixel between the 2K reference image and the 2K evaluation image calculated in step S4. Here, the interference evaluation value calculation unit 17 calculates the evaluation value for each pixel by shifting pixel by pixel in units of local regions of a predetermined size, and sets the deviation value of the evaluation value within the frame or field as the interference evaluation value. Note that a plurality of sizes are set for the local region according to the type of pixel error to be evaluated (scratches, block noise, etc.). Then, the interference evaluation value calculation unit 17 calculates the interference evaluation value for each type of pixel error.

[0070] In step S9, the interference evaluation value determination unit 18 determines whether a pixel error has occurred based on a predetermined fixed threshold value from the interference evaluation value for each pixel calculated in step S8. Then, when the interference evaluation value determination unit 18 determines that the interference evaluation value is abnormal based on the fixed threshold value, it displays the pixel error to the outside by an error message, an alarm, etc. (not shown as a step).

[0071] By the operations described above, the video comparison evaluation apparatus 1 can adaptively change the determination criteria for quality evaluation according to the characteristics of the video and compare and evaluate the video. In addition, the video comparison evaluation apparatus 1 can accurately detect pixel errors even for videos that change complexly in the spatial direction and the temporal direction.

[0072] Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and design changes and the like within the scope not departing from the gist of the present invention are also included.

[0073] Here, the video comparison evaluation device 1 is configured to compare and evaluate a reference video with evaluation videos having different resolutions that are down-converted or transcoded from the reference video. However, the present invention may be configured to evaluate videos having the same resolution.

[0074] For example, it may be configured as a video comparison evaluation device 1B shown in FIG. 12 in which the downscaling conversion unit 11 is omitted from the video comparison evaluation device 1 of FIG. 1. Since the configuration within the video comparison evaluation device 1B excluding the downscaling conversion unit 11 is the same as the configuration described for the video comparison evaluation device 1, the description thereof will be omitted. The video comparison evaluation device 1B is configured to compare and evaluate two videos having the same resolution. The video comparison evaluation device 1B uses one video as a reference video (for example, a 4K reference video) and the other video as an evaluation video having the same resolution as the reference video (for example, a 4K evaluation video). The reference video and the evaluation video are generated from videos having the same content. For example, the reference video and the evaluation video are those obtained by encoding the same video with different encoding methods, compressing it with different compression methods, etc. In this case, it is assumed that the evaluation video has a lower encoding efficiency, a lower compression efficiency, etc. than the reference video, and thus video degradation is assumed with respect to the reference video.

[0075] In this way, similar to the video comparison evaluation device 1, the video comparison evaluation device 1B can adaptively change the criteria for quality evaluation according to the characteristics of the video and compare and evaluate the video. Also, similar to the video comparison evaluation device 1, the video comparison evaluation device 1B can accurately detect pixel errors even for videos that change complexly in the spatial or temporal direction.

[0076] Here, the video comparison evaluation devices 1 and 1B (FIGS. 1 and 12) are configured to include a reference video decoding unit 10 and an evaluation video decoding unit 12. However, when a video that has already been decoded is input from the outside, the reference video decoding unit 10 and the evaluation video decoding unit 12 may be omitted from the configuration.

[0077] Alternatively, from the video comparison evaluation devices 1 and 1B (Figs. 1 and 12), the interference evaluation value calculation unit 17 and the interference evaluation value determination unit 18 may be omitted, and a simple configuration may be adopted in which the determination of pixel errors is omitted.

Description of Reference Numerals

[0078] 1, 1B Video comparison evaluation device 10 Reference video decoding unit 11 Reduction conversion unit 12 Evaluation video decoding unit 13 Inter-pixel difference calculation unit 14 Feature extraction unit 140 Edge detection unit 141 Zero-crossing detection unit 15 Quality evaluation value calculation unit 16 Quality evaluation value determination unit 17 Interference evaluation value calculation unit 18 Interference evaluation value determination unit

Claims

1. A video comparison and evaluation device that compares and evaluates two videos of the same content with different resolutions, using the video with the higher resolution as the reference video and the video with the lower resolution as the evaluation video, comprising: A reduction conversion unit that reduces the reference video to the resolution of the evaluation video to generate a reduced reference video; A feature amount extraction unit that extracts a feature amount serving as an index of complexity in the spatial and temporal directions from the reduced reference video; A pixel difference calculation unit that calculates the difference in pixel values between pixels in units of frames or fields of images from the reduced reference video and the evaluation video; A quality evaluation value calculation unit that calculates a quality evaluation value from the difference in pixel values; A quality evaluation value determination unit that relaxes the criterion for determining an abnormality in the quality evaluation value as the degree of complexity based on the feature amount increases, and determines the abnormality in the quality evaluation value; A video comparison and evaluation device, characterized by comprising the above.

2. A noise evaluation value calculation unit that calculates an evaluation value for each local area including the pixel for each pixel from the difference in pixel values, and calculates a deviation value of the evaluation value in the image as a noise evaluation value; A noise evaluation value determination unit that determines a pixel error based on a preset threshold value of the noise evaluation value; The video comparison and evaluation device according to claim 1, further comprising the above.

3. The feature amount extraction unit An edge detection unit that detects edges serving as an index of complexity in the spatial direction of the image from the reduced reference video, and extracts the number of the edges as a spatial feature amount that is one of the feature amounts; A zero-crossing detection unit that detects zero-crossings indicating fluctuations in pixel values in the temporal direction for each pixel from the reduced reference video, and extracts the number of the zero-crossings as a temporal feature amount that is one of the feature amounts; The video comparison and evaluation device according to claim 1, characterized by comprising the above.

4. A video comparison and evaluation device that compares and evaluates one of two videos of the same content as the reference video and the other as the evaluation video, comprising: A feature amount extraction unit that extracts a feature amount serving as an index of complexity in the spatial and temporal directions from the reference video; A pixel difference calculation unit that calculates the difference in pixel values between pixels in units of frames or fields of images from the reference video and the evaluation video; A quality evaluation value calculation unit that calculates a quality evaluation value from the difference in pixel values; A quality evaluation value determination unit that relaxes the criterion for determining an abnormality in the quality evaluation value as the degree of complexity based on the feature amount increases, and determines the abnormality in the quality evaluation value; A video comparison and evaluation device, characterized by comprising the above.

5. A noise evaluation value calculation unit that calculates an evaluation value for each local region including the pixel for each pixel from the difference in the pixel values, and calculates a deviation value in the image of the evaluation value as a noise evaluation value; A noise evaluation value determination unit that determines a pixel error based on a preset threshold value of the noise evaluation value; The video comparison evaluation apparatus according to claim 4, further comprising:

6. The feature amount extraction unit includes: An edge detection unit that detects an edge serving as an index of the complexity in the spatial direction of the image from the reference video, and extracts the number of the edges as a spatial feature amount that is one of the feature amounts; A zero-crossing detection unit that detects a zero-crossing indicating a change in the pixel value in the time direction for each pixel from the reference video, and extracts the number of the zero-crossings as a time feature amount that is one of the feature amounts; The video comparison evaluation apparatus according to claim 4, characterized by comprising:

7. A program for causing a computer to function as the video comparison evaluation apparatus according to any one of claims 1 to 6.

Images