VIDEO QUALITY ESTIMATION DEVICE, VIDEO QUALITY ESTIMATION METHOD, AND PROGRAM

Abstract

In this video quality estimating device for estimating a video quality of a video having a screen layout in which parts are arranged estimates the video quality of the video on the basis of the video quality of each part constituting the video, and a layout resolution of each part on the screen layout.

Description

[Technical field]

[0001] The present invention relates to video quality estimation. [Background technology]

[0002] The market size of web conferencing services is expanding due to the spread of remote work and distance learning. For web conferencing service providers, it is important that users are satisfied and continue to use the service, but it is known that a decline in the video quality of a web conferencing service leads to a decline in user satisfaction.

[0003] Non-Patent Document 1 discloses a technology for video distribution services that aims to improve user satisfaction (QoE: Quality of Experience) by estimating video quality from encoding information and visualizing the QoE to focus on improving areas where QoE has deteriorated. A technology is proposed for estimating video quality based on encoding information such as the encoding method, distribution bit rate, resolution, and frame rate. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Takanori Hayashi, "QoE-centric Operation for Optimizing User Quality of Experience," NTT Technical Journal, vol. 27, no. 7, pp. 16-19, July 2015. Summary of the Invention [Problem to be solved by the invention]

[0005] Similarly, for Web conferencing services, a video quality estimation technology for Web conferencing services is required to visualize the QoE in order to improve it. However, the video displayed on a terminal during a Web conference has a screen layout with parts arranged on it, and has different characteristics from the video displayed on a terminal in a video distribution service that does not have such a screen layout. Therefore, if conventional video quality estimation technology (video quality estimation model) for video distribution services is applied to Web conferencing video, it is not possible to estimate the quality with high accuracy.

[0006] Note that the above-mentioned issues are not limited to video from web conferences, but can occur with any video that has a screen layout with parts arranged in it.

[0007] The present invention has been made in consideration of the above-mentioned points, and has an object to provide a technique for estimating with high accuracy the quality of a video having a screen layout in which parts are arranged. [Means for solving the problem]

[0008] According to the disclosed technology, there is provided a video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: Estimating the image quality of the image based on the image quality of each part constituting the image and the layout resolution of each part on the screen layout. An apparatus for estimating video quality is provided. Effect of the Invention

[0009] According to the disclosed technology, it is possible to estimate with high accuracy the quality of a video having a screen layout in which parts are arranged, for example, the video quality in a web conference. [Brief description of the drawings]

[0010] [Figure 1] 1 is an overall configuration diagram of a system according to an embodiment of the present invention. [Diagram 2]FIG. 13 is a diagram showing an example of a screen layout for a Web conference video. [Diagram 3] FIG. 13 is a diagram showing an example of a screen layout in which parts are arranged. [Figure 4] FIG. 13 is a diagram showing an example of a screen layout in which parts are arranged. [Diagram 5] FIG. 13 is a diagram showing an example of a screen layout in which parts are arranged. [Figure 6] FIG. 13 is a diagram showing an example of a screen layout synthesis process image of a Web conference video. [Figure 7] 11A and 11B are diagrams illustrating examples of encoding characteristics of a face video and a material video. [Figure 8] 13A and 13B are diagrams illustrating an example of the effect on image quality caused by shrinking a part. [Figure 9] 11A and 11B are diagrams illustrating an example of the relationship between the area ratio of each face and image quality for each bit rate range. [Figure 10] FIG. 13 is a diagram illustrating an example of the relationship between part area and Web conference video quality. [Figure 11] FIG. 13 is a diagram illustrating an example of the relationship between part area and Web conference video quality. [Figure 12] 1 is a diagram illustrating an example of the configuration of a Web conference video quality estimation device according to an embodiment of the present invention. [Figure 13] FIG. 1 is a diagram illustrating an example of an overall processing flow of a Web conference video quality estimation device. [Figure 14] FIG. 13 is a diagram illustrating an example of a process flow for estimating video quality for each part taking into account reduction. [Figure 15] FIG. 13 is a diagram showing an example of the configuration of a Web conference video quality estimation device in Modification 1. [Figure 16] FIG. 11 is a diagram showing an example of the configuration of a Web conference video quality estimation device in Modification 2. [Figure 17] FIG. 13 is a diagram showing an example of the configuration of a Web conference video quality estimation device in Modification 3. [Figure 18] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention (the present embodiment) will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0012] In the following, an embodiment of the present invention will be described mainly using a video of a Web conference as an example of an application field of the present invention, but the technology according to the present embodiment is not limited to a video of a Web conference and can be applied to any video having a screen layout with parts arranged. Also, the "Web conference video quality estimation device" described in the following embodiment may be called a "video quality estimation device."

[0013] (System Configuration) An example of the overall configuration of a system according to an embodiment of the present invention is shown in Fig. 1. As shown in Fig. 1, the system includes a plurality of terminals 10 (which may be called clients), a Web conference video quality estimation device 100, and a conference server 200.

[0014] The terminal 10 is a general terminal on which a Web browser runs, such as a PC or a smartphone. The conference server 200 is a server having a server function for a Web conference, and in this embodiment, it is assumed to have a server function of a WebRTC SFU. A Web conference is held between a plurality of terminals 10 under the control of the conference server 200. When the technology according to this embodiment is applied to video other than a Web conference, the conference server 200 is replaced with, for example, a video distribution server that distributes video other than the conference video.

[0015] The Web conference video quality estimation device 100 is an example of a device that implements the technology according to the present invention, and estimates the quality of a Web conference video displayed on the terminal 10. The Web conference video quality estimation device 100 may be a standalone device, a function built into the terminal 10, or a function built into the conference server 200. When the function of the Web conference video quality estimation device 100 is built into the terminal 10, the terminal 10 may be called the Web conference video quality estimation device 100. When the function of the Web conference video quality estimation device 100 is built into the conference server 200, the conference server 200 may be called the Web conference video quality estimation device 100.

[0016] (About the assignment, about the example of the screen layout) As mentioned above, the video displayed on a terminal during a Web conference is different from the video displayed on a terminal in a video distribution service for video that does not have a screen layout with parts arranged on it, so if a conventional video quality estimation technology (video quality estimation model) for such video distribution services is applied to Web conference video, it is not possible to estimate the quality with high accuracy. More specifically, there are the following two issues regarding this point.

[0017] The first issue is that the video displayed on a terminal during a web conference is limited, unlike the video displayed on a video distribution service that does not have a screen layout with parts arranged in it.In the case of such video distribution services, various videos are displayed, so video quality estimation technology is built to reflect the coding characteristics of various videos with different image resolution and movement size.

[0018] However, the video displayed in a web conference is limited, and what is displayed primarily consists of the participants' facial videos and documentary video. Facial videos contain a lot of movement, such as nodding and gesturing. On the other hand, documentary video only shows movement when pages are turned, and there are many periods of still images. As such, facial videos and documentary video have different characteristics, so in order to estimate video quality with greater accuracy, a quality estimation model that reflects the coding characteristics of facial videos and documentary video is required. Note that facial videos and documentary video are both examples of video parts.

[0019] The second issue concerns screen layout. Figure 2 shows examples of screen layouts for web conference videos. In web conferences, as shown in Figure 2, in addition to layouts where faces and documents are displayed across the screen ((a) and (b)), there are various layouts that combine facial images and document images, such as layouts where multiple facial images are displayed in a grid ((c) and (d)), and layouts where the main speaker's facial image and document images are displayed large with the participants' facial images displayed smaller at the top ((e) and (f)).

[0020] In layouts that combine facial footage and archive footage, the various parts of the facial footage and archive footage are reduced in size and then integrated to display as a single image.

[0021] All of the screen layouts shown in Figure 2 are examples of screen layouts intended for web conference video.

[0022] As described above, the technology according to the present embodiment can be applied to images other than Web conference images. That is, the technology according to the present embodiment can be applied to a screen layout in which images other than face images and document images are arranged as parts.

[0023] For example, the technology according to the present embodiment is also applicable to a screen layout in which multiple streaming videos are simultaneously displayed on one screen, allowing the user to simultaneously watch multiple streaming videos. Examples of such streaming videos include game play-by-plays, idol concerts, sports videos, etc. Examples of screen layouts are shown in Figs. 3 to 5. Note that Figs. 3 to 5 are examples of screen layouts, and the present invention is not limited to these.

[0024] Figure 3 shows an example of a screen layout for an idol concert. In this example, it is possible to simultaneously watch four videos of the stage taken from four angles on four screens. Each of the four videos of the stage in this example is a part.

[0025] Figure 4 shows an example of a screen layout for sports videos. In this example, the screen is divided into five parts, each of which displays footage of a match being played at a different venue across the country, and users can watch these videos simultaneously. Each of the five game screens in this example is a part.

[0026] FIG. 5 shows another example of a screen layout for sports videos. In this example, it is possible to simultaneously watch four videos of the field taken from four angles on four screens. In addition, in an area separate from the game screen, for example, the name of a country, information about its team, the score, etc. are displayed. In this example, each game screen and the screen showing information other than the game are all parts.

[0027] In the following, the technology according to the present embodiment will be described assuming, as an example, a video of a Web conference as shown in FIG.

[0028] FIG. 6 shows an image of the compositing process when using an SFU-type Web conferencing service in which the client reduces and combines images, and the faces of four participants are composited onto a grid.

[0029] After client 1 acquires the facial images of the three people, they are reduced to a resolution that matches the screen layout (called the layout resolution) and then synthesized to create a grid display. Therefore, it is necessary to build a quality estimation model that reflects the effects of reducing and integrating parts.

[0030] (Outline of the embodiment) The Web conference video quality estimation device 100 in this embodiment estimates the Web conference video quality based on the coding information (bit rate, resolution, frame rate) of each part constituting the Web conference video, the part type, and the resolution of each part when composing the layout. More specifically, the process takes into account the following qualitative characteristics 1 to 3.

[0031] Characteristic 1 is a characteristic in which the video quality of face video is more likely to deteriorate due to encoding than that of document video. Figure 7 shows an example of characteristic 1. As shown in Figure 7, when the bit rate is lowered, the video quality of face video deteriorates more than that of document video.

[0032] Characteristic 2 is that when each part, such as a face image or a document image, is reduced in size, it becomes difficult to notice the degradation in quality. Figure 8 shows an example of the effect on image quality of reducing parts. Figure 8 shows the relationship between bit rate and image quality when the image size (resolution) is reduced in succession from (1) to (4). As shown in Figure 8, the smaller the size, the smaller the degradation in image quality with respect to a decrease in bit rate.

[0033] Fig. 9 shows an example of the relationship between the face area ratio and image quality for each bit rate range. As shown in Fig. 9, as the area ratio decreases, the image quality increases, but below a certain area ratio, the image quality saturates.

[0034] In other words, the difficulty of noticing quality degradation varies depending on the video quality, and there is a strong tendency for quality degradation to become harder to notice when the image is reduced, especially in areas with low bit rates (Figures 8 and 9). Also, once the image is reduced to a certain size, the difficulty of noticing degradation remains the same (Figures 8 and 9).

[0035] Characteristic 3 is that users' gazes tend to focus on parts that occupy a large area, such as documents and speakers. Figure 2(e) shows an example of the relationship between the part area and the video quality of a Web conference in a Web conference screen layout example.

[0036] 10 and 11, the thick line indicates the Web conference video quality, and the thin line indicates the video quality of face 1. Moreover, Fig. 10 shows a case where the video quality of faces 2 to 7 is 4.5, and Fig. 11 shows a case where the video quality of faces 2 to 7 is 2.0.

[0037] As shown in Fig. 10, even if the video quality of the small parts at the top of Fig. 2(e) is high, the video quality of the Web conference will degrade according to the quality degradation of the parts that occupy a larger area. Also, as shown in Fig. 11, even if the video quality of the small parts at the top is low, the degree of degradation of the Web conference video quality is small.

[0038] (Device configuration example) The Web conference video quality estimation device 100 of this embodiment takes into consideration at least one of the above characteristics 1 to 3, and derives the Web conference video quality using as input the encoding information (bit rate, resolution, frame rate) of each part that makes up the Web conference video, the part type of each part that makes up the Web conference video, and the resolution of each part when performing layout synthesis.

[0039] 12 shows an example of the configuration of a Web conference video quality estimation apparatus 100 according to this embodiment. As shown in FIG. 12, the Web conference video quality estimation apparatus 100 includes a part-by-part video quality estimation unit 110, a coefficient DB 120, and a screen layout reflection unit .

[0040] The part-by-part video quality estimation unit 110 is a functional unit for estimating Web conference video quality based on characteristic 1, and inputs the bit rate, resolution, frame rate, and part type for each part that makes up the Web conference video, and outputs the video quality for each part for all parts.

[0041] The screen layout reflecting unit 130 is a functional unit for estimating the Web conference video quality based on the characteristics 2 and 3. More specifically, the screen layout reflecting unit 130 includes a parts reduction influence reflecting unit 131, a reduction effect coefficient calculating unit 134, and a reduction effect coefficient DB 133 for estimating the Web conference video quality based on the characteristic 2, and includes a parts integration influence reflecting unit 132 for estimating the Web conference video quality based on the characteristic 3.

[0042] The part reduction effect reflecting unit 131 receives the video quality of each part for all parts and derives the video quality of each part taking into account the reduction. The part integration effect reflecting unit 132 receives the video quality of each part taking into account the reduction of all parts and the resolution of each part when composing the layout and derives the Web conference video quality.

[0043] Furthermore, coefficient DB 120 stores coefficients of a video quality estimation model used by part-specific video quality estimator 110 to estimate the video quality for each part.

[0044] The reduction effect coefficient DB 133 stores coefficients used for estimating the image quality for each part taking reduction into consideration in the part reduction effect reflecting unit 131. The reduction effect coefficient calculation unit 134 calculates the coefficients.

[0045] (Operation Example of Web Conference Video Quality Estimation Device 100) An example of the operation of the Web conference video quality estimation device 100 shown in Fig. 12 will be described. The Web conference video quality estimation device 100 estimates and outputs the quality of the Web conference video displayed on the terminal 10. The inputs are the bit rate BR(i), resolution RS(i), frame rate FR(i), part type P(i), and layout resolution RS_p(i) of the Web conference video displayed on the terminal 10, and the output is the video quality MOS of the Web conference video displayed on the terminal 10.

[0046] The Web conference video quality estimation device 100 may acquire input data from the terminal 10, may acquire input data from the conference server 200, may acquire input data from communication packets between the terminal 10 and the conference server 200, or may acquire input data in some other way.

[0047] Furthermore, the Web-conference video quality estimation device 100 may visualize and display the estimated Web-conference video quality on a display of the Web-conference video quality estimation device 100, or may display it as a Web screen on a remote management terminal or the like.

[0048] Next, an operation example of the Web conference video quality estimation device 100 will be described in more detail according to the procedure of the flowchart in FIG. 13. In the following processing, each part is assumed to be a face video or a document video. However, the fact that each part is a face video or a document video is just an example, and the application scope of the present invention is not limited to face videos or document videos.

[0049] <S101: Video Quality Estimation for Each Part> In S101, first, for each part i (i = 1 to n, where n is the number of parts), the video quality estimation unit 110 for each part takes the bitrate BR(i), resolution RS(i), and frame rate FR(i) as inputs and outputs the video quality MOS_p(i) for each part of all parts. The resolution RS(i) is the resolution before reduction for layout. For the estimation of the video quality of each part, for example, a video quality estimation model disclosed in Reference 1, "Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport, Recommendation ITU-T P.1203, 2017", etc. may be used.

[0050] In order to reflect characteristic 1, the coefficients of the video quality estimation model such as Reference 1 are determined based on the results of the Web conference video quality evaluation experiment performed in advance and stored in the coefficient DB120. In the estimation of the video quality for each part by the video quality estimation unit 110 for each part, a coefficient corresponding to the part type P(i) is selected from the coefficient DB120, and after reflecting it in the video quality estimation model such as Reference 1, the video quality for each part is estimated. Note that the part type P(i) indicates whether it is a face video or a document video.

[0051] <S102: Video Quality Estimation for Each Part Considering Reduction> In S102, the part reduction effect reflecting section 131 in the screen layout reflecting section 130 receives the image quality MOS_p(i) for each part for all parts as input, and derives the image quality MOS_pr(i) for each part taking the reduction into consideration using the reduction effect coefficient for each part. The processing content in S102 will be described with reference to the flowchart in Fig. 14. In the following equation with i, i = 1 to n, and n is the number of parts.

[0052] In S1021, the reduction effect coefficient calculation unit 134 derives the area ratio RS_r(i) for each part from the layout resolution RS_p(i) for each part by the following formula (1). Here, the resolution (reference resolution) at the time of single screen display is indicated as RS_b. RS_b may be a predetermined value or a value included in the input data. Note that formula (1) is only an example, and other formulas may be used as long as they can take into account the effect of reduction.

[0053] RS_r(i)=RS_p(i) / RS_b (1) In S1022, the reduction effect coefficient calculation unit 134 derives the reduction effect coefficient re(i) by the irrational function (2) in order to reflect the characteristics of FIG. 9, that is, the smaller the "area ratio of each part when a resolution of 1920x1080 is set to 1," the "higher the image quality becomes than that of an area ratio of 1," and the image quality saturates at an area ratio below a certain level. The derived reduction effect coefficient re(i) is stored in the reduction effect coefficient DB 133. Note that the formula (2) is just an example, and any other formula may be used as long as it can reflect the characteristics of FIG. 9.

[0054] re(i)=a*(1-RS_r(i))^(1 / 2) (2) The coefficient a in equation (2) is determined for each bit rate region based on the results of a Web conference video quality evaluation experiment conducted in advance, and is stored in the reduction effect coefficient DB 133. When calculating the reduction effect coefficient for a part with a bit rate of BR(i), the reduction effect coefficient calculation unit 134 reads out the coefficient a corresponding to the bit rate region of BR(i) from the reduction effect coefficient DB 133 and uses it.

[0055] In S1023, the component reduction impact reflection unit 131 takes the video quality MOS_p(i) for each component and the reduction effect coefficient re(i) read from the reduction effect coefficient DB133 as inputs, and derives the video quality MOS_pr(i) for each component considering reduction according to the following formula (3). Note that formula (3) is just an example, and other formulas can be used as long as they can consider the impact of reduction.

[0056] MOS_pr(i)=re(i)*MOS_p(i) (3) <S103: Web Conference Video Quality Estimation> In S103 of FIG. 10, the component integration impact reflection unit 132 takes the video quality MOS_pr(i) for each component considering reduction and the layout resolution RS_p(i) for all components as inputs, and derives the Web conference video quality MOS.

[0057] Considering the characteristics shown in FIGS. 10 and 11 that the video quality of components occupying a large area is more likely to be affected, and considering the area ratio of the components to the entire screen, the Web conference video quality MOS is derived by integrating the video quality MOS_pr(i) for each component considering reduction according to formula (4). "RS_p(i) / Σ n k=1 RS_p(i)" in formula (4) corresponds to the above area ratio.

[0058] Note that formula (4) is just an example, and other formulas can be used as long as they can reflect the characteristics shown in FIGS. 10 and 11.

[0059]

Number

[0060] (Modification Example 1) 15 shows a configuration of the Web conference video quality estimation device 100 in Modification 1. As shown in FIG. 15, the Web conference video quality estimation device 100 in Modification 1 includes a parts integration effect reflection unit 132. The parts integration effect reflection unit 132 estimates the Web conference video quality based on characteristic 3.

[0061] In the first modification, the video quality MOS_p(i) for each part and the layout resolution RS_p(i) for each part in the video to be estimated are input to the Web conference video quality estimation device 100. In the Web conference video quality estimation device 100, the video quality MOS_p(i) and the layout resolution RS_p(i) are input to the part integration influence reflection unit 132. The video quality MOS_p(i) for each part is the video quality estimated by existing technology (for example, Reference 1).

[0062] The parts integration influence reflecting unit 132 integrates the video quality MOS_p(i) using the above-mentioned equation (4) in which MOS_pr(i) is replaced with MOS_p(i), to derive and output the Web conference video quality MOS.

[0063] (Variation 2) Fig. 16 shows a configuration of a Web conference video quality estimation device 100 in Modification 2. As shown in Fig. 16, the Web conference video quality estimation device 100 in Modification 2 corresponds to a configuration obtained by removing the part-specific video quality estimation unit 110 and the coefficient DB 120 from the Web conference video quality estimation device 100 shown in Fig. 12.

[0064] In the second modification, the video quality estimated by an existing technique (e.g., Reference 1) is used as the video quality MOS_p(i) for each part. Except for this, the Web conference video quality estimation device 100 in the second modification is the same as the Web conference video quality estimation device 100 shown in FIG.

[0065] (Variation 3) Fig. 17 shows a configuration of a Web conference video quality estimation device 100 in Modification 2. As shown in Fig. 17, the Web conference video quality estimation device 100 in Modification 3 corresponds to a configuration obtained by removing the coefficient DB 120 from the Web conference video quality estimation device 100 shown in Fig. 12.

[0066] In the third modification, the method of estimating the video quality MOS_p(i) for each part by the part-by-part video quality estimation unit 110 is different from the estimation method in the Web conference video quality estimation device 100 shown in Fig. 12. Except for this point, the Web conference video quality estimation device 100 in the third modification is the same as the Web conference video quality estimation device 100 shown in Fig. 12.

[0067] That is, in the Web conference video quality estimation device 10 shown in FIG. 12, the video quality for each part is estimated using a parametric objective evaluation model, but in variant example 3, it is possible to estimate the video quality for each part using a packet layer objective evaluation technique or a bit stream layer objective evaluation technique.

[0068] When the packet layer objective evaluation technique is used, the part-by-part video quality estimation unit 110 inputs packet header information such as IP and RTP in the video to be evaluated, and estimates and outputs the video quality MOS_p(i) for each part using a packet layer model.

[0069] When using the bitstream layer objective evaluation technique, the part-by-part video quality estimation unit 110 inputs payload information (encoded bit sequence information before decoding) in addition to packet header information in the video to be evaluated, and estimates and outputs the video quality MOS_p(i) for each part using a bitstream layer model.

[0070] (Hardware configuration example> The Web conference video quality estimation device 100 can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0071] That is, the Web conference video quality estimation device 100 can be realized by using hardware resources such as a CPU and memory built into a computer to execute a program corresponding to the processing performed by the Web conference video quality estimation device 100. The above program can be recorded on a computer-readable recording medium (such as a portable memory) and stored or distributed. The above program can also be provided via a network such as the Internet or email.

[0072] Fig. 18 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 18 has a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, etc., which are all connected to each other via a bus BS.

[0073] The program for realizing the processing on the computer is provided by a recording medium 1001 such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0074] The memory device 1003 reads out and stores the program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the Web conference video quality estimation device 100 in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network or the like. The display device 1006 displays a GUI (Graphical User Interface) or the like according to a program. The input device 1007 is composed of a keyboard and mouse, buttons, a touch panel, or the like, and is used to input various operation instructions. The output device 1008 outputs the calculation results.

[0075] (Effects of the embodiment) According to the technology of the present embodiment, it becomes possible to estimate the quality of a video having a screen layout in which parts are arranged with high accuracy. For example, it becomes possible to estimate the video quality in a Web conference with high accuracy. For example, a Web conference service provider can visualize the Web conference video quality based on the encoding quality and screen layout information of the service being provided, and by improving the encoding quality that affects the Web conference video quality based on the visualized information, it becomes possible to maintain and improve the Web conference video quality.

[0076] (Additional Note) This specification discloses at least the following video quality estimation apparatus, video quality estimation method, and program: (Additional note 1) A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: Estimating the image quality of the image based on the image quality of each part constituting the image and the layout resolution of each part on the screen layout. Video quality estimation device. (Additional note 2) A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: Memory, at least one processor coupled to the memory; Including, The processor, Estimating the image quality of the image based on the image quality of each part constituting the image and the layout resolution of each part on the screen layout. Video quality estimation device. (Additional note 3) A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: Memory, at least one processor coupled to the memory; Including, The processor, Estimating the image quality of each part constituting the image; A video quality of the video is estimated based on the estimated video quality for each part and a layout resolution for each part on the screen layout. Video quality estimation device. (Additional note 4) A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: Memory, at least one processor coupled to the memory; Including, The processor, Estimating a video quality for each part based on the coding information and the part type of each part constituting the video; A video quality of the video is estimated based on the estimated video quality for each part and a layout resolution for each part on the screen layout. Video quality estimation device. (Additional note 5) A video quality estimation device that estimates video quality of a video having a screen layout in which a plurality of parts are arranged, comprising: Memory, at least one processor coupled to the memory; Including, The processor, Estimating a video quality for each part based on the coding information and the part type of each part constituting the video; A video quality of the video is estimated based on the video quality of each part estimated by the video quality estimation unit and a layout resolution of each part on the screen layout. Video quality estimation device. (Additional note 6) The processor estimates the video quality for each part by using a video quality estimation model that reflects a coefficient corresponding to the part type. Item 5. A video quality estimation device according to claim 5. (Additional note 7) The processor calculates, for each part, a video quality that reflects an effect of reducing the part in order to configure the screen layout; Calculating the image quality of the image by reflecting the effect of integrating a plurality of parts that constitute the screen layout 7. A video quality estimation device according to claim 5 or 6. (Additional note 8) The processor calculates the image quality of each part taking the reduction into consideration by using a reduction effect coefficient set based on the relationship between the area ratio of the part to the entire screen, the bit rate of the part, and the image quality of the part. 8. A video quality estimation device according to claim 7. (Additional note 9) The processor calculates the image quality of the image by integrating the image quality of all parts, taking into account the area ratio of the parts to the entire screen and taking into account the reduction. 9. A video quality estimation device according to claim 7 or 8. (Additional note 10) A video quality estimation method executed by a video quality estimation device for estimating video quality of a video having a screen layout in which parts are arranged, comprising: Estimating the image quality of the image based on the image quality of each part constituting the image and the layout resolution of each part on the screen layout. Video quality estimation method. (Additional note 11) A non-transitory storage medium storing a program for causing a computer to function as the video quality estimation device according to any one of claims 1 to 9.

[0077] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0078] This patent application claims priority to international patent application PCT / JP2021 / 030205, filed on August 18, 2021, the entire contents of which are incorporated herein by reference. [Explanation of symbols]

[0079] 10 Terminal 100 Web conference video quality estimation device 110 Part-by-part video quality estimation unit 120 Coefficient DB 130 Screen layout reflection section 131 Parts reduction effect reflection section 132 Parts Integration Impact Reflection Section 133 Reduction Effect Coefficient DB 134 Reduction Effect Coefficient Calculation Section 200 Conference Server 300 Network 1000 Drive Device 1001 Recording media 1002 Auxiliary storage 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input Device 1008 Output device

Claims

1. A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: Estimating the image quality of the image based on the image quality of each part constituting the image and the layout resolution of each part on the screen layout. Video quality estimation device.

2. A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: a screen layout reflecting unit that estimates a video quality of the video based on a video quality of each part constituting the video and a layout resolution of each part on the screen layout; A video quality estimation device comprising:

3. A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: A video quality estimation unit that estimates a video quality for each part constituting the video; a screen layout reflecting unit that estimates a video quality of the video based on the video quality of each part estimated by the video quality estimation unit and a layout resolution of each part on the screen layout; A video quality estimation device comprising:

4. A video quality estimation device that estimates video quality of a video having a screen layout in which parts are arranged, comprising: a video quality estimation unit that estimates a video quality for each part based on coding information and a part type of each part constituting the video; a screen layout reflecting unit that estimates a video quality of the video based on the video quality of each part estimated by the video quality estimation unit and a layout resolution of each part on the screen layout; A video quality estimation device comprising:

5. A video quality estimation device that estimates video quality of a video having a screen layout in which a plurality of parts are arranged, comprising: a video quality estimation unit that estimates a video quality for each part based on coding information and a part type of each part constituting the video; a screen layout reflecting unit that estimates a video quality of the video based on the video quality of each part estimated by the video quality estimation unit and a layout resolution of each part on the screen layout; A video quality estimation device comprising:

6. The video quality estimation unit estimates video quality for each part by using a video quality estimation model that reflects a coefficient corresponding to a part type.

6. A video quality estimation apparatus according to claim 5.

7. The screen layout reflecting unit includes a part reduction effect reflecting unit that calculates, for each part, a video quality reflecting an effect of reducing the part in order to configure the screen layout; a parts integration effect reflecting unit that calculates a video quality of the video by reflecting an effect of integrating a plurality of parts that configure the screen layout; The video quality estimation apparatus according to claim 5 .

8. The part reduction effect reflecting unit calculates the image quality of each part taking the reduction into consideration by using a reduction effect coefficient set based on the relationship between the area ratio of the part to the entire screen, the bit rate of the part, and the image quality of the part. The video quality estimation apparatus according to claim 7.

9. The part integration effect reflection unit calculates the image quality of the image by integrating the image quality of all parts, taking into account the area ratio of the parts to the entire screen and taking into account the reduction. The video quality estimation apparatus according to claim 7.

10. A video quality estimation method executed by a video quality estimation device for estimating video quality of a video having a screen layout in which parts are arranged, comprising: Estimating the image quality of the image based on the image quality of each part constituting the image and the layout resolution of each part on the screen layout. Video quality estimation method.

11. A program for causing a computer to function as the video quality estimation device according to any one of claims 1 to 9.

Images