Three-dimensional video quality evaluation method, device, equipment and storage medium

Abstract

The present invention discloses a three-dimensional stereoscopic video quality evaluation method, device, equipment and storage medium, belonging to the field of digital video technology. The present invention obtains a reference three-dimensional stereoscopic video corresponding to the three-dimensional stereoscopic video to be evaluated; obtains an image set corresponding to the three-dimensional stereoscopic video to be evaluated, and an image set corresponding to the reference three-dimensional stereoscopic video; calculates a binocular index value according to the image set corresponding to the three-dimensional stereoscopic video to be evaluated and the image set corresponding to the reference three-dimensional stereoscopic video; evaluates the quality of the three-dimensional stereoscopic video to be evaluated according to the binocular index value, and uses the binocular index value to evaluate the quality of the three-dimensional stereoscopic video, comprehensively measuring the video quality, and improving the accuracy of the three-dimensional stereoscopic video quality evaluation.

Description

Technical Field

[0001] The present invention relates to the field of digital video technology, and in particular to a three-dimensional video quality evaluation method, device, equipment and storage medium. Background Art

[0002] As one of the emerging multimedia formats, the success of 3D stereoscopic video depends largely on the improvement of the quality and experience it provides to viewers. Quality assessment plays an important role in the design and optimization of 3D video processing systems.

[0003] At present, 3D stereoscopic video quality evaluation methods can be roughly divided into two categories: subjective evaluation and objective evaluation. One is subjective evaluation, which uses statistics on the test subject's preferences for the test video quality to determine the final video score. However, subjective evaluation methods are time-consuming, expensive, and have poor stability and portability, making them unsuitable for real-time video quality evaluation. The other is objective evaluation, which includes using PSNR to evaluate the quality of compressed stereoscopic videos, taking the average PSNR of the left and right views as the evaluation result; using a 3D-DCT transform stereoscopic video quality evaluation model without a reference stereoscopic perception domain view; and inputting features extracted from sparse dictionaries and stacked autoencoders into support vector machines to obtain the final quality score.

[0004] In objective evaluation, the average value of the PSNR of the left and right views is used as the evaluation result, and the sparse dictionary and stacked autoencoder methods do not consider the depth characteristics of the video and do not conform to the characteristics of human stereoscopic vision. In addition, the PSNR and 3D-DCT transform algorithms only use a single indicator evaluation model with limited evaluation dimensions and do not consider occlusion information, making it impossible to accurately evaluate the video quality of 3D stereoscopic videos.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide a three-dimensional stereoscopic video quality evaluation method, device, equipment and storage medium, aiming to solve the technical problem that the existing technology cannot accurately evaluate the video quality of 3D stereoscopic video.

[0007] To achieve the above object, the present invention provides a three-dimensional stereoscopic video quality evaluation method, the three-dimensional stereoscopic video quality evaluation method comprising the following steps:

[0008] Obtain a reference 3D stereoscopic video corresponding to the 3D stereoscopic video to be evaluated;

[0009] Acquire an image set corresponding to the 3D stereoscopic video to be evaluated and an image set corresponding to the reference 3D stereoscopic video;

[0010] Calculating a binocular index value according to an image set corresponding to the three-dimensional stereo video to be evaluated and an image set corresponding to the reference three-dimensional stereo video;

[0011] The quality of the three-dimensional stereoscopic video to be evaluated is evaluated according to the binocular index value.

[0012] Optionally, the binocular index value includes a binocular competition index value and a binocular depth index value;

[0013] The calculating of the binocular index value according to the image set corresponding to the three-dimensional stereo video to be evaluated and the image set corresponding to the reference three-dimensional stereo video includes:

[0014] Acquire a first view, a second view, and a disparity map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated;

[0015] Acquire a disparity map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video;

[0016] Calculating a binocular rivalry index value according to the first view, the second view, and the disparity map of the three-dimensional stereoscopic video to be evaluated;

[0017] A binocular depth index value is determined according to a preset feature combination, the disparity map of the three-dimensional stereoscopic video to be evaluated, and the disparity map of the reference three-dimensional stereoscopic video.

[0018] Optionally, the calculating the binocular rivalry index value according to the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated includes:

[0019] Determining the number of matching pixels between a first view and a second view of the 3D stereoscopic video to be evaluated according to the disparity map of the 3D stereoscopic video to be evaluated;

[0020] A binocular rivalry index value is calculated according to the preset feature combination, the number of pixels, the first view and the second view of the three-dimensional stereoscopic video to be evaluated.

[0021] Optionally, before performing quality evaluation on the to-be-evaluated 3D stereoscopic video according to the binocular index value, the method further includes:

[0022] Acquire an occlusion map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated;

[0023] Acquire a third view, a fourth view, a disparity map, and an occlusion map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video;

[0024] Determine a monocular view of the 3D stereoscopic video to be evaluated according to the first view, the second view, the disparity map, and the occlusion map of the 3D stereoscopic video to be evaluated;

[0025] Determine a monocular view of the reference 3D stereoscopic video according to a third view, a fourth view, a disparity map, and an occlusion map of the reference 3D stereoscopic video, wherein the first view and the third view have the same viewing direction, the second view and the fourth view have the same viewing direction, and the first view and the second view have opposite viewing directions;

[0026] Calculating a monocular view difference index value according to the preset feature combination, the monocular view of the 3D stereoscopic video to be evaluated, and the monocular view of the reference 3D stereoscopic video;

[0027] Accordingly, the quality evaluation of the three-dimensional stereoscopic video to be evaluated according to the binocular index value includes:

[0028] The quality of the three-dimensional stereoscopic video to be evaluated is evaluated according to the monocular view difference index value and the binocular index value.

[0029] Optionally, the first view is one of a left view or a right view, and the second view is a remaining view of the left view or the right view after the first view is determined;

[0030] The determining of the monocular view of the 3D stereoscopic video to be evaluated according to the first view, the second view, the disparity map, and the occlusion map of the 3D stereoscopic video to be evaluated comprises:

[0031] Determine a reference mapping image according to the first view and the disparity map of the 3D stereoscopic video to be evaluated;

[0032] Correcting the reference mapping image according to the occlusion map and the second view of the 3D stereoscopic video to be evaluated to obtain a target mapping image;

[0033] A monocular view of the 3D stereoscopic video to be evaluated is determined according to the second view of the 3D stereoscopic video to be evaluated and the target mapping map.

[0034] Optionally, determining a reference mapping image according to the first view and the disparity map of the 3D stereoscopic video to be evaluated includes:

[0035] Obtaining the grayscale value of each pixel in the disparity map of the three-dimensional video to be evaluated;

[0036] Determine, according to the grayscale value, a pixel offset between a first view and a second view of the three-dimensional stereoscopic video to be evaluated;

[0037] A reference mapping image is determined according to the pixel offset and the first view of the 3D stereoscopic video to be evaluated.

[0038] Optionally, the performing quality evaluation on the to-be-evaluated 3D stereoscopic video according to the monocular view difference index value and the binocular index value includes:

[0039] Inputting the monocular view difference index value and the binocular index value into a preset regression model;

[0040] Obtaining an evaluation index value output by the preset regression model based on the monocular view difference index value and the binocular index value;

[0041] The quality of the three-dimensional stereoscopic video to be evaluated is evaluated according to the evaluation index value.

[0042] In addition, to achieve the above-mentioned purpose, the present invention further provides a three-dimensional stereoscopic video quality evaluation device, the three-dimensional stereoscopic video quality evaluation device comprising:

[0043] An acquisition module, used for acquiring a reference 3D stereoscopic video corresponding to the 3D stereoscopic video to be evaluated;

[0044] The acquisition module is further used to acquire an image set corresponding to the 3D stereoscopic video to be evaluated and an image set corresponding to the reference 3D stereoscopic video;

[0045] A calculation module, used for calculating a binocular index value according to an image set corresponding to the three-dimensional stereo video to be evaluated and an image set corresponding to the reference three-dimensional stereo video;

[0046] An evaluation module is used to evaluate the quality of the three-dimensional stereo video to be evaluated according to the binocular index value.

[0047] In addition, to achieve the above-mentioned purpose, the present invention also proposes a three-dimensional stereoscopic video quality evaluation device, which includes: a memory, a processor, and a three-dimensional stereoscopic video quality evaluation program stored in the memory and running on the processor, and the three-dimensional stereoscopic video quality evaluation program is configured to implement the three-dimensional stereoscopic video quality evaluation method described above.

[0048] In addition, to achieve the above objectives, the present invention also proposes a storage medium, on which a three-dimensional stereoscopic video quality evaluation program is stored, and when the three-dimensional stereoscopic video quality evaluation program is executed by a processor, the three-dimensional stereoscopic video quality evaluation method as described above is implemented.

[0049] The present invention obtains a reference three-dimensional stereoscopic video corresponding to a three-dimensional stereoscopic video to be evaluated; obtains an image set corresponding to the three-dimensional stereoscopic video to be evaluated and an image set corresponding to the reference three-dimensional stereoscopic video; calculates a binocular index value according to the image set corresponding to the three-dimensional stereoscopic video to be evaluated and the image set corresponding to the reference three-dimensional stereoscopic video; performs quality evaluation on the three-dimensional stereoscopic video to be evaluated according to the binocular index value, and uses the binocular index value to evaluate the quality of the three-dimensional stereoscopic video, thereby comprehensively measuring the video quality and improving the accuracy of the three-dimensional stereoscopic video quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a structural schematic diagram of a three-dimensional video quality evaluation device in a hardware operating environment involved in an embodiment of the present invention;

[0051] Figure 2 It is a flowchart of a first embodiment of a three-dimensional stereoscopic video quality assessment method of the present invention;

[0052] Figure 3 It is a flowchart of a second embodiment of a three-dimensional stereoscopic video quality assessment method of the present invention;

[0053] Figure 4 It is a flowchart of a third embodiment of a method for evaluating quality of three-dimensional stereoscopic video according to the present invention;

[0054] Figure 5 It is a structural block diagram of the first embodiment of the three-dimensional stereoscopic video quality assessment device of the present invention.

[0055] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0057] Reference Figure 1 , Figure 1 The figure is a schematic diagram of the structure of a three-dimensional video quality evaluation device in a hardware operating environment involved in an embodiment of the present invention.

[0058] like Figure 1As shown, the three-dimensional stereoscopic video quality assessment device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0059] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the three-dimensional stereoscopic video quality assessment device, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0060] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a 3D stereoscopic video quality evaluation program.

[0061] exist Figure 1 In the three-dimensional stereoscopic video quality evaluation device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the three-dimensional stereoscopic video quality evaluation device of the present invention can be set in the three-dimensional stereoscopic video quality evaluation device, and the three-dimensional stereoscopic video quality evaluation device calls the three-dimensional stereoscopic video quality evaluation program stored in the memory 1005 through the processor 1001, and executes the three-dimensional stereoscopic video quality evaluation method provided by the embodiment of the present invention.

[0062] Based on the above hardware structure, an embodiment of a three-dimensional stereoscopic video quality evaluation method of the present invention is proposed.

[0063] Reference Figure 2 , Figure 2 It is a flowchart diagram of a first embodiment of a three-dimensional stereoscopic video quality assessment method of the present invention.

[0064] In a first embodiment, the three-dimensional stereoscopic video quality evaluation method comprises the following steps:

[0065] Step S10: obtaining a reference 3D stereoscopic video corresponding to the 3D stereoscopic video to be evaluated.

[0066] In this embodiment, the executor of this embodiment can be a three-dimensional stereoscopic video quality evaluation device, and the three-dimensional stereoscopic video quality evaluation device can be a computer or a tablet or other device, or can be other electronic devices that can achieve the same or similar functions. This embodiment is not limited to this. In this embodiment and the following embodiments, the three-dimensional stereoscopic video quality evaluation method of the present invention is explained by taking the three-dimensional stereoscopic video quality evaluation device as an example.

[0067] It should be noted that the three-dimensional stereoscopic video in this embodiment is also 3D stereoscopic video. The 3D stereoscopic video quality evaluation method currently adopted is roughly divided into subjective evaluation and objective evaluation. The subjective evaluation method is an evaluation method that uses statistics on the test subject's preferences for the test video quality to determine the final video score. However, the subjective evaluation method is time-consuming, expensive, and has poor stability and portability, and is not suitable for real-time video quality evaluation. Secondly, objective evaluation, which includes using PSNR to evaluate the quality of the compressed stereoscopic video, and taking the average value of the PSNR of the left and right views as the evaluation result; a 3D-DCT transform stereoscopic video quality evaluation model without a reference stereoscopic perception domain view; and inputting the features extracted from the sparse dictionary and stacked autoencoder into the support vector machine to obtain the final quality score. In the objective evaluation, the average value of the PSNR of the left and right views is used as the evaluation result, and the sparse dictionary and stacked autoencoder methods do not consider the depth characteristics of the video, which does not conform to the human stereoscopic vision characteristics, and the PSNR and 3D-DCT transformation algorithms only use a single indicator evaluation model, the evaluation dimension is limited, and occlusion information is not considered, and the video quality of the 3D stereoscopic video cannot be accurately evaluated.

[0068] In order to solve the above problems, in this embodiment, multiple groups of indicators are fused, and the quality of the three-dimensional stereoscopic video model is more accurately evaluated by fusing the depth characteristics and occlusion information. Specifically, it can be implemented as follows.

[0069] In the specific implementation, before performing quality evaluation on the 3D stereoscopic video in this embodiment, it is necessary to first obtain a reference 3D stereoscopic video of the 3D stereoscopic video to be evaluated. The reference 3D stereoscopic video is the original, unprocessed 3D stereoscopic video. The 3D stereoscopic video to be evaluated in this embodiment is a distorted 3D stereoscopic video that needs to be evaluated for quality.

[0070] It should be noted that the video content of the 3D stereoscopic video to be evaluated and the reference 3D stereoscopic video in this embodiment are consistent, and the difference lies in color or clarity, etc. The selection of the 3D stereoscopic video to be evaluated and the reference 3D stereoscopic video can be made according to actual needs, and this is not limited in this embodiment.

[0071] In a specific implementation, this embodiment can receive a video quality evaluation instruction input by a user, and obtain a reference three-dimensional stereoscopic video corresponding to the three-dimensional stereoscopic video to be evaluated according to the video quality evaluation instruction. A preset time can also be set. When the preset time is reached, the quality of the three-dimensional stereoscopic video to be evaluated stored by the user is automatically evaluated. The preset time can be set according to actual needs, and the above method can also be selected accordingly according to user needs, which is not limited in this embodiment.

[0072] Step S20: obtaining an image set corresponding to the 3D stereoscopic video to be evaluated and an image set corresponding to the reference 3D stereoscopic video.

[0073] In a specific implementation, after obtaining the three-dimensional stereoscopic video to be evaluated and the reference three-dimensional stereoscopic video, the present embodiment further obtains an image set corresponding to the three-dimensional stereoscopic video to be evaluated and an image set corresponding to the reference three-dimensional stereoscopic video. The image set corresponding to the three-dimensional stereoscopic video to be evaluated includes multiple views of the three-dimensional stereoscopic video to be evaluated, such as a left view, a right view, a disparity map, and an occlusion map. Similarly, the image set corresponding to the reference three-dimensional stereoscopic video includes multiple views of the reference three-dimensional stereoscopic video, such as a left view, a right view, a disparity map, and an occlusion map.

[0074] Step S30: calculating a binocular index value according to an image set corresponding to the three-dimensional stereo video to be evaluated and an image set corresponding to the reference three-dimensional stereo video.

[0075] In a specific implementation, in this embodiment, a no-reference evaluation method can be used to calculate the binocular competition index value based on the image set corresponding to the three-dimensional stereo video to be evaluated and the image set corresponding to the reference three-dimensional stereo video. Of course, in addition to the above method, other methods can be used to calculate the binocular index value according to user needs, which is not limited in this embodiment.

[0076] Step S40: performing quality evaluation on the to-be-evaluated 3D stereoscopic video according to the binocular index value.

[0077] In a specific implementation, after obtaining the binocular index value, the quality of the 3D video to be evaluated can be evaluated according to the binocular index value. Specifically, in this embodiment, the quality score of the 3D video to be evaluated is calculated according to the binocular index value, and the quality evaluation is performed based on the quality score.

[0078] This embodiment obtains a reference 3D stereo video corresponding to the 3D stereo video to be evaluated; obtains an image set corresponding to the 3D stereo video to be evaluated, and an image set corresponding to the reference 3D stereo video; calculates a binocular index value according to the image set corresponding to the 3D stereo video to be evaluated and the image set corresponding to the reference 3D stereo video; performs quality evaluation on the 3D stereo video to be evaluated according to the binocular index value, and uses the binocular index value to evaluate the quality of the 3D stereo video, thereby comprehensively measuring the video quality and improving the accuracy of the 3D stereo video quality evaluation.

[0079] refer to Figure 3 , Figure 3 It is a flowchart diagram of a second embodiment of a three-dimensional stereoscopic video quality assessment method of the present invention.

[0080] Based on the first embodiment, in the three-dimensional stereoscopic video quality evaluation method of this embodiment, step S30 specifically includes:

[0081] Step S301: acquiring a first view, a second view and a disparity map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated.

[0082] Step S302: obtaining a disparity map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video.

[0083] In a specific implementation, the present embodiment evaluates the 3D stereo video to be evaluated by making a reference comparison with the reference 3D stereo video. First, the present embodiment needs to obtain the first view, the second view and the disparity map of the 3D stereo video to be evaluated from the image set corresponding to the 3D stereo video to be evaluated, and obtain the disparity map of the reference 3D stereo video from the image set corresponding to the reference 3D stereo video. The first view may be a left view or a right view. When the first view is a left view, the second view is a right view. Conversely, when the first view is a right view, the second view is a left view.

[0084] Step S303: calculating a binocular rivalry index value according to the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated.

[0085] In a specific implementation, the disparity map can be used to match pixels between the first view and the second view, find similar pixels in the first view and the second view, and obtain the number of pixels of these pixels. For example, if pixel A in the first view is similar to pixel A' in the second view, the number of pixels is increased by 1. Then, the binocular competition index value is calculated based on the preset feature combination, the number of pixels, the left view and the right view of the 3D stereo video to be evaluated, and the calculation formula is as follows:

[0086]

[0087] Among them, VQA is the preset feature combination, N is the number of pixels, and I Ldis is the first view of the 3D stereo video to be evaluated, I Rdis is the second view of the 3D stereoscopic video to be evaluated.

[0088] Step S304: determining a binocular depth index value according to a preset feature combination, the disparity map of the 3D stereo video to be evaluated, and the disparity map of the reference 3D stereo video.

[0089] In a specific implementation, the binocular depth index value may be calculated according to a preset feature combination, a disparity map of the three-dimensional stereo video to be evaluated, and a disparity map of a reference three-dimensional stereo video.

[0090] Specifically, in this embodiment, the binocular depth index value is calculated according to the following formula:

[0091] QA BD =VQA(DETA ref ,DETA dis ),

[0092] Among them, VQA is a preset feature combination, DETA ref is the disparity map of the 3D stereo video to be evaluated, DETA dis See the disparity map of 3D stereo video.

[0093] This embodiment obtains the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated from the image set corresponding to the three-dimensional stereoscopic video to be evaluated; obtains the disparity map of the reference three-dimensional stereoscopic video from the image set corresponding to the reference three-dimensional stereoscopic video; calculates the binocular competition index value according to the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated; determines the binocular depth index value according to a preset feature combination, the disparity map of the three-dimensional stereoscopic video to be evaluated and the disparity map of the reference three-dimensional stereoscopic video, and by combining the preset feature combination, the first view, the second view and the disparity map, the binocular competition index value and the binocular depth index value can be accurately calculated, and the quality of the three-dimensional stereoscopic video is evaluated by the binocular competition index value and the binocular depth index value, thereby further improving the accuracy of video evaluation.

[0094] refer to Figure 4 , Figure 4 It is a flowchart diagram of a third embodiment of a three-dimensional stereoscopic video quality assessment method of the present invention.

[0095] Based on the above second embodiment, a third embodiment of a three-dimensional stereoscopic video quality evaluation method is proposed.

[0096] In this embodiment, before step S40, the following steps are also included:

[0097] Step S040: obtaining an occlusion map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated.

[0098] Step S140: acquiring a third view, a fourth view, a disparity map, and an occlusion map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video.

[0099] In the specific implementation, in order to make the quality evaluation of the 3D stereo video more comprehensive, the monocular difference index value needs to be considered on the basis of the binocular index value. First, in this embodiment, the occlusion map of the 3D stereo video to be evaluated needs to be obtained from the image set corresponding to the 3D stereo video to be evaluated, and the third view, fourth view, disparity map and occlusion map of the reference 3D stereo video need to be obtained from the image set corresponding to the reference 3D stereo video.

[0100] Step S240: determining a monocular view of the 3D stereoscopic video to be evaluated according to the first view, the second view, the disparity map, and the occlusion map of the 3D stereoscopic video to be evaluated.

[0101] Step S340: determining a monocular view of the reference 3D stereoscopic video according to the third view, the fourth view, the disparity map, and the occlusion map of the reference 3D stereoscopic video.

[0102] In a specific implementation, the monocular disparity index value is calculated through a monocular view. In this embodiment, what is needed to calculate the monocular disparity index value is a monocular view of the 3D stereo video to be evaluated and a monocular view of the reference 3D stereo video.

[0103] Specifically, the monocular view of the 3D stereo video to be evaluated can be determined based on the first view, the second view, the disparity map and the occlusion map of the 3D stereo video to be evaluated, and the monocular view of the reference 3D stereo video can be determined based on the third view, the fourth view, the disparity map and the occlusion map of the reference 3D stereo video. And it should be emphasized that in this embodiment, the first view and the third view have the same viewing direction, the second view and the fourth view have the same viewing direction, the first view and the second view have opposite viewing directions, the first view is one of the left view or the right view, and the second view is the remaining view in the left view or the right view after the first view is determined, for example, when the first view is the left view, the third view is also the left view, and the second view and the fourth view are the right views.

[0104] Further, after obtaining the first view and the disparity map, a reference mapping map can be calculated according to the first view and the disparity map. When the first view is a right view, the reference mapping map is a mapping map from right to left. When the second view is a left view, the reference mapping map is a mapping map from left to right. It can be calculated according to the following formula:

[0105] R2L(x,y)=I R (x+△(x,y),y),

[0106] Where (x, y) represents the position of the pixel, W and H represent the image size of the first view, W: width, H: height, x=1,...,W, y=1,...,H, I R represents the first view, △(x, y) is the pixel offset between the first view and the second view of the 3D stereo video to be evaluated, and the pixel offset in this embodiment can be determined by the grayscale value of each pixel in the disparity map.

[0107] Furthermore, after obtaining the reference mapping image, the reference mapping image is corrected by the occlusion map so that the quality evaluation takes into account the occlusion information in the three-dimensional stereoscopic video. Specifically, the correction can be performed according to the following formula, which is as follows:

[0108]

[0109] Among them, I L (x, y) is the left view of the 3D stereo video to be evaluated, R2L(x, y) is the reference mapping map, θ(x, y) represents the binary occlusion map, θ(x, y) = 1 represents the marked occluded pixels, if the pixel points in the reference mapping map are not occluded, the pixel points in the original reference mapping map are used, if the pixel points in the reference mapping map are occluded, the corresponding pixel points in the second view are used for replacement.

[0110] After obtaining the target mapping map, the monocular view of the 3D stereoscopic video to be evaluated can be calculated according to the left view of the 3D stereoscopic video to be evaluated and the target mapping map according to the following formula:

[0111]

[0112] Among them, I L is the second view of the 3D stereo video to be evaluated, R2L fix The method of obtaining the monocular view of the reference 3D stereoscopic video is similar to the above process and will not be described in detail in this embodiment.

[0113] In a specific implementation, after obtaining a monocular view of the 3D stereo video to be evaluated and a monocular view of the reference 3D stereo video, a monocular view difference index value can be obtained by combining a preset feature combination and the above two monocular views.

[0114] Step S440: Calculating a monocular view difference index value according to the preset feature combination, the monocular view of the 3D stereoscopic video to be evaluated, and the monocular view of the reference 3D stereoscopic video.

[0115] In a specific implementation, the monocular view difference index value may be calculated by combining a preset feature combination and evaluating a monocular view of a 3D stereoscopic video and a monocular view of a reference 3D stereoscopic video.

[0116] Specifically, in this embodiment, the monocular view difference index value is calculated according to the following formula:

[0117]

[0118] Among them, VQA is a preset feature combination, is the monocular view of the 3D stereo video to be evaluated, is the monocular view of the reference 3D stereo video.

[0119] Furthermore, step S40 in this embodiment specifically includes:

[0120] Step S401: performing quality evaluation on the to-be-evaluated 3D stereoscopic video according to the monocular view difference index value and the binocular index value.

[0121] In a specific implementation, after obtaining the monocular view difference index value, the binocular competition index value, and the binocular depth index value, the quality of the 3D stereo video to be evaluated can be evaluated according to these index values. Specifically, in this embodiment, the quality score of the 3D stereo video to be evaluated is calculated according to the above index values, and the quality evaluation is performed based on the quality score.

[0122] Furthermore, in this embodiment, the quality of the three-dimensional stereoscopic video is comprehensively evaluated by means of index fusion. Specifically, the obtained monocular view difference index value, binocular competition index value and binocular depth index value can be input into a preset regression model for fusion. Among them, the preset regression model in this embodiment can adopt a 3DVQA model, and can also adopt other models, which can be selected accordingly according to actual needs, and this is not limited in this embodiment. And it should be emphasized that the 3DVQA model in this embodiment is trained by SVM_SVR and support vector machine.

[0123] In a specific implementation, after the monocular view difference index value, the binocular competition index value and the binocular depth index value are input into a preset regression model, the preset regression model outputs a corresponding evaluation index value based on the input index value. The evaluation index value is the quality score of the three-dimensional stereoscopic video to be evaluated, which can be used to perform quality evaluation on the three-dimensional stereoscopic video to be evaluated.

[0124] In a specific implementation, after obtaining the evaluation index value, the quality of the 3D stereo video to be evaluated can be evaluated according to the evaluation index value. The higher the evaluation index value, the better the video quality of the 3D stereo video to be evaluated. Conversely, the lower the evaluation index value, the worse the video quality of the 3D stereo video to be evaluated. The value range of the evaluation index value in this embodiment can be set to 0 to 100. Of course, it can also be set to other ranges according to the actual needs of the user, which is not limited in this embodiment.

[0125] This embodiment more accurately calculates the monocular view difference index value according to the preset feature combination, the monocular view of the 3D stereo video to be evaluated, and the monocular view of the reference 3D stereo video, and then combines the monocular view difference index value with the binocular index value to evaluate the video quality. By considering video occlusion through monocular characteristics and binocular competition characteristics and combining the depth characteristics of the video, the video quality is comprehensively measured, thereby improving the accuracy of 3D video quality evaluation.

[0126] In addition, an embodiment of the present invention further proposes a storage medium storing a 3D stereoscopic video quality evaluation program, which implements the steps of the 3D stereoscopic video quality evaluation method described above when executed by a processor.

[0127] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0128] Reference Figure 5 , Figure 5 It is a structural block diagram of the first embodiment of the three-dimensional stereoscopic video quality assessment device of the present invention.

[0129] like Figure 5 As shown, the three-dimensional stereoscopic video quality evaluation device proposed in the embodiment of the present invention includes:

[0130] The acquisition module 10 is used to acquire a reference 3D stereoscopic video corresponding to the 3D stereoscopic video to be evaluated.

[0131] The acquisition module 10 is further used to acquire an image set corresponding to the 3D stereoscopic video to be evaluated and an image set corresponding to the reference 3D stereoscopic video.

[0132] The calculation module 20 is used to calculate the binocular index value according to the image set corresponding to the three-dimensional stereo video to be evaluated and the image set corresponding to the reference three-dimensional stereo video.

[0133] The evaluation module 30 is used to evaluate the quality of the three-dimensional stereo video to be evaluated according to the binocular index value.

[0134] This embodiment obtains a reference 3D stereo video corresponding to the 3D stereo video to be evaluated; obtains an image set corresponding to the 3D stereo video to be evaluated, and an image set corresponding to the reference 3D stereo video; calculates a binocular index value according to the image set corresponding to the 3D stereo video to be evaluated and the image set corresponding to the reference 3D stereo video; performs quality evaluation on the 3D stereo video to be evaluated according to the binocular index value, and uses the binocular index value to evaluate the quality of the 3D stereo video, thereby comprehensively measuring the video quality and improving the accuracy of the 3D stereo video quality evaluation.

[0135] In one embodiment, the calculation module 20 is further used to obtain the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated from the image set corresponding to the three-dimensional stereoscopic video to be evaluated; obtain the disparity map of the reference three-dimensional stereoscopic video from the image set corresponding to the reference three-dimensional stereoscopic video; calculate the binocular competition index value according to the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated; determine the binocular depth index value according to a preset feature combination, the disparity map of the three-dimensional stereoscopic video to be evaluated and the disparity map of the reference three-dimensional stereoscopic video.

[0136] In one embodiment, the calculation module 20 is further used to determine the number of matching pixels between the first view and the second view of the three-dimensional stereoscopic video to be evaluated based on the disparity map of the three-dimensional stereoscopic video to be evaluated; and calculate the binocular competition index value based on the preset feature combination, the number of pixels, the first view and the second view of the three-dimensional stereoscopic video to be evaluated.

[0137] In one embodiment, the calculation module 20 is further used to obtain an occlusion map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated; obtain a third view, a fourth view, a disparity map and an occlusion map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video; determine a monocular view of the 3D stereoscopic video to be evaluated according to the first view, the second view, the disparity map and the occlusion map of the 3D stereoscopic video to be evaluated; determine a monocular view of the reference 3D stereoscopic video according to the third view, the fourth view, the disparity map and the occlusion map of the reference 3D stereoscopic video, wherein the first view and the third view have the same viewing direction, the second view and the fourth view have the same viewing direction, and the first view and the second view have opposite viewing directions; calculate a monocular view difference index value according to the preset feature combination, the monocular view of the 3D stereoscopic video to be evaluated and the monocular view of the reference 3D stereoscopic video;

[0138] The evaluation module 30 is further used to evaluate the quality of the to-be-evaluated 3D stereoscopic video according to the monocular view difference index value and the binocular index value.

[0139] In one embodiment, the calculation module 20 is further used to determine a reference mapping map based on the first view and the disparity map of the three-dimensional stereoscopic video to be evaluated; correct the reference mapping map based on the occlusion map and the second view of the three-dimensional stereoscopic video to be evaluated to obtain a target mapping map; and determine a monocular view of the three-dimensional stereoscopic video to be evaluated based on the second view of the three-dimensional stereoscopic video to be evaluated and the target mapping map.

[0140] In one embodiment, the calculation module 20 is further used to obtain the grayscale value of each pixel in the disparity map of the three-dimensional stereoscopic video to be evaluated; determine the pixel offset between the first view and the second view of the three-dimensional stereoscopic video to be evaluated according to the grayscale value; and determine a reference mapping map according to the pixel offset and the first view of the three-dimensional stereoscopic video to be evaluated.

[0141] In one embodiment, the evaluation module 30 is further used to input the monocular view difference index value and the binocular index value into a preset regression model; obtain an evaluation index value output by the preset regression model based on the monocular view difference index value and the binocular index value; and perform quality evaluation on the three-dimensional stereo video to be evaluated according to the evaluation index value.

[0142] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0143] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0144] In addition, for technical details not fully described in this embodiment, reference may be made to the three-dimensional stereoscopic video quality evaluation method provided in any embodiment of the present invention, and will not be repeated here.

[0145] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0146] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0147] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0148] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A three-dimensional video quality evaluation method, characterized in that: The three-dimensional stereoscopic video quality evaluation method comprises: Obtain a reference 3D stereoscopic video corresponding to the 3D stereoscopic video to be evaluated; Acquire an image set corresponding to the 3D stereoscopic video to be evaluated and an image set corresponding to the reference 3D stereoscopic video; Calculating a binocular index value according to an image set corresponding to the three-dimensional stereo video to be evaluated and an image set corresponding to the reference three-dimensional stereo video; Determine a reference mapping image according to the first view and the disparity map of the 3D stereoscopic video to be evaluated; Correcting the reference mapping image according to the occlusion map and the second view of the 3D stereoscopic video to be evaluated to obtain a target mapping image; Determining a monocular view of the 3D stereoscopic video to be evaluated according to the second view of the 3D stereoscopic video to be evaluated and the target mapping map, wherein the first view is one of a left view or a right view, and the second view is a remaining view of the left view or the right view after the first view is determined; Calculating a monocular view difference index value according to a preset feature combination, a monocular view of the 3D stereoscopic video to be evaluated, and a monocular view of the reference 3D stereoscopic video; The quality of the three-dimensional stereoscopic video to be evaluated is evaluated according to the monocular view difference index value and the binocular index value.

2. The three-dimensional video quality evaluation method according to claim 1, characterized in that: The binocular index value includes a binocular competition index value and a binocular depth index value; The calculating of the binocular index value according to the image set corresponding to the three-dimensional stereo video to be evaluated and the image set corresponding to the reference three-dimensional stereo video includes: Acquire a first view, a second view, and a disparity map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated; Acquire a disparity map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video; Calculating a binocular rivalry index value according to the first view, the second view, and the disparity map of the three-dimensional stereoscopic video to be evaluated; A binocular depth index value is determined according to the preset feature combination, the disparity map of the three-dimensional stereoscopic video to be evaluated, and the disparity map of the reference three-dimensional stereoscopic video.

3. The three-dimensional video quality evaluation method according to claim 2, characterized in that: The calculating of the binocular rivalry index value according to the first view, the second view and the disparity map of the three-dimensional stereoscopic video to be evaluated includes: Determining the number of matching pixels between a first view and a second view of the 3D stereoscopic video to be evaluated according to the disparity map of the 3D stereoscopic video to be evaluated; A binocular rivalry index value is calculated according to the preset feature combination, the number of pixels, the first view and the second view of the three-dimensional stereoscopic video to be evaluated.

4. The three-dimensional video quality evaluation method according to claim 2, characterized in that: Before calculating the monocular view difference index value according to the preset feature combination, the monocular view of the 3D stereoscopic video to be evaluated and the monocular view of the reference 3D stereoscopic video, the method further includes: Acquire an occlusion map of the 3D stereoscopic video to be evaluated from an image set corresponding to the 3D stereoscopic video to be evaluated; Acquire a third view, a fourth view, a disparity map, and an occlusion map of the reference 3D stereoscopic video from an image set corresponding to the reference 3D stereoscopic video; A monocular view of the reference 3D stereoscopic video is determined according to the third view, the fourth view, the disparity map and the occlusion map of the reference 3D stereoscopic video, wherein the first view and the third view have the same viewing direction, and the second view and the fourth view have the same viewing direction.

5. The three-dimensional video quality evaluation method according to claim 1, characterized in that: The step of determining a reference mapping image according to the first view and the disparity map of the 3D stereoscopic video to be evaluated comprises: Obtaining the grayscale value of each pixel in the disparity map of the three-dimensional video to be evaluated; Determine, according to the grayscale value, a pixel offset between a first view and a second view of the three-dimensional stereoscopic video to be evaluated; A reference mapping image is determined according to the pixel offset and the first view of the 3D stereoscopic video to be evaluated.

6. The three-dimensional video quality evaluation method according to claim 1, characterized in that: The step of evaluating the quality of the to-be-evaluated 3D stereoscopic video according to the monocular view difference index value and the binocular index value comprises: Inputting the monocular view difference index value and the binocular index value into a preset regression model; Obtaining an evaluation index value output by the preset regression model based on the monocular view difference index value and the binocular index value; The quality of the three-dimensional stereoscopic video to be evaluated is evaluated according to the evaluation index value.

7. A three-dimensional video quality evaluation device, characterized in that: The three-dimensional video quality evaluation device comprises: An acquisition module, used for acquiring a reference 3D stereoscopic video corresponding to the 3D stereoscopic video to be evaluated; The acquisition module is further used to acquire an image set corresponding to the 3D stereoscopic video to be evaluated and an image set corresponding to the reference 3D stereoscopic video; A calculation module, used for calculating a binocular index value according to an image set corresponding to the three-dimensional stereo video to be evaluated and an image set corresponding to the reference three-dimensional stereo video; The calculation module is further used to determine a reference mapping map according to the first view and the disparity map of the 3D stereoscopic video to be evaluated; correct the reference mapping map according to the occlusion map and the second view of the 3D stereoscopic video to be evaluated to obtain a target mapping map; determine a monocular view of the 3D stereoscopic video to be evaluated according to the second view of the 3D stereoscopic video to be evaluated and the target mapping map, wherein the first view is one of the left view or the right view, and the second view is a remaining view in the left view or the right view after the first view is determined; calculate a monocular view difference index value according to a preset feature combination, the monocular view of the 3D stereoscopic video to be evaluated, and the monocular view of the reference 3D stereoscopic video; An evaluation module is used to evaluate the quality of the three-dimensional stereo video to be evaluated according to the monocular view difference index value and the binocular index value.

8. A three-dimensional video quality evaluation device, characterized in that: The 3D stereoscopic video quality assessment device includes: a memory, a processor, and a 3D stereoscopic video quality assessment program stored in the memory and running on the processor, wherein the 3D stereoscopic video quality assessment program is configured to implement the 3D stereoscopic video quality assessment method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a three-dimensional stereoscopic video quality assessment program, and when the three-dimensional stereoscopic video quality assessment program is executed by the processor, the three-dimensional stereoscopic video quality assessment method according to any one of claims 1 to 6 is implemented.

Images