Subjective quality evaluation method and device for volume video

Abstract

The invention discloses a subjective quality evaluation method and device for a volume video. The method comprises the following steps: reading sequence data of a tested volume video through test software; the test software renders a corresponding picture in real time based on the read sequence data and a preset initial position and orientation and presents the picture to a tester through a presentation device; adjusting the position and orientation of the picture through an input device, re-rendering the picture in real time based on the adjusted position and orientation, and presenting the picture to a tester through a presentation device; and receiving and recording subjective opinion scores made by the testers after the watching time is over. According to the invention, a tester can subjectively evaluate the quality of the volume video while actively and interactively watching the volume video.

Description

Technical Field

[0001] The present invention relates to the technical field of video quality evaluation, and in particular to a subjective quality evaluation method and device for volumetric video. Background Art

[0002] With the continuous development of related technologies and the increasing demand for video consumption, people are beginning to pursue a freer way to watch videos. Volumetric video allows multi-viewpoint and multi-angle viewing, which provides more freedom than omnidirectional video and can meet people's needs. Therefore, it has gradually become an important video form. However, low-quality volumetric video will lead to a poor user experience. Therefore, a certain quality evaluation method is needed to guide the improvement of the quality of volumetric video.

[0003] Image and video quality evaluation can be divided into subjective quality evaluation and objective quality evaluation. Among them, subjective quality evaluation refers to the quality of an image or video obtained by subjective judgment. It is the most direct and reliable method to understand the observer's feelings about the quality of an image or video. Subjective quality evaluation is an important part of studying image or video quality.

[0004] With the development of related technologies and people's pursuit of higher-level consumption methods, more and more users begin to consume volumetric videos in an active interactive way. This interactive behavior can provide users with a strong sense of immersion. However, the current subjective quality evaluation methods of volumetric videos are all tested under the condition of simulating users moving along a fixed trajectory, without interactive behavior, which does not conform to people's actual habit of consuming volumetric videos. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for subjective quality evaluation of volumetric video, so that a tester can freely watch the volumetric video in an active interactive manner and perform subjective evaluation on the quality of the volumetric video.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A first aspect of the present invention provides a subjective quality assessment method for volumetric video, comprising the following steps:

[0008] Read the sequence data of the tested volume video through the test software;

[0009] The test software renders the corresponding images in real time based on the read sequence data and the preset initial position and orientation, and presents them to the tester through the presentation device;

[0010] Adjust the position and orientation of the screen through the input device, re-render the screen in real time based on the adjusted position and orientation, and present it to the tester through the presentation device;

[0011] Accept and record subjective opinion scores from testers after viewing time is complete.

[0012] Furthermore, the method further comprises:

[0013] Determine whether the position of the picture exceeds the space range;

[0014] If so, the position and orientation of the image are corrected and an alarm is issued.

[0015] Furthermore, the method for determining whether the position of the picture exceeds the spatial range is as follows:

[0016] The coordinates of the viewing position and the representation of the viewable space of the tested sequence data are obtained to determine whether the position is within the viewable space; if so, it is considered that the position of the picture does not exceed the spatial range; if not, it is considered that the position of the picture exceeds the spatial range.

[0017] Furthermore, the correction is made by at least one of the following methods:

[0018] Freeze the position and orientation of the image;

[0019] Reset the position and orientation of the screen to the initial position and orientation;

[0020] The alarm is issued by at least one of the following methods:

[0021] sound an audible alarm;

[0022] Change the screen color.

[0023] Furthermore, the method also includes:

[0024] The spatial preview sequence corresponding to the tested sequence data is presented to the tester through a presentation device, wherein the spatial preview sequence is used to display the spatial range.

[0025] Furthermore, the spatial preview sequence uses at least one of the following methods to display the spatial range:

[0026] Render the boundary of the space as a non-completely transparent surface and add it to the source sequence corresponding to the tested sequence;

[0027] Render the boundary of the bottom surface of the space as a non-completely transparent closed figure and add it to the source sequence corresponding to the tested sequence;

[0028] Render the boundary of the space as a non-completely transparent surface and add it to the tested sequence;

[0029] Render the boundary of the bottom surface of the space as a non-completely transparent closed shape and add it to the tested sequence.

[0030] A second aspect of the present invention provides a subjective quality assessment device for volumetric video, which is used to implement the method described in the first aspect, including:

[0031] The test software is used to read the sequence data of the tested volumetric video, and based on the read sequence data and the preset initial position and orientation, render the corresponding picture in real time and output it to the presentation device, or re-render the picture in real time based on the adjusted position and orientation and output it to the presentation device;

[0032] A presentation device, used to present the screen output by the received test software to the tester;

[0033] Input device, used to adjust the position and orientation of the screen according to the user's input operation;

[0034] The testing software is also used to accept and record the subjective opinion scores given by the testers after the viewing time is over.

[0035] Furthermore, the input device adjusts the position and orientation of the picture according to the user's input operation in at least one of the following ways:

[0036] Adjust the position and orientation of the screen through the keyboard;

[0037] Adjust the position and orientation of the image through spatial positioning equipment;

[0038] Use the joystick to adjust the position and orientation of the screen.

[0039] Furthermore, the presenting device presents the received screen output by the test software to the tester in at least one of the following ways:

[0040] The images are presented to the tester through a normal display;

[0041] The images are presented to the tester through a head-mounted display;

[0042] The images are presented to the testers through VR glasses.

[0043] Compared with the prior art, the advantages and positive effects of the present invention are:

[0044] The subjective quality evaluation method for volumetric video provided by the present invention is different from the existing quality evaluation method for passive non-interactive viewing, and enables the tester to subjectively evaluate the quality of the volumetric video while actively and interactively watching. In order to solve the problem that the scope of high-quality viewing of some volumetric videos is limited, the present invention also limits the spatial range of viewing of the tester, so that the tester can always watch relatively high-quality pictures. In addition, the present invention also allows the tester to watch a spatial preview sequence before watching the tested sequence, so that the tester can walk more smoothly and freely in the viewable space during the test time period, thereby significantly reducing the frequency of the tester touching the spatial boundary, which may enable the tester to focus more on the quality evaluation task. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is an overall flow chart of an embodiment of a subjective quality assessment method for volumetric video of the present invention.

[0046] Figure 2 A schematic diagram of a possible implementation method for obtaining a viewable space representation in an embodiment of the present invention.

[0047] Figure 3 It is a schematic diagram of another possible implementation method for obtaining a viewable space representation in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0049] First, the terms involved in the embodiments of the present invention are explained.

[0050] Omnidirectional video: Omnidirectional video, also known as panoramic video or 360° video, is an important scene representation method in virtual reality and computer vision. Omnidirectional video supports users to watch from multiple perspectives, but does not support users to watch from multiple viewpoints.

[0051] Volumetric video: Volumetric video is a type of video that captures information in three-dimensional space and generates a three-dimensional model representation. Volumetric video supports users to watch from multiple viewpoints and perspectives.

[0052] Head-mounted display: A head-mounted display is a display worn on the head that projects images onto the user's retina through a set of optical systems, allowing the user to watch omnidirectional and volumetric videos.

[0053] Volumetric video allows users to watch videos through active interaction and gain a three-dimensional experience, which is more immersive. This is an important feature of volumetric video. With the development of related technologies and people's pursuit of higher-level consumption methods, more and more users are beginning to consume volumetric videos in an active interactive manner. Therefore, the present invention evaluates the subjective quality of volumetric video under the premise of active interactive viewing, which not only retains the important features of volumetric video, but also conforms to the consumption habits of users.

[0054] Omnidirectional video supports users to watch videos of any angle from a single viewpoint, so only the video sequence of the viewpoint is needed to obtain the currently viewed picture through steps such as cropping, rotation and scaling. Volumetric video supports users to watch videos of any viewpoint and any angle, and it is necessary to render the currently viewed picture based on the video sequence of the basic viewpoint. In this process, in order to ensure that the tester feels that the picture is manipulable, the rendering must be completed in real time. Therefore, the present invention uses the video sequence of the basic viewpoint to render the picture presented to the tester in real time according to the position and orientation of the picture manipulated by the tester.

[0055] refer to Figure 1 The first embodiment of the present invention provides a subjective quality assessment method for volumetric video. In this embodiment, the method is described as a series of steps or operations. It should be understood that the steps included in the method in this embodiment can be performed in various orders and / or occur simultaneously, not limited to Figure 1 Regarding the expressions in the embodiments, it should be understood that "in one possible implementation manner" refers to the implementation method of the step.

[0056] like Figure 1 As shown, the subjective quality evaluation method for volumetric video in this embodiment includes the following steps:

[0057] S1. Read the sequence data of the tested volume video through the test software.

[0058] S2. Based on the read sequence data and the preset initial position and orientation, the test software renders the corresponding picture in real time and presents it to the tester through the presentation device.

[0059] S3. Adjust the position and orientation of the picture through the input device, re-render the picture in real time based on the adjusted position and orientation, and present it to the tester through the presentation device.

[0060] S4. Accept and record the subjective opinion scores given by the testers after the viewing time is over.

[0061] Omnidirectional video only allows users to watch from a single viewpoint, and users can always obtain high-quality window video, so there is no restriction on the viewing space in three rotational degrees of freedom. Volumetric video allows users to watch from any viewpoint, and when watching, the video of the user's current viewpoint needs to be interpolated from the video of the basic viewpoint. At this time, the video of the interpolated viewpoint often has a higher quality, while the quality of the extrapolated viewpoint video often decreases. This results in the spatial range of high-quality viewing of volumetric video being limited. Therefore, the present invention will always determine whether the position and orientation of the picture manipulated by the tester exceeds the spatial range. When the position and orientation of the picture exceed the spatial range, the position and orientation are corrected to prevent the quality of the picture presented to the tester from being seriously degraded; at the same time, an alarm is issued to prevent the tester from continuing to move outside the spatial range.

[0062] Based on this, as a further improved implementation scheme, in this embodiment, the method also includes:

[0063] Determine whether the position of the picture exceeds the space range;

[0064] If so, the position and orientation of the image are corrected and an alarm is issued.

[0065] As a preferred implementation, the following method is used to determine whether the position of the picture exceeds the spatial range:

[0066] The coordinates of the viewing position and the representation of the viewable space of the tested sequence data are obtained to determine whether the position is within the viewable space; if so, it is considered that the position of the picture does not exceed the spatial range; if not, it is considered that the position of the picture exceeds the spatial range.

[0067] As a preferred embodiment, the following method is used to obtain the representation of the viewable space:

[0068] In one possible implementation, Figure 2 As shown, the front side forms a ring according to the position of the camera array when the test sequence is captured, and the rear side retains additional convex corners to obtain a representation of the viewable space.

[0069] In one possible implementation, Figure 3 As shown, a sphere is extended at the camera position at each shot of the tested sequence, and all spheres are connected to obtain a representation of the viewable space.

[0070] As a preferred implementation, the following method is used to correct the position and orientation of the picture:

[0071] In one possible implementation, the correction is made by freezing the position and orientation.

[0072] In a possible implementation, the initial position and orientation are corrected by resetting the position and orientation.

[0073] In one possible implementation, no correction is made.

[0074] As a preferred embodiment, the alarm is issued using the following method:

[0075] In one possible implementation, the alarm is issued by emitting a sound.

[0076] In a possible implementation, an alarm is issued by changing the color of a picture.

[0077] The boundaries of the space available for testers to view are virtual walls that do not exist in the actual scene, but prevent testers from passing through like ordinary walls. Before the testers touch the boundaries, the space is invisible and unknown to them, so touching the boundaries may confuse them, which may distract their attention and waste time during the test. Therefore, it is necessary to reduce the frequency with which testers touch the boundaries. The present invention allows testers to watch the spatial preview sequence corresponding to the tested sequence, so that testers can walk more smoothly and freely in the space during the test time period, thereby significantly reducing the frequency of testers touching the spatial boundaries, which may enable testers to focus more on the quality evaluation task.

[0078] Based on this, as a further improved implementation scheme, in this embodiment, the method also includes:

[0079] The spatial preview sequence corresponding to the tested sequence data is presented to the tester through a presentation device, wherein the spatial preview sequence is used to display the spatial range.

[0080] As a preferred implementation, the spatial preview sequence uses the following method to display the spatial range:

[0081] In a possible implementation, the boundary of the space is rendered as a non-completely transparent surface and added to the source sequence corresponding to the tested sequence to obtain a space preview sequence.

[0082] In a possible implementation, the boundary of the bottom surface of the space is rendered as a partially transparent closed graphic, and added to the source sequence corresponding to the tested sequence to obtain a space preview sequence.

[0083] In a possible implementation, the boundary of the space is rendered as a non-completely transparent surface and added to the tested sequence to obtain a space preview sequence.

[0084] In a possible implementation, the boundary of the bottom surface of the space is rendered as a partially transparent closed graphic and added to the tested sequence to obtain a space preview sequence.

[0085] As a preferred implementation scheme, in this embodiment, the tester gives a subjective opinion score on the visual quality of the video according to the five-level visual quality scale shown in Table 1.

[0086] Table 1

[0087]

[0088] It should be noted that the specific target of quality evaluation and the appropriate quality level scale can be selected according to the requirements of the volumetric video subjective test, and the present invention does not impose any specific restrictions on this. The target of quality evaluation can be visual quality, experience quality, etc.

[0089] A second embodiment of the present invention provides a subjective quality assessment device for volumetric video, which is used to implement the method described in the first embodiment, including:

[0090] The test software is used to read the sequence data of the tested volumetric video, and based on the read sequence data and the preset initial position and orientation, render the corresponding picture in real time and output it to the presentation device, or re-render the picture in real time based on the adjusted position and orientation and output it to the presentation device;

[0091] A presentation device, used to present the screen output by the received test software to the tester;

[0092] Input device, used to adjust the position and orientation of the screen according to the user's input operation;

[0093] The testing software is also used to accept and record the subjective opinion scores given by the testers after the viewing time is over.

[0094] As a preferred embodiment, the input device adjusts the position and orientation of the picture according to the user's input operation in the following manners:

[0095] In a possible implementation, the position and orientation of the picture are adjusted through a keyboard.

[0096] In a possible implementation, the position and orientation of the picture are adjusted by a spatial positioning device.

[0097] In a possible implementation, the position and orientation of the picture are adjusted by a joystick.

[0098] As a preferred embodiment, the presenting device presents the screen output by the received test software to the tester in the following manners:

[0099] In a possible implementation, the image is presented to the tester via a common display.

[0100] In a possible implementation, the image is presented to the tester via a head-mounted display.

[0101] In one possible implementation, the images are presented to the tester through VR glasses.

[0102] It should be noted that the method of presenting the screen to the tester can be any method that can be thought of by a person skilled in the art, and the present invention does not specifically limit this. However, it should be noted that the selected method of presenting the screen to the tester and the method of manipulating the position and orientation of the screen need to match. For example: while presenting the screen through an ordinary display, the position and orientation of the screen are manipulated through the keyboard; while presenting the screen through a head-mounted display, the position and orientation of the screen are manipulated through a spatial positioning device, etc.

[0103] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0104] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0105] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A subjective quality assessment method for volumetric video, characterized in that: The steps include: Read the sequence data of the tested volume video through the test software; The test software renders the corresponding images in real time based on the read sequence data and the preset initial position and orientation, and presents them to the tester through the presentation device; Adjust the position and orientation of the screen through the input device, re-render the screen in real time based on the adjusted position and orientation, and present it to the tester through the presentation device; Accept and record subjective opinion scores from testers after viewing time is complete.

2. The method according to claim 1, characterized in that The method further comprises: Determine whether the position of the picture exceeds the space range; If so, the position and orientation of the image are corrected and an alarm is issued.

3. The method according to claim 2, characterized in that The method for determining whether the position of the image exceeds the space range is as follows: The coordinates of the viewing position and the representation of the viewable space of the tested sequence data are obtained to determine whether the position is within the viewable space; if so, it is considered that the position of the picture does not exceed the spatial range; if not, it is considered that the position of the picture exceeds the spatial range.

4. The method according to claim 2, characterized in that: The correction is made by at least one of the following methods: Freeze the position and orientation of the image; Reset the position and orientation of the screen to the initial position and orientation; The alarm is issued by at least one of the following methods: sound an audible alarm; Change the screen color.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: The spatial preview sequence corresponding to the tested sequence data is presented to the tester through a presentation device, wherein the spatial preview sequence is used to display the spatial range.

6. The method according to claim 5, characterized in that The spatial preview sequence uses at least one of the following methods to present the spatial range: Render the boundary of the space as a non-completely transparent surface and add it to the source sequence corresponding to the tested sequence; Render the boundary of the bottom surface of the space as a non-completely transparent closed figure and add it to the source sequence corresponding to the tested sequence; Render the boundary of the space as a non-completely transparent surface and add it to the tested sequence; Render the boundary of the bottom surface of the space as a non-completely transparent closed shape and add it to the tested sequence.

7. A subjective quality assessment device for volumetric video, used to implement the method according to any one of claims 1 to 6, characterized in that: include: The test software is used to read the sequence data of the tested volumetric video, and based on the read sequence data and the preset initial position and orientation, render the corresponding picture in real time and output it to the presentation device, or re-render the picture in real time based on the adjusted position and orientation and output it to the presentation device; A presentation device, used to present the screen output by the received test software to the tester; Input device, used to adjust the position and orientation of the screen according to the user's input operation; The testing software is also used to accept and record the subjective opinion scores given by the testers after the viewing time is over.

8. The device according to claim 7, characterized in that The input device adjusts the position and orientation of the screen according to the user's input operation in at least one of the following ways: Adjust the position and orientation of the screen through the keyboard; Adjust the position and orientation of the image through spatial positioning equipment; Use the joystick to adjust the position and orientation of the screen.

9. The device according to claim 7, characterized in that The presentation device presents the received test software output screen to the tester in at least one of the following ways: The images are presented to the tester through a normal display; The images are presented to the tester through a head-mounted display; The images are presented to the testers through VR glasses.

Images