METHOD, DEVICE AND SYSTEM FOR DISPLAYING DANMAKE COMMENTS IN FREE VIEWPOINT VIDEO

Abstract

The method, device and system for displaying barrage comments of a free viewpoint video relates to the field of Internet technology, and helps a user to concentrate on viewing a current video image and the contents of the barrage comments related to the current video image, thereby improving the viewing experience. The method includes: a terminal displays a first interface of a first free viewpoint video at a first playback time point, the first interface including a video image of the first free viewpoint video at a first viewpoint and a first barrage comment, and receives a first operation, the first operation being an operation of switching a playback viewpoint of the first free viewpoint video from the first viewpoint to a second viewpoint. In response to receiving the first operation, the terminal displays a second interface of the first free viewpoint video at a first playback time point, the second interface including a video image of the first free viewpoint video at a second viewpoint, and the second interface does not include the first barrage comment.

Description

[Technical Field]

[0001] [ Technical Field] This application relates to the field of Internet technology, and in particular to a method, apparatus and system for displaying bullet comments for free viewpoint video. [Background technology]

[0002] Free viewpoint video (FVV) is a special-format video file in which a target object is jointly photographed by multiple surrounding cameras to generate a video stream that is encoded into a file such as a special-format video file. The generated free viewpoint video is then uploaded to a media server for storage and then distributed to a terminal through a content distribution network (CDN). When viewing the free viewpoint video by using a terminal, a user may freely switch viewpoints through a finger drag operation. Compared with traditional video, free viewpoint video allows users to freely select viewpoints, thereby overcoming the drawbacks of traditional video in which the viewpoint is determined by a director. Free viewpoint video provides users with an immersive and interactive playback experience.

[0003] During playback of a free viewpoint video, a user can enable a barrage comment function. Barrage comments refer to the scrolling, leaving, or even displaying a large number of user comments on a video image in an action-like manner, which is currently a popular pastime. It should be noted that a terminal not only displays a video image of a free viewpoint video from a certain viewpoint (e.g., viewpoint A), but also displays barrage comments from all viewpoints at the current playback point. For example, a terminal not only displays barrage comments from viewpoint A, but also barrage comments from viewpoint B, viewpoint C, etc. In this case, in some scenarios, the content of the barrage comments from viewpoint B and / or viewpoint C may not be consistent with the content of the video image from viewpoint A, resulting in confusion and a disorienting and unsatisfactory video experience for the user. Summary of the Invention

[0004] The method, device, and system for displaying danmaku comments in free viewpoint video provided in this application improve the viewing experience by helping users focus on viewing the current video image and the content of the danmaku comments related to the current video image.

[0005] To achieve the above objectives, the embodiments of this application provide the following technical solutions:

[0006] According to a first aspect, there is provided a method for displaying danmaku comments of a free viewpoint video, which is applied to a terminal. The method includes: the terminal displays a first interface of a first free viewpoint video at a first playback time point, the first interface including a video image of the first free viewpoint video at a first viewpoint and a first danmaku comment; the terminal receives a first operation, the first operation being an operation for switching a playback viewpoint of the first free viewpoint video from the first viewpoint to a second viewpoint; and in response to receiving the first operation, the terminal displays a second interface of the first free viewpoint video at the first playback time point, the second interface including a video image of the first free viewpoint video at a second viewpoint, and the second interface not including the first danmaku comment.

[0007] The offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is within a first preset angle range. Therefore, the first danmaku comment is displayed in the first interface. The offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment is outside the first preset angle range. Therefore, the first danmaku comment is not displayed in the second interface.

[0008] When playing the first free viewpoint video, the terminal (Hereafter It can be seen that danmaku comments within the first preset angle range (the visible viewpoint range below) can be displayed, and danmaku comments outside the first preset angle range are not displayed. Since the content of danmaku comments within the first preset angle range is usually strongly related to the content of the video image currently being viewed by the user at the playback viewpoint, the user can concentrate on viewing the current video image and the content of danmaku comments related to the current video image, improving the viewing experience.

[0009] In a possible implementation, the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

[0010] For example, the terminal may predefine the visible viewpoint range of the danmaku comment, or may receive the visible viewpoint range of the danmaku comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than an angle threshold E (e.g., 45 degrees). In this case, the terminal may determine that the visible viewpoint range of the danmaku comment is [playback viewpoint of the first free viewpoint video - angle threshold E, playback viewpoint of the first free viewpoint video + angle threshold E].

[0011] It can be seen that the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is within a first preset angle range, and the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment is outside the first preset angle range. It can be inferred that the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

[0012] In a possible implementation manner, before the terminal receives the first operation, the method further includes: the terminal receives a second operation to pause the playback of the first free viewpoint video.

[0013] In one example, in the process of switching the viewpoint, the terminal automatically pauses the playback of the first free viewpoint video. Therefore, before the user performs the first operation, the terminal plays the video image of the first free viewpoint video at the first viewpoint. When the terminal detects that the user has performed the first operation at the first playback time point, it automatically pauses the playback of the first free viewpoint video. After the first operation is performed and the playback viewpoint of the first free viewpoint video is switched from the first viewpoint to the second viewpoint, the first free viewpoint video starts to be played from the first playback time point.

[0014] In another example, in the process of switching viewpoints, the terminal continuously plays the first free viewpoint video. In this case, before performing the first operation, the user may first manually pause the playback of the first free viewpoint video. In this case, the first free viewpoint video remains at the first playback point. Then, after the user performs the first operation and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, the video image of the first free viewpoint video still remains at the first playback point.

[0015] Alternatively, after the first operation is performed and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, playback starts from a playback point after the first playback point. Alternatively, the user may manually drag the playback progress bar to adjust the playback point back to the first playback point.

[0016] Possible implementations include: , th The second interface includes a second danmaku comment, and the offset angle between the second viewpoint and the viewpoint corresponding to the second danmaku comment is within a first preset angle range.

[0017] In a possible implementation, the sending time of the first danmaku comment is relative to the first playback time, and the sending time of the second danmaku comment is relative to the first playback time.

[0018] It can be understood that the first danmaku comment is played back in the first interface and displayed on the video image at a first playback time point. In this case, the first playback time point is equal to the sending time point of the first danmaku comment, or is after the sending time point of the first danmaku comment and within the continuous display duration of the first danmaku comment. Similarly, the first playback time point is equal to the sending time point of the second danmaku comment, or is after the sending time point of the second danmaku comment and within the continuous display duration of the second danmaku comment.

[0019] In a possible implementation, the first interface displayed by the terminal further includes a first control, the first control being related to a first number of danmaku comments, the first number of danmaku comments being the number of danmaku comments obtained when an offset angle between a viewpoint of the danmaku comments and a playback viewpoint of the terminal is within a second preset angle range, and the second preset angle range does not overlap with the first preset angle range.

[0020] In one example, the first control displays the number of danmaku comments obtained when the offset angle between the viewpoint of the danmaku comments and the playback viewpoint of the terminal is within a second preset angle range.

[0021] In a possible implementation, the method further includes: the terminal receives a third operation for switching viewpoints; the terminal displays a fourth interface during the third operation process, and the number of barrage comments indicated by the first control changes in the fourth interface; and after the third operation process, the terminal displays a second interface of the first free viewpoint video at the second playback point.

[0022] In one possible implementation, the terminal receives a third operation for the first control, and in response to receiving the third operation, the terminal displays a third interface, where the third interface includes the first barrage comment.

[0023] It can be seen that when a user is interested in the content of the barrage comments outside the visible viewpoint range, the content of the barrage comments outside the visible viewpoint range may alternatively be expanded by using corresponding controls to meet the user's viewing requirements in multiple scenarios.

[0024] In a possible implementation, after the terminal displays a second interface of the first free viewpoint video at the first playback time point in response to receiving the first operation, the method further includes: the terminal receives a fourth operation, the fourth operation being an operation for sending a third danmaku comment; and the terminal sends data of the third danmaku comment to a danmaku comment server in response to receiving the fourth operation, where the data of the third danmaku comment includes a fourth viewpoint.

[0025] It can be seen that when the user sends a third danmaku comment in the second interface, an association is established between the viewpoint of the third danmaku comment and the playback viewpoint of the first free viewpoint video in the second interface, i.e., the fourth viewpoint. Subsequently, when the third danmaku comment is displayed, it is determined whether to present the third danmaku comment based on the relationship between the playback viewpoint of the first free viewpoint video and the viewpoint of the third danmaku comment.

[0026] In a possible implementation, the first interface further includes a fourth barrage comment, the fourth barrage comment corresponding to the first viewpoint.

[0027] In a possible implementation, in the first interface, the first style of the first barrage comment is different from the second style of the fourth barrage comment.

[0028] It can be seen that the user may distinguish the danmaku comments of the first free viewpoint video at the playback viewpoint from the danmaku comments at other viewpoints based on the presentation style of the danmaku comments (e.g., the first style or the second style). In this way, the user will not be confused or suspicious when the video image at the playback viewpoint does not match the content of the danmaku comments at other viewpoints. This improves the viewing experience of the first free viewpoint video.

[0029] In a possible implementation, before the terminal displays a first interface of the first free viewpoint video at the first playback time point, the method includes: the terminal determines to display the first danmaku comment in a first style on the first interface based on a first offset angle between the first viewpoint and a third viewpoint corresponding to the first danmaku comment.

[0030] In a possible implementation, when the first offset angle is positive, the first style is , bullets The text direction of the curtain comment is a direction obtained by rotating the first direction clockwise by a first angle, and the first direction is a direction parallel to a first bezel of a screen of a terminal, or when the first offset angle is a negative value, the first style is , bullets The text direction of the curtain comment is the direction obtained by rotating the first direction counterclockwise by the first angle.

[0031] In a possible implementation, the method further includes: the terminal receives a fifth operation for switching viewpoints; the terminal displays a fifth interface during the fifth operation, where the fifth interface and the first interface are the same; and after the fifth operation, the terminal displays a sixth interface of the first free viewpoint video at the second playback time point.

[0032] In other words, in the fifth operation process, the video image of the first free viewpoint video displayed by the terminal remains unchanged, and the presentation method of the barrage comments also remains unchanged.

[0033] In a possible implementation manner, the method further includes: the terminal receives a sixth operation for switching a viewpoint.

[0034] The terminal displays a seventh interface during the sixth operation process, and the seventh interface includes a video image of the first free viewpoint video at a third playback point in time. After the sixth operation process, the terminal displays an eighth interface of the first free viewpoint video at a second playback point in time, and the third playback point in time is after the first playback point in time and before the second playback point in time.

[0035] In other words, in the sixth operation process, the terminal continuously plays the first free viewpoint video, and therefore the playback time point of the first free viewpoint video changes.

[0036] In a possible implementation, the method further includes: the seventh interface further includes a fifth barrage comment, and the first interface does not include the fifth barrage comment.

[0037] In other words, in the sixth operation process, the terminal continuously plays the first free viewpoint video, and the barrage comments displayed by the terminal also change accordingly as the playback time of the first free viewpoint video changes.

[0038] In a possible implementation, the method further includes: the seventh interface further includes a first barrage comment, and the display style of the first barrage comment in the seventh interface is different from the display style of the first barrage comment in the first interface.

[0039] In other words, in the sixth operation process, the terminal adjusts the display style of each barrage comment in real time according to the change of the playback viewpoint.

[0040] In a possible implementation, the seventh interface further includes a sixth danmaku comment, and the appearance time of the sixth danmaku comment is after the first playback time.

[0041] In other words, after the viewpoint of the first free viewpoint is switched, the terminal continuously plays back the first free viewpoint video, and the content of the barrage comments changes accordingly as the playback time point of the first free viewpoint video changes.

[0042] According to a second aspect, there is provided a terminal including a processor, a memory, and a touchscreen. The memory and the touchscreen are coupled to the processor, and the memory is configured to store computer program code, the computer program code including computer instructions. When the processor reads the computer instructions from the memory, the terminal is enabled to perform the method according to the first aspect and any one of the possible implementations of the first aspect.

[0043] According to a third aspect, there is provided a computer-readable storage medium comprising computer instructions which, when executed on a terminal, enable the terminal to perform a method according to the first aspect and any one of the possible implementations of the first aspect.

[0044] According to a fourth aspect, there is provided a computer program product. or terminal When executed above, Computer or The terminal is enabled to execute the method according to the first aspect and any one of the possible implementations of the first aspect.

[0045] For technical effects that can be achieved by the terminal, computer-readable storage medium, and computer program product provided in the second to fourth aspects, please refer to the description of the technical effects in the first aspect and possible implementation methods of the first aspect, and details will not be described again here. [Brief explanation of the drawings]

[0046] [Figure 1] 1 is a schematic diagram of the structure of a system for playing free viewpoint video according to an embodiment of this application; [Figure 2]1 is a schematic diagram of the structure of a terminal according to an embodiment of this application; [Figure 3] FIG. 2 is a schematic diagram of the structure of a server according to an embodiment of the present application; [Figure 4] 1 is a schematic flowchart of a method for transmitting barrage comments of a free viewpoint video according to an embodiment of the present application; [Figure 5(1)] 1 is a schematic diagram of several terminal interfaces according to an embodiment of the present application; [Figure 5(2)] 1 is a schematic diagram of several terminal interfaces according to an embodiment of the present application; [Figure 6(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 6(2)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 6(3)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 7] 1 is a schematic flowchart of a method for displaying barrage comments according to an embodiment of the present application; [Figure 8] 1 is a schematic diagram of a relationship between a viewpoint of a barrage comment and a playback viewpoint of a free viewpoint video according to an embodiment of the present application; [Figure 9(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 9(2)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 9(3)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 10A] 1 is a schematic flowchart of a method for displaying barrage comments in a process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 10B(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 10B(2)]3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 10B(3)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 11A] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 11B(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 11B(2)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 11B(3)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 12A] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 12B] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 13] FIG. 10 is a schematic diagram of another relationship between the viewpoint of the barrage comments and the playback viewpoint of the free viewpoint video according to an embodiment of the present application. [Figure 14A(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 14A(2)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 14A(3)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 14B(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 14B(2)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 14B(3)]3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 15A] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 15B(1)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 15B(2)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 15B(3)] 3A-3C are schematic diagrams of some other terminal interfaces according to embodiments of the present application. [Figure 16A] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 16B] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; [Figure 17] 10 is a schematic flowchart of another method for displaying barrage comments in the process of switching viewpoints of a free viewpoint video according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0047] In order to facilitate understanding of the technical solutions provided in the embodiments of this application, technical terms in the embodiments of this application will be explained first.

[0048] In free viewpoint video, a target object is simultaneously photographed by multiple cameras, and during playback, this is equivalent to a user selecting the photographing angle of one of the cameras for viewing. Free viewpoint video may be generated in several ways:

[0049] Method 1: Multiple cameras arranged around a target object jointly photograph the target object by using the target object as the center of the image, and specific video encoding is performed on the jointly photographed video stream to generate a special format video file. When a user views a free viewpoint video, this is equivalent to the viewer selecting a viewpoint around the photographed target object for viewing.

[0050] Method 2: Multiple cameras are arranged around the viewer using the viewer as the center to jointly capture a target object, and specific video encoding is performed on the jointly captured video stream to generate a special format video file, for example, to generate a 3-DoF (degrees of freedom) video (e.g., a 360-degree panoramic video). When a user views a free viewpoint video, this is equivalent to the viewer rotating their head to view the surrounding scenery without moving.

[0051] Method 3: The above two methods are combined. For example, a 6-DoF video is generated. When viewing a free viewpoint video, a user can move forward, backward, left, or right to view the video, or rotate their head to view the surrounding scenery.

[0052] The camera position is the number of the position where each camera is located when multiple cameras around the target object or view capture the target object. Generally, one camera position corresponds to one shooting angle. At one shooting angle, images of the target object within a specific angle range may be captured.

[0053] Perspective is also viewing point , also called a playback viewpoint, is a shooting angle at which a user views a free viewpoint video. Specifically, the image of the target object seen at a viewpoint is captured at the viewpoint of the camera. of It may be considered an image captured when performing photography at a position (i.e., camera position).

[0054] Video playback is the process of displaying a group of images in chronological order in a display window and playing the corresponding audio after decoding, rendering, etc. have been performed on a video file (including free viewpoint video).

[0055] Danmaku comments are comments or annotations that are displayed on a video (including free viewpoint video) at a specific time or for a specific period of time according to specific rules, providing an interesting and unexpected video viewing experience.

[0056] 1 is a schematic diagram of a system architecture according to an embodiment of this application. The system architecture includes a camera array 500, a content distribution network (CDN) 200, a media server 300, a CDN 400, a terminal 100, and a barrage comment server 600.

[0057] The camera array 500 (Photographed Shadow Object / Photography scene ) and / or a plurality (e.g., 18-36) of cameras or other devices with camera lenses located around the viewer. These cameras correspond to camera position 0, camera position 1, ..., and camera position n, respectively. Camera array 500 separately captures images of a target object from multiple angles and uploads the video streams captured from the multiple angles to media server 300. In some examples, synchronization of the images captured by the multiple cameras in the camera array or other devices with camera lenses may be achieved using a metronome.

[0058] By using functional modules such as a load balancing module, a content distribution module, and a scheduling module on a central platform, the CDN 200 enables video streams collected by a camera array 500 from multiple angles to be uploaded to a media server 300 in an accelerated manner by using edge servers located at each location.

[0059] The media server 300 is configured to transcode the received video streams at multiple angles to generate free-viewpoint video. For example, processing processes performed by the media server 300 on the video streams at multiple angles include, but are not limited to, focusing, frame alignment synchronization encoding, rotation auxiliary stream generation, virtual viewpoint synthesis, etc. Focusing may be used to ensure a smooth transition of the image of a captured object in the process of switching viewpoints, such as by cropping an image. Frame alignment synchronization encoding is used to align video frames captured by multiple cameras at the same time. In virtual viewpoint synthesis, a virtual video stream is inserted between two real video streams to reduce the number of cameras that need to be used during shooting or to implement a smooth transition of the image of a captured object in the process of switching viewpoints. The rotation auxiliary stream is used for displaying in a rotational manner, which helps reduce the total stream usage.

[0060] It should be noted that the above processing process of the media server 300 is merely an example, and the specific technical solutions used in the process of generating free-viewpoint videos based on the video streams uploaded by the camera array 500 are not limited to the embodiments of this application.

[0061] It should be further noted that the media server 300 herein may be at least one of an independent physical server, a plurality of independent physical servers, a cloud server providing cloud computing, a cloud computing platform, and a virtual center.

[0062] It can be understood that the media server 300 stores and maintains a large amount of free-viewpoint video. When the media server 300 receives a request sent by the terminal 100 for free-viewpoint video content, the media server 300 quickly returns the corresponding free-viewpoint video content to the terminal 100 through the CDN 400.

[0063] It should be noted that the media server 300 here may be at least one of an independent physical server, a plurality of independent physical servers, a cloud server providing cloud computing, a cloud computing platform, and a virtual center.

[0064] The CDN 400 enables terminal users (e.g., terminal 100) to obtain nearby required content such as free-viewpoint video by using functional modules such as a load balancing module, a content distribution module, and a scheduling module on a central platform, and by using edge servers located at each location, thereby reducing network congestion and improving user access response speed and hit rate.

[0065] The terminal 100 may request a free-viewpoint video from the media server 300 by using a free-viewpoint video client / application installed on the terminal 100, and then receive the free-viewpoint video file returned by the media server 300 through the CDN 400. Alternatively, the terminal 100 downloads the free-viewpoint video file from the corresponding server based on the playback address of the free-viewpoint video returned by the media server 300. Finally, the terminal performs processing such as decoding and rendering on the free-viewpoint video file, and then plays the free-viewpoint video. If the terminal 100 also enables the barrage comment function when playing back the free-viewpoint video, the terminal 100 needs to further request barrage comment data corresponding to the current playback time point or period from the barrage comment server 600.

[0066] In this embodiment of the present application, the barrage comment data returned by the barrage comment server 600 includes the content of the barrage comment, the time when the barrage comment appeared, the viewpoint of the barrage comment, etc. The viewpoint of the barrage comment is the viewpoint of the free viewpoint video that was being viewed by the user when the barrage comment was sent. The content of the barrage comment sent by the user is usually The view For example, the content of a barrage of comments sent by a user is usually related to the video image at the current time. The view In other words, in this embodiment of the present application, the barrage comments sent by the user are related to the current playback viewpoint.

[0067] In this embodiment of the application, the terminal 100 uses different display methods for multiple danmaku comments at different viewpoints based on the viewpoint of the danmaku comments to help the user distinguish between multiple danmaku comments at different viewpoints. In other words, the danmaku comments at different viewpoints are displayed differently. For example, in a video image at the user's current playback viewpoint 1, danmaku comments at viewpoint 1 and viewpoint 2 are currently displayed. In this case, the terminal 100 separately displays the danmaku comments at viewpoint 1 and the danmaku comments at viewpoint 2 in different display methods, thereby allowing the user to intuitively distinguish between the danmaku comments at viewpoint 1 and the danmaku comments at viewpoint 2. Danmaku comments at viewpoint 2 is seen The user will not have confusion or embarrassment when there is a mismatch with the video image at point 1. This improves the video viewing experience.

[0068] Furthermore, when playing back the free viewpoint video, terminal 100 may further receive new content of the danmaku comment entered by the user, establish an association between the new content of the danmaku comment and the viewpoint of the video currently being viewed by the user, and upload the association to danmaku comment server 600 for storage.

[0069] The terminal 100 further receives a viewpoint switching operation performed by the user (for example, a user's finger sliding operation on the screen or a user's operation of pressing a button on a remote control), and calculates a viewpoint switching direction (for example, switching the viewpoint to the left or switching the viewpoint to the right) and a switching angle amplitude (for example, switching by 10 degrees or 20 degrees) based on the viewpoint switching operation performed by the user, and outputs a current free viewpoint video. The view In some examples, the display of the terminal 100 is a touch screen. In this case, the terminal 100 captures screen events, the screen events including detected finger slide events, including finger slide direction and distance, and performs a calculation based on the screen events. , visual In some other examples, the terminal 100 may be configured as a remote control. In this case, the terminal 100 may receive button events from the remote control, where the button events include a button identifier, the number of button presses or the duration of button presses, etc., and then perform a point switching operation based on the button events. Look The switching direction and switching angle of the points may be determined, for example, by identifying the button operated by the user based on the button identifier, e.g., tapping the left button or the right button, to determine the switching direction, and the switching angle amplitude may be identified based on the number of button presses or the duration of the button presses.

[0070] For example, the terminal 100 in this embodiment of the present application may be, for example, a mobile phone, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a netbook, a wearable device, an augmented reality (AR) device, a virtual reality (VR) device, an in-vehicle device, a smart screen, or an intelligent vehicle. The specific form of the terminal 100 is not particularly limited in this application.

[0071] The barrage comment server 600 is configured to receive barrage comment information transmitted by each terminal (including, for example, terminal 100), and the barrage comment information includes, but is not limited to, an identifier of the free viewpoint video, the time when the barrage comment appears, the duration of the barrage comment, the content of the barrage comment, the viewpoint of the barrage comment, etc.

[0072] It should be noted that the barrage comment server 600 here may be at least one of an independent physical server, a plurality of independent physical servers, a cloud server providing cloud computing, a cloud computing platform, and a virtual center.

[0073] The system shown in this embodiment of the present invention architecture It can be understood that the above does not constitute a specific limitation on the system architecture to which the technical solutions are applicable. In some other embodiments of this application, the system architecture to which the technical solutions provided in the embodiments of this application are applicable may include more or fewer devices than those shown in the drawings. or combination of parts of the device or separation from parts of the device It may also include stomach .

[0074] 2 is an exemplary diagram of the structure of a terminal 100 according to an embodiment of this application. The terminal 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, a subscriber identification module (SIM) card interface 195, etc.

[0075] It can be understood that the structure shown in this embodiment of the present invention does not constitute a particular limitation on the terminal 100. In some other embodiments of this application, the terminal 100 may include more or fewer components than those shown in the drawings. or the combination of parts of the components, or the division of parts of the components, or the arrangement of different components It may include stomach The components shown in the figures may be realized in hardware, software, or a combination of software and hardware.

[0076] Processor 110 may include one or more processing units. For example, processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural-network processing unit (NPU), and / or the like. The different processing units may be separate components or may be integrated into one or more processors.

[0077] A video codec is configured to compress or decompress digital video. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record video in multiple encoding formats, such as Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-3, and MPEG-4. In this embodiment of the present application, the video codec may decode free-viewpoint video downloaded by terminal 100 from media server 300 to enable playback on terminal 100.

[0078] The internal memory 121 may be configured to store computer-executable program code. The executable program code includes instructions. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, applications required by at least one function (e.g., a free-viewpoint video playback function), etc. The data storage area may store data generated when the terminal 100 is used (e.g., audio data and contacts), etc. Furthermore, the internal memory 121 may include high-speed random access memory, or may include non-volatile memory, such as at least one magnetic disk storage device, flash memory, or universal flash storage (UFS). The processor 110 executes instructions stored in the internal memory 121 and / or instructions stored in memory located in the processor to perform various functional applications and data processing of the terminal 100.

[0079] In some embodiments of this application, internal memory 121 stores program code required by a free viewpoint video client / application. The free viewpoint video client / application may implement functions for downloading free viewpoint video from media server 300 and playing the free viewpoint video, receive the content of danmaku comments entered by a user, upload the content of the danmaku comments entered by a user and other danmaku comment information (e.g., danmaku comment viewpoint information) to danmaku comment server 600, receive danmaku comment data sent by danmaku comment server 600, and display danmaku comments at different viewpoints differently based on the viewpoint information carried in the danmaku comment data, etc.

[0080] The wireless communication function of the terminal 100 may be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, the baseband processor, and so on.

[0081] Mobile communication module 150 may provide wireless communication solutions, including 2G / 3G / 4G / 5G / 6G / next generation wireless communication technologies, etc., applied to terminal 100. In some embodiments, at least some of the functional modules in mobile communication module 150 may be located in processor 110. In some embodiments, at least some of the functional modules of mobile communication module 150 may be located in the same device as at least some of the modules of processor 110.

[0082] The wireless communication module 160 may provide a solution for wireless communication applied to the terminal 100, including a wireless local area network (WLAN) (e.g., a wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0083] Terminal 100 may implement display functions by using a GPU, display 194, an application processor, etc. The GPU is a microprocessor for image processing and is connected to display 194 and the application processor. The GPU is configured to perform mathematical and geometric calculations and render images. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0084] The terminal 100 may realize the photographing function by using an ISP, a camera 193, a video codec, a GPU, a display 194, an application processor, and the like.

[0085] The external memory interface 120 may be used to connect an external memory card, such as a microSD card, to expand the storage of the terminal 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and video files may be stored on the external memory card.

[0086] Terminal 100 may implement audio functions such as music playback and recording using audio module 170, speaker 170A, receiver 170B, microphone 170C, headset jack 170D, application processor, and the like.

[0087] Charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, charging management module 140 may receive the charging input of the wired charger through USB interface 130. In some embodiments in which wireless charging is performed, charging management module 140 may receive the wireless charging input through a wireless charging coil of terminal 100. When charging battery 142, charging management module 140 may also provide power to the terminal by using power management module 141.

[0088] Power management module 141 is configured to connect to battery 142, charging management module 140, and processor 110. Power management module 141 receives input from battery 142 and / or charging management module 140 and provides power to processor 110, internal memory 121, display 194, camera 193, wireless communication module 160, etc. Power management module 141 may be further configured to monitor parameters such as battery capacity, battery cycle count, and battery health status (fault or impedance). In some other embodiments, power management module 141 may alternatively be located in processor 110. In some other embodiments, power management module 141 and charging management module 140 may alternatively be located in the same component.

[0089] 3 is a schematic diagram of the structure of the barrage comment server 600. The barrage comment server 600 includes one or more processors 610, one or more memories 620, and one or more communication interfaces 630.

[0090] The processor 610, memory 620, and communication interface 630 are connected via a bus. The processor 610 may include a general-purpose central processing unit (CPU) (e.g., CPU0 and CPU1), a microprocessor, an application-specific integrated circuit (ASIC), a graphics processing unit (GPU), a neural-network processing unit (NPU), an integrated circuit configured to control program execution of the solutions of this application, etc.

[0091] The memory 620 may be configured to store computer-executable program code. The executable program code includes instructions. The memory 620 may include a program storage area and a data storage area. The program storage area may store an operating system, application code, etc. In some examples, the data storage area stores danmaku comment data for each free viewpoint video uploaded by a user, and the danmaku comment data includes the viewpoint of each danmaku comment. Furthermore, the memory 620 may include high-speed random access memory and may further include non-volatile memory, such as at least one magnetic disk storage device, flash memory, or universal flash storage (UFS). The processor 610 executes instructions stored in the memory 620 to perform various functional applications and data processing of the danmaku comment server 600. In one example, the processor 610 may alternatively include multiple CPUs, and the processor 610 may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. A processor herein may be one or more devices, circuits, or processing cores configured to process data (e.g., computer program instructions).

[0092] The communication interface 630 may be configured to communicate with other devices or communication networks, for example, Ethernet or a wireless local area network (WLAN).

[0093] It should be noted that the media server 300, the servers in CDN 200 or CDN 400, etc. shown in Fig. 1 refer to the structure of the barrage comment server 600 shown in Fig. 3. The media server 300 and the servers in CDN 200 or CDN 400 may include more or fewer components than the barrage comment server 600, or some components may be combined, or some components may be separated, or a different component layout may exist. This is not limited to the embodiments of this application.

[0094] DETAILED DESCRIPTION OF THE INVENTION With reference to the accompanying drawings and specific embodiments, the following describes in detail the technical solutions provided in the embodiments of this application.

[0095] 1. Solution for sending barrage comments

[0096] 4 is a schematic flowchart of a method for sending barrage comments according to an embodiment of the present application. The method includes the following steps:

[0097] S400: Terminal 100 plays back the video image of the first free viewpoint video at viewpoint A.

[0098] For example, a client (e.g., a web page) or an application (e.g., a video application for a mobile terminal) used to play free-viewpoint video is installed on terminal 100. The client / application may be referred to as a free-viewpoint video client / application. A user may launch the client / application and select to play the first free-viewpoint video in the display interface of the client / application. In some embodiments, terminal 100 plays a video image of the first free-viewpoint video from viewpoint A by default. When detecting a user's operation to select playback of the first free-viewpoint video, terminal 100 requests video file 1 of the first free-viewpoint video from viewpoint A from media server 300 via CDN 400. In response, media server 300 returns video file 1 to terminal 100 via CDN 400, and terminal 100 displays a video image of the first free-viewpoint video from viewpoint A based on video file 1. In some other embodiments, terminal 100 first plays a video image of the first free-viewpoint video from a viewpoint other than viewpoint A. When the terminal 100 detects an operation by the user indicating switching to viewpoint A, the terminal 100 requests video file 1 of the first free viewpoint video at viewpoint A from the media server 300 via the CDN 400. In response, the media server 300 returns video file 1 to the terminal 100 via the CDN 400, and the terminal 100 displays a video image of the first free viewpoint video at viewpoint A based on video file 1.

[0099] In some other examples, when terminal 100 plays back the first free viewpoint video, the danmaku comment function is enabled. In this case, terminal 100, based on the playback progress, further requests danmaku comment data at the corresponding playback time point from danmaku comment server 600. In this case, the video image played back by terminal 100 further includes the corresponding danmaku comment.

[0100] For example, an example in which the terminal 100 is a mobile phone is used for explanation. A video application installed on the mobile phone supports free-viewpoint video playback by integrating a software development kit (SDK) for playing free-viewpoint videos. In this case, a user can launch the video application on the mobile phone and select to play a first free-viewpoint video in the video application interface. For example, FIG. 5A shows a playback interface 601 displayed by the mobile phone and used to play a first free-viewpoint video. The playback interface 601 includes a video image 602 of the first free-viewpoint video at a specific viewpoint and danmaku comments (including, for example, danmaku comments 603) that appear on the video image 602 according to specific rules. The playback interface 601 may further include several function controls, such as a playback progress bar 604, a playback control 609, a next episode control 610, a danmaku enable / disable control 605, a danmaku comment input box 606, a danmaku comment send control 607, and a viewpoint control 608.

[0101] The playback progress bar 604 is used to display the playback progress of the current video, and is used by the user to adjust the playback progress of the current video by dragging, etc. The playback control 609 is used to pause or continuously play the current video. The next episode control 610 is used to switch to the next episode or the next video. The danmaku enable / disable control 605 is used to enable or disable the danmaku comment function. The danmaku comment input box 606 is used to accept the content of the danmaku comment entered by the user. The danmaku comment send control 607 is used to send the content of the danmaku comment entered by the user or the default content of the danmaku comment.

[0102] The viewpoint control 608 is used to display a viewpoint (e.g., 60 degrees) corresponding to the video image 602 currently being played by the mobile phone, and is used to call up a viewpoint ruler used to switch viewpoints. When it is detected that the user has operated the viewpoint control 608, the mobile phone displays a playback interface 611 shown in FIG. 5(2), and a viewpoint ruler 612 is displayed on the playback interface 611. The user may switch the viewpoint of the free viewpoint video currently being played by the mobile phone by sliding left or right on the viewpoint ruler 612 using their finger. For example, when it is detected that the user's finger slides right on the ruler, the mobile phone displays a playback interface 613 shown in FIG. 6(1). The playback interface 613 is a video image of the first free viewpoint video at viewpoint A (e.g., 70 degrees).

[0103] It should be noted that the example in which the user invokes the viewpoint ruler 612 by operating the viewpoint control 608 is used here for the purpose of explanation. In some other examples, the mobile phone may alternatively directly display the viewpoint ruler 612 on the free viewpoint video playback interface 601, invoke the viewpoint ruler 612 in other ways, etc. The interface for displaying the viewpoint ruler 612 is not limited in the embodiments of this application.

[0104] In some examples, after not receiving any operation performed by the user on the viewpoint ruler for preset duration 1, the mobile phone may hide the viewpoint ruler, for example, as shown in playback interface 614 in Fig. 6(2). In some other examples, when detecting a user tapping operation on the display in playback interface 614, the mobile phone displays playback interface 615 shown in Fig. 6(3), and the above-mentioned function controls, such as playback progress bar 604, playback control 609, next episode control 610, danmaku enable / disable control 605, danmaku comment input box 606, danmaku comment send control 607, and viewpoint control 608, are redisplayed in playback interface 615.

[0105] S401: Terminal 100 receives an operation by a user to input the content of a first barrage comment and send the first barrage comment, and terminal 100 generates data of the first barrage comment, and the data of the first barrage comment includes the content of the first barrage comment, the time when the first barrage comment appeared, and the viewpoint of the first barrage comment.

[0106] For example, a user may input and send a barrage comment through the video playback interface 601 shown in Fig. 5(1) or the video playback interface 615 shown in Fig. 6(3). Here, an example in which a user inputs a first barrage comment into the video playback interface 615 shown in Fig. 6(3) and sends the first barrage comment will be used for explanation.

[0107] As shown in FIG. 6(3), when the mobile phone plays back the first free viewpoint video at viewpoint A, the mobile phone detects that the user has input the content of the first danmaku comment into the danmaku comment input box 606 and operated the danmaku comment send control 607. The mobile phone generates data of the first danmaku comment. In an embodiment of this application, the data of the first danmaku comment generated by the mobile phone includes the content of the first danmaku comment (e.g., "Amazing"), the time when the first danmaku comment appeared (e.g., 20:10), the viewpoint of the first danmaku comment, and the time when the first danmaku comment appeared (e.g., 20:10). (Visual The data of the first danmaku comment further includes an identifier of the first free viewpoint video, an identifier of the first danmaku comment, a duration ΔT of the first danmaku comment, a format of the first danmaku comment, and a transmission position of the first danmaku comment. (No. The data format further includes one or more of the following: (a) a playback time at which the first barrage comment is sent; (b) a playback time at which the first barrage comment is sent; etc. Table 1 below shows an example of a data format used for the data of the first barrage comment.

[0108] The value of the movie identifier field is the identifier of the first free viewpoint video, for example, movie 1. The value of the danmaku comment identifier field is the identifier of the first danmaku comment, for example, danmaku comment 1. The value of the danmaku comment content field is the content of the first danmaku comment entered by the user, for example, "Amazing."

[0109] The value of the appearance time field is the playback time when the display starts to display the first danmaku comment, for example, playback time T1. In some embodiments, playback time T1 is equal to the transmission time of the first danmaku comment. For example, terminal 1 00If a user sends a first danmaku comment at 20:10 in the free viewpoint video, the appearance time of the first danmaku comment, i.e., playback time T1, may be 20:10. It may be understood that in some other embodiments, the appearance time of the first danmaku comment is different from the transmission time of the first danmaku comment. For example, in some scenarios, the duration of a highlight in a video is long, or a user is accustomed to sending a danmaku comment at the start of a highlight. In this case, the transmission time of the first danmaku comment is determined as playback time T1 to remind other users to pay attention to the highlight in the video. In some other embodiments, playback time T1 is equal to the transmission time of the first danmaku comment minus preset time offset value 1. Preset time offset value 1 may be predefined or may be set by the user. This is not limited to the embodiments of this application. The setting and calculation methods of other preset time offset values ​​in this specification (e.g., preset time offset value 2) are similar to those of preset time offset value 1 and may be the same as or different from those of preset time offset value 1. Details will not be described below. For example, if the sending time of the first danmaku comment is 20:10 and the preset time offset value 1 is 5 seconds, the appearance time of the first danmaku comment, i.e., the playback time T1, is 20:05. In some other scenarios, it may be understood that the duration of a highlight in a video is short, or that a user is accustomed to sending a danmaku comment at the end of a highlight. In this case, the sending time of the first danmaku comment minus the preset time offset value 1 is determined as the playback time T1, to remind other users in advance to pay attention to the highlight in the video. In some other embodiments, the playback time T1 is equal to the sending time of the first danmaku comment plus the preset time offset value 2. For example, if the sending time of the first danmaku comment is 20:10 and the preset time offset value 2 is 3 seconds, the appearance time of the first danmaku comment, i.e., the playback time T1, is 20:08. The preset time offset value 1 and the preset time offset value 2 may be the same or different.In some other scenarios, it may be understood that users are accustomed to holding discussions etc. before highlights. In this case, the sending time point of the first barrage comment plus the preset time offset value 2 is determined as the playback time point T1, reminding other users to pay attention to the highlights in the video in a timely manner.

[0110] The value of the Duration field is the duration ΔT (e.g., 15 seconds) during which the first danmaku comment is continuously displayed on the display. In some examples, the first danmaku comment starts to appear from the rightmost side of the terminal display and then moves from the rightmost side to the leftmost side of the display until it disappears from the display. The entire process from when the first danmaku comment is displayed from the rightmost side of the display to when it disappears from the leftmost side of the display takes ΔT, e.g., 15 seconds. The duration ΔT may be a preset value or may be set by the user. For example, the user may set a danmaku comment mode, and different danmaku comment modes correspond to different durations. The value of the Font or Color field, etc. is set by the user when the user enters the content of the first danmaku comment, or is set by default on the terminal 100.

[0111] The value of the viewpoint field is the viewpoint corresponding to the video image currently being displayed by terminal 100 when the first danmaku comment was sent, i.e., viewpoint A. In other words, a correspondence relationship is established between the first danmaku comment and viewpoint A.

[0112] S402: The terminal 100 transmits the data of the first barrage comment to the barrage server 600.

[0113] In some embodiments, after receiving the data of the first barrage comment sent by terminal 100, the barrage comment server reviews the data of the first barrage comment, for example, reviews the content and reviews the format. After the review is successful, the barrage comment server stores the data of the first barrage comment.

[0114] In this way, the barrage comment server 600 stores the barrage comment data of the first free viewpoint video transmitted from each viewpoint and uploaded by each user.

[0115] It can be understood that the danmaku comment server 600 stores a large amount of danmaku comment data for free viewpoint video at each viewpoint. Subsequently, other terminals (including terminal 100) may request danmaku comment data for a specific free viewpoint video at a specific playback point from the danmaku comment server 600. Furthermore, the danmaku comment data returned by the danmaku comment server 600 includes the content of the danmaku comment, the time when the danmaku comment appeared, the viewpoint of the danmaku comment, etc. Subsequently, when displaying the danmaku comments, the other terminals may determine different display methods for the danmaku comments at different viewpoints.

[0116] 2. Solution for displaying barrage comments

[0117] For example, the method for displaying barrage comments provided in the embodiments of this application will be described by using an example in which a terminal 100 (here, the terminal 100 may be the same as or different from the terminal 100 that sends the first barrage comment) plays a first free viewpoint video.

[0118] 7 is a schematic flowchart of a method for displaying barrage comments according to an embodiment of the present application. The method includes the following steps:

[0119] S701: The terminal 100 requests a video file of a first free viewpoint video at viewpoint B at playback time point T2 from the media server 300. In response, the media server 300 returns the video file of the first free viewpoint video at viewpoint B at playback time point T2 to the terminal 100.

[0120] For example, the user may select to play a video from any viewpoint by using a free viewpoint video client / application of the terminal 100. Based on the playback progress of the first free viewpoint video, the terminal 100 requests a video file at the corresponding playback time point from the media server 300. For example, the terminal 100 requests a video file of the first free viewpoint video from viewpoint B at playback time point T2 from the media server 300.

[0121] S702: The terminal 100 further requests danmaku comment data for the first free viewpoint video at playback time point T2 from the danmaku comment server 600. In response, the danmaku comment server 600 returns the danmaku comment data for the first free viewpoint video at playback time point T2 to the terminal 100.

[0122] When the terminal 100 is playing back a first free viewpoint video and the danmaku comment function is enabled, the terminal 100 must further request danmaku comment data at the corresponding playback time point from the danmaku comment server 600, based on the playback progress of the first free viewpoint video. For example, the terminal 100 requests danmaku comment data for the first free viewpoint video at playback time point T2 from the media server 300. In this case, it should be noted that the danmaku comment data for the first free viewpoint video at playback time point T2 returned to the terminal 100 by the danmaku comment server 600 includes danmaku comments for each viewpoint, for example, a danmaku comment for viewpoint B, a danmaku comment for viewpoint A, and danmaku comments for other viewpoints.

[0123] It should be noted that step S702 may be performed after, before, or simultaneously with step S701. This is the case in the embodiment of this application. is limitedNot determined.

[0124] S703: The terminal 100 calculates the offset angle between viewpoint B and the viewpoint of each barrage comment.

[0125] In some embodiments, the offset angle includes an offset direction and / or an offset angle value. The offset direction includes a left offset direction and a right offset direction. If the offset direction is a left offset direction, this means that viewpoint B needs to be switched to the left when it is switched to the viewpoint of the danmaku comment. If the offset direction is a right offset direction, this means that viewpoint B needs to be switched to the right when it is switched to the viewpoint of the danmaku comment. The offset angle value is the absolute value of the difference between viewpoint B and the viewpoint of the danmaku comment. In other embodiments, offset angle = viewpoint of the danmaku comment - viewpoint B. When the offset angle is a positive number, this indicates that the danmaku comment is a right offset danmaku comment. When the offset angle is a negative number, this indicates that the danmaku comment is a left offset danmaku comment. The definition and calculation of the offset angle are described in the embodiments of this application. is limited This specification will be described by using an example where the offset angle is equal to the viewpoint of the barrage comment minus viewpoint B. Details will not be given later.

[0126] For example, in the playback interface 613 shown in FIG. 6(1), the current playback viewpoint (Visual Point B) is 70 degrees, and the viewpoint of Danmaku Comment 1 is 60 degrees. In this case, the offset angle between viewpoint B and the viewpoint of Danmaku Comment 1 is a left offset of 10 degrees. When the viewpoint of Danmaku Comment 2 is 100 degrees, the offset angle between viewpoint B and the viewpoint of Danmaku Comment 2 is a right offset of 30 degrees.

[0127] In another example, as described above, the free viewpoint video is video captured by a camera around the target object, and thus the free viewpoint video includes images of the target object from multiple viewpoints. As shown in FIG. 8 , the current viewer angle (the playback viewpoint of the free viewpoint video, i.e., also referred to as viewpoint B) is 0 degrees. When the transmission viewpoint of danmaku comment 1 (also referred to as the viewpoint of danmaku comment 1) is −30 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 1 is a left offset of 30 degrees. When the transmission viewpoint of danmaku comment 2 (also referred to as the viewpoint of danmaku comment 2) is +25 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 2 is a right offset of 25 degrees. When the transmission viewpoint of danmaku comment 3 (also referred to as the viewpoint of danmaku comment 3) is −100 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 3 is a left offset of 100 degrees. When the sending viewpoint of danmaku comment 4 (also called the viewpoint of danmaku comment 4) is +90 degrees, the offset angle between viewpoint B and the viewpoint of danmaku comment 4 is a right offset of 90 degrees.

[0128] Figure 9 shows a circle around the target object. (3 It should be noted that the free-viewpoint footage shown in the figures is captured at a 360-degree angle (60 degrees). In other words, the free-viewpoint footage is a 360-degree ambient footage. Obviously, in some other examples, the free-viewpoint footage may not be captured in a circle around the target object. For example, the free-viewpoint footage may be a 180-degree ambient footage, a 130-degree ambient footage, or a 270-degree ambient footage.

[0129] S704: The terminal 100 displays a video image of the first free viewpoint video at viewpoint B, and presents each danmaku comment based on the offset angle between viewpoint B and the viewpoint of each danmaku comment.

[0130] It can be understood that when the value of the offset angle between viewpoint B and the viewpoint of the barrage comment is different, the barrage comment is presented in a different manner. For example, the difference in presentation manner includes a different tilt manner (including tilt direction and / or tilt degree) of the text direction of the barrage comment, a different rotation manner (including rotation direction and / or rotation degree) of the text of the barrage comment, a different font, a different three-dimensional effect, a different movement manner, etc.

[0131] In the embodiment, the playback viewpoint (Visual The danmaku comment of the first free viewpoint video at point B) is presented in a different manner from the danmaku comments at other viewpoints. In this example embodiment, different danmaku comments may be presented by using the following policy: When |Viewpoint of danmaku comment-Viewpoint B|≦Threshold A (e.g., Threshold A is 0 degrees or 5 degrees), the danmaku comment is displayed in a first presentation manner. When |Viewpoint of danmaku comment-Viewpoint B|>Threshold A, the danmaku comment is displayed in a second presentation manner, which is different from the first presentation manner. For example, the first presentation manner is such that the text of the danmaku comment is displayed normally (e.g., the text of the danmaku comment is not slanted or rotated, the font is not transformed, and a flat effect is used). In this case, the text direction of the danmaku comment is parallel to the first direction. For example, the first direction is a direction parallel to the upper bezel / lower bezel of the screen of terminal 100. The second presentation method performs display using one or more of tilt, rotation, font deformation, and three-dimensional effect, i.e., the text direction of the barrage comments is a direction obtained by rotating a first direction (e.g., parallel to the upper bezel / lower bezel of the screen of terminal 100) clockwise by a specific angle. In this case, a specific included angle is formed between the text direction of the barrage comments and a bezel on one side of the screen of terminal 100. In another example, in the first presentation method, the barrage comments move in a first direction (e.g., left), and in the second presentation method, the barrage comments move in a second direction (e.g., right). The first direction is different from the second direction.

[0132] For example, in the playback interface 901 of the first free viewpoint video shown in FIG. 9(1), the viewpoint of the barrage comment 902 (the viewpoint is 60 degrees) is the same as the playback viewpoint of the first free viewpoint video. (Visual Point B, Similarly The viewpoint of danmaku comment 903 (the viewpoint is 90 degrees), the viewpoint of danmaku comment 904 (the viewpoint is 30 degrees), the viewpoint of danmaku comment 905 (the viewpoint is 120 degrees), and the viewpoint of danmaku comment 906 (the viewpoint is 10 degrees) are all different from the playback viewpoint of the first free viewpoint video, and the text of the four danmaku comments are all displayed in a tilted manner. It may be understood that in some other examples, when displaying danmaku comments, terminal 100 may or may not display viewpoint information of the danmaku comments.

[0133] The user can select the playback viewpoint based on the presentation method of the barrage comments. (Visual It can be seen that the danmaku comments in the first free viewpoint video at point B) can be distinguished from the danmaku comments in other viewpoints. In this way, the user will not be confused or suspicious when the video in the playback viewpoint does not match the content of the danmaku comments in other viewpoints. This improves the viewing experience of the first free viewpoint video.

[0134] In other embodiments, when the offset direction between viewpoint B and the viewpoint of the danmaku comment is different, the danmaku comment is presented in a different manner. In one example, when the offset angle between viewpoint B and the viewpoint of the danmaku comment is a left offset, the danmaku comment is displayed in a third presentation manner. When the offset angle between viewpoint B and the viewpoint of the danmaku comment is a right offset, the danmaku comment is displayed in a fourth presentation manner. The third presentation manner is different from the fourth presentation manner.

[0135] For example, an example in which the first direction is a direction parallel to the top bezel of the screen of the terminal 100 is used for explanation. In the third presentation method, the text direction of the barrage comment is a direction obtained by rotating the first direction counterclockwise by a specific angle. In the fourth presentation method, the text direction of the barrage comment is a direction obtained by rotating the first direction clockwise by a specific angle. For example, in the first free viewpoint video playback interface 907 shown in FIG. 9(2), the viewpoint of the barrage comment 902 (the viewpoint is 60 degrees) is the same as the playback viewpoint of the first free viewpoint video. (Visual Point B, Similarly The offset angle between the viewpoint of danmaku comment 903 (the viewpoint is 90 degrees) and the playback viewpoint of the first free viewpoint video is a right offset, and the text direction of danmaku comment 903 is a direction obtained by rotating the first direction clockwise by a specific angle. Similarly, the offset angle between the viewpoint of danmaku comment 905 (the viewpoint is 120 degrees) and the playback viewpoint of the first free viewpoint video is also a right offset, and the text direction of danmaku comment 905 is a direction obtained by rotating the first direction clockwise by a specific angle. The offset angle between the viewpoint of danmaku comment 904 (the viewpoint is 30 degrees) and the playback viewpoint of the first free viewpoint video is a left offset, and the text direction of danmaku comment 904 is a direction obtained by rotating the first direction counterclockwise by a specific angle. Similarly, the offset angle between the viewpoint of the barrage comment 906 (the viewpoint is 10 degrees) and the playback viewpoint of the first free viewpoint video is also a left offset, and the text direction of the barrage comment 906 is the direction obtained by rotating the first direction counterclockwise by a specific angle.

[0136] In another example, the third presentation scheme includes a first mark (eg, a left arrow) and the fourth presentation scheme includes a second mark (eg, a right arrow).

[0137] It can be seen that when the offset direction between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is different and the presentation methods of the danmaku comments are also different, the user may distinguish the offset direction of the viewpoint corresponding to the danmaku comment based on the presentation method of the danmaku comment. In this case, when the user is interested in some danmaku comments (referred to as target danmaku comments), the user may determine the viewpoint switching direction based on the presentation method of the target danmaku comment and switch to the viewpoint corresponding to the target danmaku comment. In other words, different presentation methods of the danmaku comments may notify the user of the direction that needs to be switched when the user switches the viewpoint, thereby enabling the user to switch the viewpoint quickly.

[0138] For example, in the playback interface 907 shown in FIG. 9(2), if a user is interested in danmaku comment 903 and wishes to switch to the viewpoint corresponding to danmaku comment 903, the user will know that the viewpoint offset between playback viewpoint B and the viewpoint of danmaku comment 903 is a right offset, based on the presentation method in which danmaku comment 903 tilts to the right. Thus, the user may invoke the viewpoint ruler and slide to the right on the viewpoint ruler until the viewpoint is switched to that corresponding to danmaku comment 903. If a user is interested in danmaku comment 904 and wishes to switch to the viewpoint corresponding to danmaku comment 904, the user will know that the viewpoint offset between playback viewpoint B and the viewpoint of danmaku comment 904 is a left offset, based on the presentation method in which danmaku comment 904 tilts to the left. Thus, the user may invoke the viewpoint ruler and slide to the left on the viewpoint ruler until the viewpoint is switched to that corresponding to danmaku comment 904.

[0139] In yet another embodiment, when the offset direction between viewpoint B and the viewpoint of the danmaku comment is the same but the offset angle value is different, the danmaku comment is presented in a different manner. For example, the greater the offset angle between viewpoint B and the viewpoint of the danmaku comment, the greater the degree of tilt present when the danmaku comment is presented, i.e., the greater the tilt angle of the danmaku comment. For example, in the playback interface 908 shown in FIG. 9(3), both danmaku comment 903 and danmaku comment 905 are offset to the right with respect to the playback viewpoint of the first free viewpoint video. When |viewpoint of danmaku comment 903 - viewpoint B|<{viewpoint of danmaku comment 905 - viewpoint B|, then tilt angle a of danmaku comment 903 < tilt angle b of danmaku comment 905. Both danmaku comment 904 and danmaku comment 906 are offset to the right with respect to the playback viewpoint of the first free viewpoint video. When |viewpoint of danmaku comment 904 - viewpoint B|<|viewpoint of danmaku comment 906 - viewpoint B|, then the tilt angle c of danmaku comment 904 < the tilt angle d of danmaku comment 906.

[0140] one In the example, the tilt angle of the text direction of the barrage comment is directly proportional to the value of the offset angle of the barrage comment. In other words, the tilt angle of the text direction of the barrage comment has a linear increasing relationship based on the value of the offset angle of the barrage comment. In another example, the tilt angle of the text direction of the barrage comment may alternatively have a stepwise increasing relationship based on the value of the offset angle of the barrage comment. For example, the following policies may be used to present different barrage comments:

[0141] When |Viewpoint of danmaku comment-Viewpoint B|≦threshold B (for example, threshold B is 10 degrees or 15 degrees), the danmaku comment is displayed as usual.

[0142] When threshold B<|viewpoint of danmaku comment-viewpoint B|≦threshold C (for example, threshold C is 30 degrees or 45 degrees), the text direction of the danmaku comment is the direction obtained by rotating the first direction counterclockwise / clockwise by the first angle.

[0143] When threshold C<|viewpoint of danmaku comment-viewpoint B|≦threshold D (e.g., threshold D is 60 degrees), the text direction of the danmaku comment is the direction obtained by rotating the first direction counterclockwise / clockwise by a second angle, where the second angle is greater than the first angle.

[0144] ...

[0145] Table 2 below provides an example of the correspondence between the offset angle of the barrage comment and the tilt angle of the barrage comment. [Table 2]

[0146] In conclusion, when the offset angle between viewpoint B and the viewpoint of the danmaku comments is different and the presentation methods of the danmaku comments are different, the user may distinguish the value of the offset angle between viewpoint B and the viewpoint of the danmaku comments based on the presentation method of the danmaku comments. In this case, when the user is interested in some danmaku comments (referred to as target danmaku comments), the user may determine the viewpoint switching amplitude (e.g., the finger sliding distance, or the duration or number of presses of a button on a remote control) based on the presentation method of the target danmaku comments, and switch to the viewpoint corresponding to the target danmaku comment. In other words, different presentation methods of the danmaku comments may notify the user of the viewpoint switching amplitude (e.g., the finger sliding distance, or the duration or number of presses of a button on a remote control), thereby enabling the user to quickly switch viewpoints.

[0147] It can be appreciated that in some other embodiments, the danmaku comments in the playback interface in all viewpoints, such as danmaku comment 902, danmaku comment 903, danmaku comment 904, danmaku comment 905, and danmaku comment 906 in playback interface 908, may be presented in the same manner, e.g., all are displayed normally. The user can select the playback viewpoint based on the viewpoint conveyed in each danmaku comment. (VisualIt can be understood that the danmaku comments in the first free viewpoint video at point B) may be distinguished from the danmaku comments in other viewpoints. In this way, the user will not be confused or suspicious when the video in the playback viewpoint does not match the content of the danmaku comments in other viewpoints. This improves the viewing experience of the first free viewpoint video.

[0148] As described above, the user may randomly switch the playback viewpoint of the first free viewpoint video. In some embodiments, in the process of switching the viewpoint of the first free viewpoint video, terminal 100 leaves the content and presentation method of the danmaku comments unchanged. After the viewpoint of the first free viewpoint video is switched, terminal 100 reacquires the danmaku comment data acquired after the viewpoint is switched, and determines the display content of the danmaku comments and the specific presentation method of each danmaku comment based on the reacquired danmaku comment data.

[0149] For example, Figure 10A shows a method for displaying barrage comments in the process of switching viewpoints according to an embodiment of the present application. The method includes the following steps:

[0150] S1001: At time t0, the terminal 100 detects that the user has started to perform a viewpoint switching operation, and the viewpoint switching operation indicates switching the viewpoint of the first free viewpoint video.

[0151] It should be noted that before step S1001, the terminal 100 may display a video image of the first free viewpoint video at any viewpoint and the corresponding danmaku comments by using the method in steps S701 to S704. In one example, the terminal 100 displays a video image of the first free viewpoint video at viewpoint B at playback time point T2 and the danmaku comments corresponding to playback time point T2.

[0152] The viewpoint switching operation may be, for example, a slide operation performed by the user on the screen of the terminal 100, an operation of tilting the terminal 100 by the user, or an operation of operating a button on a remote control by the user.

[0153] For example, when the terminal 100 detects that the user's finger has remained on the viewpoint ruler for longer than a preset duration 2 (e.g., 3 seconds), the terminal 100 determines that the user has started to perform a viewpoint switching operation and records the current time as time t0. In another example, when the terminal 100 is configured with a device such as a gravity acceleration or IMU device, the attitude of the terminal 100 can be detected. When it is detected that the tilt angle of the terminal 100 with respect to the horizontal plane is equal to or greater than threshold 1, the terminal 100 determines that the user has started to perform a viewpoint switching operation and records the current time as time t0. In another example, when the terminal 100 receives a signal corresponding to a specific button transmitted by a remote control, the terminal 100 determines that the user has started to perform a viewpoint switching operation and records the current time as time t0.

[0154] S1002: From time t0 to time t1, the terminal 100 leaves the presentation method of each barrage comment unchanged.

[0155] S1003: At time t1, the terminal 100 detects that the viewpoint switching operation has been completed.

[0156] For example, after time t0, the terminal continuously detects that the user's finger has slid across the screen. When it detects that the user's finger has been removed from the screen, the terminal determines that the user has completed the viewpoint change operation and records the current time as time t1. In another example, after time t0, the terminal continuously detects the tilt angle of the terminal 100 with respect to the horizontal plane. When it detects that the tilt angle of the terminal 100 with respect to the horizontal angle is less than threshold 1, it determines that the user has completed the viewpoint change operation and records the current time as time t1. In another example, after time t0, the terminal continuously detects a signal corresponding to a specific button transmitted by a remote control. When a signal corresponding to the specific button is not detected for a preset duration 3 (e.g., 5 seconds) after the last detection of the signal corresponding to the specific button, it determines that the user has completed the viewpoint change operation and records the current time as time t1.

[0157] For example, in the process of the user performing a viewpoint switching operation, i.e., between time t0 and time t1 (excluding time t1), terminal 100 may leave the presentation method of each danmaku comment unchanged, for example, may leave the tilt direction and tilt angle of each danmaku comment unchanged. In other words, regardless of whether the playback viewpoint of the first free viewpoint video changes between time t0 and time t1, terminal 100 always maintains the presentation method of the danmaku comments displayed by terminal 100 at time t0.

[0158] It should be noted that in some examples, the playback viewpoint of the video image of the first free viewpoint video changes between time t0 and time t1. For example, terminal 100 may alternatively play back the video image of the first free viewpoint video at another viewpoint (e.g., one or more viewpoints between viewpoint B and viewpoint C) based on a viewpoint switching operation performed by the user (e.g., the sliding distance of a finger, or the number of times or duration of pressing a button on a remote control). In other words, from time t0 to time t1, the playback viewpoint of the first free viewpoint video is switched from viewpoint B to viewpoint C, and the playback viewpoint of the first free viewpoint video plays back all or part of the viewpoints from viewpoint B to viewpoint C. CoverFor example, when the viewpoint of the first free viewpoint video is switched from 60 degrees to 90 degrees, the viewpoint of the first free viewpoint video is switched to all angles between 60 degrees and 90 degrees. Covering or some angle between 60 and 90 degrees Covers for example 、6 0 degrees, 70 degrees, 80 degrees and 90 degrees Covering angles including .

[0159] In some other examples, the playback viewpoint of the video image of the first free viewpoint video does not change between time t0 and time t1, i.e., the terminal 100 continues to play the video image of the first free viewpoint video at angle B.

[0160] The terminal 100 continues playing back the first free viewpoint video. (Re It should be further noted that the terminal 100 may determine the danmaku comments to be displayed and the display positions of these danmaku comments based on the current playback time (i.e., the current live time). In some examples, in the process of a viewpoint switching operation performed by a user, i.e., between time t0 and time t1 (excluding time t1), the terminal 100 continuously plays back the first free viewpoint video. In this case, the playback time of the video images of the first free viewpoint video changes. In this case, the terminal 100 determines in real time the danmaku comments to be displayed by the terminal 100 and the display positions of the danmaku comments based on the change in the playback time. However, the presentation manner of each danmaku comment (e.g., the tilt direction and tilt angle of the danmaku comment) remains unchanged. In some other examples, in the process of a viewpoint switching operation performed by a user, i.e., between time t0 and time t1 (excluding time t1), the terminal 100 pauses the playback of the first free viewpoint video. During this period, the playback time of the video images of the first free viewpoint video does not change. In this case, both the barrage comments displayed by the terminal 100 and the display positions of the barrage comments remain unchanged.

[0161] S1004: After time t1, the terminal 100 requests the video file of the first free viewpoint video at viewpoint C at playback time T3 from the media server.

[0162] In response, the media server returns to the terminal 100 the video file of the first free viewpoint video at viewpoint C at playback time T3.

[0163] In some embodiments, after detecting that the user has completed a viewpoint switching operation, the terminal 100 sends a request 1 to the media server, where the request 1 is used to request a video file corresponding to the viewpoint acquired after switching. In one example, the request 1 includes an identifier of the first free viewpoint video, a viewpoint C, and a playback time point T3. The playback time point T3 is related to the playback time point T2. For example, the playback time point T3 = the playback time point T2, the playback time point T3 = the playback time point T2 - the preset time offset value 1, or the playback time point T3 = the playback time point T2 + the preset time offset value 2. The preset time offset value 1 and the preset time offset value 2 may be the same or different.

[0164] S1005: After time t1, the terminal 100 further requests, from the barrage comment server, barrage comment data corresponding to playback time T3 of the first free viewpoint video.

[0165] In response, the barrage comment server returns to the terminal 100 barrage comment data corresponding to playback time point T3 of the first free viewpoint video.

[0166] In some embodiments, after detecting that the user has completed a viewpoint switching operation, the terminal 100 sends a request 2 to the media server, where the request 2 is used to request barrage comment data corresponding to the playback time T3 of the first free viewpoint video. In one example, the request 2 includes an identifier of the first free viewpoint video and the playback time T3. The playback time T3 is related to the playback time T2. For example, the playback time T3 = the playback time T2, the playback time T3 = the playback time T2 - the preset time offset value 1, or the playback time T3 = the playback time T2 + the preset time offset value 2. In some other embodiments, the terminal 100 continuously plays the first free viewpoint video in the process of the user performing a viewpoint switching operation. In this case, the playback time T3 = the playback time T2 + (time t1 - time t0).

[0167] It should be noted that step S1005 may be performed after, before, or simultaneously with step S1004.

[0168] It should be further noted that this step is optional. In some other embodiments, playback time point T3 = playback time point T2, or playback time point T3 = playback time point T2 - preset time offset value 1. Terminal 100 stores the danmaku comment data corresponding to playback time point T3. In this case, terminal 100 does not need to request the danmaku comment data at playback time point T3 again from the danmaku comment server. As playback of the first free viewpoint video progresses, terminal 100 requests, from the danmaku comment server, danmaku comment data that exists after playback time point T3 or playback time point T2.

[0169] S1006: The terminal 100 recalculates the offset angle between viewpoint C and the viewpoint of each barrage comment after the viewpoint has been switched.

[0170] It should be noted that before the viewpoint is switched, the terminal 100 calculates the offset angle between viewpoint B and the viewpoint of each danmaku comment. After the viewpoint is switched, the terminal 100 calculates the offset angle between viewpoint C and the viewpoint of each danmaku comment. To calculate the offset angle, For thoseFor the method, see the related content in step S703.

[0171] S1007: The terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time point T3, and displays each barrage comment based on the offset angle between viewpoint C and the viewpoint of each barrage comment.

[0172] For this step, please refer to the relevant content in step S704, and the details will not be described again here.

[0173] For example, as shown in FIG. 10B(1), the terminal 100 has a 60-degree viewpoint. (Visual The first free viewpoint image at point B Playback Interface 1001. The playback interface 1001 further displays danmaku comments at different viewpoints, such as danmaku comment 1002, danmaku comment 1003, danmaku comment 1004, danmaku comment 1005, and danmaku comment 1006. At time t0, the terminal 100 detects that the user has dragged the viewpoint ruler in the playback interface 1001 (the playback time in this case is 20:05, i.e., playback time T2), and at time t1, the terminal 100 detects that the user's finger has been lifted from the display of the terminal 100. At time t1, the user performs a viewpoint switching operation (Yu In response to the information indicating that the user has performed an operation of dragging the viewpoint ruler, the terminal 100 switches the playback viewpoint of the first free viewpoint video and displays the playback interface 1007 shown in FIG. 10B(2). In this case, the playback viewpoint of the first free viewpoint video is a 70-degree viewpoint. (Visual It can be noted that the display mode of each danmaku comment in the playback interface 1007 is changed. For example, the display mode of each danmaku comment in the playback interface 1001 is changed as follows: thing Compared to the bullet comment 1002 Presentation method changes from normal display to tilted display. ChangeThe tilt angles of the danmaku comments 1003 and 1005 decrease (a changes to a', and b changes to b'). The tilt angles of the danmaku comments 1004 and 1006 increase (c changes to c', and d changes to d'). In some examples, the terminal 100 pauses the playback of the first free viewpoint video between time t0 and time t1 (excluding time t1), i.e., the video image displayed by the terminal 100 remains unchanged. In this case, the playback time of the first free viewpoint video in the playback interface 1007 is 20:05. (Re The raw time point T3 is the same as the replay time point T2).

[0174] In some other examples, the terminal 100 continuously plays back the first free viewpoint video between time t0 and time t1 (excluding time t1). In this case, if the user performs a viewpoint switching operation at time t1, (Yu In response to the information indicating that the user has performed an operation (dragging the viewpoint ruler with the user), the terminal 100 switches the playback viewpoint of the first free viewpoint video and displays the playback interface 1008 shown in FIG. 10B(3). The playback point of the first free viewpoint video in the playback interface 1008 is 20:10 (specifically, the playback point T 3= The playback time is T2+ (time t1-time t0).

[0175] In conclusion, in this embodiment, in the process of switching the viewpoint by the user, i.e., at the time t0 and time During t1, terminal 100 does not update the specific presentation format of the barrage comments, which helps the user compare the change in the presentation format of the barrage comments before and after the viewpoint is switched, reduces the processing load on terminal 100, and reduces the power consumption of terminal 100.

[0176] In some other embodiments, in the process of switching the viewpoint of the first free viewpoint video, the presentation manner of each barrage comment displayed by terminal 100 changes in real time based on the current playback viewpoint of the first free viewpoint video. In other words, step S1002 in Fig. 10A may be replaced by step S1101 in Fig. 11A.

[0177] 11A shows another method for displaying barrage comments in a process of switching viewpoints according to an embodiment of the present application, which includes step S1001, step S1101, and step S1003 to step S1007.

[0178] For step S1001 and steps S1003 to S1007, please refer to the explanation of the corresponding steps in Fig. 10A, and the details will not be explained again here.

[0179] S1101: From time t0 to time t1, terminal 100 calculates in real time the viewpoint offset between the viewpoint of each barrage comment and the playback viewpoint of the first free viewpoint video, and displays each barrage comment based on the viewpoint offset calculated in real time.

[0180] In some embodiments, the playback viewpoint of the video image of the first free viewpoint video changes between time t0 and time t1. For example, terminal 100 may alternatively play back the video image of the first free viewpoint video at another viewpoint (e.g., one or more viewpoints between viewpoint B and viewpoint C) based on a viewpoint switching operation performed by the user (e.g., the sliding distance of a finger, or the number of times or duration of pressing a button on a remote control). In other words, from time t0 to time t1, the playback viewpoint of the first free viewpoint video is switched from viewpoint B to viewpoint C, and the playback viewpoint of the first free viewpoint video plays back all or part of the viewpoints from viewpoint B to viewpoint C. Cover For example, when the viewpoint of the first free viewpoint video is switched from 60 degrees to 90 degrees, the viewpoint of the first free viewpoint video is switched to all angles between 60 degrees and 90 degrees. Coveringor some angle between 60 and 90 degrees Cover , for example, angles including 60 degrees, 70 degrees, 80 degrees, and 90 degrees Cover .

[0181] From time t0 to time t1, the terminal 100 determines that as the playback viewpoint of the first free viewpoint video changes, the presentation method of each barrage comment changes accordingly. for example When the playback viewpoint of the first free viewpoint video is changed, the terminal 100 calculates in real time an angle offset between the viewpoint of each barrage comment and the changed playback viewpoint of the first free viewpoint video, and determines a presentation method for each barrage comment based on the angle offset calculated in real time.

[0182] The terminal 100 continues playing back the first free viewpoint video. (Re It should be further noted that the terminal 100 may determine the danmaku comments to be displayed and the display positions of these danmaku comments based on the playback time (the playback time of the first free viewpoint video). In some examples, in the process of performing a viewpoint switching operation by the user, i.e., between time t0 and time t1 (excluding time t1), the terminal 100 continuously plays back the first free viewpoint video. In this case, the playback time of the video images of the first free viewpoint video changes. In this case, the terminal 100 determines in real time the danmaku comments to be displayed by the terminal 100 and the display positions of the danmaku comments based on the change in the playback time. In other words, the content of the danmaku comments displayed by the terminal 100 changes as the playback of the first free viewpoint video progresses. Optionally, in this scenario, between time t0 and time t1 (excluding time t1), the terminal 100 may further request danmaku comment data corresponding to each playback time in real time from the danmaku comment server based on the playback progress of the first free viewpoint video.

[0183] In some other examples, in the process of the user performing a viewpoint switching operation, that is, between time t0 and time t1 (excluding time t1), terminal 100 pauses the playback of the first free viewpoint video. During this period, the playback time point of the video image of the first free viewpoint video does not change. In this case, both the barrage comments displayed by terminal 100 and the display positions of the barrage comments remain unchanged.

[0184] For example, as shown in FIG. 11B(1), the terminal 100 has a 60-degree viewpoint. (Visual The first free viewpoint image at point B Playback Interface 1101. The playback interface 1101 further displays danmaku comments at different viewpoints, such as danmaku comment 1102, danmaku comment 1103, danmaku comment 1104, danmaku comment 1105, and danmaku comment 1106. At time t0, the terminal 100 detects that the user has dragged the viewpoint ruler in the playback interface 1101 (the playback time in this case is 20:05, i.e., playback time T2), and at time t1, the terminal 100 detects that the user's finger has been lifted from the display of the terminal 100.

[0185] When the terminal 100 detects that the user has performed a viewpoint switching operation between time t0 and time t1 (excluding time t0 and time t1), the playback viewpoint of the first free viewpoint video displayed by the terminal 100 changes continuously, for example, from 60 degrees to 65 degrees, as shown in the playback interface 1107 shown in FIG. 11B(2). It may be noted that the presentation method of each barrage comment in the playback interface 1107 is changed. For example, thing Compared to the bullet comment 1102 Presentation method changes from a normal display to a tilted display (tilt angle is e'). The tilt angles of danmaku comments 1003 and 1005 decrease (a changes to a'' and b changes to b''). The tilt angles of danmaku comments 1004 and 1006 increase (c changes to c'' and d changes to d'').

[0186] At time t1, the terminal 100 further switches the playback viewpoint of the first free viewpoint video and displays the playback interface 1108 shown in Fig. 11B(3). In this case, the playback viewpoint of the first free viewpoint video is a 70-degree viewpoint. (Visual It can be noted that the display mode of each danmaku comment in the playback interface 1108 is switched to point C). teeth, Each barrage comment in the playback interface 1107 For example, The tilt angle of the barrage comment 1102 has increased further. death (e' is changed to e) 、 The tilt angle of Danmaku Comment 1003 and Danmaku Comment 1005 has decreased. death (a'' is changed to a', and b'' is changed to b') 、 The tilt angles of barrage comments 1004 and 1006 increase (c'' changes to c', and d'' changes to d').

[0187] In conclusion, in this embodiment, in the process of switching the viewpoint by the user, the terminal 100 adjusts the presentation manner of the barrage comments in real time based on the change in the playback viewpoint of the first free viewpoint video, which helps the user know the distribution status of the barrage comments of the first free viewpoint video at each viewpoint, and finally indicates to the user to select the switched playback viewpoint.

[0188] In some other embodiments, the execution of the viewpoint switching operation may be paused in the process of switching the viewpoint of the first free viewpoint video. In this case, when the viewpoint switching operation is paused, terminal 100 may re-request the barrage comment data corresponding to the current playback time point from the barrage comment server, and may determine the display content of the barrage comments and the specific presentation method for each barrage comment based on the re-acquired barrage comment data.

[0189] 12A and 12B are schematic flowcharts of another method for displaying barrage comments in a process of switching viewpoints according to an embodiment of the present application. The method includes the following steps:

[0190] S1201: At time t0, the terminal 100 detects that the user has started to perform a viewpoint switching operation, and the viewpoint switching operation indicates switching the viewpoint of the first free viewpoint video.

[0191] In one example, the terminal 100 displays the video image of the first free viewpoint video at viewpoint B at playback time point T2 and the barrage comment data of the first free viewpoint video at playback time point T2.

[0192] The viewpoint switching operation may be, for example, a slide operation performed by the user on the screen of the terminal 100, an operation of tilting the terminal 100 by the user, or an operation of operating a button on a remote control by the user. For example, if the terminal 100 detects that the user's finger remains on the viewpoint ruler for longer than a preset duration 2 (e.g., 3 seconds), the terminal 100 determines that the user has started to perform a viewpoint switching operation and records the current time point as time point t0.

[0193] For other details of this step, please refer to the relevant details of step S1001 in Fig. 10A, and the details will not be described again here.

[0194] S1202: From time t0 to time t2, the terminal 100 leaves the presentation method of the displayed barrage comments unchanged.

[0195] Time point t2 is a time point between time points t0 and t1.

[0196] For other details of this step, please refer to the relevant details of step S1002 in Fig. 10A, and the details will not be described again here.

[0197] In some other embodiments, between time t0 and time t2, terminal 100 may alternatively adjust the presentation manner of each danmaku comment in real time based on a change in the playback viewpoint of the first free viewpoint video. For related content, refer to the description of step S1101 in FIG. 11A. The details will not be described again here.

[0198] S1203: At time t2, the terminal 100 detects that the user has temporarily stopped the execution of the viewpoint switching operation.

[0199] For example, after time t0, the terminal continuously detects that the user's finger slides on the screen, and when it detects that the duration during which the user's finger does not slide and the finger does not leave the screen exceeds preset duration 4 (for example, 3 seconds), the terminal determines that the user has paused the execution of the viewpoint switching operation and records the current time as time t2.

[0200] S1204: After time point t2, the terminal 100 requests the video file of the first free viewpoint video at viewpoint D at playback time point T4 from the media server.

[0201] In other words, after detecting that the user has paused the execution of the viewpoint switching operation, the terminal 100 determines the current playback viewpoint of the first free viewpoint video, for example, viewpoint D. The playback time point T4 is relative to the playback time point T2. In some examples, when the terminal 100 pauses the playback of the first free viewpoint video from time t0 to time t2, the playback time point T4 = playback time point T2, or the playback time point T4 = playback time point T2 - preset time offset value 1. For example, when the playback time point T2 is 20:10, the playback time point T4 is 20:10 or 20:05. In some other examples, when the terminal 100 continuously plays the first free viewpoint video from time t0 to time T2, the playback time point T4 = playback time point T2 + (time t2 - time t0). In another example, when the playback time point T2 is 20:10, the playback time point T4 is 20:15.

[0202] S1205: After time point t2, the terminal 100 further requests, from the barrage comment server, barrage comment data corresponding to playback time point T4 of the first free viewpoint video.

[0203] S1206: The terminal 100 recalculates the offset angle between viewpoint D and the viewpoint of each barrage comment after the viewpoint has been switched.

[0204] S1207: The terminal 100 displays the video image of the first free viewpoint video at viewpoint D at playback time point T4, and displays each barrage comment based on the offset angle between viewpoint D and the viewpoint of each barrage comment.

[0205] For other details of steps S1204 to S1207, refer to the relevant details of steps S1004 to S1007 in Fig. 10A, and the details will not be described again here.

[0206] S1208: At time t1, the terminal 100 detects that the viewpoint switching operation has been completed.

[0207] For example, after time t2, the terminal continuously detects that the user's finger has slid across the screen. When the terminal detects that the user's finger has been removed from the screen, the terminal determines that the user has completed the viewpoint switching operation and records the current time as time t1.

[0208] S1209: After time point t1, the terminal 100 requests the video file of the first free viewpoint video at viewpoint C at playback time point T3 from the media server.

[0209] S1210: After time t1, the terminal 100 further requests, from the barrage comment server, barrage comment data corresponding to playback time T3 of the first free viewpoint video.

[0210] In some examples, playback time T3 is relative to playback time T4, such as playback time T3 = playback time T4, playback time T3 = playback time T4 - preset time offset value 1, or playback time T3 = playback time T4 + preset time offset value 2. In other examples, playback time T3 = playback time T4 + (time t1 - time t0).

[0211] S1211: The terminal 100 recalculates the offset angle between viewpoint C and the viewpoint of each barrage comment after the viewpoint has been switched.

[0212] S1212: The terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time point T3, and displays each barrage comment based on the offset angle between viewpoint C and the viewpoint of each barrage comment.

[0213] 12A and 12B are described by using an example in which the viewpoint switching operation is paused only once, and it can be understood that the technical solutions provided in the embodiments of this application are also applicable to the case in which the viewpoint switching operation is paused more than once.

[0214] For other details of steps S1208 to S1212, refer to the relevant details of steps S1003 to S1007 in Fig. 10A, and the details will not be described again here.

[0215] In this embodiment, in the process of switching the viewpoint by the user, after the user pauses the viewpoint switching operation, the terminal 100 may adjust the display manner of the danmaku comments in real time based on the current playback viewpoint of the first free viewpoint video, which helps the user know the distribution status of the danmaku comments of the first free viewpoint video at the current playback viewpoint, and finally indicates to the user to select the switched playback viewpoint.

[0216] In another embodiment, the terminal 100 may further predefine a visible viewpoint range of the barrage comments, or the terminal 100 may provide an option for the user to set the visible viewpoint range of the barrage comments. The visible viewpoint range of the barrage comments means that the terminal 100 displays the content of the barrage comments within the viewpoint range and does not display the content of the barrage comments outside the viewpoint range.

[0217] For example, terminal 100 may predefine the visible viewpoint range of the danmaku comment, or may receive the visible viewpoint range of the danmaku comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than an angle threshold E (e.g., 45 degrees). In this case, terminal 100 may determine that the visible viewpoint range of the danmaku comment is [playback viewpoint of the first free viewpoint video - angle threshold E, playback viewpoint of the first free viewpoint video + angle threshold E].

[0218] In another example, the terminal 100 may predefine the visible viewpoint range of the barrage comment, or may receive the visible viewpoint range of the barrage comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the barrage comment and the playback viewpoint of the first free viewpoint video is less than the angle threshold E, and the offset angle is equal to or less than 0. (O (The offset direction is a left offset.) In this case, the terminal 100 may determine that the visible viewpoint range of the barrage comment is [playback viewpoint of the first free viewpoint video - angle threshold E, playback viewpoint of the first free viewpoint video].

[0219] In another example, the terminal 100 may predefine the visible viewpoint range of the barrage comment, or may receive the visible viewpoint range of the barrage comment set by the user, as follows: The absolute value of the offset angle between the viewpoint of the barrage comment and the playback viewpoint of the first free viewpoint video is less than the angle threshold E, and the offset angle is greater than or equal to 0. (O(The offset direction is a right offset.) In this case, the terminal 100 may determine that the visible viewpoint range of the barrage comment is [playback viewpoint of the first free viewpoint video, playback viewpoint of the first free viewpoint video + angle threshold E].

[0220] In some other embodiments, terminal 100 may further collect and count the number of danmaku comments outside the visible viewpoint range of the danmaku comments, and display the number of danmaku comments outside the visible viewpoint range to notify the user of the status of the danmaku comments outside the visible viewpoint range. Furthermore, terminal 100 may further count the number of left-offset danmaku comments and the number of right-offset danmaku comments separately for the danmaku comments outside the visible viewpoint range of the danmaku comments.

[0221] Alternatively, the terminal 100 may further display a control related to the number of barrage comments outside the visible viewpoint range. For example, the control related to the number of barrage comments outside the visible viewpoint range may indicate the enthusiasm of users participating in discussions about video images outside the visible viewpoint range. In a specific implementation, when the number of barrage comments outside the visible viewpoint range is equal to or greater than a specific number, the control may indicate that the viewpoints outside the visible viewpoint range are hot areas and may be used to prompt users to pay attention to video images outside the visible viewpoint range. In another specific implementation, the terminal 100 displays control 1, which is related to the number of left-offset barrage comments. When the number of left-offset barrage comments is equal to or greater than a specific number, control 1 may indicate that the left-offset viewpoints outside the visible viewpoint range are hot areas. The terminal 100 displays control 2, which is related to the number of right-offset barrage comments. When the number of right-offset barrage comments is equal to or greater than a specific number, control 2 may indicate that the right-offset viewpoints outside the visible viewpoint range are hot areas.

[0222] For example, the visible viewpoint range of the danmaku comments is set as follows: the absolute value of the offset angle between the viewpoint of the danmaku comment and the playback viewpoint of the first free viewpoint video is less than an angle threshold E, the first free viewpoint video is a 360-degree surrounding video, the number of left offset danmaku comments may be calculated by using equation (1), and the number of right offset danmaku comments may be calculated by using equation (2).

[0223] Number of left offset barrage comments = Σ(-180 degrees < offset angle < - angle threshold E) Equation (1)

[0224] Number of right offset barrage comments = Σ(angle threshold E<offset angle<180 degrees) Equation (2)

[0225] It can be understood that if the first free-viewpoint image is a 180-degree ambient image, then 180 degrees in equations (1) and (2) are replaced with 90 degrees. If the first free-viewpoint image is a 270-degree ambient image, then 180 degrees in equations (1) and (2) are replaced with 135 degrees.

[0226] As shown in FIG. 13, when the angle threshold E is set to 45 degrees, the visible viewpoint range of the barrage comments in the first free viewpoint video is [playback viewpoint of the first free viewpoint video -45 degrees, playback viewpoint of the first free viewpoint video +45 degrees].

[0227] When the playback viewpoint of the first free viewpoint video is 0 degrees, the visible viewpoint range of the danmaku comments is [-45 degrees, +45 degrees]. Danmaku comment 1 and danmaku comment 2 are located within the visible viewpoint range of the danmaku comments, and the terminal 100 2The contents are displayed. Danmaku comment 3 and danmaku comment 5 are located at [-180 degrees, -45 degrees], are outside the visible viewpoint range of the danmaku comments, and are left-offset danmaku comments. In this case, terminal 100 may display prompt information, for example, +2, on the left side of the screen to notify the user that there are two left-offset danmaku comments located outside the visible viewpoint range of the danmaku comments. Danmaku comment 4 and danmaku comment 6 are located at (+45 degrees, +180 degrees) and are outside the visible viewpoint range of the danmaku comments, and are right-offset danmaku comments. In this case, terminal 100 may display prompt information, for example, +2, on the right side of the screen to notify the user that there are two right-offset danmaku comments located outside the visible viewpoint range of the danmaku comments.

[0228] As shown in playback interface 1401 in FIG. 14A(1), playback interface 1401 includes danmaku comments within the visible viewpoint range, for example, danmaku comment 1 and danmaku comment 2. Playback interface 1401 further includes control 1402 and control 1403. Control 1402 is used to display the number of danmaku comments that are located outside the visible viewpoint range of the danmaku comments and offset to the left. Control 1403 is used to display the number of danmaku comments that are located outside the visible viewpoint range of the danmaku comments and offset to the right. In some examples, a user may operate control 1402 to display danmaku comments that are located outside the visible viewpoint range of the danmaku comments and offset to the left. For example, in response to detecting that a user has operated control 1402 in playback interface 1401, terminal 100 displays playback interface 1404 shown in FIG. 14A(2). Playback interface 1404 displays danmaku comments that are located outside the visible viewpoint range of the danmaku comments and offset to the left, for example, danmaku comment 3 and danmaku comment 5. It should be noted that the position of each danmaku comment on the display is relative to the current playback time point of the first free viewpoint video and the appearance time of each danmaku comment, and is not relative to the angular offset between the viewpoint of each danmaku comment and the current playback viewpoint of the first free viewpoint video. Playback interface 1404 may further include a control 1405 used to hide danmaku comments outside the visible viewpoint range. Optionally, in response to detecting that the user has operated control 1402 in playback interface 1404, terminal 100 displays playback interface 1406 shown in FIG. 14A(3). Playback interface 1406 displays danmaku comments that are located outside the visible viewpoint range of the danmaku comments and are offset to the right, for example, danmaku comment 4 and danmaku comment 6. Playback interface 1406 may further include a control 1405 used to hide danmaku comments outside the visible viewpoint range.

[0229] It should be noted that the barrage comments in playback interface 1401, playback interface 1404, and playback interface 1406 may be presented in the same manner, for example, may be displayed normally. In some other embodiments, the barrage comments in playback interface 1401, playback interface 1404, and playback interface 1406 may alternatively be different. For example, any one of the above-mentioned barrage comment display methods may be used for presentation. The specific presentation manner of the barrage comments is not limited herein.

[0230] In another example, as shown in playback interface 1407 shown in Fig. 14B(1), playback interface 1408 shown in Fig. 14B(2), and playback interface 1409 shown in Fig. 14B(3), the presentation method for each danmaku comment is determined based on the offset angle between the viewpoint of each danmaku comment and the playback viewpoint of the first free viewpoint video. For specific determination methods, refer to the above explanation. Furthermore, for functional control in each playback interface, refer to the explanations related to playback interface 1401 shown in Fig. 14A(1), playback interface 1404 shown in Fig. 14A(2), and playback interface 1406 shown in Fig. 14A(2). Details will not be described again here.

[0231] In some other examples, the terminal 100 may automatically further display hidden danmaku comments outside the visible viewpoint range based on a viewpoint switching operation performed by the user. For example, if the terminal 100 detects that the user has performed an operation to switch the viewpoint to the left in the interface 1401 shown in FIG. 14A(1), the terminal 100 may also display danmaku comments that are located outside the visible viewpoint range of the danmaku comments and that are offset to the left when switching the viewpoint. In another example, if the terminal 100 detects that the user has performed an operation to switch the viewpoint to the right in the interface 1401 shown in FIG. 14A(1), the terminal 100 may also display danmaku comments that are located outside the visible viewpoint range of the danmaku comments and that are offset to the right when switching the viewpoint.

[0232] Since the content of the danmaku comments within the visible viewpoint range is usually strongly related to the content of the video image currently being viewed by the user at the playback viewpoint, when the terminal 100 displays only the content of the danmaku comments within the visible viewpoint range and does not display the content of the danmaku comments outside the visible viewpoint range, it can be seen that the user can concentrate on viewing the current video image and the content of the danmaku comments related to the current video image, improving the viewing experience. When the user is interested in the content of the danmaku comments outside the visible viewpoint range, the content of the danmaku comments outside the visible viewpoint range may alternatively be expanded by using corresponding controls to meet the user's viewing requirements in multiple scenarios.

[0233] As described above, the user may randomly switch the playback viewpoint of the first free viewpoint video. In some embodiments, in the process of switching the viewpoint of the first free viewpoint video, terminal 100 leaves the number of danmaku comments outside the visible viewpoint range unchanged. After the viewpoint of the first free viewpoint video is switched, terminal 100 reacquires the danmaku comment data acquired after the viewpoint was switched, and determines the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range based on the reacquired danmaku comment data.

[0234] For example, Figure 15A shows a method for displaying barrage comments in the process of switching viewpoints according to an embodiment of the present application. The method includes the following steps:

[0235] S1501: At time t0, the terminal 100 detects that the user has started to perform a viewpoint switching operation, and the viewpoint switching operation indicates switching the viewpoint of the first free viewpoint video.

[0236] It should be noted that before step S1501, terminal 100 may display a video image of a first free viewpoint video at any viewpoint and corresponding danmaku comments by using the method in steps S701 to S704. In one example, terminal 100 displays a video image of a first free viewpoint video at viewpoint B at playback time point T2 and danmaku comments corresponding to playback time point T2. The danmaku comments corresponding to playback time point T2 include the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range.

[0237] S1502: From time t0 to time t1, the terminal 100 displays the message that the number of barrage comments outside the visible viewpoint range remains unchanged.

[0238] In some embodiments, between time t0 and time t1, terminal 100 retains the number of barrage comments outside the visible viewpoint range that were displayed by terminal 100 at time t0. In other words, between time t0 and time t1, the number of barrage comments outside the visible viewpoint range that were displayed by terminal 100 remains unchanged.

[0239] It should be noted that the video image of the first free viewpoint video displayed by the terminal 100 may or may not change between time t0 and time t1. This is not limited to the embodiment of this application. For the change of the video image of the first free viewpoint video displayed by the terminal 100 between time t0 and time t1, please refer to the description of the related content above. The details will not be described again here.

[0240] Between time t0 and time t1, the presentation manner of the barrage comments within the visible viewing range displayed by the terminal 100 may or may not change. For specific details, please refer to the above related content. Details will not be described again here.

[0241] S1503: At time t1, the terminal 100 detects that the viewpoint switching operation has been completed.

[0242] S1504: After time t1, the terminal 100 requests the video file of the first free viewpoint video at viewpoint C at playback time T3 from the media server.

[0243] In response, the media server returns to the terminal 100 the video file of the first free viewpoint video at viewpoint C at playback time T3.

[0244] The playback time point T3 is relative to the playback time point T2, for example, playback time point T3=playback time point T2, playback time point T3=playback time point T2-preset time offset value 1, or playback time point T3=playback time point T2+preset time offset value 2.

[0245] S1505: After time t1, the terminal 100 further requests, from the barrage comment server, barrage comment data corresponding to playback time T3 of the first free viewpoint video.

[0246] In response, the barrage comment server returns to the terminal 100 barrage comment data corresponding to playback time point T3 of the first free viewpoint video.

[0247] The playback time point T3 is relative to the playback time point T2, for example, playback time point T3=playback time point T2, playback time point T3=playback time point T2-preset time offset value 1, or playback time point T3=playback time point T2+preset time offset value 2.

[0248] It should be noted that this step is optional. In some other embodiments, playback time point T3 = playback time point T2, or playback time point T3 = playback time point T2 - preset time offset value 1. Terminal 100 stores the danmaku comment data corresponding to playback time point T3. In this case, terminal 100 does not need to request the danmaku comment data at playback time point T3 again from the danmaku comment server. Instead, as playback of the first free viewpoint video progresses, terminal 100 requests, from the danmaku comment server, danmaku comment data that exists after playback time point T3 or playback time point T2.

[0249] It should further be noted that step S1505 may be performed after, before, or simultaneously with step S1504.

[0250] S1506: The terminal 100 recalculates the number of barrage comments outside the visible viewpoint range after the viewpoint has been switched.

[0251] In some embodiments, terminal 100 further recalculates the offset angle between viewpoint C and the viewpoint of each danmaku comment within the visible viewpoint range, and determines the presentation method of each danmaku comment within the visible viewpoint range based on the recalculated offset angle.

[0252] S1507: The terminal 100 displays the video image of the first free viewpoint video at viewpoint C at playback time point T3, the number of barrage comments within the visible viewpoint range, and the number of barrage comments outside the visible viewpoint range.

[0253] For other details of this embodiment, please refer to the relevant contents of steps S1001 to S1007 in Fig. 10A, and the details will not be described again here.

[0254] In some other embodiments, the number of danmaku comments outside the visible viewpoint range displayed by terminal 100 in step 1502 may alternatively be changed. For example, when terminal 100 continuously plays back the first free viewpoint video between time t0 and time t1, the playback time points of the first free viewpoint video continuously change. Correspondingly, the number of danmaku comments corresponding to the playback time points of the first free viewpoint video also changes, and the number of danmaku comments outside the visible viewpoint range also changes accordingly.

[0255] For example, Figure 15 B( As shown in 1), the terminal 100 has a 60-degree viewpoint. (VisualThe terminal 100 plays a video image 1501 of the first free viewpoint video at point B). The playback interface 1501 includes danmaku comments within the visible viewpoint range, such as danmaku comment 1, danmaku comment 2, and danmaku comment 3. The playback interface 1501 further displays a control 1502 and a control 1503. The control 1502 is used to indicate the number (e.g., 2) of danmaku comments located outside the visible viewpoint range and offset to the left. The control 1503 is used to indicate the number (e.g., 2) of danmaku comments located outside the visible viewpoint range and offset to the right. When the first free viewpoint video has been played up to 20:05, the terminal 100 detects a viewpoint switching operation performed by the user (e.g., sliding the viewpoint ruler to the right). The terminal 100 continuously plays the first free viewpoint video and switches the playback viewpoint of the first free viewpoint video in real time based on the progress of the user's viewpoint switching operation. For example, in the process of the user performing the viewpoint switching operation, the terminal 100 B( 2) is displayed. In this case, the playback viewpoint of the first free viewpoint video is 65 degrees, and the playback time is 20:15. It can be seen that when the playback time changes to 20:15, the danmaku comments corresponding to the playback time also change accordingly. As shown in the playback interface 1504, the danmaku comments within the current visible viewpoint range change to danmaku comment 2, danmaku comment 3, and danmaku comment 4. The number of danmaku comments that are outside the visible viewpoint range and offset to the left changes to 3. The number of danmaku comments that are outside the visible viewpoint range and offset to the right changes to 5. It should be noted that in this case, the danmaku comments in the playback interface 1504 change accordingly as the playback time of the first free viewpoint video changes. In this case, the terminal 100 does not request new danmaku comment data from the danmaku comment server.

[0256] Furthermore, after the terminal 100 detects that the user has completed the viewpoint switching operation, the terminal 100 requests the barrage comment data corresponding to the current playback point from the barrage comment server. The content of the barrage comment changes. B( As shown in playback interface 1505 of 3), in this case, the playback viewpoint of the first free viewpoint video is 70 degrees, the playback time point is 20:15, and the danmaku comments within the visible viewpoint range change from danmaku comment 3 to danmaku comment 7. The number of danmaku comments that are outside the visible viewpoint range and offset to the left changes to 12. The number of danmaku comments that are outside the visible viewpoint range and offset to the right changes to 9.

[0257] In some other embodiments, the execution of the viewpoint switching operation may be paused during the process of the user performing the viewpoint switching operation. In this case, when the execution of the viewpoint switching operation is paused, the terminal 100 may re-request the danmaku comment data corresponding to the current playback time point from the danmaku comment server, and then update the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range based on the re-acquired danmaku comment data.

[0258] For example, Figure 16 A and Figure 16B 1 is a schematic flowchart of another method for displaying barrage comments in a process of switching viewpoints according to an embodiment of the present application. The method includes the following steps:

[0259] S1601: At time t0, the terminal 100 detects that the user has started to perform a viewpoint switching operation on the first free viewpoint video, and the viewpoint switching operation indicates switching the viewpoint of the first free viewpoint video.

[0260] In one example, the terminal 100 displays the video image of the first free viewpoint video at viewpoint B at playback time point T2 and the danmaku comment data of the first free viewpoint video at playback time point T2. The danmaku comment corresponding to playback time point T2 includes the content of the danmaku comments within the visible viewpoint range and the number of danmaku comments outside the visible viewpoint range.

[0261] For example, if the terminal 100 detects that the user's finger remains on the viewpoint ruler for longer than a preset duration 2 (e.g., 3 seconds), the terminal 100 determines that the user has begun to perform a viewpoint switching operation and records the current time as time t0.

[0262] S1602: From time t0 to time t2, the terminal 100 leaves the number of displayed barrage comments outside the visible viewpoint range unchanged.

[0263] In some embodiments, between time t0 and time t1, terminal 100 retains the number of danmaku comments outside the visible viewpoint range that were displayed by terminal 100 at time t0. In other words, the number of danmaku comments outside the visible viewpoint range that were displayed by terminal 100 between time t0 and time t1 remains unchanged. In some other embodiments, the number of danmaku comments outside the visible viewpoint range that were displayed by terminal 100 may alternatively change. For example, when terminal 100 continuously plays back the first free viewpoint video between time t0 and time t1, the playback time points of the first free viewpoint video change continuously. Correspondingly, the number of danmaku comments corresponding to the playback time points of the first free viewpoint video also changes, and the number of danmaku comments outside the visible viewpoint range also changes accordingly.

[0264] It should be noted that between time t0 and time t1, the video image of the first free viewpoint video displayed by the terminal 100 may or may not change. This is not limited to the embodiment of this application. Between time t0 and time t1, the presentation manner of the danmaku comments within the visible viewpoint range displayed by the terminal 100 may not change. Obviously, the presentation manner of the danmaku comments within the visible viewpoint range displayed by the terminal 100 may alternatively change. This is not limited to the embodiment of this application.

[0265] S1603: At time t2, the terminal 100 detects that the user has temporarily stopped the execution of the viewpoint switching operation.

[0266] For example, after time t0, the terminal continuously detects that the user's finger has slid across the screen. When the terminal detects that the user's finger has not slid or been removed from the screen, the terminal determines that the user has paused the execution of the viewpoint switching operation and records the current time as time t2.

[0267] S1604: After time point t2, the terminal 100 requests the video file of the first free viewpoint video at viewpoint D at playback time point T4 from the media server.

[0268] In other words, after detecting that the user has paused the execution of the viewpoint switching operation, the terminal 100 determines the current playback viewpoint of the first free viewpoint video, for example, viewpoint D. The playback time point T4 is related to the playback time point T2. For example, the playback time point T4 = the playback time point T2, the playback time point T4 = the playback time point T2 - the preset time offset value 1, or the playback time point T4 = the playback time point T2 + the preset time offset value 2.

[0269] S1605: After time t2, the terminal 100 further requests, from the barrage comment server, barrage comment data corresponding to playback time T4 of the first free viewpoint video.

[0270] S1606: The terminal 100 recalculates the number of barrage comments outside the visible viewpoint range after the viewpoint has been switched.

[0271] In some embodiments, terminal 100 further recalculates the offset angle between viewpoint C and the viewpoint of each danmaku comment within the visible viewpoint range, and determines the presentation method of each danmaku comment within the visible viewpoint range based on the recalculated offset angle.

[0272] S1607: Terminal 100 displays a video image of the first free viewpoint video from viewpoint D at playback time T4, and terminal 100 displays a video image of the first free viewpoint video from viewpoint C at playback time T3, the number of barrage comments within the visible viewpoint range, and the number of barrage comments outside the visible viewpoint.

[0273] S1608: At time t1, the terminal 100 detects that the viewpoint switching operation has been completed.

[0274] For example, after time t2, the terminal detects that the user's finger is on the screen. That's it When it detects that the user's finger has been lifted from the screen, the terminal determines that the user has completed the viewpoint switching operation and records the current time as time t1.

[0275] S1609: After time point t1, the terminal 100 requests the video file of the first free viewpoint video from viewpoint C at playback time point T3 from the media server.

[0276] S1610: After time t1, the terminal 100 further requests, from the barrage comment server, barrage comment data corresponding to playback time T3 of the first free viewpoint video.

[0277] In some examples, playback time T3 is relative to playback time T4, for example, playback time T3 = playback time T4, playback time T3 = playback time T4 - preset time offset value 1, or playback time T3 = playback time T4 + preset time offset value 2.

[0278] S1611: The terminal 100 recalculates the number of barrage comments outside the visible viewpoint range after the viewpoint has been switched.

[0279] S1612: Terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time T3, and terminal 100 displays the video image of the first free viewpoint video from viewpoint C at playback time T3, the barrage comments within the visible viewpoint range, and the number of barrage comments outside the visible viewpoint.

[0280] For other details of steps S1601 to S1612, refer to the related details of steps S1001 to S1007 in Fig. 10A and the related details of steps S1501 to S1507 in Fig. 15A. Details will not be described again here.

[0281] Figure 16 A and Figure 16B It should be noted that the embodiments described in are described by using an example in which the viewpoint switching operation is paused only once. It can be understood that the technical solutions provided in the embodiments of this application are also applicable to the case in which the viewpoint switching operation is paused more than once.

[0282] 17 is a schematic flowchart of another method for displaying barrage comments in a free viewpoint video according to an embodiment of the present application. The procedure includes the following steps:

[0283] S1701: The terminal displays a first interface of a first free viewpoint video at a first playback time point.

[0284] The first interface includes a video image of a first free viewpoint video at a first viewpoint and a first barrage comment.

[0285] For example, the first playback time point is related to the appearance time of the first danmaku comment. For example, the first playback time point is the appearance time of the first danmaku comment. Alternatively, the first playback time point is after the appearance time of the first danmaku comment and within the duration during which the first danmaku comment is continuously displayed.

[0286] The viewpoint offset between the third viewpoint corresponding to the first barrage comment and the first viewpoint is within a first preset angle range, the first preset angle range being a preset visible viewpoint range.

[0287] S1702: The terminal receives a first operation, and the first operation is an operation to switch the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint.

[0288] S1073: In response to receiving the first operation, the terminal displays a second interface of the first free viewpoint video at the first playback point in time.

[0289] The second interface includes a video image of the first free viewpoint video at a second viewpoint, and the second interface does not include the first barrage comment.

[0290] The viewpoint offset between the third viewpoint corresponding to the first danmaku comment and the second viewpoint is outside the first preset angle range. In other words, the viewpoint offset between the third viewpoint corresponding to the first danmaku comment and the second viewpoint is not within the preset visible viewpoint range. It can be understood that the offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment is smaller than the offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

[0291] In one example, in the process of switching the viewpoint, the terminal automatically pauses the playback of the first free viewpoint video. Therefore, before the user performs the first operation, the terminal plays the video image of the first free viewpoint video at the first viewpoint. When the terminal detects that the user has performed the first operation at the first playback time point, it automatically pauses the playback of the first free viewpoint video. After the first operation is performed and the playback viewpoint of the first free viewpoint video is switched from the first viewpoint to the second viewpoint, the first free viewpoint video starts to be played from the first playback time point.

[0292] In another example, in the process of switching viewpoints, the terminal continuously plays the first free viewpoint video. In this case, before performing the first operation, the user may first manually pause the playback of the first free viewpoint video. In this case, the first free viewpoint video remains at the first playback point. Then, after the user performs the first operation and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, the video image of the first free viewpoint video still remains at the first playback point.

[0293] Alternatively, after the first operation is performed and the terminal switches the playback viewpoint of the first free viewpoint video from the first viewpoint to the second viewpoint, playback starts from a playback point after the first playback point. Alternatively, the user may manually drag the playback progress bar to adjust the playback point back to the first playback point.

[0294] Optionally, the second interface includes a second danmaku comment, and an offset angle between the second viewpoint and a viewpoint corresponding to the second danmaku comment is within a first preset angle range. It may be understood that a playback time point of the second danmaku comment is relative to the first playback time point.

[0295] In some other embodiments, the first interface displayed by the terminal further includes a first control, the first control being associated with a number of first danmaku comments, the number of first danmaku comments being the number of danmaku comments obtained when an offset angle between a viewpoint of the danmaku comments and a playback viewpoint of the terminal is within a second preset angle range, the second preset angle range not overlapping with the first preset angle range. The terminal displays a third interface upon receiving a third operation on the first control, the third interface including the first danmaku comments. In other words, a user may expand a danmaku comment hidden by the terminal by using the first control.

[0296] An embodiment of this application further provides an apparatus. The apparatus is included in a terminal, and the apparatus has a function for realizing the behavior of the terminal in any of the methods in the above embodiments. The function may be realized by hardware, or by hardware executing corresponding software. The hardware or software includes at least one module or unit corresponding to the above function, such as a detection module or unit, a display module or unit, a determination module or unit, and a calculation module or unit.

[0297] An embodiment of this application further provides a computer storage medium containing computer instructions, which, when executed on a terminal, enable the terminal to perform any of the methods in the above embodiments.

[0298] An embodiment of this application further provides a computer program product, which, when executed on a computer, enables the computer to execute any of the methods in the above embodiments.

[0299] An embodiment of the present application further provides a graphical user interface on a terminal, the terminal including a display, a camera, a memory, and one or more processors configured to execute one or more computer programs stored in the memory, the graphical user interface including a graphical user interface displayed when the terminal executes any of the methods in the above embodiments.

[0300] It can be understood that to realize the above functions, the terminal etc. includes corresponding hardware structures and / or software modules for executing the functions. In combination with the examples described in the embodiments disclosed in this specification, those skilled in the art should easily recognize that the units, algorithms, and steps may be realized by hardware or a combination of hardware and computer software in the embodiments of this application. Whether the functions are performed by hardware or by hardware driven by computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to realize the described functions for each specific application, but the implementation method should not be considered to exceed the scope of the embodiments of the present invention.

[0301] In the embodiments of this application, a terminal or the like may be divided into functional modules based on the above-mentioned method examples. For example, each functional module may be obtained through division based on its corresponding function, or two or more functions may be integrated into one processing module. The integrated module may be implemented in the form of hardware or in the form of a software functional module. It should be noted that in the embodiments of the present invention, the division into modules is an example and is merely a logical division of functions, and other divisions may be used in actual implementation.

[0302] The above description of the implementation manner allows those skilled in the art to clearly understand that the above division of functional modules is taken as an example for illustration purposes for the purpose of convenient and concise description. In actual application, the above functions can be allocated to different modules and implemented as needed, that is, the internal structure of the device is divided into different functional modules to realize all or part of the above functions. For detailed operation processes of the above systems, devices and units, please refer to the corresponding processes in the above method embodiments, and the details will not be described again here.

[0303] The functional units in the embodiments of this application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional unit.

[0304] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application may essentially, or all or part of the parts contributing to the prior art or the technical solutions may be realized in the form of a software product. A computer software product is stored in a storage medium and includes several instructions for instructing a computer device (which may be a personal computer, a server, or a network device) or a processor to perform all or part of the steps of the method described in the embodiments of this application. The above storage medium includes any medium that can store program code, such as a flash memory, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0305] The above description is merely a specific implementation of this application and is not intended to limit the scope of protection of this application. Any variations or substitutions within the technical scope disclosed in this application shall be included in the scope of protection of this application. Therefore, the scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A method for displaying barrage comments of a free viewpoint video, which is applied to a terminal, comprising: a step of displaying a first interface of a first free viewpoint video at a first playback time point, the first interface including a video image of the first free viewpoint video at a first viewpoint and a first danmaku comment, an offset angle between the first viewpoint and a third viewpoint corresponding to the first danmaku comment being within a first preset angle range, the first interface further including a first control, the first control being related to the number of first danmaku comments, the number of first danmaku comments being the number of danmaku comments obtained when the offset angle between the viewpoint of the danmaku comment and a playback viewpoint of the terminal being within a second preset angle range, the second preset angle range not overlapping with the first preset angle range; receiving a first operation, the first operation being an operation for switching a playback viewpoint of the first free viewpoint video from the first viewpoint to a second viewpoint; a step of displaying a second interface of the first free viewpoint video at the first playback time point in response to receiving the first operation, the second interface including a video image of the first free viewpoint video at the second viewpoint, and the second interface not including the first barrage comment; A method comprising:

2. 2. The method of claim 1, wherein an offset angle between the first viewpoint and a third viewpoint corresponding to the first danmaku comment is smaller than an offset angle between the second viewpoint and the third viewpoint corresponding to the first danmaku comment.

3. Before receiving the first operation, The method of claim 1 , further comprising receiving a second operation to pause playback of the first free viewpoint video.

4. The method described in claim 1, wherein the offset angle between the second viewpoint and the third viewpoint corresponding to the first barrage comment is outside the first preset angle range.

5. 5. The method of claim 4, wherein the second interface includes a second danmaku comment, and an offset angle between the second viewpoint and a viewpoint corresponding to the second danmaku comment is within the first preset angle range.

6. The method of claim 5 , wherein a sending time of the first barrage comment is relative to the first playback time, and a sending time of the second barrage comment is relative to the first playback time.

7. receiving a third operation for the first control; displaying a third interface in response to receiving the third operation, the third interface including the first barrage comment; The method of claim 1 further comprising:

8. In response to receiving the first operation, after displaying a second interface of the first free viewpoint video at the first playback point in time, receiving a fourth operation, the fourth operation being an operation of sending a third barrage comment; a step of transmitting data of the third barrage comment to a barrage comment server in response to receiving the fourth operation, wherein the data of the third barrage comment includes a fourth perspective; The method of claim 1 further comprising:

9. The method of claim 1 , wherein the first interface further includes a fourth barrage comment, the fourth barrage comment corresponding to the first viewpoint.

10. 10. The method of claim 9, wherein in the first interface, a first style of the first barrage comment is different from a second style of the fourth barrage comment.

11. before displaying a first interface of a first free viewpoint video at a first playback time point; 11. The method of claim 10, further comprising determining to display the first danmaku comment in the first style in the first interface based on a first offset angle between the first viewpoint and the third viewpoint corresponding to the first danmaku comment.

12. 12. The method of claim 11, wherein, when the first offset angle is a positive value, the first style is a direction in which the text direction of the barrage comments is rotated clockwise by the first angle with respect to a first direction, the first direction being parallel to a first bezel of a screen of the terminal, and when the first offset angle is a negative value, the first style is a direction in which the text direction of the barrage comments is rotated counterclockwise by the first angle with respect to the first direction.

13. receiving a fifth operation for switching the viewpoint; displaying a fifth interface in the process of the fifth operation, where the fifth interface and the first interface are the same; After the process of the fifth operation, displaying a sixth interface of the first free viewpoint video at a second playback point in time; The method of claim 1 further comprising:

14. receiving a sixth operation for switching the viewpoint; displaying a seventh interface in the sixth operation process, the seventh interface including a video image of the first free viewpoint video at a third playback point in time; after the process of the sixth operation, displaying an eighth interface of the first free viewpoint video at a second playback point in time, where the third playback point is after the first playback point in time and before the second playback point in time; The method of claim 1 further comprising:

15. 15. The method of claim 14, wherein the seventh interface further includes a fifth barrage comment, and the first interface further includes no fifth barrage comment.

16. 15. The method of claim 14, wherein the seventh interface further includes the first barrage comment, and further comprising: a display style of the first barrage comment in the seventh interface that is different from a display style of the first barrage comment in the first interface.

17. A terminal including a processor, a memory, and a touchscreen, The memory and the touch screen are coupled to the processor, the memory is configured to store computer program code, the computer program code includes computer instructions, and when the processor reads the computer instructions from the memory, the terminal is enabled to execute the method for displaying barrage comments in free viewpoint video described in any one of claims 1 to 16.

18. A computer-readable storage medium containing computer instructions, A computer-readable storage medium, the computer instructions being, when executed on a terminal, capable of causing the terminal to execute the method for displaying barrage comments in free viewpoint video described in any one of claims 1 to 16.

19. A computer program which, when executed by a processing device, causes the processing device to carry out a method according to any one of claims 1 to 16.

Images