Bullet screen display method and device, head-mounted display equipment, storage medium and program product

Abstract

The embodiment of the invention provides a bullet screen display method and device, head-mounted display equipment, a computer readable storage medium and a computer program product, and relates to the technical field of virtual reality. The bullet screen display method comprises the following steps: collecting eyeball movement data of a user in real time; determining a first bullet screen focused by the sight line of the user at the current moment according to the eyeball movement data; and if the time length that the sight line of the user focuses on the target bullet screen exceeds the preset time length, displaying the target bullet screen in a three-dimensional space in a floating manner according to preset bullet screen configuration information, the bullet screen configuration information comprises a floating track of the target bullet screen in the three-dimensional space and depth information when the target bullet screen is located at a plurality of track points included in the floating track. According to the invention, the user can perform interactive operation with the bullet screen in the head display device, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of virtual reality technology, and in particular to a bullet screen display method, apparatus, head-mounted display device, computer-readable storage medium, and computer program product. Background Art

[0002] Head-mounted display devices, also known as head-mounted display devices (HMD), are devices that provide an immersive visual experience by placing the display screen close to the user's eyes. Head-mounted display devices are usually used in virtual reality (VR), augmented reality (AR) and mixed reality (MR) applications. In order to make head-mounted display devices better used in video scenes, various companies have developed VR apps that are compatible with head-mounted display devices. These VR apps are used in head-mounted display devices for video playback, and simulate realistic virtual environments through three-dimensional technology, allowing users to experience and interact immersively.

[0003] However, the inventors have found that when existing head-mounted display devices display the barrage input by the user during video playback, although the barrage can be displayed on a two-dimensional plane in a three-dimensional space, the user cannot interact with the barrage, resulting in a poor user experience when using the head-mounted display device.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a bullet screen display method, apparatus, head-mounted display device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a bullet screen display method, which is applied to a head display device, and the method includes: Collect user's eye movement data in real time; Determine the first bullet screen that the user's line of sight is currently focused on according to the eye movement data; If the user's line of sight is focused on the target barrage for more than a preset time, the target barrage will be displayed floating up in the three-dimensional space according to the preset barrage configuration information, wherein the barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when it is located at multiple trajectory points contained in the floating trajectory.

[0007] Optionally, determining the first bullet screen at which the user's line of sight is focused at the current moment according to the eye movement data includes: Determine the screen position where the user's line of sight is currently focused according to the eye movement data; Obtaining a bullet screen area of ​​at least one second bullet screen at the current moment; The first barrage on which the user's line of sight is focused at the current moment is determined according to the barrage area of ​​the at least one second barrage and the screen position.

[0008] Optionally, determining the first bullet screen at which the user's line of sight is focused at the current moment according to the eye movement data includes: Preprocessing the eye movement data, wherein the preprocessing is used to filter out invalid eye movement data; If the eye movement data is not filtered out, the first bullet screen that the user's line of sight is focused on at the current moment is determined according to the eye movement data.

[0009] Optionally, the method further comprises: After determining the first bullet screen that the user's line of sight is focused on at the current moment, starting a timer; If it is detected that the user's sight is no longer focused on the first bullet screen, the timer is reset.

[0010] Optionally, the floating display of the target bullet chat in three-dimensional space according to preset bullet chat configuration information includes: Obtaining a floating animation corresponding to the target bullet screen, wherein the number of animation frames included in the floating animation is the same as the number of trajectory points; Each animation frame in the floating animation is rendered in sequence through a 3D rendering engine to realize floating display of the target barrage in the three-dimensional space. When rendering the floating animation, the depth of each animation frame in the three-dimensional space is adjusted according to the barrage configuration information.

[0011] Optionally, the barrage configuration information further includes size information of the target barrage when they are respectively located at a plurality of track points included in the floating track, and the method further includes: When rendering the floating animation, the size of each animation frame in the three-dimensional space is adjusted according to the bullet screen configuration information.

[0012] Optionally, the method further comprises: In response to the user's interaction instruction to the target barrage, an interaction operation corresponding to the interaction instruction is performed.

[0013] Another aspect of an embodiment of the present application provides a bullet screen display device, which is applied to a head-mounted display device, and the device includes: A collection module, used to collect the user's eye movement data in real time; A determination module, configured to determine the first bullet screen that the user's line of sight is focused on at the current moment according to the eye movement data; A display module is used to display the target barrage in a floating manner in a three-dimensional space according to preset barrage configuration information if the user's line of sight is focused on the target barrage for more than a preset time. The barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when it is located at multiple trajectory points contained in the floating trajectory.

[0014] Another aspect of an embodiment of the present application provides a head mounted display device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein: the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as described above.

[0015] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0016] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0017] The above technical solution adopted in the embodiment of the present application may have the following advantages: the embodiment of the present application collects the user's eye movement data in real time when the user uses the head display device to watch the video, and then determines the first bullet screen that the user's line of sight is focused on at the current moment according to the eye movement data, and when it is detected that the user's line of sight is focused on the target bullet screen for more than a preset time, it is determined that the user needs to interact with the target bullet screen. At this time, the target bullet screen will be displayed in a floating manner in a three-dimensional space according to the preset bullet screen configuration information, so as to realize the user's interactive operation with the target bullet screen. In addition, since the interactive operation is realized by floating the target bullet screen in a three-dimensional space, that is, by displaying the target bullet screen in three dimensions, compared with the interactive operation in a two-dimensional plane, this interactive method can improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0019] Figure 1 The operating environment diagram of the bullet screen display method according to the first embodiment of the present application is schematically shown.

[0020] Figure 2 The flowchart of the bullet screen display method according to the first embodiment of the present application is schematically shown.

[0021] Figure 3 Schematically shows Figure 2 Detailed diagram of the step flow of step S202.

[0022] Figure 4 Schematically shows Figure 2 Detailed diagram of the step flow of step S202.

[0023] Figure 5 Schematically shows Figure 2 Detailed diagram of the step flow of step S204.

[0024] Figure 6 A block diagram of a bullet screen display device according to the second embodiment of the present application is schematically shown.

[0025] Figure 7 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0027] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0028] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0029] First, the following terms are explained: Head-mounted display device: also known as head-mounted display device (HMD), is a device that provides an immersive visual experience by placing the display screen close to the user's eyes.

[0030] Danmu: subtitles that pop up and move in a predetermined direction when watching a video on the Internet. There is no fixed word for Danmu in English, and it is usually called: comment, danmaku, barrage, bullet screen, bullet-screen comment, etc. Danmu allows viewers to comment or feel something, but unlike ordinary video sharing websites that only display comments in the dedicated comment area under the player, it will appear in real time on the video screen in the form of sliding subtitles to ensure that all viewers can notice it. Danmu can be managed by the Danmu system. Some Danmu systems can use CSS (Cascading StyleSheets) or JavaScript language to provide specific Danmu forms, such as the appearance or disappearance of Danmu in a specific position, control the speed of Danmu pop-up, Danmu position, etc. In addition, the Danmu that appears fixed at the bottom or top of the screen will also be used as subtitles for videos without subtitles.

[0031] For example, the data structure of a bullet comment can be as follows: { The time of emission in the video is in seconds; Types of barrages; The size of the barrage text; The color of the barrage text; Date the comment was created; The class name of the bullet comment; The user ID that created the bullet comment; Barrage ID (e.g. barrage group ID); Barrage content (e.g., text, images, controls); The duration of the barrage; Whether the bullet screen has a border; The color of the bullet screen border; The interline style of the bullet comment; }.

[0032] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: When users use head-mounted display devices to watch videos, although users can experience an immersive video viewing experience, the barrage is still only displayed on a two-dimensional plane in the three-dimensional space. Users cannot interact with the barrage, resulting in a poor user experience when using the head-mounted display device.

[0033] To this end, the embodiment of the present application provides a technical solution for bullet screen display. In this technical solution, when the user uses a head-mounted display device to watch a video, the user's eye movement data is collected in real time, and then the first bullet screen that the user's line of sight is focused on at the current moment is determined based on the eye movement data, and when it is detected that the user's line of sight is focused on the target bullet screen for more than a preset time, it is determined that the user needs to interact with the target bullet screen. At this time, the target bullet screen will be displayed in a floating manner in three-dimensional space according to the preset bullet screen configuration information, so that the user can interact with the target bullet screen. See below for details.

[0034] Finally, for ease of understanding, an exemplary operating environment is provided below.

[0035] like Figure 1 As shown, the environmental application schematic diagram includes: a service platform 2 and a client.

[0036] The service platform 2 can be connected to the client terminals ( 4A, 4B, ..., 4N) via the network.

[0037] The service platform 2 can provide various content and bullet screen services for the client. For example, the bullet screen is sent to the client via the network. The network includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices and / or the like. The network may include physical links, such as coaxial cable links, twisted pair cable links, optical fiber links, combinations thereof, etc. The network may include wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0038] The service platform 2 may be a single server, a server cluster or a cloud computing service center, which is used to provide various services. For example: The service platform 2 may provide content services. The content services may be configured to provide content such as video, audio, text data, or a combination thereof. The content may include content streams (e.g., video streams, audio streams, information streams), content files (e.g., video files, audio files, text files), and / or other data.

[0039] The service platform 2 can be configured to manage messages for various content items. Users can browse content and access different content items to view comments for specific content, such as comments posted by other users for the specific content. Comments from users associated with a specific content item can be output to other users who view the specific content item. For example, all users who access a content item (e.g., a video) can view comments associated with the content item. The input comment content can be output in real time or near real time.

[0040] The service platform 2 may be configured to receive a plurality of messages. The plurality of messages may include a plurality of bullet comments associated with the content.

[0041] The service platform 2 may be configured to process multiple messages, for example, various processing operations such as message storage, message filtering, and message push. Among them, message storage is used to store multiple messages in a data storage such as a database. Message filtering may include rejecting or marking messages that match the filtering criteria. Among them, the filtering criteria may specify terms and / or phrases, such as profanity, hate speech, indecent language, etc. The filtering criteria may specify characters, such as symbols, fonts, etc. The filtering criteria may specify languages, computer-readable code patterns, etc.

[0042] The service platform 2 can perform natural language processing, topic recognition, pattern recognition, artificial intelligence, etc. to automatically determine the characteristics of the message and / or group the message. As an example, frequently occurring phrases or patterns can be identified as topics. As another example, a database of topics associated with the content can be maintained. Topics can include genres (e.g., action, drama, comedy), personalities (e.g., actors, actresses, directors), languages, etc. Messages can be grouped based on the characteristics of the client device and / or the user sending the message. Demographics, interests, history, and / or the like can be stored for multiple users to determine potential groupings of messages. For example, the bullet screens uploaded by each user can be screened, filtered, sorted, and personalized push, etc.

[0043] The service platform 2 may provide a bullet screen service, which is configured to allow users to comment and / or share comments (i.e. bullet screens) associated with the content. The bullet screens are presented on the same screen together with the content. The bullet screens may be displayed in an overlay above the content and may have animation effects. For example, the bullet screens may scroll (e.g., from right to left, from left to right, from top to bottom, ...), and the animation effect may be implemented based on the transition property of CSS3.

[0044] The client (4A, 4B, ..., 4N) may be configured to access the content and services of the service platform 2. The client (4A, 4B, ..., 4N) may include an electronic device carrying or connected to a display panel, such as a mobile device, a tablet device, a laptop computer, a workstation, a head-mounted display device with a virtual reality function, a gaming device, a digital streaming device, a vehicle client, a smart TV, a set-top box, etc., and may also include a virtualized computing instance. The virtualized computing instance may include a virtual machine, such as a simulation of a computer system, an operating system, a server, etc.

[0045] The client (4A, 4B, ..., 4N) may be associated with one or more users. A single user may also use one or more of the clients (4A, 4B, ..., 4N) to access the service platform 2. The client (4A, 4B, ..., 4N) may travel to various locations and use different networks to access the service platform 2.

[0046] The client (4A, 4B, ..., 4N) may include multiple client programs, such as a video APP, a live broadcast APP or a mini program with a built-in bullet screen system. In this embodiment, the video and other contents provided by the service platform 2 may be presented through the client program, and the bullet screen system may present the bullet screen contents in a specific manner.

[0047] The client (4A, 4B, ..., 4N) may include an interface, which may include a touch pad, a touch screen, a mouse, a keyboard or other sensing elements. For example, the input element may be configured to receive user instructions, and the user instructions may enable the client (4A, 4B, ..., 4N) to perform various operations, such as inputting bullet comments, etc.

[0048] It should be noted that the above devices are exemplary, and the number and type of devices are adjustable in different scenarios or according to different needs.

[0049] The technical solutions of the present application are described below through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described here.

[0050] Embodiment 1 Figure 2 The flowchart of the bullet screen display method according to the first embodiment of the present application is schematically shown. The bullet screen display method is applied in a head-mounted display device.

[0051] like Figure 2 As shown, the bullet screen display method may include steps S200 to S204, wherein: Step S200, collecting the user's eye movement data in real time.

[0052] Step S202, determining the first bullet screen that the user's line of sight is focused on at the current moment according to the eye movement data.

[0053] Step S204, if the user's line of sight is focused on the target barrage for more than a preset time, the target barrage is displayed floating up in the three-dimensional space according to the preset barrage configuration information, wherein the barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when it is located at multiple trajectory points contained in the floating trajectory.

[0054] The bullet screen display method provided in this embodiment collects the user's eye movement data in real time when the user uses a head-mounted display device to watch a video, and then determines the first bullet screen that the user's line of sight is focused on at the current moment based on the eye movement data, and when it is detected that the user's line of sight is focused on the target bullet screen for more than a preset time, it is determined that the user needs to interact with the target bullet screen. At this time, the target bullet screen will be displayed in a floating manner in a three-dimensional space according to the preset bullet screen configuration information, so as to realize the user's interactive operation with the target bullet screen. In addition, since the interactive operation is realized by floating the target bullet screen in a three-dimensional space, that is, by displaying the target bullet screen in three dimensions, compared with the interactive operation in a two-dimensional plane, this interactive method can improve the user experience.

[0055] The following combination Figure 2 , each step in steps S200~S204 and other optional steps are described in detail.

[0056] Step S200 , collect users’ eye movement data in real time.

[0057] When a user watches a video through a head-mounted display device, the user's eye movement data can be collected in real time through the eye movement sensor built into the head-mounted display device. The frequency of the eye movement sensor collecting eye movement data can be set according to actual conditions, for example, the collection frequency is set to 10 times / second.

[0058] Among them, the eye movement data may include the user's line of sight direction, the user's line of sight focus position, etc. The user's line of sight focus position may be referred to as human eye spatial position information, which may be expressed in the form of three-dimensional coordinates, for example, it includes the distance information between the user's eyes / face and the screen of the head display device (that is, the depth information of the user's eyes / face), the horizontal position information of the viewing eyes / face on the screen of the head display device, and the vertical position information of the user's eyes / face on the screen of the head display device. The human eye spatial position may also be expressed in the form of two-dimensional coordinates containing any two of the distance information, the horizontal position information, and the vertical position information.

[0059] Step S202 , determine the first bullet screen that the user's line of sight is focused on at the current moment based on the eye movement data.

[0060] Since the eye movement data contains the screen position where the user's line of sight is focused (including the horizontal and vertical position information of the screen), and the barrage is displayed by sliding on the screen according to the preset barrage trajectory, after obtaining the eye movement data, the screen position where the user's line of sight is focused contained in the eye movement data collected at each moment can be matched with the barrage currently displayed at that screen position to obtain the first barrage that the user's line of sight is focused on at the current moment.

[0061] As an example, assuming that the screen position where the user's line of sight is focused contained in the eye movement data collected at 0.1s is represented by a two-dimensional coordinate of (100, 200), and the barrage currently displayed on the two-dimensional coordinate is barrage A, then barrage A can be used as the first barrage focused on by the user's line of sight at 0.1s. Similarly, assuming that the screen position where the user's line of sight is focused contained in the eye movement data collected at 0.5s is represented by a two-dimensional coordinate of (300, 500), and the barrage currently displayed on the two-dimensional coordinate is barrage B, then barrage B can be used as the first barrage focused on by the user's line of sight at 0.1s.

[0062] In an optional embodiment, if Figure 3 As shown, step S202 includes: S300: Determine the screen position where the user's line of sight is currently focused according to the eye movement data.

[0063] S302, obtaining a bullet screen area of ​​at least one second bullet screen at the current moment.

[0064] S304, determining the first barrage on which the user's line of sight is focused at the current moment according to the barrage area of ​​the at least one second barrage and the screen position.

[0065] In a specific implementation, the barrage area is the area occupied by all the contents of the second barrage, that is, the barrage area is determined by the length of the barrage, and the longer the second barrage is, the larger its barrage area is. In this embodiment, the barrage area can be defined by a geometric area such as a rectangle or a circle.

[0066] The at least one second barrage is the barrage displayed on the screen at the current moment.

[0067] In this embodiment, after obtaining the bullet screen area of ​​at least one second bullet screen and the screen position, a collision detection algorithm may be used to detect within which bullet screen area of ​​the second bullet screen the screen position falls.

[0068] It should be noted that, if it is not detected that the screen position falls within the bullet screen area of ​​the at least one second bullet screen, it indicates that the user's current line of sight is not focused on the bullet screen.

[0069] In this embodiment, the first bullet screen on which the user's line of sight is focused at the current moment is determined directly based on the screen position on which the user's line of sight is focused and the bullet screen areas of each second bullet screen, so that the bullet screen on which the user's line of sight is focused can be accurately found.

[0070] In other embodiments, when determining the first barrage on which the user's line of sight is focused at the current moment based on the eye movement data, the barrage area of ​​at least one second barrage can also be converted into a barrage area in a three-dimensional space, and then the first barrage on which the user's line of sight is focused at the current moment can be determined based on the three-dimensional coordinates corresponding to the eye movement data and the barrage areas of each second barrage in the three-dimensional space.

[0071] In an optional embodiment, if Figure 4 As shown, step S202 includes: Step S400, preprocessing the eye movement data, wherein the preprocessing is used to filter out invalid eye movement data.

[0072] Step S402: If the eye movement data is not filtered out, determine the first bullet screen that the user is focusing on at the current moment according to the eye movement data.

[0073] In the specific implementation, since the user's line of sight may not necessarily be focused on the bullet screen, it may be focused on an area without bullet screen. When the user's line of sight is focused on an area without bullet screen, the collected eye movement data is an invalid eye movement data, which can be filtered out. In addition, when the user's line of sight quickly scans the bullet screen on the screen, the collected eye movement data in this case is also an invalid eye movement data, which can also be filtered out.

[0074] In this embodiment, after the eye movement data is collected, the eye movement data is first pre-processed to determine whether the eye movement data is invalid eye movement data. If it is invalid eye movement data, there is no need to perform subsequent processing procedures. If it is valid eye movement data, the subsequent processing procedures will be executed, thereby reducing the consumption of processing resources.

[0075] In an optional embodiment, the method further comprises: After determining the first bullet screen on which the user's line of sight is focused at the current moment, start a timer; if it is detected that the user's line of sight is no longer focused on the first bullet screen, reset the timer.

[0076] In the specific implementation, in order to accurately count the time that the user's eyes are focused on the bullet screen, after determining the first bullet screen that the user's eyes are focused on at the current moment, a timer will be started for the first bullet screen. When it is detected that the user's eyes are no longer focused on the first bullet screen, the timer will be reset, thereby clearing the timer of the first bullet screen.

[0077] As an example, after determining the first bullet screen a that the user's line of sight is focused on at the moment, timer a will be started to count the focus time of the first bullet screen. Similarly, after determining the first bullet screen b that the user's line of sight is focused on at the moment, timer b will be started to count the focus time of the first bullet screen b.

[0078] In this embodiment, a timer is maintained for each bullet screen that the user's line of sight has focused on to count its focus time, so that the focus time of each bullet screen that the user's line of sight has focused on can be accurately counted.

[0079] Step S204 If the user's line of sight is focused on the target barrage for more than a preset time, the target barrage will be displayed floating up in the three-dimensional space according to the preset barrage configuration information, wherein the barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when it is located at multiple trajectory points contained in the floating trajectory.

[0080] In specific implementation, the preset duration can be set and modified according to actual conditions, for example, the preset duration is 2 seconds. The floating trajectory can be a straight floating along the Z axis of the three-dimensional space, or a diagonal floating along the Z axis of the three-dimensional space, which is not limited in this embodiment.

[0081] The depth information can be represented by the Z-axis coordinate of the three-dimensional space. Different depths represent different Z-axis coordinate values ​​corresponding to the bullet screen when it is displayed in the three-dimensional space.

[0082] In this embodiment, the target barrage will be displayed in a floating manner only when the user's line of sight is focused on the target barrage for more than a preset period of time, thereby highlighting the target barrage and facilitating the user to further interact with the target barrage.

[0083] In an optional embodiment, if Figure 5 As shown, the step S204 includes: S500, obtaining a floating animation corresponding to the target bullet screen, wherein the number of animation frames included in the floating animation is the same as the number of the track points.

[0084] S502, rendering each animation frame in the floating animation in sequence through a 3D rendering engine to realize floating display of the target barrage in the three-dimensional space, wherein when rendering the floating animation, adjusting the depth of each animation frame in the three-dimensional space according to the barrage configuration information.

[0085] In the specific implementation, in order to realize that the target bullet screen can be displayed in a floating state in three-dimensional space, a floating animation corresponding to the target bullet screen will be created before the display. The floating animation can be created by key frames, or directly combined by combining multiple animation frames in the playback order.

[0086] After the floating animation is obtained, the built-in 3D rendering engine in the head display device can be used to render each animation frame in the floating animation in sequence, so as to realize floating display of the target bullet screen in the three-dimensional space. In the process of rendering different animation frames, the rendering engine will adjust the rendering parameters of the target bullet screen according to the depth information, the size information of the bullet screen subtitles, the position information of the bullet screen, etc. in the bullet screen configuration information, so as to realize the adjustment of the depth, position and size of each animation frame in the three-dimensional space.

[0087] It should be noted that the rendering parameters may also include light and shadow parameters, angle information of the barrage display, etc.

[0088] The 3D rendering engine may be a rendering engine such as Metal or SceneKit.

[0089] In this embodiment, when performing a floating animation through 3D rendering, the depth of each animation frame in the three-dimensional space is adjusted, so that the target bullet screen can show an obvious floating effect.

[0090] In an optional embodiment, the barrage configuration information further includes size information of the target barrage when the target barrage is located at a plurality of track points included in the floating track, and the method further includes: When rendering the floating animation, the size of each animation frame in the three-dimensional space is adjusted according to the bullet screen configuration information.

[0091] In a specific implementation, by adjusting the size of each animation frame in the three-dimensional space according to the bullet screen configuration information when rendering the floating animation, the target bullet screen can be enlarged during the floating process.

[0092] In this embodiment, the target barrage is enlarged while the target barrage is floating up, so that the target barrage can be more prominent, further enhancing the user experience.

[0093] In an optional embodiment, the method further comprises: In response to the user's interaction instruction to the target barrage, an interaction operation corresponding to the interaction instruction is performed.

[0094] During specific implementation, the interaction instruction may be a gesture instruction, a voice instruction, etc.

[0095] In order to subsequently perform corresponding interactive operations according to the interactive instructions, mapping relationships between different interactive instructions and interactive operations can be pre-stored. For example, the interactive operation corresponding to interactive instruction 1 is to collect the barrage, the interactive operation corresponding to interactive instruction 2 is to mark the barrage as "favorite", the interactive operation corresponding to interactive instruction 3 is to block the barrage, the interactive operation corresponding to interactive instruction 4 is to report the barrage, and the interactive operation corresponding to interactive instruction 5 is to delete the barrage, etc.

[0096] In this embodiment, by responding to the user's interactive instructions for the target barrage, corresponding interactive operations are implemented, so that the head display device can provide richer functions and enhance the user experience.

[0097] Embodiment 2 Figure 6 The block diagram of the bullet screen display device 600 according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium, and are executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Figure 6 As shown, the device 600 may include: a collection module 601, a determination module 602 and a display module 603, wherein: The collection module 601 is used to collect the user's eye movement data in real time.

[0098] The determination module 602 is used to determine the first bullet screen that the user's line of sight is focused on at the current moment according to the eye movement data.

[0099] Display module 603 is used to display the target barrage in a floating manner in three-dimensional space according to preset barrage configuration information if the user's line of sight is focused on the target barrage for more than a preset time. The barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when they are respectively located at multiple trajectory points contained in the floating trajectory.

[0100] As an optional embodiment, determination module 602 is also used to: determine the screen position on which the user's line of sight is focused at the current moment based on the eye movement data; obtain the barrage area of ​​at least one second barrage at the current moment; and determine the first barrage on which the user's line of sight is focused at the current moment based on the barrage area of ​​the at least one second barrage and the screen position.

[0101] As an optional embodiment, the determination module 602 is also used to: pre-process the eye movement data, wherein the pre-processing is used to filter out invalid eye movement data; if the eye movement data is not filtered out, determine the first barrage that the user's line of sight is focused on at the current moment based on the eye movement data.

[0102] As an optional embodiment, the device 600 is also used to: start a timer after determining the first barrage that the user's line of sight is focused on at the current moment; if it is detected that the user's line of sight is no longer focused on the first barrage, reset the timer.

[0103] As an optional embodiment, the display module 603 is also used to: obtain a floating animation corresponding to the target barrage, the number of animation frames contained in the floating animation being the same as the number of trajectory points; render each frame of the floating animation in sequence through a 3D rendering engine to realize a floating display of the target barrage in the three-dimensional space, wherein when rendering the floating animation, the depth of each frame of the animation in the three-dimensional space is adjusted according to the barrage configuration information.

[0104] As an optional embodiment, the barrage configuration information also includes size information of the target barrage when they are respectively located at multiple trajectory points contained in the floating trajectory; the device 600 is also used to: when rendering the floating animation, adjust the size of each frame of the animation in the three-dimensional space according to the barrage configuration information.

[0105] As an optional embodiment, the device 600 is further used to: respond to the user's interaction instruction to the target barrage, perform an interaction operation corresponding to the interaction instruction.

[0106] Embodiment 3 Figure 7 The schematic diagram of the hardware architecture of a head display device 700 suitable for implementing the bullet screen display method according to the third embodiment of the present application is schematically shown. In some embodiments, the head display device 700 can be VR glasses, VR helmets, etc. Figure 7 As shown, the head display device 700 includes but is not limited to: a memory 701, a processor 702, and a network interface 703 that can communicate with each other through a system bus. The memory 701 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 701 can be an internal storage module of the head display device 700, such as a hard disk or memory of the head display device 700. In other embodiments, the memory 701 can also be an external storage device of the head display device 700, such as a plug-in hard disk equipped on the head display device 700, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 701 can also include both the internal storage module of the head display device 700 and its external storage device. In this embodiment, the memory 701 is generally used to store the operating system and various application software installed on the head display device 700, such as the program code of the bullet screen display method, etc. In addition, the memory 701 can also be used to temporarily store various types of data that have been output or are to be output.

[0107] In some embodiments, the processor 702 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 702 is generally used to control the overall operation of the head display device 700, such as performing control and processing related to data interaction or communication with the head display device 700. In this embodiment, the processor 702 is used to run the program code stored in the memory 701 or process data.

[0108] The network interface 703 may include a wireless network interface or a wired network interface, and the network interface 703 is generally used to establish a communication link between the head display device 700 and other computer devices. For example, the network interface 703 is used to connect the head display device 700 to an external terminal through a network, and to establish a data transmission channel and a communication link between the head display device 700 and the external terminal. The network can be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0109] It should be pointed out that Figure 7 Only a computer device having components 701 - 703 is shown, but it should be understood that implementing all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0110] In this embodiment, the bullet screen display method stored in the memory 701 can also be divided into one or more program modules and executed by one or more processors (such as processor 702) to complete the embodiment of the present application.

[0111] Embodiment 4 An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the barrage display method in the embodiment are implemented.

[0112] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium may be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Of course, the computer-readable storage medium may also include both an internal storage unit of a computer device and an external storage device thereof. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on a computer device, such as the program code of the bullet screen display method in the embodiment, etc. In addition, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.

[0113] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0114] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0115] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A bullet screen display method, applied to a head-mounted display device, characterized in that: The method comprises: Collect user's eye movement data in real time; Determine the first bullet screen that the user's line of sight is currently focused on according to the eye movement data; If the user's line of sight is focused on the target barrage for more than a preset time, the target barrage will be displayed floating up in the three-dimensional space according to the preset barrage configuration information, wherein the barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when it is located at multiple trajectory points contained in the floating trajectory.

2. The method according to claim 1, characterized in that: The first bullet screen that determines the user's sight focus at the current moment according to the eye movement data includes: Determine the screen position where the user's line of sight is currently focused according to the eye movement data; Obtaining a bullet screen area of ​​at least one second bullet screen at the current moment; The first barrage on which the user's line of sight is focused at the current moment is determined according to the barrage area of ​​the at least one second barrage and the screen position.

3. The method according to claim 1, characterized in that The first bullet screen that determines the user's sight focus at the current moment according to the eye movement data includes: Preprocessing the eye movement data, wherein the preprocessing is used to filter out invalid eye movement data; If the eye movement data is not filtered out, the first bullet screen that the user's line of sight is focused on at the current moment is determined according to the eye movement data.

4. The method according to claim 1, characterized in that: The method further comprises: After determining the first bullet screen that the user's line of sight is focused on at the current moment, starting a timer; If it is detected that the user's sight is no longer focused on the first bullet screen, the timer is reset.

5. The method according to claim 1, characterized in that The step of floating and displaying the target bullet screen in three-dimensional space according to the preset bullet screen configuration information includes: Obtaining a floating animation corresponding to the target bullet screen, wherein the number of animation frames included in the floating animation is the same as the number of trajectory points; Each animation frame in the floating animation is rendered in sequence through a 3D rendering engine to realize floating display of the target barrage in the three-dimensional space. When rendering the floating animation, the depth of each animation frame in the three-dimensional space is adjusted according to the barrage configuration information.

6. The method according to claim 5, characterized in that The barrage configuration information also includes size information of the target barrage when they are respectively located at a plurality of track points included in the floating track, and the method further includes: When rendering the floating animation, the size of each animation frame in the three-dimensional space is adjusted according to the bullet screen configuration information.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: In response to the user's interaction instruction to the target barrage, an interaction operation corresponding to the interaction instruction is performed.

8. A bullet screen display device, applied to a head-mounted display device, characterized in that: The device comprises: A collection module, used to collect the user's eye movement data in real time; A determination module, configured to determine the first bullet screen that the user's line of sight is focused on at the current moment according to the eye movement data; A display module is used to display the target barrage in a floating manner in a three-dimensional space according to preset barrage configuration information if the user's line of sight is focused on the target barrage for more than a preset time. The barrage configuration information includes the floating trajectory of the target barrage in the three-dimensional space and the depth information of the target barrage when it is located at multiple trajectory points contained in the floating trajectory.

9. A head mounted display device, characterized in that: include: at least one processor; and a memory communicatively connected to the at least one processor; wherein: The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to claims 1 to 7 are implemented.

Images