Display device and viewing angle switching method

Abstract

The present application shows a display device and a method for switching viewing angles, which receives a control instruction input by a user for entering a virtual reality application; in response to the control instruction, obtains media; wherein the media is a panoramic viewing angle media; when the user views the media at the current viewing angle, receives a first message sent by a server; wherein the first message includes the camera position data and the camera rotation data of the center viewing angle; the center viewing angle is determined by the terminal and is used to set the viewing angle of the user's viewing range of the media; the display device, the server and the terminal are connected based on a network; and according to the first message, the current viewing angle of the media is switched to the center viewing angle. The technical solution shown in the present application can control the viewing angle of the user in real time during the user's use of the virtual reality application.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a method for switching viewing angles. Background Art

[0002] Virtual Reality (VR) technology is a display technology that uses computers to simulate virtual environments, giving people a sense of immersion. Virtual reality applications use scenes built with virtual reality technology to give users a better viewing experience. Virtual reality applications based on virtual reality technology (such as VR games, VR design, VR live broadcasts, etc.) are widely used in tourism, events, real estate, and medical industries.

[0003] However, in existing virtual reality applications, users usually control the viewing angle by themselves. For example, when users watch live broadcasts in VR, they do not have the spatial stereoscopic sense of watching live. In the process of controlling the viewing angle by themselves, users cannot determine the focus of the VR live broadcast in a timely and accurate manner. In order to help users better determine the focus of the VR live broadcast, in the existing way of watching VR live broadcasts, users are usually prompted with the focus position of the VR live broadcast through guidance by the director, subtitle prompts, and switching of the playback source.

[0004] However, only through the director's guidance, subtitle prompts and switching of playback sources in the VR live broadcast screen, it is impossible to interfere with the user's viewing perspective, and it cannot meet the needs of real-time control of the live broadcast. Summary of the invention

[0005] The present application provides a display device and a method for switching viewing angles, which can control the viewing angle of a user in real time when the user is using a virtual reality application.

[0006] In the first aspect, the present application shows a display device, including: a display; a controller, configured to: receive a control instruction input by a user for entering a virtual reality application; obtain media in response to the control instruction; wherein the media is a panoramic perspective media, and when the user views the media at the current perspective, a first message sent by a server is received; wherein the first message includes the camera position data and camera rotation data of the center perspective; the center perspective is determined by the terminal to set the perspective of the user viewing the media; the display device, the server and the terminal are connected based on a network; according to the first message, the current perspective of the media is switched to the center perspective. With this implementation, the user's current perspective can be switched to the first perspective, and the user's viewing perspective can be adjusted in real time so that the user can obtain the viewing focus in a timely and accurate manner and obtain the best viewing experience.

[0007] In the second aspect, the present application shows a terminal, including: a display; a controller, configured to: receive a control instruction input by a user for entering a virtual reality application; in response to the control instruction, send a third request instruction to the first network element for obtaining a second virtual reality player; wherein the first network element is a network element in the server that provides the second virtual reality player to the display device; load the second virtual reality player sent by the first network element in response to the third request instruction; play the media collected in real time by the image acquisition device through the second virtual reality player; when the user determines the central perspective of the media in the second virtual reality player, switch the perspective operation. The technical solution shown in the present application can determine the focus of the scene through manual means or other intelligent methods at the terminal, and can help users determine the viewing perspective in real time.

[0008] In a third aspect, the present application shows a display device, comprising: a display; a controller, configured to: receive a control instruction input by a user for entering a virtual reality application; in response to the control instruction, send a fourth request instruction to a first network element for obtaining a third virtual reality player; wherein the first network element is a network element in a server that provides the third virtual reality player to the display device; load the third virtual reality player sent by the first network element in response to the fourth request instruction; when the third virtual reality player is loaded on the display device, send a fifth request instruction to a second network element for obtaining media resources; wherein the second network element is a network element in a server that receives media resources collected by a terminal in real time and provides the media resources to the display device; control the third virtual reality player to play the media resources sent by the second network element to the display device in response to the fifth request instruction; the terminal has determined a central viewing angle of the media resources; when the user watches the media resources at the current viewing angle, the initial viewing angle of the media resources is the central viewing angle.

[0009] In a fourth aspect, the present application also shows a method for switching perspectives, which includes: obtaining media resources, which are panoramic perspective media resources, and when a user watches the media resources at a current perspective, receiving a first message sent by a server; the first message includes camera position data and camera rotation data of a center perspective; the center perspective is determined by a terminal; a display device, a server, and a terminal are connected based on a network; and according to the first message, the current perspective of the media resources is switched to the center perspective.

[0010] In a fifth aspect, the present application also shows a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the above-mentioned perspective switching method.

[0011] The technical solution shown above can solve the problem that the viewer's perspective cannot be directly intervened only by the director's guidance, message prompts, and switching playback sources during the user's use of virtual reality applications. This application shows a display device and a method for switching perspectives, which receives a control instruction input by the user for entering the virtual reality application; in response to the control instruction, obtains media assets; the media assets are panoramic perspective media assets collected by the terminal in real time, and when the user watches the media assets at the current perspective, receives a first message sent by the server; the first message includes the camera position data and camera rotation data of the center perspective; the center perspective is determined by the terminal; the display device, the server, and the terminal are connected based on a network; according to the first message, the current perspective of the media assets is switched to the center perspective. The technical solution shown in this application can control the user's viewing perspective in real time during the user's use of virtual reality applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 A diagram showing a usage scenario of a display device according to some embodiments;

[0014] Figure 2 shows a block diagram of a configuration of a control device 100 according to some embodiments;

[0015] Figure 3 shows a hardware configuration block diagram of a display device 200 according to some embodiments;

[0016] Figure 4 shows a software configuration diagram of a display device 200 according to some embodiments;

[0017] Figure 5 A schematic diagram showing an icon control interface display of an application according to some embodiments is shown;

[0018] Figure 6 A schematic diagram of a virtual reality application usage scenario according to some embodiments is shown;

[0019] Figure 7 A schematic diagram of a usage scenario of a first user terminal according to some embodiments is shown;

[0020] Figure 8 A schematic diagram of a usage scenario of a collection terminal according to some embodiments is shown;

[0021] Fig. 9 shows a flow chart of display device configuration according to some embodiments;

[0022] Fig.10 A schematic diagram of entering a virtual reality application through a URL according to some embodiments is shown;

[0023] Fig.11 shows a flow chart of display device configuration according to some embodiments;

[0024] Fig.12 shows a flow chart of display device configuration according to some embodiments;

[0025] Fig.13 shows a flow chart of display device configuration according to some embodiments;

[0026] Fig.14 shows a first prompt schematic diagram according to some embodiments;

[0027] Fig.15 shows a second prompt schematic diagram according to some embodiments;

[0028] Fig.16 shows a flow chart of display device configuration according to some embodiments;

[0029] Fig.17 A schematic diagram of the architecture of a collection end display device according to some embodiments is shown;

[0030] Fig.18 A schematic diagram of an application scenario of a virtual reality device according to some embodiments is shown. DETAILED DESCRIPTION

[0031] In order to make the purpose and implementation method of the present application clearer, the exemplary implementation method of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0032] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0033] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0034] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0035] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0036] References throughout this specification to "multiple embodiments", "some embodiments", "one embodiment" or "an embodiment" etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment. Therefore, phrases such as "in multiple embodiments", "in some embodiments", "in at least another embodiment" or "in an embodiment" etc. that appear throughout this specification do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, specific features, structures or characteristics may be combined in any suitable manner. Therefore, without limitation, specific features, structures or characteristics shown or described in conjunction with one embodiment may be combined in whole or in part with features, structures or characteristics of one or more other embodiments. Such modifications and variations are intended to be included within the scope of this application.

[0037] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to one or more embodiments of the present application, such as Figure 1 As shown, the user can operate the display device 200 through the mobile terminal 300 and the control device 100. The control device 100 can be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication, Bluetooth protocol communication, wireless or other wired methods to control the display device 200. The user can input user instructions through buttons on the remote controller, voice input, control panel input, etc. to control the display device 200. In some embodiments, a mobile terminal, a tablet computer, a computer, a laptop computer, and other smart devices can also be used to control the display device 200.

[0038] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and realize connection communication through the network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to realize the synchronous display function. The display device 200 also communicates data with the server 400 through various communication methods. The display device 200 can be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks. The server 400 can provide various content and interactions to the display device 200. The display device 200 can be a liquid crystal display, an OLED display, or a projection display device. In addition to providing a broadcast receiving television function, the display device 200 can also provide an intelligent network TV function that provides a computer support function.

[0039] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive the user's input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, and play the role of an interactive intermediary between the user and the display device 200. The communication interface 130 is used to communicate with the outside, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module. The user input / output interface 140 includes at least one of a microphone, a touchpad, a sensor, a button or an alternative module.

[0040] Figure 3 FIG. 2 shows a hardware configuration block diagram of a display device 200 according to an exemplary embodiment. Figure 3The display device 200 shown includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface 280. The controller includes a central processing unit, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The display 260 can be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and can also be a projection device and a projection screen. The tuner 210 receives broadcast television signals by wired or wireless reception, and demodulates audio and video signals from multiple wireless or wired broadcast television signals, such as and EPG data signals. The detector 230 is used to collect signals of the external environment or interact with the outside. The controller 250 and the tuner 210 can be located in different split devices, that is, the tuner 210 can also be in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0041] In some embodiments, the controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. The user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0042] In some embodiments, "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on a display screen of an electronic device, wherein a control may include at least one of icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0043] Figure 4 FIG. 1 is a schematic diagram of software configuration in a display device 200 according to one or more embodiments of the present application. Figure 4As shown in FIG. 1 , the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer), the application framework layer (Application Framework layer), the Android runtime and system library layer (System runtime library layer), and the kernel layer. The kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0044] Figure 5 FIG. 1 is a schematic diagram showing an icon control interface of an application in a display device 200 according to one or more embodiments of the present application. Figure 5 As shown in , the application layer includes at least one application that can display corresponding icon controls in the display, such as: live TV application icon control, video on demand application icon control, media center application icon control, application center icon control, game application icon control, etc. Live TV application can provide live TV through different signal sources. Video on demand application can provide videos from different storage sources. Unlike live TV application, video on demand provides video display from certain storage sources. Media center application can provide applications for playing various multimedia content. Application center can provide storage for various applications.

[0045] The display device in the present application refers to a terminal device that can output a specific display screen, which can be a mobile phone, smart TV, tablet computer, desktop, laptop, notebook computer, ultra-mobile personal computer (ultra-mobile personal computer, UMPC), handheld computer, netbook, personal digital assistant (personal digital assistant, PDA), wearable electronic device, virtual display device and other terminal devices. The embodiments of the present application do not impose any restrictions on the specific type of terminal device.

[0046] Figure 6 The following is a schematic diagram of a virtual reality application scenario according to an exemplary embodiment. Virtual reality applications can be VR games, VR design, VR live broadcast, etc. The technical solution shown in this application includes but is not limited to being applied to any VR application that requires real-time control of the user's viewing angle. Figure 6As shown, taking VR live broadcast as an example, in the VR live broadcast scene, the display device, server and terminal are connected based on the network; the terminal collects media resources in real time through the VR camera at the VR live broadcast site; and the server provides media resources to the display device. Among them, the virtual reality application provides the viewer version to the display device and the supplier version to the terminal. The viewer version is used to watch the media resources provided by the server; the supplier version is used to watch the media resources collected in real time by the terminal's image acquisition device and control the media viewing angle.

[0047] In some embodiments, live broadcast can also be video data that is preset data and users can adjust the viewing angle to watch, and audio data is live data, which requires multiple display devices to play synchronously. Taking VR design as an example of virtual reality application, multiple display devices, servers and terminals are connected based on the network, and multiple display devices enter the VR design application to allow users to watch the house interior decoration scene. The house interior decoration scene is preset data, and users can adjust the viewing angle on the display device to watch. The director provides audio data in real time on the terminal for explanation, and provides audio data to the display device through the server. When the director enters the corresponding room for explanation, the director can control the video viewing angle so that the user can watch the viewing angle corresponding to the audio data, so as to provide users with a better explanation experience.

[0048] In some embodiments, live broadcast can also be live broadcast data with both audio data and video data, which requires multiple display devices to play synchronously. Taking VR live broadcast as an example, multiple display devices, servers and terminals are connected based on a network. When multiple display devices enter the VR live broadcast application to allow users to watch live conference scenes, the terminal collects video data through an image acquisition device at the VR live broadcast site, and collects audio data through an audio acquisition device. Users can adjust the viewing angle to watch video data on the display device, and the director can control the video viewing angle so that users can watch the video viewing angle corresponding to the audio data, so as to provide users with a better viewing experience.

[0049] It should be noted that the virtual reality applications in this application include but are not limited to web applications built based on the Hyper Text Markup Language 5 (HTML5) format. HTML5 is a language description method for building web page content. There are many existing methods for implementing virtual reality players based on the HTML5 format. For example, a 3D engine (such as Three.js) is used to create a three-dimensional site model, a virtual camera, a spherical model and a web graphics (Web Graphics Library, Web GL) renderer; the picture or video resources are converted into a texture and pasted on the spherical model. The user adjusts the camera position and rotation angle of the virtual camera through the touch screen or mouse to select the display perspective, and the picture or video of the display perspective is rendered through the Web GL renderer to watch the picture or video in 360 degrees. If the display device is equipped with a gyroscope, the user can watch the picture or video in 360 degrees by rotating the display device.

[0050] It should be noted that the virtual camera in this application can also be called a camera model, which is a virtual model used to capture images in the virtual scene of the above-mentioned three-dimensional venue model. The virtual camera has camera information similar to that of a physical camera, such as position, posture, aperture size, focal length, etc. The virtual camera is determined by the camera information of the virtual camera in the virtual scene.

[0051] For ease of expression, a virtual reality player loaded with media assets for playing on a display device is described as a first virtual reality player, a virtual reality player loaded with media assets for playing on a terminal is described as a second virtual reality player, and a virtual reality player loaded with media assets for playing on another display device is described as a third virtual reality player.

[0052] Figure 7 Schematic diagram of a usage scenario of a display device according to an exemplary embodiment is shown. Figure 7 As shown, the user inputs a control instruction for entering a virtual reality application, and in response to the control instruction, a first virtual reality player is acquired, so that the user can watch media through the first virtual reality player. During the process of watching the media, the user can interact with the display device.

[0053] In some embodiments, the display device may include a display unit, which may be used to receive input digital or character information and generate signal input related to user settings and function control of the display device. Specifically, the display unit may include a touch screen arranged on the front of the communication terminal, which may collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc. The display unit may also include a display screen arranged on the front of the display device. Among them, the display screen may be configured in the form of a liquid crystal display, a light emitting diode, etc. Among them, the touch screen may be covered on the display screen, or the touch screen may be integrated with the display screen to realize the input and output functions of the display device, and the integration may be referred to as a touch display screen. The display unit in the present application may be used to display the virtual reality application interface in the present application. Users can adjust the viewing angle of the media asset on the virtual reality application interface through touch operations.

[0054] In some embodiments, the interaction can be performed through a peripheral device, for example, through a mouse or by operating a user interface. The user can adjust the viewing angle of the media asset on the virtual reality application by moving the mouse.

[0055] Figure 8 FIG. 1 is a schematic diagram showing a terminal usage scenario according to an exemplary embodiment. Figure 8 As shown, the terminal uses a real-scene camera to shoot the real-scene site at the live broadcast site, and constructs a three-dimensional site model in the virtual reality application according to the real-scene site to form a virtual reality picture, and the terminal can play the virtual reality picture.

[0056] In the embodiments of the present application, the real scene site refers to the target site photographed by the physical camera. Taking the conference live broadcast as an example, the target site photographed is the live broadcast venue. Among them, one real scene site can correspond to multiple physical cameras. The video images shot by multiple physical cameras are spliced ​​into a live video stream by partitioning or time division to upload to the server. The three-dimensional site model is a three-dimensional virtual model of the real scene site, and the three-dimensional virtual model can be regarded as the result of scaling the real scene site according to a certain ratio. The three-dimensional site model of the real scene site can be generated and stored in advance.

[0057] In the scenario of media playback on display devices, take the example of a user participating in a communication meeting online on a mobile phone through VR live broadcast. Before the meeting starts, the user slides on the touch screen to switch the scene perspective in order to experience the on-site environment of the communication meeting. The user's viewing perspective is controlled by the user. During the audience's speech at the communication meeting, the on-site participants can determine the position of the audience who is speaking on the scene in a timely and accurate manner based on the sense of spatial orientation. However, in online VR live broadcasts, users do not have a sense of spatial orientation on the scene, so they cannot turn the perspective in time to confirm the position of the audience who is speaking / cannot determine the person / object corresponding to the audio data, thereby gaining a better sense of participation.

[0058] In order to help users better determine the focus of the VR live screen / or take the person / object corresponding to the audio data as the focus, the existing way of watching VR live broadcasts usually prompts users the focus position in the VR live screen through director guidance, subtitle prompts, switching playback sources, etc. However, only through the director guidance, subtitle prompts and switching playback sources in the VR live screen, it is impossible to interfere with the user's viewing angle and cannot meet the needs of real-time control of live broadcasts.

[0059] It can be seen that in existing VR live broadcast applications, users are unable to determine important viewing content in a timely manner. Therefore, helping users control the viewing angle in real time can improve the user's viewing experience.

[0060] The technical solution shown in this application can switch the viewing angle of the user on the display device to the central viewing angle determined by the terminal user when the user is using the VR live broadcast application, so as to help the user on the display device to determine the best viewing angle in a timely and accurate manner, thereby gaining a better sense of participation in the VR live broadcast. It should be noted that the technical solution shown in this application includes but is not limited to application in VR live broadcast, and the technical solution shown in this application can be applied to any VR application that requires real-time control of the user's viewing angle.

[0061] To this end, the present application shows a display device, including: a display; a controller, configured to perform the following Fig. 9 The following steps S901-S904 are shown.

[0062] Step S901: receiving a control instruction input by a user for entering a virtual reality application.

[0063] In some embodiments, the user clicks a Uniform Resource Locator (URL) to enter the virtual reality application on the web page. The URL can be a link shared by other users to the display device user, a link pushed by a media provider to the display device user, or a link entered by the display device user in the browser.

[0064] Fig.10 FIG. 1 is a schematic diagram showing a virtual reality application entered through a URL according to an exemplary embodiment. Fig.10 As shown, when a user on a display device clicks on the URL of a VR live broadcast application, the user enters an HTML5-formatted browser page to load the VR live broadcast room.

[0065] In some embodiments, the server creates a live broadcast room for the live broadcast, and the display device can access the live broadcast room after starting the virtual reality application. Synchronously, the first display device, the second display device, and the Nth display device can access the live broadcast room. The terminal controlling the viewing angle can be one of the display devices, and the acquisition terminal can be a part of the terminal, or it can only be used as an acquisition device to upload data to the live broadcast room.

[0066] In some embodiments, when the acquisition end may be a part of the terminal, the terminal not only uploads the data collected by the acquisition end, but also pulls down the virtual reality data from the server.

[0067] In some embodiments, when the acquisition end exists independently, it can only serve as an acquisition device to upload data to the live broadcast room without downloading virtual reality data.

[0068] Step S902, in response to the control instruction, obtaining media assets; wherein the media assets are panoramic perspective media assets.

[0069] In some embodiments, the media assets are panoramic perspective media assets collected by the terminal in real time.

[0070] In some embodiments, the video media in the media assets are preset panoramic view images, and the audio media assets are audio data collected in real time by the collection end.

[0071] Step S903, when the user watches the media at the current viewing angle, the first message sent by the server is received; wherein the first message includes the camera position data and the camera rotation data of the center viewing angle; the center viewing angle is determined by the terminal; the display device, the server and the terminal are connected based on a network; optionally, the display device, the server and the terminal can be connected based on a wired network or a wireless network.

[0072] It should be noted that the central viewing angle in the present application can be a viewing angle determined by a free viewing angle adjustment operation for setting the range of the user viewing the media asset, that is, in the embodiment of the present application, the target viewing angle is not limited to a fixed viewing angle, and can also be a viewing angle freely adjusted by the terminal user. As the host of the live broadcast, the terminal can set the central viewing angle. And by triggering the first message, all viewing display devices can view the content of the same viewing angle.

[0073] In some embodiments, the central viewing angle may be set by the host of the conference, or may be determined based on the acquisition angle of the terminal, with the purpose of presenting people / objects at a specified angle on the interface of the display device.

[0074] In some embodiments, after receiving the first instruction from the conference host, the server sends a first message to each display device, wherein the first instruction is used to control the presentation of a person / object at a specified angle in the live broadcast image on the interface of the display device.

[0075] In some embodiments, the wireless network or wired network described above uses standard communication technology and / or protocols. The network is typically the Internet, but may also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including HyperText Markup Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) may be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.

[0076] The server in this application may be one server, or several servers, or a virtualization platform, or a cloud computing service center.

[0077] It should be noted that the media resources in this application include but are not limited to video images of live video streams, video images of video files, and images of video animations.

[0078] In some embodiments, the server includes: a first network element, a second network element, a third network element, and a fourth network element; the first network element is used to provide a first virtual reality player to a display device; the second network element is used to receive media resources collected by a terminal in real time and provide the media resources to the display device; the third network element is used to send messages between the display device, the server, and the terminal, and the messages are used to transmit camera position data and camera rotation data; the fourth network element is used to control the media resources through the terminal;

[0079] The first network element and the fourth network element in this application are interconnected through the second network element, and the third network element is connected to the first network element, the second network element and the fourth network element respectively. Among them, the first network element in this application is a Web network element; the second network element is a service network element; the third network element is a message network element; and the fourth network element is an Oss network element. The network element is composed of one or more disks or frames, and can independently complete certain transmission functions. In this application, the Web network element is used to load a virtual reality player on a display device and a terminal, the Oss network element is used to control media resources on the terminal, the service network element is an API Server, which is used to provide data interaction between the Web network element and the Oss network element, and the message network element is used to transmit messages between the Web network element, the Oss network element, and the service network element.

[0080] In some embodiments, the live video stream includes audio data. In some embodiments, the audio data and the video data may have different sources.

[0081] In some embodiments, the terminal uploads the locally recorded live video stream to the second network element, and the second network element performs transcoding and other related processing on the live video stream and then sends it to the first virtual reality player for media playback.

[0082] In some embodiments, the video data in the media data is pre-stored on the server or terminal, and the sound data is uploaded in real time by the acquisition terminal. For example, the model introduced in the conference is three-dimensional video data, and each terminal can watch and switch the perspective. The audio is real-time data. When the video data introduces a specified target object, the perspective can be switched so that all participating terminals can synchronously watch the target object in the three-dimensional video data.

[0083] In some embodiments, the controller performs the step of obtaining media resources and is further configured to perform the following steps: Fig.11 Steps S1101-S1104 shown;

[0084] Step S1101: Send a first request instruction for obtaining a first virtual reality player to a first network element; wherein the first network element is a network element in a server that provides the first virtual reality player to a display device;

[0085] Step S1102, loading a first virtual reality player sent by the first network element to the display device in response to the first request instruction;

[0086] Step S1103: When the first virtual reality player is loaded on the display device, a second request instruction for obtaining media resources is sent to the second network element; wherein the second network element is a network element in the server that receives the media resources collected by the terminal in real time and provides the media resources to the display device;

[0087] Step S1104: Control the first virtual reality player to play the media resource sent by the second network element to the display device in response to the second request instruction.

[0088] In some embodiments, the controller performs the step of receiving a first message sent by the server, and is further configured to perform the following steps: Fig.12 Steps S1201-S1203 shown;

[0089] Step S1201: When the user watches the acquired media resources at the current viewing angle, the user receives a first message sent by a third network element through a first virtual reality player; wherein the third network element is a network element in the server that sends messages between the display device, the server, and the terminal; the message is used to transmit camera position data and camera rotation data;

[0090] Step S1202, parsing the camera position data and camera rotation data of the center viewing angle according to the first message;

[0091] Step S1203: Switch the current perspective of the media asset to the central perspective according to the camera position data and the camera rotation data of the central perspective.

[0092] It should be noted that the first message in the present application includes but is not limited to camera position data and camera rotation data, and may also include camera aperture data and camera focal length data. In a specific implementation, the camera position data and camera rotation data in the present application may be the camera position data and camera rotation data of the virtual camera adjusted in the display device of the second user acquisition terminal.

[0093] When the terminal determines the central perspective, the terminal controls the media to switch the current perspective to the first perspective through the fourth network element. The fourth network element obtains the camera position information and camera rotation information of the first perspective, and sends the obtained camera position information and camera rotation information of the first perspective to the second network element. The second network element stores the received camera position information and camera rotation information of the first perspective in the database.

[0094] After the second network element stores the received camera position information and camera rotation information of the first perspective into the database, the second network element sends a first message to the first network element through the third network element. The first message includes the camera position data and camera rotation data of the first perspective for collecting media assets.

[0095] After receiving the first message, the first network element sends the first message to the user terminal.

[0096] In some embodiments, after the controller executes the step of receiving the first message sent by the server, the controller is further configured to:

[0097] The first virtual reality player is controlled to monitor whether a touch event or a mouse movement event is triggered to determine whether the user is currently switching the perspective; if the touch event or the mouse movement event is triggered, it is determined that the user is currently switching the perspective; when it is determined that the user is currently switching the perspective, the monitoring of the touch event or the mouse movement event is paused; if the touch event or the mouse movement event is not triggered, it is determined that the user is not currently switching the perspective.

[0098] Touch events include touchstart events, touchmove events, and touchend events, where touchstart is triggered when the first user touches a Document Object Model (DOM) element; touchmove is triggered when the first user slides the finger on a DOM element; touchend is triggered when the first user moves away from a DOM element. The triggering conditions of touch events in this application can be the triggering of touchmove events, or both touchstart events and touchmove events. The mouse move event (mousemove) triggers the event handler when moving on the page. In this event, the document object can be used to read the position of the mouse on the page in real time.

[0099] It should be noted that touch events and mouse movement events cannot be triggered at the same time.

[0100] In some embodiments, after the controller executes the step of controlling the first virtual reality player to monitor whether a touch event or a mouse movement event is triggered, the controller is further configured to:

[0101] Determine whether the current perspective is the first perspective;

[0102] If the current perspective is not the first perspective, a step of switching the current perspective of the media asset to a center perspective is performed according to the first message.

[0103] In some embodiments, after the controller executes the step of controlling the first virtual reality player to monitor whether a touch event or a mouse movement event is triggered, the controller is further configured to:

[0104] If the current perspective is the first perspective, stop executing the step of switching the current perspective of the media asset to the first perspective according to the camera position data and the camera rotation data of the first perspective.

[0105] Step S904: switch the current perspective of the media asset to the center perspective according to the first message.

[0106] In some embodiments, different display terminals may have independent operations for viewing panoramic video data. When the host of a live conference wants everyone to simultaneously focus on the video content of a certain perspective, the above solution can achieve this goal. This is conducive to enhancing the transmission of conference messages.

[0107] In some embodiments, after the controller executes the step of switching the current perspective of the media asset to the central perspective according to the first message, the controller is further configured to: execute the following steps: Fig.13 Steps S1301-S1303 shown;

[0108] Step S1301, determining whether the current perspective of the media asset is successfully switched to the center perspective;

[0109] Step S1302: If the current viewing angle of the media asset is successfully switched to the center viewing angle, the display is controlled to display a first prompt, where the first prompt is used to prompt the user terminal that the current viewing angle has been switched to the center viewing angle;

[0110] Fig.14 The first prompt schematic diagram according to an exemplary embodiment is exemplarily shown. Exemplarily, the first prompt in the present application may be "The current user perspective has been forced to switch."

[0111] Step S1303: If the current viewing angle of the media asset is not successfully switched to the central viewing angle, the display is controlled to display a second prompt, where the second prompt is used to prompt the user that the current viewing angle needs to be switched to the central viewing angle.

[0112] Fig.15 The second prompt schematic diagram according to an exemplary embodiment is exemplarily shown. For example, the second prompt in the present application may be "Playback error, it is recommended to refresh and try again."

[0113] In some embodiments, after the controller executes the step of switching the current perspective of the media asset to the central perspective according to the first message, the controller is further configured to:

[0114] Re-listen for the touch event or mouse movement event.

[0115] In the process of switching the current perspective to the first perspective, in order to ensure the smoothness of the perspective switching process, it is necessary to pause monitoring of touch events or mouse movement events to avoid conflicts with the technical method shown in this application when the first user switches the viewing perspective through the touch screen or mouse; after pausing monitoring of touch events or mouse movement events, it is necessary to re-monitor touch events or mouse movement events to allow the first user to restore the authority to actively switch perspectives.

[0116] The present application also shows a terminal, comprising: a display; a controller, configured to execute Fig.16Steps S1601-S1605 shown;

[0117] Step S1601, receiving a control instruction input by a user for entering a virtual reality application;

[0118] Step S1602: In response to the control instruction, a third request instruction for obtaining a second virtual reality player is sent to the first network element; wherein the first network element is a network element in the server that provides the second virtual reality player to the display device;

[0119] Step S1603: loading a second virtual reality player sent by the first network element in response to the third request instruction;

[0120] Step S1604, playing the media resources collected in real time by the image collection device through the second virtual reality player;

[0121] Step S1605: When the user determines the central viewing angle of the media resource in the second virtual reality player, a viewing angle switching operation is performed.

[0122] Among them, the terminal can use a built-in image acquisition device to acquire media images in real time or use an external image acquisition device to acquire media images and send them to the controller. At the same time, the terminal can acquire media audio through a built-in audio acquisition device or use an external audio acquisition device to acquire media audio and send them to the controller. In a specific implementation, the image acquisition device can be a physical camera, and the image acquisition device and the audio acquisition device can be part of the display device, for example, the image acquisition device and the audio acquisition device can be a built-in camera and a built-in microphone of the display device; or, the image acquisition device and the audio acquisition device can be external devices connected to the display device, such as a camera and a microphone connected to the display device; or, the image acquisition device and the audio acquisition device can also be partially built into the display device and partially used as external devices connected to the display device, such as the image acquisition device is a built-in camera of the display device, and the audio acquisition device is a microphone in a headset connected to the display device. The embodiment of the present application does not limit the specific implementation form of the image acquisition device and the audio acquisition device.

[0123] In some embodiments, the user interface of the virtual reality application includes: a switch perspective control;

[0124] When the controller performs the steps of switching the view angle, it is further configured as follows:

[0125] When the user clicks the perspective switch control, the perspective switch operation is performed.

[0126] The "user interface (UI)" for virtual reality applications involved in this application is a media interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the terminal device, and finally presented as content that the user can recognize, such as pictures, text, buttons and other controls. Controls, also known as widgets, are basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scroll bars, pictures and text. The properties and contents of controls in the interface are defined by tags or nodes, such as XML through <textview> 、 <imgview> 、 <videoview>The controls contained in the interface are specified by nodes such as <head> and <body>. A node corresponds to a control or attribute in the interface. After being parsed and rendered, the node is presented as user-visible content. In addition, many applications, such as hybrid applications, usually also contain web pages in their interfaces. A web page, also known as a page, can be understood as a special control embedded in the application interface. A web page is source code written in a specific computer language, such as hypertext markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed as user-recognizable content by a browser or a web page display component with similar functions to a browser. The specific content contained in a web page is also defined by tags or nodes in the web page source code. For example, HTML is defined by <body> tags or nodes in the web page source code. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0127] The most common form of user interface is graphical user interface (GUI), which refers to a user interface related to computer operation that is displayed in a graphical manner. It can be an icon, window, control or other interface element displayed on the display screen of an electronic device, where a control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets and other visual interface elements.

[0128] Fig.17 The schematic diagram of the architecture of the terminal according to the exemplary embodiment is shown as an example. Fig.17 As shown, the terminal includes several physical cameras, a renderer, and a virtual reality user interface. The second user controls the virtual camera to browse the perspective of the three-dimensional venue model, and the renderer renders and generates video images and switching animations based on the images captured by the physical camera to display at least one of the perspectives of the virtual camera controlled by the interface.

[0129] based on Fig.17 In the terminal architecture shown, the user controls the rendering engine through the virtual reality application interface, generates media resources based on the images captured by the physical camera, and uploads the media resources to the third network element. When the user wants to determine the center viewing angle, the user freely adjusts the viewing angle of the three-dimensional site model through the touch screen or mouse. After determining the center viewing angle, click the switch viewing angle control.

[0130] The present application also shows a display device, including: a display; a controller, configured to:

[0131] Receiving a control instruction input by a user for entering a virtual reality application;

[0132] In response to the control instruction, sending a fourth request instruction for acquiring a third virtual reality player to the first network element;

[0133] Loading a third virtual reality player sent by the first network element in response to the fourth request instruction;

[0134] When the third virtual reality player is loaded on the display device, a fifth request instruction for obtaining media resources is sent to the third network element;

[0135] Controlling the third virtual reality player to play the media resource sent by the third network element to the display device in response to the fifth request instruction; the media resource has a central viewing angle determined by the terminal;

[0136] When the user watches the media asset at the current viewing angle, the initial viewing angle of the media asset is the central viewing angle.

[0137] Fig.18 A schematic diagram of a virtual reality device application scenario according to an exemplary embodiment is shown as an example. Fig.18 As shown, a user watches VR live broadcast through a virtual reality device. The present application also shows a virtual reality device, which includes:

[0138] monitor;

[0139] Controller;

[0140] The controller is further configured to:

[0141] Receiving a control instruction input by a user for entering a virtual reality application;

[0142] In response to the control instruction, media resources are obtained; the media resources are media resources collected by the terminal in real time;

[0143] When the user watches the media at the current viewing angle, a first message sent by the server is received; the first message includes the camera position data and the camera rotation data of the central viewing angle; the central viewing angle is determined by the terminal; the virtual reality device, the server and the terminal are connected based on a network;

[0144] The current perspective of the media asset is switched to the central perspective according to the first message.

[0145] In some embodiments, the virtual reality device further includes: a posture sensor configured to detect user posture data, wherein the posture data includes a rotation angle of the user's head;

[0146] Determine whether the rotation angle of the user's head is less than a first threshold;

[0147] If the rotation angle of the user's head is less than the first threshold, executing the step of switching the current perspective of the media asset to the center perspective according to the first message;

[0148] If the user's head rotation angle is greater than the first threshold, the display is controlled to display a third prompt, where the third prompt is used to prompt the user that the current head rotation angle needs to be adjusted and the current viewing angle needs to be switched to a center viewing angle.

[0149] It should be noted that the first threshold can be set according to user preferences, which is an angle suitable for users to watch VR live broadcasts. In a specific implementation, if a user sits on a sofa and watches VR live broadcasts, the first threshold can be set to an angle of slightly twisting the head, such as -45° to +45°.

[0150] In the embodiments of the present application, the virtual reality device generally refers to a display device that can be worn on the user's face to provide the user with an immersive experience, including but not limited to VR glasses, augmented reality devices (AR), VR gaming devices, mobile computing devices, and other wearable computers. Some embodiments of the present application use VR glasses as an example to illustrate the technical solution. It should be understood that the provided technical solution can also be applied to other types of virtual reality devices. The virtual reality device can run independently or be connected to other smart display devices as an external device, wherein the display device can be a smart TV, computer, tablet computer, server, etc.

[0151] The present application also shows a method for switching viewing angles, the method comprising:

[0152] Acquire media resources; the media resources are panoramic perspective media resources collected by the terminal in real time,

[0153] When the user watches the media asset at the current viewing angle, a first message sent by the server is received; the first message includes camera position data and camera rotation data of the central viewing angle; the central viewing angle is determined by the terminal; the display device, the server and the terminal are connected based on a network;

[0154] The current perspective of the media asset is switched to the central perspective according to the first message.

[0155] In some embodiments, the present application also shows a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the perspective switching method described in the above embodiments.

[0156] It should be understood that the specific implementation of each step in the above-mentioned method for switching perspectives can refer to the above-mentioned display device embodiment, which will not be described in detail here. It can be seen from the above-mentioned embodiment that the technical solution shown in this application can solve the problem that the viewer's perspective cannot be directly intervened only by the director's guidance, sending message prompts and switching the playback source during the user's use of the virtual reality application. By receiving the control instruction input by the first user for entering the virtual reality application in this application; responding to the control instruction, receiving the first message sent by the server; parsing the camera position data and camera rotation data of the first perspective according to the first message; switching the current perspective of the media asset to the first perspective according to the camera position data and camera rotation data of the first perspective, the user's viewing perspective can be controlled in real time during the user's use of the virtual reality application.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0158] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A display device, characterized in that: include: monitor; The controller is configured as: receiving a control instruction input by a user for entering a virtual reality application; the virtual reality application is loaded with a camera model; In response to the control instruction, media assets are acquired; wherein the media assets are panoramic perspective media assets; When the user watches the media asset at the current viewing angle, a first message sent by the server is received; wherein the first message is a message sent by the server to the display device after receiving the first instruction sent by the terminal; the first instruction is used to control the presentation of a target at a specified angle in the media asset on the interface of the display; the first message includes camera position data and camera rotation data of a central viewing angle; the central viewing angle is a viewing angle determined by the terminal for setting a range for the user to view the media asset; the display device, the server and the terminal are connected based on a network; the terminal is located at a real-life shooting site for the media asset; the camera position data is used to adjust the camera position of the camera model, and the camera rotation data is used to adjust the rotation angle of the camera model; The current perspective of the media asset is switched to the central perspective according to the first message.

2. The display device according to claim 1, characterized in that The controller performs the step of acquiring media resources and is further configured to: Sending a first request instruction for obtaining a first virtual reality player to a first network element; wherein the first network element is a network element in the server that provides the first virtual reality player to the display device; Loading a first virtual reality player sent by the first network element to the display device in response to the first request instruction; When the first virtual reality player is loaded on the display device, a second request instruction for obtaining media resources is sent to a second network element; wherein the second network element is a network element in the server that receives the media resources collected by the terminal in real time and provides the media resources to the display device; Control the first virtual reality player to play the media asset sent by the second network element to the display device in response to the second request instruction.

3. The display device according to claim 2, characterized in that The controller performs the step of receiving a first message sent by the server, and is further configured to: When the user watches the acquired media resource at the current viewing angle, the first message sent by the third network element is received through the first virtual reality player; wherein the third network element is a network element in the server that sends messages between the display device, the server, and the terminal; and the message is used to transmit the camera position data and the camera rotation data; Parsing the camera position data and the camera rotation data of the central viewing angle according to the first message; The current perspective of the media asset is switched to the central perspective according to the camera position data and the camera rotation data of the central perspective.

4. The display device according to claim 1, characterized in that After the controller executes the step of switching the current perspective of the media asset to the central perspective according to the first message, the controller is further configured to: Determining whether the current perspective of the media asset is successfully switched to the central perspective; If the current viewing angle of the media asset is successfully switched to the central viewing angle, controlling the display to display a first prompt, where the first prompt is used to prompt the user terminal that the current viewing angle has been switched to the central viewing angle; If the current viewing angle of the media asset is not successfully switched to the central viewing angle, the display is controlled to display a second prompt, where the second prompt is used to prompt the user that the current viewing angle needs to be switched to the central viewing angle.

5. The display device according to claim 2, characterized in that After the controller executes the step of receiving the first message sent by the server, the controller is further configured to: Controlling the first virtual reality player to monitor whether a touch event or a mouse movement event is triggered, so as to determine whether the user is currently switching the viewing angle; If the touch event or mouse movement event is triggered, determine that the user is currently switching perspectives; When it is determined that the user is currently switching the viewing angle, suspending monitoring of the touch event or the mouse movement event; If the touch event or the mouse movement event is not triggered, it is determined that the user is not currently switching the viewing angle.

6. The display device according to claim 5, characterized in that After the controller executes the step of controlling the first virtual reality player to monitor whether a touch event or a mouse movement event is triggered, the controller is further configured to: Determining whether the current viewing angle is a first viewing angle; If the current perspective is not the first perspective, the step of switching the current perspective of the media asset to the center perspective according to the first message is performed.

7. The display device according to claim 6, characterized in that After the controller executes the step of switching the current perspective of the media asset to the central perspective according to the first message, the controller is further configured to: Re-listen for the touch event or mouse movement event.

8. A display device, characterized in that: include: monitor; The controller is configured as: Receiving a control instruction input by a user for entering a virtual reality application; In response to the control instruction, a fourth request instruction for obtaining a third virtual reality player is sent to the first network element; wherein the first network element is a network element in the server that provides the third virtual reality player to the display device; Loading a third virtual reality player sent by the first network element in response to the fourth request instruction; When the third virtual reality player is loaded on the display device, a fifth request instruction for obtaining media resources is sent to the second network element; wherein the second network element is a network element in the server that receives the media resources collected by the terminal in real time and provides the media resources to the display device; Controlling the third virtual reality player to play the media asset sent by the second network element to the display device in response to the fifth request instruction; the media asset has a central viewing angle determined by the terminal; the central viewing angle is a viewing angle used to set a range for a user to watch the media asset, so that a target at a specified angle in the media asset is presented on the interface of the display; When the user watches the media asset at the current viewing angle, the initial viewing angle of the media asset is the central viewing angle.

9. A method for switching viewing angles, characterized in that: The method comprises: Acquire media assets, where the media assets are panoramic perspective media assets; When the user watches the media asset at the current viewing angle, a first message sent by the server is received; wherein the viewing angle of the user for viewing the media asset is determined by the camera model loaded in the application for viewing the media asset; the first message is a message sent by the server to the display device after receiving the first instruction sent by the terminal; the first instruction is used to control the target of the specified angle in the media asset to be presented on the interface of the display of the display device; the first message includes camera position data and camera rotation data of the center viewing angle; the center viewing angle is a viewing angle determined by the terminal for setting a specified range for the user to view the media asset; the display device, the server and the terminal are connected based on a network; the terminal is located at the real-life shooting site of the media asset; the camera position data is used to adjust the camera position of the camera model, and the camera rotation data is used to adjust the rotation angle of the camera model; The current perspective of the media asset is switched to the central perspective according to the first message.

Images