A virtual reality device and a method for playing screen-cast media assets

Abstract

The present application provides a virtual reality device and a method for playing screen-cast media assets. The method can receive screen-cast data after receiving a control instruction from a user, and extract screen-cast media asset information from the screen-cast data. Then, the screen-cast media asset information is used to obtain source addresses of video sources in different playback modes in a server, so as to obtain media data in a specified playback mode by accessing the source addresses and play the screen-cast picture. The method can obtain media data in a specified playback mode while displaying the screen-cast data, so as to play the media content cast from a smart terminal in a 3D or panoramic mode, and solve the problem that the playback form is single and it is not conducive to obtaining an immersive effect.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, and in particular, to a virtual reality device and a method for playing screen-cast media resources. Background Art

[0002] Virtual Reality (VR) technology is a display technology that simulates a virtual environment through a computer, thus giving people a sense of immersion in the environment. A virtual reality device is a device that uses virtual display technology to present a virtual picture to a user. Usually, a virtual reality device includes two display screens for presenting virtual picture content, corresponding to the user's left and right eyes respectively. When the content displayed on the two display screens comes from images of the same object from different perspectives, a stereoscopic viewing experience can be brought to the user.

[0003] A virtual reality device can be used as a screen-cast receiver (sink) to play the screen-cast picture sent by a smart terminal (source). During the screen-casting process, the smart terminal can send the screen-cast picture to the virtual reality device through a screen-casting protocol, and the virtual reality device then renders the screen-cast picture, so as to display the picture content in a specific interface.

[0004] A virtual reality device can play multimedia resources of various source types, for example, 2D source, 3D source, and panoramic source, etc. However, the screen-cast pictures sent by smart terminals such as mobile phones are usually presented in 2D form. Therefore, the virtual reality device can only present the screen-cast pictures in 2D form, which is not only not conducive to rendering an immersive effect, but also easily causes a single viewing form and reduces the user experience. Summary of the Invention

[0005] This application provides a virtual reality device and a method for playing screen-cast media resources to solve the problem that the traditional playing method has a single playing form and is not conducive to obtaining an immersive effect.

[0006] On the one hand, this application provides a virtual reality device, including: a display, a communicator, and a controller, where the display is configured to display a playing interface and other user interfaces; the communicator is configured to establish a screen-cast connection with a smart terminal; the controller is configured to execute the following program steps:

[0007] Obtain a control instruction input by a user for establishing a screen-cast connection;

[0008] In response to the control instruction, receive screen-cast data sent by the smart terminal, where the screen-cast data includes screen-cast media information;

[0009] According to the screen-cast media information, obtain a source address adapted to at least one playing mode from a server;

[0010] Access the source address of the video to control the display to display the media screen corresponding to the source address.

[0011] On the other hand, the present application also provides a method for playing screen-cast media, which is applied to the above virtual reality device. The method for playing screen-cast media includes the following steps:

[0012] Obtain a control instruction input by the user for establishing a screen-cast connection;

[0013] In response to the control instruction, receive the screen-cast data sent by the smart terminal, where the screen-cast data includes screen-cast media information;

[0014] According to the screen-cast media information, obtain a source address adapted to at least one playback mode from the server;

[0015] Access the source address to control the display to display the media screen corresponding to the source address.

[0016] As can be seen from the above technical solutions, the virtual reality device and the method for playing screen-cast media provided by the present application can, after receiving the user's control instruction, receive the screen-cast data, and extract the screen-cast media information from the screen-cast data. Then, use the screen-cast media information to obtain the source address in different playback modes in the server, so as to obtain the media data in the specified playback mode by accessing the source address and play the screen-cast screen. The method can obtain the media data in the specified playback mode while displaying the screen-cast data, so as to play the media content cast from the smart terminal in a 3D or panoramic mode, and solve the problem that the playback form is single and it is not conducive to obtaining an immersive effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

[0018] Figure 1 It is a schematic structural diagram of a display system including a virtual reality device in some exemplary embodiments of the present application;

[0019] Figure 2 It is a schematic diagram of a VR scene global interface in some exemplary embodiments of the present application;

[0020] Figure 3 It is a schematic diagram of the recommended content area of the global interface in some exemplary embodiments of the present application;

[0021] Figure 4 It is a schematic diagram of the application quick operation entry area of the global interface in some exemplary embodiments of the present application;

[0022] Figure 5 Schematic diagram of the suspended matter on the global interface in some exemplary embodiments of the present application;

[0023] Figure 6 Schematic diagram of the playback interface in some exemplary embodiments of the present application;

[0024] Figure 7 Schematic diagram of the regional division of the playback interface in some exemplary embodiments of the present application;

[0025] Figure 8 Schematic diagram of the mode switching operation interface in some exemplary embodiments of the present application;

[0026] Figure 9 Schematic diagram of the process of the screen mirroring media playback method in some exemplary embodiments of the present application;

[0027] Figure 10 Schematic diagram of the process of the screen mirroring interaction operation in some exemplary embodiments of the present application;

[0028] Figure 11 Schematic diagram of the process of playing the screen mirroring picture according to the playback mode in some exemplary embodiments of the present application;

[0029] Figure 12 Schematic diagram of the process of playing the screen mirroring picture based on the rendering scene in some exemplary embodiments of the present application;

[0030] Figure 13 Schematic diagram of the process of creating a database using the MyBatis framework in some exemplary embodiments of the present application;

[0031] Figure 14 Schematic diagram of the interaction process based on the switching interface in some exemplary embodiments of the present application;

[0032] Figure 15 Schematic diagram of the window switching when a 3D video source is detected in some exemplary embodiments of the present application;

[0033] Figure 16 Schematic diagram of the window switching when a 360 video source is detected in some exemplary embodiments of the present application;

[0034] Figure 17 Schematic diagram of the window switching with a preview picture in some exemplary embodiments of the present application;

[0035] Figure 18 Schematic diagram of the process of the screen mirroring media playback method in some exemplary embodiments of the present application. Detailed implementation manners

[0036] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the following will describe the technical solutions in the exemplary embodiments of this application clearly and completely in conjunction with the accompanying drawings in the exemplary embodiments of this application. Obviously, the described exemplary embodiments are only a part of the embodiments of this application, rather than all of the embodiments.

[0037] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application. In addition, although the disclosed content in this application is introduced according to one or several exemplary instances, it should be understood that each aspect of these disclosed contents can also constitute a complete technical solution alone.

[0038] It should be understood that the terms "first", "second", "third", etc. in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances. For example, it can be implemented in an order other than those given in the illustrations or descriptions of the embodiments of this application.

[0039] In addition, the terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not necessarily have to be limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0040] The term "module" used in this application refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code that can perform functions related to that element.

[0041] References throughout this specification to "multiple embodiments", "some embodiments", "one embodiment", or "an embodiment", etc., mean that the specific features, structures, or characteristics described in connection with that embodiment are included in at least one embodiment. Therefore, the phrases "in multiple embodiments", "in some exemplary implementations", "in at least another embodiment", or "in an embodiment", etc., that appear throughout this specification do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, the specific features, structures, or characteristics can be combined in any suitable manner. Therefore, without limitation, the specific features, structures, or characteristics shown or described in connection with one embodiment can be combined in whole or in part with the features, structures, or characteristics of one or more other embodiments. Such modifications and variations are intended to be included within the scope of this application.

[0042] In some exemplary embodiments of the present application, the virtual reality device 500 generally refers to a display device that can be worn on the user's face and provide an immersive experience for the user, including but not limited to VR glasses, augmented reality devices (AR), VR game devices, mobile computing devices, and other wearable computers, etc. In some embodiments of the present application, the technical solution is described by taking VR glasses as an example. It should be understood that the provided technical solution can also be applied to other types of virtual reality devices. The virtual reality device 500 can operate independently or be connected as an external device to other intelligent display devices, where the display device can be an intelligent TV, a computer, a tablet computer, a server, etc.

[0043] After being worn on the user's face, the virtual reality device 500 can display media content images, providing close-range images for the user's eyes to bring an immersive experience. To present the media content images, the virtual reality device 500 may include multiple components for displaying images and for facial wearing. Taking VR glasses as an example, the virtual reality device 500 may include a housing, a position fixing member, an optical system, a display component, an attitude detection circuit, an interface circuit, and other components. In practical applications, the optical system, the display component, the attitude detection circuit, and the interface circuit may be arranged inside the housing for presenting specific display images; the two sides of the housing are connected to the position fixing member to be worn on the user's face.

[0044] During use, attitude detection elements such as a gravity acceleration sensor and a gyroscope are built into the attitude detection circuit. When the user's head moves or rotates, the user's attitude can be detected, and the detected attitude data is transmitted to a processing element such as a controller, so that the processing element can adjust the specific image content in the display component according to the detected attitude data.

[0045] As Figure 1 shown, in some exemplary embodiments, the virtual reality device 500 shown can be connected to the display device 200, and a network-based display system can be constructed between the virtual reality device 500, the display device 200, and the server 400. Data interaction can be carried out in real time among the virtual reality device 500, the display device 200, and the server 400. For example, the display device 200 can obtain media data from the server 400 for playback and transmit the specific image content to the virtual reality device 500 for display.

[0046] Among them, the display device 200 can be a liquid crystal display, an OLED display, or a projection display device. The specific type, size, and resolution of the display device are not limited. Those skilled in the art can understand that the display device 200 can be changed in terms of performance and configuration as needed. The display device 200 can provide a broadcast receiving television function and can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0047] The display device 200 and the virtual reality device 500 also communicate with the server 400 through various communication methods for data communication. It is allowed that the display device 200 and the virtual reality device 500 communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various contents and interactions to the display device 200. Exemplarily, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 can be a cluster or multiple clusters and can include one or more types of servers. Other network service contents such as video-on-demand and advertising services are provided through the server 400.

[0048] During the data interaction process, the user can operate the display device 200 through the mobile terminal 300 and the remote control 100. The mobile terminal 300 and the remote control 100 can communicate with the display device 200 in a direct wireless connection manner or in a non-direct connection manner. That is, in some exemplary embodiments, the mobile terminal 300 and the remote control 100 can communicate with the display device 200 through direct connection methods such as Bluetooth and infrared. When sending a control instruction, the mobile terminal 300 and the remote control 100 can directly send the control instruction data to the display device 200 through Bluetooth or infrared.

[0049] In some other embodiments, the mobile terminal 300 and the remote control 100 can also access the same wireless network with the display device 200 through a wireless router to establish non-direct connection communication with the display device 200 through the wireless network. When sending a control instruction, the mobile terminal 300 and the remote control 100 can first send the control instruction data to the wireless router, and then the wireless router forwards the control instruction data to the display device 200.

[0050] In some exemplary embodiments, the user can also directly interact with the virtual reality device 500 using the mobile terminal 300 and the remote control 100. For example, the mobile terminal 300 and the remote control 100 can be used as handles in a virtual reality scenario to achieve functions such as somatosensory interaction.

[0051] In some exemplary embodiments, the display component of the virtual reality device 500 includes a display screen and a driving circuit related to the display screen. To present specific images and bring a stereoscopic effect, the display component may include two display screens, corresponding to the user's left eye and right eye respectively. When presenting a 3D effect, the image content displayed on the left and right screens will be slightly different, and the left and right cameras during the shooting of the 3D video source can be displayed respectively. Due to the image content observed by the user's left and right eyes, a display image with a strong sense of three-dimensionality can be observed when wearing the device.

[0052] The optical system in the virtual reality device 500 is an optical module composed of multiple lenses. The optical system is arranged between the user's eyes and the display screen. By refracting the optical signal through the lenses and the polarization effect of the polarizers on the lenses, the optical path can be increased, so that the content presented by the display component can be clearly presented within the user's field of view. At the same time, in order to adapt to the vision conditions of different users, the optical system also supports focusing, that is, by adjusting the position of one or more of the multiple lenses through a focusing component, changing the mutual distance between the multiple lenses, thereby changing the optical path and adjusting the image clarity.

[0053] The interface circuit of the virtual reality device 500 can be used to transmit interaction data. In addition to transmitting the above-mentioned attitude data and display content data, in practical applications, the virtual reality device 500 can also be connected to other display devices or peripherals through the interface circuit to perform data interaction with the connected devices to achieve more complex functions. For example, the virtual reality device 500 can be connected to a display device through the interface circuit, so as to output the displayed image to the display device for display in real time. For another example, the virtual reality device 500 can also be connected to a handle through the interface circuit, and the handle can be held and operated by the user to perform related operations in the VR user interface.

[0054] Among them, the VR user interface can be presented in various different types of UI layouts according to user operations. For example, the user interface may include a global interface. The global UI after the startup of the AR / VR terminal is as Figure 2 shown. The global UI can be displayed on the display screen of the AR / VR terminal or on the display of the display device. The global UI may include a recommended content area 1, a business classification expansion area 2, an application quick operation entry area 3, and a floating object area 4.

[0055] The recommended content area 1 is used to configure different classification TAB columns; different types of media assets, special topics, etc. can be selected and configured in the columns; the media assets may include 2D movies, educational courses, tourism, 3D, 360-degree panoramas, live broadcasts, 4K movies, program applications, games, tourism and other services with media asset content, and different template styles can be selected for the columns, and it supports the simultaneous recommendation and arrangement of media assets and special topics, asFigure 3 as shown

[0056] In some exemplary embodiments, the content recommendation area 1 may further include a main interface and a secondary interface. As Figure 3 shown, the part located at the center of the UI layout is the main interface, while the parts located on both sides of the main interface are the secondary interfaces. The main interface and the secondary interfaces can be used to display different recommended contents respectively. For example, according to the type of the recommended video source, the business of 3D video sources can be displayed on the main interface; while the business of 2D video sources can be displayed on the left secondary interface, and the business of panoramic video sources can be displayed on the right secondary interface.

[0057] Obviously, for the main interface and the secondary interfaces, while displaying different business contents, they can be presented in different content layouts. And, the user can control the switching between the main interface and the secondary interfaces through specific interaction actions. For example, by controlling the left - right movement of the focus flag, when the focus flag is at the far right of the main interface and then moves further to the right, the right secondary interface can be controlled to be displayed at the center of the UI layout. At this time, the main interface switches to display the business of panoramic video sources, the left secondary interface switches to display the business of 3D video sources; and the right secondary interface switches to display the business of 2D video sources.

[0058] In addition, for the convenience of the user's viewing, the main interface and the secondary interfaces can also be displayed with different display effects respectively. For example, the transparency of the secondary interface can be increased to make the secondary interface obtain a blurred effect, thereby highlighting the main interface. The secondary interface can also be set to a grayscale effect, while keeping the main interface in a color effect, to highlight the main interface.

[0059] In some exemplary embodiments, a status bar may also be set at the top of the recommended content area 1. Multiple display controls can be set in the status bar, including common options such as time, network connection status, battery level, etc. The content included in the status bar can be customized by the user. For example, contents such as weather, user avatar, etc. can be added. The content included in the status bar can be selected by the user to execute corresponding functions. For example, when the user clicks on the time option, the virtual reality device 500 can display a time device window in the current interface, or jump to the calendar interface. When the user clicks on the network connection status option, the virtual reality device 500 can display a WiFi list in the current interface, or jump to the network settings interface.

[0060] The content displayed in the status bar can be presented in different content forms according to the setting status of the specific project. For example, the time control can be directly displayed as specific time text information and display different texts at different times; the battery control can be displayed as different pattern styles according to the remaining battery level of the virtual reality device 500.

[0061] The status bar is used to enable the user to perform commonly used control operations and quickly set up the virtual reality device 500. Since the setting program for the virtual reality device 500 includes many items, it is usually not possible to display all commonly used setting options in the status bar. To this end, in some exemplary embodiments, an extended option may also be provided in the status bar. After the extended option is selected, an extended window may be presented in the current interface, and a plurality of setting options may be further provided in the extended window to implement other functions of the virtual reality device 500.

[0062] For example, in some exemplary embodiments, after the extension option is selected, a "Quick Center" option may be set in the extension window. After the user clicks the Quick Center option, the virtual reality device 500 may display the Quick Center window. The Quick Center window may include "Screenshot", "Screen Recording" and "Screen Casting" options for waking up corresponding functions respectively.

[0063] The business category extension area 2 supports the configuration of extended categories of different categories. If there is a new business type, it supports the configuration of an independent TAB to display the corresponding page content. The extended categories in the business category extension area 2 can also be sorted and adjusted and offline business operations can be performed. In some exemplary embodiments, the business category extension area 2 may include the following content: film and television, education, travel, applications, and mine. In some exemplary embodiments, the business category extension area 2 is configured to display a large business category TAB, and supports the configuration of more categories, and its icons support configuration, such as Figure 3 shown.

[0064] The application quick operation entry area 3 can specify pre-installed applications to be displayed in front for operation recommendation, and supports configuration of special icon styles to replace default icons. The pre-installed applications can be specified as multiple. In some exemplary embodiments, the application quick operation entry area 3 also includes a left-moving control and a right-moving control for moving the option target, which are used to select different icons, such as Figure 4 shown.

[0065] The floating object area 4 can be configured to be located at the upper left side or the upper right side of the fixed area, and can be configured as a replaceable image or a jump link. For example, after receiving a confirmation operation, the floating object jumps to a certain application or displays a specified function page, such as Figure 5 In some exemplary embodiments, the suspended object may not be configured with a jump link and is simply used for image display.

[0066] In some exemplary embodiments, the global UI further includes a status bar at the top for displaying time, network connection status, battery status, and more quick operation entrances. After selecting an icon using the handle of the AR / VR terminal, i.e., the handheld controller, the icon will display text prompts that expand left and right, and the selected icon will stretch and expand left and right according to its position for display.

[0067] For example, after selecting the search icon, the search icon will display the text "Search" and the original icon. After further clicking on the icon or the text, the search icon will jump to the search page; for another example, clicking on the favorite icon will jump to the favorite TAB, clicking on the history icon will default to locate and display the history page, clicking on the search icon will jump to the global search page, and clicking on the message icon will jump to the message page.

[0068] In some exemplary embodiments, interactions can be performed through peripherals. For example, the handle of the AR / VR terminal can operate on the user interface of the AR / VR terminal, including a back button; a home key, and long pressing it can achieve a reset function; volume up and down buttons; a touch area, and the touch area can achieve functions such as clicking, swiping, and holding and dragging of the focus.

[0069] Users can enter different scene interfaces through the global interface. For example, as Figure 6 , Figure 7 shown, users can enter the playback interface through the "Playback Interface" entry in the global interface, or start the playback interface by selecting any media asset in the global interface. In the playback interface, the virtual reality device 500 can create a 3D scene through the Unity 3D engine and render specific picture content in the 3D scene.

[0070] In the playback interface, users can watch specific media asset content. To obtain a better viewing experience, different virtual scene controls can also be set in the playback interface to cooperate with the media asset content to present specific scenes or implement interactions. For example, in the playback interface, panel controls can be loaded in the Unity 3D scene to present picture content, and then combined with other home virtual controls to simulate the effect of a cinema screen.

[0071] The virtual reality device 500 can display operation UI content in the playback interface. For example, in front of the display panel in the Unity 3D scene, a media asset list UI control can also be displayed. In the media asset list, the media asset icons stored locally in the current virtual reality device 500 can be displayed, or the network media asset icons that can be played in the virtual reality device 500 can be displayed. Users can select any icon in the media asset list and play the media asset data corresponding to the icon, and then the selected media asset can be displayed in real time on the display panel.

[0072] The media assets that can be displayed in the Unity 3D scene can be in various forms such as pictures, videos, etc. Moreover, due to the display characteristics of the VR scene, the media assets displayed in the Unity 3D scene at least include 2D pictures or videos, 3D pictures or videos, panoramic pictures or videos, etc.

[0073] Among them, the 2D picture or video is a traditional picture or video file. When being displayed, the same image can be shown on the two display screens of the virtual reality device 500. In this application, the 2D picture or video is collectively referred to as the 2D source; the 3D picture or video, that is, the 3D source, is made by at least two cameras shooting the same object at different angles and can display different images on the two displays of the virtual reality device 500 to achieve a stereoscopic effect; the panoramic picture or video, that is, the panoramic source, is a panoramic image obtained by a panoramic camera or special shooting means and can be displayed by creating a display sphere in the Unity3D scene to present a panoramic effect.

[0074] For the 3D source, it can be further divided into left - right type 3D source, up - down type 3D source, etc. according to the layout of each frame of the image in the source. Among them, each frame of the left - right type 3D source includes two parts, the left and right, which are the image frames shot by the left - eye camera and the right - eye camera respectively. For the panoramic source, it can be further divided into 360 - degree panoramic, 180 - degree panoramic, fisheye panoramic and other source forms according to the field of view of the panoramic source. The synthesis method of each frame of the image is different in different panoramic source forms. In order to present a better stereoscopic effect, the panoramic source can also include true - panoramic type, left - right type panoramic, up - down type panoramic and other forms.

[0075] Since the media asset data that can be displayed in the playback interface includes various source types and different source types require different image output methods, there can also be UI controls for playback control in the playback interface. For example, the UI control for playback control can be set in front of the display panel. This UI control is a floating interactive UI control, that is, it can be triggered to display according to a specific trigger action. As Figure 8 shown, in the UI control, there can be a "mode switch" option. When the user clicks the "mode switch" option, a mode list can be displayed, including mode options such as "2D mode", "3D mode", etc. After the user selects any mode option in the mode list, the virtual reality device can be controlled to play the media asset data in the media asset list according to the selected mode.

[0076] Similarly, when the user selects a media asset item in the list, the playback interface can play the media asset item, that is, display the corresponding picture of the media asset item on the display panel. During the playback of the media asset item, the user can also call the UI control for playback control and select any mode option in the UI control interface to switch the playback mode, so that the virtual reality device 500 can play the selected media data according to the switched playback mode.

[0077] To adapt to different playback modes, in some exemplary embodiments, a media asset item corresponds to multiple data forms. For example, some media asset items include both 2D-formatted media data and 3D-formatted media data. That is, a media asset item corresponds to two media files, where each frame of the image in one media file contains only one specific picture, and each frame of the image in the other media file contains two specific pictures on the left and right (or up and down). For such media asset items, during playback, one of the media files can be selected for playback according to different playback modes to obtain different effects.

[0078] The virtual reality device 500 can also establish a screen mirroring connection with the smart terminal to play the video data sent from the smart terminal through the virtual reality device 500. Among them, the smart terminal that establishes a screen mirroring connection with the virtual reality device 500 can be a device with media asset playback and picture processing functions, including but not limited to mobile phones, smart TVs (display device 200), computers, tablets, etc. After establishing the screen mirroring connection relationship, the smart terminal can send screen mirroring data to the virtual reality device 500. After receiving the screen mirroring data, the virtual reality device 500 can render the screen mirroring data in the rendering scene and output the left-eye image frame and the right-eye image frame for display on the display.

[0079] According to different screen mirroring methods, the forms of the screen mirroring data sent by the smart terminal to the virtual reality device 500 are also different. For example, for the virtual reality device 500 and the smart terminal that support the Digital Living Network Alliance (DLNA) protocol, after establishing the screen mirroring connection, the smart terminal can send the Uniform Resource Locator (URL) address of the currently played media asset to the virtual reality device 500, so that after receiving the screen mirroring data, the virtual reality device 500 can obtain the media asset data by accessing the media asset URL address. The smart terminal can also directly send the video data stream to the virtual reality device 500, that is, the smart terminal can send the played media file or the currently displayed picture to the virtual reality device 500 in the form of a video data stream. The virtual reality device 500 then renders the received video data stream and displays it on the display.

[0080] Since intelligent terminals such as mobile phones can usually only present a 2D display screen, the media data sent to the virtual reality device 500 through screen mirroring connection is generally in the form of a 2D video source. However, the virtual reality device 500 can support various video source forms such as 3D and panoramic, resulting in not only the problem of a single video source but also low clarity during the screen mirroring process, and a better immersive effect cannot be obtained. Therefore, in order to adapt to the screen mirroring process, in some embodiments of the present application, a virtual reality device 500 is provided, including: a display, a communicator, and a controller. Among them, the display includes a left display and a right display, which are used to present a playback interface and other user interfaces. The communicator can achieve data communication based on a wired / wireless network connection method and is used to establish a screen mirroring connection with an intelligent terminal. As Figure 9 shown, the controller is further configured to execute the following program steps:

[0081] S1: Obtain a control instruction input by the user for establishing a screen mirroring connection.

[0082] In practical applications, the virtual reality device 500 can receive various control instructions input by the user. Different control instructions can implement different functions and correspond to different interaction actions. The control instruction input by the user for establishing a screen mirroring connection can be completed through the virtual reality device 500 or through the intelligent terminal.

[0083] For example, as Figure 9 shown, the user can control the virtual reality device 500 to access a specified wireless local area network and also connect the mobile phone to the same wireless local area network. Then, through the mobile phone, perform an interaction operation and click the screen mirroring function button on the operation interface. At this time, the mobile phone will present a list of screen mirroring devices, and the user can select the current virtual reality device 500 in the list of screen mirroring devices to establish a screen mirroring connection relationship. After the above interaction action process, the intelligent terminal can send a control instruction for establishing a screen mirroring connection function to the virtual reality device 500.

[0084] For the virtual reality device 500 that supports other interaction methods, the input of the control instruction can also be completed through the supported input methods. For example, for a virtual reality device 500 or an intelligent terminal that supports an intelligent voice system, the user can also control the intelligent terminal and the virtual reality device 500 to establish a screen mirroring connection relationship by inputting voice content such as "establish screen mirroring" or "I want to screen mirror".

[0085] S2: In response to the control instruction, receive the screen mirroring data sent by the intelligent terminal.

[0086] After establishing a screen mirroring connection relationship, the smart terminal can send screen mirroring data to the virtual reality device 500, and the virtual reality device 500 can receive the screen mirroring data sent by the smart terminal. Since the screen mirroring data sent by different screen mirroring connection methods is different, the virtual reality device 500 can adopt different data parsing methods for different screen mirroring connection methods to parse out the screen mirroring media information from the screen mirroring data. Among them, the screen mirroring media information refers to the media data related information corresponding to the screen mirroring data, including content such as media name and source type.

[0087] For example, the user performs an interactive operation on the mobile phone to play a movie resource online. The name of the movie resource is "A". Since it is currently played through the mobile phone, the source type is a 2D source. During the process of playing the movie resource on the mobile phone, the user clicks the "Screen Mirroring" button on the playback interface and selects the current virtual reality device 500 from the pop-up list of screen mirroring devices, thereby sending screen mirroring data to the current virtual reality device 500. The virtual reality device 500 can then parse out the movie resource name "A" and the source type as 2D from the screen mirroring data.

[0088] For different screen mirroring connection methods, the virtual reality device 500 may need to adopt different methods to parse out the screen mirroring resource information from the screen mirroring data. For example, for a smart terminal that sends screen mirroring data using the DLNA protocol, the URL address of the movie "A" can be parsed out from the screen mirroring data. The virtual reality device 500 then accesses this URL address to obtain the corresponding media information and extracts content such as the movie name and source type from the media information.

[0089] S3: According to the screen mirroring media information, obtain the source address adapted to at least one playback mode from the server.

[0090] After the virtual reality device 500 parses out the screen mirroring media information in the screen mirroring data, it can use the screen mirroring media information to perform a match in the database of the server. After a match hits any entry in the database, it can obtain the source address in different playback modes. For example, after parsing and obtaining that the movie name is "A" and the source type is 2D, use this movie name as the index value to match the sources of other types except the 2D source in the database. Through the match, if it is determined that there are also 3D format and / or panoramic format sources for the currently screen mirrored movie "A", the media link address in the 3D source format or panoramic source format can be obtained from the database so that the virtual reality device 500 can play the 3D source or panoramic source of the same movie name.

[0091] Among them, the database is a relational table constructed according to a specific mapping relationship framework. In the database, the name, source type of each media asset, and the media asset data storage address corresponding to each source type can be stored under the specified media asset platform. The database can be maintained by the virtual reality device 500. For example, the virtual reality device 500 can construct a mapping relationship table by reading the media asset data and its corresponding media asset data storage address in the specified media asset platform, and add the mapping relationship table to the database according to the mapping relationship framework corresponding to the database. The database can also be uniformly maintained by the operator of the virtual reality device 500. For example, the operator reads the platform media asset data and its corresponding storage addresses of each source through the server, generates a mapping relationship table, and adds the generated mapping relationship table to the database. When the virtual reality device 500 accesses the media asset selection interface, the database can be synchronized through the background program.

[0092] In some exemplary embodiments, the database can also be jointly maintained by the virtual reality device 500 and the operator. For example, the virtual reality device 500 can save the different source data of some media assets in a local storage manner. Therefore, the virtual reality device 500 can obtain the database maintained by the operator through the server, and when saving the local media asset data in the virtual reality device 500, construct a mapping relationship table according to the mapping relationship framework corresponding to the database, and store the mapping relationship table in the database to complete the matching through the database after obtaining the screen mirroring media asset information.

[0093] In some exemplary embodiments, the virtual reality device 500 can generate a source acquisition request according to the screen mirroring media asset information; and then send the source acquisition request to the cloud server, so that the cloud server can respond to the source acquisition request and feedback the source address, that is, the virtual reality device 500 can receive the source address adapted to at least one playback mode.

[0094] S4: Access the source address to control the display to display the media asset screen corresponding to the source address.

[0095] After matching and obtaining the source addresses adapted to different playback modes in the server, the virtual reality device 500 can obtain the media asset data corresponding to the specified playback mode by accessing the obtained source addresses, and after rendering according to the corresponding rendering method, form a specific display screen.

[0096] For example, if the virtual reality device 500 obtains through matching that the source address of the 3D source type of the movie "A" is recorded in the database of the server, it can obtain the 3D version of the movie "A" by accessing this source address, so that the virtual reality device 500 can play the movie "A" according to the playback mode of the 3D source.

[0097] It can be seen that in the above embodiments, after receiving the user's control instruction, the virtual reality device 500 can extract the screen mirroring media resource information from the screen mirroring data. Then, the screen mirroring media resource information is used to obtain the source addresses in different playback modes in the server, so as to obtain the media data in the specified playback mode by accessing the source addresses and play the screen mirroring picture. By matching in the server, the virtual reality device 500 can play the media content mirrored from the smart terminal in 3D or panoramic mode, making the playback form of the screen mirroring process more diverse and improving the picture quality.

[0098] It should be noted that when the virtual reality device 500 matches the source address through the database, it can not only match the media data addresses of different source types corresponding to the screen mirroring data. That is, when the media source type corresponding to the screen mirroring data is a 2D source, the 3D source media or panoramic source media with the same name can be matched from the database; it can also match the media source addresses of the same source type corresponding to the screen mirroring data in the database. That is, the virtual reality device 500 can match the source addresses of the 2D source type in the database. By matching the source addresses of the same type, the virtual reality device 500 can obtain a source with better picture quality effect through the database to improve the playback effect of the screen mirroring media. For example, the movie "A" shared through the screen mirroring data is suitable for playing on the mobile phone. Therefore, during the screen mirroring process, the resource of the movie "A" of the 2D type but suitable for the virtual reality device 500 can also be matched from the database to obtain a better display effect.

[0099] In some exemplary embodiments, when the smart terminal and the virtual reality device 500 send the screen mirroring data based on the DLNA protocol, the virtual reality device 500 can use the URL address, media title, and media type in the screen mirroring data for matching. That is, as Figure 11 shown, the steps of obtaining the source address adapted to at least one playback mode in the server using the screen mirroring media resource information further include:

[0100] S310: Parse the screen mirroring media resource information from the screen mirroring data;

[0101] S320: Detect the current playback mode;

[0102] S330: If the media type is adapted to the current playback mode, access the current media URL address;

[0103] S340: If the media type is not adapted to the current playback mode, obtain the source address adapted to the current playback mode from the server according to the current media URL address and / or media title.

[0104] Before performing the matching, the virtual reality device 500 can parse the screen mirroring information from the screen mirroring data. Since the screen mirroring data is sent based on the DLNA protocol, the URL address, media title, and media type of the screen mirroring media can be parsed from the screen mirroring data. For example, the URL address of the movie "B" can be extracted from the screen mirroring data as "HTTP: / / ×××", the corresponding media title is "B", and the media type is "2D source".

[0105] After parsing the screen mirroring media information, the current playback mode of the virtual reality device 500 can also be detected. For example, by detecting the input and output method of the current rendering scene and the layout of the display panel in the rendering scene, it can be determined that the current playback mode is the 3D mode, that is, it is detected that the rendering scene includes two display panels that are visible to the left display camera and the right display camera respectively, and are used to present the left-eye image and the right-eye image in each frame of the picture respectively.

[0106] The virtual reality device 500 can also compare the media type with the current playback mode to determine whether the media source type in the screen mirroring data is compatible with the current playback mode. When the media type is compatible with the current playback mode, the media data can be directly obtained according to the URL address specified in the screen mirroring data to play the screen mirroring picture. For example, if the media type parsed from the screen mirroring data is a 2D source and the current playback mode is also the 2D mode, the parsed URL address can be directly accessed, so as to play the screen mirroring media in the 2D mode.

[0107] When the media type is not compatible with the current playback mode, the source address corresponding to the current playback mode can be queried from the server, that is, the virtual reality device 500 can use the current media URL address and / or media title to match the source address compatible with the current playback mode in the server. For example, if the media type parsed from the screen mirroring data is a 2D source and the current playback mode is the 3D mode, it is necessary to use the current media URL address "HTTP: / / ×××" or the media title "B" to perform matching in the server to query the 3D source of the movie B.

[0108] Since different playback modes require different rendering methods for media assets data. For example, when the current playback mode is the 3D mode, the virtual reality device 500 can split each frame of image and input the left-eye picture part and the right-eye picture part into the display panel in the rendering scene for display respectively; while when the current playback mode is the 2D mode, the virtual reality device 500 does not need to split each frame of image and directly presents the picture content on the display panel. Therefore, in this embodiment, after obtaining the screen mirroring data, it can compare whether the media asset type is suitable for the current playback mode, so that the screen mirroring data adapts to the current playback form of the virtual reality device 500, and complete the rendering according to the suitable rendering method, avoiding incorrect rendering methods and improving the playback quality.

[0109] It should be noted that when the virtual reality device 500 matches the current media asset URL address and / or media asset title in the server, in some cases, it may not be able to match the appropriate source address. That is, there is no source corresponding to other playback modes for some screen mirroring media assets in the server. Therefore, when the virtual reality device 500 fails to match the source address suitable for the current playback mode in the server, it can also automatically switch the playback mode to adapt to the screen mirroring data.

[0110] After obtaining the specified source address, the virtual reality device 500 can obtain the media asset data suitable for the current playback mode through this source address to play the screen recording picture. As Figure 12 shown, in order to be able to play the media asset data, the virtual reality device 500 can execute the following steps in the step of accessing the source address to play the screen mirroring picture:

[0111] S410: Obtain the video data stream from the source address;

[0112] S420: Display the video data stream in the virtual rendering scene;

[0113] S430: Perform image shooting on the virtual rendering scene to generate a screen mirroring picture;

[0114] S440: Send the screen mirroring picture to the display for display.

[0115] By accessing the source address, the virtual reality device 500 can obtain the video data stream from the source address and display the video data stream in the rendering scene. Among them, the virtual rendering scene includes a left display camera and a right display camera. The virtual reality device 500 can perform image shooting on the virtual rendering scene where the video data stream is displayed through the left display camera and the right display camera, thereby generating a screen mirroring picture, and finally sending the screen mirroring picture to the display to display the screen mirroring picture content through the display.

[0116] For the process of displaying a video data stream, the virtual reality device 500 can decode the video data stream to obtain multiple frames of media resource images. Then, according to the current playback mode, a display panel is added to the virtual rendering scene to display the media resource images frame by frame through the loaded display panel.

[0117] In different playback modes, the form of the display panel loaded into the virtual rendering scene is also different. When the playback mode is the 3D mode, the virtual reality device 500 can add a left panel and a right panel to the virtual rendering scene. Among them, the left panel is visible to the left display camera, and the right panel is visible to the right display camera. The virtual reality device 500 can split each frame of the video data stream to obtain a left-eye image part and a right-eye image part, and display the left-eye image part on the left panel and the right-eye image part on the right panel. Then, by simultaneously taking pictures of the virtual rendering scene with the left display camera and the right display camera, a rendered scene image with the video stream image can be obtained.

[0118] Since the left panel is only visible to the left display camera, the image captured by the left display camera includes the content of the left-eye image part. Similarly, the right panel is only visible to the right display camera, which enables the image captured by the right display camera to include the content of the right-eye image part. Through the simultaneous shooting of the left and right display cameras, the final projected screen can have the 3D effect in the video data stream and the 3D effect in the rendering scene, providing a better immersive experience.

[0119] If the current playback mode is the panoramic mode, the virtual reality device 500 can add a curved panel to the virtual rendering scene. According to the different panoramic forms of each frame in the panoramic video source, the shape of the added curved panel is also different. For example, when the panoramic video source is of the 180° panoramic type, a hemispherical curved panel can be added to the rendering scene; when the panoramic video source is of the 360° panoramic type, a spherical curved panel can be added to the rendering scene; when the panoramic video source is of the fish-eye panoramic type, an annular curved panel can be added to the rendering scene.

[0120] To facilitate the output of the final image, the curved panel added to the rendering scene should have the midpoint between the left display camera and the right display camera as the arc center. For example, in the 360° panoramic playback mode, the virtual reality device 500 can display the video image on the inner surface of a 3D sphere model in the scene. This 3D sphere model has the midpoint between the left display camera and the right display camera as the center of the sphere, and the radius of the 3D sphere model can be defined according to the shooting range of each frame of the panoramic image in the video data stream, so as to display the panoramic image completely on the 3D sphere model and realize the playback of the 360° panoramic media resource image.

[0121] In some exemplary embodiments, in order to enable the virtual reality device 500 to match the source address in the database, the server may also pre-create a database, where the database is created based on the MyBatis framework, as Figure 13 shown, including the following steps:

[0122] S501: Read the mapping table by reading the reader object;

[0123] S502: Obtain the SQL session of the current thread to start a transaction;

[0124] S503: Read the operation number in the mapping table through the SQL session, and read the SQL statement;

[0125] S504: Commit the transaction to create the database.

[0126] The mapping table includes source tags, source addresses, and the mapping relationship between source tags and source addresses. The server can create a database in the initial control system to record the mapping relationship between playback modes and source types.

[0127] During the process of matching the source address, since a media asset item can support multiple playback modes, there is a "one-to-many" / "many-to-one" mapping relationship between the media asset item and the playback mode. Among them, the one-to-many mapping relationship means that from the perspective of the picture / video resource, one picture / video has multiple playback modes, which is one-to-many; the many-to-one mapping relationship means that from the perspective of the playback mode, multiple playback modes correspond to one picture / video resource, which is many-to-one. Based on this one-to-many / many-to-one mapping relationship, a database can be created through a mapping framework. Frameworks that can create databases include open-source object-relational mapping frameworks (Hibernate), Java Database Connectivity (JDBC), and persistence layer frameworks (MyBatis), etc.

[0128] Among them, the MyBatis framework has an interface binding function, that is, it includes annotation binding of Structured Query Language (SQL) and Extensible Markup Language binding of SQL. Moreover, the MyBatis framework supports the Object Graph Navigation Language (OGNL) expression for dynamic SQL. Therefore, the MyBatis framework can flexibly configure the SQL statements to be executed in XML or annotation mode, map Java objects and SQL statements to generate the finally executed SQL, and finally map the results of SQL execution to generate Java objects. This makes the learning threshold of the MyBatis framework low. Database maintainers can directly write native SQL, strictly control the SQL execution performance, and have high flexibility.

[0129] In the process of using the MyBatis framework to create and maintain a database, the mapping file of the MyBatis framework, that is, the mapping relation table, can be read through the Reader object in the MyBatis framework. Then, the SqlSessionFactory object can be created through the SqlSessionFactoryBuilder object, and the SQLSession of the current thread can be obtained. After obtaining the SQLSession, the transaction can be set to be enabled by default to read the operation number in the mapping file through the SQLSession, so as to read the SQL statement, and then commit the transaction to store the mapping relation table in the database.

[0130] It can be seen that in this embodiment, the MyBatis framework and the mapping relation table can quickly establish a one-to-many / many-to-one mapping relation between the media asset project and the playback mode. Moreover, the database based on the MyBatis framework can not only reduce the workload and learning content of developers, but also facilitate the unified maintenance of media asset projects in different playback interfaces, enabling the virtual reality device 500 to quickly query the appropriate playback parameters through the database.

[0131] Based on the MyBatis framework to create a database, after extracting the screen projection media asset information, the virtual reality device 500 can extract the mapping relation table from the database, then query the source label from the mapping relation table according to the current media asset URL address and / or media asset title, and finally select the URL address suitable for the current playback mode through the source label, that is, obtain the source address suitable for the specified playback mode.

[0132] In the above embodiments, after receiving the screen mirroring data, the virtual reality device 500 can match the source addresses of different playback modes through the server, so as to obtain various source types. In order to obtain a better interaction experience and improve the interaction efficiency of the screen mirroring connection, after receiving the screen mirroring data, the virtual reality device 500 can first play the screen mirroring data in the 2D mode to retain the interaction effect during the traditional screen mirroring connection. At the same time, the switching window is displayed in the display interface to prompt the user that they can switch to other playback modes to obtain a better screen mirroring interaction effect.

[0133] After the user determines to switch to other playback modes, they access the media resource addresses corresponding to the other playback modes from the server and display the media resources corresponding to the modes on the display.

[0134] That is, as Figure 14 shown, in some exemplary embodiments, the step of accessing the source address to play the screen mirroring image further includes:

[0135] S451: Control the display to display a switching window according to the source address;

[0136] S452: Receive the switching instruction input by the user through the switching interface;

[0137] S453: Respond to the switching instruction and switch the playback mode;

[0138] S454: Access the source address to play the screen mirroring image according to the switched mode.

[0139] In this embodiment, after receiving the screen mirroring data, the virtual reality device 500 can first play the screen mirroring data according to the media source type specified in the screen mirroring media information, that is, first present the screen mirroring image of the 2D source type through the 2D mode. At the same time, the virtual reality device 500 can also execute the step of using the screen mirroring media information to obtain the source addresses adapted to at least one playback mode in the server in the background operation mode to match the source corresponding to other playback modes.

[0140] When the virtual reality device 500 queries the source addresses of different playback modes in the server, it can display a switching window according to the matched source address. Among them, the switching window can be used to prompt that there are sources corresponding to different playback modes for the current screen mirroring media, so that when the user wants to experience different playback modes, they can complete the switching of the playback mode through the switching window. In order to achieve the prompting effect, the switching window can include prompt information. For example, as Figure 15 、 Figure 16As shown, while playing the screen mirroring data in 2D, a switching window can be displayed on the current interface. The switching window may include the following prompt text: "It is detected that the video contains a 3D (360) mode video source. Do you want to switch to the 3D (360) mode for playback?" And two options, "Yes" and "No", are displayed in the switching window for the user to select.

[0141] For this reason, while displaying the switching window, the virtual reality device 500 can also receive a switching instruction input by the user for switching the playback mode. And in response to this switching instruction, the playback mode is switched, and at the same time, the video source address that matches and hits is accessed, so as to play the screen mirroring picture according to the switched playback mode. For example, when the user clicks the "Yes" option on the switching window, it represents the input of the switching instruction. The virtual reality device 500 can respond to this switching instruction and switch the playback mode to the 3D mode, that is, add a left panel and a right panel in the rendered scene. At the same time, according to the matched 3D video source address, the 3D type of media data can be obtained by accessing this video source address, and the obtained media data is played in the 3D mode to switch to the 3D mode.

[0142] Obviously, the user can also input a control instruction in the switching window for not switching the playback mode. For example, the user can click the "No" option on the switching window to control the virtual reality device 500 not to switch the playback mode and still play the screen mirroring data in the 2D mode.

[0143] In order to more intuitively experience the viewing effect after switching and guide the user to complete the mode switching, in some exemplary embodiments, the switching window may also include a preview screen, and the preview screen can present a media picture or a poster picture according to the corresponding video source type. For example, when it is matched in the server that the current screen mirroring data has a 3D type of video source, the 3D type of media data can be obtained by accessing the 3D video source address, and the media data is played in the 3D mode to obtain a 3D media picture. Then, through the preview window, the 3D media picture is displayed so that the user can experience the viewing effect in the 3D mode. Obviously, in order to reduce the data processing volume, the preview window can only preview a part of the video segments of the media data in different modes, for example, only preview the first 5s video segments of the media data.

[0144] While displaying the switching window with the preview screen, the virtual reality device 500 can also receive the switching instruction input by the user through the switching window. For example, as Figure 17As shown, the switching window may only include a preview screen and a close button. When the user clicks on any area of the switching window other than the close button, it represents that the user inputs a switching instruction. At this time, the playback mode of the virtual reality device 500 can be switched to the corresponding playback mode according to the switching method in the above embodiment. When the user clicks the close button, it represents an input of a close instruction. After receiving the close instruction input by the user through the preview window, the virtual reality device 500 responds to the close instruction, parses the screen mirroring data to obtain the screen mirroring image, and plays the screen mirroring image in the 2D playback mode.

[0145] In some other embodiments of the present application, intelligent terminals such as mobile phones, tablets, computers, etc. have a 2D presentation mode, and the media resources generally played on intelligent terminals are 2D resources. Therefore, the intelligent terminal generally mirrors and sends the media images related to 2D resources. After receiving the screen mirroring URL link, the virtual reality device 500 can access the URL link through the server and display the video source corresponding to the URL on the display. The virtual display device 500 obtains the media resources of the screen mirroring link related to the 2D resource and presents the 2D media resources on the main interface of the display to retain the interaction effect during traditional screen mirroring connections.

[0146] Further, after receiving the screen mirroring link related to the 2D resource sent by the mobile terminal, the server parses the link. When a 3D video source corresponding to the link is matched in the server, the server sends a preview screen or a poster screen corresponding to the 3D video source to the virtual display device 500. When the user selects to watch the 3D video source, the playback mode of the player is switched to the 3D mode, and the images related to the 3D video source are played on the main interface.

[0147] That is, as Figure 18 shown, in some exemplary embodiments, the steps of playing the screen mirroring image further include:

[0148] S601: Obtain a control instruction input by the user for establishing a screen mirroring connection to control the media resources to be sent;

[0149] S602: In response to the control instruction, receive the screen mirroring data sent by the intelligent terminal, where the screen mirroring data includes screen mirroring media information;

[0150] S603: Control the display to present the screen mirroring media image in the 2D mode on the main interface and present a recommended screen on the secondary interface; where the recommended screen is an entry screen of a recommended resource different from the 2D mode sent by the server, and the recommended resource is associated with the media resource;

[0151] S604: In response to the user's selection of the recommended screen, control the display to present the recommended resource in a presentation mode different from the 2D mode on the main interface of the display.

[0152] In this embodiment, after the virtual reality device 500 receives the screen mirroring data, the screen mirroring data or the screen mirroring media information actually includes a URL data. The virtual display device requests the screen mirroring media resources from the server based on this URL data. After receiving the screen mirroring media resources sent by the server, it can first play the screen mirroring media resources according to the media source type specified in the screen mirroring media information, that is, first present the screen mirroring picture of the 2D media source type on the main interface in the 2D mode.

[0153] At the same time, the server parses the source address information, such as the source name, etc., and queries whether there are sources of other playback modes, such as 3D, panoramic, etc., according to this source address information. If there are sources of other playback modes and it is detected that the screen mirroring target is the virtual display device 500, it sends a recommended picture (taking the 3D source as an example below), such as a preview picture, etc., of the source of the other playback mode to the virtual display device 500. The preview picture can present the media picture or the poster picture according to the corresponding source type, as well as the entry link of the source of the other playback mode, and is displayed on the secondary interface on the side of the main interface.

[0154] After the virtual display device 500 receives the recommended picture, it presents this recommended picture on the secondary interface. At the same time, the main interface still normally plays the screen mirroring picture of the 2D source.

[0155] When the user selects the preview picture or the poster picture corresponding to the 3D source by means of an air mouse, gesture or other selection methods, the main interface switches from the state of playing the 2D source to the state of playing the 3D source, and accesses the picture link corresponding to the 3D source from the server to present this recommended content on the main interface. At the same time, the recommended picture on the secondary interface disappears.

[0156] In order to be able to more intuitively experience the viewing effect after switching, both guide the user to complete the mode switch and not affect the user's immersive experience. In some exemplary embodiments, the recommended picture is generally on the secondary interface, and can be in the form of filling the entire secondary interface or in the form of a small window. After receiving the recommended picture sent by the server, it is only displayed for a preset time, such as one minute. Within this minute, the user's selection on the secondary interface can be received and switched to playing the 3D source on the main interface. After one minute, the recommended picture on the secondary interface is hidden, and only the screen mirroring media picture is displayed on the main interface, so that the user can better focus on the media playback on the main interface. And when the user triggers the selection instruction or the control instruction again, the recommended picture on the secondary interface and the content of the service classification expansion area can be displayed again.

[0157] In this embodiment, since the virtual display device 500 can be distinguished from a display device that only displays 2D content sources, it can support viewing in 3D, 360-degree, and panoramic modes. However, the screen mirroring of a mobile device is only suitable for scenarios with 2D content sources. When the screen mirroring image is a 2D content source, in order to provide a more immersive viewing mode for the virtual reality device, a request can be sent to the server, or a link or entry for a 3D content source directly sent by the server can be received, when the screen mirroring is a 2D content source. While the user is viewing the 2D content source, they can learn about the possible 3D content sources available for viewing. When the user clicks to view, they can be directly redirected to the 3D content source for viewing. This provides a better screen mirroring experience for the user and further highlights the viewing advantages of the virtual display device 500.

[0158] Based on the above virtual reality device 500, in some embodiments of the present application, a screen mirroring media asset playback method is further provided, including the following steps:

[0159] S1: Obtain a control instruction input by the user for establishing a screen mirroring connection;

[0160] S2: In response to the control instruction, receive the screen mirroring data sent by the smart terminal, where the screen mirroring data includes screen mirroring media asset information;

[0161] S3: According to the screen mirroring media asset information, obtain a content source address adapted to at least one playback mode from the server;

[0162] S4: Access the content source address to control the display to display the media asset image corresponding to the content source address.

[0163] As can be seen from the above technical solutions, the screen mirroring media asset playback method provided in this embodiment can, after receiving the user's control instruction, receive the screen mirroring data, and extract the screen mirroring media asset information from the screen mirroring data. Then, use the screen mirroring media asset information to obtain the content source addresses in different playback modes in the server, so as to obtain the media asset data in the specified playback mode by accessing the content source addresses and play the screen mirroring image. The method can obtain the media asset data in the specified playback mode while displaying the screen mirroring data, so as to play the media asset content screen mirrored from the smart terminal in 3D or panoramic mode, solving the problem of a single playback form and being not conducive to obtaining an immersive effect.

[0164] For the similar parts between the embodiments provided in the present application, reference can be made to each other. The specific embodiments provided above are only several examples under the overall concept of the present application and do not constitute a limitation on the protection scope of the present application. For those skilled in the art, any other implementation manner extended based on the solution of the present application without creative efforts belongs to the protection scope of the present application.

Claims

1. A virtual reality device, characterized in that, Comprising: A display configured to present a user interface, the user interface including a main interface arranged in the center and a secondary interface located on the side of the main interface; A communicator configured to establish a screen mirroring connection with a smart terminal and perform data interaction with a server; A controller configured to: Obtain a control instruction input by a user for establishing a screen mirroring connection and controlling the delivery of media resources; In response to the control instruction, receive screen mirroring data sent by the smart terminal, the screen mirroring data including a URL address, wherein the smart terminal is one of a mobile phone, a tablet, and a computer; Send the URL address to the server; Present a screen mirroring media picture in 2D mode on the main interface and present a recommended picture on the secondary interface, wherein the screen mirroring media picture is feedback by the server based on the URL address, and the recommended picture is obtained by the server querying based on the source name parsed from the address, and the recommended picture is an entrance picture of a 3D mode source or a panoramic mode source; In response to the user's selection of the recommended picture, add a display panel adapted to the current playback mode in a virtual rendering scene according to the 3D mode or the panoramic mode to present the recommended resource through the display panel, wherein the 3D mode is adapted to the 3D source, the panoramic mode is adapted to the panoramic source, when the current playback mode is the 3D mode, add a left panel and a right panel in the virtual rendering scene; if the current playback mode is the panoramic mode, add a curved panel in the virtual rendering scene; Perform image capture on the virtual rendering scene to generate a display picture; Send the display picture to the display for display.

2. The virtual reality device according to claim 1, wherein, After sending the URL address to the server, the controller is further configured to: Receive the screen mirroring media picture feedback by the server.

3. The virtual reality device according to claim 2, wherein, After sending the URL address to the server, the controller is further configured to: Receive the recommended picture sent down by the server.

4. A screen mirroring method, characterized in that: Obtain a control instruction input by a user for establishing a screen mirroring connection and controlling the delivery of media resources; In response to the control instruction, receive screen mirroring data sent by the smart terminal, the screen mirroring data including a URL address, wherein the smart terminal is one of a mobile phone, a tablet, and a computer; Send the URL address to the server; Present a screen mirroring media picture in 2D mode on the main interface and present a recommended picture on the secondary interface, wherein the screen mirroring media picture is feedback by the server based on the URL address, and the recommended picture is obtained by the server querying based on the source name parsed from the address, and the recommended picture is an entrance picture of a 3D mode source or a panoramic mode source; In response to the user's selection of the recommended screen, a display panel adapted to the current playback mode is added to the virtual rendering scene according to the 3D mode or the panoramic mode, so as to present the recommended resources through the display panel, wherein the 3D mode is adapted to the 3D video source, the panoramic mode is adapted to the panoramic video source, when the current playback mode is the 3D mode, a left panel and a right panel are added to the virtual rendering scene; if the current playback mode is the panoramic mode, a curved panel is added to the virtual rendering scene; Perform image capture on the virtual rendering scene to generate a display screen; Send the display screen to the display for display.

5. The screen mirroring method according to claim 4, wherein After sending the URL address to the server, it further includes: Receiving the screen mirroring media resource screen feedback from the server.

6. The screen mirroring method according to claim 4, wherein After sending the URL address to the server, receiving the recommended screen sent by the server.

Images