Virtual reality device and screenshot picture playing method

Abstract

The present application provides a virtual reality device and a screenshot image playback method. The method can parse the screenshot image file after obtaining the control command input by the user, so as to obtain the screenshot image screen and the field of view; then call the player model according to the field of view, so as to map the screenshot image screen to the player model. The screenshot image playback method can use the player model to display the screenshot image. The player model has a display area shape corresponding to the field of view angle, so that the screenshot image can fill the entire field of view during playback, alleviating the confusion between the sky box screen and the screenshot screen. In addition, the screenshot image scene is restored through a surface model adapted to the screenshot screen, thereby obtaining a better immersive experience.

Description

[0001] This application claims priority to a Chinese patent application filed with the Chinese Patent Office on January 18, 2021, with application number 202110065015.6 and invention name “A virtual reality device and quick interaction method”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of virtual reality technology, and in particular to a virtual reality device and a screenshot image playback method. Background Art

[0003] Virtual Reality (VR) technology is a display technology that uses computers to simulate virtual environments, giving people a sense of immersion in the environment. A virtual reality device is a device that uses virtual display technology to present virtual images to users. Usually, a virtual reality device includes two display screens for presenting virtual image content, one for the user's left eye and the other for the left eye. When the content displayed on the two display screens comes from images of the same object from different perspectives, it can bring a three-dimensional viewing experience to the user.

[0004] When displaying pictures on a virtual reality device, the virtual reality device can complete the picture playback through a specific playback interface. The playback interface consists of a picture imaging area and a skybox. The imaging area can display multimedia content such as pictures, and the skybox is used to present the rendering scene background of the playback interface, thereby providing users with an immersive experience.

[0005] In actual use, the virtual reality device can output the displayed content in the form of an image through the screenshot operation, so as to display the user's perspective image at the time of the screenshot on the display device or the virtual reality device. Since the screenshot image obtained by the virtual reality device screenshot includes not only the interface screen content but also the rendered scene screen content, when the virtual reality device displays the screenshot image in the imaging area, it will display the interface screen and the rendered scene screen at the same time, which will cause the rendered scene screen to be inconsistent with the skybox screen content, that is, the playback screen is chaotic, and a good immersive experience cannot be obtained. Summary of the invention

[0006] The present application provides a virtual reality device and a screenshot image playback method to solve the problem that the rendering image displayed by the traditional screenshot image playback method is not coordinated with the skybox image.

[0007] In a first aspect, the present application provides a virtual reality device, including a display and a controller. The display is used to display a user interface such as a playback interface, and the user interface includes a picture imaging area and a skybox area. The picture imaging area is used to present multimedia content; the skybox area is located around the image area and is used to present rendered background content. The controller is configured to perform the following program steps:

[0008] Obtaining the control command input by the user for playing the screenshot image;

[0009] In response to the control instruction, parsing the screenshot image file to be played to obtain the screenshot image picture and field of view;

[0010] Calling a player model according to the field of view angle, wherein the player model is a surface model pre-established according to the field of view angle in a rendering scene;

[0011] The screenshot image is mapped to the player model.

[0012] In a second aspect, the present application further provides a display device, including a display and a controller. The display is used to display a user interface such as a playback interface, and the user interface includes a picture imaging area and a skybox area. The picture imaging area is used to present multimedia content; the skybox area is located around the image area and is used to present the rendered background content. The controller is configured to execute the following program steps:

[0013] Receive the screenshot command input by the user;

[0014] In response to the screenshot instruction, performing image capture on the current rendering scene to obtain a screenshot image;

[0015] Recording a shooting position and a field of view angle in the rendering scene when performing image shooting;

[0016] The screenshot image and the field of view are saved to generate a screenshot picture file.

[0017] In a third aspect, the present application further provides a screenshot image playback method, which is applied to the above-mentioned virtual reality device, and the screenshot image playback method comprises:

[0018] Obtaining the control command input by the user for playing the screenshot image;

[0019] In response to the control instruction, parsing the screenshot image file to be played to obtain the screenshot image picture and field of view;

[0020] Calling a player model according to the field of view angle, wherein the player model is a surface model pre-established according to the field of view angle in a rendering scene;

[0021] The screenshot image is mapped to the player model.

[0022] In a fourth aspect, the present application further provides a screenshot image generation method, which is applied to the above-mentioned virtual reality device, and the screenshot image generation method comprises:

[0023] Receive the screenshot command input by the user;

[0024] In response to the screenshot instruction, performing image capture on the current rendering scene to obtain a screenshot image;

[0025] Recording a shooting position and a field of view angle in the rendering scene when performing image shooting;

[0026] The screenshot image and the field of view are saved to generate a screenshot picture file.

[0027] It can be seen from the above technical solutions that the virtual reality device and screenshot picture playback method provided by the present application can parse the screenshot picture file after obtaining the control command input by the user, so as to obtain the screenshot image screen and the field of view angle; then call the player model according to the field of view angle, so as to map the screenshot image screen to the player model. The screenshot picture playback method can use the player model to display the screenshot picture. The player model has a display area shape corresponding to the field of view angle, so that the screenshot picture can fill the entire field of view during playback, alleviating the confusion between the sky box picture and the screenshot picture. In addition, the screenshot picture scene is restored through a surface model adapted to the screenshot picture, so as to obtain a better immersive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 A schematic diagram of the structure of a display system including a virtual reality device in an embodiment of the present application;

[0030] Figure 2 This is a schematic diagram of the global interface of the VR scene in the embodiment of the present application;

[0031] Figure 3 A schematic diagram of the recommended content area of ​​the global interface in an embodiment of the present application;

[0032] Figure 4 This is a schematic diagram of the application quick operation entry area of ​​the global interface in an embodiment of the present application;

[0033] Figure 5A schematic diagram of suspended matter at a global interface in an embodiment of the present application;

[0034] Figure 6 This is a schematic diagram of entering the quick center through the status bar in an embodiment of the present application;

[0035] Figure 7 This is a schematic diagram of the quick center window in the embodiment of this application;

[0036] Figure 8 This is a schematic diagram of entering the shortcut center by pressing a button in an embodiment of the present application;

[0037] Fig. 9 This is a schematic diagram of starting to take screenshots in the embodiment of this application;

[0038] Fig.10 This is a schematic diagram of a prompt text window when a screenshot is successfully taken in an embodiment of the present application;

[0039] Fig.11 This is a screenshot of the traditional image browsing interface.

[0040] Fig.12 This is a schematic diagram of the process of the screenshot picture playback method in the embodiment of the present application;

[0041] Fig.13 This is a schematic diagram of the screen capture picture playback effect in the embodiment of the present application;

[0042] Fig.14 This is a schematic diagram of the process of matching the player model in an embodiment of the present application;

[0043] Fig.15 A schematic diagram of the process of creating a player model in an embodiment of the present application;

[0044] Fig.16 This is a schematic diagram of spherical coordinates in an embodiment of the present application;

[0045] Fig.17 This is a schematic diagram of the texture mapping process in an embodiment of the present application;

[0046] Fig.18 Schematic diagram of the mapping relationship between the normal vector coordinates and the texture coordinates of a point on a spherical surface in an embodiment of the present application;

[0047] Fig.19 This is a schematic diagram of the process of parsing a screenshot image file to be played in an embodiment of the present application;

[0048] Fig. 20 A schematic diagram of the process of outputting a screenshot image in an embodiment of the present application;

[0049] Fig.21 A schematic diagram of a rendering scene when outputting a screenshot image in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0051] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0052] It should be understood that the terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented according to an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0053] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0054] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0055] References throughout this specification to "multiple embodiments", "some embodiments", "one embodiment" or "an embodiment" etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment. Therefore, phrases such as "in multiple embodiments", "in some embodiments", "in at least another embodiment" or "in an embodiment" etc. that appear throughout this specification do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, specific features, structures or characteristics may be combined in any suitable manner. Therefore, without limitation, specific features, structures or characteristics shown or described in conjunction with one embodiment may be combined in whole or in part with features, structures or characteristics of one or more other embodiments. Such modifications and variations are intended to be included within the scope of this application.

[0056] In the embodiments of the present application, the virtual reality device 500 generally refers to a display device that can be worn on the user's face to provide the user with an immersive experience, including but not limited to VR glasses, augmented reality devices (AR), VR gaming devices, mobile computing devices, and other wearable computers. Some embodiments of the present application use VR glasses as an example to illustrate the technical solution. It should be understood that the provided technical solution can also be applied to other types of virtual reality devices. The virtual reality device 500 can operate independently or be connected to other smart display devices as an external device, wherein the display device can be a smart TV, computer, tablet computer, server, etc.

[0057] The virtual reality device 500 can be worn on the user's face, display media images, and provide close-up images for the user's eyes to bring an immersive experience. In order to present media images, the virtual reality device 500 may include multiple components for displaying images and wearing on the face. Taking VR glasses as an example, the virtual reality device 500 may include but is not limited to at least one of the components such as a housing, a facial firmware component, an optical system, a display component, a posture detection circuit, and an interface circuit. In actual applications, the optical system, the display component, the posture detection circuit, and the interface circuit can be arranged in the housing to present a specific display image; facial fixings are connected to both sides of the housing to be worn on the user's face.

[0058] When in use, the posture detection circuit has built-in posture detection elements such as gravity acceleration sensors and gyroscopes. When the user's head moves or rotates, the user's posture can be detected, and the detected posture data can be transmitted to processing elements such as controllers, so that the processing elements can adjust the specific screen content in the display component according to the detected posture data.

[0059] In some embodiments, Figure 1 The virtual reality device 500 shown can be connected to the display device 200, and a network-based display system can be established between the virtual reality device 500, the display device 200 and the server 400. Real-time data interaction can be performed between the virtual reality device 500, the display device 200 and the server 400. For example, the display device 200 can obtain media data from the server 400 and play it, and transmit specific picture content to the virtual reality device 500 for display.

[0060] The display device 200 may be a liquid crystal display, an OLED display, or a projection display device. The specific display device type, size, and resolution are not limited. It is understood by those skilled in the art that the display device 200 may be modified in performance and configuration as needed. The display device 200 may provide a broadcast receiving television function, and may also provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0061] The display device 200 and the virtual reality device 500 also communicate data with the server 400 through various communication methods. The display device 200 and the virtual reality device 500 may be allowed to communicate and connect via a local area network (LAN), a wireless local area network (WLAN) and other networks. The server 400 may provide various contents and interactions to the display device 200. For example, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 may be one cluster or multiple clusters, and may include one or more types of servers. Other network service contents such as video on demand and advertising services are provided through the server 400.

[0062] During the data interaction, the user can operate the display device 200 through the mobile terminal 300 and the remote controller 100. The mobile terminal 300 and the remote controller 100 can communicate with the display device 200 by direct wireless connection or by indirect connection. That is, in some embodiments, the mobile terminal 300 and the remote controller 100 can communicate with the display device 200 by direct connection such as Bluetooth or infrared. When sending a control command, the mobile terminal 300 and the remote controller 100 can directly send the control command data to the display device 200 by Bluetooth or infrared.

[0063] In other embodiments, the mobile terminal 300 and the remote controller 100 may also access the same wireless network as the display device 200 through a wireless router to establish indirect connection communication with the display device 200 through the wireless network. When sending a control command, the mobile terminal 300 and the remote controller 100 may first send the control command data to the wireless router, and then forward the control command data to the display device 200 through the wireless router.

[0064] In some embodiments, the user can also use the mobile terminal 300 and the remote control 100 to directly interact with the virtual reality device 500. For example, the mobile terminal 300 and the remote control 100 can be used as handles in a virtual reality scene to achieve functions such as somatosensory interaction.

[0065] In some embodiments, the display component of the virtual reality device 500 includes, but is not limited to, a display screen and a driving circuit related to the display screen. In order to present a specific picture and bring a stereoscopic effect, the display component may include two display screens, corresponding to the left eye and the right eye of the user respectively. When presenting a 3D effect, the picture content displayed on the left and right screens will be slightly different, and the left camera and the right camera of the 3D film source during the shooting process can be displayed respectively. Due to the picture content observed by the left and right eyes of the user, a display picture with a strong sense of stereoscopic can be observed when wearing it.

[0066] The optical system in the virtual reality device 500 is an optical module composed of multiple lenses. The optical system is arranged between the user's eyes and the display screen, and can increase the optical path through the refraction of the light signal by the lens and the polarization effect of the polarizer on the lens, so that the content presented by the display component can be clearly presented within the user's field of vision. At the same time, in order to adapt to the vision conditions of different users, the optical system also supports focusing, that is, adjusting the position of one or more of the multiple lenses through the focusing component, changing the mutual distance between the multiple lenses, thereby changing the optical path and adjusting the picture clarity.

[0067] The interface circuit of the virtual reality device 500 can be used to transmit interactive data. In addition to the above-mentioned transmission of posture data and display content data, in actual applications, the virtual reality device 500 can also be connected to other display devices or peripherals through the interface circuit to achieve more complex functions by exchanging data with the connected devices. For example, the virtual reality device 500 can be connected to the display device through the interface circuit, so that the displayed picture is output to the display device in real time for display. For another example, the virtual reality device 500 can also be connected to a handle through the interface circuit, and the handle can be held by the user to perform related operations in the VR user interface.

[0068] The VR user interface can be presented in a variety of different types of UI layouts according to user operations. For example, the user interface can include a global interface. The global UI after the AR / VR terminal is started is as follows: Figure 2 As shown, the global UI can be displayed on the display screen of the AR / VR terminal or on the display of the display device. The global UI can include but is not limited to at least one of the recommended content area 1, the business classification extension area 2, the application quick operation entrance area 3 and the floating object area 4.

[0069] The recommended content area 1 is used to configure different classification TAB columns; in the columns, media resources, special topics, etc. can be selected; the media resources may include but are not limited to 2D movies, educational courses, travel, 3D, 360-degree panoramic views, live broadcasts, 4K movies, program applications, games, travel and other businesses with media content, and the columns can select different template styles and support the simultaneous recommendation and arrangement of media resources and special topics, such as Figure 3 shown.

[0070] In some embodiments, a status bar may be provided at the top of the recommended content area 1, and a plurality of display controls may be provided in the status bar, including but not limited to common options such as time, network connection status, and power. The content included in the status bar may be customized by the user, for example, weather, user avatars, and other content may be added. The content included in the status bar may be selected by the user to perform a corresponding function. For example, when the user clicks on the time option, the virtual reality device 500 may display a time device window in the current interface, or jump to a calendar interface. When the user clicks on the network connection status option, the virtual reality device 500 may display a WiFi list in the current interface, or jump to a network settings interface.

[0071] The content displayed in the status bar can be presented in different content forms according to the setting status of the specific project. For example, the time control can directly display specific time text information, and display different texts at different times; the power control can display different patterns according to the current remaining power of the virtual reality device 500.

[0072] The status bar is used to enable the user to perform commonly used control operations and quickly set up the virtual reality device 500. Since the setting program for the virtual reality device 500 includes many items, it is usually not possible to display all commonly used setting options in the status bar. For this reason, in some embodiments, an extended option may also be provided in the status bar. After the extended option is selected, an extended window may be presented in the current interface, and a plurality of setting options may be further provided in the extended window to implement other functions of the virtual reality device 500.

[0073] For example, in some embodiments, after the extension option is selected, a "Quick Center" option can be set in the extension window. After the user clicks the Quick Center option, the virtual reality device 500 can display the Quick Center window. The Quick Center window may include but is not limited to "Screenshot", "Screen Recording" and "Screen Casting" options, which are used to wake up the corresponding functions respectively.

[0074] The business category extension area 2 supports the configuration of extended categories of different categories. If there is a new business type, it supports the configuration of an independent TAB to display the corresponding page content. The extended categories in the business category extension area 2 can also be sorted and adjusted and offline business operations can be performed. In some embodiments, the business category extension area 2 may include the following content: film and television, education, travel, applications, and mine. In some embodiments, the business category extension area 2 is configured to display a large business category TAB and supports the configuration of more categories. Its icons support configuration, such as Figure 3 shown.

[0075] The application quick operation entry area 3 can specify pre-installed applications to be displayed in front for operation recommendation, and supports configuration of special icon styles to replace default icons. The pre-installed applications can be specified as multiple. In some embodiments, the application quick operation entry area 3 also includes a left-moving control and a right-moving control for moving the option target, which are used to select different icons, such as Figure 4 shown.

[0076] The floating object area 4 can be configured to be located at the upper left side or the upper right side of the fixed area, and can be configured as a replaceable image or a jump link. For example, after receiving a confirmation operation, the floating object jumps to a certain application or displays a specified function page, such as Figure 5 In some embodiments, the floating object may not be configured with a jump link and is simply used for image display.

[0077] In some embodiments, the global UI also includes a status bar at the top for displaying time, network connection status, battery status, and more quick operation entries. When an icon is selected using the handle of the AR / VR terminal, that is, a handheld controller, the icon will display a text prompt including left and right expansion, and the selected icon will be stretched left and right according to its position.

[0078] For example, after selecting the search icon, the search icon will display the text "Search" and the original icon. After further clicking the icon or text, the search icon will jump to the search page. For example, clicking the favorite icon will jump to the favorites TAB, clicking the history icon will display the history page by default, clicking the search icon will jump to the global search page, and clicking the message icon will jump to the message page.

[0079] In some embodiments, interaction can be performed through peripherals. For example, the handle of the AR / VR terminal can operate the user interface of the AR / VR terminal, including a back button; a home button, and a long press of the home button can realize a reset function; a volume plus or minus button; and a touch area, which can realize the focus click, slide, hold and drag functions.

[0080] The user can perform interactive operations through the global UI interface and jump to a specific interface in some interactive modes. For example, in order to play media data, the user can click on any media link icon in the global UI interface to start playing the media file corresponding to the media link. At this time, the virtual reality device 500 can control the jump to the media playback interface.

[0081] After jumping to a specific interface, the virtual reality device 500 may also display a status bar at the top of the playback interface and execute corresponding setting functions according to the set interaction mode. Figure 6 As shown, when the virtual reality device 500 plays video media, if the user wants to take a screenshot of the media screen, the user can click the extension option on the status bar to call out the extension window, and click the quick center option in the extension window to display the quick center window on the playback interface of the virtual reality device 500. Figure 7 As shown, finally click the "Screenshot" option in the extension center window to enable the virtual reality device 500 to perform a screenshot operation and store the display screen at the current moment in image form.

[0082] Among them, the status bar can be hidden when the virtual reality device 500 plays the media screen to avoid blocking or interfering with the media screen. When the user performs a specific interactive action, the display is triggered. For example, when the user does not use the handle to perform an action, the status bar can be hidden, and when the user uses the handle to perform an action, the status bar is displayed. To this end, the virtual reality device 500 can be configured to detect the state of the orientation sensor or the state of any button in the handle when playing the media screen. When it is detected that the detection value of the orientation sensor changes, or the button is pressed, the status bar can be controlled to be displayed at the top of the playback interface. When it is detected that the orientation sensor has not changed within the set time, or the button is not pressed, the status bar is hidden in the playback interface.

[0083] It can be seen that in the above embodiment, the user can call up the quick center through the status bar, and then click the corresponding option in the quick center window to complete the screenshot, screen recording and screen projection operations. The quick center window can also use other interactive methods to call the quick center and display the quick center window. For example, Figure 8 As shown, the user can call up the shortcut center window by double-clicking the home button on the controller.

[0084] The user can select any icon in the shortcut center window to start the corresponding function. The startup method of the corresponding function can be determined according to the actual interaction method of the virtual reality device 500. For example, Fig. 9As shown, after calling up the quick center window, the user can move the handle downward to move the focus mark to the screenshot option in the quick center window, and then start running the screenshot function by pressing the "Confirm / OK" key on the handle.

[0085] After executing the above-mentioned startup and running of the screenshot function, the virtual reality device 500 can call the screenshot operation program from the memory, and by running the screenshot program, take a screenshot of the currently displayed screen. For example, the virtual reality device 500 can run the screenshot program to overlay and synthesize the display contents of all layers to generate a picture file of the currently displayed pattern. The generated picture file can also be stored according to a preset storage path.

[0086] Since the virtual reality device 500 includes two displays, corresponding to the left and right eyes of the user respectively, when displaying part of the media images, in order to obtain a three-dimensional viewing effect, the contents displayed by the two displays respectively correspond to the left and right virtual playback cameras in the 3D scene, that is, the images displayed on the two displays are slightly different. Therefore, when performing a screenshot operation, different screens can obtain screenshots of different contents.

[0087] To this end, the virtual reality device 500 can detect the form of the screen displayed at the time of the screenshot when executing the screenshot. When it is detected that the user uses the 3D mode, the screen displayed on the left display and the right display can be respectively screenshotted, that is, two screenshots are output through one screenshot operation. However, since the content displayed on the left and right displays in the 3D mode is slightly different, some users do not need two screenshots. Therefore, in order to save the storage space of the virtual reality device 500, in some embodiments, when executing the screenshot operation, the screenshot program can be used to specify that a screenshot is performed on one of the displayed contents of the two displays, for example, specifying to perform a screenshot on the content displayed on the left display, thereby obtaining a screenshot picture and storing it.

[0088] In some embodiments, after completing the storage of the screenshot image, the virtual reality device 500 may also display prompt content in the displayed interface, for example, Fig.10 As shown, a prompt text window (toast window) can be displayed floating on the playback interface, including the text content "Screen capture successful, saved to ××", where "××" is the specific saving path. Obviously, the prompt text window will automatically cancel the display after a certain period of time to avoid excessive obstruction of the playback interface. For example, the prompt text window is displayed after the screenshot is successfully captured, and disappears after being displayed for 2 seconds.

[0089] In addition, the prompt text window can also dynamically change the specific prompt text content according to the saving process of the screenshot image. For example, after the user confirms to execute the screenshot function operation, the prompt text window displays "Screenshot successful, the screenshot image is being saved", and after the saving is completed, the prompt text window displays "Saved to ××".

[0090] It should be noted that, since users generally do not want the screenshot image to include the quick center interface when performing a screenshot operation, in order to be able to capture the played media content, the quick center window can be hidden after the user clicks the screenshot icon.

[0091] In some embodiments, after the screenshot operation is completed, the screenshot result can be displayed on the playback interface, that is, a display window is displayed floating on the upper layer of the playback interface, and the screenshot image is presented in the display window for the user to view. Furthermore, in the process of presenting the screenshot image, some drawing tool options can also be displayed in the display window, such as a line drawing tool, an ellipse tool, a rectangle tool, a text tool, etc. The user can block, annotate, crop, etc. the screenshot image by clicking these drawing tools to output a better screenshot image result.

[0092] It can be seen that in the above embodiment, the virtual reality device 500 can quickly perform a screenshot operation through the quick center window or shortcut keys, so as to save a screenshot image according to the content displayed by the virtual reality device 500. According to different application scenarios, the screenshot object of the screenshot operation can also be different. For example, the virtual reality device 500 can take a screenshot of the content displayed on the display, or a screenshot of a partial area in the rendering scene.

[0093] When playing media, the virtual reality device 500 can render the media screen, that is, set a display panel in the rendering scene to present the media screen content. Then, by adding virtual objects such as seats and speakers, a virtual scene is formed, so that according to the virtual scene, the effects of simulated theaters, home scenes, etc. can be output. At this time, if the virtual reality device 500 takes a screenshot of the displayed content, the screenshot includes not only the media screen, but also the rendered virtual object screen.

[0094] The virtual reality device 500 can also take a screenshot of the screen presented by the display panel in the rendering scene, that is, it can only take a screenshot of the media screen content. The specific screenshot method can be to perform a screenshot operation on the display panel screen area in the rendering scene, or the virtual reality device 500 directly extracts the media screen frame data after parsing the media data, and copies the extracted frame data, so as to obtain a screen without the virtual objects of the rendered virtual items.

[0095] In some embodiments, the virtual reality device 500 can also take screenshots of a portion of the rendered scene. For example, when the user wears the virtual reality device 500 and moves to any viewing angle, a screenshot can be taken of the display panel area and / or the nearby rendered screen content at the current viewing angle, thereby obtaining the screenshot content in the key area or the user-set area.

[0096] After the user performs the screenshot operation in the above manner, a screenshot image can be obtained. The obtained screenshot image can be played or displayed in the virtual reality device 500 or other display devices. When the screenshot image is played using the virtual reality device 500, the virtual reality device 500 can present the screenshot image content through the browsing interface.

[0097] The browsing interface is an interface output to the display after being rendered by the rendering engine of the virtual reality device 500, including but not limited to the picture display area, the operation area and the background area. Each area can obtain specific screen content through the rendering scene. For example, the screen content corresponding to the picture display area can be the screen shot picture content played in real time through the display panel set in the rendering scene; the screen content corresponding to the operation area is the UI model set in the rendering scene, which is used to perform the corresponding user operation; the screen content corresponding to the background area is the screen pattern presented by the rendering scene skybox, which may include a plane pattern content screen and may also include other model screen content set in the rendering scene. Among them, the skybox is the background screen content set by the virtual reality device 500 to present various interfaces, which may be a black background, a solid color background or other backgrounds with specific patterns.

[0098] Since the display panel is a flat shape, the image display area in the browsing interface can also be presented in a flat shape. The operation area can present different screen contents according to different browsing interfaces, and the background area can present different screens according to the skybox type. For example, Fig.11 As shown in FIG. 1 , in order to simulate an indoor viewing scene, indoor furniture images such as a sofa can be displayed in the background area. Fig.11 In the browsing interface shown, when the screenshot includes content of a rendered scene (such as tables, chairs, ground, background wall, etc.), the rendered scene is not compatible with the skybox, making the picture confusing and failing to provide users with a good immersive experience.

[0099] In order to obtain a better immersive experience, some embodiments of the present application further provide a virtual reality device 500, including but not limited to a display and a controller. The display is used to display a playback interface, and the controller can run a screenshot image playback method program to play the screenshot image according to the screenshot image playback method. Fig.12As shown, the controller is configured to perform the following program steps:

[0100] S1: Obtaining a control instruction input by a user for playing a screenshot image.

[0101] When using the virtual reality device 500, the user can output control instructions for various purposes through different interactive actions. When the user inputs a control instruction for playing a screenshot, the virtual reality device 500 can play the screenshot in response to the control instruction. The virtual reality device 500 will jump to the playback interface after the user inputs the control instruction; or, in the playback interface, jump from the state of displaying a screenshot to the state of displaying another screenshot. Therefore, the control instruction input by the user can be an interactive action input in other interfaces, or an interactive action input in the playback interface.

[0102] For example, the user can click on the icon of the screenshot image file in the file list to trigger the virtual reality device 500 to play the selected image. At this time, the action of clicking the image file icon input by the user is a control instruction for playing the screenshot image. Since the virtual reality device 500 can also present the image list, "previous", "next" and other UI interactive controls on the playback interface when displaying the playback interface, when the user displays a screenshot image in the playback interface, by clicking the "previous" or "next" UI interactive controls, or clicking any image icon in the image list, it can switch to displaying adjacent or selected images. At this time, the action of clicking the interactive control or image icon input by the user is a control instruction for playing the screenshot image.

[0103] In addition, the control command for playing the screenshot image can also be automatically input by the controller by judging the working state of the virtual reality device 500. For example, when the user uses the virtual reality device 500 to take a screenshot, the virtual reality device 500 can display the screenshot result after completing the screenshot. Therefore, when displaying the screenshot result, the virtual reality device 500 can automatically input the control command for playing the screenshot image.

[0104] S2: In response to the control instruction, the screenshot image file to be played is parsed to obtain the screenshot image picture and field of view.

[0105] After obtaining the control command input by the user, the virtual reality device 500 can analyze the screenshot image file to be played in response to the control command input by the user, including but not limited to decompression, reading pixel information, etc., to obtain the screenshot image information. In the process of parsing the screenshot image file, the virtual reality device 500 can also read the rendering scene information corresponding to the screenshot image to determine the field of view angle through the rendering scene information.

[0106] The screen content viewed by the user on the virtual reality device 500 is displayed in a sphere with the camera as the center and R as the radius in the rendering scene. The range that the user can see is related to the field angle of view (FOV) of the virtual reality device 500, and the range between the boundaries of the field angle is the range that the user can see.

[0107] It should be noted that, in this embodiment, when the screenshot includes a rendered scene screen, it can be displayed by the above-mentioned screenshot playback method, and when the screenshot does not include a rendered scene screen, it is not necessary to display it according to the above-mentioned screenshot playback method. Therefore, when parsing the screenshot image file to be played, it is also possible to detect whether the screenshot contains a rendered scene screen. For example, the source of the screenshot file can be determined by reading the description information of the screenshot file. When it is read that the screenshot file is generated by the virtual reality device 500, the shape of the effective area in the picture can be further detected. When it is detected that the picture is not distorted, it is determined that the current screenshot is obtained by the texture data in the rendering scene, and there may be a rendering scene screen. At this time, the screenshot can be parsed in the above manner to obtain data such as the screenshot image and the field of view.

[0108] S3: calling a player model according to the field of view angle.

[0109] After reading the field of view angle, the virtual reality device 500 can also call the player model according to the read field of view angle. The player model is used to display the screenshot image content in the rendering scene, so the player model is a surface model pre-established in the rendering scene according to the field of view angle.

[0110] Since the images that the user can see through the virtual reality device 500 are in a spherical area under the field of view, the player model can be set to a spherical shape in order to obtain a better immersive experience. For different field of view angles, the spherical range of the player model is also different. For example, the spherical shape range of a wide-angle virtual reality device is larger than that of a conventional virtual reality device.

[0111] In the virtual reality device 500, player models suitable for corresponding field of view angles can be pre-stored, that is, multiple spherical models of different specifications are provided. After the virtual reality device 500 reads the current field of view angle, it can match the model list according to the value of the field of view angle, and extract the matching player model to add it to the rendering scene for displaying the screenshot image.

[0112] S4: Mapping the screenshot image to the player model.

[0113] After calling the player model, the virtual reality device 500 can also map the screenshot image to the player model, that is, display the screenshot image on the player model through the position relationship of the pixels in the image. Since the screenshot image to be played is a flat image, and the player model display area is a curved surface, the virtual reality device 500 can merge or add pixels in the screenshot image to display the flat image in the curved surface area.

[0114] In the embodiment of the present application, by mapping the screenshot image to the player model, the screenshot image can be displayed on the player model after the mapping is completed, such as Fig.13 As shown. Since the spherical area corresponding to the player model can cover the field of view, the player model can make the screenshot image full of the user's field of view, that is, the screenshot image replaces the skybox image in the traditional browsing interface, alleviating the overlapping display of the rendered scene image and the skybox image in the screenshot image, so as to bring a better immersive experience.

[0115] In the above embodiment, the virtual reality device 500 can obtain the screenshot image and the field of view by parsing the screenshot image file, and thus call the player model according to the field of view. In order to obtain the field of view in the screenshot image file, in some embodiments, the virtual reality device 500 can also receive a screenshot instruction input by the user.

[0116] According to the rules set in the operating system of the virtual reality device 500, the user can input the screenshot command in different ways. For example, the user can double-click the home button on the handle to call out the shortcut center window, and click the "Screenshot" option in the shortcut center window to input the screenshot command.

[0117] The user can also input the screenshot command through the shortcut key operation. The shortcut key can be set according to the physical key setting on the virtual reality device 500, that is, different virtual reality devices 500 can be provided with different shortcut key combinations for implementing the screenshot operation. For example, the user can input the screenshot command through the combination of the "power button" and the "volume +" key. For the virtual reality device 500 with an interactive device such as an external handle, the user can also input the screenshot command through the combination of the handle button and the virtual reality device 500 button.

[0118] For some virtual reality devices 500, users can also use other interactive devices or interactive systems to complete the input of control instructions. For example, an intelligent voice system can be built into the virtual reality device 500, and the user can input voice information such as "screenshot" or "I want to keep the current screen" through an audio input device such as a microphone. The intelligent voice system recognizes the meaning of the voice information by converting, analyzing, and processing the user's voice information, and generates a control instruction based on the recognition result to control the virtual reality device 500 to perform the screenshot operation.

[0119] After receiving the screenshot command input by the user, the virtual reality device 500 can obtain the screenshot image from the rendering scene in response to the screenshot command, that is, by performing image capture on the rendering scene, thereby obtaining the screenshot image. Among them, the rendering scene refers to a virtual scene constructed by the rendering engine of the virtual reality device 500 through a rendering program. For example, the virtual reality device 500 based on the unity 3D rendering engine can construct a unity 3D scene when presenting a display screen. In the unity 3D scene, various virtual objects and function controls can be added to render a specific usage scenario. For example, when playing multimedia resources, a display panel can be added to the unity 3D scene, and the display panel is used to present the multimedia resource screen. At the same time, virtual object models such as seats, audio, and characters can also be added to the unity 3D scene to create a theater effect.

[0120] In order to output the rendered images, the virtual reality device 500 can also set a virtual camera in the unity 3D scene. For example, the virtual reality device 500 can set a left eye camera and a right eye camera in the unity 3D scene according to the positional relationship between the user's eyes, and the two virtual cameras can simultaneously shoot objects in the unity 3D scene, thereby outputting the rendered images to the left display and the right display respectively. In order to obtain a better immersive experience, the angles of the two virtual cameras in the unity 3D scene can be adjusted in real time with the posture sensor of the virtual reality device 500, so that when the user wears the virtual reality device 500 to move, the rendered images in the unity 3D scene under different viewing angles can be output in real time.

[0121] Therefore, the virtual reality device 500 can use the virtual camera (Camera) set in the rendering scene, output the TargetTexture of the Camera to the RenderTexture, capture images of virtual objects and media images in the rendering scene, and thus directly obtain images in key areas from the rendering scene.

[0122] In order to capture images of key areas, the virtual reality device 500 can directly obtain screenshot images through the virtual display camera in the rendering scene, or through a special virtual screenshot camera. That is, a virtual screenshot camera can be set in the unity 3D scene, and the virtual screenshot camera is set to only capture images of key areas. After the user enters the screenshot command, the virtual screenshot camera can execute image capture and output the target image. Since the acquired image is an image directly generated by each virtual control or object in the rendering scene and has not been distorted or processed, the obtained target image will not be affected by distortion, and the image in the rendering scene can be better preserved.

[0123] In the process of obtaining the screenshot image, the virtual reality device 500 may also record the position information when performing the image shooting, including recording the shooting position and field of view angle during shooting. For example, when using a screenshot camera to shoot an image, the position of the screenshot camera in the current rendering scene may be recorded, and the field of view angle of the screenshot camera may be recorded at the same time. Since the recorded position information may be used for parsing when playing the screenshot image file, thereby restoring the virtual rendering scene, in order to obtain a better restoration effect, the position information of the virtual objects in the rendering scene may also be recorded, such as recording the position of the display panel.

[0124] After recording the field of view, the virtual reality device 500 can save the screenshot image. Since the image obtained by image capture is a kind of pixel information, it needs to be converted into a picture file of a specific format before it can be read, called and sent by the virtual reality device 500 or other devices in subsequent use. Therefore, when saving the screenshot image, the virtual reality device 500 can encode, compress and other processes on the screenshot image to generate a picture file. The generated picture file can be saved in the file save path specified in the memory of the virtual reality device 500, or sent to other devices through a communication connection. In the process of storing the screenshot picture file, the recorded location information can also be integrated into the picture file so that it can be parsed and obtained during subsequent playback.

[0125] It can be seen that in the above embodiment, after obtaining the screenshot operation instruction, the virtual reality device 500 can obtain an undistorted screenshot image by shooting an image in the rendering scene, so as to generate a screenshot image file. This screenshot method can not only avoid the distortion processing affecting the image content, but also enable the image file obtained by the screenshot to carry position information such as the field of view angle and the shooting position, which is convenient for parsing when playing the file.

[0126] In the above embodiment, the screenshot image can be presented by a pre-created player model that conforms to the current field of view. However, since the screenshot image can come from different virtual reality devices, and different virtual reality devices may have different fields of view, that is, the virtual reality device 500 used by the user may not have a player model corresponding to the field of view pre-stored. Fig.14 As shown, in some embodiments, in the step of calling the player model according to the field of view angle, the controller is further configured to:

[0127] S310: Obtain a player model list;

[0128] S320: Match a player model in the player model list using the field of view angle;

[0129] S330: If a player model under the viewing angle is matched in the player model list, calling the matched player model;

[0130] S340: If no player model under the field of view angle is matched in the player model list, create a player model according to the field of view angle.

[0131] After reading the field of view angle, the virtual reality device 500 can match the player model according to the read field of view angle. During the matching process, the virtual reality device 500 can first obtain a player model list, and then complete the model matching through the player model list. The player model list may include multiple entries, each of which records the storage address of a player model in the current virtual reality device 500 and its corresponding field of view angle.

[0132] The controller then matches each item in the player model list according to the field of view. When the field of view value recorded in any item is equal to the read field of view value, it means that the player model under the field of view is matched in the player model list, so the model can be called and loaded into the rendering scene. If there is no item in the player model list that is equal to the field of view value, it is determined that the player model under the field of view is not matched in the player model list. At this time, in order to continue playing the current screenshot, the virtual reality device can create a new player model based on the current field of view and the screenshot.

[0133] Among them, the player model can be created by the virtual reality device 500 through the rendering engine, or it can be created through a third-party application. For example, the modeling tools include but are not limited to: Maya software, 3dsmax software, ZBrush software, Headus UVLayout software, BodyPaint 3D software, PS software and Vray renderer software. The model creation can be completed by a third-party application, or it can be completed by multiple third-party applications in collaboration. After the model is successfully created, it is imported into the unity development project and added to the rendering scene, so that the screenshot is presented in the user's entire field of view, and the same experience effect as in the screenshot scene is obtained.

[0134] In some embodiments, Fig.15 As shown, in order to create a new player model, in the step of creating a player model according to the field of view angle, the controller is further configured as follows:

[0135] S341: Obtain the resolution of the screenshot to be played;

[0136] S342: Calculating a curved surface radius according to the resolution and the field of view angle;

[0137] S343: Creating a spherical coordinate system in the rendering scene according to the curved surface radius;

[0138] S344: intercepting the curved surface range under the field of view angle in the spherical coordinate system to generate a player model.

[0139] Since the player model is part of a sphere, when creating the player model, the player model can be represented by a spherical coordinate system. The spherical coordinate system is a three-dimensional coordinate system used to determine the position of points, lines, surfaces, and bodies in three-dimensional space. It uses the coordinate origin as a reference point and consists of azimuth, elevation, and distance.

[0140] like Fig.16 As shown, in the spherical coordinate system, ρ can be used to represent the distance from the spherical point to the origin, and the value range of ρ is 0≤ρ≤+∞. On the ρz plane, the angle of deflection from the positive half axis of the z axis to ρ is The value range is The angle deflected from the x-axis to the plane is θ, and the value range of θ is 0≤θ≤2π. The player model can be generated by defining the surface range under the field of view angle in the spherical coordinate system.

[0141] because The maximum value on the sphere is 180°. When the viewing angle of the Camera is FOV, the vertical viewing range is FOV. In the spherical coordinate system The starting value is (180-FOV) / 2=90-FOV / 2, The end value is 90-FOV / 2+FOV=90+FOV / 2; that is The vertical range of the area is (90-FOV / 2, 90+FOV / 2). Converting this range into radians, we get The value range is (90-FOV / 2) / 180 to (90+FOV / 2) / 180.

[0142] Similarly, the maximum value of θ on the sphere is 360°, and the visible range in the horizontal direction is FOV, then the starting value of θ is (360-FOV) / 2=180-FOV / 2, and the ending value of θ is 180-FOV / 2+FOV=180+FOV / 2; that is, the horizontal area of ​​θ ranges from 180-FOV / 2 to 180+FOV / 2. Converting this range into radians, the value range of θ is (180-FOV / 2) / 180 to (180+FOV / 2) / 180.

[0143] Based on the content of the spherical coordinate system, when the player model corresponding to the current field of view is not matched, the virtual reality device 500 can determine the size of the current picture and the state information most suitable for displaying the current picture by reading the resolution of the screenshot picture to be played, such as the space shape suitable for display, viewing distance and other data. Then, the surface radius is calculated according to the resolution and field of view, that is, ρ in the spherical coordinate system is determined, so as to create a spherical coordinate system in the rendering scene according to the calculated surface radius.

[0144] The virtual reality device 500 then intercepts the surface range in the spherical coordinate system according to the value range corresponding to the field of view angle to generate a player model. That is, after creating the spherical coordinate system, the vertical area range within (90-FOV / 2, 90+FOV / 2) and the horizontal area range within (180-FOV / 2, 180+FOV / 2) are intercepted in the spherical coordinate system. After the player model is generated, the virtual reality device 500 can load the generated player model into the rendering scene for presenting the screenshot image.

[0145] It can be seen that in the above embodiment, the virtual reality device 500 can call or create a player model through field of view matching, so as to play the screenshot image through the player model adapted to the current screenshot image. By using the screenshot image content to replace the skybox image in the browsing interface, the inconsistency problem between the skybox image and the screenshot image content can be alleviated, and the user's immersive experience can be improved.

[0146] In some embodiments, in order to map the screenshot image to the player model, the controller is further configured to:

[0147] S410: constructing two-dimensional texture coordinates according to the screenshot image, and constructing spherical texture coordinates according to the player model;

[0148] S420: Setting the range of the two-dimensional texture coordinates and the spherical texture coordinates;

[0149] S430: Mapping the points in the two-dimensional texture coordinates to the spherical texture coordinates according to the normal vector coordinates of each point on the spherical texture coordinates.

[0150] When the player model is successfully created, the virtual reality device 500 can display the screenshot image on the player model. Since the screenshot image is a flat texture shape, the shape that can be displayed on the player model is a spherical texture shape. Fig.17 As shown, in this embodiment, mapping the screenshot image onto the playback model is to map a plane texture onto a spherical surface.

[0151] Specifically, the virtual reality device 500 can first construct two-dimensional texture coordinates based on the screenshot image, and construct spherical texture coordinates based on the player model, and then determine the correspondence between the two-dimensional texture coordinates and the spherical texture coordinates by setting the range of the two-dimensional texture coordinates and the spherical texture coordinates, so as to map the points in the two-dimensional texture coordinates to the spherical texture coordinates according to the normal vector coordinates of each point on the spherical texture coordinates.

[0152] For example, Fig.18 As shown, the two-dimensional texture coordinates can be represented by the outer box in the figure, and their range is (u, v)min=(0, 0); (u, v)max=(1, 1). The circle in the box in the figure represents the spherical normal vector coordinates, and their x, y components range is (x, y)min=(-1, -1); (x, y)max=(1, 1). It can be seen that mapping the two-dimensional texture coordinates to the spherical coordinates is the mapping of two sets of coordinates, that is, mapping the interval (x, y)min-(x, y)max to the interval (u, v)min-(u, v)max.

[0153] In order to improve the accuracy of mapping, the virtual reality device 500 can also segment the screenshot image according to the resolution of the screenshot image; then perform texture sampling on each sub-area obtained by the segmentation; and use the inverse sine function to convert the coordinates of the texture sampling point in the sub-area into spherical point coordinates.

[0154] For example, the virtual reality device 500 can use the inverse sine function y=arcsin(x). The domain of the inverse sine function y=arcsin(x) is x=(-1, 1), and the range is y=(-π / 2, π / 2). Combined with the radius of the three-dimensional sphere being r, the horizontal rotation angle being h([0, 2π]), and the vertical rotation angle being p([-π / 2, π / 2]), the three-dimensional coordinates of a point on the sphere are After the reverse transformation, we can get θ = arcsin (z / r),

[0155] Therefore, when the vertical rotation angle p corresponds to the V direction of the texture, and the horizontal rotation angle h corresponds to the U direction of the texture, the range of UV is [0,1]. After knowing the spherical coordinates xyz and the radius r, the texture coordinates corresponding to the spherical point are: U = arctan (y / x) / 2 / π; V = arcsin (z / r) / π + 0.5. Based on the above results, the virtual reality device 500 can convert the two-dimensional texture coordinates into spherical coordinates to display the screenshot image on the player model.

[0156] In the above embodiment, when displaying the screenshot, the virtual reality device 500 can use the corresponding player model to present the screenshot image according to the scene in the screenshot. However, in actual applications, some virtual reality devices 500 can output multiple types of screenshots, for example, 2D pictures, 3D pictures, and panoramic pictures. For different types of pictures, the virtual reality device 500 needs to use different playback methods, such as Fig.19 As shown, in some embodiments, in the step of parsing the screenshot image file to be played, the controller is further configured to:

[0157] S210: Reading the screen type of the screenshot to be played;

[0158] S220: If the image type is the first type, executing the step of parsing the screenshot image file to be played to obtain the screenshot image and the field of view;

[0159] S230: If the picture type is the second type, hide the rendering background in the play interface to display the screenshot in the rendering scene.

[0160] Since the 2D images are regular images among the types of images output by the screenshots of the virtual reality device 500, they can be displayed according to the above screenshot image playback method. The 3D images include two parts of the screen corresponding to the left eye and the right eye, respectively. The two parts of the screen can be presented in the screenshot image in a specific arrangement. For example, the left eye image of the left-right 3D image is on the left side of the image, and the right eye image is on the right side of the image; the left eye image of the top-bottom 3D image is on the top of the image, and the right eye image is on the bottom of the image. Therefore, when playing the 3D image, the screen displayed by the virtual reality device 500 in the player model will include the left eye image and the right eye image, that is, a display error problem occurs.

[0161] In this regard, in this embodiment, the virtual reality device 500 can read the screen type of the screenshot image when parsing the screenshot image to be played. The screenshot image type includes a first type and a second type, the first type can be a regular image such as a 2D image, and the second type is a picture with multiple screen parts such as a left eye image and a right eye image in a 3D image or the like. By reading the screen type of the screenshot image, if the screen type is the first type, the screenshot image file to be played can be parsed in the manner in the above embodiment to obtain the screenshot image screen and field of view.

[0162] If the screen type is the second type, since the screenshot image file itself has a 3D effect, the screenshot image can be directly displayed in the rendering scene to bring an immersive experience to the user. Obviously, in the process of displaying the screenshot image, the left eye image in the 3D image can be output to the left display by setting; the right eye image in the 3D image can be output to the right display. For example, the image is segmented for the 3D image, the left eye image and the right eye image are separated, and they are displayed in the rendering scene at the same time. By setting the left eye image to be visible to the virtual display camera corresponding to the left display, and setting the right eye image to be visible to the virtual display camera corresponding to the right display, the image screen is output in the left display and the right display. When displaying a 3D picture, the virtual reality device 500 can also hide the original sky box area in the rendering scene, retaining only the screenshot image content, to alleviate the interference of the sky box screen on the screenshot picture.

[0163] In addition, for panoramic image types, since they can be directly displayed on a spherical surface, when it is determined that the type of screenshot image to be played is a panoramic image, the virtual reality device 500 can directly display the panoramic image on the spherical surface corresponding to the player model, thereby obtaining a surround effect.

[0164] As can be seen from the above embodiments, the virtual reality device 500 can display the screenshot image content in the rendering scene after mapping the screenshot image to the player model. In order to output the image content in the rendering scene to the display for display, Fig. 20 As shown, in some embodiments, after the step of mapping the screenshot image to the player model, the controller is further configured to:

[0165] S501: Adding the player model mapped with the screenshot image in the rendering scene;

[0166] S502: Setting the position of the player model in the rendering scene so that the screenshot image covers the entire field of view;

[0167] S503: photographing the current rendering scene through a virtual display camera to output the screenshot image.

[0168] After mapping the screenshot image to the player model, the virtual reality device 500 can add the player model with the screenshot image to the rendering scene. Then, by setting the position of the player model in the rendering scene, the screenshot image can cover the field of view of the current virtual reality device 500, that is, Fig.21 As shown, the screenshot image fills the user's field of view.

[0169] The virtual reality device 500 then adds interactive controls to the rendering scene, such as adding a screenshot image list, "previous", "next" and other UI interactive controls, so that the user can select, switch and perform other operations on the displayed content through the UI interactive controls to control the playback process.

[0170] After adding all the contents of the playback interface to the rendering scene, the virtual reality device 500 can also shoot the current rendering scene through the virtual display camera, so as to output the display screen in real time. Two virtual display cameras can be set in the rendering scene of the virtual reality device 500, and the two virtual display cameras can correspond to the left and right displays of the virtual reality device 500 respectively, so as to simulate the viewing method of the human eye and present a three-dimensional effect to the user. The virtual display camera can adjust the shooting angle following the posture sensor in the virtual reality device 500, so as to adjust the picture content according to the user's movements when wearing it, so that the user can get a better immersive experience.

[0171] Based on the above virtual reality device 500, some embodiments of the present application further provide a method for playing screenshot images, the method comprising the following steps:

[0172] S1: Obtaining a control instruction input by a user for playing a screenshot image;

[0173] S2: In response to the control instruction, parsing the screenshot image file to be played to obtain the screenshot image picture and field of view;

[0174] S3: calling a player model according to the field of view angle, where the player model is a surface model pre-established in the rendering scene according to the field of view angle;

[0175] S4: Mapping the screenshot image to the player model.

[0176] It can be seen from the above technical solution that the screenshot picture playback method provided in the above embodiment can parse the screenshot picture file after obtaining the control instruction input by the user, so as to obtain the screenshot image screen and the field of view; and then call the player model according to the field of view, so as to map the screenshot image screen to the player model. The screenshot picture playback method can use the player model to display the screenshot picture. The player model has a display area shape corresponding to the field of view angle, so that the screenshot picture can fill the entire field of view during playback, alleviating the confusion between the sky box picture and the screenshot picture. In addition, the screenshot picture scene is restored through a surface model adapted to the screenshot picture, so as to obtain a better immersive experience.

[0177] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A virtual reality device, characterized in that: include: A display configured to display a user interface, wherein the user interface includes a picture imaging area and a sky box area, wherein the picture imaging area is used to present multimedia content; The skybox area is located around the image area and is used to present the rendered background content; The controller is configured as: Obtaining the control command input by the user for playing the screenshot image; In response to the control instruction, reading a picture type of the screenshot picture to be played, the picture type being a 2D picture or a 3D picture; If the picture type is a 2D picture, parse the screenshot picture file to be played to obtain the screenshot image picture and field of view; Calling a player model according to the field of view angle, wherein the player model is a surface model pre-established according to the field of view angle in a rendering scene; Mapping the screenshot image to the player model; If the picture type is a 3D picture, the rendering background in the playback interface is hidden to display the screenshot in the rendering scene.

2. The virtual reality device according to claim 1, characterized in that: Before obtaining the control instruction input by the user for playing the screenshot image, the controller is further configured to: Receive the screenshot command input by the user; In response to the screenshot instruction, performing image capture on the current rendering scene to obtain a screenshot image; Recording a shooting position and a field of view angle in the rendering scene when performing image shooting; The screenshot image and the field of view are saved to generate a screenshot picture file.

3. The virtual reality device according to claim 1, characterized in that: In the step of calling the player model according to the field of view angle, the controller is further configured to: Get the player model list; Matching a player model in the player model list using the field of view angle; If a player model under the viewing angle is matched in the player model list, calling the matched player model; If no player model under the field of view angle is matched in the player model list, a player model is created according to the field of view angle.

4. The virtual reality device according to claim 3, characterized in that: In the step of creating a player model according to the field of view angle, the controller is further configured to: Get the resolution of the screenshot to be played; Calculate the radius of the curved surface according to the resolution and the field of view; Creating a spherical coordinate system in the rendering scene according to the surface radius; The curved surface range under the field of view angle is intercepted in the spherical coordinate system to generate a player model.

5. The virtual reality device according to claim 1, characterized in that: In the step of mapping the screenshot image to the player model, the controller is further configured to: Constructing two-dimensional texture coordinates according to the screenshot image, and constructing spherical texture coordinates according to the player model; Setting the range of the two-dimensional texture coordinates and the spherical texture coordinates; According to the normal vector coordinates of each point on the spherical texture coordinates, the points in the two-dimensional texture coordinates are mapped to the spherical texture coordinates.

6. The virtual reality device according to claim 5, characterized in that: In the step of mapping the points in the two-dimensional texture coordinates to the spherical texture coordinates, the controller is further configured to: Segmenting the screenshot image according to the resolution of the screenshot image; Perform texture sampling for each sub-region obtained by segmentation; Use the inverse sine function to convert the texture sampling point coordinates within the sub-region into spherical point coordinates.

7. The virtual reality device according to claim 1, characterized in that: After the step of mapping the screenshot image to the player model, the controller is further configured to: Adding the player model with the screenshot image mapped to the rendering scene; Setting the position of the player model in the rendering scene so that the screenshot image covers the entire field of view; The current rendering scene is photographed by a virtual display camera to output the screenshot image.

8. A screenshot picture playing method, characterized in that: Applied to a virtual reality device, the virtual reality device comprising a display and a controller; The display is configured to display a user interface, the user interface comprising a picture imaging area and a sky box area, the picture imaging area is used to present multimedia content; the sky box area is located around the image area and is used to present rendered background content; The controller is configured to: Receive the screenshot command input by the user; In response to the screenshot instruction, performing image capture on the current rendering scene to obtain a screenshot image; Recording a shooting position and a field of view angle in the rendering scene when performing image shooting; Saving the screenshot image and the field of view to generate a screenshot picture file; The screenshot picture playing method comprises: Obtaining the control command input by the user for playing the screenshot image; In response to the control instruction, reading a picture type of the screenshot picture to be played, the picture type being a 2D picture or a 3D picture; If the picture type is a 2D picture, parse the screenshot picture file to be played to obtain the screenshot image picture and field of view; Calling a player model according to the field of view angle, wherein the player model is a surface model pre-established according to the field of view angle in a rendering scene; Mapping the screenshot image to the player model; If the picture type is a 3D picture, the rendering background in the playback interface is hidden to display the screenshot in the rendering scene.

Images