Virtual reality device and high-definition screenshot method

Abstract

This application provides a virtual reality device and a high-definition screenshot method. During the screenshot process, the method first receives a screenshot control command input from the user, then retrieves a target image of a specified resolution from the rendering scene according to the command, and saves the target image to obtain a screenshot image file. This method allows for image capture of an image area within the rendering scene during the screenshot process, ensuring the obtained image contains more of the main content, reducing the occupation of the screenshot image area by non-essential elements in the rendering scene, and improving the clarity of the main content.

Description

[0001] This application claims priority to Chinese Patent Application No. 202110065015.6, filed on January 18, 2021, entitled "A Virtual Reality Device and a Quick Interaction Method", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of virtual reality technology, and in particular to a virtual reality device and a high-definition screenshot method. Background Technology

[0003] Virtual Reality (VR) technology is a display technology that uses computers to simulate virtual environments, thereby creating a sense of immersion. A VR device is a device that uses virtual display technology to present virtual images to the user. Typically, a VR device includes two display screens to display virtual content, corresponding to the user's left and right eyes respectively. When the content displayed on the two screens comes from different perspectives of the same object, it can provide the user with a stereoscopic viewing experience.

[0004] Virtual reality (VR) devices can capture screenshots to output the displayed content as images for display on other display devices or VR devices. During the screenshot process, VR devices typically capture the currently displayed content on the screen directly. However, the image obtained by directly capturing the screenshot deviates from what the user actually sees, affecting the final quality of the screenshot. For example, the lenses in the optical components of VR devices have a structure that is thicker in the middle and thinner at the edges, causing the image viewed by the user to be displayed correctly in the center but distorted at the edges.

[0005] To avoid discomfort caused by distortion, virtual reality devices are designed so that the central area of ​​the screen is visible while the edges are invisible. This results in a screenshot showing a larger area than the human eye can see. Therefore, by directly capturing the content displayed on the screen, the area of ​​interest to the user occupies only a small portion of the entire image, leading to lower clarity in the screenshot showing the main content. Summary of the Invention

[0006] This application provides a virtual reality device and a high-definition screenshot method to solve the problem of low clarity of the main content in traditional screenshot methods.

[0007] In a first aspect, this application provides a virtual reality device, including a display and a controller. The display is configured to display a user interface, which includes an image area and a skybox area. The image area is used to present multimedia content; the skybox area is located around the image area and is used to present rendered background content. The controller is configured to execute the following program steps:

[0008] Receive user input for screen capture control commands;

[0009] In response to the control command, a target image is obtained from the rendering scene, wherein the target image is an image of a specified resolution obtained by performing image capture on an image region in the rendering scene;

[0010] Save the target image to generate a screenshot file.

[0011] Secondly, this application also provides a virtual reality device, including a display and a controller. The display is configured to display a user interface, which includes an image area and a skybox area. The image area is used to present multimedia content; the skybox area is located around the image area and is used to present rendered background content. The controller is configured to execute the following program steps:

[0012] Get the playback command input by the user to display the screenshot image file;

[0013] In response to the playback command, the screenshot image file is parsed to obtain the target image;

[0014] The target image is displayed within the image area.

[0015] Thirdly, this application also provides a high-definition screenshot method, applied to the aforementioned virtual reality device, the high-definition screenshot method comprising:

[0016] Receive user input for screen capture control commands;

[0017] In response to the control command, a target image is obtained from the rendering scene, wherein the target image is an image of a specified resolution obtained by performing image capture on an image region in the rendering scene;

[0018] Save the target image to generate a screenshot file.

[0019] As can be seen from the above technical solutions, the virtual reality device and high-definition screenshot method provided in this application can first receive screenshot control commands input by the user during the screenshot process, then obtain a target image of a specified resolution from the rendering scene according to the screenshot control commands, and save the target image to obtain a screenshot image file. The method can capture images of image areas in the rendering scene during the screenshot process, ensuring that the obtained image contains more of the main content, alleviating the occupation of the screenshot image area by non-main scenes in the rendering scene, and improving the clarity of the main content. Attached Figure Description

[0020] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the display system structure including a virtual reality device in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the global interface of the VR scene in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of the recommended content area of ​​the global interface in this embodiment of the application;

[0024] Figure 4 This is a schematic diagram of the application shortcut operation entry area of ​​the global interface in this application embodiment;

[0025] Figure 5 This is a schematic diagram of the floating objects on the global interface in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram illustrating accessing the quick access center via the status bar in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the quick center window in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram illustrating access to the quick access center via a button in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram illustrating the start of a screenshot in an embodiment of this application;

[0030] Figure 10 This is a schematic diagram of the text window that prompts a successful screenshot in an embodiment of this application;

[0031] Figure 11 This is an illustration of the effect of directly capturing screen content;

[0032] Figure 12 This is a flowchart illustrating the high-definition screenshot method in the embodiments of this application;

[0033] Figure 13 This is a schematic diagram illustrating the screenshot effect of the high-definition screenshot method in the embodiments of this application;

[0034] Figure 14 This is a schematic diagram illustrating the process of setting up a virtual screenshot camera in an embodiment of this application;

[0035] Figure 15 This is a schematic diagram illustrating the process of setting the sampling resolution of the virtual screenshot camera in an embodiment of this application;

[0036] Figure 16 This is a schematic diagram of the interface for setting the encoding method in an embodiment of this application;

[0037] Figure 17 This is a schematic diagram of the process for saving the target image in an embodiment of this application;

[0038] Figure 18 This is a schematic diagram of the media asset playback interface process in the embodiments of this application;

[0039] Figure 19 This is a schematic diagram of the media asset playback interface screenshot process in the embodiments of this application. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0041] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete technical solution on its own.

[0042] It should be understood that the terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate, for example, to allow implementation in orders other than those given in the embodiments illustrated or described in this application.

[0043] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0044] As used in this application, the term "module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0045] Throughout this specification, references to "multiple embodiments," "some embodiments," "one embodiment," or simply "embodiment" indicate that a specific feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment. Therefore, phrases such as "in multiple embodiments," "in some embodiments," "in at least another embodiment," or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment. Furthermore, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner. Therefore, without limitation, a specific feature, structure, or characteristic shown or described in connection with one embodiment may be combined, in whole or in part, with features, structures, or characteristics of one or more other embodiments. Such modifications and variations are intended to be included within the scope of this application.

[0046] In this application embodiment, the virtual reality device 500 generally refers to a display device that can be worn on a user's face to provide an immersive experience, including but not limited to VR glasses, augmented reality (AR) devices, VR gaming devices, mobile computing devices, and other wearable computers. Some embodiments of this application use VR glasses as an example to illustrate the technical solution; it should be understood that the provided technical solution can also be applied to other types of virtual reality devices. The virtual reality device 500 can operate independently or be connected as an external device to other smart display devices, wherein the display device can be a smart TV, computer, tablet computer, server, etc.

[0047] The virtual reality device 500, when worn on a user's face, displays media assets, providing close-up images to the user's eyes for an immersive experience. To present the media assets, the virtual reality device 500 can include multiple components for displaying the images and for wearing on the face. Taking VR glasses as an example, the virtual reality device 500 can include a shell, temples, an optical system, a display component, a posture detection circuit, and an interface circuit. In practical applications, the optical system, display component, posture detection circuit, and interface circuit can be housed within the shell to display the specific images; the temples are connected to both sides of the shell for wearing on the user's face.

[0048] When in use, the attitude detection circuit has built-in attitude detection components such as gravity acceleration sensor and gyroscope. When the user's head moves or turns, the user's attitude can be detected and the detected attitude data can be transmitted to the controller and other processing components, so that the processing components can adjust the specific screen content in the display component according to the detected attitude data.

[0049] It should be noted that the specific content presented varies depending on the type of virtual reality device 500. For example, as... Figure 1 As shown, for some thin and light VR glasses, the built-in controller generally does not directly participate in the control process of the displayed content. Instead, it sends the posture data to an external device, such as a computer, for processing. The external device then determines the specific screen content to be displayed and sends it back to the VR glasses to display the final screen.

[0050] In some embodiments, the virtual reality device 500 can be connected to the display device 200 and a network-based display system can be built between it and the server 400. Data interaction can be performed in real time between the virtual reality device 500, the display device 200 and the server 400. For example, the display device 200 can obtain media data from the server 400 and play it, and transmit specific screen content to the virtual reality device 500 for display.

[0051] The display device 200 can be a liquid crystal display, an OLED display, or a projection display device. The specific type, size, and resolution of the display device are not limited. Those skilled in the art will understand that the display device 200 can be modified in terms of performance and configuration as needed. The display device 200 can provide broadcast television reception functionality and may also include, but is not limited to, intelligent network television functionality with computer support, including but not limited to, internet television, smart television, and Internet Protocol television (IPTV).

[0052] Display device 200 and virtual reality device 500 also communicate with server 400 via various communication methods. Display device 200 and virtual reality device 500 can communicate via local area network (LAN), wireless local area network (WLAN), and other networks. Server 400 can provide display device 200 with various content and interactive features. For example, display device 200 can interact by sending and receiving information, as well as electronic program guides (EPGs), receiving software updates, or accessing remotely stored digital media libraries. Server 400 can be a cluster or multiple clusters, and may include one or more types of servers. Other network services such as video-on-demand and advertising services can be provided through server 400.

[0053] During data interaction, users can operate the display device 200 via the mobile terminal 300 and the remote control 100. The mobile terminal 300 and the remote control 100 can communicate with the display device 200 via a direct wireless connection or a non-direct connection. Specifically, in some embodiments, the mobile terminal 300 and the remote control 100 can communicate with the display device 200 via direct connection methods such as Bluetooth or infrared. When sending control commands, the mobile terminal 300 and the remote control 100 can directly transmit the control command data to the display device 200 via Bluetooth or infrared.

[0054] In other embodiments, the mobile terminal 300 and the remote controller 100 can also access the same wireless network as the display device 200 via a wireless router to establish a non-direct connection communication with the display device 200 through the wireless network. When sending control commands, the mobile terminal 300 and the remote controller 100 can first send the control command data to the wireless router, and then the wireless router forwards the control command data to the display device 200.

[0055] In some embodiments, users can also use the mobile terminal 300 and the remote control 100 to interact directly with the virtual reality device 500. For example, the mobile terminal 300 and the remote control 100 can be used as controllers in a virtual reality scene to achieve functions such as motion-sensing interaction.

[0056] In some embodiments, the display component of the virtual reality device 500 includes a display screen and driving circuitry associated with the display screen. To present a concrete image and provide a stereoscopic effect, the display component may include two display screens, corresponding to the user's left and right eyes respectively. When presenting a 3D effect, the content displayed on the left and right screens will be slightly different, and can respectively display the left and right cameras used during the filming of the 3D source material. Because the user observes the image content with their left and right eyes, a more stereoscopic display image can be observed when the device is worn.

[0057] The optical system in the virtual reality device 500 is an optical module composed of multiple lenses. Positioned between the user's eyes and the display screen, the optical system increases the optical path through the refraction of light signals by the lenses and the polarization effect of polarizers on the lenses, ensuring that the content displayed is clearly presented within the user's field of vision. Furthermore, to accommodate different users' visual acuity, the optical system also supports focusing. This involves adjusting the position of one or more lenses using a focusing component, changing the distance between the lenses, and thus altering the optical path and adjusting the image clarity.

[0058] The interface circuit of the virtual reality device 500 can be used to transmit interactive data. Besides transmitting posture data and display content data, in practical applications, the virtual reality device 500 can also connect to other display devices or peripherals through the interface circuit to achieve more complex functions through data interaction with the connected devices. For example, the virtual reality device 500 can connect to a display device through the interface circuit to output the displayed image to the display device in real time. As another example, the virtual reality device 500 can also connect to a controller through the interface circuit, which can be held and operated by the user to perform related operations in the VR user interface.

[0059] The VR user interface can be presented in various different UI layouts based on user operations. For example, the user interface may include a global interface, such as the global UI after the AR / VR terminal is started. Figure 2 As shown, the global UI can be displayed on the display screen of the AR / VR terminal or on the display device's monitor. The global UI may include a recommended content area 1, a business category extension area 2, an application quick operation entry area 3, and a floating element area 4.

[0060] Recommended content area 1 is used to configure different category tabs; within these tabs, media assets, special features, etc., can be configured; the media assets may include 2D films, educational courses, tourism, 3D, 360-degree panoramic, live streaming, 4K films, applications, games, and other businesses with media asset content, and these tabs can select different template styles and support simultaneous recommendation and arrangement of media assets and special features, such as... Figure 3 As shown.

[0061] In some embodiments, a status bar may also be provided at the top of the recommended content area 1. The status bar may contain multiple display controls, including commonly used options such as time, network connection status, and battery level. The content included in the status bar can be customized by the user; for example, weather or user avatars can be added. The content included in the status bar can be selected by the user to perform corresponding functions. For example, when the user clicks the time option, the virtual reality device 500 may display a time device window on the current interface or jump to a calendar interface. When the user clicks the network connection status option, the virtual reality device 500 may display a WiFi list on the current interface or jump to a network settings interface.

[0062] The content displayed in the status bar can vary depending on the specific settings of the project. For example, the time control can directly display the specific time text information, and display different text at different times; the battery control can display different pattern styles according to the current remaining battery level of the virtual reality device 500.

[0063] The status bar enables users to perform frequently used control operations, allowing for quick setup of the virtual reality device 500. Since the setup process for the virtual reality device 500 involves numerous steps, not all commonly used settings options can typically be displayed in the status bar. Therefore, in some embodiments, the status bar may also include extended options. When an extended option is selected, an extended window is displayed on the current interface, where multiple settings options can be further configured to implement other functions of the virtual reality device 500.

[0064] For example, in some embodiments, when the extended options are selected, a "Quick Center" option can be set in the extended window. After the user clicks the Quick Center option, the virtual reality device 500 can display the Quick Center window. The Quick Center window can include "Screenshot," "Screen Recording," and "Screen Casting" options, used to activate the corresponding functions respectively.

[0065] The Business Category Extension Area 2 supports configuring extended categories for different business types. When a new business type is added, a separate tab can be configured to display the corresponding page content. The extended categories in Business Category Extension Area 2 can also be sorted, and businesses can be removed from the platform. In some embodiments, Business Category Extension Area 2 may include the following content: Movies & TV, Education, Travel, Applications, and My Account. In some embodiments, Business Category Extension Area 2 is configured to display large business category tabs and supports configuring more categories; its icons are configurable, such as... Figure 3 As shown.

[0066] The application quick access area 3 can prioritize pre-installed applications for promotional purposes, and supports configuring special icon styles to replace default icons. Multiple pre-installed applications can be specified. In some embodiments, the application quick access area 3 also includes left-hand and right-hand movement controls for selecting different icons, such as... Figure 4 As shown.

[0067] The floating area 4 can be configured to be located above the left or right diagonal side of a fixed area, and can be configured as a replaceable image or a jump link. For example, after receiving a confirmation operation, the floating area can jump to an application or display a specified function page, such as... Figure 5 As shown. In some embodiments, the suspended object may not be configured with a jump link and may be used simply for visual display.

[0068] In some embodiments, the global UI also includes a status bar at the top for displaying the time, network connection status, battery status, and more quick access points. When an icon is selected using the AR / VR terminal's controller, the icon will display a text prompt that expands to the left and right, and the selected icon will stretch and expand to the left and right according to its position.

[0069] For example, after selecting the search icon, the search icon will display the text "Search" and the original icon. Clicking the icon or text will take you to the search page. Another example is that clicking the favorites icon will take you to the favorites tab, clicking the history icon will display the history page by default, clicking the search icon will take you to the global search page, and clicking the message icon will take you to the message page.

[0070] In some embodiments, interaction can be performed through peripheral devices, such as the controller of an AR / VR terminal, which can operate the user interface of the AR / VR terminal, including a back button; a home button, which can be long-pressed to achieve a reset function; volume up and down buttons; and a touch area that can realize the functions of clicking, sliding, holding and dragging the focus.

[0071] Users can perform interactive operations through the global UI interface and, in some interactive modes, jump to specific interfaces. For example, to play media asset data, users can click on any media asset link icon in the global UI interface to start playing the media asset file corresponding to that link. At this time, the virtual reality device 500 can control the jump to the media asset playback interface.

[0072] After navigating to a specific interface, the virtual reality device 500 can also display a status bar at the top of the playback interface and execute corresponding settings functions according to the set interaction method. For example, ... Figure 6 As shown, when the virtual reality device 500 is playing video media, if the user wants to take a screenshot of the media image, they can click the extended option on the status bar to bring up the extended window, and then click the quick center option in the extended window to display the quick center window on the playback interface of the virtual reality device 500. Figure 7 As shown, finally click the "Screenshot" option in the extended center window to enable the virtual reality device 500 to perform a screenshot operation and store the current display screen as an image.

[0073] The status bar can be hidden when the virtual reality device 500 is playing media assets to avoid obstructing the image. It is triggered when the user performs a specific interactive action. For example, the status bar can be hidden when the user is not using the controller, and displayed when the user is using the controller. To this end, the virtual reality device 500 can be configured to detect the status of the orientation sensor in the controller or the status of any button while playing media assets. When a change in the orientation sensor reading is detected, or a button is pressed, the status bar can be displayed at the top of the playback interface. If no change in the orientation sensor reading is detected within a set time, or a button is not pressed, the status bar is hidden from the playback interface.

[0074] As can be seen in the above embodiments, users can bring up the quick access center through the status bar, and then click the corresponding options in the quick access center window to complete screenshot, screen recording, and screen casting operations. The quick access center window can also be accessed and displayed using other interactive methods. For example, as... Figure 8 As shown, users can bring up the quick access center window by double-clicking the home button on the controller.

[0075] Users can select any icon in the quick access center window to activate the corresponding function. The activation method for each function can be determined based on the actual interaction method of the virtual reality device 500. For example... Figure 9 As shown, after bringing up the quick access center window, the user can move the focus icon down to the screenshot option in the quick access center window by moving the handle down, and then press the "OK" button on the handle to start the screenshot function.

[0076] After executing the above-mentioned screenshot function, the virtual reality device 500 can retrieve the screenshot operation program from its memory and run the program to capture a screenshot of the currently displayed screen. For example, the virtual reality device 500 can run the screenshot program to overlay and composite the display content of all layers to generate an image file of the currently displayed pattern. The generated image file can also be stored according to a pre-defined storage path.

[0077] Because the virtual reality device 500 includes two displays, one for the user's left eye and one for their right, and when displaying media assets, to achieve a stereoscopic viewing effect, the content displayed on the two displays corresponds to the left and right virtual playback cameras in the 3D scene, meaning there are slight differences between the images displayed on the two displays. Therefore, when taking a screenshot, different screens can capture screenshots with different content.

[0078] Therefore, the virtual reality device 500 can detect the display format at the moment of screenshotting. When it detects that the user is using 3D mode, it can perform screenshots on the left and right displays separately, outputting two screenshot images in a single operation. Since the content displayed on the left and right displays is relatively similar in 3D mode, some users do not need two screenshot images. Therefore, to save storage space on the virtual reality device 500, in some embodiments, the screenshot program can specify to capture the content displayed on one of the two displays, for example, specifying to capture the content displayed on the left display, thereby obtaining a single screenshot image and storing it.

[0079] After saving the screenshot, the virtual reality device 500 can also display prompts on the screen, such as... Figure 10 As shown, a notification text window (toast window) can be displayed floating on the playback interface, including the text "Screenshot successful, saved to ××", where "××" is the specific save path. Obviously, the notification text window will automatically disappear after a certain period of time to avoid excessively obscuring the playback interface; for example, the notification text window appears after a successful screenshot and disappears after 2 seconds.

[0080] Furthermore, the prompt text window can dynamically change its content according to the screenshot saving process. For example, after the user confirms the screenshot operation, the prompt text window displays "Screenshot successful, saving screenshot image," and after saving is complete, it displays "Saved to ××."

[0081] It should be noted that since users generally do not want the quick settings center interface to be included in the screenshot image, the quick settings center window can be hidden after the user clicks the screenshot icon in order to capture the media content being played.

[0082] In some embodiments, after the screenshot operation is completed, the screenshot result can be displayed on the playback interface. That is, a display window is displayed floating above the playback interface, showing the screenshot image for the user to view. Furthermore, during the display of the screenshot image, some drawing tools can be displayed in the display window, such as line tools, ellipse tools, rectangle tools, and text tools. Users can click on these drawing tools to perform operations such as masking, annotation, and cropping on the screenshot image to output a better screenshot result.

[0083] As can be seen, in the above embodiments, the virtual reality device 500 can quickly perform a screenshot operation through the quick access center window or a shortcut key, so as to save the screenshot image based on the content displayed by the virtual reality device 500. Depending on the application scenario, the screenshot object can also be different. For example, the virtual reality device 500 can take a screenshot of the content displayed on the monitor, or it can take a screenshot of a portion of the rendered scene.

[0084] When playing media assets, the virtual reality device 500 can render the media asset images, that is, set up a display panel in the rendering scene to display the content of the media asset images. Then, by adding virtual objects such as seats and speakers, a virtual scene is constructed, so as to output effects simulating a cinema or home scene. At this time, if the virtual reality device 500 takes a screenshot of the displayed content, the screenshot will include not only the media asset images but also the rendered virtual object images.

[0085] The virtual reality device 500 can also take screenshots of the display panel in the rendered scene, meaning it can capture only the content of the media asset screen. Specifically, the screenshot method can be to perform a screenshot operation on a specific area of ​​the display panel in the rendered scene, or the virtual reality device 500 can directly extract the frame data of the media asset screen after parsing the media asset data and copy the extracted frame data to obtain an image of a virtual object without rendered virtual items.

[0086] In some embodiments, the virtual reality device 500 can also take screenshots of specific areas in the rendered scene. For example, when a user wearing the virtual reality device 500 moves to any viewpoint, a screenshot can be taken of the display panel area and / or nearby rendered screen content at the current viewpoint, thereby obtaining screenshot content of key areas or user-defined areas.

[0087] Based on the above screenshot operation, users can generate screenshot image files while wearing the virtual reality device 500. Because the main area of ​​interest to the user occupies only a small portion of the screenshot image obtained by directly capturing the screen display content, the clarity of the user's focus area is poor in the screenshot, such as... Figure 11 As shown. Therefore, in some embodiments of this application, a high-definition screenshot method is provided. This method can be applied to a virtual reality device 500. By configuring the controller of the virtual reality device 500, the image clarity obtained by the screenshot taken by the virtual reality device 500 can be improved, such as... Figure 12 As shown, it specifically includes the following:

[0088] S1: Receives user input for taking screenshots.

[0089] Since the virtual reality device 500 may include a display and a controller, the display can be configured to show a user interface during user use, through which the user can perform specific interactive actions to achieve corresponding functions. The controller can be configured to receive various control commands input by the user; for example, to perform a screenshot operation, the user can input control commands for taking a screenshot.

[0090] According to the rules set in the operating system of the virtual reality device 500, users can input screenshot control commands in different ways. For example, users can double-click the home button on the controller to bring up the quick settings window, and then click the "screenshot" option in the quick settings window to input screenshot control commands.

[0091] Users can also input screenshot control commands using keyboard shortcuts. These shortcuts can be configured based on the physical button settings on the virtual reality device 500; different virtual reality devices 500 can have different shortcut combinations for screenshot operations. For example, users can input screenshot control commands using the combination of the "power button" and "volume +". For virtual reality devices 500 connected to external controllers or other interactive devices, users can also input screenshot control commands using a combination of controller buttons and virtual reality device 500 buttons.

[0092] For some virtual reality devices 500, users can also input control commands using other interactive devices or systems. For example, a built-in intelligent voice system can be incorporated into the virtual reality device 500, allowing users to input voice information such as "screenshot" or "I want to save the current image" through audio input devices like microphones. The intelligent voice system identifies the meaning of the voice information by converting, analyzing, and processing it, and generates control commands based on the recognition results to control the virtual reality device 500 to perform the screenshot operation.

[0093] S2: In response to the control command, obtain the target image from the rendering scene.

[0094] Upon receiving a screenshot control command from the user, the virtual reality device can respond by acquiring a target image from the rendered scene. This is achieved by capturing an image of a specified resolution within an image area of ​​the rendered scene. In this embodiment, the image area is a pre-defined region based on the usage environment of the virtual reality device 500. The content of the image area differs depending on the user interface being displayed. For example, when the virtual reality device 500 displays the global UI interface, the image area may be located where the UI controls are situated and may include all UI interface controls. When the virtual reality device 500 displays a media playback interface, the image area may be located within the media playback area, covering the display panel in the rendered scene and some playback-related UI controls.

[0095] In this context, a rendered scene refers to a virtual scene constructed by the rendering engine of the 500 virtual reality device through a rendering program. For example, the 500 virtual reality device, based on the Unity 3D rendering engine, can construct a Unity 3D scene when displaying images. Various virtual objects and functional controls can be added to the Unity 3D scene to render specific usage scenarios. For instance, when playing multimedia resources, a display panel can be added to the Unity 3D scene to display the multimedia resource images. Simultaneously, virtual object models such as seats, speakers, and characters can be added to the Unity 3D scene to create a cinematic effect.

[0096] To output the rendered images, the virtual reality device 500 can also set up virtual cameras within the Unity 3D scene. For example, the virtual reality device 500 can set up a left-eye camera and a right-eye camera in the Unity 3D scene according to the position of the user's eyes. The two virtual cameras can simultaneously capture images of objects in the Unity 3D scene, thus outputting rendered images to the left and right displays respectively. To achieve a better immersive experience, the angles of the two virtual cameras in the Unity 3D scene can be adjusted in real time using the pose sensor of the virtual reality device 500, so that as the user moves while wearing the virtual reality device 500, it can output rendered images of the Unity 3D scene from different viewing angles in real time.

[0097] Therefore, the virtual reality device 500 can utilize a virtual camera set up in the rendering scene, outputting the Camera's TargetTexture to the RenderTexture to capture images of image areas within the rendering scene, thus directly obtaining images within those areas from the rendering scene. For example, if the image area in the current rendering scene is a UI interactive control within the rendering scene, such as... Figure 11 As shown, the image area includes the file list area, tab area, and other file management interface areas, while the area outside the interface area is the skybox area, which includes the background wall, tables, chairs, and other images.

[0098] To capture images of a specific image area, a virtual screenshot camera can be set up in the Unity 3D scene. This virtual camera is configured to capture images only of the image area. After the user inputs a screenshot control command, the virtual screenshot camera can capture the image and output the target image. Because the acquired image is directly generated by the virtual controls or objects in the rendering scene without distortion or other processing, the obtained target image is not affected by distortion and can better preserve the scene in the rendering.

[0099] S3: Save the target image to generate a screenshot image file.

[0100] After acquiring the target image, the virtual reality device 500 can save it. Since an image acquired through image capture is pixel-based, it needs to be converted into a specific image file format before it can be read, retrieved, and sent by the virtual reality device 500 or other devices in subsequent use. Therefore, when saving the target image, the virtual reality device 500 can encode, compress, and process the target image to generate an image file. The generated image file can be saved in a specified file save path within the virtual reality device 500's memory, or sent to other devices via a communication connection.

[0101] As can be seen, in the above embodiments, the virtual reality device 500, after receiving a screenshot operation command, can capture an image of an image region in the rendered scene to obtain an undistorted target image of the image region, which is then used to generate a screenshot image file. For example... Figure 13 As shown, the above screenshot method can not only avoid distortion processing from affecting the image content, but also ensure that all or most of the image content in the screenshot is the content corresponding to the image area, thereby increasing the proportion of the main content in the screenshot and improving the clarity of the main content.

[0102] After capturing an image through a screenshot, the virtual reality device 500 can also present the screenshot to the user. To do this, the user can input a playback command to display the screenshot, and the virtual reality device 500 can respond to this command and play the screenshot. The virtual reality device 500 will either jump to the playback interface after the user inputs the playback command, or, within the playback interface, jump from displaying an image to displaying the screenshot.

[0103] For example, a user can click the icon of a screenshot file in the file list, triggering the virtual reality device 500 to play the selected image. In this case, the user's action of clicking the image file icon is the control command for playing the screenshot. Since the virtual reality device 500 can also display an image list, "previous," and "next" UI interactive controls on the playback interface, when a user displays a screenshot image in the playback interface, clicking the "previous" or "next" UI interactive controls, or clicking any image icon in the image list, can switch to displaying adjacent or selected images. In this case, the user's action of clicking the interactive controls or image icons is the control command for playing the screenshot.

[0104] Furthermore, the control commands for playing screenshots can also be automatically input by the controller based on the operating status of the virtual reality device 500. For example, when a user takes a screenshot using the virtual reality device 500, the device can display the screenshot result after completion. Therefore, when displaying the screenshot result, the virtual reality device 500 can automatically input playback commands for displaying the screenshot image.

[0105] Upon receiving a playback command from the user, the virtual reality device 500 can respond by parsing the screenshot file to be played, including but not limited to decompression and reading pixel information, to obtain the target image corresponding to the screenshot. This means capturing an image of a specified resolution from an image region within the rendered scene during the screenshot process. The parsed target image is then displayed within the image region of the playback interface, completing the display of the screenshot.

[0106] The virtual reality device 500 can display a target image within a rendering scene by adding a display panel control and combining it with other controls to form an image area. To this end, the virtual reality device 500 can also set the size, shape, and position of the display panel within the rendering scene based on information such as the size or resolution of the target image. By setting the display panel, a suitable distance can be maintained between the display panel and the virtual camera. After a screenshot is displayed on the display panel, a suitably sized virtual reality image can be obtained through scene rendering, thus providing the user with a better viewing experience.

[0107] like Figure 14 As shown, in some embodiments, in order to obtain the target image from the rendering scene, the controller is further configured to perform the following program steps:

[0108] S210: Enable the virtual screenshot camera in the rendered scene;

[0109] S220: Set the image area in the rendering scene according to the current user interface type;

[0110] S230: Use the virtual screenshot camera to capture an image of the image area to generate the target image.

[0111] After receiving the screenshot control command input by the user, the virtual reality device 500 can activate the virtual screenshot camera set in the rendering scene to capture images of the image area in the rendering scene. In practical applications, to reduce the waste of the rendering engine's overall processing resources on the virtual reality device 500, the automatic on / off mode of the virtual screenshot camera can be configured after setting it in the rendering scene.

[0112] For example, the virtual screenshot camera is off by default when no screenshot is being taken, but it turns on when the user inputs a screenshot control command. Correspondingly, during the screenshot process, the virtual reality device 500 can also detect the saving progress of the target image; if the target image is saved, the virtual screenshot camera is turned off. To accommodate multiple screenshot operations performed by the user in a short period, the virtual screenshot camera can be set to close after a delay, such as 30 seconds after the scheduled time for completing the screenshot operation (i.e., saving the screenshot image file). Therefore, the virtual screenshot camera can minimize the consumption of rendering engine processing resources while meeting screenshot call performance requirements.

[0113] Since different user interfaces correspond to different image areas in the rendering scene, the image area in the rendering scene can be set according to the current user interface type after enabling the virtual screenshot camera. For example, when a user is browsing a file management interface, the corresponding rendering scene may include UI interface controls and a skybox model as the background. At this time, after receiving the screenshot control command input by the user, the virtual reality device 500 can detect the boundaries of the current UI controls, determine the leftmost, rightmost, topmost, and bottommost boundary positions of the UI interface, and then set the image area according to the determined boundary positions, that is, the image area covers the above boundary positions.

[0114] After defining the image area, the virtual reality device 500 uses a virtual screenshot camera to capture images of the image area to generate the target image. Clearly, during the capture process, in order to capture all the main content within the image area, the virtual screenshot camera's shooting range should be able to cover the entire image area. In some embodiments, the shooting range of the virtual screenshot camera can be set to be equal to the image area's range, so that the captured target image contains only the content corresponding to the image area.

[0115] In some embodiments, users can also customize the specifications of the screenshot as needed, such as... Figure 15 As shown, in the step of obtaining the target image from the rendering scene, the controller is further configured to:

[0116] S241: After receiving the control command, control the display to show the parameter setting interface;

[0117] S242: Obtain screenshot parameters input by the user through the parameter setting interface, the screenshot parameters including screenshot resolution and encoding method;

[0118] S243: Based on the screenshot resolution, set the sampling resolution of the virtual screenshot camera to obtain the target image at the specified resolution.

[0119] After receiving control commands from the user, the virtual reality device 500 can control the monitor to display a parameter setting interface. In this interface, the user can set the image resolution, output format, and other settings. For example, a text input box can be displayed, allowing the user to input the desired screenshot size or resolution. To quickly complete parameter settings, multiple resolution options can be displayed, such as "Normal (720P 1280×720)," "High Definition (1080P 1920×1080)," and "Ultra High Definition (4K 3840×2160)." Users can quickly input screenshot parameters by clicking any option after the parameter setting interface pops up.

[0120] The virtual reality device 500 can obtain screenshot parameters input by the user through a parameter setting interface, and then configure the virtual screenshot camera according to these parameters to output the target image according to the specified screenshot parameters. For example, when the user selects the high-definition option, the image resolution can be set in the RenderTexture by setting the Size, i.e., setting the Size to 1920×1080. Then, based on the input screenshot resolution, the sampling resolution of the virtual screenshot camera is set to 1920×1080 to obtain a target image with a resolution of 1920×1080.

[0121] It should be noted that, in addition to screenshot resolution, the virtual reality device 500 also supports user settings for other screenshot parameters. Therefore, these parameters include screenshot resolution and encoding method. Users can input the encoding method of the screenshot image through the parameter setting interface, so that the virtual reality device 500 can output a screenshot image file in a specified format according to this encoding method. For example, ... Figure 16 As shown, by setting the Color Format to encode the screenshot image in PNG format, the corresponding virtual reality device 500 generates a screenshot image file in PNG format.

[0122] In some embodiments, after the user specifies the encoding method in the parameter setting interface, the virtual reality device 500 can save the target image according to the following steps:

[0123] S310: Outputs the pixel information of the target image to the texture image;

[0124] S320: Output the texture image to a byte array according to the encoding method described above;

[0125] S330: Save the byte array as a screenshot image file.

[0126] After acquiring the target image, the virtual reality device 500 can output the pixel information of the target image to a texture image. Then, according to the encoding rules specified by the user in the parameter settings interface, the texture image is output to a byte array, thereby saving the byte array as a screenshot image file. For example, the virtual reality device 500 can output the pixel information of the RenderTexture to the Texture, and then output the Texture to a byte array according to PNG encoding, finally generating a PNG format screenshot image file.

[0127] During the storage process, the virtual reality device 500 can also store the screenshot image files in a designated storage path for subsequent processing, such as playback, network transmission, and recognition analysis. It should be noted that the virtual reality device 500 can also use built-in or external storage to save the screenshot image files. That is, in this embodiment, the storage device for the screenshot image files can be the internal storage of the virtual reality device 500, or it can be the storage device in an external device connected to the virtual reality device 500, such as the storage of a display device or a server.

[0128] During the screenshot process, all captured image files are saved in a single folder, such as the "DCIM / Camera" folder on Android systems. Therefore, to ensure consistency in screenshot save locations, such as... Figure 17 As shown, in some embodiments, during the step of saving the target image, the controller is further configured to:

[0129] S330: Traverse the file save paths of the current system;

[0130] S340: If the current system includes a file save path at a preset location, save the target image according to the file save path at the preset location;

[0131] S350: If the current system does not include a file save path at a preset location, create a new folder at the preset location and save the target image to the newly created folder.

[0132] After storing the texture image information as a byte array, the virtual reality device 500 can traverse the current system's file save paths to determine if a folder for saving screenshot images exists in the current system. For example, the virtual reality device 500 can read the folder names in the current system's registry entries one by one to determine if a folder with the file path "DCIM / Camera" exists in the current system. If the folder exists in the current system, the byte array can be saved as an image file, completing the saving of the screenshot image. If the folder does not exist in the current system, a folder with the path "DCIM / Camera" can be created to save the screenshot image file.

[0133] It should be noted that the path where users save screenshot image files varies depending on the operating system. Therefore, different file save paths can be preset in different virtual reality devices 500. Furthermore, users can customize the file save path according to the intended use of the screenshot image file. For example, for screenshot images output to display device 200, the file save path can be directly set to the address of display device 200, so that after the screenshot operation is completed, the image file obtained from the screenshot is directly sent to display device 200 according to the set address.

[0134] As can be seen, in the above embodiments, users can input the screenshot resolution and the encoding method of the screenshot image through the parameter setting interface during the screenshot process, so as to control the virtual reality device 500 to output the screenshot image file according to the set parameters, thereby realizing the customization of the screenshot operation and allowing users to obtain different image files according to different needs.

[0135] The screenshot image obtained by the above screenshot method is a part of the entire rendered scene, that is, the image after removing rendered content such as the skybox, controllers, and rays, so as to retain more of the main content that users are concerned about in the screenshot image. Since the virtual reality device 500 can adjust the shooting angle according to the user's head movements, the image viewed by the user can be a tilted perspective, that is, the displayed content will tilt with the user's head movement. When the tilt angle is too large, the area occupied by the main content in the screenshot image will be reduced, resulting in lower clarity of the screenshot image content. Therefore, in some embodiments, the step of obtaining the target image from the rendered scene further includes:

[0136] S251: Detect the range of the image region;

[0137] S252: Based on the range of the image area, set the position and shooting angle of the virtual screenshot camera in the rendering scene so that the shooting range of the virtual screenshot camera covers the image area and the shooting direction is perpendicular to the image area;

[0138] S253: Lock the position and shooting angle of the virtual screenshot camera.

[0139] Since the virtual reality device 500 can acquire target images through a virtual screenshot camera set independently in the rendering scene, during the screenshot process, the shooting content of the virtual screenshot camera can be limited by setting the settings parameters of the virtual screenshot camera in the rendering scene, so that the shooting angle of the captured target image is fixed, thereby outputting an image area image under a fixed perspective.

[0140] That is, after receiving a screenshot control command, the virtual reality device 500 can detect the image area range in the rendered scene, and set the setting parameters of the virtual screenshot camera according to the detected image area range, including: setting the shooting angle of the virtual screenshot camera so that the shooting direction of the virtual screenshot camera is perpendicular to the plane corresponding to the image area; and setting the position of the virtual screenshot camera in the rendered scene so that the shooting range of the virtual screenshot camera covers the image area.

[0141] For example, when a user takes a screenshot of the global UI, since the global UI controls are displayed on a plane in the rendering scene—that is, the image area is the area on that plane where the UI controls are located—the virtual reality device 500 can first set the shooting angle of the virtual screenshot camera so that the shooting direction of the virtual screenshot camera is perpendicular to the plane where the UI controls are located. Then, by adjusting the distance between the virtual screenshot camera and the plane where the UI controls are located, the virtual screenshot camera can cover the image area on that plane.

[0142] After setting the virtual screenshot camera position and shooting angle, the virtual reality device 500 can also lock the position and shooting angle of the virtual screenshot camera to prevent the virtual screenshot camera from adjusting the shooting angle according to the user's head movements. This ensures that the content of the screenshot will not be tilted as the user's head moves, and will remain consistent with the design direction of the controls in the rendering scene, thus improving clarity.

[0143] As can be seen, the screenshot operation process of the aforementioned virtual reality device 500 can achieve image capture of different image areas under different user interfaces to obtain screenshots of the corresponding main content. However, for some user interfaces, the area of ​​focus for the user is smaller than the image area range defined by the UI interface controls and display panel controls.

[0144] For example, in a media playback interface, the central area of ​​the playing video content, the left area displaying the video playlist, and the area below the playback control bar are all image areas, but users usually focus on the central area of ​​the playing video content. This means that when using the above-mentioned method of cropping image areas, it's impossible to save only the content from the video as an image, and the saved image cannot maintain the same resolution as the original video. Therefore, as... Figure 18 As shown, in some embodiments, the step of obtaining the target image from the rendering scene further includes:

[0145] S261: Detect the type of the current user interface;

[0146] S262: If the current user interface is a media asset playback interface, generate the target image based on the media asset file being played;

[0147] S263: If the type of the current user interface is not a media playback interface, perform the step of obtaining the target image from the rendering scene.

[0148] After the user inputs the screenshot control command, the virtual reality device 500 can also detect the type of the currently displayed user interface. The specific detection method can be accomplished through the display control program of the virtual reality device 500. For example, it can determine the type of the currently displayed user interface by detecting the execution status of the display control program code of the current controller. When the controller is running a UI control program, the type of the currently displayed user interface is determined to be a UI interface; when the controller is running a player program, the type of the currently displayed user interface is determined to be a media playback interface.

[0149] By detecting the current user interface type, different screenshot methods can be used for different user interface types. If the current user interface type is a UI interface or other interface without media asset playback functionality, the step of obtaining the target image from the rendering scene is executed, that is, the screenshot operation is completed according to the screenshot method in the above embodiment. However, if the current user interface type is a media asset playback interface with media asset playback functionality, the target image is generated based on the media asset file being played. Generating the target image based on the media asset file being played, such as... Figure 19 As shown, in some embodiments, the controller is also configured to perform the following program steps:

[0150] S2621: Obtain the video data stream of the media asset file being played, and obtain the input time of the control command;

[0151] S2622: Extract the image of the frame at the input time from the video data stream;

[0152] S2623: Copy the frame image to output the frame image as the target image.

[0153] To generate a target image based on the media asset file being played, the virtual reality device 500 can acquire the video data stream of the media asset file when it determines that the current user interface type is a media asset playback interface. The video data stream can be directly captured from the player; that is, after decoding the media asset file, the player sends the video data stream to the rendering scene. This video data stream is composed of multiple frames of images. Each frame in the video data stream can be called a picture frame image, and multiple picture frames images can be arranged sequentially in time to form a video image.

[0154] While acquiring the video data stream, the virtual reality device 500 can also extract the input time of the screenshot control command, thereby extracting the frame image of the input time from the video data stream. For example, if the user inputs a control command at 12:32:05:028, the virtual reality device 500, after capturing the video data stream for the corresponding time period, can search the video data stream according to the input time to obtain the frame image of 12:32:05:010.

[0155] Obviously, the frames in a video data stream are arranged at certain frame rate intervals. For example, for a 60Hz video data stream, there are 60 frames per second. When the time corresponding to the first frame is 12:32:05:000, the time corresponding to the second frame is 12:32:05:017. At this time, there is no frame data at the input time of 12:32:05:010. Therefore, the virtual reality device 500 can extract the image with the shortest time relative to the input time as the frame data, i.e., the second frame.

[0156] As can be seen, in this embodiment, when the virtual reality device 500 receives a screenshot event, it can determine whether the current user interface is playing a video. If a video is playing, the input time can be obtained, and then the nearby frame images can be obtained based on the input time, i.e., a Bitmap object can be obtained. The Bitmap is then saved as a screenshot image file, so that the size of the screenshot image is the same as the size of the video frame, ensuring that the captured image has accurate resolution.

[0157] Based on the aforementioned virtual reality device 500, some embodiments of this application also provide a high-definition screenshot method, which can be applied to the aforementioned virtual reality device 500. The high-definition screenshot method includes:

[0158] S1: Receives user input for taking screenshots;

[0159] S2: In response to the control command, obtain a target image from the rendering scene, wherein the target image is an image of a specified resolution obtained by performing image capture on an image region in the rendering scene;

[0160] S3: Save the target image to generate a screenshot image file.

[0161] As can be seen from the above technical solutions, the high-definition screenshot method provided in the above embodiments can first receive user input screenshot control commands during the screenshot process, then obtain a target image of a specified resolution from the rendering scene according to the screenshot control commands, and save the target image to obtain a screenshot image file. The method can capture images of image areas in the rendering scene during the screenshot process, ensuring that the obtained image contains more of the main content, alleviating the occupation of the screenshot image area by non-main scenes in the rendering scene, and improving the clarity of the main content.

[0162] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A virtual reality device, characterized in that, include: The display is configured to display a user interface, the user interface including an image area and a skybox area, the image area being used to present multimedia content; The skybox area is located around the image area and is used to present the rendered background content; The controller is configured as follows: Receive user input for screen capture control commands; In response to the control command, the virtual screenshot camera configured in the rendering scene is activated; The image area in the rendering scene is set according to the current user interface type; Based on the range of the image area, the position and shooting angle of the virtual screenshot camera in the rendering scene are set so that the shooting range of the virtual screenshot camera covers the image area and the shooting direction is perpendicular to the image area; Lock the position and shooting angle of the virtual screenshot camera; The virtual screenshot camera is controlled to capture images of the image area to obtain a target image from the rendering scene. The target image is an image of a specified resolution obtained by capturing images of the image area in the rendering scene. Save the target image to generate a screenshot file.

2. The virtual reality device according to claim 1, characterized in that, The controller is further configured to: Get the playback command input by the user to display the screenshot image file; In response to the playback command, the screenshot image file is parsed to obtain the target image; The target image is displayed within the image area.

3. The virtual reality device according to claim 1, characterized in that, In the step of obtaining the target image from the rendering scene, the controller is further configured to: Upon receiving the control command, the control display shows the parameter setting interface; Obtain screenshot parameters input by the user through the parameter setting interface, the screenshot parameters including screenshot resolution and encoding method; Based on the screenshot resolution, the sampling resolution of the virtual screenshot camera is set to obtain the target image at the specified resolution.

4. The virtual reality device according to claim 3, characterized in that, In the step of saving the target image, the controller is further configured to: Output the pixel information of the target image to the texture image; The texture image is output to a byte array according to the encoding method described above; Save the byte array as a screenshot image file.

5. The virtual reality device according to claim 1, characterized in that, In the step of saving the target image, the controller is further configured to: Detect the saving process of the target image; Once the target image has been saved, close the virtual screenshot camera.

6. The virtual reality device according to claim 1, characterized in that, In the step of obtaining the target image from the rendering scene, the controller is further configured to: Detect the type of the current user interface; If the current user interface is a media asset playback interface, the target image is generated based on the media asset file being played; If the current user interface is not a media playback interface, perform the step of obtaining the target image from the rendering scene.

7. A high-definition screenshot method, characterized in that, Applied to virtual reality devices, wherein the virtual reality includes a display and a controller, the high-definition screenshot method includes: Receive user input for screen capture control commands; In response to the control command, the virtual screenshot camera configured in the rendering scene is activated; Based on the current user interface type, the image area in the rendering scene is set; the rendering scene also includes a skybox area, which is located around the image area and is used to present the rendering background content; Based on the range of the image area, the position and shooting angle of the virtual screenshot camera in the rendering scene are set so that the shooting range of the virtual screenshot camera covers the image area and the shooting direction is perpendicular to the image area; Lock the position and shooting angle of the virtual screenshot camera; The virtual screenshot camera is controlled to capture images of the image area to obtain a target image from the rendering scene. The target image is an image of a specified resolution obtained by capturing images of the image area in the rendering scene. Save the target image to generate a screenshot file.

Images