A virtual reality device and a picture display method

Abstract

The present application provides a virtual reality device and a picture display method. After obtaining a user's control instruction, the method can capture a target picture, obtain an intermediate picture with a fixed resolution based on the target picture, and display the intermediate picture on a display. When the resolution of the target picture does not match the VR scene, the method can transform the target picture to meet the display effect, and at the same time, it is not necessary to send the target picture to other devices for compression, thereby improving the picture display rate.

Description

Technical Field

[0001] This application relates to the technical field of virtual reality devices, and particularly to a virtual reality device and a picture display method. Background Art

[0002] Virtual Reality (VR) technology is a display technology that simulates a virtual environment through a computer, thus giving people a sense of immersion in the environment. A virtual reality device is a device that uses virtual display technology to present a virtual picture to users to achieve a sense of immersion. Generally, a virtual reality device includes two display screens for presenting virtual picture content, corresponding to the left and right eyes of the user respectively. When the content displayed on the two display screens comes from images of the same object from different perspectives, it can bring a stereoscopic viewing experience to the user.

[0003] A virtual reality device can display various media contents through a VR scene, that is, render a playback interface through the Unity 3D engine, and display picture, video and other picture contents in the rendered playback interface. In a VR scene, the played media will be set on a specific display panel. The shape and size of the display panel can be fixed to a set value. When playing media content, the media picture is scaled to fit the size of the display panel, so as to obtain a VR picture.

[0004] However, when a virtual reality device plays a picture file, since it is impossible to estimate the resolution of the picture to be played, it is difficult to match the size of the picture to be played with the size of the display panel, resulting in a poor display effect of the scaled picture. For example, when the picture resolution is low, the picture will show jaggedness when enlarged; when the picture resolution is large, moiré patterns will appear on the picture, making the picture display effect distorted and affecting the user experience. Summary of the Invention

[0005] This application provides a virtual reality device and a picture display method to solve the problem of poor picture display effect of traditional virtual reality devices.

[0006] On the one hand, this application provides a virtual reality device, including a display and a controller. Among them, the display is configured to display a user interface, and the controller is configured to execute the following program steps:

[0007] Obtain a control instruction input by the user for displaying a target picture;

[0008] In response to the control instruction, perform an image capture on the target picture to generate an intermediate image, and the intermediate image is an image with a fixed resolution;

[0009] Control the display to display the intermediate image.

[0010] On the other hand, the present application also provides a method for displaying pictures. The method can be applied to the above virtual reality device, and the method includes:

[0011] Obtain a control instruction input by a user for displaying a target picture;

[0012] In response to the control instruction, perform an image capture on the target picture to generate an intermediate image, where the intermediate image is an image with a fixed resolution;

[0013] Control the display to display the intermediate image.

[0014] As can be seen from the above technical solutions, the virtual reality device and the picture display method provided by the present application can, after obtaining the control instruction of the user, capture the target picture, thereby obtaining an intermediate image with a fixed resolution according to the target picture, and display the intermediate image on the display. When the resolution of the target image does not match the VR scene, the method can transform the target picture, meet the display effect, and at the same time, without sending the target picture to other devices for compression, improve the picture display rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0016] Figure 1 Schematic diagram of the display system structure including a virtual reality device in an embodiment of the present application;

[0017] Figure 2 Schematic diagram of the VR scene global interface in an embodiment of the present application;

[0018] Figure 3 Schematic diagram of the recommended content area of the global interface in an embodiment of the present application;

[0019] Figure 4 Schematic diagram of the application quick operation entry area of the global interface in an embodiment of the present application;

[0020] Figure 5 Schematic diagram of the suspended matter of the global interface in an embodiment of the present application;

[0021] Figure 6 Schematic diagram of the VR picture in an embodiment of the present application;

[0022] Figure 7 Schematic diagram of the flow of a picture display method in an embodiment of the present application;

[0023] Figure 8Schematic diagram showing the effect of high - resolution pictures in the embodiments of the present application;

[0024] Figure 9 Schematic diagram of the process of generating an intermediate image by an intermediate camera in the embodiments of the present application;

[0025] Figure 10 Schematic diagram showing the effect of the intermediate image in the embodiments of the present application;

[0026] Figure 11 Schematic diagram of the process of displaying a target picture according to the picture resolution in the embodiments of the present application;

[0027] Figure 12 Schematic diagram of setting an intermediate camera and a playback camera in a VR scenario in the embodiments of the present application;

[0028] Figure 13 Schematic diagram of the process of setting the display panel type according to the source type of the film in the embodiments of the present application. Detailed implementation manners

[0029] To make the objectives, technical solutions and advantages of the exemplary embodiments of the present application clearer, the technical solutions in the exemplary embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the exemplary embodiments of the present application. Apparently, the described exemplary embodiments are only a part rather than all of the embodiments of the present application.

[0030] Based on the exemplary embodiments shown in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application. In addition, although the disclosure in the present application is introduced according to one or several exemplary instances, it should be understood that each aspect of these disclosures can also be independently constituted as a complete technical solution.

[0031] It should be understood that the terms "first", "second", "third", etc. in the specification, claims and the above - mentioned drawings of the present application are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances, for example, it can be implemented in an order other than those given in the illustration or description of the embodiments of the present application.

[0032] In addition, the terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device including a series of components does not necessarily have to be limited to those components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0033] As used in this application, the term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware and / or software code that can perform functions related to that element.

[0034] The terms "multiple embodiments", "some embodiments", "one embodiment", or "an embodiment" mentioned throughout this specification mean that the specific features, structures, or characteristics described in connection with that embodiment are included in at least one embodiment. Therefore, the phrases "in multiple embodiments", "in some embodiments", "in at least another embodiment", or "in an embodiment" that appear throughout this specification do not necessarily all refer to the same embodiment. In addition, in one or more embodiments, the specific features, structures, or characteristics can be combined in any suitable manner. Therefore, without limitation, the specific features, structures, or characteristics shown or described in connection with one embodiment can be combined in whole or in part with the features, structures, or characteristics of one or more other embodiments. Such modifications and variations are intended to be included within the scope of this application.

[0035] In the embodiments of this application, the virtual reality device 500 generally refers to a display device that can be worn on the user's face and provide an immersive experience for the user, including but not limited to VR glasses, augmented reality devices (AR), VR game devices, mobile computing devices, and other wearable computers, etc. The virtual reality device 500 can operate independently or be connected as an external device to other intelligent display devices, where the display device can be a smart TV, computer, tablet, server, etc.

[0036] After being worn on the user's face, the virtual reality device 500 can display media content images, providing close-range images for the user's eyes to bring an immersive experience. To present the media content images, the virtual reality device 500 can include multiple components for displaying images and for facial wearing. Taking VR glasses as an example, the virtual reality device 500 can include components such as a housing, temple arms, an optical system, a display component, an attitude detection circuit, an interface circuit, etc. In practical applications, the optical system, display component, attitude detection circuit, and interface circuit can be arranged inside the housing to be used for presenting specific display images; the two sides of the housing are connected to the temple arms for wearing on the user's face.

[0037] When in use, attitude detection elements such as a gravity acceleration sensor and a gyroscope are built into the attitude detection circuit. When the user's head moves or turns, the user's attitude can be detected, and the detected attitude data is transmitted to a processing element such as a controller, enabling the processing element to adjust the specific image content in the display component according to the detected attitude data.

[0038] It should be noted that, depending on the type of the virtual reality device 500, the way it presents specific picture content is also different. For example, as Figure 1 shown, for some thin and light VR glasses, the built-in controllers generally do not directly participate in the control process of the displayed content, but send the attitude data to an external device, such as a computer. The external device processes the data and determines the specific picture content to be displayed, and then sends it back to the VR glasses to display the final picture.

[0039] In some embodiments, the virtual reality device 500 can be connected to the display device 200, and a network-based display system can be constructed between the virtual reality device 500, the display device 200, and the server 400. Data interaction can be carried out in real time among the virtual reality device 500, the display device 200, and the server 400. For example, the display device 200 can obtain media data from the server 400 and play it, and transmit the specific picture content to the virtual reality device 500 for display.

[0040] Among them, the display device 200 can be a liquid crystal display, an OLED display, or a projection display device. The specific type, size, and resolution of the display device are not limited. Those skilled in the art can understand that the display device 200 can make some changes in performance and configuration according to needs. The display device 200 can provide a broadcast receiving TV function, and can also additionally provide an intelligent network TV function with computer support functions, including but not limited to, network TV, smart TV, Internet Protocol TV (IPTV), etc.

[0041] The display device 200 and the virtual reality device 500 also communicate with the server 400 through various communication methods. It is allowed for the display device 200 and the virtual reality device 500 to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various contents and interactions to the display device 200. By way of example, the display device 200 can receive software program updates or access a remotely stored digital media library by sending and receiving information and interacting with an electronic program guide (EPG). The server 400 can be a cluster or multiple clusters, and can include one or more types of servers. Other network service contents such as video-on-demand and advertising services are provided through the server 400.

[0042] During the process of data interaction, the user can operate the display device 200 through the mobile terminal 100A and the remote controller 100B. The mobile terminal 100A and the remote controller 100B can communicate with the display device 200 in a direct wireless connection manner or in a non-direct connection manner. That is, in some embodiments, the mobile terminal 100A and the remote controller 100B can communicate with the display device 200 through direct connection methods such as Bluetooth and infrared. When sending a control instruction, the mobile terminal 100A and the remote controller 100B can directly send the control instruction data to the display device 200 through Bluetooth or infrared.

[0043] In some other embodiments, the mobile terminal 100A and the remote controller 100B can also access the same wireless network with the display device 200 through a wireless router to establish non-direct connection communication with the display device 200 through the wireless network. When sending a control instruction, the mobile terminal 100A and the remote controller 100B can send the control instruction data to the wireless router first, and then the wireless router forwards the control instruction data to the display device 200.

[0044] In some embodiments, the user can also directly interact with the virtual reality device 500 using the mobile terminal 100A and the remote controller 100B. For example, the mobile terminal 100A and the remote controller 100B can be used as handles in a virtual reality scene to achieve functions such as somatosensory interaction.

[0045] In some embodiments, the display component of the virtual reality device 500 includes a display screen and a driving circuit related to the display screen. In order to present a specific picture and bring a stereoscopic effect, the display component can include two display screens, respectively corresponding to the user's left eye and right eye. When presenting a 3D effect, the picture contents displayed on the left and right screens will be slightly different, and can respectively display the left camera and the right camera during the shooting process of the 3D video source. Due to the picture contents observed by the user's left and right eyes, when wearing it, a display picture with a strong sense of three-dimensionality can be observed.

[0046] The optical system in the virtual reality device 500 is an optical module composed of multiple lenses. The optical system is arranged between the user's eyes and the display screen. Through the refraction of the light signal by the lenses and the polarization effect of the polarizers on the lenses, the optical path can be increased, so that the content presented by the display component can be clearly presented within the user's visual field. At the same time, in order to adapt to the vision conditions of different users, the optical system also supports focusing, that is, by adjusting the position of one or more of the multiple lenses through a focusing component, changing the mutual distance between the multiple lenses, thereby changing the optical path and adjusting the picture clarity.

[0047] The interface circuit of the virtual reality device 500 can be used to transmit interactive data. In addition to transmitting posture data and display content data as described above, in actual applications, the virtual reality device 500 can also connect to other display devices or peripherals through the interface circuit to implement more complex functions by exchanging data with the connected devices. For example, the virtual reality device 500 can be connected to a display device through the interface circuit, thereby outputting the displayed image to the display device in real time for display. For another example, the virtual reality device 500 can also be connected to a controller through the interface circuit, which can be held by the user to perform related operations in the VR user interface.

[0048] The VR user interface can be presented in a variety of different types of UI layouts according to user operations. For example, the user interface can include a global interface, and the global UI after the AR / VR terminal is started can be as follows: Figure 2 As shown, the global UI can be displayed on the display screen of the AR / VR terminal or on the display of the display device. The global UI can include a recommended content area 1, a service classification expansion area 2, an application quick operation entry area 3, and a floating object area 4.

[0049] The recommended content area 1 is used to configure different classification TAB columns; in the columns, media resources, special topics, etc. can be selected; the media resources may include 2D movies, educational courses, travel, 3D, 360-degree panorama, live broadcast, 4K movies, program applications, games, travel and other businesses with media content, and the columns can select different template styles and support the simultaneous recommendation and arrangement of media resources and special topics, such as Figure 3 shown.

[0050] The business classification extension area 2 supports the configuration of extended categories of different categories. If there is a new business type, it supports the configuration of an independent TAB to display the corresponding page content. The extended categories in the business classification extension area 2 can also be sorted and adjusted and offline business operations can be performed. In some embodiments, the business classification extension area 2 may include the following content: film and television, education, travel, applications, and mine. In some embodiments, the business classification extension area 2 is configured to display a large business category TAB, and supports the configuration of more categories, and its icons support configuration, such as Figure 3 shown.

[0051] The application quick operation entry area 3 can specify pre-installed applications to be displayed in front for operation recommendation, and supports configuring special icon styles to replace the default icons. The pre-installed applications can be specified as multiple. In some embodiments, the application quick operation entry area 3 also includes a left-moving control and a right-moving control for moving the option target, which are used to select different icons, such as Figure 4 shown.

[0052] The suspension area 4 can be configured to be above the upper left or upper right of a fixed area, and can be configured with a replaceable image or a jump link. For example, after receiving a confirmation operation, the suspension jumps to a certain application or displays a specified function page, such as Figure 5 shown. In some embodiments, the suspension may not be configured with a jump link and is simply used for image display.

[0053] In some embodiments, the global UI also includes a status bar at the top for displaying time, network connection status, battery status, and more quick operation entrances. After using the handle of the AR / VR terminal, that is, the handheld controller, to select an icon, the icon will display a text prompt that expands left and right. The selected icon stretches and expands left and right according to its position.

[0054] For example, after selecting the search icon, the search icon will display the text "Search" and the original icon. After further clicking on the icon or the text, the search icon will jump to the search page; for another example, clicking on the favorite icon jumps to the favorite TAB, clicking on the history icon defaults to display the history page, clicking on the search icon jumps to the global search page, and clicking on the message icon jumps to the message page.

[0055] In some embodiments, interactions can be performed through external devices. For example, the handle of the AR / VR terminal can operate on the user interface of the AR / VR terminal, including a back button; a home key, and long pressing it can achieve a reset function; volume up and down buttons; a touch area, and the touch area can achieve functions such as clicking, sliding, and holding and dragging of the focus.

[0056] Users can enter different scene interfaces through the global interface. For example, as Figure 6 shown, users can enter the browsing interface through the "Browsing Interface" entry in the global interface, or start the browsing interface by selecting any media asset in the global interface. In the browsing interface, the virtual reality device 500 can create a 3D scene through the Unity 3D engine and render specific picture contents in the 3D scene.

[0057] In the browsing interface, users can watch specific media content. To obtain a better viewing experience, different virtual scene controls can also be set in the browsing interface to cooperate with the media content to present specific scenes or real-time interactions. For example, in the browsing interface, a panel can be set in the Unity 3D scene to present picture contents, and in combination with other virtual home controls, the effect of a cinema screen can be achieved.

[0058] The virtual reality device 500 can display the operation UI content in the browsing interface. For example, in front of the display panel in the Unity 3D scene, a list UI can also be displayed. In the list UI, the media asset icons locally stored in the current virtual reality device 500 can be displayed, or the network media asset icons that can be played in the virtual reality device 500 can be displayed. The user can select any icon in the list UI, and the selected media asset can be displayed in real time on the display panel.

[0059] Since the selected media asset needs to be displayed on the display panel, there should be a matching relationship between the display panel and the picture of the selected media asset, that is, the selected media asset should be able to be fully displayed on the display panel. For example, when displaying a picture, the maximum resolution supported by the display panel should be greater than the resolution of the selected picture. However, since the resolvable picture resolution is not fixed and the VR scene space is limited, the display area (space) of the display panel is limited to a fixed value. Therefore, some pictures will exceed the display panel area, resulting in incomplete picture content not being presented on the display panel. For example, as Figure 8 shown, for an 8K-level picture, its picture resolution can reach 7680×4320, while the display screen of a general virtual reality device is only 2K level, that is, it supports a resolution of 2560×1440. At this time, when displaying an 8K-level picture on the display panel, it will exceed the display range and cannot be fully displayed.

[0060] In order to be able to fully display the picture, in some embodiments, the virtual reality device can scale the picture with a higher resolution, and by adjusting the picture size, make the picture displayed on the display panel. However, in the display method of adjusting the size by scaling, when shrinking a high-resolution picture, a moiré phenomenon will occur, that is, the picture viewed by the user will flicker, reducing the user experience.

[0061] The virtual reality device can also send the picture to the Android side for compression when displaying a high-resolution picture, compress an 8K-level picture to 2K level, and then display it through the display panel. However, this processing method will occupy CPU resources, and the processing process and transmission process will consume a certain amount of time, resulting in slower picture display and also reducing the user experience.

[0062] To improve the user experience, in some embodiments of the present application, an image display method is also provided. The picture display method can be applied to a virtual reality device, so that mid- to low-end VR devices with low hardware configurations can obtain better display effects. As Figure 7 shown, the image display method includes the following content:

[0063] The user inputs a control instruction for displaying a target picture to the virtual reality device; among them, the control instruction for displaying the target picture can be completed by a series of interaction operations of the user. For example, the user can enter the file management application through the global interface. The file management application can present a list of local pictures saved in the current virtual reality device in the application interface. The user inputs the control instruction for displaying the target picture by selecting any local picture in the local picture list through interaction operations.

[0064] The target picture is the picture selected by the user in the list. It should be noted that in the embodiments of the present application, the target picture is not limited to local pictures, and can also be pictures obtained by the virtual reality device from a network server. For example, on the media asset recommendation page, multiple picture links can be displayed. When the user selects to open any picture link through an interaction action, the virtual reality device can send a download request to the server in the network, download the picture file corresponding to the picture link, and automatically display it after the download is completed. Therefore, the control instruction for displaying the target picture also includes the above-mentioned network download operation instruction.

[0065] In addition, the control instruction for displaying the target picture can also be completed by other means. For example, when the user clicks on the switching control in the browsing interface, the previous picture or the next picture of the currently displayed picture in the current browsing interface needs to be displayed. At this time, the control instruction for displaying the target picture is the instruction of clicking on the switching control.

[0066] After the user inputs the control instruction, the virtual reality device can display the target picture in response to the control instruction. Since there may be a mismatch between the resolution of the target picture and the resolution supported by the display area in the current VR scene, the target picture can be photographed to generate an intermediate image.

[0067] Among them, photographing the target picture can be completed by setting a virtual intermediate camera in the VR scene. The intermediate camera is a virtual image generation device that can photograph the specified picture content in the VR scene to generate an intermediate image. Obviously, in order to photograph the target picture, a region for displaying the target picture can be set in the VR scene first, such as setting a virtual panel. Since this panel is only used to display the target picture for the intermediate camera to perform shooting, the set virtual panel can be called a temporary panel.

[0068] The generated intermediate image can be output in the current VR scene or on a specified device. In order to enable the target picture to be completely displayed, the intermediate camera is configured to output an image with a fixed resolution, that is, the intermediate camera can generate an image with a specified resolution by setting shooting parameters. For example, Figure 8As shown, the resolution of the target image is 7680×4320, and the resolution of the intermediate image output by the intermediate camera can be 2560×1440. By using the intermediate camera to capture the target image, an intermediate image with a resolution of 2560×1440 can be obtained, and this intermediate image with such a resolution can be completely displayed in the VR scene.

[0069] It can be seen that by using the intermediate camera to capture the target image, an intermediate image with the same content but a lower resolution can be generated based on the high-resolution picture, equivalently reducing the resolution of the target image. Obviously, in order to retain as much content of the target image as possible while reducing the resolution, in some embodiments of the present application, the output resolution of the intermediate camera can be set equal to the maximum resolution of the picture display area in the VR scene. For example, as Figure 9 shown, the display screen of the virtual reality device has a 2K resolution, that is, it supports a maximum resolution of 2560×1440, and correspondingly, the resolution of the intermediate camera is also set to 2560×1440.

[0070] After obtaining the intermediate image, the specific picture content in the VR scene can be displayed through the display. In order to be able to display pictures through the virtual reality device, while the user inputs a control instruction for displaying the target picture, the VR scene can also be initialized and rendered, that is, virtual content is added to the VR scene, and a dedicated area for displaying the picture is set, and the intermediate image can be displayed within the set picture display area, as Figure 10 shown.

[0071] It should be noted that since the virtual reality device constructs a specific viewing scene by combining the virtual pictures in the VR scene while displaying the target picture. Therefore, generally, the content displayed on the display is not only the intermediate image but also includes other virtual pictures. For example, when simulating a cinema scene through the virtual reality device, the intermediate image can be displayed in the virtual cinema screen area, and virtual pictures related to the cinema, such as seats, audio equipment, ambient lights, and virtual characters, etc., can be displayed in other areas. And as the user's actions of wearing the virtual reality device change, the picture content presented through the display also changes in real time.

[0072] From the above technical solutions, it can be seen that the picture display method provided in this embodiment can convert the target picture with an uncertain specific resolution into an intermediate image with a fixed resolution for output display by using the intermediate camera to capture the target picture, thereby overcoming the defect that the resolution of some target pictures does not match the resolution of the display area and ensuring the correct display of the target picture.

[0073] Since the entire display process does not scale the picture itself, it can alleviate the jaggedness and flickering generated during the picture scaling process and ensure the picture display quality. In addition, the above picture display method can be directly and independently completed by the GPU in the virtual reality device without the need for the CPU to perform resolution conversion. Therefore, it can save the time of resolution conversion and data transmission processes, enabling the target picture to be displayed quickly and enhancing the user experience.

[0074] During the process of displaying a picture, since the resolution of the target picture is uncertain, that is, the resolution of some target pictures is greater than the supported resolution of the display panel, while the resolution of some target pictures is less than or equal to the supported resolution of the display panel. Among them, the target pictures with a smaller resolution can be directly displayed on the display panel. Therefore, for the convenience of quick display, as Figure 11 shown, in some embodiments of the present application, the step of using the intermediate camera to capture the target picture in response to the control instruction further includes:

[0075] Analyze the target picture specified in the control instruction;

[0076] Obtain the picture resolution of the target picture and the supported resolution of the display panel in the current VR scene;

[0077] If the picture resolution is greater than the supported resolution, perform image capture on the target picture;

[0078] If the picture resolution is less than or equal to the supported resolution, display the target picture through the display panel.

[0079] After obtaining the control instruction input by the user, the GPU of the virtual reality device can parse the target picture to be displayed according to the control instruction, extract the storage path of the target picture, and extract the original file of the target picture from the memory according to the storage path. Then, by extracting the picture resolution of the target picture and the maximum resolution supported by the display panel in the current VR scene and comparing them, it is determined whether the target picture matches the supported resolution of the display panel in the current VR scene.

[0080] When the resolution of the target picture is greater than the supported resolution of the display panel, it is determined that the target picture cannot be fully displayed in the current VR scene. Therefore, the target picture can be converted into a low-resolution intermediate image for display according to the picture display method provided in the above embodiments, that is, the step of using the intermediate camera to capture the target picture is performed. When the resolution of the target picture is less than or equal to the supported resolution of the display panel, it is determined that the target picture can be fully displayed by the current display panel. Therefore, the target picture can be directly displayed through the display panel without converting the target picture, thereby improving the display efficiency of the target picture.

[0081] For example, by parsing the target picture, the picture resolution of the current target picture is obtained as 1920×1080, while the maximum supported resolution of the current display panel is 2560×1440. That is, the picture resolution is less than the supported resolution, and the current target picture can be fully displayed on the display panel. At this time, the step of using the intermediate camera to capture the target picture can be skipped, and the target picture can be directly displayed on the display panel.

[0082] It can be seen that in this embodiment, before the step of using the intermediate camera to capture the target picture, the picture resolution of the target picture can be extracted first, and compared with the resolution supported by the display panel to determine whether the current display panel supports fully displaying the current target picture. If the display panel can fully display the target picture, the step of using the intermediate camera for shooting can be skipped and directly displayed. Therefore, this embodiment can reduce the amount of data processing and improve the display speed of the picture on the basis of ensuring that the target picture can be fully displayed.

[0083] For a target picture with a large resolution, since an intermediate camera needs to be used to capture the picture to generate an intermediate image. Therefore, as Figure 12 shown, in some embodiments, in response to the control instruction, the step of using the intermediate camera to capture the target picture further includes:

[0084] Setting the intermediate camera and the temporary panel in the current VR scene; displaying the target picture in the temporary panel; using the intermediate camera to perform shooting on the temporary panel to output an intermediate image. After starting the VR scene, the controller of the virtual reality device can judge the resolution of the target picture. When the resolution of the target picture is greater than the supported resolution of the display panel, the intermediate camera and the temporary panel can be set in the VR scene, and the target picture can be displayed through the temporary panel. When displaying, the target picture can maintain the original picture size and resolution without scaling the target picture, so there will be no display defects such as jagged or flickering. Then use the intermediate camera to shoot the temporary panel so that the intermediate camera can capture the target image content displayed in the temporary panel, thereby outputting an intermediate image.

[0085] When setting up the intermediate camera and the temporary panel, by setting the parameters of the intermediate camera and the temporary panel, the intermediate camera can capture the temporary panel. For example, by setting the mutual distance between the intermediate camera and the temporary panel, the shooting field of view of the intermediate camera can cover the temporary panel, enabling the capture of all the content displayed on the temporary panel. Another example is that by setting camera parameters such as the focal length and projection ratio of the intermediate camera, the shooting effect of the camera can be adjusted so that, even when the distance between the intermediate camera and the temporary panel is fixed, the field of view can still cover the temporary panel to capture all the content displayed on the temporary panel.

[0086] In addition, since the intermediate camera and the temporary panel set in the VR scene may be viewed by the user during subsequent viewing, in order not to affect the subsequent viewing experience, in this embodiment, the intermediate camera and the temporary panel can also be hidden by setting their parameters. For example, the temporary panel can be set to be visible only to the intermediate camera, that is, only the intermediate camera can capture the content in the temporary panel, and the playback camera (main camera) enabled during subsequent viewing cannot capture the temporary panel, thus preventing the user from seeing the temporary panel in the VR scene.

[0087] The position of the intermediate camera and the temporary panel can also be set so that the playback camera (main camera) cannot capture them, to mitigate the impact of the temporary panel on the viewing experience. For example, the intermediate camera and the temporary panel can be set at the farthest position in the VR scene so that the playback camera cannot capture them. The intermediate camera and the temporary panel can also be set to a dynamic position, that is, in an area outside the field of view of the playback camera. When the shooting angle of the playback camera changes, the set positions of the intermediate camera and the temporary panel also change synchronously, thus avoiding being captured by the playback camera.

[0088] In some embodiments, the step of setting the intermediate camera and the temporary panel in the current VR scene further includes:

[0089] Obtain the picture resolution of the target picture; set the display area area of the temporary panel according to the picture resolution; set the projection ratio between the intermediate camera and the display area according to the display area area, so that the intermediate camera can capture the entire display area area of the temporary panel.

[0090] For target images with different resolutions, it is also possible to adjust the size of the temporary panel to display images with different resolutions. For example, when the resolution of the target image is 7680×4320, a temporary panel with a size corresponding to a resolution of 7680×4320 can be set in the VR scene; when the resolution of the target image is 4096×2160, a temporary panel with a size corresponding to a resolution of 4096×2160 can be set in the VR scene. Then, by adjusting the camera parameters of the middle camera and / or the relative positional relationship between the middle camera and the temporary panel, the middle camera can be adapted to temporary panels of different sizes, so as to capture all the content displayed on the temporary panel.

[0091] After setting the middle camera and the temporary panel in the VR scene, the controller can further set a display panel in the VR scene to present the middle image, that is, as Figure 12 shown, in some embodiments, the step of outputting the middle image further includes:

[0092] Setting a display panel in the current VR scene; displaying the middle image in the display panel. In order to present the content in the target image, in this embodiment, a display panel can be set in the VR scene to display the image captured by the middle camera in real time. The position where the display panel is set in the VR scene can be determined according to the specific characteristics of the VR scene. For example, for the application scenario of a simulated cinema, the display panel can be set at the position of the screen of the virtual cinema. After setting the display panel, the middle image can be displayed through the display panel.

[0093] Limited by the specific VR scene, the size of the display panel is usually fixed, and it can be set that the supported resolution of the display panel is equal to the resolution of the middle image. Among them, the resolution of the middle image is the resolution of the middle camera, which can be set according to the GPU performance of the virtual reality device and the specific application scenario. For example, on the premise that the GPU performance meets the requirements, if the maximum supported resolution of the display panel is 2560×1440, the output resolution of the middle camera can be set to 2560×1440, and the resolution of the corresponding output middle image is also 2560×1440. It can be seen that by setting the output resolution of the middle camera to be equal to the supported resolution of the display panel, target images of any resolution can be displayed on the display panel at the set resolution, ensuring that the target images can be fully displayed.

[0094] It should be noted that for different types of target pictures, the shooting methods of the middle camera are different, and the corresponding types of output middle images are also different. For example, for static images, the middle camera can take a single shot and output a static middle image, which can also be displayed as static picture content on the display panel. For dynamic images, the middle camera needs to take pictures of the dynamic image within one change cycle according to the change cycle and frame rate of the dynamic image, so as to generate a dynamic middle image with the same change cycle and frame rate, and play the dynamic image on the display panel.

[0095] In some embodiments, the video picture can also be displayed through a temporary panel. While the video picture is being displayed, continuous shooting is performed using the middle image to generate a middle video, and the middle video is displayed on the display panel. It can be seen that the picture display method provided in this embodiment can convert the resolution of pictures or video pictures to adapt to a display panel of a fixed size, enabling mid- to low-end virtual display devices to also obtain good display effects when displaying high-resolution images.

[0096] While the display panel displays the middle image, the virtual reality device can display the VR scene according to its specific playback method, presenting some picture content in the VR scene on the display. Since the VR scene includes the display panel for displaying the middle image, the purpose of displaying the middle image on the display can be achieved. Specifically, as Figure 12 shown, the steps of controlling the display to display the middle image further include:

[0097] Setting a playback camera in the current VR scene; using the playback camera to take pictures of the display panel and / or the VR scene to generate a final picture with the middle image; controlling the display to display the final picture.

[0098] In order to present specific VR picture content to the user, a playback camera can be set in the VR scene before the display panel displays the middle image. The playback camera can take pictures of the VR scene and output the captured pictures to the display for display in real time. The playback camera can be associated with the attitude detection circuit of the virtual display device, that is, it can adjust the shooting direction of the playback camera in real time according to the user attitude detected by the attitude detection circuit. In different shooting directions, the playback camera can take pictures of the picture content in the display panel and / or the VR scene to generate a final picture with the middle image and output it for display on the display.

[0099] It can be seen that according to the technical solutions provided in the above embodiments, the picture display method provided in the present application can, before displaying a picture, set an intermediate camera and a playback camera, as well as a temporary panel and a display panel in a VR scene. When playing a target picture, the original picture of the target picture is displayed through the temporary panel, and then the intermediate camera is used to capture the content displayed on the temporary panel, so as to obtain an intermediate image and output it to the display panel for display, so that the playback camera can capture the VR scene and the display panel to generate a final picture and display it on the display.

[0100] The entire display process can be completed entirely in the VR scene, that is, completed by the GPU alone, without sending the target picture to the Android layer for compression processing, thereby reducing the time consumed in the compression process and the data transfer process. While ensuring the display effect, the display speed of high-resolution pictures is improved, enabling the picture display method to be applied to mid- and low-end VR devices to improve the picture display effect.

[0101] During the process of displaying the target picture, different types of target pictures require different display methods. For example, for traditional 2D pictures, they can be directly displayed through a flat surface; while for 360 panoramic pictures, they need to be displayed through a curved surface. Therefore, as Figure 13 shown, in order to obtain a better display effect, in some embodiments of the present application, the step of setting the display panel in the VR scene further includes:

[0102] Obtain the source type of the target picture; set the display panel type according to the source type of the target picture. In this embodiment, before setting the display panel, the source type of the target image can be detected, and the source type at least includes 2D source, 3D source, and 360 panoramic source. The specific detection method can be that after receiving a control instruction, the controller extracts information such as the media classification, format, extension name, file description, etc. of the displayed media to determine the source type of the current displayed media. For example, for network resources presented in the user interface, when sharing the media, the source type of the media can be indicated in the file description.

[0103] It is also possible to combine the specific picture content to judge the source type of the currently displayed media. For example, by analyzing the similarity of the left and right sides of the picture, when the similarity of the pictures on both sides is small, it can be determined that the source type of the currently to-be-identified image is a 2D source; when the similarity of the pictures on both sides is large, it can be determined that the source type of the currently to-be-identified image is a 3D source.

[0104] After detecting the source type of the target picture, the display panel type can be set according to the source type of the target picture. Specifically, if the source type of the target picture is a first-type source such as a 2D source or a 3D source, a flat display panel is set in the VR scene; if the source type of the target picture is a second-type source such as a 360-degree panoramic source, a spherical display panel is set in the VR scene.

[0105] For the first-type source, since its presentation forms are all flat patterns and its specific picture content usually has not been deformed, the intermediate image can be directly displayed through a flat display panel. Among them, the picture of the 3D source usually includes two halves. The left half is the picture taken by the left-eye camera, and the right half is the picture taken by the right-eye camera. The specific picture content of the two parts is slightly different. When the 3D source is displayed, the left half can be displayed on the left screen, and the right half can be displayed on the right screen, so that the picture seen by the user presents a stereoscopic effect.

[0106] Therefore, when setting the flat display panel, two levels can be set. In each level of the display panel, the left half and the right half can be displayed respectively. And the level picture showing the left half is output to the left screen for display, and the level picture showing the right half is output to the right screen for display.

[0107] For the second-type source, since it is a flat pattern formed by rotating the camera or combining multiple lenses during shooting, there is partial deformation in its picture content itself. In order to adapt to the deformed area, the intermediate image can be displayed through a curved display panel. For example, when setting the display panel, a display sphere can be added in the VR scene, so as to display the 360-degree panoramic source image through the display sphere to form a panoramic effect.

[0108] It should be noted that since the pictures of different sources can all be in the two-dimensional flat form in terms of picture presentation forms and the temporary panel will not be observed by the user, when setting the temporary panel, a flat panel can be directly set, and the original picture of the picture is directly displayed on the temporary panel, so that the intermediate camera can capture the picture content in the picture in the normal shooting mode and complete the conversion of the intermediate image.

[0109] However, in some embodiments, in order to obtain better display effects, when setting up the temporary panel, the source type of the target picture can be detected, and different forms of temporary panels and intermediate cameras can be set according to different source types of the picture. For example, if the source type of the target picture is a first type of picture source such as a 2D picture source or a 3D picture source, a flat-form temporary panel is set in the VR scene, and the shooting mode of the intermediate camera is set to the conventional shooting mode; if the source type of the target picture is a second type of picture source such as a 360-degree panoramic picture source, a spherical-form display panel is set in the VR scene, and the shooting mode of the intermediate camera is set to the circular shooting mode. Through different forms of temporary panels and intermediate cameras with different shooting modes, the generated intermediate image can be perfectly matched with the final display panel, alleviating the impact of resolution changes on the deformed pictures in the image during the shooting process of the intermediate camera and improving the display effect.

[0110] Based on the above picture display method, in some embodiments of the present application, a virtual reality device is further provided, including: a display and a controller, wherein, the display is configured to display a user interface; the controller is configured to execute the following program steps:

[0111] Obtain a control instruction input by a user for displaying a target picture;

[0112] In response to the control instruction, perform image shooting on the target picture to generate an intermediate image;

[0113] Control the display to display the intermediate image.

[0114] Wherein, the intermediate camera is configured to output an image with a fixed resolution. The virtual reality device provided in this embodiment can, after obtaining the control instruction of the user, use the intermediate camera to shoot the target picture, thereby obtaining an intermediate image with a fixed resolution according to the target picture and displaying the intermediate image on the display. When the resolution of the target image does not match the VR scene, the virtual display device can directly convert the target picture through the intermediate camera, meet the display effect while independently completing the conversion of the target picture, without sending the target picture to other devices for compression, and improving the picture display rate.

[0115] In some embodiments, before performing image shooting on the target picture, the controller of the virtual reality device is further configured to:

[0116] Analyze the target picture specified in the control instruction to obtain the picture resolution of the target picture and the resolution supported by the display panel in the current VR scene; if the picture resolution is greater than the supported resolution, perform image shooting on the target picture; if the picture resolution is less than or equal to the supported resolution, display the target picture through the display panel.

[0117] In this embodiment, before the controller performs image capture on the target picture, it can first detect the resolution of the target picture and compare it with the resolution supported by the display panel in the current scene to determine whether the current scene can completely display the current target picture. If the picture resolution is less than or equal to the supported resolution, the target picture is directly displayed through the display panel without performing the image capture step, reducing the amount of data processing.

[0118] In some embodiments, in order to perform image capture on the target picture, the controller 250 of the virtual reality device is further configured to:

[0119] Set an intermediate camera and a temporary panel in the current VR scene; display the target picture in the temporary panel; and then use the intermediate camera to perform shooting on the temporary panel to output an intermediate image.

[0120] Among them, the intermediate camera and the temporary panel are virtual scene controls in the VR scene, which can be called by the controller during the display process to achieve shooting of the target picture. The intermediate camera can output an image with a fixed resolution, so as to generate an intermediate image with a fixed resolution according to the target picture. The temporary panel can be used to display the target picture for the intermediate camera to shoot. For this purpose, the temporary panel can only be visible to the intermediate camera to avoid the content displayed on the temporary panel affecting the final VR image presentation effect.

[0121] In some embodiments, in order to enable the intermediate camera to capture the content displayed on the temporary panel, when setting the intermediate camera and the temporary panel, the controller is further configured to:

[0122] Obtain the picture resolution of the target picture; then set the display area area of the temporary panel according to the picture resolution; and set the projection ratio between the intermediate camera and the display area according to the display area area, so that the intermediate camera can capture the entire display area area of the temporary panel.

[0123] By setting the projection ratio, the intermediate camera can cover the entire target picture through the set projection ratio within the limited VR scene space, so that the intermediate camera can capture the entire display area area of the temporary panel and adapt to target pictures of different sizes.

[0124] In some embodiments, after generating the intermediate image, the controller can present the final display screen in the following manner:

[0125] Set a display panel and a playback camera in the current VR scene; display the intermediate image in the display panel; and use the playback camera to shoot the display panel and / or the VR scene to generate a final picture with the intermediate image; finally, control the display to display the final picture.

[0126] Among them, the display panel supports a resolution equal to that of the intermediate image. By setting the display panel, the obtained intermediate image can be displayed. Since the resolution supported by the display panel is equal to that of the intermediate image, the intermediate image can be completely displayed on the display panel without scaling the target picture.

[0127] For the similar parts between the embodiments provided in this application, reference can be made to each other. The specific embodiments provided above are only several examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other embodiments extended based on the solution of this application without creative efforts belong to the protection scope of this application.

Claims

1. A virtual reality device, characterized in that: include: a display configured to display a user interface; The controller is configured as: Obtaining the control instruction input by the user for displaying the target image; In response to the control instruction, parsing a target image specified in the control instruction; Obtain the image resolution of the target image and the supported resolution of the display panel in the current VR scene; If the picture resolution is greater than the supported resolution, performing image capture on the target picture to generate an intermediate image; The intermediate image is an image with a fixed resolution; controlling the display to display the intermediate image; If the image resolution is less than or equal to the supported resolution, the target image is displayed through the display panel.

2. A virtual reality device, characterized in that: include: a display configured to display a user interface; The controller is configured as: Obtaining the control instruction input by the user for displaying the target image; In response to the control instruction, an intermediate camera and a temporary panel are set in the current VR scene, where the intermediate camera and the temporary panel are virtual scene controls of the VR scene, the intermediate camera is configured to output an image with a fixed resolution, and the temporary panel is configured to be visible only to the intermediate camera; Displaying the target image in the temporary panel; photographing the temporary panel using the intermediate camera to output an intermediate image; The display is controlled to display the intermediate image.

3. The virtual reality device according to claim 2, wherein: In the step of outputting the intermediate image, the controller is further configured to: Setting a display panel in the current VR scene, wherein the display panel supports a resolution equal to the resolution of the intermediate image; The intermediate image is displayed on the display panel.

4. The virtual reality device according to claim 3, wherein: In the step of controlling the display to display the intermediate image, the controller is further configured to: Set the playback camera in the current VR scene; Using the playback camera to shoot the display panel and / or the VR scene to generate a final picture with the intermediate image; The display is controlled to display the final image.

5. The virtual reality device according to claim 3, wherein: In the step of setting the display panel in the VR scene, the controller is further configured to: Obtaining a source type of the target image, where the source type includes a first type and a second type; The display panel type is set according to the source type of the target image.

6. The virtual reality device according to claim 5, characterized in that In the step of setting the display panel type according to the source type of the target image, the controller is further configured to: If the source type of the target image is the first type, setting the display panel in a planar form in the VR scene; If the source type of the target image is the second type, the display panel in a spherical form is set in the VR scene.

7. The virtual reality device according to claim 2, wherein: In the step of setting the intermediate camera and the temporary panel in the VR scene, the controller is further configured to: Obtaining the image resolution of the target image; Setting the display area of the temporary panel according to the image resolution; According to the area of the display area, a projection ratio between the intermediate camera and the display area is set so that the intermediate camera can capture the entire display area of the temporary panel.

8. A method for displaying an image, characterized in that: Applied to a virtual reality device, the method comprises: Obtaining the control instruction input by the user for displaying the target image; In response to the control instruction, parsing a target image specified in the control instruction; Obtain the image resolution of the target image and the supported resolution of the display panel in the current VR scene; If the image resolution is greater than the supported resolution, performing image capture on the target image to generate an intermediate image; the intermediate image is an image with a fixed resolution; controlling the display to display the intermediate image; If the image resolution is less than or equal to the supported resolution, the target image is displayed through the display panel.

9. A method for displaying an image, characterized in that: Applied to a virtual reality device, the method comprises: Obtaining the control instruction input by the user for displaying the target image; In response to the control instruction, an intermediate camera and a temporary panel are set in the current VR scene, where the intermediate camera and the temporary panel are virtual scene controls of the VR scene, the intermediate camera is configured to output an image with a fixed resolution, and the temporary panel is configured to be visible only to the intermediate camera; Displaying the target image in the temporary panel; photographing the temporary panel using the intermediate camera to output the intermediate image; The display is controlled to display the intermediate image.

Images