Method, device, storage medium and equipment for adjusting viewing picture in virtual environment

Abstract

The application discloses a viewing picture adjusting method and device in a virtual environment, a storage medium and equipment, and the method comprises the steps of displaying a virtual environment, and a ray cursor and a video viewing area are presented in the virtual environment, wherein the ray cursor points to the direction of the video viewing area, and a first included angle is formed between the ray cursor and the video viewing area; in response to drag control information, and based on the initial position of the ray cursor and the first included angle, the video viewing area is dragged; and based on the current dragging position of the video viewing area in the virtual environment, the video picture displayed by the video viewing area is determined. The application designs a dragging mode for the three-dimensional space of 2D video, VR180 video and VR360 video, so that the user can experience the charm of the VR space and different viewing angles in the video field, and the immersive experience of viewing in the virtual reality space is improved.

Description

Technical Field

[0001] This application relates to the field of virtual reality technology, specifically to a method, apparatus, storage medium, and device for adjusting viewing images in a virtual environment. Background Technology

[0002] Currently, in viewing modes such as 2D video, VR180 video, and VR360 video, the video screen is displayed in a fixed position and within a fixed frame, which cannot effectively reflect the immersive experience of virtual reality viewing, resulting in a poor user experience. Summary of the Invention

[0003] This application provides a method, apparatus, storage medium, and device for adjusting the viewing screen in a virtual environment. It designs a drag-and-drop method for 2D video, VR180 video, and VR360 video in three-dimensional space, which allows users to experience the charm of VR space and different viewing angles in the video field, thereby enhancing the immersive experience of watching movies in virtual reality space.

[0004] On one hand, embodiments of this application provide a method for adjusting the viewing screen in a virtual environment, the method comprising:

[0005] A virtual environment is displayed, in which a ray cursor and a video viewing area are presented, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area;

[0006] In response to drag control information, and based on the initial position of the ray cursor and the first angle, drag the video viewing area;

[0007] The video image displayed in the video viewing area is determined based on the current drag position of the video viewing area in the virtual environment.

[0008] In some embodiments, the step of responding to drag control information and dragging the video viewing area based on the initial position of the ray cursor and the first angle includes:

[0009] In response to drag control information, the video viewing area is dragged along the spherical surface of the spherical space with the initial position of the ray cursor as the origin of the spherical space and the first included angle fixed.

[0010] In some embodiments, if the viewing mode of the virtual environment is a two-dimensional video viewing mode, then the video viewing area is a virtual screen, and the virtual screen is in a display state before responding to drag control information.

[0011] The process of dragging the video viewing area along the spherical surface of the spherical space includes:

[0012] The virtual screen is dragged along the spherical surface of the spherical space in at least one of the x-axis and y-axis directions.

[0013] In some embodiments, the method further includes:

[0014] If the virtual screen is dragged along the spherical surface of the spherical space in the y-axis direction to the top or bottom in the y-axis direction, the virtual screen will be controlled to rotate 180 degrees around the center of the virtual screen.

[0015] In some embodiments, the method further includes:

[0016] If the virtual screen is dragged along the spherical surface of the spherical space to the virtual ground in the virtual environment, and the virtual screen and the virtual ground clip through each other, then the virtual ground is hidden.

[0017] In some embodiments, the method further includes:

[0018] During the dragging process, the border of the virtual screen is highlighted.

[0019] When the drag is finished, control the border of the virtual screen to return to normal display.

[0020] In some embodiments, determining the video frame displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment includes:

[0021] Based on the current drag position of the virtual screen in the virtual environment, it is determined that the video displayed in the video viewing area is a full-screen 2D video, wherein the video displayed by the virtual screen at the current drag position and the video displayed by the virtual screen at the drag start position are both full-screen 2D videos.

[0022] In some embodiments, if the viewing mode of the virtual environment is a 180-degree or 360-degree panoramic video viewing mode, then the video viewing area is a viewfinder with a preset ratio, and the viewfinder is in a hidden state before responding to drag control information.

[0023] The step of responding to drag control information, using the initial position of the ray cursor as the origin of the spherical space, and fixing the first included angle, to drag the video viewing area along the spherical surface of the spherical space includes:

[0024] In response to drag control information, the viewfinder is controlled to be displayed in the virtual environment;

[0025] Using the initial position of the ray cursor as the origin of the spherical space and fixing the first included angle, drag the viewfinder along the spherical surface of the spherical space in the x-axis direction.

[0026] In some embodiments, the method further includes:

[0027] During the dragging process, control the display of the mask in the area of ​​the virtual environment other than the viewfinder;

[0028] When the drag is finished, hide the viewfinder and the mask.

[0029] In some embodiments, the method further includes:

[0030] When the left edge of the viewfinder extends beyond the left edge of the 180-degree panoramic video by a preset distance, dragging stops, and in response to a reset view control command, the left edge of the 180-degree panoramic video moves to the left edge of the viewfinder; or

[0031] When the distance between the right edge of the viewfinder and the right edge of the 180-degree panoramic video reaches a preset distance value, the dragging stops, and in response to the reset view control command, the right edge of the 180-degree panoramic video is moved to the right edge of the viewfinder.

[0032] In some embodiments, prior to responding to the reset view control command, the method further includes:

[0033] A reset view prompt message is displayed in the viewfinder, which prompts the user to input the reset view control command.

[0034] In some embodiments, determining the video frame displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment includes:

[0035] Based on the current drag position of the viewfinder in the virtual environment, the video image displayed by the viewfinder is determined to be the video texture corresponding to the current drag position in the panoramic video, wherein the video image displayed by the viewfinder at the current drag position is different from the video image displayed by the viewfinder at the drag start position.

[0036] In some embodiments, the method further includes: during dragging, hiding the ray cursor and displaying the cursor focus of the ray cursor located on the video viewing area.

[0037] In some embodiments, the method further includes:

[0038] If the drag control information is generated based on the buttons of the object-controlled interactive device, then in response to the drag control information, a vibration prompt message is sent to the interactive device. The vibration prompt message is used to indicate that the interactive device vibrates to indicate that the drag operation has been triggered.

[0039] In some embodiments, the method further includes:

[0040] If the drag control information is generated based on the object's bare hand gesture, then a voice prompt is issued in response to the drag control information, and the voice prompt is used to indicate that the drag operation has been triggered.

[0041] On the other hand, embodiments of this application provide a viewing screen adjustment device in a virtual environment, the device comprising:

[0042] The display unit is used to display a virtual environment, in which a ray cursor and a video viewing area are presented, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area;

[0043] A control unit is configured to respond to drag control information and drag the video viewing area based on the initial position of the ray cursor and the first angle.

[0044] The determining unit is used to determine the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment.

[0045] On the other hand, embodiments of this application provide a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the viewing screen adjustment method in a virtual environment as described in any of the above embodiments.

[0046] On the other hand, this application provides a virtual reality device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the viewing screen adjustment method in the virtual environment as described in any of the above embodiments by calling the computer program stored in the memory.

[0047] On the other hand, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method for adjusting the viewing screen in a virtual environment as described in any of the above embodiments.

[0048] This application embodiment displays a virtual environment with a ray cursor and a video viewing area. The ray cursor points towards the video viewing area and forms a first angle with it. Responding to drag control information and based on the initial position of the ray cursor and the first angle, the video viewing area is dragged. The video image displayed in the video viewing area is determined based on its current drag position within the virtual environment. This application embodiment designs a drag-and-drop method for 2D videos, VR180 videos, and VR360 videos in three-dimensional space, allowing users to experience the charm of VR space and different viewing angles in the video field, thus enhancing the immersive experience of watching videos in virtual reality space. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating a method for adjusting the viewing screen in a virtual environment, as provided in an embodiment of this application.

[0051] Figure 2 This is a schematic diagram of a first application scenario for the method of adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0052] Figure 3 This is a schematic diagram of a second application scenario for the method of adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0053] Figure 4 This is a schematic diagram of a third application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0054] Figure 5 This is a schematic diagram of a fourth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0055] Figure 6 This is a schematic diagram of the fifth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0056] Figure 7 This is a schematic diagram of the sixth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0057] Figure 8 This is a schematic diagram of the seventh application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0058] Figure 9 This is a schematic diagram of the eighth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0059] Figure 10 This is a schematic diagram of the ninth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0060] Figure 11 This is a schematic diagram of the tenth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0061] Figure 12 This is a schematic diagram of the eleventh application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0062] Figure 13 This is a schematic diagram of the twelfth application scenario of the method for adjusting the viewing screen in a virtual environment provided in the embodiments of this application.

[0063] Figure 14 This is a schematic diagram of the structure of the viewing screen adjustment device in the virtual environment provided in the embodiments of this application.

[0064] Figure 15 This is a first structural schematic diagram of a virtual reality device provided in an embodiment of this application.

[0065] Figure 16 This is a schematic diagram of the second structure of a virtual reality device provided in an embodiment of this application. Detailed Implementation

[0066] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] This application provides a method, apparatus, computer-readable storage medium, virtual reality device, server, and computer program product for adjusting the viewing image in a virtual environment. Specifically, the method for adjusting the viewing image in a virtual environment according to this application embodiment can be executed by a virtual reality device or by a server.

[0068] The embodiments of this application can be applied to various application scenarios such as extended reality (XR), virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0069] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0070] Extended Reality (XR) is a concept that includes Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). It refers to the technology that creates an environment that connects the virtual world with the real world, allowing users to interact with that environment in real time.

[0071] Virtual Reality (VR) is a technology for creating and experiencing virtual worlds. It computationally generates a virtual environment, which is a multi-source information (virtual reality mentioned in this article includes at least visual perception, and may also include auditory perception, tactile perception, motion perception, and even taste perception, olfactory perception, etc.) that realizes the fusion of virtual environment, interactive three-dimensional dynamic visual scenes and simulation of physical behavior, allowing users to immerse themselves in the simulated virtual reality environment, and enabling applications in various virtual environments such as maps, games, videos, education, medical care, simulation, collaborative training, sales, assisted manufacturing, maintenance and repair.

[0072] Augmented Reality (AR) is a technology that calculates the camera's pose parameters in the real world (or 3D world, the real world) in real time during image capture, and adds virtual elements to the captured images based on these parameters. Virtual elements include, but are not limited to, images, videos, and 3D models. The goal of AR technology is to overlay the virtual world onto the real world on a screen for interactive experiences.

[0073] Mixed Reality (MR) is a simulated scene that integrates computer-created sensory input (e.g., virtual objects) with sensory input or its representation from a physical setting. In some MR scenes, the computer-created sensory input can adapt to changes in sensory input from the physical setting. Additionally, some electronic systems used to present MR scenes can monitor orientation and / or position relative to the physical setting, enabling virtual objects to interact with real objects (i.e., physical elements from the physical setting or their representations). For example, the system can monitor motion so that virtual plants appear stationary relative to physical buildings.

[0074] Augmented Virtuality (AV): An AV scene refers to a computer-created scene or virtual scene that incorporates at least one sensory input from a physical scene. The one or more sensory inputs from the physical scene can be a representation of at least one feature of the physical scene. For example, virtual objects can present the colors of physical elements captured by one or more imaging sensors. As another example, virtual objects can present features consistent with actual weather conditions in a physical scene, such as those identified via weather-related imaging sensors and / or online weather data. In another example, an augmented reality forest can have virtual trees and structures, but animals can have features accurately reproduced from images taken of physical animals.

[0075] The virtual field of view (FOV) is the area in the virtual environment that a user can perceive through lenses in a virtual reality device. The perceived area is represented by the field of view angle (FOV).

[0076] Virtual reality devices, the terminals that enable virtual reality effects, can typically be provided in the form of glasses, head-mounted displays (HMDs), or contact lenses to achieve visual perception and other forms of perception. Of course, the form of virtual reality devices is not limited to these, and they can be further miniaturized or enlarged as needed.

[0077] The virtual reality devices described in this application may include, but are not limited to, the following types:

[0078] PC-based virtual reality (PCVR) devices utilize a PC for calculations and data output related to virtual reality functions. External PC-based virtual reality devices then use the data output from the PC to achieve the virtual reality effect.

[0079] Mobile virtual reality devices support setting up mobile terminals (such as smartphones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connections with the mobile terminal, the mobile terminal performs calculations related to virtual reality functions and outputs data to the mobile virtual reality device, such as watching virtual reality videos through a mobile terminal's app.

[0080] All-in-one virtual reality devices have a processor for performing virtual functions, thus possessing independent virtual reality input and output capabilities. They do not require connection to a PC or mobile terminal, offering a high degree of freedom of use.

[0081] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0082] This application provides a method for adjusting the viewing screen in a virtual environment. This method can be executed by a terminal or a server, or by both a terminal and a server. This application uses the example of the method for adjusting the viewing screen in a virtual environment being executed by a terminal (virtual reality device) to illustrate the method.

[0083] Please see Figures 1 to 13 , Figure 1 This is a flowchart illustrating the method for adjusting the viewing screen in a virtual environment provided in an embodiment of this application. Figures 2 to 13 These are all schematic diagrams illustrating relevant application scenarios provided in the embodiments of this application, wherein, Figure 2 The blank background in the image can be a three-dimensional virtual space layer. The method includes:

[0084] Step 110: Display a virtual environment, in which a ray cursor and a video viewing area are presented, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area.

[0085] For example, such as Figure 2 As shown, a virtual environment 10 is displayed, containing a ray cursor 11 and a video viewing area 12. The ray cursor 11 points towards the video viewing area 12, forming a first angle α between them. For example, to enrich the visual presentation of the virtual environment 10, a virtual controller 13 can be displayed at the starting position 111 of the ray cursor 11. A ray cursor 11 radiating towards the video viewing area 12 is positioned directly in front of the virtual controller 13, forming a first angle α between the ray cursor 11 and the video viewing area 12. This first angle α is the angle formed between the ray cursor 11 and the plane containing the video viewing area 12.

[0086] In 2D, the viewing area is the video panel (hereinafter referred to as the virtual screen) shown in the presentation; in VR180 and VR360, the viewing area is the visual focus area, which is the viewfinder that is dragged later.

[0087] In the virtual environment 10, the ray cursor 11 and the video viewing area 12 can be displayed normally or hidden. For example, if the viewing mode of the virtual environment 10 is a two-dimensional video viewing mode, the video viewing area 12 can be a virtual screen. Before and during the dragging of the virtual screen, the virtual screen can be in a display state to simulate an object (which can be a user) in a movie theater or other viewing scenario. The ray cursor 11 can be in a display state before the virtual screen is dragged, and it can be in a hidden state during the dragging of the virtual screen. For example, if a virtual handle 13 is also present at the starting position 111 of the ray cursor 11, the virtual handle 13 can be in a display state before the virtual screen is dragged, and it can be in a hidden state during the dragging of the virtual screen. For example, if the viewing mode of the virtual environment 10 is a 180-degree or 360-degree panoramic video viewing mode, then the video viewing area 12 can be a viewfinder with a preset ratio (equivalent to the user's visual focus area in the virtual environment). Before the viewfinder is dragged, the viewfinder can be hidden to simulate the user being in a panoramic viewing scene, and the entire virtual environment displays a panoramic video. During the dragging process, the viewfinder can be displayed. The ray cursor 11 can be displayed before the viewfinder is dragged, and it can be hidden during the dragging process. For example, if a virtual handle 13 is also present at the starting position 111 of the ray cursor 11, then the virtual handle 13 can be displayed before the viewfinder is dragged, and it can be hidden during the dragging process.

[0088] Step 120: In response to the drag control information, and based on the initial position of the ray cursor and the first angle, drag the video viewing area.

[0089] For example, drag-and-drop control information can be generated based on the buttons of an object-oriented interactive device. For example, a virtual environment can be displayed on the screen of a virtual reality device. This virtual reality device can be connected to an external interactive device, which can drive a raycast cursor in the virtual environment by manipulating the buttons of the interactive device, and can control the raycast cursor to drive drag operations on the video viewing area in response to drag-and-drop control information. For example, the interactive device can include a controller, digital gloves, or a specific interactive device.

[0090] For example, drag control information can be generated based on bare-hand gestures of an object. This drag control information can then be used to control the raycast cursor to perform drag operations on the video viewing area.

[0091] In some embodiments, the step of responding to drag control information and dragging the video viewing area based on the initial position of the ray cursor and the first angle includes:

[0092] In response to drag control information, the video viewing area is dragged along the spherical surface of the spherical space with the initial position of the ray cursor as the origin of the spherical space and the first included angle fixed.

[0093] For example, such as Figure 3 As shown, a spherical space 20 can be set up as a viewing container. When entering the viewing mode, the video can be viewed in the viewing container of the spherical space 20. When responding to drag control information, the origin A of the spherical space 20 is taken as the starting position 111 of the ray cursor 11, and the first angle α between the ray cursor 11 and the plane where the video viewing area 12 is located is fixed. The video viewing area 12 is dragged along the spherical surface of the spherical space 20. By controlling the cursor focus 112 of the ray cursor 11 to be located within the draggable video viewing area, and along the x-direction and / or y-axis direction, human-computer interaction can be performed in the spherical space to achieve dragging of the viewing screen.

[0094] For example, a cube-shaped space can also be set up as a viewing container. When entering viewing mode, the video can be watched within the viewing container of this cube-shaped space. In response to drag control information, the initial position of the ray cursor is used as the center point of the cube-shaped space, and the first angle between the ray cursor and the plane containing the video viewing area is fixed. The video viewing area is then dragged along the surface of the cube-shaped space. By controlling the cursor focus of the ray cursor to be within the draggable video viewing area, and by dragging the viewing screen along the x-axis and / or y-axis, human-computer interaction can be performed within the cube-shaped space.

[0095] In some embodiments, if the viewing mode of the virtual environment is a two-dimensional video viewing mode, then the video viewing area is a virtual screen, and the virtual screen is in a display state before responding to drag control information; dragging the video viewing area along the spherical surface of the spherical space includes: dragging the virtual screen along the spherical surface of the spherical space in at least one of the x-axis direction and the y-axis direction.

[0096] For example, if the viewing mode of the virtual environment is a two-dimensional (2D) video viewing mode, then the video viewing area is a virtual screen, which is in a display state before responding to drag control information. For example, the 2D video viewing mode can be referenced... Figure 2 or Figure 4The virtual environment 10 shown includes a ray cursor 11 and a video viewing area 12 (i.e., a virtual screen 121). The ray cursor 11 points towards the video viewing area 12 and forms a first angle α with the video viewing area 12. For example, to enrich the presentation of the virtual environment 10, a virtual controller 13 can be presented at the starting position 111 of the ray cursor 11. Directly in front of the virtual controller 13, a ray cursor 11 is positioned towards the video viewing area 12 (i.e., the virtual screen 121), and the ray cursor 11 forms a first angle α with the video viewing area 12 (i.e., the virtual screen 121).

[0097] For example, in 2D video viewing mode, the drag trigger conditions include the ray cursor 11 pointing to the video viewing area 12 (i.e., virtual screen 121), and the cursor focus 112 of the ray cursor 11 being within the video viewing area 12 (i.e., virtual screen 121). The video viewing area 12 (i.e., virtual screen 121) can be understood as the video hot zone. The object generates drag control information by long-pressing a preset button in the interactive device (such as a controller), and when the virtual reality device obtains the drag control information, it can respond to the drag control information by controlling the displacement of the ray cursor 11 in the x / y axis direction to be greater than or equal to xdp, so as to drag the video viewing area 12 (i.e., virtual screen 121).

[0098] For example, 'x' in xdp is a dynamic value that can be adjusted based on the force of the user's drag, allowing for different values ​​to be assigned in different scenarios. 'dp' stands for pixel, which is density-independent and is an abstract unit based on screen density. On a 160-dot-per-inch display, 1dp = 1px.

[0099] For example, preset buttons may include the Grip button, the Trigger button, and the A / X button (which functions the same as the Trigger button).

[0100] For example, in addition to the drag-and-drop trigger conditions mentioned above, interaction avoidance conditions can be set before responding to drag-and-drop control information. For example, interaction avoidance conditions include either of the following: dragging occurs when the "Settings Panel" pops up; or dragging occurs when the "Settings Panel" disappears. Either of these two conditions can be selected for interaction avoidance, such as setting it to trigger dragging when the "Settings Panel" pops up, or setting it to trigger dragging when the "Settings Panel" disappears.

[0101] For example, when there is a conflict in web page interaction, the interaction avoidance conditions may also include: when drag is triggered, "video viewing area drag" is executed first, and "web page interaction" is not executed.

[0102] For example, in 2D video viewing mode, when responding to drag control information, the dragged object is the video viewing area 12 (i.e., virtual screen 121). Following the dragged video viewing area 12 (i.e., virtual screen 121), the video displayed in the video viewing area 12 (i.e., virtual screen 121) is a 2D full-screen video. During the dragging process, the first angle α between the ray cursor 11 and the video viewing area 12 (i.e., virtual screen 121) is fixed, and the video viewing area 12 (i.e., virtual screen 121) is dragged by moving the ray cursor 11.

[0103] like Figure 3 As shown, in the 2D video viewing mode, the video viewing area 12 (i.e., the virtual screen 121) is dragged spherically along the surface of the spherical space 20 in at least one of the x-axis and y-axis directions, i.e., the video viewing area 12 (i.e., the virtual screen 121) moves with full xy-degree-of-freedom spherical movement. Specifically, the origin A of the spherical space 20 is taken as the starting position 111 of the ray cursor 11, and the radius of the spherical space 20 is taken as the radius. The first angle α between the ray cursor 11 and the video viewing area 12 (i.e., the virtual screen 121) is fixed, and the position (x-axis coordinate and y-axis coordinate) of the landing point B is moved spherically. That is, the content seen by the object on the display screen of the virtual reality device is always the position of point B, which is tangent to the spherical surface of the spherical space 20, and the first angle α is kept unchanged during dragging. For example, the line segment AB in the figure represents the ray cursor 11 radiating from the virtual controller 13 to the video viewing area 12 (i.e., the virtual screen 121).

[0104] For example, during the dragging process, the RotationZ (horizontal rotation) value of the video viewing area is always 0.

[0105] like Figure 5 As shown, when dragging, the center point of the virtual screen 121 moves along the spherical surface of the spherical space 20 in at least one of the x-axis and y-axis directions.

[0106] In some embodiments, the method further includes: if the virtual screen is dragged along the spherical surface of the spherical space in the y-axis direction to the top or bottom in the y-axis direction, then the virtual screen is controlled to rotate 180 degrees around the center of the virtual screen.

[0107] For example, in 2D video viewing mode, after reaching the top / bottom along the y-axis, the virtual screen is rotated 180°. For example, always ensure that the top margin of the virtual screen is greater than or equal to the bottom margin along the y-axis. When dragging to the point where the top margin equals the bottom margin (dragdling to the very top / bottom), simultaneously respond to the drag and rotate the screen 180° around its center point.

[0108] For example, such as Figure 6As shown, during dragging, the center point of the virtual screen 121 moves along the spherical surface of the spherical space 20 in the y-axis direction. The center point of the virtual screen 121 is tangent to the spherical surface. During the continued dragging process, when the virtual screen 121 is dragged to the top or bottom in the y-axis direction, the virtual screen 121 is controlled to rotate 180°. If the dragging operation is still ongoing, it continues to respond to the displacement changes of the dragging after the rotation.

[0109] In some embodiments, the method further includes: if the virtual screen is dragged along the spherical surface of the spherical space to a virtual ground in the virtual environment, and the virtual screen and the virtual ground clip through each other, then the virtual ground is hidden.

[0110] For example, such as Figure 2 As shown, in 2D video viewing mode, when dragging causes the virtual screen 121 to clip through the virtual ground 14, the virtual ground 14 disappears, resulting in a scene without the virtual ground 14. The smooth dragging experience can be ensured by setting disappearance handling conditions. For example, the disappearance handling condition could be that when the virtual ground 14 is clipped through by the virtual screen 121, the virtual ground 14 gradually disappears based on the dragging speed; and when the virtual screen 121 completely passes through the virtual ground 14, the virtual ground 14 is completely hidden.

[0111] In some embodiments, the method further includes:

[0112] During the dragging process, the border of the virtual screen is highlighted.

[0113] When the drag is finished, control the border of the virtual screen to return to normal display.

[0114] For example, in 2D video viewing mode, during the dragging process, such as Figure 7 As shown, the border 1211 of the virtual screen 121 is highlighted. For example, when the drag ends, the border of the virtual screen returns to its normal state. For example, during the drag, the controller connected to the virtual reality device will vibrate; when the drag ends, the controller will stop vibrating.

[0115] For example, you can generate a reset view command by long-pressing the "Home" button, and then reset the view in response to the reset view command, resetting the position of the virtual screen to the default position in the view.

[0116] like Figure 8 As shown, when the playback control bar 15 and the settings panel 16 are displayed, if the focus of the ray cursor is within the virtual screen 121 and dragging is triggered outside the area shown in the settings panel 16, the virtual screen 121 will still respond to the dragging operation; when dragging, the playback control bar 15 and the settings panel 16 can be temporarily hidden.

[0117] In some embodiments, if the viewing mode of the virtual environment is a 180-degree or 360-degree panoramic video viewing mode, then the video viewing area is a viewfinder with a preset ratio, and the viewfinder is hidden before responding to drag control information; the step of responding to drag control information, using the initial position of the ray cursor as the origin of the spherical space, and fixing the first included angle, and dragging the video viewing area along the spherical surface of the spherical space, includes:

[0118] In response to drag control information, the viewfinder is controlled to be displayed in the virtual environment;

[0119] Using the initial position of the ray cursor as the origin of the spherical space and fixing the first included angle, drag the viewfinder along the spherical surface of the spherical space in the x-axis direction.

[0120] For example, if the virtual environment's viewing mode is a 180-degree or 360-degree panoramic video viewing mode, then the video viewing area is a viewfinder with a preset aspect ratio, and the viewfinder is hidden before responding to drag control information. For example, the preset aspect ratio viewfinder can be a 16:9 viewfinder. For example, the viewfinder can be a 1280*720 area.

[0121] For example, in 180-degree or 360-degree panoramic video viewing mode, the drag trigger conditions include the ray cursor pointing to the video viewing area (i.e., the viewfinder), and the ray cursor's focus being within the video viewing area (i.e., the viewfinder). This video viewing area (i.e., the viewfinder) can be understood as the video hotspot range (which can also be understood as the visual focus area in the immersive experience). Furthermore, the object generates drag control information by long-pressing a preset button on the interactive device (such as a controller), and when the virtual reality device obtains this drag control information, it can respond to the drag control information by controlling the displacement of the ray cursor in the x / y axis direction to be greater than or equal to xdp, so as to drag the video viewing area (i.e., the viewfinder).

[0122] For example, preset buttons may include the Grip button, the Trigger button, and the A / X button (which functions the same as the Trigger button).

[0123] For example, in addition to the drag-and-drop trigger conditions mentioned above, interaction avoidance conditions can also be set before responding to drag-and-drop control information. For example, interaction avoidance conditions include any of the following: dragging when the "Settings Panel" pops up; dragging when the "Settings Panel" pops up; dragging when the "Settings Panel" and "Immersive Bar" disappear; dragging when the "Immersive Bar" is displayed; dragging when the "Immersive Bar" disappears. Only one of these four conditions can be selected for interaction avoidance.

[0124] For example, see Figure 3 In 180-degree or 360-degree panoramic video viewing mode, Figure 3 The video viewing area 12 can be a viewfinder 122. When responding to drag control information, the dragged object is the video viewing area 12 (i.e., viewfinder 122). Following the dragged video viewing area 12 (i.e., viewfinder 122), the video image displayed in the video viewing area 12 (i.e., viewfinder 122) is the video texture corresponding to the current drag position in the panoramic video. During the dragging process, the first included angle α between the ray cursor 11 and the video viewing area 12 (i.e., viewfinder 122) is fixed, and the video viewing area 12 (i.e., viewfinder 122) is dragged by moving the ray cursor 11.

[0125] like Figure 3 As shown, in the 180-degree or 360-degree panoramic video viewing mode, in the horizontal direction of the x-axis, with the starting position 111 of the ray cursor 11 as the origin A of the spherical space 20, and the ray cursor 11 as the radius of the spherical space 20, the first angle α between the ray cursor 11 and the video viewing area 12 (i.e., the viewfinder 122) is fixed, and the video viewing area 12 (i.e., the viewfinder 122) is dragged horizontally along the spherical surface of the spherical space 20.

[0126] In some embodiments, the method further includes:

[0127] During the dragging process, control the display of the mask in the area of ​​the virtual environment other than the viewfinder;

[0128] When the drag is finished, hide the viewfinder and the mask.

[0129] For example, such as Figure 9 or Figure 10 As shown, in 180-degree or 360-degree panoramic video viewing modes, during dragging, the area of ​​the virtual environment 10 excluding the viewfinder 122 displays a mask 17. For example, this mask 17 can be a black mask or a mask of another color. For instance, during dragging, only the video texture at the corresponding position is visible within the viewfinder 122; the mask 17 outside the viewfinder 122 is displayed as black. At the end of the drag, the viewfinder 122 and the mask 17 are hidden, and the entire immersive video image of the virtual environment 10 is visible in the immersive experience. For instance, during dragging, the controller connected to the virtual reality device vibrates; at the end of the drag, the controller stops vibrating.

[0130] For example, you can generate a reset view command by long-pressing the "Home" button, and then reset the view in response to the reset view command, resetting the position of the viewfinder 122 to the default position in the view.

[0131] For example, such as Figure 11As shown, when the playback control bar 15 and the settings panel 16 are displayed, if the focus of the ray cursor 11 is within the viewfinder 122 and dragging is triggered outside the area shown in the settings panel 16, the viewfinder 122 will still respond to the dragging operation; when dragging, the playback control bar 15 and the settings panel 16 can be temporarily hidden.

[0132] In some embodiments, the method further includes:

[0133] When the left edge of the viewfinder extends beyond the left edge of the 180-degree panoramic video by a preset distance, dragging stops, and in response to a reset view control command, the left edge of the 180-degree panoramic video moves to the left edge of the viewfinder; or

[0134] When the distance between the right edge of the viewfinder and the right edge of the 180-degree panoramic video reaches a preset distance value, the dragging stops, and in response to the reset view control command, the right edge of the 180-degree panoramic video is moved to the right edge of the viewfinder.

[0135] In some embodiments, prior to responding to the reset view control command, the method further includes:

[0136] A reset view prompt message is displayed in the viewfinder, which prompts the user to input the reset view control command.

[0137] For example, such as Figure 12 As shown, if the right edge of the viewfinder 122 is dragged beyond the right edge xdp (preset distance value) of the panoramic video, it cannot be dragged further. At this time, the video image displayed in the viewfinder 122 is incomplete. A reset view prompt message can be displayed in the viewfinder as a toast message box 1221. For example, the content of the reset view prompt message is "Press and hold the Home button on one side of the controller to reset the view" or "Press and hold the ○ button on one side of the controller to reset the view". The user triggers the reset view control command by pressing and holding the Home button or the ○ button according to the content of the reset view prompt message, so that the virtual reality device responds to the reset view control command to perform a reset view operation, such as controlling the left edge of the 180-degree panoramic video to move to the left edge of the viewfinder, or controlling the right edge of the 180-degree panoramic video to move to the right edge of the viewfinder.

[0138] For example, a toast message box 1221 is used to display a reset view prompt in the viewfinder 122. This toast message box 1221 has no control buttons, does not receive focus, and disappears automatically after a period of time.

[0139] For example, when resetting the viewfinder, the screen you see is the video edge bouncing back to the viewfinder edge; that is, the left edge of the video bouncing back to the left edge of the viewfinder, or the right edge of the video bouncing back to the right edge of the viewfinder. If the viewfinder edge is not dragged beyond the video edge, no toast message will appear.

[0140] For example, the preset distance value is xdp, where x is a dynamic value that can be set according to the force of the user's drag. Different values ​​can be assigned in different scenarios.

[0141] In some embodiments, the method further includes:

[0142] If the drag control information is generated based on the buttons of the object-controlled interactive device, then in response to the drag control information, a vibration prompt message is sent to the interactive device. The vibration prompt message is used to indicate that the interactive device vibrates to indicate that the drag operation has been triggered.

[0143] For example, taking a controller as an example of an interactive device, when a drag operation is triggered, a vibration alert is sent to the controller. The vibration alert indicates that the controller is vibrating, and the controller responds by vibrating momentarily to indicate to the object that a drag operation has been triggered. For example, the vibration duration is x seconds, such as 3 seconds.

[0144] In some embodiments, the method further includes:

[0145] If the drag control information is generated based on the object's bare hand gesture, then a voice prompt is issued in response to the drag control information, and the voice prompt is used to indicate that the drag operation has been triggered.

[0146] For example, when a drag-and-drop operation is triggered, a voice prompt can be issued to indicate that the drag-and-drop operation has been triggered.

[0147] In some embodiments, the method further includes: during dragging, hiding the ray cursor and displaying the cursor focus of the ray cursor located on the video viewing area.

[0148] For example, during dragging in 2D video viewing mode: the ray cursor 11 can be hidden, only the cursor focus 112 can be displayed. The playback bar 15 can also be hidden; a non-normal (standard) state can be displayed in full-screen video; the video playback / pause state displayed in the virtual screen 121 remains unchanged. For example, during dragging in 2D video viewing mode, the virtual environment is displayed as follows: Figure 7 As shown, the video displayed in the video viewing area 12 (i.e., virtual screen 121) is a 2D full-screen video.

[0149] For example, during dragging in 180-degree or 360-degree panoramic video viewing mode: the ray cursor 11 can be hidden, and only the cursor focus 112 can be displayed. The video play / pause status displayed in the viewfinder 122 remains unchanged. For example, during dragging in 180-degree or 360-degree panoramic video viewing mode, a virtual environment is displayed such as... Figure 9 As shown, the video image displayed in the video viewing area 12 (i.e., viewfinder 122) is the video texture corresponding to the dragged position.

[0150] Step 130: Based on the current drag position of the video viewing area in the virtual environment, determine the video screen displayed in the video viewing area.

[0151] In some embodiments, if the viewing mode of the virtual environment is a two-dimensional video viewing mode, determining the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment includes:

[0152] Based on the current drag position of the virtual screen in the virtual environment, it is determined that the video displayed in the video viewing area is a full-screen 2D video, wherein the video displayed by the virtual screen at the current drag position and the video displayed by the virtual screen at the drag start position are both full-screen 2D videos.

[0153] For example, in 2D video viewing mode, when an object releases a preset button after a long press, a drag-to-end command is generated, causing the virtual reality device to respond to the drag-to-end command and cancel the drag. At this point, the virtual screen remains at the current drag position. Figure 13 As shown, after the drag is complete, the entire full-screen UI display of the virtual screen remains as a whole, maintaining the relative positions of the "Full-screen bar," "Full-screen video," and "Settings panel." The full-screen video is a two-dimensional video played in full-screen mode.

[0154] Before, during, and after dragging, the video displayed in the video viewing area in the 2D video viewing mode is always a full-screen video, and the video playback or pause status remains unchanged during the dragging process.

[0155] For example, in 2D video viewing mode, after dragging and exiting the virtual environment, the drag position is not recorded. The next time you enter the virtual environment, the display shows the default state. For example, the default state represents the initial state of the display system. For instance, when a user is wearing a head-mounted virtual reality device and lying down to watch a movie, the displayed video image will be tilted at 45° (the video image after dragging); however, the next time the user wears the virtual reality device, the displayed video image will be the normal 90° image (the default state).

[0156] For example, you don't need to record the virtual screen's position dragging information, but you can record the ray cursor's scaling information. This scaling information includes the ray cursor's scaling size, which can be used to zoom the virtual screen closer or further away based on the screen's center point.

[0157] In some embodiments, if the viewing mode of the virtual environment is a 180-degree or 360-degree panoramic video viewing mode, determining the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment includes:

[0158] Based on the current drag position of the viewfinder in the virtual environment, the video image displayed by the viewfinder is determined to be the video texture corresponding to the current drag position in the panoramic video, wherein the video image displayed by the viewfinder at the current drag position is different from the video image displayed by the viewfinder at the drag start position.

[0159] For example, in 180-degree or 360-degree panoramic video viewing mode, when the object releases the preset button after a long press, a drag-to-end command is generated, causing the virtual reality device to respond to the drag-to-end command and cancel the drag. At this time, the viewfinder remains at the current drag position and disappears with an alpha change. The video image displayed in the viewfinder remains at the current drag position and plays. The video image displayed in the viewfinder is the video texture corresponding to the current drag position in the panoramic video. The video image displayed in the viewfinder at the current drag position is different from the video image displayed in the viewfinder at the start of the drag.

[0160] For example, in VR180° immersive viewing mode, it's a 180° panoramic view, equivalent to a hemisphere. When an object wears a virtual reality device, they can view a panoramic video within a 180° range. When the video content displayed in the viewfinder is dragged outside the 180° viewable area due to the movement of the viewfinder, a black mask or alpha gradient appears to blend it into the viewing experience. For instance, if the scene is pure black and the viewing screen is in color, dragging the video content outside the 180° range will create a transition from the color viewing screen to the pure black scene; this transition is the alpha change.

[0161] For example, in 180-degree or 360-degree panoramic video viewing mode, after dragging and exiting the virtual environment, the drag position is not recorded. The next time you enter the virtual environment, the display shows the default state.

[0162] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0163] This application embodiment displays a virtual environment with a ray cursor and a video viewing area. The ray cursor points towards the video viewing area and forms a first angle with it. Responding to drag control information and based on the initial position of the ray cursor and the first angle, the video viewing area is dragged. The video image displayed in the video viewing area is determined based on its current drag position within the virtual environment. This application embodiment designs a drag-and-drop method for 2D videos, VR180 videos, and VR360 videos in three-dimensional space, allowing users to experience the charm of VR space and different viewing angles in the video field, thus enhancing the immersive experience of watching videos in virtual reality space.

[0164] To facilitate better implementation of the viewing screen adjustment method in a virtual environment according to the embodiments of this application, the embodiments of this application also provide a viewing screen adjustment device in a virtual environment. Please refer to... Figure 14 , Figure 14 A schematic diagram of the structure of a viewing screen adjustment device in a virtual environment provided in this application embodiment. The viewing screen adjustment device 200 in the virtual environment may include:

[0165] Display unit 210 is used to display a virtual environment, wherein a ray cursor and a video viewing area are presented in the virtual environment, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area;

[0166] Control unit 220 is configured to respond to drag control information and drag the video viewing area based on the initial position of the ray cursor and the first angle.

[0167] The determining unit 230 is used to determine the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment.

[0168] In some embodiments, the control unit 220 is specifically used for:

[0169] In response to drag control information, the video viewing area is dragged along the spherical surface of the spherical space with the initial position of the ray cursor as the origin of the spherical space and the first included angle fixed.

[0170] In some embodiments, if the viewing mode of the virtual environment is a two-dimensional video viewing mode, then the video viewing area is a virtual screen, and the virtual screen is in a display state before responding to drag control information.

[0171] When the control unit 220 drags the video viewing area along the spherical surface of the spherical space, it is specifically used to: drag the virtual screen along the spherical surface of the spherical space in at least one of the x-axis direction and the y-axis direction.

[0172] In some embodiments, the control unit 220 is further configured to: if the virtual screen is dragged along the spherical surface of the spherical space in the y-axis direction to the top or bottom in the y-axis direction, control the virtual screen to rotate 180 degrees around the center of the virtual screen.

[0173] In some embodiments, the control unit 220 is further configured to: hide the virtual ground if the virtual screen is dragged along the spherical surface of the spherical space to the virtual ground in the virtual environment, and the virtual screen and the virtual ground clip through each other.

[0174] In some embodiments, the control unit 220 is further configured to:

[0175] During the dragging process, the border of the virtual screen is highlighted.

[0176] When the drag is finished, control the border of the virtual screen to return to normal display.

[0177] In some embodiments, the determining unit 230 is specifically used for:

[0178] Based on the current drag position of the virtual screen in the virtual environment, it is determined that the video displayed in the video viewing area is a full-screen 2D video, wherein the video displayed by the virtual screen at the current drag position and the video displayed by the virtual screen at the drag start position are both full-screen 2D videos.

[0179] In some embodiments, if the viewing mode of the virtual environment is a 180-degree or 360-degree panoramic video viewing mode, then the video viewing area is a viewfinder with a preset ratio, and the viewfinder is in a hidden state before responding to drag control information.

[0180] The control unit 220 is used for:

[0181] In response to drag control information, the viewfinder is controlled to be displayed in the virtual environment;

[0182] Using the initial position of the ray cursor as the origin of the spherical space and fixing the first included angle, drag the viewfinder along the spherical surface of the spherical space in the x-axis direction.

[0183] In some embodiments, the control unit 220 is further configured to:

[0184] During the dragging process, control the display of the mask in the area of ​​the virtual environment other than the viewfinder;

[0185] When the drag is finished, hide the viewfinder and the mask.

[0186] In some embodiments, the control unit 220 is further configured to:

[0187] When the left edge of the viewfinder extends beyond the left edge of the 180-degree panoramic video by a preset distance, dragging stops, and in response to a reset view control command, the left edge of the 180-degree panoramic video moves to the left edge of the viewfinder; or

[0188] When the distance between the right edge of the viewfinder and the right edge of the 180-degree panoramic video reaches a preset distance value, the dragging stops, and in response to the reset view control command, the right edge of the 180-degree panoramic video is moved to the right edge of the viewfinder.

[0189] In some embodiments, the control unit 220 is further configured to: [further details about the control unit and its configuration before responding to a reset view control command]

[0190] A reset view prompt message is displayed in the viewfinder, which prompts the user to input the reset view control command.

[0191] In some embodiments, the determining unit 230 is specifically used for:

[0192] Based on the current drag position of the viewfinder in the virtual environment, the video image displayed by the viewfinder is determined to be the video texture corresponding to the current drag position in the panoramic video, wherein the video image displayed by the viewfinder at the current drag position is different from the video image displayed by the viewfinder at the drag start position.

[0193] In some embodiments, the control unit 220 is further configured to: hide the ray cursor during dragging and display the cursor focus of the ray cursor located on the video viewing area.

[0194] In some embodiments, the control unit 220 is further configured to:

[0195] If the drag control information is generated based on the buttons of the object-controlled interactive device, then in response to the drag control information, a vibration prompt message is sent to the interactive device. The vibration prompt message is used to indicate that the interactive device vibrates to indicate that the drag operation has been triggered.

[0196] In some embodiments, the control unit 220 is further configured to: if the drag control information is drag control information generated based on the bare hand gesture of an object, then, in response to the drag control information, issue a voice prompt message, the voice prompt message being used to indicate that the drag operation has been triggered.

[0197] Each unit in the viewing screen adjustment device 200 in the aforementioned virtual environment can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in the virtual reality device in hardware form, or stored in the memory of the virtual reality device in software form, so that the processor can call and execute the operations corresponding to each unit.

[0198] The viewing screen adjustment device 200 in the virtual environment can be integrated into a terminal or server that has storage and a processor and thus computing power, or the viewing screen adjustment device 200 in the virtual environment can be the terminal or server.

[0199] In some embodiments, this application also provides a virtual reality device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0200] like Figure 15 As shown, Figure 15 This is a schematic diagram of the structure of a virtual reality device provided in an embodiment of this application. The virtual reality device 300 can typically be provided in the form of glasses, a head-mounted display (HMD), or contact lenses to achieve visual perception and other forms of perception. Of course, the form of the virtual reality device is not limited to these, and it can be further miniaturized or enlarged as needed. The virtual reality device 300 may have, but is not limited to, the following components:

[0201] Detection module 301: Uses various sensors to detect user operation commands and apply them to the virtual environment, such as continuously updating the images displayed on the screen according to the user's gaze, realizing the interaction between the user and the virtual environment and the scene, such as continuously updating the real content based on the detected direction of the user's head rotation.

[0202] Feedback module 302: Receives data from sensors and provides real-time feedback to the user; wherein, the feedback module 302 can be used to display a graphical user interface, such as displaying a virtual environment on the graphical user interface. For example, the feedback module 302 may include a display screen, etc.

[0203] Sensor 303: On the one hand, it receives operation commands from the user and applies them to the virtual environment; on the other hand, it provides the results of the operation to the user in various forms of feedback.

[0204] Control module 304: Controls sensors and various input / output devices, including acquiring user data (such as actions and voice) and outputting sensor data, such as images, vibrations, temperatures, and sounds, which affect the user, the virtual environment, and the real world.

[0205] Modeling module 305: Constructs a 3D model of the virtual environment, and may also include various feedback mechanisms such as sound and touch in the 3D model.

[0206] In this embodiment, a virtual environment can be constructed by the modeling module 305 and displayed by the feedback module 302. The virtual environment presents a ray cursor and a video viewing area, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area; then, the control module 304 responds to the drag control information and drags the video viewing area based on the initial position of the ray cursor and the first angle; then, the control module 304 determines the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment.

[0207] In some embodiments, such as Figure 16 As shown, Figure 16 This is another schematic diagram of the structure of a virtual reality device provided in an embodiment of this application. The virtual reality device 300 further includes a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, and a computer program stored in the memory 320 and executable on the processor. The processor 310 is electrically connected to the memory 320. Those skilled in the art will understand that the virtual reality device structure shown in the figures does not constitute a limitation on the virtual reality device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0208] The processor 310 is the control center of the virtual reality device 300. It connects various parts of the virtual reality device 300 through various interfaces and lines. By running or loading software programs and / or modules stored in the memory 320, and calling data stored in the memory 320, it performs various functions of the virtual reality device 300 and processes data, thereby monitoring the virtual reality device 300 as a whole.

[0209] In this embodiment, the processor 310 in the virtual reality device 300 loads the instructions corresponding to the processes of one or more applications into the memory 320 according to the following steps, and the processor 310 runs the applications stored in the memory 320 to achieve various functions:

[0210] A virtual environment is displayed, in which a ray cursor and a video viewing area are presented. The ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area. In response to drag control information, and based on the initial position of the ray cursor and the first angle, the video viewing area is dragged. Based on the current drag position of the video viewing area in the virtual environment, the video screen displayed in the video viewing area is determined.

[0211] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0212] In some embodiments, the processor 310 may include a detection module 301, a control module 304, and a modeling module 305.

[0213] In some embodiments, such as Figure 16 As shown, the virtual reality device 300 also includes: a radio frequency circuit 306, an audio circuit 307, and a power supply 308. The processor 310 is electrically connected to the memory 320, the feedback module 302, the sensor 303, the radio frequency circuit 306, the audio circuit 307, and the power supply 308. Those skilled in the art will understand that... Figure 15 or Figure 16 The virtual reality device structure shown does not constitute a limitation on the virtual reality device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0214] The radio frequency circuit 306 can be used to transmit and receive radio frequency signals to establish wireless communication with network devices or other virtual reality devices, and to transmit and receive signals with network devices or other virtual reality devices.

[0215] Audio circuitry 307 can be used to provide an audio interface between the user and the virtual reality device via a speaker and a microphone. Audio circuitry 307 converts received audio data into electrical signals, transmits them to the speaker, and the speaker converts them into sound signals for output. Conversely, the microphone converts collected sound signals into electrical signals, which are then received by audio circuitry 307, converted back into audio data, and processed by processor 301. The audio data is then transmitted via radio frequency circuitry 306 to, for example, another virtual reality device, or output to memory for further processing. Audio circuitry 307 may also include an earphone jack to facilitate communication between peripheral headphones and the virtual reality device.

[0216] Power supply 308 is used to power the various components of virtual reality device 300.

[0217] although Figure 15 or Figure 16As not shown in the diagram, the virtual reality device 300 may also include a camera, a wireless fidelity module, a Bluetooth module, an input module, etc., which will not be described in detail here.

[0218] In some embodiments, this application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a virtual reality device or server, and the computer program causes the virtual reality device or server to execute the corresponding processes in the viewing screen adjustment method in the virtual environment of this application embodiment; for brevity, these will not be elaborated further here.

[0219] In some embodiments, this application also provides a computer program product comprising a computer program stored in a computer-readable storage medium. The processor of a virtual reality device reads the computer program from the computer-readable storage medium and executes the computer program, causing the virtual reality device to perform the corresponding processes in the viewing image adjustment method in the virtual environment of this application embodiment. For simplicity, these details are not elaborated here.

[0220] This application also provides a computer program, which includes a computer program stored in a computer-readable storage medium. The processor of the virtual reality device reads the computer program from the computer-readable storage medium and executes the computer program, causing the virtual reality device to perform the corresponding process in the viewing screen adjustment method in the virtual environment of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0221] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0222] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0223] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0224] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0226] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0227] In addition, the functional units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0228] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a virtual reality device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0229] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for adjusting the viewing image in a virtual environment, characterized in that, The method includes: A virtual environment is displayed, in which a ray cursor and a video viewing area are presented, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area; In response to drag control information, and based on the initial position of the ray cursor and the first included angle, dragging the video viewing area includes: in response to drag control information, taking the initial position of the ray cursor as the origin of the spherical space, fixing the first included angle, and dragging the video viewing area along the spherical surface of the spherical space; if the viewing mode of the virtual environment is a two-dimensional video viewing mode, then the video viewing area is a virtual screen, and the virtual screen is in a display state before responding to drag control information; if the virtual screen is dragged along the spherical surface of the spherical space to the virtual ground in the virtual environment, and the virtual screen and the virtual ground clip through each other, then the virtual ground gradually disappears based on the drag speed control, and when the virtual screen completely passes through the virtual ground, the virtual ground is completely hidden; The ray cursor is hidden during the dragging process; The video image displayed in the video viewing area is determined based on the current drag position of the video viewing area in the virtual environment.

2. The method for adjusting the viewing image in a virtual environment as described in claim 1, characterized in that, The process of dragging the video viewing area along the spherical surface of the spherical space includes: The virtual screen is dragged along the spherical surface of the spherical space in at least one of the x-axis and y-axis directions.

3. The method for adjusting the viewing image in a virtual environment as described in claim 2, characterized in that, The method further includes: If the virtual screen is dragged along the spherical surface of the spherical space in the y-axis direction to the top or bottom in the y-axis direction, the virtual screen will be controlled to rotate 180 degrees around the center of the virtual screen.

4. The method for adjusting the viewing screen in a virtual environment as described in claim 2, characterized in that, The method further includes: During the dragging process, the border of the virtual screen is highlighted. When the drag is finished, control the border of the virtual screen to return to normal display.

5. The method for adjusting the viewing image in a virtual environment as described in any one of claims 2-4, characterized in that, The step of determining the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment includes: Based on the current drag position of the virtual screen in the virtual environment, it is determined that the video displayed in the video viewing area is a full-screen 2D video, wherein the video displayed by the virtual screen at the current drag position and the video displayed by the virtual screen at the drag start position are both full-screen 2D videos.

6. The method for adjusting the viewing image in a virtual environment as described in claim 1, characterized in that, If the viewing mode of the virtual environment is a 180-degree or 360-degree panoramic video viewing mode, then the video viewing area is a viewfinder with a preset ratio, and the viewfinder is in a hidden state before responding to drag control information. The step of responding to drag control information, using the initial position of the ray cursor as the origin of the spherical space, and fixing the first included angle, to drag the video viewing area along the spherical surface of the spherical space includes: In response to drag control information, the viewfinder is controlled to be displayed in the virtual environment; Using the initial position of the ray cursor as the origin of the spherical space and fixing the first included angle, drag the viewfinder along the spherical surface of the spherical space in the x-axis direction.

7. The method for adjusting the viewing image in a virtual environment as described in claim 6, characterized in that, The method further includes: During the dragging process, control the display of the mask in the area of ​​the virtual environment other than the viewfinder; When the drag is finished, hide the viewfinder and the mask.

8. The method for adjusting the viewing image in a virtual environment as described in claim 6, characterized in that, The method further includes: When the left edge of the viewfinder extends beyond the left edge of the 180-degree panoramic video by a preset distance, dragging stops, and in response to a reset view control command, the left edge of the 180-degree panoramic video moves to the left edge of the viewfinder; or When the distance between the right edge of the viewfinder and the right edge of the 180-degree panoramic video reaches a preset distance value, the dragging stops, and in response to the reset view control command, the right edge of the 180-degree panoramic video is moved to the right edge of the viewfinder.

9. The method for adjusting the viewing image in a virtual environment as described in claim 8, characterized in that, Prior to the response to the reset view control command, the following is also included: A reset view prompt message is displayed in the viewfinder, which prompts the user to input the reset view control command.

10. The method for adjusting the viewing image in a virtual environment as described in any one of claims 6-9, characterized in that, The step of determining the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment includes: Based on the current drag position of the viewfinder in the virtual environment, the video image displayed by the viewfinder is determined to be the video texture corresponding to the current drag position in the panoramic video, wherein the video image displayed by the viewfinder at the current drag position is different from the video image displayed by the viewfinder at the drag start position.

11. The method for adjusting the viewing screen in a virtual environment as described in claim 1, characterized in that, The method further includes: Displays the cursor focus of the ray cursor located on the video viewing area.

12. The method for adjusting the viewing screen in a virtual environment as described in claim 1, characterized in that, The method further includes: If the drag control information is generated based on the buttons of the object-controlled interactive device, then in response to the drag control information, a vibration prompt message is sent to the interactive device. The vibration prompt message is used to indicate that the interactive device vibrates to indicate that the drag operation has been triggered.

13. The method for adjusting the viewing screen in a virtual environment as described in claim 1, characterized in that, The method further includes: If the drag control information is generated based on the object's bare hand gesture, then a voice prompt is issued in response to the drag control information, and the voice prompt is used to indicate that the drag operation has been triggered.

14. A viewing screen adjustment device in a virtual environment, characterized in that, The device includes: The display unit is used to display a virtual environment, in which a ray cursor and a video viewing area are presented, wherein the ray cursor points in the direction of the video viewing area and forms a first angle with the video viewing area; A control unit, configured to respond to drag control information and drag the video viewing area based on the initial position of the ray cursor and the first included angle, includes: responding to drag control information, using the initial position of the ray cursor as the origin of the spherical space and fixing the first included angle, dragging the video viewing area along the spherical surface of the spherical space; if the viewing mode of the virtual environment is a two-dimensional video viewing mode, then the video viewing area is a virtual screen, and the virtual screen is in a display state before responding to drag control information; if the virtual screen is dragged along the spherical surface of the spherical space to the virtual ground in the virtual environment, and the virtual screen and the virtual ground clip through each other, then the virtual ground is gradually hidden based on the drag speed; when the virtual screen completely passes through the virtual ground, the virtual ground is completely hidden. The control unit is also used to hide the ray cursor during the dragging process; The determining unit is used to determine the video screen displayed in the video viewing area based on the current drag position of the video viewing area in the virtual environment.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the viewing screen adjustment method in a virtual environment as described in any one of claims 1-13.

16. A virtual reality device, characterized in that, The virtual reality device includes a processor and a memory, the memory storing a computer program, and the processor executing the viewing screen adjustment method in the virtual environment according to any one of claims 1-13 by calling the computer program stored in the memory.

17. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method for adjusting the viewing screen in a virtual environment as described in any one of claims 1-13.

Images