VR video control method

Abstract

The present application relates to the technical field of virtual reality. Provided is a VR video control method The method is applied to a mobile terminal, and comprises: in response to an operation of a user, determining a target control mode, wherein the target control mode comprises a gyroscope control mode, a rocker control mode or a touch control mode; acquiring a control parameter corresponding to the target control mode; and on the basis of the control parameter, performing interaction control over a VR video on a display screen of a display terminal. In the method provided in the present application, a control parameter corresponding to a target control mode is acquired by means of a mobile terminal, such that interaction control of a user over a VR video is realized on the basis of the control parameter.

Description

VR video control method

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 2023110791893 and application date of August 24, 2023, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present application relates to the field of virtual reality technology, and in particular to a VR video control method. Background Art

[0004] Virtual reality (VR) technology is maturing, and the widespread adoption of fifth-generation mobile communication technology (5G) is driving the rapid development of VR applications. VR technology utilizes computer simulation systems to simulate the external environment, primarily focusing on the environment, skills, sensors, and perception, providing users with a multi-information, three-dimensional, dynamic, and interactive simulation experience.

[0005] Android-based smart TVs are widely used for VR panoramic content due to their low cost, large screen size, and ability to reduce user discomfort from wearing VR headsets for extended periods. However, current smart TVs lack interactive features like controllers and touchscreens. Remote controls only support up, down, left, and right gestures, making interactive control of VR videos impossible.

[0006] Summary of the Invention

[0007] The embodiments of the present application provide a VR video control method to solve the technical problem in the prior art that the existing remote controller cannot realize interactive control of VR video.

[0008] In a first aspect, an embodiment of the present application provides a VR video control method, applied to a mobile terminal, the method comprising:

[0009] In response to a user operation, determining a target control mode, the target control mode including a gyroscope control mode, a joystick control mode, or a touch control mode;

[0010] Obtaining control parameters corresponding to the target control mode;

[0011] The VR video on the display screen of the display terminal is interactively controlled based on the control parameters.

[0012] In a second aspect, an embodiment of the present application provides a VR video control method, applied to a display terminal, the method comprising:

[0013] Receiving control parameters corresponding to a target control mode sent by a mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode;

[0014] Based on the control parameters, the VR video on the display screen of the display terminal is interactively controlled.

[0015] In a third aspect, an embodiment of the present application provides a VR video control device, applied to a mobile terminal, comprising:

[0016] a first determining module configured to determine a target control mode in response to a user operation, the target control mode including a gyroscope control mode, a joystick control mode, or a touch control mode;

[0017] an acquisition module configured to acquire control parameters corresponding to the target control mode;

[0018] The control module is configured to interactively control the VR video on the display screen of the display terminal based on the control parameters.

[0019] In a fourth aspect, an embodiment of the present application provides a VR video control device, applied to a display terminal, comprising:

[0020] a receiving module configured to receive control parameters corresponding to a target control mode sent by a mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode;

[0021] The control module is configured to interactively control the VR video on the display screen of the display terminal based on the control parameters.

[0022] In a fifth aspect, an embodiment of the present application provides a mobile terminal, comprising a gyroscope, a touch screen, a processor, and a memory storing a computer program;

[0023] The gyroscope is configured to obtain swing parameters; the touch screen is configured to receive touch operations; and when the processor executes the computer program, the steps of any one of the VR video control methods described in the first aspect above are implemented.

[0024] In a sixth aspect, an embodiment of the present application provides a display terminal, comprising a display, a processor, and a memory storing a computer program;

[0025] The display is configured to display VR video; when the processor executes the computer program, the steps of any one of the VR video control methods described in the second aspect above are implemented.

[0026] In the seventh aspect, an embodiment of the present application provides a VR video control system, comprising the mobile terminal described in the fifth aspect and the display terminal described in the sixth aspect.

[0027] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the VR video control method described in the first aspect or the second aspect.

[0028] In a ninth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the VR video control method described in the first aspect or the second aspect.

[0029] The present application provides a VR video control method in which a mobile terminal responds to a user's operation and determines a target control mode. The method then obtains control parameters corresponding to the target control mode to interactively control the VR video based on the control parameters. The target control modes include a gyroscope control mode, a joystick control mode, or a touch control mode. In any of these target control modes, the user can interactively control the VR video based on the obtained control parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 is a flow chart of a VR video control method according to an embodiment of the present application;

[0031] FIG2 is a functional schematic diagram of a gyroscope control mode in a VR video control method provided in an embodiment of the present application;

[0032] FIG3 is a functional schematic diagram of a joystick control mode in a VR video control method provided in an embodiment of the present application;

[0033] FIG4 is a functional schematic diagram of a touch control mode in a VR video control method provided in an embodiment of the present application;

[0034] FIG5 is a second flow chart of the VR video control method provided in an embodiment of the present application;

[0035] FIG6 is a schematic diagram of virtual camera construction in the VR video control method provided in an embodiment of the present application;

[0036] FIG7 is a flowchart of a VR video control method provided in an embodiment of the present application;

[0037] FIG8 is a signaling flow chart of a VR video control method provided in an embodiment of the present application;

[0038] FIG9 is a structural diagram of a VR video control device according to an embodiment of the present application;

[0039] FIG10 is a second structural diagram of the VR video control device provided in an embodiment of the present application;

[0040] FIG11 is a schematic diagram of the physical structure of a mobile terminal provided in an embodiment of the present application;

[0041] FIG12 is a schematic diagram of the physical structure of the display terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0043] It should be noted that the serial numbers assigned to the objects described in this application, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning.

[0044] Virtual reality (VR) video can be any VR application displayed in video format, such as VR tours, VR games, or VR movies. In addition to using dedicated VR devices like VR headsets, VR videos can also be displayed on display terminals like smart TVs.

[0045] Existing smart TVs are usually controlled by traditional remote controls, and traditional remote controls are only equipped with control buttons that support two-dimensional remote control operations such as up, down, left, and right. The remote control does not have a built-in gyroscope or a touch screen, which makes it impossible to obtain three-dimensional remote control actions input by the user. Therefore, when using a smart TV to display VR videos, the three-dimensional control parameters input by the user cannot be obtained through the remote control, resulting in poor controllability of VR video control and inability to meet user needs for VR panoramic applications.

[0046] Based on this problem, an embodiment of the present application provides a VR video control method, wherein the VR video control method applied to a mobile terminal can obtain control parameters corresponding to the target control mode through the mobile terminal, and interactively control the VR video on the display screen displayed by the display terminal based on the control parameters to meet user needs.

[0047] FIG1 is a flowchart of a VR video control method according to an embodiment of the present application. Referring to FIG1 , an embodiment of the present application provides a VR video control method, which is applied to a mobile terminal and may include steps 110 to 130.

[0048] Step 110 : In response to the user's operation, determine a target control mode, where the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode.

[0049] Specifically, the control mode can be various modes suitable for controlling VR videos through mobile terminals. For example, the control mode can be a gyroscope control mode, a joystick control mode, or a touch control mode. A control mode is determined in each control mode, and the control mode is the target control mode. For example, if a user chooses to control a VR video through a gyroscope control mode among gyroscope control mode, joystick control mode, or touch control mode, the gyroscope control mode is the target control mode. The target control mode may be determined in response to a user selection through a selection control, or in response to other user operations, and the embodiments of the present application do not limit this.

[0050] Gyroscope control mode can be a control mode that uses the gyroscope in the mobile terminal to control the field of view of the VR video. Joystick control mode can be a control mode that uses the joystick to control the field of view of the VR video. Touch control mode can be a control mode that uses the touch area to control the field of view of the VR video.

[0051] Step 120: Obtain control parameters corresponding to the target control mode.

[0052] Specifically, a user inputs control parameters in a target control mode through a mobile terminal, and the mobile terminal obtains the control parameters corresponding to the target control mode through its own control parameter acquisition device. The mobile terminal can be a mobile phone, tablet computer, smartwatch, or other intelligent mobile terminal device. The control parameter acquisition device can be a gyroscope or touch screen, etc.

[0053] For example, when a user uses a mobile phone to input control parameters in gyroscope control mode, the mobile phone can obtain the control parameters input by the user by collecting three-degree-of-freedom coordinate parameters through the gyroscope; or, when a user uses a mobile phone to input control parameters through the touch screen in joystick control mode, the mobile phone can obtain the control parameters input by the user by collecting electrical signal parameters through the touch screen.

[0054] Step 130: Interactively control the VR video on the display screen of the display terminal based on the control parameters.

[0055] Specifically, after obtaining the control parameters, the mobile terminal can send the control parameters to the display terminal. For example, the mobile terminal can establish a communication connection with the display terminal and transmit the control parameters using the communication connection. The display terminal can be a smart TV, a smart screen, or other electronic device that can display VR videos. Interactive control of the VR video on the display screen of the display terminal can be performed, for example, controlling the field of view of the VR video, controlling the game characters in the VR video or selecting options, switching the VR video to a 360-degree perspective, and so on. Among them, the field of view can be understood as the visible range of the VR video displayed in the display interface of the display terminal.

[0056] For example, using field of view control as an example, the control parameters can be used to instruct the display terminal to determine the first three-dimensional coordinates of the mobile terminal in three-dimensional space, map the first three-dimensional coordinates to the display screen of the display terminal, obtain first two-dimensional coordinates corresponding to the first three-dimensional coordinates, and then control the field of view of the VR video on the display screen based on the first two-dimensional coordinates. The first three-dimensional coordinates can be the coordinates of the mobile terminal in three-dimensional space in the real world. For example, the mobile terminal obtains data such as the azimuth and pitch angles from a gyroscope. Based on the obtained data, the display terminal can determine the first three-dimensional coordinates of the mobile terminal in three-dimensional space and map the first three-dimensional coordinates to the display screen of the display terminal to obtain first two-dimensional coordinates corresponding to the first three-dimensional coordinates. It can be understood that, based on the obtained control parameters, the display terminal establishes a correspondence between the position of the mobile terminal in three-dimensional space and the center position of the virtual light source in the VR video. The center position of the virtual light source can also be understood as the position of the virtual camera in the VR video. The display terminal can control the field of view of the VR video on the display screen based on the first two-dimensional coordinates. For example, the field of view of the VR video can be adjusted or the viewing angle can be switched.

[0057] For example, taking a mobile phone as the mobile terminal and a smart TV as the display terminal, the mobile phone, in response to a user's control mode selection operation, determines that the target control mode is the gyroscope control mode; the mobile phone obtains control parameters input by the user in the gyroscope control mode in real time and sends the obtained control parameters to the smart TV. After receiving the control parameters sent by the mobile phone, the smart TV determines the first three-dimensional coordinates of the mobile phone in three-dimensional space based on the control parameters, maps the first three-dimensional coordinates to the display screen of the smart TV, and obtains first two-dimensional coordinates corresponding to the first three-dimensional coordinates. Based on the first two-dimensional coordinates, the field of view of the VR video on the display screen can be controlled, for example, the field of view can be adjusted to meet the user's viewing needs of a 360-degree panoramic VR video.

[0058] For example, by operating the control joystick, you can complete 360-degree perspective switching, game character control walking and other functions in the VR video of the smart TV terminal, and realize the separate control of selection control commands and perspective switching commands.

[0059] For another example, after establishing the connection through command data, the TV side uses a three-dimensional ellipsoid fitting algorithm to start verifying the original instruction set of the mobile phone side, thereby determining the three-dimensional sphere center of the TV side VR video, and according to the ray principle of the camera, the azimuth and pitch angle data and joystick parameters of the mobile terminal gyroscope are mapped to the TV side screen, rendered as virtual rays, and respond to the user's corresponding operation events at the ray projection point to realize the interaction between the mobile terminal and the smart TV side VR video.

[0060] The present application provides a VR video control method in which a mobile terminal responds to a user's operation and determines a target control mode. The method then obtains control parameters corresponding to the target control mode to interactively control the VR video based on the control parameters. The target control modes include a gyroscope control mode, a joystick control mode, or a touch control mode. In any of these target control modes, the user can interactively control the VR video based on the obtained control parameters.

[0061] It should be noted that in order to accurately obtain the control parameters corresponding to the target control mode, the parameter type to be obtained under the target control mode can be obtained based on the target control strategy corresponding to the target control mode, and the control parameters can be accurately obtained based on the parameter type.

[0062] In one embodiment, obtaining the control parameters corresponding to the target control mode can be achieved in the following manner: based on the correspondence between the control mode and the control strategy, determining the target control strategy corresponding to the target control mode; the target control strategy is used to characterize the parameter type to be obtained under the target control mode; based on the parameter type, obtaining the control parameters corresponding to the target control mode.

[0063] Specifically, a control strategy can be a strategy set for parameters corresponding to a control mode. The control strategy can be used to characterize the parameter types to be obtained within the control mode. A control strategy corresponding to each control mode can be preset. Based on the correspondence between control modes and control strategies, the corresponding target control strategy can be determined once the target control mode is determined. A parameter type can be the type to which a specific parameter value belongs. For example, parameter types can include gyroscope swing parameters, joystick parameter types, and touch parameter types.

[0064] For example, a gyroscope control mode corresponds to a gyroscope control strategy, which indicates that the parameter types to be obtained by the mobile terminal in the gyroscope control mode include gyroscope swing parameters. For another example, a joystick control mode corresponds to a joystick control strategy, which indicates that the parameter types to be obtained by the mobile terminal in the joystick control mode include joystick parameter types. For another example, a touch control mode corresponds to a touch control strategy, which indicates that the parameter types to be obtained by the mobile terminal in the touch control mode include touch parameter types.

[0065] In this embodiment, based on the correspondence between the control mode and the control strategy, after determining the target control mode, the target control strategy corresponding to the target control mode can be determined. Based on the parameter type to be obtained represented by the target control strategy, the mobile terminal can accurately obtain the control parameters corresponding to the target control mode, and then based on the control parameters, the controllability of the VR video can be improved.

[0066] Existing display terminals, such as smart TVs, are equipped with remote controls without built-in gyroscopes. When users use smart TVs to watch 360-degree VR videos, they control the VR video's field of view by individually pressing the up, down, left, and right keys. This adjustment has low precision and flexibility, resulting in poor controllability and a fragmented user experience. In the embodiments of the present application, a mobile terminal can be used to obtain a first swing parameter of the gyroscope within the mobile terminal. Determining the first swing parameter as a control parameter can improve the controllability of the VR video.

[0067] In one embodiment, the target control mode is a gyroscope control mode, and the parameter type includes a swing parameter type of a gyroscope in the mobile terminal. Based on the parameter type, the control parameter corresponding to the target control mode is obtained, which can be specifically implemented as follows:

[0068] When a field of view adjustment instruction input by a user is received, a first swing parameter of the gyroscope is acquired based on the swing parameter type of the gyroscope; and the first swing parameter is determined as a control parameter.

[0069] Specifically, when the target control mode is a gyroscope control mode, the parameter type may include a swing parameter type of a gyroscope in the mobile terminal, and corresponding control parameters may be obtained based on the swing parameter type of the gyroscope. The field of view adjustment command input by the user may be a command for instructing to adjust the field of view of the VR video, for example, by long pressing a command button to input the field of view adjustment command.

[0070] The first swing parameter of the gyroscope can be a parameter used to adjust the field of view of the VR video, and the first swing parameter can be acquired by the gyroscope in the mobile terminal. The first swing parameter can be, for example, a three-degree-of-freedom coordinate parameter acquired by the gyroscope, or a parameter such as the azimuth angle, pitch angle, roll angle, or angular velocity acquired when the gyroscope swings. When the target control mode is the gyroscope control mode and a field of view adjustment instruction input by the user is received, the first swing parameter of the gyroscope in the mobile terminal obtained is determined as the control parameter based on the swing parameter type of the gyroscope.

[0071] Exemplarily, FIG2 is a functional diagram of the gyroscope control mode provided in an embodiment of the present application. Referring to FIG2 , the gyroscope control mode can be displayed in the interface corresponding to the mobile terminal. Taking the mobile terminal as a mobile phone and the display terminal as a smart TV as an example, when the user needs to control the field of view of the VR video, when the target control mode is the gyroscope control mode, the user can hold the mobile phone, long press the confirmation key 31 (i.e., the ok key 31) in the mobile phone interface and move the mobile phone, thereby inputting the control parameters in the target control mode, wherein the ok key 31 is the command key, and long pressing the ok key 31 is the field of view adjustment command. After the mobile phone obtains the first swing parameter generated by the gyroscope when the user moves the mobile phone, the first swing parameter can be determined as the control parameter and the control parameter can be sent to the smart TV of the display terminal. Based on the control parameter, the smart TV can adjust the viewing angle of the VR video to enable the user to browse the VR video in 360 degrees.

[0072] In this embodiment, the target control mode is a gyroscope control mode, and the parameter type includes the swing parameter type of the gyroscope in the mobile terminal. Upon receiving a field of view adjustment instruction input by the user, the first swing parameter of the gyroscope is obtained and determined as the control parameter. Based on this control parameter, the field of view of the VR video can be controlled. This enables more flexible interaction and more precise control of the VR video displayed on the display terminal by leveraging the mobile terminal's built-in gyroscope component and using the mobile terminal as a virtual reality controller, thereby improving the controllability of the VR video control.

[0073] It should be noted that compared to controlling the field of view of the VR video by operating the up, down, left and right keys separately, controlling the field of view of the VR video by controlling the joystick is more flexible and accurate, which can improve the controllability of the VR video.

[0074] In one embodiment, the target control mode is a joystick control mode, and the parameter type includes a joystick parameter type. Based on the parameter type, the control parameter corresponding to the target control mode is obtained, which can be specifically implemented as follows:

[0075] Based on the joystick parameter type, the joystick movement distance and joystick movement direction input by the user through the joystick are obtained; the joystick is displayed in the interface corresponding to the joystick control mode; and the joystick movement distance and joystick movement direction are determined as control parameters.

[0076] Specifically, when the target control mode is a joystick control mode, the parameter type may include a joystick parameter type. Based on the joystick parameter type, the mobile terminal obtains the joystick movement distance and joystick movement direction input by the user through the joystick control, and determines the joystick movement distance and joystick movement direction as control parameters. For example, a joystick control mode interface is displayed on the touch screen of the mobile terminal, where a control joystick is displayed. When the user operates the control joystick to move, the mobile terminal obtains specific data on the joystick movement distance and joystick movement direction generated by the joystick movement, and determines the specific data on the joystick movement distance and joystick movement direction as control parameters.

[0077] The joystick movement distance can be understood as the distance from the initial position to the final position of the joystick; the joystick movement direction can be understood as the direction from the initial position to the final position of the joystick. For example, a joystick is displayed on the touch screen of a mobile terminal. By collecting the changes in the electrical signals on the touch screen when the user operates the joystick, the joystick movement distance and joystick movement direction can be calculated. The obtained joystick movement distance and joystick movement direction can be determined as control parameters, based on which the field of view of the VR video can be controlled.

[0078] For example, FIG3 is a functional schematic diagram of the joystick control mode provided in an embodiment of the present application. Referring to FIG3 , the joystick control mode can be displayed in the corresponding interface of a mobile terminal. For example, in the case of a mobile phone as the mobile terminal and a smart TV as the display terminal, the touch screen of the mobile phone can display a control joystick 32 in the joystick control mode. When a user needs to control the field of view of a VR video, if the target control mode is the joystick control mode, the user can hold the mobile phone, press and hold the control joystick 32 in the mobile phone interface, and thus input control parameters in the target control mode. After the mobile phone obtains the joystick movement distance and direction generated by the user dragging the joystick, it can determine the joystick movement distance and direction as control parameters and send these control parameters to the smart TV as the display terminal. Based on these control parameters, the smart TV can adjust the field of view of the VR video in real time or switch the viewing angle to enable 360-degree viewing of the VR video. Alternatively, based on these control parameters, the smart TV can control the movement of game characters within the field of view of the VR video in real time, enabling real-time interaction during VR gaming.

[0079] In this embodiment, the target control mode is the joystick control mode, and the parameter type includes the joystick parameter type; based on the parameter type, the joystick movement distance and joystick movement direction of the user's operation to control the joystick input are obtained, and the joystick movement distance and the joystick movement direction are determined as control parameters. Based on the control parameters, the field of view of the VR video can be controlled, and the multi-touch screen of the mobile terminal is relied upon. The mobile terminal is used as a virtual reality handle, and more flexible interaction and more precise control of the VR video displayed in the display terminal are achieved, thereby improving the controllability of the VR video control.

[0080] It should be noted that compared to controlling the field of view of VR video by operating the up, down, left and right keys separately, controlling the field of view of VR video through the touch area is more flexible and accurate, which can improve the controllability of VR video.

[0081] In one embodiment, the target control mode is a touch control mode, and the parameter type includes a touch parameter type. Based on the parameter type, the control parameter corresponding to the target control mode is obtained, which can be specifically implemented as follows:

[0082] Based on the touch parameter type, the touch distance and touch direction input by the user through the touch area are obtained; the touch area is displayed in an interface corresponding to the touch control mode; and the touch distance and touch direction are determined as control parameters.

[0083] Specifically, when the target control mode is a touch control mode, the parameter type may include a touch parameter type. Based on the touch parameter type, the mobile terminal obtains the touch distance and touch direction input by the user through the touch area, and determines the touch distance and touch direction as control parameters. For example, in the interface of the touch control mode displayed on the touch screen of the mobile terminal, a touch area is displayed. The user slides within the touch area using a finger or a touch tool to generate a touch distance and touch direction. By determining the touch distance and the touch direction as control parameters, the field of view of the VR video can be controlled based on the control parameters. Among them, the touch tool can be an operating tool that causes the touch screen to generate an electrical signal, such as a capacitive pen or a touch pen.

[0084] The touch distance can be understood as the distance from the starting position to the ending position of the touch operation, and the touch direction can be understood as the direction from the starting position to the ending position of the touch operation. For example, a touch area is displayed on the touch screen of a mobile terminal. By collecting the changes in the electrical signals of the touch screen during the user's touch operation, the touch distance and touch direction can be calculated. The obtained touch distance and touch direction are determined as control parameters, and the field of view of the VR video can be controlled based on the control parameters.

[0085] For example, FIG4 is a functional diagram of the touch control mode provided in an embodiment of the present application. Referring to FIG4 , the touch control mode can be displayed in the interface corresponding to the mobile terminal. Taking the mobile terminal as a mobile phone and the display terminal as a smart TV as an example, the touch screen of the mobile phone can display the touch area 33 in the touch control mode. When the user needs to control the field of view of the VR video, when the target control mode is the touch control mode, the user can hold the mobile phone, press and hold the touch area with a finger and slide, thereby inputting the control parameters in the target control mode. After the mobile phone obtains the touch distance and touch direction generated when the user's finger slides, it can determine the touch distance and touch direction as control parameters and send the control parameters to the smart TV of the display terminal. Based on the control parameters, the smart TV can adjust the field of view of the VR video in real time, or switch the viewing angle to achieve 360-degree viewing of the VR video.

[0086] In this embodiment, the target control mode is the touch control mode, and the parameter type includes the joystick parameter type. Based on the parameter type, the touch distance and touch direction input by the user are obtained, and the touch distance and the touch direction are determined as control parameters. Based on the control parameters, the field of view of the VR video can be controlled, and a multi-touch screen based on the mobile terminal is realized. The mobile terminal is used as a virtual reality handle, and the VR video displayed in the display terminal is more flexibly interacted with and more accurately controlled, thereby improving the controllability of the VR video control.

[0087] Optionally, as shown in Figures 2-4, the interfaces of the gyroscope control mode, the joystick control mode and the touch control mode may include a volume up key 1, a volume down key 2, a return key 4, a home key 5, a lock key 6, a connection control key 7 and a mode switch key 8.

[0088] Among them, the volume up key 1 can be used to increase the volume of the display terminal; the volume down key 2 can be used to decrease the volume of the display terminal; the back key 4 can be used to control the display terminal VR application system to return to the previous level in the current interface; the home key 5 can be used to control the VR application system to return to the homepage. For example, regardless of the current interface, the display terminal VR application system will directly terminate the current operation and return to the application homepage; the lock key 6 can be used to lock the current perspective or task and no longer transmit control parameters to the display terminal; the connection control key 7 can be used to control the mobile terminal to open the connection menu. For example, after clicking the connection control key 7, the mobile terminal will display the connection control menu, allowing operations such as disconnecting, reconnecting, and connecting to other display terminals. After disconnecting from the current display terminal VR application system, the display terminal VR application system will no longer be able to control operations. The mode switch key 8 can be used to switch the target control mode. For example, after clicking the mode switch key 8, different control modes can be selected, enabling rapid switching between the target control modes. Optionally, the touch control mode interface can only retain the back key 4, the connection control key 7, and the mode switch key 8.

[0089] It should be noted that in order to further improve the user's control visibility when experiencing VR videos, the camera's ray principle can be used to determine a virtual ray based on the second swing parameter of the gyroscope in the mobile terminal, and the virtual ray is pointed at the first object to be selected. This can improve the control visibility when controlling VR videos and enhance the experience of human-computer interaction.

[0090] In one embodiment, obtaining control parameters corresponding to the target control mode includes: obtaining a second swing parameter of a gyroscope in a mobile terminal, and determining the second swing parameter as a control parameter; interactively controlling a VR video on a display screen of a display terminal based on the control parameter, including: sending the second swing parameter to the display terminal; the second swing parameter is used to instruct the display terminal to display a virtual ray determined based on the second swing parameter, the virtual ray pointing to a first target object to be selected in the VR video; obtaining a first selection instruction input by a user for the first target object to be selected; sending the first selection instruction to the display terminal; the first selection instruction is used to instruct the display terminal to select the first target object to be selected.

[0091] Specifically, the second swing parameter can be a swing parameter obtained based on the gyroscope in the mobile terminal and used to instruct the display terminal to display virtual rays. The second swing parameter can be, for example, a three-degree-of-freedom coordinate parameter collected by the gyroscope, or parameters such as azimuth, pitch angle, roll angle or angular velocity obtained when the gyroscope swings. The second swing parameter can be determined as a control parameter.

[0092] The virtual ray may be a ray displayed on the display terminal pointing to the first object to be selected in the VR video. The virtual ray may be understood as a fictitious auxiliary ray used to assist the user in locating the first object to be selected. The first object to be selected may be any selectable object in the VR video. After the first object to be selected is designated by the virtual ray, the user may more clearly identify the first object to be selected. The first object to be selected may be, for example, an icon or an option in a menu; the first target object to be selected may be the first object to be selected selected by the user, for example, an icon selected by the user.

[0093] Exemplarily, the first selection instruction can be any instruction for instructing the display terminal to determine the first to-be-selected object currently pointed to by the virtual ray as the first target to-be-selected object. For example, when the target control mode is the gyroscope control mode, the first selection instruction can be a single click of the OK key 31; when the target control mode is the joystick control mode, the first selection instruction can be a single click of the center of the joystick 32; when the target control mode is the touch control mode, the first selection instruction can be a single click of any position in the touch area 33. In the target control mode, the first selection instruction can also be an instruction of other types, which is not limited in this embodiment of the present invention.

[0094] For example, taking a mobile phone as the mobile terminal and a smart TV as the display terminal, when the target control mode is the gyroscope control mode, an OK button 31 is displayed on the touch screen of the mobile phone. The user holds the mobile phone and swings it, and the gyroscope of the mobile phone can obtain a second swing parameter. The mobile phone sends the second swing parameter to the smart TV in real time. After receiving the second swing parameter, the smart TV can determine and display a virtual ray. In the VR video displayed on the smart TV, the virtual ray points to different icons as the mobile phone swings. When the user clicks the OK button 31, the icon pointed to by the virtual ray is determined as the first target to be selected, and this icon is the icon selected by the user.

[0095] For example, taking a mobile phone as the mobile terminal and a smart TV as the display terminal, if the target control mode is the joystick control mode, a control joystick 32 is displayed on the touch screen of the mobile phone. The user holds the mobile phone and swings it, and the mobile phone's gyroscope can obtain a second swing parameter. The mobile phone sends the second swing parameter to the smart TV in real time. After receiving the second swing parameter, the smart TV can determine and display a virtual ray. In the VR video displayed on the smart TV, the virtual ray points to different icons as the phone swings. When the user clicks the center position of the control joystick 32, the icon pointed by the virtual ray is determined as the first target to be selected, and this icon is the icon selected by the user.

[0096] For example, taking a mobile phone as the mobile terminal and a smart TV as the display terminal, when the target control mode is the touch control mode, a touch area 33 is displayed on the touch screen of the mobile phone. The user holds the mobile phone and swings it, and the gyroscope of the mobile phone can obtain a second swing parameter. The mobile phone sends the second swing parameter to the smart TV in real time. After receiving the second swing parameter, the smart TV can determine and display a virtual ray. In the VR video displayed on the smart TV, the virtual ray points to different icons as the phone swings. When the user clicks anywhere in the touch area 33, the icon pointed to by the virtual ray is determined as the first target to be selected, and this icon is the icon selected by the user.

[0097] Based on this, the virtual ray determined by the second swing parameter can effectively point to the viewing direction within the field of view of the VR video, thereby improving the flexibility and control visibility of the VR video control; at the same time, based on the first selection instruction, menu selection, page switching and other functions can be efficiently executed, further improving the controllability of the VR video.

[0098] It should be noted that during VR video control, the selection control command and the view switching command are prone to conflict, and the problem of how to operate the selection control command and the view switching command separately is particularly prominent. In order to achieve the operation of the selection control command and the view switching command separately, a target direction control can be set.

[0099] In one embodiment, obtaining control parameters corresponding to the target control mode includes: when the target control mode includes a joystick control mode, obtaining a user's click instruction for the target direction control, and determining the click instruction as a control parameter; the target direction control is displayed in an interface corresponding to the joystick control mode; interactively controlling the VR video on the display screen of the display terminal based on the control parameters includes: sending the click instruction to the display terminal; the click instruction is used to instruct the display terminal to determine a second object to be selected in the VR video based on the click instruction; obtaining a second selection instruction input by the user for the second target object to be selected; sending the second selection instruction to the display terminal; the second selection instruction is used to instruct the display terminal to select the second target object to be selected.

[0100] Specifically, the target direction control can be a control for receiving click commands. For example, in the interface corresponding to the joystick control mode, target direction controls are provided around the joystick. By clicking the target direction in the target direction control, the user's click command can be received. The received click command can be determined as a control parameter. As shown in Figure 3, the target direction controls provided around the joystick include triangle controls for up, down, left, and right directions. Clicking the triangle control in any direction inputs a click command in that direction.

[0101] The second object to be selected can be any selectable object in the VR video. After receiving the click instruction, the second object to be selected currently indicated in the VR video can be switched based on the click instruction. The second object to be selected can be, for example, an icon or an option in a menu; the second target object to be selected can be the second object to be selected selected by the user. The second selection instruction is used to instruct the display terminal to select the second target object to be selected.

[0102] Exemplarily, the second selection instruction may be any instruction for instructing the display terminal to determine the current second to-be-selected object as the second target to-be-selected object. For example, when the target control mode includes a joystick control mode, the second selection instruction may be an instruction generated by clicking any position of the joystick 32.

[0103] In this embodiment, a target direction control is used to obtain a click instruction input by the user, and the click instruction is determined as a control parameter. The click instruction is sent to the display terminal. Based on the click instruction, the display terminal can determine the second object to be selected in the VR video, that is, implement a switching operation for each second object to be selected. Based on the user's operation of the joystick, the joystick movement distance and joystick movement direction are determined as control parameters, and the field of view of the VR video can be controlled. Therefore, this embodiment can realize the separate control of the selection control instruction and the perspective switching instruction, solving the problem of the selection control instruction and the perspective switching instruction easily conflicting during the control process of the VR video. This makes this method applicable to VR videos that require both controlling the perspective and switching the objects to be selected, thereby improving the applicability of this method.

[0104] For example, in order to facilitate controlling the field of view of the VR video in different target control modes, the control modes can be switched.

[0105] In one embodiment, the gyroscope control mode, the joystick control mode and the touch control mode are switchable.

[0106] Specifically, when the current target control mode is any one of the gyroscope control mode, the joystick control mode, and the touch control mode, the current target control mode can be switched to another of the gyroscope control mode, the joystick control mode, and the touch control mode by switching the control mode. For example, as shown in Figures 2-4, switching the control mode can be achieved by using the mode switch key 8; for example, switching the control mode can also be achieved by voice commands or gesture commands.

[0107] For example, when the current target control mode is the gyroscope control mode, the user can switch the target control mode to the joystick control mode or the touch control mode by clicking the mode switch button in the mobile phone interface. When switching, the control mode can be switched according to the preset switching sequence of the gyroscope control mode, the joystick control mode, and the touch control mode, or it can be directly switched to the control mode required by the user according to user needs. In this embodiment, the gyroscope control mode, the joystick control mode, and the touch control mode can be switched, and the control mode can be quickly changed to determine the target control mode, thereby improving the applicability of the method of the present invention and enhancing the user experience.

[0108] For example, existing display terminals such as smart TVs rely on remote control commands and cannot be controlled through touch screens. The logic of remote control control cannot implement effective control selection commands such as text input and three-dimensional (3D) cursor display. By controlling the field of view of VR videos based on mobile terminals, a text input box or 3D cursor can be called up using the mobile terminal's touch screen, enabling text input or 3D cursor display within VR videos.

[0109] FIG5 is a second flow chart of a VR video control method provided in an embodiment of the present application. Referring to FIG5 , an embodiment of the present application provides a VR video control method, which is applied to a display terminal and may include steps 510 and 520.

[0110] Step 510: Receive control parameters corresponding to a target control mode sent by a mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode.

[0111] Specifically, referring to the description of steps 110 to 130, the control parameters input by the user in the target control mode can be obtained through the mobile terminal, and the control parameters can be sent to the display terminal through the mobile terminal. After the display terminal receives the control parameters, the field of view of the VR video can be controlled based on the control parameters.

[0112] Step 520: Based on the control parameters, interactively control the VR video on the display screen of the display terminal.

[0113] In one embodiment, based on control parameters, interactive control of a VR video on a display screen of a display terminal is performed, including: determining a first three-dimensional coordinate of a mobile terminal in a three-dimensional space based on the control parameters; mapping the first three-dimensional coordinate to the display screen of the display terminal to obtain a first two-dimensional coordinate corresponding to the first three-dimensional coordinate; and controlling the field of view of the VR video on the display screen based on the first two-dimensional coordinate.

[0114] Specifically, the first three-dimensional coordinates may be the coordinates of the mobile terminal in the three-dimensional space of the real world. For example, the mobile terminal obtains data such as the azimuth and pitch angles of the gyroscope, or obtains control parameters such as the joystick movement distance and joystick movement direction input through the control stick, or obtains control parameters such as the touch distance and touch direction input through the touch area, and can determine the first three-dimensional coordinates of the virtual camera in the three-dimensional space, where the virtual camera can be understood as the light source center of the VR video. The position of the mobile terminal in the three-dimensional space is mapped to the position of the virtual camera in the VR video.

[0115] Exemplarily, the first two-dimensional coordinate may be a coordinate corresponding to the first three-dimensional coordinate after the first three-dimensional coordinate is mapped to the display screen of the display terminal. For example, based on the first three-dimensional coordinate, a graphics processing unit (GPU) rendering pipeline is used to map the first three-dimensional coordinate of the virtual camera in three-dimensional space to the first two-dimensional coordinate on the display screen of the display terminal.

[0116] For example, the first three-dimensional coordinate is mapped onto the display screen of the display terminal. The first two-dimensional coordinate corresponding to the first three-dimensional coordinate can be obtained by using a three-dimensional ellipsoid fitting algorithm to verify the data. The first two-dimensional coordinate corresponding to the first three-dimensional coordinate can be determined. The determined first two-dimensional coordinate is the position of the sphere center. When using the three-dimensional ellipsoid fitting algorithm, the standard ellipsoid equation shown below must be used:

[0117] Wherein, x, y, and z represent the coordinate values ​​of the first three-dimensional coordinate respectively; x , O y , O z Represent the coordinates of the center of the sphere, R x 、R y 、R z Represent the lengths of the three semi-axes of the sphere respectively. After expanding the standard ellipsoid equation, we can get the following polynomial:

[0118] After sorting, we can get: y 2 A+z 2 B+xC+yD+zE+F=-x 2

[0119] Right now:

[0120] Write the above formula as KX=Y. According to the least square solution, for the ellipsoid fitting data of the gyroscope data set, the form to be estimated is: X=(K T K) -1 K T Y

[0121] Where:

[0122] The final equation for ellipsoid calibration is:

[0123] After verification according to this method, the fitting rate of the results obtained by this method with the origin in actual virtual reality applications exceeds 99.8%. The ellipsoid fitting result is accurate and the position of the center point of the sphere can be determined.

[0124] Specifically, the display terminal receives the control parameters sent by the mobile terminal in real time, and updates the first three-dimensional coordinates of the mobile terminal in the three-dimensional space in real time based on the control parameters, maps the first three-dimensional coordinates to the display screen of the display terminal, and obtains the updated first two-dimensional coordinates. The first two-dimensional coordinates can be used to control the field of view of the VR video on the display screen.

[0125] Figure 6 is a schematic diagram of the virtual camera construction provided by an embodiment of the present application. As shown in Figure 6, the first three-dimensional coordinates of the mobile terminal in the three-dimensional space are determined, that is, the center position of the sphere where the virtual camera is located is determined, and the first three-dimensional coordinates are mapped to the display screen of the display terminal to obtain the first two-dimensional coordinates corresponding to the first three-dimensional coordinates. The field of view of the VR video on the display screen can be controlled based on the first two-dimensional coordinates.

[0126] In this embodiment, the display terminal can determine the first three-dimensional coordinates of the mobile terminal based on the received control parameters, and map the first three-dimensional coordinates to the display screen of the display terminal. The first two-dimensional coordinates corresponding to the first three-dimensional coordinates can be obtained, and the field of view of the VR video on the display screen can be controlled based on the first two-dimensional coordinates. In this way, the field of view of the VR video displayed on the display terminal can be controlled based on the mobile terminal, improving the controllability of the VR video.

[0127] In one embodiment, the control parameters include a second swing parameter of the gyroscope obtained by the mobile terminal in a target control mode; based on the control parameters, the VR video on the display screen of the display terminal is interactively controlled, including: displaying a virtual ray based on the second swing parameter; pointing the virtual ray to an object in the VR video as a first object to be selected; receiving a first selection instruction sent by the mobile terminal; and selecting a first target object to be selected corresponding to the first selection instruction in response to the first selection instruction.

[0128] Specifically, the display terminal can receive the second swing parameter sent by the mobile terminal. The second swing parameter can be, for example, a three-degree-of-freedom coordinate parameter collected by a gyroscope, or parameters such as azimuth, pitch, roll or angular velocity obtained when the gyroscope swings.

[0129] When the target control mode includes a gyroscope control mode, a touch control mode, or a joystick control mode, the display terminal may display a virtual ray based on the second swing parameter. The virtual ray may be a ray displayed on the display terminal pointing to the first object to be selected in the VR video. The virtual ray may be understood as a fictitious auxiliary ray used to assist the user in locating the first object to be selected.

[0130] The first object to be selected can be any selectable object in the VR video. After the first object to be selected is designated by the virtual ray, the user can more clearly identify the first object to be selected. The first object to be selected can be, for example, an icon or an option in a menu; the first target object to be selected can be the first object to be selected selected by the user, for example, an icon selected by the user.

[0131] Exemplarily, the first selection instruction may be any instruction for instructing the display terminal to determine the first to-be-selected object currently pointed to by the virtual ray as the first target to-be-selected object. For example, if the target control mode includes a gyroscope control mode, the first selection instruction may be single-clicking the OK key 31; if the target control mode includes a joystick control mode, the first selection instruction may be single-clicking the center of the joystick 32; if the target control mode includes a touch control mode, the first selection instruction may be a command generated by single-clicking any position of the touch area 33.

[0132] Based on this, the virtual ray determined by the second swing parameter improves the control visibility during VR video control, and based on the second swing parameter, functions such as menu selection and page switching can be realized, thereby improving the controllability of VR video control.

[0133] In one embodiment, displaying a virtual ray based on a second swing parameter includes: determining a second three-dimensional coordinate of the mobile terminal in three-dimensional space based on the second swing parameter; mapping the second three-dimensional coordinate to a display screen of the display terminal to obtain a second two-dimensional coordinate corresponding to the second three-dimensional coordinate; and displaying the virtual ray at the position of the second two-dimensional coordinate.

[0134] Specifically, the second three-dimensional coordinates may be coordinates of the mobile terminal in a three-dimensional space in the real world, determined based on the second swing parameter. For example, the mobile terminal may obtain data such as the azimuth and pitch angles of a gyroscope, or obtain control parameters such as the joystick movement distance and direction input via a joystick, or obtain control parameters such as the touch distance and direction input via a touch area, to determine the second three-dimensional coordinates of the mobile terminal in the three-dimensional space. Based on the second three-dimensional coordinates being mapped onto the display screen of the display terminal, a second two-dimensional coordinate corresponding to the second three-dimensional coordinate may be obtained, and a virtual ray may be displayed at the position of the second two-dimensional coordinates.

[0135] A virtual ray can be understood as a ray emitted from a virtual camera, pointing toward the first target to be selected in the VR video. Displaying this virtual ray on the display terminal's screen can improve the visibility of VR video manipulation, helping users clearly identify the first target to be selected, and avoiding the phenomenon of incorrectly determining the first target to be selected due to unclear first target selection. It can more conveniently implement menu selection, page switching, and other functions, and can enhance the user-computer interaction experience.

[0136] In one embodiment, the control parameters include a user's click instruction for a target direction control obtained by the mobile terminal in a target control mode; the target control mode includes a joystick control mode; the target direction control is displayed in an interface corresponding to the joystick control mode; based on the control parameters, the VR video on the display screen of the display terminal is interactively controlled, including: responding to the click instruction, determining a second object to be selected corresponding to the click instruction; receiving a second selection instruction sent by the mobile terminal; and responding to the second selection instruction, selecting a second target object to be selected corresponding to the second selection instruction.

[0137] Specifically, the click instruction input by the user can be obtained through the target direction control. After the display terminal receives the click instruction sent by the mobile terminal, it can determine the second to-be-selected object corresponding to the click instruction in response to the click instruction.

[0138] For example, when a user clicks on a right-pointing triangle control in a target direction control, the mobile terminal sends the click instruction to the display terminal in real time. The display terminal receives and responds to the click instruction, and determines a second object to be selected to the right of the second object to be selected determined before receiving the click instruction as the current second object to be selected.

[0139] The second selection instruction may be any instruction for instructing the display terminal to determine the current second object to be selected as the second target object to be selected. For example, after receiving the second selection instruction, the display terminal selects the current second object to be selected as the second target object to be selected in response to the second selection instruction.

[0140] In this embodiment, when the target control mode includes a joystick control mode, the display terminal can flexibly select the second target to-be-selected object corresponding to the second selection instruction in response to a click instruction, thereby further improving the controllability of controlling the VR video.

[0141] In the smart TV market, limited by the limitations of traditional remote controls, VR applications still primarily rely on directional keys for interaction, resulting in suboptimal experiences for VR box panoramic content interaction and text input. With the widespread adoption of technologies like gyroscopes and accelerometers in mobile devices, the advantages of mobile device hardware can be leveraged to create new VR video interaction models, significantly improving the VR panoramic content interaction experience in the smart TV market.

[0142] By acquiring data from the mobile device's gyroscope, accelerometer, and other devices, the system dynamically calculates the mobile device's orientation or position in real space. This real-time data is then transmitted to the smart TV via a proprietary wireless fidelity (WiFi) or Bluetooth protocol. A 3D pointer is then mapped within the smart TV's VR three-dimensional space, enabling a seamless interaction between the mobile device and the smart TV's 3D pointer. Furthermore, content can be input into the VR video on the smart TV screen using the mobile device's soft keyboard, providing a flexible and precise VR interactive experience.

[0143] The VR video control method provided in the embodiment of the present application is based on a mobile terminal and provides three control modes including a gyroscope control mode, a joystick control mode and a touch control mode. It can also realize menu selection, perspective switching and other functions of VR videos such as VR panoramic roaming and VR games in a display terminal or set-top box through virtual rays.

[0144] The following uses a smart TV as an example, and describes the implementation steps of the VR video control method when applied to a mobile terminal and a display terminal in conjunction with Figure 7. Figure 7 is a flowchart of the VR video control method provided in an embodiment of the present application. As shown in Figure 7, the method may include five steps S1 to S5, where S1 is the step of establishing a connection; S2 is the step of sending a command; S3 is the step of receiving a command; S4 is the step of parsing the command; and S5 is the step of calibrating the command.

[0145] Specifically, S1: Connect the mobile terminal and smart TV to the same network and establish a physical channel via WiFi. The mobile terminal sends a connection request to the device address and port of the smart TV by manually selecting or scanning a code, establishing a port number (i.e., socket) connection, and the smart TV performs the socket binding verification with the mobile terminal.

[0146] S2: After the mobile terminal and the smart TV are connected, the mobile terminal transmits its gyroscope parameters, joystick parameters, touch parameters, and key control instructions to the smart TV in real time. The functional diagrams of the three modes are shown in Figures 2 to 4.

[0147] S3: The smart TV receives the corresponding original parameters and instructions transmitted by the mobile terminal through the Sokect connection in real time and in sequence.

[0148] S4: The smart TV parses the received original parameters and instructions, establishes a data transmission channel through the parameters, completes the connection between the mobile terminal data and the smart TV instruction protocol, and prepares to start the data verification process.

[0149] S5: After establishing the connection through the command data, the smart TV uses the three-dimensional ellipsoid fitting algorithm to start verifying the original instruction set of the mobile terminal, thereby determining the center of the three-dimensional sphere of the VR video displayed by the smart TV. Based on the ray principle of the camera, the azimuth and pitch angle data and the joystick parameters of the mobile terminal's gyroscope are mapped to the display screen of the smart TV, rendered as virtual rays, and responding to the user's corresponding operation events at the ray projection point, thereby realizing VR video control and interaction between the mobile terminal and the smart TV.

[0150] In this embodiment, a new interactive control method is proposed to address the problem of being unable to control VR videos using traditional remote controls when playing VR videos on display terminals such as smart TVs or set-top boxes. Based on a mobile terminal with a multi-touch screen and a built-in gyroscope, the mobile terminal controls the display device to display the VR video, improving the controllability of VR video.

[0151] FIG8 is a signaling flow chart of a VR video control method provided in an embodiment of the present application. As shown in FIG8 , the method includes:

[0152] Step 810: The mobile terminal determines a target control mode in response to a user operation. The target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode.

[0153] Step 820: The mobile terminal obtains control parameters corresponding to the target control mode.

[0154] Step 830: The mobile terminal interactively controls the VR video on the display screen of the display terminal based on the control parameters.

[0155] The VR video control method provided in this embodiment has a specific implementation process and technical effects similar to those of the above embodiments. For details, please refer to the detailed introduction in the above embodiments, which will not be repeated here.

[0156] The following describes a VR video control device on a mobile terminal side provided in an embodiment of the present application. The VR video control device described below and the VR video control method described above can refer to each other.

[0157] FIG9 is a structural diagram of a VR video control device according to an embodiment of the present application. As shown in FIG9 , the VR video control device 900 includes:

[0158] A first determining module 910 is configured to determine a target control mode in response to a user operation, where the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode;

[0159] An acquisition module 920 is configured to acquire control parameters corresponding to a target control mode;

[0160] The control module 930 is configured to interactively control the VR video on the display screen of the display terminal based on the control parameters.

[0161] In an exemplary embodiment, the acquisition module 920 is specifically configured to:

[0162] Based on the correspondence between the control mode and the control strategy, the target control strategy corresponding to the target control mode is determined; the target control strategy is used to characterize the parameter type to be obtained under the target control mode; based on the parameter type, the control parameters corresponding to the target control mode are obtained.

[0163] In an exemplary embodiment, the target control mode is a gyroscope control mode, and the parameter type includes a swing parameter type of a gyroscope in the mobile terminal; the acquisition module 920 is specifically configured as follows:

[0164] When a field of view adjustment instruction input by a user is received, a first swing parameter of the gyroscope is acquired based on the swing parameter type of the gyroscope; and the first swing parameter is determined as a control parameter.

[0165] In an exemplary embodiment, the target control mode is a joystick control mode, and the parameter type includes a joystick parameter type; the acquisition module 920 is specifically configured as follows:

[0166] Based on the joystick parameter type, the joystick movement distance and joystick movement direction input by the user through the joystick are obtained; the joystick is displayed in the interface corresponding to the joystick control mode; and the joystick movement distance and joystick movement direction are determined as control parameters.

[0167] In an exemplary embodiment, the target control mode is a touch control mode, and the parameter type includes a touch parameter type; the acquisition module 920 is specifically configured as follows:

[0168] Based on the touch parameter type, the touch distance and touch direction input by the user through the touch area are obtained; the touch area is displayed in an interface corresponding to the touch control mode; and the touch distance and touch direction are determined as control parameters.

[0169] In an exemplary embodiment, the acquisition module 920 is specifically configured to acquire a second swing parameter of a gyroscope in the mobile terminal and determine the second swing parameter as a control parameter;

[0170] The control module 930 is specifically configured to send the second swing parameter to the display terminal; the second swing parameter is used to instruct the display terminal to display a virtual ray determined based on the second swing parameter, where the virtual ray points to the first object to be selected in the VR video;

[0171] The acquisition module 920 is further configured to acquire a first selection instruction input by the user for the first target object to be selected;

[0172] The control module 930 is further configured to send a first selection instruction to the display terminal; the first selection instruction is used to instruct the display terminal to select a first target object to be selected.

[0173] In an exemplary embodiment, the acquisition module 920 is specifically configured to, when the target control mode includes a joystick control mode, acquire a user's click instruction for a target direction control and determine the click instruction as a control parameter; and display the target direction control in an interface corresponding to the joystick control mode;

[0174] The control module 930 is specifically configured to send a click instruction to the display terminal; the click instruction is used to instruct the display terminal to determine a second object to be selected in the VR video based on the click instruction;

[0175] The acquisition module 920 is further configured to acquire a second selection instruction input by the user for a second target object to be selected;

[0176] The control module 930 is further configured to send a second selection instruction to the display terminal; the second selection instruction is used to instruct the display terminal to select a second target object to be selected.

[0177] In an exemplary embodiment, a gyroscope control mode, a joystick control mode, and a touch control mode are switchable.

[0178] The device of this embodiment can be configured to execute the method of any embodiment of the VR video control method embodiment on the mobile terminal side. Its specific implementation process and technical effects are similar to those in the embodiment on the VR video control method side. For details, please refer to the detailed description in the embodiment on the VR video control method side, which will not be repeated here.

[0179] The following describes a VR video control device on the display terminal side provided in an embodiment of the present application. The VR video control device described below and the VR video control method described above can refer to each other.

[0180] FIG10 is a second structural diagram of a VR video control device provided in an embodiment of the present application. As shown in FIG10 , the VR video control device 1000 includes:

[0181] The receiving module 1010 is configured to receive control parameters corresponding to a target control mode sent by the mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode;

[0182] The control module 1020 is configured to interactively control the VR video on the display screen of the display terminal based on the control parameters.

[0183] In an exemplary embodiment, the control module 1020 is specifically configured to:

[0184] Based on the control parameters, the first three-dimensional coordinates of the mobile terminal in the three-dimensional space are determined; the first three-dimensional coordinates are mapped to the display screen of the display terminal to obtain the first two-dimensional coordinates corresponding to the first three-dimensional coordinates; and the field of view of the VR video on the display screen is controlled based on the first two-dimensional coordinates.

[0185] In an exemplary embodiment, the control parameter includes a second swing parameter of a gyroscope obtained by the mobile terminal in the target control mode; the control module 1020 is specifically configured as follows:

[0186] A virtual ray is displayed based on a second swing parameter; an object in the VR video pointed by the virtual ray is determined as a first object to be selected; a first selection instruction sent by the mobile terminal is received; and in response to the first selection instruction, a first target object to be selected corresponding to the first selection instruction is selected.

[0187] In an exemplary embodiment, the control module 1020 is specifically configured to:

[0188] Determine the second three-dimensional coordinates of the mobile terminal in the three-dimensional space based on the second swing parameter; map the second three-dimensional coordinates to the display screen of the display terminal to obtain the second two-dimensional coordinates corresponding to the second three-dimensional coordinates; and display the virtual ray at the position of the second two-dimensional coordinates.

[0189] In an exemplary embodiment, the control parameter includes a user click instruction for a target direction control obtained by the mobile terminal in a target control mode; the target control mode includes a joystick control mode; the target direction control is displayed in an interface corresponding to the joystick control mode;

[0190] The control module 1020 is specifically configured as follows:

[0191] In response to the click instruction, a second to-be-selected object corresponding to the click instruction is determined; a second selection instruction sent by the mobile terminal is received; and in response to the second selection instruction, a second target to-be-selected object corresponding to the second selection instruction is selected.

[0192] The device of this embodiment can be configured to execute the method of any embodiment of the display terminal side VR video control method embodiment. Its specific implementation process and technical effects are similar to those in the VR video control method side embodiment. For details, please refer to the detailed description in the VR video control method side embodiment, which will not be repeated here.

[0193] FIG11 is a schematic diagram of the physical structure of a mobile terminal provided in an embodiment of the present application. As shown in FIG11 , the mobile terminal may include: a processor 1110, a communications interface 1120, a memory 1130, a bus interface 1140, a gyroscope 1150, and a touch screen 1160. The processor 1110, communications interface 1120, memory 1130, gyroscope 1150, and touch screen 1160 communicate with each other via the bus interface 1140. The memory 1130 is configured to store computer programs; the gyroscope 1150 is configured to obtain swing parameters; the touch screen 1160 is configured to receive touch operations; the processor 1110 can call the logic instructions in the memory 1130 to execute the above-mentioned VR video control method, which includes: determining the target control mode in response to the user's operation, the target control mode including the gyroscope control mode, the joystick control mode or the touch control mode; obtaining the control parameters corresponding to the target control mode; and interactively controlling the VR video on the display screen of the display terminal based on the control parameters.

[0194] In addition, the logic instructions in the above-mentioned memory 1130 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0195] The embodiment of the present application also provides a mobile terminal, which may include: a processor, a communication interface, a memory, a bus interface, a transceiver, a gyroscope, and a touch screen. Among them, the processor, the communication interface, the memory, the transceiver, the gyroscope, and the touch screen communicate with each other through the bus interface. The memory is configured to store computer programs; the transceiver is configured to send and receive data under the control of the processor; the gyroscope is configured to obtain swing parameters; the touch screen is configured to receive touch operations; the processor can call the logic instructions in the memory to execute the above-mentioned VR video control method, which includes: determining the target control mode in response to the user's operation, the target control mode including the gyroscope control mode, the joystick control mode, or the touch control mode; obtaining the control parameters corresponding to the target control mode; and interactively controlling the VR video on the display screen of the display terminal based on the control parameters.

[0196] FIG12 is a schematic diagram of the physical structure of a display terminal provided in an embodiment of the present application. As shown in FIG12 , the display terminal may include: a processor 1210, a communications interface 1220, a memory 1230, a bus interface 1240, and a display 1250. The processor 1210, the communications interface 1220, the memory 1230, and the display 1250 communicate with each other via the bus interface 1240. The memory 1230 is configured to store computer programs; the display 1250 is configured to display VR video; the processor 1210 may call the logic instructions in the memory 1230 to execute the above-mentioned VR video control method, which includes: receiving control parameters corresponding to a target control mode sent by a mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode; and interactively controlling the VR video on the display screen of the display terminal based on the control parameters.

[0197] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0198] An embodiment of the present application also provides a display terminal, which may include: a processor, a communication interface, a memory, a bus interface, a transceiver, and a display. The processor, the communication interface, the memory, the transceiver, and the display communicate with each other through the bus interface. The memory is configured to store computer programs; the transceiver is used to send and receive data under the control of the processor; the display is configured to display VR video; the processor can call the logic instructions in the memory to execute the above-mentioned VR video control method, which includes: receiving control parameters corresponding to the target control mode sent by the mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode; based on the control parameters, the VR video on the display screen of the display terminal is interactively controlled.

[0199] On the other hand, an embodiment of the present application also provides a VR video control system, which includes a mobile terminal and a display terminal as provided in the above embodiments.

[0200] On the other hand, an embodiment of the present application also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the steps of the VR video control method provided in the above embodiments, for example, including: determining a target control mode in response to a user's operation, the target control mode including a gyroscope control mode, a joystick control mode or a touch control mode; obtaining control parameters corresponding to the target control mode; and interactively controlling the VR video on the display screen of the display terminal based on the control parameters.

[0201] Alternatively, the computer can execute the steps of the VR video control method provided in the above embodiments, for example, including: receiving control parameters corresponding to the target control mode sent by the mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode or a touch control mode; based on the control parameters, interactively controlling the VR video on the display screen of the display terminal.

[0202] On the other hand, an embodiment of the present application also provides a processor-readable storage medium, which stores a computer program. The computer program is configured to enable the processor to execute the steps of the VR video control method provided in the above embodiments, for example, including: determining the target control mode in response to the user's operation, the target control mode including a gyroscope control mode, a joystick control mode or a touch control mode; obtaining control parameters corresponding to the target control mode; and interactively controlling the VR video on the display screen of the display terminal based on the control parameters.

[0203] Alternatively, the computer can execute the steps of the VR video control method provided in the above embodiments, for example, including: receiving control parameters corresponding to the target control mode sent by the mobile terminal; the target control mode includes a gyroscope control mode, a joystick control mode or a touch control mode; based on the control parameters, interactively controlling the VR video on the display screen of the display terminal.

[0204] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NAND FLASH), solid-state drives (SSDs)), etc.

[0205] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0206] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A VR video control method, applied to a mobile terminal, the method comprising: In response to the user's operation, determining a target control mode, wherein the target control mode includes a gyroscope control mode, a joystick control mode, or a touch control mode; Obtaining control parameters corresponding to the target control mode; The VR video on the display screen of the display terminal is interactively controlled based on the control parameters.

2. The VR video control method according to claim 1, wherein: The obtaining of the control parameters corresponding to the target control mode includes: Based on the correspondence between the control mode and the control strategy, determining the target control strategy corresponding to the target control mode; the target control strategy is used to characterize the type of parameters to be obtained under the target control mode; Based on the parameter type, a control parameter corresponding to the target control mode is acquired.

3. The VR video control method according to claim 2, wherein: The target control mode is a gyroscope control mode, and the parameter type includes a swing parameter type of a gyroscope in the mobile terminal; The acquiring, based on the parameter type, a control parameter corresponding to the target control mode includes: When receiving a field of view adjustment instruction input by a user, acquiring a first swing parameter of the gyroscope based on the swing parameter type of the gyroscope; The first swing parameter is determined as the control parameter.

4. The VR video control method according to claim 2, wherein: The target control mode is a joystick control mode, and the parameter type includes a joystick parameter type; The acquiring, based on the parameter type, a control parameter corresponding to the target control mode includes: Based on the joystick parameter type, obtaining the joystick movement distance and joystick movement direction input by the user through the joystick control; the joystick control is displayed in the interface corresponding to the joystick control mode; The joystick moving distance and the joystick moving direction are determined as the control parameters.

5. The VR video control method according to claim 2, wherein: The target control mode is a touch control mode, and the parameter type includes a touch parameter type; The acquiring, based on the parameter type, a control parameter corresponding to the target control mode includes: Based on the touch parameter type, acquiring the touch distance and touch direction input by the user through the touch area; the touch area is displayed in the interface corresponding to the touch control mode; The touch distance and the touch direction are determined as the control parameters.

6. The VR video control method according to any one of claims 1 to 5, wherein: The obtaining of the control parameters corresponding to the target control mode includes: Acquire a second swing parameter of the gyroscope in the mobile terminal, and determine the second swing parameter as the control parameter; The interactively controlling the VR video on the display screen of the display terminal based on the control parameter includes: sending the second swing parameter to the display terminal; the second swing parameter is used to instruct the display terminal to display a virtual ray determined based on the second swing parameter, the virtual ray pointing to the first object to be selected in the VR video; Acquire a first selection instruction input by a user for a first target object to be selected; The first selection instruction is sent to the display terminal; the first selection instruction is used to instruct the display terminal to select the first target object to be selected.

7. The VR video control method according to any one of claims 1 to 5, wherein: The obtaining of the control parameters corresponding to the target control mode includes: In the case where the target control mode includes a joystick control mode, obtaining a click instruction of a user for a target direction control, and determining the click instruction as the control parameter; the target direction control is displayed in an interface corresponding to the joystick control mode; The interactively controlling the VR video on the display screen of the display terminal based on the control parameter includes: Sending the click instruction to the display terminal; the click instruction is used to instruct the display terminal to determine the second object to be selected in the VR video based on the click instruction; Acquire a second selection instruction input by a user for a second target object to be selected; The second selection instruction is sent to the display terminal; the second selection instruction is used to instruct the display terminal to select the second target object to be selected.

8. A VR video control method, applied to a display terminal, the method comprising: Receiving control parameters corresponding to the target control mode sent by the mobile terminal; The target control mode includes a gyroscope control mode, a joystick control mode or a touch control mode; Based on the control parameters, the VR video on the display screen of the display terminal is interactively controlled.

9. The VR video control method according to claim 8, wherein: The interactively controlling the VR video on the display screen of the display terminal based on the control parameter includes: Determining a first three-dimensional coordinate of the mobile terminal in a three-dimensional space based on the control parameter; Mapping the first three-dimensional coordinates onto a display screen of the display terminal to obtain first two-dimensional coordinates corresponding to the first three-dimensional coordinates; The field of view of the VR video on the display screen is controlled based on the first two-dimensional coordinates.

10. The VR video control method according to claim 8, wherein: The control parameter includes a second swing parameter of the gyroscope acquired by the mobile terminal in the target control mode; The interactively controlling the VR video on the display screen of the display terminal based on the control parameter includes: displaying a virtual ray based on the second swing parameter; Determine the object in the VR video pointed by the virtual ray as the first object to be selected; receiving a first selection instruction sent by the mobile terminal; In response to the first selection instruction, a first target to-be-selected object corresponding to the first selection instruction is selected.

11. The VR video control method according to claim 10, wherein: The displaying of the virtual ray based on the second swing parameter comprises: determining a second three-dimensional coordinate of the mobile terminal in the three-dimensional space based on the second swing parameter; Mapping the second three-dimensional coordinates onto the display screen of the display terminal to obtain second two-dimensional coordinates corresponding to the second three-dimensional coordinates; The virtual ray is displayed at the position of the second two-dimensional coordinate.

12. The VR video control method according to claim 8, wherein: The control parameter includes a user click instruction for a target direction control obtained by the mobile terminal in the target control mode; the target control mode includes a joystick control mode; the target direction control is displayed in an interface corresponding to the joystick control mode; The interactively controlling the VR video on the display screen of the display terminal based on the control parameter includes: In response to the click instruction, determining a second to-be-selected object corresponding to the click instruction; receiving a second selection instruction sent by the mobile terminal; In response to the second selection instruction, a second target to-be-selected object corresponding to the second selection instruction is selected.

Images