Virtual application interface update method, head mounted display device component and medium

Abstract

The embodiments of the present disclosure disclose a virtual application interface update method, a head-mounted display device component, and a medium. The method is applied to a head-mounted display device component, which includes a head-mounted display device, a handle, and a target device provided with a touch-sensitive display screen. A specific implementation of the method includes: starting processing each handle button supported by the target application in the handle button group provided on the handle, wherein the application running state of the target application is running; displaying each screen button supported by the target application in the touch-sensitive display screen of the target device; in response to detecting a user's key operation on each handle button and / or each screen button displayed, updating the virtual application interface of the corresponding target application displayed in the head-mounted display device according to the key operation. This implementation can guide the user to perform interactive operations through touch, thereby improving the user interaction experience.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a virtual application interface update method, a head-mounted display device component, and a medium. Background Art

[0002] Head-mounted display devices, such as AR (Augmented Reality) glasses or MR (Mixed Reality) glasses, allow users to watch displayed content after wearing them. Through a control terminal, users can control the display content of the head-mounted display device. Currently, users can control the screen of the application displayed in the head-mounted display device through a tablet control terminal (such as a mobile phone screen) to achieve interactive operations.

[0003] However, when the above method is used to control the screen of the application displayed in the head-mounted display device, the following technical problems often occur: the tablet control terminal has no button touch, cannot guide the user to perform interactive operations through touch, and the interaction method is single, resulting in a poor user interaction experience. Summary of the invention

[0004] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0005] Some embodiments of the present disclosure propose a virtual application interface update method based on key configuration, a head-mounted display device component, and a computer-readable medium to solve the technical problems mentioned in the above background technology section.

[0006] In a first aspect, some embodiments of the present disclosure provide a method for updating a virtual application interface based on button configuration, which is applied to a head-mounted display device component, wherein the head-mounted display device component includes a head-mounted display device, a handle, and a target device provided with a touch-sensitive display screen, the method comprising: starting processing each handle button supported by a target application in a handle button group provided on the handle, wherein the application running state of the target application is already running; displaying each screen button supported by the target application in the touch-sensitive display screen of the target device; in response to detecting a user's key operation on a handle button among the handle buttons and / or a screen button among the displayed screen buttons, updating the virtual application interface corresponding to the target application displayed in the head-mounted display device according to the key operation.

[0007] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device component, comprising: one or more processors; a storage device on which one or more programs are stored, and a handle on which a handle button group is arranged; a target device on which a touch-sensitive display screen is arranged, and the touch-sensitive display screen is configured to display screen buttons; a head-mounted display device, configured to display a virtual application interface of the target application; when one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0008] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the first aspect is implemented.

[0009] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the virtual application interface update method based on key configuration of some embodiments of the present disclosure, the user interaction experience is improved. Specifically, the reason for the poor user interaction experience is that the tablet-type control terminal has no key touch, cannot guide the user to perform interactive operations through touch, and the interaction mode is single, resulting in poor user interaction experience. Based on this, the virtual application interface update method based on key configuration of some embodiments of the present disclosure is applied to a head-mounted display device component, and the head-mounted display device component includes a head-mounted display device, a handle, and a target device provided with a touch-sensitive display screen. First, each handle button supported by the target application in the handle button group set on the handle is started. Among them, the application running state of the target application is already running. Thus, each handle button supported by the target application on the handle can be started, so that the user can perform interactive operations through each started handle button. Then, each screen button supported by the target application is displayed on the touch-sensitive display screen of the target device. Thus, each screen button supported by the target application can be displayed on the touch-sensitive display screen of the target device, so that the user can perform interactive operations through each screen button. Finally, in response to detecting the user's key operation on the handle buttons among the above-mentioned handle buttons and / or the screen buttons among the displayed screen buttons, the virtual application interface corresponding to the above-mentioned target application displayed in the above-mentioned head-mounted display device is updated according to the above-mentioned key operation. In this way, it is possible to respond to the user's key operation on the handle buttons or screen buttons. Also, because it is possible to respond to the user's key operation on the handle buttons or screen buttons, the user's interaction methods are increased. And because the handle buttons have a key touch, the user can be guided to perform interactive operations through the touch. As a result, the user interaction experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0011] Figure 1 is an architectural diagram of an exemplary system in which some embodiments of the present disclosure may be applied;

[0012] Figure 2 is a schematic diagram of an application scenario of a virtual application interface update method based on key configuration according to some embodiments of the present disclosure;

[0013] Figure 3 is a flow chart of some embodiments of a virtual application interface update method based on key configuration according to the present disclosure;

[0014] Figure 4 is a flow chart of other embodiments of a virtual application interface updating method based on key configuration according to the present disclosure;

[0015] Figure 5 It is a schematic diagram of the structure of a head-mounted display device component suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0017] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0018] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0019] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0020] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0021] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0022] Figure 1 An exemplary system architecture 100 that can be applied to an embodiment of the present disclosure and applied to a virtual application interface update method based on key configuration is shown.

[0023] like Figure 1 As shown, the exemplary system architecture 100 may include a head mounted display device assembly 11. The head mounted display device assembly 11 may include a head mounted display device 111, a handle 112, and a target device 113.

[0024] The head-mounted display device 111 may include at least one display screen 1111. The above-mentioned display screen is used to perform imaging in front of the user's eyes. In addition, the head-mounted display device 111 also includes a frame 1112. In some embodiments, the sensor, processing unit, memory and battery of the head-mounted display device 111 may be placed inside the frame 1112. In some optional implementations of some embodiments, one or more components of the sensor, processing unit, memory and battery may also be integrated into another independent accessory (not shown) and connected to the frame 1112 via a data cable. In some optional implementations of some embodiments, the head-mounted display device 111 may only have a display function and some sensors, and provide data processing, data storage, power supply and other capabilities through the target device 113.

[0025] The handle 112 is provided with a handle button group 1121. In some embodiments, the handle 112 and the target device 113 can communicate via a wireless connection. In some optional implementations of some embodiments, the handle 112 and the target device 113 can also be connected via a USB interface (not shown). In some optional implementations of some embodiments, the handle 112 and the head mounted display device 111 can communicate via a wireless connection.

[0026] The target device 113 may include a touch-sensitive display screen 1131. In some embodiments, the head-mounted display device 111 and the target device 113 may communicate via a wireless connection. In some optional implementations of some embodiments, the head-mounted display device 111 and the target device 113 may also be connected via a data cable (not shown).

[0027] It should be understood that Figure 1The number of head mounted display devices, handles and target devices in the embodiment is only illustrative. According to the implementation requirements, any suitable number of head mounted display devices, handles and target devices can be provided.

[0028] Figure 2 It is a schematic diagram of an application scenario of a virtual application interface updating method based on key configuration according to some embodiments of the present disclosure.

[0029] exist Figure 2 In the application scenario of , the head-mounted display device component includes a head-mounted display device 201, a handle 202, and a target device 203 provided with a touch-sensitive display screen. First, the head-mounted display device 201 can start processing each handle button 204 supported by the target application in the handle button group set on the above-mentioned handle. Among them, the application running state of the above-mentioned target application is running. Then, the head-mounted display device 201 can display each screen button 205 supported by the above-mentioned target application in the touch-sensitive display screen of the above-mentioned target device. Finally, the head-mounted display device 201 can respond to detecting the user's key operation on the handle buttons in the above-mentioned handle buttons 204 and / or the screen buttons in the displayed screen buttons 205, and according to the above-mentioned key operation, update the virtual application interface 206 corresponding to the above-mentioned target application displayed in the above-mentioned head-mounted display device 201.

[0030] It should be noted that the head-mounted display device 101 can be hardware or software. When the head-mounted display device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or it can be implemented as a single server or a single terminal device. When the head-mounted display device is embodied as software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0031] It should be understood that Figure 2 The number of head mounted display devices, handles and target devices in the embodiment is only illustrative. Any number of head mounted display devices, handles and target devices may be provided according to implementation requirements.

[0032] Continue to refer Figure 3 , shows a process 300 of some embodiments of the virtual application interface update method based on key configuration according to the present disclosure. The virtual application interface update method based on key configuration is applied to a head-mounted display device component, the head-mounted display device component includes a head-mounted display device, a handle and a target device with a touch-sensitive display screen, and includes the following steps:

[0033] Step 301: Start each handle button supported by the target application in the handle button group arranged on the handle.

[0034] In some embodiments, the execution body of the virtual application interface update method based on the key configuration (for example Figure 2 The head-mounted display device 201 or the target device 203 shown in the figure can start processing each handle button supported by the target application in the handle button group set on the handle. Among them, the head-mounted display device can be a device for users to watch virtual scenes after wearing it. The head-mounted display device can include but is not limited to: a head-mounted enhanced display device and a head-mounted hybrid display device. For example, the head-mounted enhanced display device can be AR glasses. The head-mounted hybrid display device can be MR glasses. The handle can be a game handle. The connection method between the handle and the target device can be a wired connection or a wireless connection. The handle can also be connected to the head-mounted display device through a wired connection or a wireless connection. The target device can be a mobile terminal with a touch-sensitive display screen. For example, the target device can be a mobile phone or a tablet computer. It should be noted that the wireless connection method can include but is not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0035] The above-mentioned handle button group may be individual buttons arranged on the above-mentioned handle. For example, the handle buttons in the above-mentioned handle button group may be physical buttons. The above-mentioned physical buttons may be buttons with a press tactile sensation. The above-mentioned target application may be an application currently running on the above-mentioned execution subject. The application running state of the above-mentioned target application is already running. For example, the above-mentioned target application may be a currently running game application. The individual handle buttons supported by the target application may be individual handle buttons for controlling the above-mentioned target application. It can be understood that when the model of the above-mentioned handle is the handle model supported by the above-mentioned target application, the above-mentioned individual handle buttons may be individual handle buttons in the above-mentioned handle button group.

[0036] In practice, the execution subject may, in response to detecting that the target application is started or running in the foreground, start processing of each handle button supported by the target application in the handle button group set on the handle. In practice, the execution subject may set the use state of each handle button to the start state to start each handle button. Thus, each handle button supported by the target application on the handle may be started, so that the user can perform interactive operations through each started handle button.

[0037] Step 302: Display various screen buttons supported by the target application on the touch-sensitive display screen of the target device.

[0038] In some embodiments, the execution subject may display the various screen buttons supported by the target application in the touch-sensitive display screen of the target device. Among them, the various screen buttons supported by the target application may be various screen buttons for controlling the target application. The screen buttons may be buttons in the form of controls displayed in the touch-sensitive display screen. It is understandable that the various screen buttons and the various handle buttons correspond to the application type of the target application. For example, when the target application is an application with a zoom or slide function, the various screen buttons may include screen buttons for users to perform zoom operations or slide in any direction. The various handle buttons may include handle buttons for users to perform forward, backward, left or right operations. In practice, the execution subject may display UI (User Interface) icons of the various screen buttons supported by the target application in the touch-sensitive display screen of the target device. The UI icons may be preset icons. Here, the specific settings of the UI icons are not limited. Thus, the various screen buttons supported by the target application may be displayed on the touch-sensitive display screen of the target device, so that users can perform interactive operations through the various screen buttons.

[0039] In some optional implementations of some embodiments, the execution entity may, in response to detecting that the handle is disposed at one end of the touch-sensitive display screen, display various screen buttons supported by the target application at the other end of the touch-sensitive display screen. Here, the handle may be a handle that can be disposed at one end of the touch-sensitive display screen of the target device. For example, when the touch-sensitive display screen is placed in landscape orientation, the handle may be disposed at the left end of the touch-sensitive display screen. Then, the execution entity may display various screen buttons supported by the target application at the right end of the touch-sensitive display screen. This makes it convenient for users to operate buttons with both hands.

[0040] Optionally, the execution subject may control the target device or the head mounted display device or the handle to perform a button configuration completion prompt operation in response to determining that the configuration of the handle buttons and the screen buttons are complete. The completion of the configuration of the handle buttons and the screen buttons may indicate that the execution subject has completed the startup processing of the handle buttons, and the touch-sensitive display screen displays the screen buttons supported by the target application. The button configuration completion prompt operation may be an operation that indicates that the configuration of the handle buttons and the screen buttons is complete. The button configuration completion prompt operation may include but is not limited to: an audio-visual prompt operation and / or a vibration prompt operation performed by at least one of the target device, the head mounted display device, and the handle. The audio-visual prompt operation may be an operation that emits sound and / or light to prompt that the configuration of the handle buttons and the screen buttons is complete. The vibration prompt operation may be an operation that emits vibration to prompt that the configuration of the handle buttons and the screen buttons is complete. Thus, after the configuration of the handle buttons and the screen buttons is complete, the user may be prompted to perform a button operation.

[0041] Step 303, in response to detecting a user's key operation on a handle button among the handle buttons and / or a screen button among the displayed screen buttons, updating the virtual application interface of the corresponding target application displayed in the head-mounted display device according to the key operation.

[0042] In some embodiments, the execution subject may respond to the key operation detected by the user on the handle buttons in the above-mentioned handle buttons and / or the screen buttons in the displayed screen buttons, and update the virtual application interface corresponding to the above-mentioned target application displayed in the above-mentioned head-mounted display device according to the above-mentioned key operation. Among them, the above-mentioned virtual application interface may be an interface displayed on the display screen of the above-mentioned head-mounted display device when the above-mentioned target application is running. For example, when the above-mentioned target application is a game application, the above-mentioned virtual application interface may be a virtual game interface. The above-mentioned key operation may be an operation of the user pressing the handle button or the user clicking and sliding the screen button. Here, the specific way in which the user performs the key operation is not limited. In one or more embodiments, in the head-mounted display device, the virtual application interface may be hidden or have a certain degree of transparency. It is understandable that the user may only operate at least one of the above-mentioned handle buttons, or only operate at least one of the above-mentioned screen buttons, or simultaneously operate at least one of the above-mentioned handle buttons and at least one of the above-mentioned screen buttons.

[0043] In practice, the execution subject may update the interface display content of the virtual application interface in response to the operation instruction corresponding to the key operation, and update the interface content of the virtual application interface corresponding to the target application displayed in the head-mounted display device according to the operation instruction. For example, the key operation may be an operation in which a user presses a handle button representing the movement of a virtual object in the virtual application interface forward. The virtual object may be an object displayed in the virtual application interface, for example, a virtual character. Then the operation instruction may be an instruction representing the movement of the virtual object forward in the virtual application interface. The execution subject may move the virtual object forward in the virtual application interface.

[0044] In practice, the execution subject may also update the application parameters of the virtual application interface displayed in the head mounted display device according to the operation instruction represented by the key operation. The application parameters may be related parameters when the application is running, for example, including but not limited to: playback volume. The key operation may be an operation in which the user clicks to increase the playback volume of the virtual application interface. The operation instruction may be an instruction to increase the playback volume of the virtual application interface. The execution subject may increase the playback volume of the virtual application interface. Thus, the user's key operation on the handle button or screen button may be responded to.

[0045] Optionally, first, the execution subject may, in response to detecting a disabling operation of the user acting on the activated handle button, disable the handle button corresponding to the disabling operation. The disabling operation may be an operation of disabling a button, which may invalidate the corresponding button. The disabling operation may include, but is not limited to: multiple consecutive click / press operations, long press operations. For example, the disabling operation may be an operation in which the user continuously clicks or presses a button for a preset number of times. The disabling operation may also be an operation in which the user presses a button for more than a preset time. The preset number of times may be 3 times, and the preset time may be 10 seconds. Here, the specific settings of the preset number of times and the preset time are not limited. In practice, the execution subject may set the use state of the handle button corresponding to the disabling operation to a disabled state, so that the handle button is invalid. Then, in response to the disabled handle button satisfying the button operation condition corresponding to the target application, the screen button corresponding to the disabled handle button may be displayed in the touch-sensitive display screen. The button operation condition may be "the necessary handle button identification group corresponding to the target application includes the handle button identification of the handle button". Among them, the above-mentioned necessary handle button identification group can be the identification of the handle buttons necessary to control the above-mentioned target application. The necessary handle button identification group can be pre-configured by the developer. In practice, the above-mentioned execution entity can display the UI icon of the screen button corresponding to the disabled handle button in the above-mentioned touch-sensitive display screen. Thereby, when the handle button disabled by the user is a necessary handle button, the corresponding screen button can be displayed in the touch-sensitive display screen of the target device.

[0046] Optionally, the execution subject may, in response to detecting a user's undisabling operation on a disabled handle button, undisabling the handle button corresponding to the undisabling operation. The undisabling operation may be an operation to undisabling the disabled button. For example, the undisabling operation may be a disabling operation performed again. It can be understood that after the user performs a disabling operation on a button, the user performs a disabling operation on the button again. For example, the user first presses a handle button three times in succession to disable the handle button. The undisabling operation corresponding to the handle button is the user pressing the handle button three times in succession again. It can be understood that the undisabling operation corresponding to the handle button is also the user pressing the handle button four times in succession again. Here, the specific setting of the undisabling operation is not limited. In practice, the execution subject may change the use state of the handle button from a disabled state to a started state. Thus, after the user disables the handle button, the handle button can be started by the undisabling operation.

[0047] Optionally, within a preset time period, for any group of corresponding handle buttons and screen buttons among the activated handle buttons and the displayed screen buttons, the execution subject may perform the following steps:

[0048] In the first step, in response to the number of handle button operations corresponding to the handle button and the number of screen button operations corresponding to the screen button satisfying the handle button priority condition, the screen button is hidden in the touch-sensitive display screen. The operation instructions corresponding to the handle button and the screen button corresponding to each other are the same. That is, the button functions of the handle button and the screen button corresponding to each other are the same. The number of handle button operations may be the number of times the user operates the handle button within the preset time period. The number of screen button operations may be the number of times the user operates the screen button within the preset time period. The handle button priority condition may be "the number of handle button operations is greater than or equal to N, and the number of screen button operations is less than or equal to M". N may be any positive integer, M may be any integer greater than or equal to 0, and M is less than N. For example, N may be 5, and M may be 0. The specific settings of the preset time period, N and M are not limited here. Thus, when the user operates the handle button more times and operates the screen button with the same function less times, the screen button may be hidden in the touch-sensitive display screen to save the display space of the touch-sensitive display screen.

[0049] In the second step, in response to the number of handle button operations corresponding to the above handle button and the number of screen button operations corresponding to the above screen button satisfying the screen button priority condition, the above handle button is disabled. The above screen button priority condition may be "the number of handle button operations is less than or equal to X, and the number of screen button operations is greater than or equal to Y". Y may be any positive integer, X may be any integer greater than or equal to 0, and X is less than Y. For example, Y may be 5, and X may be 0. There is no limitation on the specific settings of X and Y. Thus, when the user operates the screen button more times and operates the handle button with the same function less times, the handle button may be disabled.

[0050] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the virtual application interface update method based on key configuration of some embodiments of the present disclosure, the user interaction experience is improved. Specifically, the reason for the poor user interaction experience is that the tablet-type control terminal has no key touch, cannot guide the user to perform interactive operations through touch, and the interaction mode is single, resulting in poor user interaction experience. Based on this, the virtual application interface update method based on key configuration of some embodiments of the present disclosure is applied to a head-mounted display device component, and the head-mounted display device component includes a head-mounted display device, a handle, and a target device provided with a touch-sensitive display screen. First, each handle button supported by the target application in the handle button group set on the handle is started. Among them, the application running state of the target application is already running. Thus, each handle button supported by the target application on the handle can be started, so that the user can perform interactive operations through each started handle button. Then, each screen button supported by the target application is displayed on the touch-sensitive display screen of the target device. Thus, each screen button supported by the target application can be displayed on the touch-sensitive display screen of the target device, so that the user can perform interactive operations through each screen button. Finally, in response to detecting the user's key operation on the handle buttons among the above-mentioned handle buttons and / or the screen buttons among the displayed screen buttons, the virtual application interface corresponding to the above-mentioned target application displayed in the above-mentioned head-mounted display device is updated according to the above-mentioned key operation. In this way, it is possible to respond to the user's key operation on the handle buttons or screen buttons. Also, because it is possible to respond to the user's key operation on the handle buttons or screen buttons, the user's interaction methods are increased. And because the handle buttons have a key touch, the user can be guided to perform interactive operations through the touch. As a result, the user interaction experience is improved.

[0051] Further references Figure 4 , which shows a process 400 of another embodiment of a method for updating a virtual application interface based on key configuration. The process 400 of the method for updating a virtual application interface based on key configuration is applied to a head-mounted display device component, the head-mounted display device component includes a head-mounted display device, a handle, and a target device with a touch-sensitive display screen, and includes the following steps:

[0052] Step 401, in response to the presence of a handle button in the handle button group corresponding to the start handle button identifier in the start handle button identifier group, start processing is performed on each handle button in the handle button group corresponding to the start handle button identifier in the start handle button identifier group.

[0053] In some embodiments, the execution body of the virtual application interface update method based on the key configuration (for example Figure 2The head mounted display device 201 or the target device 203 shown in the figure can respond to the presence of a handle button corresponding to the start handle button identifier in the start handle button identifier group in the handle button group, and start processing for each handle button in the handle button group corresponding to the start handle button identifier in the start handle button identifier group. Among them, the start handle button identifier group corresponds to the target application. The start handle button identifier group can be the identifier of each handle button required to run the target application configured in advance. The start handle button identifier in the start handle button identifier group can uniquely identify the handle button. For example, the start handle button identifier "up" can represent a handle button that moves upward. In practice, the execution subject can set the use state of each handle button corresponding to the start handle button identifier in the start handle button identifier group to the start state. Thus, each handle button in the handle button group corresponding to the start handle button identifier in the start handle button identifier group can be started.

[0054] Step 402, disabling each unactivated handle button in the handle button group.

[0055] In some embodiments, the execution subject may disable each unactivated handle button in the handle button group. In practice, the execution subject may set the usage status of each unactivated handle button in the handle button group to a disabled status. It should be noted that pressing a handle button whose usage status is a disabled status is invalid. Thus, each handle button that does not support the target application may be disabled.

[0056] Step 403: remove the start handle button identification corresponding to each of the activated handle buttons from the start handle button identification group, and obtain the start handle button identification group after removal as the substitute screen button identification group.

[0057] In some embodiments, the execution subject may remove the start handle button identification corresponding to each activated handle button from the start handle button identification group, and obtain the removed start handle button identification group as the substitute screen button identification group. Thus, the handle buttons corresponding to each start handle button identification in the removed start handle button identification group are the handle buttons that need to be activated on the handle but have not been activated.

[0058] Step 404: Display each screen button and the screen button corresponding to each substitute screen button identifier in the substitute screen button identifier group on the touch-sensitive display screen.

[0059] In some embodiments, the execution entity may display the above-mentioned screen buttons and the screen buttons corresponding to the above-mentioned substitute screen button identifiers in the above-mentioned substitute screen button identifier group in the above-mentioned touch-sensitive display screen. In practice, the execution entity may display the UI (User Interface) icons of the screen buttons corresponding to the above-mentioned substitute screen button identifiers in the above-mentioned touch-sensitive display screen. The above-mentioned UI icons may be preset icons. Here, there is no limitation on the specific setting of the UI icons. Thus, the screen buttons corresponding to the various handle buttons that need to be activated on the handle but have not been activated can be displayed on the touch-sensitive display screen.

[0060] Step 405, in response to detecting a user's key operation on a handle button among the handle buttons and / or a screen button among the displayed screen buttons, updating the virtual application interface of the corresponding target application displayed in the head-mounted display device according to the key operation.

[0061] In some embodiments, the specific implementation of step 405 and the technical effects brought about can be referred to Figure 3 The corresponding step 303 in the embodiments will not be described in detail here.

[0062] from Figure 4 It can be seen that Figure 3 Compared with the description of some corresponding embodiments, Figure 4 The process 400 of the method for updating the virtual application interface based on button configuration in some corresponding embodiments embodies the step of expanding the substitute screen button identifier. Therefore, the scheme described in these embodiments can display the screen buttons corresponding to each handle button that needs to be activated in the handle but has not been activated on the touch-sensitive display screen. This can support the combination of different models of handles with the target device, and uniformly configure the handle buttons and screen buttons to control the virtual application interface in the head-mounted display device.

[0063] Reference below Figure 5 , which shows a head mounted display device component (e.g., Figure 1 Schematic diagram of the structure of the head-mounted display device component 11)500. Figure 5 The head mounted display device components shown are merely examples and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0064] like Figure 5As shown, the head mounted display device assembly 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the head mounted display device assembly 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0065] Typically, the following devices may be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, a handle, a target device, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, a head-mounted display device, etc.; and communication devices 509. The communication devices 509 may allow the head-mounted display device assembly 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The head mounted display device assembly 500 having various devices is shown, but it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0066] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0067] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0068] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0069] The computer-readable medium may be included in the head-mounted display device component; or it may exist independently without being assembled into the head-mounted display device component. The computer-readable medium carries one or more programs. When the one or more programs are executed by the head-mounted display device component, the head-mounted display device component: starts processing each handle button supported by the target application in the handle button group set on the handle, wherein the application running state of the target application is already running; displays each screen button supported by the target application in the touch-sensitive display screen of the target device; in response to detecting a user's key operation on each handle button and / or each screen button displayed, according to the key operation, updates the virtual application interface corresponding to the target application displayed in the head-mounted display device.

[0070] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0071] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0072] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0073] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for updating a virtual application interface based on key configuration, applied to a head-mounted display device assembly, wherein the head-mounted display device assembly includes a head-mounted display device, a handle, and a target device provided with a touch-sensitive display screen, including: Performing a start process on each handle button supported by a target application in a handle button group arranged on the handle, wherein the application running state of the target application is already running; Displaying various screen buttons supported by the target application on the touch-sensitive display screen of the target device, comprising: in response to detecting that the handle is disposed at one end of the touch-sensitive display screen, displaying various screen buttons supported by the target application at the other end of the touch-sensitive display screen; In response to detecting a key operation of a user acting on a handle button among the handle buttons and / or a screen button among the displayed screen buttons, updating a virtual application interface corresponding to the target application displayed in the head mounted display device according to the key operation; During the preset time period, for any group of corresponding handle buttons and screen buttons among the activated handle buttons and displayed screen buttons, the following steps are performed: In response to the number of handle button operations corresponding to the handle button and the number of screen button operations corresponding to the screen button satisfying a handle button priority condition, hiding the screen button in the touch-sensitive display screen; In response to the number of handle button operations corresponding to the handle button and the number of screen button operations corresponding to the screen button satisfying a screen button priority condition, disabling processing is performed on the handle button.

2. The method according to claim 1, wherein: The starting process of each handle button supported by the target application in the handle button group arranged on the handle includes: In response to the presence of a handle button in the handle button group corresponding to a start handle button identifier in the start handle button identifier group, start processing is performed on each handle button in the handle button group corresponding to the start handle button identifier in the start handle button identifier group, wherein the start handle button identifier group corresponds to the target application.

3. The method according to claim 2, wherein: After the start-up process is performed on each handle button supported by the target application in the handle button group arranged on the handle, the method further includes: Disable each unactivated handle button in the handle button group.

4. The method according to claim 2 or 3, wherein: The step of displaying various screen buttons supported by the target application on the touch-sensitive display screen of the target device includes: Eliminate the start handle button identification corresponding to each activated handle button from the start handle button identification group, and obtain the eliminated start handle button identification group as the substitute screen button identification group; The screen buttons and the screen buttons corresponding to the substitute screen button identifiers in the substitute screen button identifier group are displayed on the touch-sensitive display screen.

5. The method according to claim 1, wherein: The method further comprises: In response to detecting a disabling operation performed by a user on an activated handle button, disabling processing is performed on the handle button corresponding to the disabling operation; In response to the disabled handle button satisfying the key operation condition corresponding to the target application, a screen button corresponding to the disabled handle button is displayed in the touch-sensitive display screen.

6. The method according to claim 3 or 5, wherein: The method further comprises: In response to detecting a disabling operation performed by a user on a disabled handle button, disabling processing is performed on the handle button corresponding to the disabling operation.

7. The method according to claim 1, wherein: In response to detecting a key operation of a user acting on a handle button among the handle buttons and / or a screen button among the displayed screen buttons, before updating a virtual application interface corresponding to the target application displayed in the head mounted display device according to the key operation, the method further includes: In response to determining that the configuration of each handle button and each screen button is complete, the target device or the head mounted display device or the handle is controlled to perform a button configuration completion prompt operation.

8. A head mounted display device assembly, comprising: one or more processors; a storage device having one or more programs stored thereon; A handle, on which a handle button group is arranged; a target device having a touch-sensitive display screen disposed thereon, the touch-sensitive display screen being configured to display screen buttons; a head mounted display device configured to display a virtual application interface of a target application; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images