Interface control method, head-mounted display device and storage medium

Abstract

The embodiment of the invention provides an interface control method, head-mounted display equipment and a storage medium, and belongs to the field of interface control. The method comprises the following steps: acquiring a stack top Activity name and a stack top window name; according to the stack top Activity name and the stack top window name, setting a view mode of the head-mounted display device; in response to the fact that the view mode before setting is different from the view mode after setting, user interface UI elements displayed in the three-dimensional space are adjusted according to the view mode after setting. According to the technical scheme, the immersive effect of the user interface can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of interface control, and particularly to an interface control method, a head-mounted display device, and a storage medium. Background Art

[0002] Currently, User Interface (UI) elements can be displayed in three-dimensional space by translation and rotation from a two-dimensional plane to achieve a surround experience. However, the UI elements displayed in three-dimensional space are all window window classes and cannot apply the principle of pushing and popping of the Activity stack, so that the UI elements displayed in three-dimensional space can only be controlled by the application program that carries them and cannot be controlled by external applications or the system layer. They do not participate in the layout and interaction of the user interface of the entire system, and the immersive effect of the user interface is poor. Summary of the Invention

[0003] Embodiments of the present invention provide an interface control method, a head-mounted display device, and a storage medium, aiming to improve the immersive effect of the user interface.

[0004] In a first aspect, an embodiment of the present invention provides an interface control method applied to a head-mounted display device. The method includes:

[0005] Obtain the name of the top Activity and the name of the top window;

[0006] Set the viewing mode of the head-mounted display device according to the name of the top Activity and the name of the top window;

[0007] In response to the viewing mode before setting being different from the viewing mode after setting, adjust the User Interface (UI) elements of the user interface displayed in three-dimensional space according to the viewing mode after setting.

[0008] In a second aspect, an embodiment of the present invention further provides a head-mounted display device. The head-mounted display device includes a processor, a memory, a computer program stored on the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory. When the computer program is executed by the processor, the interface control method described in the first aspect is implemented.

[0009] In a third aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the interface control method described in the first aspect.

[0010] An embodiment of the present invention provides an interface control method, a head-mounted display device, and a storage medium. Based on the name of the top Activity and the name of the top window in the stack, the embodiment of the present invention sets the viewing mode of the head-mounted display device, so that when the viewing modes before and after the setting are different, the UI elements of the user interface displayed in the three-dimensional space can be adjusted in a timely manner according to the viewing mode after the setting, so as to achieve seamless switching between different viewing modes, so that the UI elements in the three-dimensional space can participate in the user interface layout and interaction of the entire system, improving the immersive effect of the user interface. Description of the Drawings

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0012] Figure 1 It is a schematic flowchart of an interface control method provided by an embodiment of the present invention;

[0013] Figure 2 It is a schematic diagram of a user interface in an embodiment of the present invention;

[0014] Figure 3 It is another schematic diagram of a user interface in an embodiment of the present invention;

[0015] Figure 4 It is another schematic diagram of a user interface in an embodiment of the present invention;

[0016] Figure 5 It is another schematic diagram of a user interface in an embodiment of the present invention;

[0017] Figure 6 It is another schematic diagram of a user interface in an embodiment of the present invention;

[0018] Figure 7 It is another schematic diagram of a user interface in an embodiment of the present invention;

[0019] Figure 8 It is a schematic block diagram of a head-mounted display device provided by an embodiment of the present invention. Detailed Embodiments

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] The flowchart shown in the accompanying drawings is only an example, and does not necessarily include all the contents and operations / steps, nor does it necessarily execute in the described order. For example, some operations / steps can be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0022] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0023] Currently, user interface (UI) elements can be displayed in three-dimensional space from a two-dimensional plane through translation and rotation to achieve a surround experience. However, the UI elements displayed in three-dimensional space are all window classes and cannot apply the principle of pushing and popping the Activity stack, so that the UI elements displayed in three-dimensional space can only be controlled by the application program that carries them and cannot be controlled by external applications or the system layer. They do not participate in the user interface layout and interaction of the entire system, and the immersive effect of the user interface is poor.

[0024] To solve the above problems, the embodiments of the present invention provide an interface control method, a head-mounted display device, and a storage medium. The embodiments of the present invention set the viewing mode of the head-mounted display device based on the name of the top Activity in the stack and the name of the top window, so that when the viewing modes before and after the setting are different, the UI elements of the user interface displayed in three-dimensional space can be adjusted in a timely manner according to the viewing mode after the setting to achieve seamless switching between different viewing modes, so that the UI elements in three-dimensional space can participate in the user interface layout and interaction of the entire system, improving the immersive effect of the user interface. Among them, the head-mounted display device can include augmented reality (AR) glasses, AR helmets, mixed reality (MR) glasses, and MR helmets, etc.

[0025] The following will describe in detail some embodiments of the present invention with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0026] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an interface control method provided by an embodiment of the present invention.

[0027] As Figure 1 shown, the interface control method includes steps S101 to S103.

[0028] Step S101, obtain the name of the top Activity in the stack and the name of the top window in the stack.

[0029] In this embodiment, the name of the top Activity in the stack refers to the name of the Activity located at the top of the Activity task stack, and the name of the top window in the stack refers to the name of the window located at the top of the window task stack. Among them, the name of the top Activity in the stack may include the name of the full-screen interface that is first displayed at the top layer of the screen currently or the name of the full-screen interface that is displayed at the top layer of the screen currently. The name of the top window in the stack may include the name of the focused window currently displayed on the screen or the name of the window at the top layer of the screen currently. The focused window refers to the window that can currently receive user input (such as keyboard input, mouse click, etc.).

[0030] In some embodiments, obtaining the name of the top Activity in the stack and the name of the top window in the stack may include: obtaining the name of the top Activity through a preset top-Activity collection interface and obtaining the name of the top window through a preset top-window collection interface. Among them, the top-Activity collection interface and the top-window collection interface can be implemented through instrumentation. In this embodiment, the name of the top Activity in the stack and the name of the top window in the stack can be quickly obtained through the top-Activity collection interface and the top-window collection interface.

[0031] It should be noted that the preset top-Activity collection interface (which can be a collection interface capable of collecting the name of the top Activity in the stack) and the preset top-window collection interface (which can be a collection interface capable of collecting the name of the top window in the stack) can be set based on the actual situation, as long as they can collect the name of the top Activity in the stack or the name of the top window in the stack. The embodiments of the present invention do not make specific limitations on this.

[0032] For example, taking the Android system as an example, a top-of-stack Activity collection interface is newly created in the creation and flow process logic of the Activity task stack. For example, an interface notifyFullActivityFirstCreate is newly created in the creation (ActivityStarter) process logic of the Activity task stack to monitor whether a full-screen interface is first displayed at the top of the screen. An interface notifyActivityResumed is newly created in the flow (ActivityRecord) process logic of the Activity task stack to monitor whether a full-screen interface is being displayed at the top of the screen. Another example is taking the Android system as an example, a top-of-stack window collection interface is newly created in the window creation management and sorting destruction logic. For example, an interface notifyFocueswindowChanged is newly created in the window creation management (windowManagerService) logic to monitor whether a focused window is being displayed on the screen. An interface notifySystemwindowChanged is newly created in the window sorting destruction (windowState & windowToken) logic to monitor whether a window is being displayed at the top of the screen.

[0033] Step S102: Set the viewing mode of the head-mounted display device according to the name of the top-of-stack Activity and the name of the top-of-stack window.

[0034] In this embodiment, the viewing modes of the head-mounted display device include a full-view desktop mode, a control center mode, a lock screen mode, an immersive application mode, a star flash notification mode, etc. It can be understood that the full-view desktop mode, the control center mode, the lock screen mode, the immersive application mode, and the star flash notification mode can be switched between each other. For example, as Figure 2 shown, in the full-view desktop mode, there are UI elements displayed in the central viewing area 11, UI elements displayed in the left viewing area 12, UI elements displayed in the right viewing area 13, UI elements displayed in the upper viewing area 14, and UI elements displayed in the lower viewing area 15 in the three-dimensional space 10. It should be noted that Figure 2 the rectangular frames of the viewing areas in Figure 2 are the approximate positions of the exemplary viewing areas, and the actual user interface does not have Figure 2 the rectangular frames of the viewing areas in

[0035] Among them, the UI elements displayed in the central visual field area 11 may include a status bar and one or more application icons. The UI elements displayed in the left visual field area 12 may include one or more notification bars. The UI elements displayed in the right visual field area 13 may include one or more floating windows. The UI elements displayed in the upper visual field area 14 may include widgets. For example, the widgets include a weather widget and a calendar widget. The UI elements displayed in the lower visual field area 15 may include a quick access bar 151 (the quick access bar 151 may also be referred to as a dock bar), a volume adjustment component 152, and a brightness adjustment component 153.

[0036] For another example, as Figure 3 shown, in the control center mode, a control center bar 21 spanning the left visual field area 12, the central visual field area 11, and the right visual field area 13 is displayed in the three-dimensional space 10. Among them, Figure 3 the dashed rectangular frames of the respective visual field areas in Figure 3 are the approximate positions of the respective exemplary visual field areas, and the dashed rectangular frames of the respective visual field areas do not exist in the actual user interface. For another example, as Figure 4 shown, in the immersive application mode, only the UI elements 22 of the application programs running in the foreground of the head-mounted display device and the status bar (the status bar displays UI elements such as time, wifi icon, and battery icon) are displayed in the three-dimensional space 10, and other system UI elements are not displayed, such as Figure 2 the UI elements in

[0037] Among them, the system UI elements are the UI elements inherent in the operating system of the head-mounted display device and are different from the UI elements of the application programs installed in the head-mounted display device.

[0037] For another example, as Figure 5 shown, in the lock screen mode, a lock screen UI element 23 and a status bar 24 are displayed in the three-dimensional space 10. The lock screen UI element 23 may be a lock screen wallpaper or a lock screen animation, etc. In some embodiments, in the lock screen mode, the widgets in Figure 2 may also be displayed in the three-dimensional space 10. For example, the widgets include a weather widget and a calendar widget. For another example, as Figure 6 shown, in the star flash notification mode, the head-mounted display device switches from the screen-off state to the screen-on state, and a call pop-up window 25 (in other embodiments, UI elements such as a banner notification, a charging animation, or a shutdown pop-up window may also be displayed), a calendar widget 26, a weather widget 27, etc. are displayed in the three-dimensional space 10. In addition, after the call pop-up window 25 is displayed for a certain period of time, such as 3 seconds, the head-mounted display device switches from the screen-on state to the screen-off state, as Figure 7As shown, no UI elements are displayed in the three-dimensional space 10 at this time. It should be noted that in any view desktop mode, if the display of a banner notification, an incoming call pop-up window, a charging animation, or a shutdown pop-up window is triggered, the corresponding UI elements of the banner notification, the incoming call pop-up window, the charging animation, or the shutdown pop-up window will also be displayed in the three-dimensional space 10.

[0038] In some embodiments, setting the viewing mode of the head-mounted display device according to the name of the top Activity in the stack and the name of the top window in the stack may include: determining a viewing mode identifier according to the name of the top Activity in the stack and the name of the top window in the stack, where the viewing mode identifier is used to identify the viewing mode; writing the viewing mode identifier into a database to set the viewing mode of the head-mounted display device. Among them, the viewing mode identifier can be written into the database through a ContentProvider component. In this embodiment, the viewing mode of the head-mounted display device can be set by writing the viewing mode identifier into the database, which improves the efficiency and real-time performance of setting the viewing mode.

[0039] It should be noted that different viewing modes correspond to different viewing mode identifiers, and the viewing mode identifiers can be set based on actual situations, and the embodiments of the present invention do not make specific limitations in this regard. For example, the viewing modes include a full-view desktop mode, a lock screen mode, a control center mode, an immersive application mode, and a star flash notification mode. The viewing mode identifier of the full-view desktop mode is 0, the viewing mode identifier of the lock screen mode is 1, the viewing mode identifier of the control center mode is 2, the viewing mode identifier of the immersive application mode is 3, and the viewing mode identifier of the star flash notification mode is 4.

[0040] In some embodiments, determining the viewing mode identifier according to the name of the top Activity in the stack and the name of the top window in the stack may include: querying a data table containing the mapping relationship between the package name, the name of the top window in the stack, and the viewing mode identifier according to the package name carried by the name of the top Activity in the stack and the name of the top window in the stack, so as to obtain the corresponding viewing mode identifier. Among them, the mapping relationship between the package name, the name of the top window in the stack, and the viewing mode identifier included in the data table can be set based on actual situations, and the embodiments of the present invention do not make specific limitations in this regard.

[0041] For example, the server side of the KeyguardService in the application framework layer obtains the data of the notifyFullActivityFirstCreate interface and the notifyActivityResumed interface, as well as the data of the notifyFocuesWindowChanged interface and the notifySystemWindowChanged interface. That is, the server side of the KeyguardService in the application framework layer obtains the name of the top Activity in the stack and the name of the top window in the stack, and transmits them to the client of the KeyguardService in SystemUI through the binder service. The client located in SystemUI determines the view mode identifier according to the name of the top Activity in the stack and the name of the top window in the stack, and then writes the view mode identifier to the database through the ContentProvider component to set the view mode of the head-mounted display device.

[0042] Step S103, in response to the view mode before setting being different from the view mode after setting, adjust the UI elements of the user interface displayed in the three-dimensional space according to the view mode after setting.

[0043] In this embodiment, the view mode of the head-mounted display device is set based on the name of the top Activity in the stack and the name of the top window in the stack. When the view mode before and after setting is different, the UI elements of the user interface displayed in the three-dimensional space can be adjusted in time according to the view mode after setting, so as to achieve seamless switching between different view modes, so that the UI elements in the three-dimensional space can participate in the user interface layout and interaction of the entire system, improving the immersive effect of the user interface.

[0044] In some embodiments, adjusting the UI elements of the user interface displayed in the three-dimensional space according to the view mode after setting may include: obtaining at least one UI element corresponding to the view mode after setting, where the at least one UI element includes a first UI element and / or a second UI element, the first UI element is a UI element already displayed in the three-dimensional space, and the second UI element is a UI element not displayed in the three-dimensional space; deleting the first UI element in the three-dimensional space and / or adding the second UI element in the three-dimensional space. In this embodiment, the switching of the view mode can be realized by adding and deleting UI elements in the three-dimensional space, improving the fluency of the view mode switching.

[0045] In some embodiments, in response to the field-of-view mode listener detecting a change in the field-of-view mode identifier in the database, it is determined that the field-of-view mode before setting is different from the field-of-view mode after setting. The field-of-view mode listener is registered with the database through the ContentProvider component when the UI element is first displayed in the three-dimensional space, and the field-of-view mode listener is used to detect whether the field-of-view mode identifier in the database has changed.

[0046] In some embodiments, deleting the first UI element in the three-dimensional space and / or adding a second UI element in the three-dimensional space includes: in response to the field-of-view mode before setting not being the immersive application mode, deleting the first UI element in the three-dimensional space and / or adding a second UI element in the three-dimensional space. In the immersive application mode, the UI elements of the application running in the foreground are displayed in the three-dimensional space. For example, as Figure 4 shown in the immersive application mode, the UI element 22 of the application running in the foreground and the status bar ( Figure 4 the status bar in which the time, WiFi icon, and battery level are displayed) are shown in the three-dimensional space 10, and there are no other system UI elements to achieve immersive use of the application.

[0047] For example, the field-of-view mode before setting is the global desktop mode as Figure 2 shown, and the field-of-view mode after setting is the control center mode as Figure 3 shown. Therefore, the UI element corresponding to the global desktop mode in the three-dimensional space 10 is deleted, and the control center bar 21 corresponding to the control center mode and the status bar ( Figure 3 the status bar in which the time, WiFi icon, and battery level are displayed) are added in the three-dimensional space 10. Another example is that the field-of-view mode before setting is the global desktop mode as Figure 2 shown, and the field-of-view mode after setting is the immersive application mode as Figure 4 shown. Therefore, the UI element corresponding to the global desktop mode in the three-dimensional space 10 is deleted, and the UI element 22 of the application running in the foreground and the status bar ( Figure 4 the status bar in which the time, WiFi icon, and battery level are displayed) are added in the three-dimensional space 10.

[0048] In some embodiments, after obtaining at least one UI element corresponding to the set vision mode, the method further includes: in response to the head-mounted display device being in the screen-off state and the second UI element including a UI element of a preset type, automatically controlling the head-mounted display device to switch from the screen-off state to the screen-on state and adding a second UI element in the three-dimensional space; in response to the time for displaying the second UI element reaching a preset time, automatically controlling the head-mounted display device to switch from the screen-on state to the screen-off state and deleting the second UI element in the three-dimensional space. Wherein, the UI element of the preset type may include a banner notification, an incoming call pop-up window, a charging animation, a shutdown pop-up window, etc., and the preset time can be set based on actual situations, and the embodiments of the present invention do not make specific limitations thereto. For example, the preset time is 3 seconds.

[0049] For example, if the set vision mode is the Figure 6 star flash notification mode as shown, the second UI element includes an incoming call pop-up window 25, a calendar widget 26, and a weather widget 27, and the head-mounted display device is in the Figure 7 screen-off state as shown. Therefore, the head-mounted display device is automatically controlled to switch from the screen-off state to the screen-on state, and the incoming call pop-up window 25, the calendar widget 26, and the weather widget 27 are displayed in the three-dimensional space 10. Then, after a certain time, such as 3 seconds, of displaying the incoming call pop-up window 25, the calendar widget 26, and the weather widget 27, the head-mounted display device is automatically controlled to switch from the screen-on state to the Figure 7 screen-off state as shown. At this time, no UI element is displayed in the three-dimensional space 10.

[0050] In some embodiments, adjusting the UI elements of the user interface displayed in the three-dimensional space according to the set vision mode may include: in response to the vision mode before setting being the immersive application mode, blocking the UI elements of the application program in the three-dimensional space and displaying the UI elements corresponding to the set vision mode in the three-dimensional space. In this embodiment, by blocking the UI elements of the application program in the three-dimensional space, the UI elements of the application program can survive in the background so as to be quickly switched back to the immersive application mode subsequently.

[0051] In some embodiments, blocking the UI elements of the application program in the three-dimensional space may include: displaying a mask UI element in the area where the UI elements of the application program are displayed in the three-dimensional space to block the UI elements of the application program in the three-dimensional space, where the level of the mask UI element is lower than the level of the UI elements corresponding to the set vision mode. It should be noted that the size of the mask UI element is greater than or equal to the area where the UI elements of the application program are displayed in the three-dimensional space.

[0052] For example, the vision mode before setting is the Figure 4 immersive application mode as shown, and the vision mode after setting is the Figure 5In the lock screen mode shown, a mask UI element is displayed in the area where the UI element 22 of the application is displayed in the three-dimensional space 10, so as to block the UI element 22 of the application in the three-dimensional space 10, and the UI element corresponding to the lock screen mode is displayed in the three-dimensional space 10. For another example, the viewing mode before setting is the immersive application mode shown in Figure 4 and the viewing mode after setting is the control center mode shown in Figure 3 In this case, a mask UI element is displayed in the area where the UI element 22 of the application is displayed in the three-dimensional space 10, so as to block the UI element 22 of the application in the three-dimensional space 10, and the UI element corresponding to the control center mode is displayed in the three-dimensional space 10.

[0053] For another example, the viewing mode before setting is the immersive application mode shown in Figure 4 and the viewing mode after setting is the control center mode shown in Figure 3 Moreover, among the UI elements of the application, there are UI elements displayed in the central viewing area 11, UI elements displayed in the left viewing area 12, and UI elements displayed in the right viewing area 13. In this case, a mask UI element is displayed in each of the central viewing area 11, the left viewing area 12, and the right viewing area 13, so as to completely block the UI elements of the application in the three-dimensional space 10, and the UI element corresponding to the control center mode is displayed in the three-dimensional space 10.

[0054] Please refer to Figure 8 , Figure 8 which is a schematic block diagram of the structure of a head-mounted display device provided by an embodiment of the present invention.

[0055] As shown in Figure 8 , the head-mounted display device 100 includes a processor 101 and a memory 102. The processor 101 and the memory 102 are connected through a bus 103, and this bus is, for example, an I2C (Inter-integrated Circuit) bus.

[0056] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire head-mounted display device. The processor 301 can be a Central Processing Unit (CPU), and this processor 101 can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or this processor can also be any conventional processor, etc.

[0057] Specifically, the memory 102 can be a Flash chip, Read-Only Memory (ROM), magnetic disk, optical disc, USB flash drive, or mobile hard disk, etc.

[0058] Those skilled in the art can understand that Figure 8 the structure shown in is only a block diagram of some structures related to the solution of the embodiment of the present invention, and does not constitute a limitation on the head-mounted display device to which the solution of the embodiment of the present invention is applied. The specific head-mounted display device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0059] Among them, the processor 101 is used to run the computer program stored in the memory 102 and implement any one of the interface control methods provided by the embodiments of the present invention when executing the computer program.

[0060] In one embodiment, the processor 101 is used to run the computer program stored in the memory and implement the following steps when executing the computer program:

[0061] Obtain the name of the top Activity and the name of the top window;

[0062] Set the viewing mode of the head-mounted display device according to the name of the top Activity and the name of the top window;

[0063] In response to the viewing mode before setting being different from the viewing mode after setting, adjust the user interface (UI) elements displayed in the three-dimensional space according to the viewing mode after setting.

[0064] In some embodiments, when the processor 101 implements setting the viewing mode of the head-mounted display device according to the name of the top Activity in the stack and the name of the top window in the stack, it is configured to implement:

[0065] Determine a viewing mode identifier according to the name of the top Activity in the stack and the name of the top window in the stack, where the viewing mode identifier is used to identify the viewing mode;

[0066] Write the viewing mode identifier into a database to set the viewing mode of the head-mounted display device.

[0067] In some embodiments, when the processor 101 implements adjusting the UI elements of the user interface (UI) displayed in a three-dimensional space according to the set viewing mode, it is configured to implement:

[0068] Obtain at least one UI element corresponding to the set viewing mode, where the at least one UI element includes a first UI element and / or a second UI element, the first UI element is a UI element that has been displayed in the three-dimensional space, and the second UI element is a UI element that has not been displayed in the three-dimensional space;

[0069] Delete the first UI element in the three-dimensional space and / or add the second UI element in the three-dimensional space.

[0070] In some embodiments, when the processor 101 implements deleting the first UI element in the three-dimensional space and / or adding the second UI element in the three-dimensional space, it is configured to implement:

[0071] In response to the viewing mode before the setting not being the immersive application mode, delete the first UI element in the three-dimensional space and / or add the second UI element in the three-dimensional space. In the immersive application mode, the three-dimensional space displays the UI elements of the foreground-running application and does not display the system UI elements.

[0072] In some embodiments, after the processor 101 implements obtaining at least one UI element corresponding to the set viewing mode, it is further configured to implement:

[0073] In response to the head-mounted display device being in the screen-off state and the second UI element including UI elements of a preset type, automatically control the head-mounted display device to switch from the screen-off state to the screen-on state and add the second UI element in the three-dimensional space;

[0074] In response to the time for displaying the second UI element reaching a preset time, automatically control the head-mounted display device to switch from the screen-on state to the screen-off state and delete the second UI element in the three-dimensional space.

[0075] In some embodiments, when the processor 101 adjusts the UI elements of the user interface (UI) displayed in the three-dimensional space when implementing the visual field mode after the setting, it is configured to:

[0076] In response to the visual field mode before the setting being the immersive application mode, block the UI elements of the application running in the foreground in the three-dimensional space and display the UI elements corresponding to the visual field mode after the setting in the three-dimensional space. In the immersive application mode, the UI elements of the application running in the foreground are displayed in the three-dimensional space, and the system UI elements are not displayed.

[0077] In some embodiments, when the processor 101 blocks the UI elements of the application in the three-dimensional space, it is configured to:

[0078] Display a mask UI element in the area where the UI elements of the application are displayed in the three-dimensional space to block the UI elements of the application in the three-dimensional space, where the level of the mask UI element is lower than the level of the UI elements corresponding to the visual field mode after the setting.

[0079] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the head-mounted display device described above can refer to the corresponding process in the foregoing embodiment of the interface control method, and will not be elaborated herein.

[0080] The embodiment of the present invention further provides a storage medium for computer-readable storage. The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any interface control method provided in the specification of the embodiment of the present invention.

[0081] Wherein, the storage medium may be an internal storage unit of the head-mounted display device described in the foregoing embodiment, such as the hard disk or memory of the head-mounted display device. The storage medium may also be an external storage device of the head-mounted display device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the head-mounted display device.

[0082] Those of ordinary skill in the art will understand that all or some of the steps in the methods disclosed above, and the functional modules / units in systems and devices, can be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division of functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be executed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0083] It should be understood that the term "and / or" as used in the specification and appended claims of the present invention refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. It should be noted that, in this context, the terms "comprises", "comprising", or any other variation thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or system. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article, or system comprising that element.

[0084] The serial numbers of the embodiments of the present invention above are only for description and do not represent the advantages or disadvantages of the embodiments. The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An interface control method, characterized in that: Applied to a head mounted display device, the method comprises: Get the name of the top Activity and the top window on the stack; According to the name of the top Activity and the name of the top window, set the field of view mode of the head mounted display device; In response to the field of view mode before setting being different from the field of view mode after setting, the user interface UI elements displayed in the three-dimensional space are adjusted according to the field of view mode after setting.

2. The interface control method according to claim 1, characterized in that: The step of setting the field of view mode of the head mounted display device according to the top Activity name and the top Window name of the stack includes: Determine a field of view mode identifier according to the name of the top Activity and the name of the top window, where the field of view mode identifier is used to identify the field of view mode; The field of view mode identifier is written into a database to set the field of view mode of the head mounted display device.

3. The interface control method according to claim 1, characterized in that: The method of obtaining the top Activity name and the top Window name includes: The name of the top-of-stack Activity is obtained through the preset top-of-stack Activity collection interface, and the name of the top-of-stack window is obtained through the preset top-of-stack window collection interface.

4. The interface control method according to any one of claims 1 to 3, characterized in that: The step of adjusting the user interface UI elements displayed in the three-dimensional space according to the set field of view mode includes: Acquire at least one UI element corresponding to the set field of view mode, where the at least one UI element includes a first UI element and / or a second UI element, where the first UI element is a UI element that has been displayed in the three-dimensional space, and the second UI element is a UI element that is not displayed in the three-dimensional space; The first UI element in the three-dimensional space is deleted and / or the second UI element is added in the three-dimensional space.

5. The interface control method according to claim 4, characterized in that: The deleting the first UI element in the three-dimensional space and / or adding the second UI element in the three-dimensional space includes: In response to the field of view mode before the setting being not the immersive application mode, the first UI element in the three-dimensional space is deleted and / or the second UI element is added in the three-dimensional space. In the immersive application mode, the three-dimensional space displays the UI elements of the application running in the foreground, and does not display the system UI elements.

6. The interface control method according to claim 4, characterized in that: After obtaining at least one UI element corresponding to the set field of view mode, the method further includes: In response to the head mounted display device being in a screen-off state and the second UI element including a UI element of a preset type, automatically controlling the head mounted display device to switch from the screen-off state to the screen-on state and adding the second UI element in the three-dimensional space; In response to the time for displaying the second UI element reaching a preset time, the head-mounted display device is automatically controlled to switch from the screen-on state to the screen-off state and the second UI element in the three-dimensional space is deleted.

7. The interface control method according to any one of claims 1 to 3, characterized in that: The step of adjusting the user interface UI elements displayed in the three-dimensional space according to the set field of view mode includes: In response to the field of view mode before the setting being the immersive application mode, the UI elements of the application running in the foreground in the three-dimensional space are blocked, and the UI elements corresponding to the field of view mode after the setting are displayed in the three-dimensional space. In the immersive application mode, the three-dimensional space displays the UI elements of the application running in the foreground, and does not display the system UI elements.

8. The interface control method according to claim 7, characterized in that: The blocking of the UI element of the application in the three-dimensional space includes: A mask UI element is displayed in the area where the UI elements of the application are displayed in the three-dimensional space to cover the UI elements of the application in the three-dimensional space, wherein the level of the mask UI element is lower than the level of the UI element corresponding to the set field of view mode.

9. A head mounted display device, characterized in that: The head-mounted display device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the interface control method according to any one of claims 1 to 8 is realized.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the interface control method according to any one of claims 1 to 8.

Images