A head-mounted device-based pop-up window application method and device and medium thereof

Abstract

This application discloses a floating window application method, device, and medium based on a head-mounted device, relating to the field of head-mounted device technology. It provides a floating window application method suitable for AR / VR products, addressing the current problem that AR / VR products cannot view or operate another application while one application is open. This method, based on a head-mounted device, determines the corresponding operation behavior for different user commands according to a preset command set. Thus, when the AR / VR product receives a user command via its built-in command acquisition module that triggers a floating window, the background application is displayed as a floating window in the current window, with the floating window's size smaller than the current window. This allows users to view or operate another application while keeping the current application open, further improving the user experience of AR / VR products.

Description

Technical Field

[0001] This application relates to the field of head-mounted device technology, and in particular to a method, device and medium for applying a floating window based on a head-mounted device. Background Technology

[0002] With the development of head-mounted devices, especially Augmented Reality (AR) / Virtual Reality (VR) products, it is now possible to support multiple applications running simultaneously. Typically, one application is displayed in the foreground while the others continue to run in the background, unnoticed by the user. Users cannot view or operate another application while another is open, thus impacting the user experience.

[0003] Therefore, those skilled in the art urgently need a floating window application method based on head-mounted devices to solve the problem that current head-mounted device products cannot view or operate one application while opening another. Summary of the Invention

[0004] The purpose of this application is to provide a floating window application method, device and medium based on head-mounted devices, and to provide a floating window application method suitable for head-mounted device products.

[0005] To address the aforementioned technical problems, this application provides a floating window application method based on a head-mounted device, comprising:

[0006] Obtain user commands and determine whether the user commands are commands in a preset command set; wherein, the preset command set includes: floating window call-up command and floating window exit command;

[0007] When the user command is a floating window call command, the image to be displayed by the background application will be displayed in the current window as a floating window; the size and position of the floating window are preset.

[0008] When the user command is to exit the floating window, close the floating window.

[0009] Preferably, when there are multiple background applications, the preset instruction set also includes: application selection instruction;

[0010] Correspondingly, displaying the image to be displayed by the background application as a floating window in the current window includes:

[0011] When the user command is an application selection command, the unique target application in the background application is determined based on the application selection command.

[0012] Display the target application's image as a floating window in the current window.

[0013] Preferably, the preset instruction set further includes: permission selection instructions;

[0014] Correspondingly, displaying the image to be displayed by the background application as a floating window in the current window includes:

[0015] When the user command is a permission selection command, the image to be displayed by the background application will be displayed in the current window as a floating window with the corresponding permissions, according to the permission selection command; among which, permissions include: viewing permissions and operation permissions.

[0016] Preferably, the preset instruction set further includes: application switching instructions;

[0017] Correspondingly, this method also includes:

[0018] When the user command is an application switching command, the image to be displayed in the background application is redefined according to the application switching command and displayed in the floating window.

[0019] Preferably, the preset instruction set also includes: permission switching instructions;

[0020] Correspondingly, this method also includes:

[0021] When the user command is a permission switching command, the permissions of the current floating window are switched to another permission.

[0022] Preferably, the preset instruction set also includes: a floating window scaling instruction;

[0023] Correspondingly, this method also includes:

[0024] When the user command is a floating window scaling command, the size of the floating window is adjusted according to the floating window scaling command.

[0025] Preferably, obtaining user instructions includes:

[0026] User commands are obtained through a gesture recognition module; the corresponding user commands are gesture commands.

[0027] To address the aforementioned technical problems, this application also provides a floating window application device based on a head-mounted device, comprising:

[0028] The instruction acquisition module is used to acquire user instructions and determine whether the user instructions are from a preset instruction set; wherein, the preset instruction set includes: floating window call-up instructions and floating window exit instructions;

[0029] The floating window call-out module is used to display the image to be displayed by the background application in the current window as a floating window when the user command is a floating window call-out command; the size and position of the floating window are preset;

[0030] The floating window exit module is used to close the floating window when the user command is a floating window exit command.

[0031] Preferably, it further includes:

[0032] The application selection module is used to determine a unique target application in the background applications based on the application selection instruction when the user command is an application selection instruction.

[0033] The permission selection module is used to display the image to be displayed by the background application in the current window as a floating window with the corresponding permissions when the user command is a permission selection command; the permissions include: view permissions and operation permissions.

[0034] The application switching module is used to redetermine the image to be displayed in the background application based on the user's application switching command, and then display it in the floating window.

[0035] The permission switching module is used to switch the permissions of the current floating window to another permission when the user command is a permission switching command.

[0036] The floating window scaling module is used to adjust the size of the floating window according to the user's command for scaling the floating window.

[0037] To address the aforementioned technical problems, this application also provides a floating window application device based on a head-mounted device, comprising:

[0038] Memory, used to store computer programs;

[0039] A processor is used to execute computer programs to implement the steps of the above-described floating window application method based on a head-mounted device.

[0040] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described floating window application method based on a head-mounted device.

[0041] This application provides a floating window application method based on a head-mounted device. It determines the corresponding operation behavior for different user commands based on a preset command set. This allows the head-mounted device to collect user commands through its built-in command acquisition module. When the user command is a floating window invocation command, the background application is displayed as a floating window in the current window, with the floating window's size smaller than the current window's size. This allows users to view or operate another application while keeping the current application open. This provides a floating window application method suitable for head-mounted devices, further improving the user experience when using head-mounted devices.

[0042] The floating window application device and computer-readable storage medium based on a head-mounted device provided in this application correspond to the above-described method and have the same effect. Attached Figure Description

[0043] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A flowchart of a floating window application method based on a head-mounted device provided by the present invention;

[0045] Figure 2 A flowchart illustrating how an AR device projects a real-world scene, as provided by this invention;

[0046] Figure 3 This invention provides a structural diagram of a floating window application device based on a head-mounted device;

[0047] Figure 4 This is a structural diagram of another floating window application device based on a head-mounted device provided by the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0049] The core of this application is to provide a method, device, and medium for applying a floating window based on a head-mounted device.

[0050] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] Currently, head-mounted devices, represented by AR / VR, are developing rapidly, with significant improvements in both hardware and software. However, they still lag behind mobile terminals such as smartphones, personal computers (PCs), and tablets. In certain application scenarios, such as viewing file downloads or displaying floating lyrics prompts, where users need to monitor one application while simultaneously paying attention to another, the floating functionality is particularly important.

[0052] For existing AR / VR devices, there are no applications with floating window functionality. To achieve this, the only way is to port the operating system from mobile terminals such as smartphones and PCs to the VR / AR device via software plugins. This would allow the AR / VR device's applications to be operated via floating windows by controlling the ported system. This method is extremely inconvenient and consumes a lot of the AR / VR device's processor and other hardware resources, affecting device performance and increasing costs.

[0053] For the reasons mentioned above, such as Figure 1 As shown, this application provides a floating window application method based on a head-mounted device, including:

[0054] S11: Obtain the user instruction and determine whether the user instruction is an instruction in the preset instruction set.

[0055] The preset instruction set includes: a floating window activation instruction and a floating window exit instruction. However, in practice, the instructions included in the above-mentioned preset instruction set are only one possible implementation. This application is not limited to the preset instruction set only including floating window activation and floating window exit instructions. In subsequent preferred embodiments, the preset instruction set may also include: an application selection instruction for selecting a specific background application to activate its floating window, a permission selection instruction for determining the floating window's permissions, an application switching instruction for switching the application currently displayed in the floating window, a permission switching instruction for switching the permissions of the current floating window, and a floating window scaling instruction for adjusting the size of the floating window, etc.

[0056] The method for acquiring user commands depends on the information acquisition module configured in the AR / VR device. Typically, due to the characteristics of AR / VR devices, they are equipped with a gesture recognition module to collect user gestures. Specifically, the gesture recognition module can be an image recognition module, collecting the user's motion image information, performing image recognition and parsing to extract the user's gestures, corresponding to different user commands, to perform corresponding operations on the AR / VR device. The information acquisition module can also be a voice recognition module, collecting the user's voice information to achieve corresponding control. Alternatively, the information acquisition module can be an ultrasonic detection module, detecting changes in the position of the user's hand or other body parts to determine the user's gestures, and then corresponding to different user commands to complete the operation of the AR / VR device. This application is not limited to the specific method of acquiring user commands, but provides a preferred embodiment in which acquiring user commands includes:

[0057] User commands are obtained through a gesture recognition module.

[0058] Correspondingly, user commands are gesture commands.

[0059] This embodiment uses the gesture recognition module, which is most commonly configured in AR / VR devices, to collect user commands without the need for additional hardware devices. It is also adapted to the characteristics of AR / VR devices, further improving the user experience.

[0060] S12: When the user command is a floating window call command, the image to be displayed by the background application will be displayed in the current window as a floating window.

[0061] The size and position of the floating window are preset. The floating window can be displayed in the current window at a preset size, which is smaller than the current window size. The size of the floating window can also be adjusted according to subsequent user commands. The display position of the floating window is generally set above or below the current display area.

[0062] However, in certain specific application scenarios, floating windows can also be overlaid on the real-world environment. In this case, the position and size of the floating window are determined by the actual environment. More specifically, for example, the floating window can be placed in an unobstructed area of ​​a real object, that is, an area that the user can see directly and without obstruction. Alternatively, the position of the floating window can be adjusted according to the user's gaze area. This can be achieved using eye trackers or other devices in AR / VR devices. Eye trackers track the user's eye movements to determine the user's current gaze area, and then project the floating window onto that gaze area, allowing the user to see the content and information displayed in the floating window directly.

[0063] Furthermore, if other windows exist within the user's current gaze area, the size and position of the floating window can be adjusted according to pre-configuration. For example, the floating window can be smaller than the user's entire gaze area and positioned slightly above or below it. Moreover, if other windows exist within the gaze area but do not completely cover it, the floating window can be displayed in a location not covered by other windows. This achieves the effect of allowing users to intuitively and conveniently view the content displayed in the floating window without affecting other aspects of their visual experience.

[0064] Furthermore, for AR devices, if the floating window displays a real-world scene (i.e., real-world scene information is captured and projected through a light engine or similar device), and this real-world scene has backup data in the cloud, the floating window content can be projected directly by downloading the cloud data, effectively reducing latency. This situation, where the real-world scene has backup data in the cloud, could mean that the real-world scene information being projected this time is unchanged from the last time it was captured, or it could mean that multiple devices are projecting the same scene. In this case, only one device needs to capture the real-world scene information and upload it to the cloud; other devices do not need to capture it themselves and can project the real-world scene using the cloud data. Specific application scenarios include... Figure 2 As shown, it includes: AR device 21, cloud server 22, user 23 and real scene 24.

[0065] S13: When the user command is to exit the floating window, close the floating window.

[0066] This application provides a floating window application method based on a head-mounted device. By comparing the collected user commands with a preset command set, the background application is displayed in the current window as a floating window, realizing the activation of the floating window and the closure of the current floating window under the corresponding user command. This is implemented in AR / VR devices, allowing users to view or operate other applications while the current application is open, further improving the user experience.

[0067] As can be seen from the above, the preset instruction set can also include other instructions to better realize the application of the hover function. Therefore, this embodiment provides a preferred implementation scheme, in which the preset instruction set further includes: applying a selected instruction;

[0068] Correspondingly, displaying the image to be displayed by the background application as a floating window in the current window includes:

[0069] When the user command is an application selection command, the unique target application in the background application is determined based on the application selection command.

[0070] Display the target application's image as a floating window in the current window.

[0071] It's easy to understand that in actual use of AR / VR devices, there isn't necessarily only one background application. Therefore, when you need to bring up the floating window of a background application, you need to select one or more background applications. Since the principle of selecting one background application and selecting multiple background applications is the same, the following explanation uses selecting one background application as an example:

[0072] One possible implementation is as follows: the selected instruction includes multiple sub-instructions, each corresponding to a different background application. When it is determined that the current user instruction is a certain sub-instruction, the corresponding background application floating window is invoked.

[0073] The advantage of this implementation is that it requires fewer user commands to bring up the floating window; only one corresponding subcommand needs to be detected to bring up the corresponding background application floating window.

[0074] Another possible implementation is: the application selection instruction includes at least one sub-instruction for switching the selected target, and a sub-instruction for determining the currently selected target as the target application.

[0075] When there is only one sub-command for switching the selected target, the background applications should be arranged in a preset order (either the order in which the applications were opened or the order in which they were last opened). Then, when the user command is the aforementioned sub-command for switching the selected target, the next background application is switched to the selected target in the order until the expected background application is switched. After that, the selected target can be selected by entering a sub-command to confirm that the current selected target is the target application, so that the floating window can be brought up later.

[0076] The advantage of this implementation is that the number of instructions for the corresponding selected target application is not limited by the number of background applications. Any number of background applications can be selected simply by using sub-instructions for switching the selected target and sub-instructions for determining the currently selected target as the target application.

[0077] In practical applications, the appropriate implementation method for application selection can be selected based on the number of background applications; this embodiment does not impose any limitations on this. Through the aforementioned application selection command, even when multiple background applications exist, the floating window of the target background application can still be accurately invoked for viewing or operation, further improving the user experience.

[0078] Similarly, this embodiment also provides a preferred implementation scheme, wherein the preset instruction set further includes: permission selection instructions;

[0079] Correspondingly, displaying the image to be displayed by the background application as a floating window in the current window includes:

[0080] When the user command is a permission selection command, the image to be displayed by the background application will be displayed in the current window as a floating window with the corresponding permissions, according to the permission selection command; among which, permissions include: viewing permissions and operation permissions.

[0081] Viewing permissions refer to the current floating window displaying normal content. Users can view the status of the corresponding application through the floating window, but cannot perform any operations on it. In other words, the application that invokes the floating window does not respond to the user's operation commands.

[0082] Operation permissions refer to the content currently displayed in the floating window and the corresponding user's operation commands. In addition to viewing the application status through the floating window, users can also perform corresponding operations on the application through operation commands.

[0083] As can be seen from the above, the permissions of the floating window can be divided into viewing permissions and operation permissions. Therefore, a preferred implementation is that the permission selection instruction includes two sub-instructions, which represent viewing permissions and operation permissions respectively. When the user's operation instruction is a sub-instruction of the corresponding permission, the corresponding permission is granted to the current floating window.

[0084] This embodiment provides a preferred solution that controls the permissions of the floating window through permission selection instructions in a preset instruction set. This allows users to select appropriate operation permissions when the floating window is invoked. For example, when playing a game, users typically do not want to be disturbed by other applications. If a user accidentally performs an operation due to the invoked floating window, it will greatly affect the user experience. Therefore, in this application scenario, the floating window's permission should be set to viewing permission, allowing users to only view the image displayed in the floating window but not to operate on it. Based on the same principle, in scenarios such as reading and watching movies, the floating window's permission can be set to operation permission, allowing users to not only view the image displayed in the floating window but also to operate on it. This preferred solution, by selecting floating window permissions, makes the floating window function better adaptable to users' personalized needs, further improving the user experience of AR / VR devices.

[0085] Similarly, as described above, although the above embodiments provide a method for selecting background applications when a floating window is brought up, this is a method during the process of bringing up the floating window. Even after the floating window has been brought up, the user still needs to switch the background applications displayed in the current floating window. Therefore, to solve the above problem, this embodiment also provides a preferred implementation scheme, where the preset instruction set further includes: application switching instructions;

[0086] Correspondingly, this method also includes:

[0087] When the user command is an application switching command, the image to be displayed in the background application is redefined according to the application switching command and displayed in the floating window.

[0088] It's easy to understand that the application switching command is essentially the same as the application selection command; both are used to select the desired target application from among many background applications. The difference lies in that the application selection command is used during the floating window's activation, while the application switching command is used after the floating window has already been activated. Therefore, when selecting a target application, the application switching command can also have the two possible implementations described above for the application selection command: the application switching command has multiple sub-commands corresponding one-to-one with the background applications, and the application switching command has at least one sub-command for switching the selected target and one sub-command for determining the currently selected target as the target application. Specific implementation methods have been described in the above embodiments and will not be repeated here.

[0089] This embodiment provides a preferred implementation method for switching applications displayed in a floating window when the floating window is invoked. Users can switch the background applications displayed in the floating window as needed, without having to close the current floating window and reopen the floating window of the desired background application, thus simplifying user operations.

[0090] Similarly, when a floating window is invoked, the user's permission requirements for it may change. For example, a floating window invoked while a user is playing a game typically requires viewing permissions, but when the user exits the game, they may be reading or playing music, requiring the user to interact with the floating window, thus necessitating permission switching. Therefore, this embodiment provides another preferred implementation, where the preset instruction set further includes: permission switching instructions;

[0091] Correspondingly, this method also includes:

[0092] When the user command is a permission switching command, the permissions of the current floating window are switched to another permission.

[0093] After the floating window is invoked, if the user command is a permission switching command, the current floating window's permission will be switched from view permission to operation permission; if the current floating window's permission is operation permission, it will be switched from view permission.

[0094] The preferred solution provided in this embodiment enables permission switching after the floating window is invoked, making the user's operation of the floating window more flexible. When it is necessary to switch the permissions of the floating window, there is no need to close the floating window and reopen it, which simplifies the operation process and optimizes the user experience of the floating function.

[0095] Furthermore, different users have different requirements for the size of the floating window of different applications at different times. Therefore, this embodiment also provides another preferred implementation scheme, and the preset instruction set also includes: floating window scaling instruction;

[0096] Correspondingly, this method also includes:

[0097] When the user command is a floating window scaling command, the size of the floating window is adjusted according to the floating window scaling command.

[0098] It's easy to understand that when a floating window is invoked, it is displayed in the current window at a preset size. The size of the floating window can then be adjusted according to the floating window switching command. For example, in one possible implementation, the user selects a window (in a VR device, the cursor can be placed on the corresponding window using a controller; in an AR device, the user can place their finger on the projected image at the location of the corresponding window), and drags their finger or cursor along the edge of the floating window. The size of the floating window then increases or decreases accordingly, thus achieving the scaling of the floating window.

[0099] It should be noted that the application switching command, permission switching command, and floating window scaling command are applied when the floating window has already been invoked. The specific steps should be implemented after the floating window invoke command and before the floating window exit command. There is no specific order among them, and they are triggered according to the user's command.

[0100] Similarly, the application selection command and permission selection command are performed during the floating window launch process. The implementation steps are included in the floating window launch process, so the command should be received after the floating window launch command. There is no order restriction between the application selection command and the permission selection command.

[0101] This embodiment provides a method for adjusting the size of a floating window after it is invoked, which further improves the flexibility of the floating window function based on AR / VR devices and also enhances the user experience.

[0102] In the above embodiments, a method for applying a floating window based on a head-mounted device has been described in detail. This application also provides an embodiment of a device for applying a floating window based on a head-mounted device. It should be noted that this application describes the device embodiment from two perspectives: one is based on functional modules, and the other is based on hardware.

[0103] From the perspective of functional modules, such as Figure 3 As shown, this embodiment provides a floating window application device based on a head-mounted device, including:

[0104] The instruction acquisition module 31 is used to acquire user instructions and determine whether the user instructions are instructions in a preset instruction set; wherein, the preset instruction set includes: floating window call-up instructions and floating window exit instructions;

[0105] The floating window call-out module 32 is used to display the image to be displayed in the background application as a floating window in the current window when the user command is a floating window call-out command; wherein, the size and position of the floating window are preset size and position;

[0106] The floating window exit module 33 is used to close the floating window when the user command is a floating window exit command.

[0107] Preferably, it further includes:

[0108] The application selection module is used to determine a unique target application in the background applications based on the application selection instruction when the user command is an application selection instruction.

[0109] The permission selection module is used to display the image to be displayed by the background application in the current window as a floating window with the corresponding permissions when the user command is a permission selection command; the permissions include: view permissions and operation permissions.

[0110] The application switching module is used to redetermine the image to be displayed in the background application based on the user's application switching command, and then display it in the floating window.

[0111] The permission switching module is used to switch the permissions of the current floating window to another permission when the user command is a permission switching command.

[0112] The floating window scaling module is used to adjust the size of the floating window according to the user's command for scaling the floating window.

[0113] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0114] This embodiment provides a floating window application device based on a head-mounted device. The device acquires user commands through an instruction acquisition module, compares the acquired user commands with a preset instruction set, and displays the background application as a floating window in the current window through a floating window call-up module, thus realizing the calling up of the floating window. Under the corresponding user command, the floating window can be closed through a floating window exit module. This enables users to view or operate other applications while the current application is open in AR / VR devices, further improving the user experience.

[0115] Figure 4A structural diagram of a floating window application device based on a head-mounted device is provided for another embodiment of this application, as shown below. Figure 4 As shown, a floating window application device based on a head-mounted device includes: a memory 40 for storing computer programs;

[0116] The processor 41 is used to execute computer programs to implement the steps of a floating window application method based on a head-mounted device as described in the above embodiments.

[0117] The floating window application device based on a head-mounted device provided in this embodiment may include, but is not limited to, VR headsets, AR interactive consoles, smartphones, tablets, laptops, or desktop computers.

[0118] The processor 41 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 41 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 41 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 41 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 41 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0119] The memory 40 may include one or more computer-readable storage media, which may be non-transitory. The memory 40 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 40 is used to store at least the following computer program 401, which, after being loaded and executed by the processor 41, is capable of implementing the relevant steps of a floating window application method based on a head-mounted device disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 40 may also include an operating system 402 and data 403, and the storage method may be temporary or permanent storage. The operating system 402 may include Windows, Unix, Linux, etc. The data 403 may include, but is not limited to, a floating window application method based on a head-mounted device.

[0120] In some embodiments, a floating window application device based on a head-mounted device may further include a display screen 42, an input / output interface 43, a communication interface 44, a power supply 45, and a communication bus 46.

[0121] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on a head-mounted device-based floating window application and may include more or fewer components than shown.

[0122] This application provides a floating window application device based on a head-mounted device, including a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: a floating window application method based on a head-mounted device.

[0123] This embodiment provides a floating window application device based on a head-mounted device. The processor executes a computer program stored in memory to acquire user commands and compares the acquired user commands with a preset command set to determine the operation to be performed. When the user command is a floating window activation command, the background application is displayed as a floating window in the current window, thus activating the floating window. When the user command is a floating window exit command, the current floating window is closed. This is implemented in AR / VR devices, allowing users to view or operate other applications while the current application is open, further improving the user experience.

[0124] Finally, this application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0125] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0126] This embodiment provides a computer-readable storage medium. When the computer program stored therein is executed, it can obtain user instructions and compare the user instructions to see which instruction in a preset instruction set it belongs to. The floating window call-up instruction is used to display the background application in the current window as a floating window, thereby realizing the call-up of the floating window. The floating window exit instruction is used to close the current floating window. In this way, in AR / VR devices, even when the current application is open, other applications can be viewed or operated, thereby further improving the user experience.

[0127] The above provides a detailed description of a floating window application method, apparatus, and medium based on a head-mounted device. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0128] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A method for applying a floating window based on a head-mounted device, characterized in that, Applied to head-mounted devices, including AR devices and VR devices; the method includes: Obtain user instructions and determine whether the user instructions are instructions from a preset instruction set; wherein, the preset instruction set includes: floating window call-up instructions, floating window exit instructions, permission selection instructions, permission switching instructions, application switching instructions, and floating window scaling instructions; when there are multiple background applications, the preset instruction set also includes: application selection instructions; When the user command is a floating window invocation command, the image to be displayed by the background application is displayed in the current window as a floating window; wherein, the size and position of the floating window are preset; the preset position includes: the unobstructed area of ​​the real object, or the user's gaze area; when there are other windows in the user's gaze area, the size of the floating window is smaller than the user's gaze area; when there are other windows in the user's gaze area and the window does not completely cover the user's gaze area, the floating window is displayed in a position in the user's gaze area that the window does not cover; when there are multiple background applications, displaying the image to be displayed by the background applications in the current window as a floating window includes: when the user command is an application selection command, determining the unique target application among the background applications according to the application selection command; displaying the image of the target application as the image to be displayed in the current window as a floating window; for the AR device, and when there are multiple AR devices, displaying the image to be displayed by the background applications in the current window as a floating window includes: downloading cloud data to project the content of the floating window; wherein, the cloud data is real scene information collected and uploaded to the cloud by any of the AR devices; When the user command is an exit command for the floating window, the floating window is closed; When the user instruction is the permission selection instruction, the image to be displayed by the background application is displayed in the current window as a floating window with the corresponding permission, according to the permission selection instruction; wherein, the permissions include: viewing permission and operation permission; When the user instruction is the permission switching instruction, the permission of the current floating window is switched to another permission. When the user instruction is the application switching instruction, the image to be displayed in the background application is re-determined according to the application switching instruction and displayed in the floating window; When the user instruction is the floating window scaling instruction, the size of the floating window is adjusted according to the floating window scaling instruction.

2. The floating window application method based on a head-mounted device according to claim 1, characterized in that, The user acquisition instructions include: The user command is obtained through the gesture recognition module, and the corresponding user command is a gesture command.

3. A floating window application device based on a head-mounted device, characterized in that, Applied to head-mounted devices, the head-mounted devices including: AR devices and VR devices; the device includes: The instruction acquisition module is used to acquire user instructions and determine whether the user instructions are instructions in a preset instruction set; wherein, the preset instruction set includes: floating window call-up instruction, floating window exit instruction, application switching instruction, and floating window scaling instruction; when there are multiple background applications, the preset instruction set also includes: application selection instruction; A floating window call-out module is used to display the image to be displayed by the background application in the current window as a floating window when the user command is the floating window call-out command; wherein, the size and position of the floating window are preset; the preset position includes: the unobstructed area of ​​the real object, or the user's gaze area; when there are other windows in the user's gaze area, the size of the floating window is smaller than the user's gaze area; when there are other windows in the user's gaze area and the window does not completely cover the user's gaze area, the floating window is displayed in the user's gaze area in a position not covered by the window; for the AR device, and when there are multiple AR devices, displaying the image to be displayed by the background application in the current window as a floating window includes: downloading cloud data to project the content of the floating window; wherein, the cloud data is real scene information collected and uploaded to the cloud by any of the AR devices; The floating window exit module is used to close the floating window when the user command is the floating window exit command; The application selection module is used to determine a unique target application in the background application according to the application selection instruction when the user instruction is the application selection instruction; and to display the image of the target application as an image to be displayed in the current window in the form of a floating window. The permission selection module is used to display the image to be displayed by the background application in the current window as a floating window with corresponding permissions when the user instruction is the permission selection instruction; wherein, the permissions include: viewing permissions and operation permissions; The application switching module is used to, when the user instruction is the application switching instruction, re-determine the image to be displayed in the background application according to the application switching instruction, and display it in the floating window; The permission switching module is used to switch the permissions of the current floating window to another permission when the user instruction is the permission switching instruction; The floating window scaling module is used to adjust the size of the floating window according to the floating window scaling instruction when the user instruction is the floating window scaling instruction.

4. A floating window application device based on a head-mounted device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the floating window application method based on a head-mounted device as described in claim 1 or 2 when executing the computer program.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the floating window application method based on a head-mounted device as described in claim 1 or 2.

Images