Method, apparatus, device and medium for controlling an application in an augmented reality space

Abstract

An embodiment of the present invention provides a method, apparatus, device, and medium for controlling an application in an augmented reality space, the method including: displaying a UI of the application and a first integrated control of the application in response to an instruction to open the application in the augmented reality space; the first integrated control being located outside the UI and integrating multiple interaction functions of the application; the first integrated control following the display of the UI; and controlling the UI to execute an interaction function corresponding to an operation on the first integrated control in response to an operation on the first integrated control. Integrating multiple interaction functions of the application into a single first integrated control outside the UI can simplify the UI of the application, facilitate user operation of controls on the UI, and improve the "sense of immersion" provided to the user by the XR device.

Description

[Technical Field]

[0001] This application claims priority to a Chinese invention patent application filed on November 8, 2022, entitled "Method, apparatus, device and medium for controlling applications in augmented reality space," with application number 2022113945776, the entire contents of which are incorporated herein by reference.

[0002] TECHNICAL FIELD Exemplary embodiments of the present invention relate to the technical field of electronic devices, and in particular to a method, apparatus, device and medium for controlling an application in an augmented reality space. [Background technology]

[0003] Extended reality (XR) refers to the creation of an interactive virtual environment by fusing reality and virtuality through computers. XR is a general term for a variety of technologies, including virtual reality (VR), augmented reality (AR), and mixed reality (MR). By combining the visual interaction technologies of these three, the experience provides an immersive experience, seamlessly transitioning between the virtual and real worlds.

[0004] Currently, on XR devices, many controls are displayed in the user interface (UI) of an application, such as close controls, minimize controls, zoom controls, and screen controls, and users use these controls to operate the application.

[0005] However, since the UI of an application has many controls, some of the controls may look unattractive and be inconvenient for users to operate. Summary of the Invention

[0006] Embodiments of the present invention provide a method, apparatus, device, and medium for controlling an application in an augmented reality space, which integrates multiple interactive functions of the application into a first integrated control outside the UI, thereby simplifying the UI of the application and facilitating user operation of the controls of the UI.

[0007] In a first aspect, an embodiment of the present invention provides a method for controlling an application in an augmented reality space, the method comprising: In response to an open command of an application in the augmented reality space, displaying a user interface UI of the application located outside the UI and a first integrated control of the application, the first integrated control integrating multiple interactive functions of the application and being displayed according to the UI; In response to an operation on the first integrated control, controlling the application to execute an interaction function corresponding to the operation.

[0008] In some embodiments, controlling the application to execute an interactive function corresponding to an operation on the first unified control in response to the operation includes: In response to a first operation on the first integrated control, minimizing the UI to a taskbar.

[0009] In some embodiments, the first operation functions as a click operation on the first integrated control.

[0010] In some embodiments, controlling an application to execute an interactive function corresponding to an operation on the first unified control in response to the operation includes: In response to a second operation on the first integrated control, movement of the UI is controlled.

[0011] In some embodiments, controlling the movement of the UI in response to a second operation on the first integrated control includes: Controlling the first integrated control to switch from a first state to a second state in response to a long press operation on the first integrated control; After the first unified control is switched to the second state, controlling movement of a UI in response to a drag operation on the first unified control.

[0012] In some embodiments, the second operation functions as a drag operation on the first integrated control.

[0013] In some embodiments, controlling the application to execute an interactive function corresponding to an operation on the first unified control in response to the operation includes: In response to detecting that the cursor moves within a first detection area of ​​the first integrated control and the hovering time length is equal to a first time length, a title bar is displayed at a preset position, the title bar includes a plurality of controls, and the first detection area covers at least the first integrated control.

[0014] In some embodiments, displaying the title bar at the preset position in response to detecting that the cursor moves into the first detection area of ​​the first integrated control and the hovering time length is equal to the first time length includes: In response to detecting that a cursor moves into the first detection area and a hovering time length is equal to a second time length, controlling the first integrated control to switch from a first state to a second state, the second time length being less than the first time length; and displaying the title bar at a preset position when detecting that the first integrated control is in the second state and the hovering time length is equal to the first time length.

[0015] In some embodiments, the method further comprises: The method further includes detecting that the cursor has moved out of the first detection area, and closing the title bar if the length of time the cursor has moved out is equal to a third length of time.

[0016] In some embodiments, the title bar is positioned around the first integrated control, and the method further comprises: The method further includes closing a title bar in response to detecting that the cursor has moved out of a second detection area for a length of time equal to a third length of time, the second detection area covering at least the first integrated control and the title bar.

[0017] In some embodiments, the first state is a down arrow and the second state is a horizontal bar.

[0018] In some embodiments, controlling the application to execute an interactive function corresponding to an operation on the first unified control in response to the operation includes: In response to a third operation on the first integrated control, closing the UI and the application.

[0019] In some embodiments, closing the UI and the application in response to a third operation on the first integrated control includes: In response to a third operation on the first integrated control, controlling the first integrated control to switch from a first state to a third state, the third state being used to characterize the application being turned off; and closing the UI and the application in response to the first unified control being in a third state for a fourth length of time.

[0020] In some embodiments, the third action functions as a double-click action.

[0021] In some embodiments, the first state is a down arrow and the third state is a fork.

[0022] In some embodiments, the first integrated control is a down arrow control.

[0023] In some embodiments, the UI does not include any controls and the first integrated control integrates functionality of existing controls in the application.

[0024] In some embodiments, a taskbar and a second integrated control located outside the taskbar are also displayed in the augmented reality space, the second integrated control integrating multiple interactive functions of the taskbar and displayed according to the taskbar, and the method further comprises: The method further includes controlling, in response to an operation on the second integrated control, the taskbar to execute an interactive function corresponding to the operation of the second integrated control.

[0025] In some embodiments, controlling the taskbar to perform an interactive function corresponding to the operation of the second integrated control in response to the operation of the second integrated control includes: and hiding the taskbar in response to a fourth operation on the second integrated control.

[0026] In some embodiments, the fourth operation functions as a click or double-click operation on the second unified control.

[0027] In some embodiments, the step of hiding the taskbar in response to a fourth operation of the second integrated control comprises: In response to detecting that the cursor moves into a third detection area of ​​the second integrated control and a hovering time length is equal to a fifth time length, the task bar is hidden, and the third detection area covers at least the second integrated control.

[0028] In some embodiments, controlling the taskbar to perform an interactive function corresponding to the operation of the second integrated control in response to the operation of the second integrated control includes: In response to a fifth operation of the second integrated control, controlling the movement of the taskbar or controlling the taskbar and a UI of an application running in the foreground to move together.

[0029] In some embodiments, controlling the movement of the taskbar or controlling the taskbar and a UI of an application executed in the foreground to move together in response to a fifth operation of the second unified control includes: Controlling the second integrated control to switch from a first state to a second state in response to a long press operation on the second integrated control; After the second integrated control is switched to the second state, in response to a drag operation on the second integrated control, controlling the movement of the taskbar or controlling the taskbar and a UI of an application running in the foreground to move together.

[0030] In some embodiments, the second integrated control is a down arrow control.

[0031] In some embodiments, the first state of the second integrated control is a down arrow and the second state of the second integrated control is a horizontal bar.

[0032] In some embodiments, the relative position of the first integrated control and the UI remains unchanged.

[0033] In some embodiments, the relative position of the second unified control and the taskbar remains unchanged.

[0034] On the other hand, an embodiment of the present invention provides a control device for an application in an augmented reality space, said device comprising: a display module that displays a user interface UI of the application located outside the UI and a first integrated control of the application in response to an open command of the application in the augmented reality space, the first integrated control being displayed according to the UI and integrating multiple interactive functions of the application; and a control module that controls the application to execute an interactive function corresponding to an operation on the first integrated control in response to the operation.

[0035] On the other hand, an embodiment of the present invention provides an electronic device comprising a processor for storing a computer program, a processor for calling and executing the computer program stored in the memory, and a memory for storing the computer program, in order to execute the method described in any one of the above.

[0036] Meanwhile, an embodiment of the present invention provides a computer-readable storage medium for storing a computer program, the computer program causing a computer to perform the method according to any one of the above claims.

[0037] On the other hand, an embodiment of the present invention provides a computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of the above claims.

[0038] An embodiment of the present invention provides a method, apparatus, device, and medium for controlling an application in an augmented reality space, the method including: displaying a UI of the application and a first integrated control of the application in response to an instruction to open the application in the augmented reality space; the first integrated control is located outside the UI and integrates multiple interactive functions of the application; the first integrated control is displayed according to the UI and, in response to an operation on the first integrated control, controls the first integrated control to perform an interactive function corresponding to the operation on the UI. Integrating multiple interaction functions of the application into the first integrated control outside the UI can simplify the UI of the application, facilitate user operation of the UI controls, and improve the "immersive feeling" that the XR device provides to the user. [Brief explanation of the drawings]

[0039] In order to more clearly describe the technical solutions in the embodiments of the present invention, the following briefly describes the drawings required in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without paying creative labor.

[0040] [Figure 1] 1 is a flow chart of a method for controlling an application in an augmented reality space provided by an embodiment of the present invention; [Figure 2] FIG. 1 is a schematic diagram of the UI and first integrated control of the app. [Figure 3] FIG. 10 is another schematic diagram of the UI and first integrated control of the app. [Figure 4] 10 is a flowchart of a method for controlling an application in an augmented reality space provided by embodiment 2 of the present invention. [Figure 5] 10 is a flowchart of a method for controlling an application in an augmented reality space provided by Embodiment 3 of the present invention. [Figure 6]FIG. 10 is a schematic diagram of the state change of the first integrated control of the app. [Figure 7] 10 is a flowchart of a method for controlling an application in an augmented reality space provided by embodiment 4 of the present invention. [Figure 8] FIG. 1 is a schematic diagram of a first detection area of ​​a first integrated control. [Figure 9] FIG. 1 is a schematic diagram of an interface transformation that invokes a title bar with a first integrated control. [Figure 10] 10 is a flowchart of a method for controlling an application in an augmented reality space provided by embodiment 5 of the present invention. [Figure 11] FIG. 10 is a schematic diagram of an interface transformation for closing an app via a first integrated control. [Figure 12] 10 is a flowchart of a method for controlling an application in an augmented reality space provided by embodiment 6 of the present invention. [Figure 13] This is a schematic diagram of an app and taskbar being displayed at the same time. [Figure 14] FIG. 13 is a schematic configuration diagram of a control device for an application in an augmented reality space provided by a seventh embodiment of the present invention. [Figure 15] FIG. 13 is a schematic configuration diagram of an electronic device provided according to an eighth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in combination with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and are not all embodiments. All other embodiments that can be obtained by those skilled in the art based on the embodiments of the present invention without paying creative labor belong to the protection scope of the present invention.

[0042] It should be noted that the terms "first," "second," and the like in the present specification and claims, as well as in the above-described drawings, are used to distinguish between similar items without necessarily describing a particular order or priority. It should be understood that such terms can be switched around as appropriate so that the embodiments of the present invention described herein can be performed in orders other than those shown or described. Additionally, the terms "comprises" and "having" and all variations thereof are intended to be non-exclusive inclusive. For example, a process, method, system, product, or server comprising a series of steps or units need not be limited to those explicitly recited steps or units, but may include other steps or units not explicitly recited or inherent to such process, method, product, or device.

[0043] The present invention is applied to the scenario of human-computer interaction. In the actual use process, a user always needs to interact with the virtual space provided by an electronic device. The virtual space can be understood as a virtual scene, which is a virtual scene displayed (or provided) when an application is executed on an electronic device. This virtual scene can be a simulation environment of the real world, a semi-simulated semi-fictional virtual scene, or a purely fictional virtual scene.

[0044] How to interact with this virtual space has become a current issue. Therefore, in this invention, we have designed a method for controlling applications in an augmented reality space to solve the problem of user-application interaction in virtual space.

[0045] To facilitate understanding of the embodiments of the present invention, before describing each embodiment, some concepts related to all embodiments will first be properly explained. Specifically, the following:

[0046] 1) Virtual reality (abbreviated as VR) is a technology that creates and experiences a virtual world. It sets up a virtual environment and uses multiple information sources (the virtual reality referred to in this text includes at least visual perception, but also auditory perception, tactile perception, movement perception, and even taste and smell perception) to integrate with the virtual environment, achieving interactive 3D dynamic vision and simulation of real actions, allowing the participant to immerse themselves in the simulated virtual reality environment, and realizing a variety of virtual environment applications such as maps, games, videos, education, medical care, simulation, joint training, sales, manufacturing cooperation, maintenance and repair, etc.

[0047] 2) Virtual reality equipment (VR devices) are terminals that realize virtual reality effects and are usually provided in the form of glasses, helmet-mounted displays (abbreviated as HMDs), or contact lenses, and can realize visual perception and other forms of perception. However, the forms that virtual reality equipment can realize are not limited to these, and they can be made even smaller or larger depending on actual needs.

[0048] As an option, the virtual reality devices described in the embodiments of the present invention may include, but are not limited to, several types:

[0049] 2.1) Computer-side virtual reality (PCVR) device: Uses the PC side to perform correlation calculations and data output for virtual reality functions, and the externally connected computer-side virtual reality device uses the data output by the PC side to realize the virtual reality effect.

[0050] 2.2) A mobile virtual reality device that supports the installation of a mobile terminal (such as a smartphone) in various ways (such as a head-mounted display with a dedicated card slot), and through a wired or wireless connection with the mobile terminal, performs correlation calculations for virtual reality functions through the mobile terminal, and outputs data to the mobile virtual reality device, such as watching virtual reality videos through the mobile terminal APP.

[0051] 2.3) The all-in-one virtual reality device is equipped with a processor for performing correlation calculations of virtual functions, and therefore has independent virtual reality input and output functions, and does not need to be connected to a PC or mobile device, allowing for high flexibility in use.

[0052] 3) Mixed reality (MR) refers to the fusion of real and virtual worlds to create new environments and visualizations, where physical and digital objects coexist, allowing for real-time interaction and the simulation of real objects. It combines real, augmented, virtual, and virtual reality technologies. MR is a hybrid of virtual reality (VR) and augmented reality (AR). It is an extension of virtual reality (VR) technology, enhancing the realism of the user experience by presenting virtual scenes within real scenes. The field of MR is related to computer vision, which is the science that studies how machines "see." Taking it a step further, cameras and computers perform machine vision, such as identifying, tracking, and measuring targets, instead of human eyes. Further image processing is then performed to create images that are more suitable for human observation or transmitted to devices for detection.

[0053] That is, MR is an analog set that integrates computer-generated sensory input (e.g., virtual objects) with sensory input from a physical set or representations thereof, and in some MR sets, the computer-generated sensory input can adapt to changes in the sensory input from the physical set. Furthermore, some electronic systems for presenting MR sets can monitor the orientation and / or position of virtual objects relative to a physical set so that they interact with real objects (i.e., physical elements from the physical set or representations thereof). For example, the system can monitor the movement of a virtual plant so that it appears stationary relative to a physical structure.

[0054] After describing some concepts related to the embodiments of the present invention, the following will specifically describe the application control method provided by the embodiments of the present invention in conjunction with the accompanying drawings, and the same content will not be described here as it is referred to in the description of the previous embodiments.

[0055] FIG. 1 is a flow diagram of an application control method in an augmented reality space provided by an embodiment of the present invention. The embodiment of the present invention may be applied to a human interaction scene executed by an application control device. The device may be configured with hardware and / or software and integrated into an electronic device. In an embodiment of the present invention, the electronic device may be any hardware device capable of providing an augmented reality space. For example, the electronic device may be an XR device such as a VR device, an AR device, or an MR device, but the present invention does not specifically limit this. Considering that the principles realized by different types of electronic devices are the same, for convenience of explanation, the following embodiment will be described in detail using an example in which the electronic device is a VR device.

[0056] As shown in FIG. 1, the method for controlling an application in an augmented reality space includes the following steps.

[0057] In S101, in response to an open command of the app in the augmented reality space, a UI of the app and a first integrated control of the app are displayed, and the first integrated control is located outside the UI, integrates multiple interaction functions of the UI, and is displayed according to the UI.

[0058] In the augmented reality space, a user opens an application (abbreviated as app). There are various ways to open the app. In one way, the user clicks the app icon on the home interface of the VR device to open the app. In another way, the app runs in the background, and the app icon appears in the interface taskbar. The user clicks the app icon on the taskbar to open the app.

[0059] When a user opens an app using any method, the app's UI and its first integrated controls are displayed in the augmented reality space, unlike existing technologies that are located outside of the app's UI and independent of the app's UI display. In prior technologies, app controls are displayed within the UI screen, for example, in the upper left, upper right, lower left, lower right, or intermediate positions at the bottom of the UI. Generally, controls are positioned so as not to obstruct the main UI interface and avoid obstructing the content of the main interface or interfering with the user's viewing or operation.

[0060] The first integrated control may be understood to be for operating the UI and integrating multiple interaction functions of the UI, or to trigger multiple interaction functions of the UI, including, but not limited to, minimizing the UI, turning off the app, controlling UI movement, controlling UI scaling, pausing, fast-forwarding, switching, turning off or adjusting the volume of content played in the UI, and controlling the screen turn-on of the app. Therefore, since the interaction functions of different apps are different, the functions of the first integrated control of different apps are considered to be different.

[0061] The first integrated control is displayed according to the UI display. In other words, the first integrated control is not displayed independently. Unlike virtual objects in the augmented reality space, the first integrated control can be displayed and operated independently. The first integrated control is displayed when the app's UI is displayed, and is not displayed when the app is closed or the UI is minimized.

[0062] The first integrated control differs from a conventional control panel in that a conventional control panel typically has multiple controls that allow users to control apps by manipulating these controls, or a conventional control panel does not have any controls but allows different controls through different areas on the panel, with each area having a different function and each area corresponding to one control. Overall, a conventional control panel is equivalent to having multiple operable points (each operating point corresponds to one control or area). In contrast, the first integrated control of the present embodiment does not have other controls, and different areas of the first integrated control cannot be operated, i.e., the first integrated control has only one operable point.

[0063] Optionally, the number of first integrated controls may be one or more. Illustratively, the UI has a first integrated control that integrates multiple interaction functions of the UI.

[0064] In one embodiment, the multiple interaction functions integrated into the first integrated control each correspond to a multiple number of controls, and each interaction function corresponds to an existing control. In this manner, common controls of the app are integrated, and the common controls of the UI include a minimize control, a maximize control, a shut down control, a pause control, and a volume adjustment control.

[0065] Alternatively, the app's UI may contain no controls, and the first integrated control may integrate the functionality of the app's existing controls.

[0066] In another embodiment, among the multiple interaction functions integrated into the first unified control, some of the interaction functions correspond to existing controls, and some of the interaction functions do not correspond to controls. For example, the UI movement function does not usually set a movement control, but rather realizes movement by dragging the UI border.

[0067] After integrating multiple interaction functions of an app into the first integrated control, different functions can be triggered through different operations of the first integrated control. For example, clicking the first integrated control minimizes the app's UI, i.e., closes the app's UI, and double-clicking the first integrated control closes the app. Alternatively, operating the first integrated control opens a new title bar, which is used to control the app. The first integrated control can be considered the app's first-level control, and the title bar can be considered the app's second-level control. The second-level control must be opened through the first-level control to control the app.

[0068] This embodiment is an arrow-type control, and the direction of the arrow can be down, up, left, or right, and is arbitrary. The first integrated control can be a circular control, a spherical control, a horizontal stripe control, etc.

[0069] In addition, in this embodiment, the position of the first integrated control is not limited, and the first integrated control may be located below or to the right of the UI. Optionally, the relative positions of the first integrated control and the UI may not change. For example, the relative positions of the center positions of the first integrated control and the UI may not change. When the first integrated control moves, the UI also moves, and after the movement, the relative positions of the two remain unchanged. As another example, the Z-axis coordinate points of the first integrated control and the UI in the virtual reality space may be the same, or the distance between the Z-axis coordinate points of the first integrated control and the UI may satisfy a fixed, preset relationship.

[0070] 2 is a schematic diagram of the UI and the first integrated control of the app. As shown in FIG. 2, the first integrated control is a downward arrow control located at the bottom center of the UI, and the arrow is an open arrow.

[0071] Figure 3 is another schematic diagram of the app's UI and the first integrated control. The difference between Figure 3 and Figure 2 is that the arrows are solid arrows.

[0072] The use of a down arrow control helps users understand the functionality of the primary integrated control. In the prior art, a down arrow control typically indicated that the control had a secondary menu or other functionality.

[0073] Since the first integrated control in Figures 2 and 3 is located at the bottom center of the UI, the arrow points downward, and when the first integrated control is located on the right side of the UI, the arrow may point to the right, that is, the direction of the arrow may change depending on the position of the first integrated control.

[0074] By integrating multiple interaction functions of an app into one control, the app's UI can be made more concise, the UI can be free of controls, and the UI can avoid occlusion of the main interface while enhancing the user's "immersion" feeling.

[0075] The method of this embodiment also facilitates user interaction with UI controls. Augmented reality space is typically a 3D space, and directly using user app operations in a 2D space can create difficulties for some user operations. For example, to shut down an app, the prior art required the user to move the cursor to the shutdown control in the upper left corner of the UI. However, due to the user's general usage habits, the cursor is located in the lower area of ​​the UI, requiring a long cursor movement distance. The method of this embodiment enables app shutdown by operating the first integrated control.

[0076] In S102, in response to an operation on the first integrated control, the app is controlled to execute an interaction function corresponding to the operation.

[0077] In this embodiment, the user operates the first unified control, including but not limited to operations such as clicking, long pressing, dragging, double clicking, and hovering over the first unified control.

[0078] In the XR device, the user can operate the first integrated control by a handheld device, gesture operation, voice control, gaze control, or other methods, and the present invention is not limited thereto.

[0079] In this embodiment, in response to an instruction to open an application in the augmented reality space, a UI of the application and a first integrated control of the application are displayed. The first integrated control is located outside the UI and integrates multiple interaction functions of the application. In response to an operation on the first integrated control, the UI is controlled to execute an interaction function corresponding to the operation. Integrating multiple interaction functions of the application into the first integrated control outside the UI makes the UI of the application simpler, facilitating user operation of the UI controls, and improving the "immersive feeling" that the XR device provides to the user.

[0080] Based on the first embodiment, the second embodiment of the present invention provides a method for controlling an application in an augmented reality space to store the UI of the app, and Fig. 4 is a flowchart of the method for controlling an application in an augmented reality space provided by the second embodiment of the present invention. As shown in Fig. 4, the method provided by this embodiment includes the following steps:

[0081] In S201, in response to an open command of the app in the augmented reality space, a UI of the app and a first integrated control of the app are displayed, and the first integrated control is located outside the UI, integrates multiple interaction functions of the app, and is displayed according to the UI.

[0082] In S202, in response to a first operation on the first integrated control, the UI is minimized to the taskbar.

[0083] The augmented reality space provided by VR devices displays not only the app's UI and integrated controls, but also the taskbar. The taskbar is a shortcut window for displaying and switching tasks. You can display and switch between running apps, or launch programs by clicking the app's icon on the taskbar. Clicking the app's minimize control minimizes the UI to the taskbar. The taskbar is similar to the taskbar in Windows systems or the dock bar in Apple systems.

[0084] For example, the first operation may be a click operation on the first integrated control, in which the user clicks the first integrated control to minimize the UI on the taskbar. After minimizing the UI on the taskbar, the user can reopen the app icon by clicking the app icon on the taskbar. After reopening the app, the UI of the app and the first integrated control can be displayed on the screen.

[0085] In the past, the minimize control for an app's UI was usually located in the upper right corner of the app, requiring the user to move the cursor to the upper right corner. The UI contains many controls, which can lead to false triggers and require a long cursor movement. In this embodiment, after the user clicks the first integrated control, the app's UI is minimized to the taskbar, facilitating user operation.

[0086] Optionally, to increase interest, you can animate the minimization process when minimizing the UI to the taskbar, giving the user a better experience.

[0087] In this embodiment, in response to an open command for an app in the augmented reality space, the UI of the app and a first integrated control for the app are displayed, the first integrated control is located outside the UI and integrates multiple interaction functions of the app, and in response to a first operation on the first integrated control, the UI is minimized on the taskbar, this first operation is a click operation, and clicking the first integrated control minimizes the UI of the app, making it easier for the user to operate.

[0088] Based on embodiment 1, embodiment 3 of the present invention provides a method for controlling an application in an augmented reality space, which controls movement of a UI in response to a second operation on a first integrated control. FIG. 5 is a flowchart of the method for controlling an application in an augmented reality space provided in embodiment 3 of the present invention. As shown in FIG. 5, the method provided in this embodiment includes the following steps:

[0089] In S301, in response to an open command of the app in the augmented reality space, a UI of the app and a first integrated control of the app are displayed, and the first integrated control is located outside the UI, integrates multiple interaction functions of the app, and is displayed according to the UI.

[0090] In S302, in response to a long press operation on the first unified control, the first unified control is controlled to switch from the first state to the second state.

[0091] The first state is the default state of the first integrated control. For example, the first state is a downward arrow type, and the second state is a horizontal bar type. FIG. 6 is a schematic diagram showing the state changes of the first integrated control of an app. As shown in FIG. 6, in the normal state, the first integrated control is a downward arrow type as shown in FIG. 6(a). When a user presses and holds the first integrated control, it changes to a horizontal bar type as shown in FIG. 6(b). After the first integrated control is switched to the second state, the second state can be used to prompt the user that the first integrated control can be dragged.

[0092] In S303, after the first unified control is switched to the second state, movement of the UI is controlled in response to a drag operation on the first unified control.

[0093] After the first integrated control is switched to the second state, the user drags the first integrated control and controls the movement of the UI through the movement of the first integrated control. Taking the control of a VR device via a steering wheel as an example, the user presses and holds the trigger key on the steering wheel to trigger the first integrated control to switch from the first state to the second state. At this time, the user continues to press and hold the trigger key to drag the first integrated control, and the UI moves along with the movement of the first integrated control, that is, the first integrated control and the UI move together.

[0094] In one embodiment, the movement distance, movement direction, and movement speed of the UI and the first integrated control are the same.

[0095] In another embodiment, at least one of the movement distance, movement direction, and movement speed of the UI and the first unified control is different. For example, the movement speed and movement direction of the UI and the first unified control are the same, but the movement distances of the UI and the first unified control are different, and the movement distance of the UI is n times the movement distance of the first unified control, where n is greater than 1.

[0096] Optionally, in some embodiments of the present invention, in response to a press and hold operation on the first integrated control, the state of the first integrated control does not change, and after the press and hold operation ends, when the user drags the first integrated control, it controls movement of the UI.

[0097] In another embodiment of the present invention, the movement of the UI is controlled in response to a drag operation on the first integrated control.

[0098] In this embodiment, in response to a long press operation on the first integrated control, the first integrated control is switched from a first state to a second state, and after the first integrated control is switched to the second state, movement of the UI is controlled in response to a drag operation on the first integrated control. Conventionally, movement of the UI has been achieved by dragging the frame of the UI, but in the method of this embodiment, movement of the UI is controlled by a long press operation on the first integrated control, which has a small area relative to the UI and is relatively easy to operate compared to the UI, and then dragging the first integrated control.

[0099] Based on the first embodiment, the fourth embodiment of the present invention provides a method for controlling an application in an augmented reality space, in which a title bar is called by a hover first integrated control and the application can be controlled by the title bar. Figure 7 is a flowchart of the method for controlling an application in an augmented reality space provided by the fourth embodiment of the present invention. As shown in Figure 7, the method provided by this embodiment includes the following steps:

[0100] In S401, in response to an open command of the app in the augmented reality space, a UI of the app and a first integrated control of the app are displayed, and the first integrated control is located outside the UI, integrates multiple interaction functions of the app, and is displayed according to the UI.

[0101] At S402, in response to detecting that the cursor moves into a first detection area of ​​a first integrated control and the hovering time length is equal to a first time length, a title bar is displayed at a preset position, the title bar including a plurality of controls, and the first detection area covers at least the first integrated control.

[0102] This step utilizes the hover function of the VR device. Hover usually refers to the state where the cursor is floating over an element. Hover can be used for highlight hints, floating explanatory frames, sub-navigation, etc. Some buttons change state after being hovered. A button box is usually displayed or highlighted to draw the user's attention to the feature and let them know that the cursor is in the clickable area of ​​the feature. In particular, for characters that do not have a strong click feeling, hover can inform the user that they can be operated. In this embodiment, the hover function is used to call up the title bar.

[0103] In the virtual reality space, since the user cannot touch 2D or 3D virtual objects, a ray can be emitted from the center of the user's palm, or from the hand controller of the XR device, and this ray can be regarded as an extension of the hand. A ring cursor is connected to the end of the ray to indicate the position where the ray intersects with the target object, and the target object pointed by the ring cursor can receive commands.

[0104] In this embodiment, when the hovering time length of the cursor in the first detection area of ​​the first integrated control is equal to the first time length, a title bar of a size that covers at least the position of the first integrated control is displayed at a preset position. The shape of this first detection area may be a rectangular area, a circular area, or other shapes, but this embodiment is not limited thereto. Figure 8 is a schematic diagram of the first detection area of ​​the first integrated control, and the first detection area shown in Figure 8 is a rectangular area.

[0105] The title bar contains a number of controls that allow interaction with the app, including, but not limited to, a minimize control, a close control, a maximize control, a help control, a projection control, a volume control, a fast forward control, etc. The title bar controls relate to the functionality of the app.

[0106] The position of the title bar may be preset around the first integrated control, for example, the title bar may be located above, below, or to the right of the first integrated control, and the position of the title bar is not limited in this embodiment.

[0107] In one embodiment, in response to detecting that a cursor moves into the first detection area and that a hovering time length is equal to a second time length, the first integrated control is controlled to switch from a first state to a second state, where the second time length is less than the first time length, and when the first integrated control is in the second state and that a hovering time length is equal to the first time length, a title bar is displayed at a preset position.

[0108] After the first integrated control is switched from the first state to the second state, the second state can be used to prompt the user to hover, and the function corresponding to the hover will be triggered shortly. Illustratively, the first state of the first integrated control is a down arrow type, and the second state is a horizontal bar type.

[0109] Figure 9 is a schematic diagram of an interface transformation that invokes a title bar using the first integrated control. Assume that the first time length is 1 second and the second time length is 0.6 seconds. When the cursor moves into the first detection area, the UI displays the first integrated control as a downward arrow, as shown in Figure 9(a). When the cursor moves into the first detection area and detects that the hovering time length is equal to 0.6 seconds, the control first integrated control displays the title bar in the first integrated control, as shown in Figure 9(b). When the cursor hovering time in the first detection area is equal to 1 second, the control first integrated control switches from a downward arrow to a horizontal bar, as shown in Figure 9(c).

[0110] The title bar shown in FIG. 9(c) has four controls, which are, from left to right, a minimize control, a close control, a maximize control, and a projection screen control.

[0111] In another embodiment, in response to detecting that the cursor has moved into the first detection area and the hovering time length is equal to the first time length, the title bar is displayed at a predetermined position, and in this manner, the state of the first integrated control does not change.

[0112] Optionally, in some embodiments, the first integrated control is hidden after displaying the title bar in a preset position.

[0113] In S403, if the cursor moves out of the first detection area and the length of time that the cursor has moved out is equal to the third length of time, the title bar is closed.

[0114] If the first integrated control was switched from the first state to the second state when the title bar was invoked, the first integrated control is switched from the second state to the first state when the title bar is closed.

[0115] In this embodiment, to avoid user error, a third time period is set, and the title bar cannot be closed unless the time period during which the cursor moves outside the first detection area reaches the third time period. If it is detected that the cursor has moved outside the first detection area but the time period is shorter than the third time period, the title bar will not be closed. For example, if it is detected that the cursor has moved outside the first detection area and then back into the first detection area within the third time period, the title bar will not be closed.

[0116] There is no necessary relationship in magnitude between the first and third durations, and they may be the same or different.

[0117] Optionally, in some embodiments of the present invention, in response to detecting that the cursor has moved out of the second detection area for a length of time equal to a third length of time, the title bar is closed, where the second detection area covers at least the first integrated control and the title bar, where the title bar is around the first integrated control, e.g., the title bar is above the first integrated control and both are adjacent to each other.

[0118] Correspondingly, if the cursor is moved out of the second detection area and detected to have moved into the second detection area within a third time period, the title bar will not close.

[0119] In this embodiment, in response to detecting that the cursor moves into a first detection area of ​​a first integrated control and the hovering time length is equal to a first time length, a title bar is displayed at a preset position including a plurality of controls, and in response to detecting that the first detection area covers at least the first integrated control, the cursor moves out of the first detection area and the time length of the cursor movement out is equal to a third time length, the title bar is closed. By hovering over the first integrated control, the title bar of an application can be called up, the application can be controlled through the title bar, and the title bar can be closed when the control is completed.

[0120] Based on the first embodiment, the fifth embodiment of the present invention provides a method for controlling an application in an augmented reality space to close the application. Fig. 10 is a flowchart of the method for controlling an application in an augmented reality space provided by the fifth embodiment of the present invention. As shown in Fig. 10, the method provided by this embodiment includes the following steps:

[0121] In S501, in response to an open command of the app in the augmented reality space, a UI of the app and a first integrated control of the app are displayed, and the first integrated control is located outside the UI, integrates multiple interaction functions of the app, and is displayed according to the UI.

[0122] In S502, in response to a third operation on the first integrated control, the UI and the app are closed.

[0123] As a selection mode, the third operation is a double-click operation, and by double-clicking the first integrated control, the application is shut down.

[0124] As a mode of selection, in response to a third operation on the first integrated control, the first integrated control is controlled to switch from the first state to a third state, and the third state is used to characterize the time that the first integrated control is in the third state reaching a fourth time length, thereby closing the UI and the app.

[0125] For example, the first state is a downward arrow style, and the third state is a fork style. Figure 11 is a schematic diagram of an interface transformation for closing an app using the first integrated control. Figure 11(a) shows the state of the first integrated control when the app is operating normally. When a user double-clicks the first integrated control, the interface in the augmented reality space transforms to Figure 11(b), and then, after a fourth time, the app closes.

[0126] As an option, to increase interest, when closing an app, you can animate the closing process to bring a better experience to users.

[0127] In this embodiment, in response to an open command for an app in the augmented reality space, the UI of the app and a first integrated control for the app are displayed, the first integrated control is located outside the UI and integrates multiple interaction functions of the app, and in response to a third operation on the first integrated control, the UI and the app are closed, and the third operation can quickly and conveniently close the app.

[0128] Embodiment 6 of the present invention provides a method for controlling an application in an augmented reality space to control a taskbar. Figure 12 is a flowchart of the method for controlling an application in an augmented reality space provided by embodiment 6 of the present invention. As shown in Figure 12, the method provided by this embodiment includes the following steps:

[0129] In S601, a taskbar and a second integrated control of the taskbar are displayed in the augmented reality space, and the second integrated control is located outside the taskbar, integrates multiple interaction functions of the taskbar, and is displayed according to the taskbar.

[0130] The taskbar is an application or function of the XR device itself. When the XR device starts up, the taskbar is displayed by default and is usually located at the bottom of the screen.

[0131] The second integrated control is for operating the taskbar, and the second integrated control integrates multiple interaction functions of the taskbar, and the interaction functions include, but are not limited to, hiding (which may be understood as closing), moving, and calling the taskbar.

[0132] The second integrated control is displayed according to the taskbar. In other words, the second integrated control is not displayed independently. Unlike virtual objects in the augmented reality space, the second integrated control can be displayed and operated independently. The second integrated control is displayed when the taskbar is displayed. When the taskbar is not displayed, the first integrated control is not displayed.

[0133] This embodiment does not limit the specific form of the second integrated control. For example, the second integrated control may be an arrow control, and the arrow direction may be down, up, left, or right. Optionally, the second integrated control may be a circular control, a spherical control, a horizontal bar control, etc.

[0134] Furthermore, this embodiment does not limit the position of the second integrated control, which may be located below or to the right of the taskbar, etc. The relative position of the second integrated control and the taskbar does not change. For example, the relative position of the center of the second integrated control and the center of the taskbar does not change, and when the second integrated control moves, the taskbar also moves, and after the movement, the relative position of the two remains unchanged. As another example, the Z-axis coordinate points of the second integrated control and the taskbar in the virtual reality space satisfy a predetermined relationship, for example, the Z-axis coordinate points of both are the same or the distance between the Z-axis coordinate points of both is fixed.

[0135] The shapes of the second integrated control and the first integrated control may be the same or different. Figure 13 is a schematic diagram showing an app and a taskbar displayed simultaneously. As shown in Figure 13, the first integrated control and the second integrated control have the same shape, both of which are downward arrows, but their functions are different.

[0136] In particular, the second integrated control is different from the conventional control panel, while the first integrated control has only one operable point, and the function of the second integrated control is similar to that of the first integrated control, so the relevant description should refer to the first integrated control and will not be described here.

[0137] In S602, in response to an open command for the app in the augmented reality space, a UI for the app and a first integrated control for the app are displayed.

[0138] In S603, in response to the operation on the second integrated control, the taskbar is controlled to execute an interaction function corresponding to the operation on the second integrated control.

[0139] It is understood that steps S602 and S603 do not change priority when executed depending on the order in which they are triggered by the user.

[0140] One interactive method includes hiding the taskbar in response to a fourth operation on the second integrated control, illustratively serving as a click, double-click, or hover operation on the second integrated control.

[0141] When the taskbar is hidden by a hover operation, the taskbar is hidden in response to detecting that the cursor moves into a third detection area of ​​the second integrated control and the hovering time length is equal to a fifth time length, and the third detection area covers at least the second integrated control.

[0142] The third detection area may be rectangular or have another shape, and the size of the third detection area covers at least the second integrated control.

[0143] Optionally, when the cursor moves into the third detection area and detects that the hovering time length is equal to a sixth time length, the second integrated control switches from the first state to the second state; when the hovering time length is equal to a fifth time length, the taskbar is hidden. The sixth time length is less than the fifth time length, and the second state is used to prompt the user to activate the hover function. Illustratively, the first state of the second integrated control is a downward arrow, and the second state is a horizontal bar.

[0144] In another interaction manner, the movement of the taskbar is controlled in response to a fifth operation of the second unified control. Illustratively, the fifth operation may be a drag operation on the second unified control, such that when the second unified control is dragged, the taskbar moves along with the second unified control.

[0145] Alternatively, in response to a long press operation on the second integrated control, the second integrated control is switched from a first state to a second state, and after the second integrated control is switched to the second state, movement of the taskbar is controlled in response to a drag operation on the second integrated control or the taskbar.

[0146] In this interaction, the taskbar and app UI are moved independently.

[0147] In another interaction mode, in response to a fifth operation of the second unified control, the taskbar and the UI of the app running in the foreground are controlled to move together. Illustratively, the fifth operation may be a drag operation on the second unified control, whereby the taskbar and the UI of the app running in the foreground move together in accordance with the second unified control when the second unified control is dragged.

[0148] Alternatively, in response to a long press operation on the second integrated control, the second integrated control is switched from a first state to a second state, and after the second integrated control is switched to the second state, in response to a drag operation on the second integrated control or the taskbar, the taskbar and the UI of the app running in the foreground are controlled to move together.

[0149] In this interaction, the drag operation to the second integrated control is a higher-level control operation than the drag operation to the first integrated control, and the drag operation to the second integrated control allows the taskbar and the app's UI to move together.

[0150] In this interaction mode, when the second unified control is dragged, an app that is not running in the foreground (i.e., an app that is not open) or an open app that is minimized to the taskbar and running in the background in the augmented reality space moves, and when the second unified control is dragged, if there is an app running in the foreground in the augmented reality space, the taskbar moves along with the UI of the app running in the foreground.

[0151] The principles for realizing the taskbar moving operation and the principles for realizing the taskbar and app UI moving together are the same as the principles for realizing the app UI moving operation in the above-mentioned embodiment, and will not be described further here.

[0152] In this embodiment, a taskbar and a second integrated control for the taskbar are displayed in the augmented reality space, and the second integrated control is located outside the taskbar and integrates multiple interaction functions of the taskbar. In response to an operation on the second integrated control, the taskbar is controlled to perform an interaction function corresponding to the operation on the second integrated control, thereby facilitating the user's operation of the taskbar.

[0153] Shrinking and enlarging the UI of an app is a common interaction function during interaction with the app. When the first integrated control is in any state, the user can shrink and enlarge the UI of the app using the handle of the XR device. Alternatively, the user can long press the first integrated control to switch the first integrated control from the first state to the second state, and then control the shrinking and enlarging of the UI of the app using the handle. It should be understood that the user can also control the shrinking or enlarging of the UI of an application in other ways, such as by voice or gesture.

[0154] In order to more easily implement the method for controlling an application in an augmented reality space according to the embodiment of the present invention, the embodiment of the present invention further provides a control device for an application in an augmented reality space. Figure 14 is a schematic diagram of the configuration of the control device for an application in an augmented reality space provided by the seventh embodiment of the present invention. As shown in Figure 14, the control device 100 for an application in an augmented reality space includes: a display module 11 that displays a user interface UI and a first integrated control of the application in response to an open command of the application in the augmented reality space, the first integrated control being located outside the UI, integrating multiple interaction functions of the application, and being displayed according to the UI; and a control module 12 that controls the application to execute an interaction function corresponding to an operation on the first integrated control in response to the operation.

[0155] In some embodiments, the control module 12 specifically: In response to a first operation on the first integrated control, the UI is minimized to a taskbar.

[0156] In some embodiments, the first operation functions as a click operation on the first integrated control.

[0157] In some embodiments, the control module 12 specifically: In response to a second operation on the first integrated control, movement of the UI is controlled.

[0158] In some embodiments, the control module 12 specifically: Controlling the first integrated control to switch from a first state to a second state in response to a long press operation on the first integrated control; After the first unified control is switched to the second state, the first unified control is used to control movement of the UI in response to a drag operation on the first unified control.

[0159] In some embodiments, the second operation functions as a drag operation on the first integrated control.

[0160] In some embodiments, the control module 12 specifically: In response to detecting that the cursor moves into a first detection area of ​​the first integrated control and the hovering time length is equal to a first time length, a title bar including a plurality of controls is displayed at a preset position, and the first detection area is used to cover at least the first integrated control.

[0161] In some embodiments, the control module 12 specifically: in response to detecting that a cursor moves into the first detection area and a hovering time length is equal to a second time length, controlling the first integrated control to switch from a first state to a second state, the second time length being less than the first time length; When the first integrated control is in the second state and it is detected that the hovering time length is equal to the first time length, it is used to display the title bar at a preset position.

[0162] In some embodiments, the control module 12 further comprises: It is used to detect that the cursor has moved out of the first detection area, and close the title bar if the length of time that the cursor has moved out is equal to a third length of time.

[0163] In some embodiments, the title bar is positioned around the first integrated control, and the control module 12 further: In response to detecting that the cursor has moved out of a second detection area for a time length equal to a third time length, the title bar is closed, and the second detection area is used to cover at least the first integrated control and the title bar.

[0164] In some embodiments, the first state is a down arrow and the second state is a horizontal bar.

[0165] In some embodiments, the control module 12 specifically: In response to a third operation on the first integrated control, the UI and the application are closed.

[0166] In some embodiments, the control module 12 specifically: In response to a third operation on the first integrated control, controlling the first integrated control to switch from a first state to a third state, the third state characterizing an application being turned off; In response to the first integrated control being in the third state for a fourth time, the UI and the application are used to close.

[0167] In some embodiments, the third action functions as a double-click action.

[0168] In some embodiments, the first state is a down arrow and the third state is a fork.

[0169] In some embodiments, the first integrated control is a down arrow control.

[0170] In some embodiments, the UI does not include any controls and the first integrated control integrates functionality of existing controls in the application.

[0171] In some embodiments, a taskbar and a second integrated control for the taskbar are also displayed in the augmented reality space, the second integrated control being located outside the taskbar and integrating multiple interaction functions of the taskbar, and the control module 12 further: In response to an operation on the second integrated control, the taskbar is controlled to execute an interaction function corresponding to the operation on the second integrated control.

[0172] In some embodiments, the control module 12 specifically: The control is used to hide the taskbar in response to a fourth operation on the second integrated control.

[0173] In some embodiments, the fourth operation functions as a click or double-click operation on the second unified control.

[0174] In some embodiments, the control module 12 specifically: In response to detecting that the cursor moves into a third detection area of ​​the second integrated control and the hovering time is equal to a fifth time length, the task bar is hidden, and the third detection area is used to cover at least the second integrated control.

[0175] In some embodiments, the control module 12 specifically: In response to a fifth operation of the second integrated control, the second integrated control is used to control the movement of the taskbar or to control the movement of the taskbar and the UI of the application running in the foreground together.

[0176] In some embodiments, the control module 12 specifically: Controlling the second integrated control to switch from a first state to a second state in response to a long press operation on the second integrated control; After the second integrated control is switched to the second state, the control is used to control the movement of the taskbar in response to a drag operation on the second integrated control, or to control the movement of the taskbar and the UI of the application running in the foreground together.

[0177] In some embodiments, the second integrated control is a down arrow control.

[0178] In some embodiments, the first state of the second integrated control is a down arrow and the second state of the second integrated control is a horizontal bar.

[0179] In some embodiments, the relative position of the first integrated control and the UI remains unchanged.

[0180] In some embodiments, the relative positions of the second integrated control and the taskbar remain unchanged.

[0181] It should be understood that the device embodiment and the method embodiment can correspond to each other, and similar descriptions can refer to the method embodiment, which will not be described in detail here to avoid redundancy.

[0182] The above describes the apparatus 100 of the embodiment of the present invention in terms of functional modules, combined with the accompanying drawings. It should be understood that the functional modules may be implemented in hardware, software instructions, or a combination of hardware and software modules. Specifically, each step of the method embodiment in the embodiment of the present invention can be completed by a hardware integrated logic circuit and / or software instructions in a processor. The method combination steps disclosed in the embodiment of the present invention may be directly implemented by a hardware decoding processor or may be implemented by a combination of hardware and software modules in the decoding processor. Alternatively, the software modules may reside in a storage medium well-established in the art, such as a random memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or register. The storage medium is located in the memory, and the processor reads information in the memory and completes the steps of the above-described method embodiment in accordance with the hardware.

[0183] In addition, an embodiment of the present invention provides an electronic device. Figure 15 is a structural schematic diagram of an electronic device provided by embodiment 8 of the present invention. As shown in Figure 15, this electronic device 200 includes: The system includes a memory 21 that stores a computer program and transmits the program code to the processor 22, and the processor 22. In other words, the processor 22 can call and execute the computer program from the memory 21 to implement the method in the embodiment of the present invention.

[0184] For example, the processor 22 may be used to carry out the method embodiments described above according to instructions in a computer program.

[0185] In some embodiments herein, the processor 22 may include, but is not limited to:

[0186] General purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0187] In some embodiments of the present invention, the memory 21 includes, but is not limited to:

[0188] Volatile and / or non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synch Link DRAM, SLDRAM), and direct memory access memory (Direct Rambus RAM, DR RAM).

[0189] In some embodiments of the present invention, the computer program may be divided into one or more modules that are stored in the memory 21 and executed by the processor 22 to complete the method provided by the present invention. The one or more modules may be a series of computer program instruction segments that describe the execution process of the computer program in an electronic device.

[0190] As shown in FIG. 15, the electronic device may further include a transceiver 23 connectable to the processor 22 or the memory 21 .

[0191] The processor 22 controls the transceiver 23 to communicate with other devices, specifically to transmit information or data to other devices and receive information or data transmitted by other devices. The transceiver 23 may include a transmitter and a receiver. The transceiver 23 may further include an antenna, and the number of antennas may be one or more.

[0192] Although not shown in FIG. 15, the electronic device 200 may also include a camera module, a wireless fidelity WIFI module, a positioning module, a Bluetooth module, a display, a controller, etc., which will not be further described here.

[0193] The components of this electronic device may be understood to be connected via a bus system that may include a power bus, a control bus, and a status signal bus in addition to a data bus.

[0194] The present invention also provides a computer storage medium having stored thereon a computer program which, when executed by a computer, enables the computer to perform the method of the above-described method embodiments. Alternatively, an embodiment of the present invention provides a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the above-described method embodiments.

[0195] The present invention also provides a computer program product, including a computer program stored in a computer-readable storage medium, wherein a processor of an electronic device reads the computer program from the computer-readable storage medium to execute a corresponding flow of the method for controlling a user position in a virtual scene in an embodiment of the present invention, and the description of the processor is omitted here for brevity.

[0196] In some embodiments provided herein, it is apparent that the disclosed systems, devices, and methods may be implemented in other ways. For example, the device embodiments described above are merely exemplary, and the division of modules is merely a logical functional division. In actual implementation, there may be other division methods, such as multiple modules or components being combined or integrated into another system, or some features being ignored or not implemented. In other respects, the couplings or direct couplings or communication connections shown or discussed between each other may be indirect couplings or communication connections via some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0197] Modules described as separate components may be physically separated, and components displayed as modules may not be physical modules, i.e., they may be located in one place or distributed across multiple network units. This embodiment may select some or all of its modules to achieve its purpose according to actual needs. For example, each functional module in each embodiment of the present invention may be integrated into a single processing module, each module may exist physically separately, or two or more modules may be integrated into a single module.

[0198] The above are only specific embodiments of the present invention, and the scope of protection of the present invention is not limited thereto. Those skilled in the art can easily think of modifications or changes within the technical scope disclosed in the present invention, and all of these should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be in accordance with the scope of protection of the claims.

Claims

1. In response to an open command of an application in the augmented reality space, display a user interface UI of the application and a first integrated control of the application, the first integrated control being located outside the UI, integrating multiple interaction functions of the application, and being displayed according to the UI; and controlling the application to execute an interaction function corresponding to the operation in response to the operation on the first integrated control. How to control applications in an augmented reality space.

2. In response to an operation on the first integrated control, controlling the application to execute an interaction function corresponding to the operation includes: minimizing the UI to a taskbar in response to a first operation on the first integrated control. The method of claim 1.

3. the first operation functions as a click operation on the first integrated control; The method of claim 2.

4. In response to an operation on the first integrated control, controlling the application to execute an interaction function corresponding to the operation includes: controlling movement of the UI in response to a second operation on the first integrated control; The method of claim 1.

5. Controlling the movement of the UI in response to a second operation on the first integrated control includes: Controlling the first integrated control to switch from a first state to a second state in response to a long press operation on the first integrated control; and controlling movement of the UI in response to a drag operation on the first integrated control after the first integrated control is switched to the second state. The method of claim 4.

6. the second operation functions as a drag operation on the first unified control; The method of claim 4.

7. In response to an operation on the first integrated control, controlling the application to execute an interaction function corresponding to the operation includes: displaying a title bar at a preset position in response to detecting that a cursor moves into a first detection area of ​​the first integrated control and a hovering time length is equal to a first time length, the title bar including a plurality of controls, and the first detection area covering at least the first integrated control; The method of claim 1.

8. displaying a title bar at a preset position in response to detecting that a cursor moves into a first detection area of ​​the first integrated control and a hovering time length is equal to a first time length; controlling the first integrated control to switch from a first state to a second state in response to detecting that a cursor moves into the first detection area and a hovering time length is equal to a second time length, the second time length being less than the first time length; and displaying the title bar at a preset position when detecting that the first integrated control is in the second state and the hovering time length is equal to the first time length. The method of claim 7.

9. detecting that the cursor has moved out of a first detection area and closing the title bar when the length of time the cursor has moved out is equal to a third length of time; 9. The method according to claim 7 or 8.

10. the title bar is positioned around the first integrated control, and the method further comprises: and further including, in response to detecting that the cursor has moved out of a second detection area for a time length equal to a third time length, closing the title bar, wherein the second detection area covers at least the first integrated control and the title bar. The method of claim 8.

11. The first state is a downward arrow type and the second state is a horizontal bar type.

9. The method according to claim 5 or 8.

12. In response to an operation on the first integrated control, controlling the application to execute an interaction function corresponding to the operation includes: and closing the UI and the application in response to a third operation on the first integrated control. The method of claim 1.

13. closing the UI and the application in response to a third operation on the first integrated control; In response to a third operation on the first integrated control, controlling the first integrated control to switch from a first state to a third state, the third state being used to characterize the application being turned off; and closing the UI and the application in response to the first integrated control being in the third state for a fourth length of time. The method of claim 12.

14. the third operation functions as a double-click operation; 14. The method according to claim 12 or 13.

15. The first state is a down arrow type and the third state is a fork type. The method of claim 13.

16. the first integrated control is a down arrow control; The method according to any one of claims 1 to 5, 7 and 12.

17. No controls are included within the UI, and the first integrated control integrates functionality of existing controls in the application. The method according to any one of claims 1 to 5, 7 and 12.

18. A taskbar and a second integrated control of the taskbar are also displayed in the augmented reality space, the second integrated control being located outside the taskbar, integrating multiple interaction functions of the taskbar, and being displayed according to the taskbar; and the method further includes: and further comprising controlling, in response to an operation on the second integrated control, the taskbar to execute an interaction function corresponding to the operation on the second integrated control. The method according to any one of claims 1 to 5, 7 and 12.

19. Controlling the taskbar to execute an interaction function corresponding to the operation of the second integrated control in response to the operation of the second integrated control includes: hiding the task bar in response to a fourth operation on the second integrated control.

20. The method of claim 18.

20. the fourth operation functions as a click operation or a double-click operation on the second integrated control.

20. The method of claim 19.

21. hiding the taskbar in response to a fourth operation on the second integrated control; hiding the task bar in response to detecting that a cursor moves into a third detection area of ​​the second integrated control and a hovering time length is equal to a fifth time length, the third detection area covering at least the second integrated control.

20. The method of claim 19.

22. Controlling the taskbar to execute an interaction function corresponding to the operation of the second integrated control in response to the operation of the second integrated control includes: and controlling, in response to a fifth operation on the second integrated control, movement of the task bar or control to move the task bar and a UI of the application executed in the foreground together.

20. The method of claim 18.

23. In response to a fifth operation on the second integrated control, controlling the movement of the task bar or controlling the movement of the task bar and a UI of the application executed in the foreground together includes: Controlling the second integrated control to switch from a first state to a second state in response to a long press operation on the second integrated control; After the second integrated control is switched to the second state, in response to a drag operation on the second integrated control, controlling the movement of the taskbar or controlling the taskbar and a UI of the application executed in the foreground to move together.

23. The method of claim 22.

24. the second integrated control is a down arrow control; 20. The method of claim 18.

25. a first state of the second integrated control being a down arrow and a second state of the second integrated control being a horizontal bar; 24. The method of claim 23.

26. The relative position of the first integrated control and the UI does not change. The method according to any one of claims 1 to 5, 7 and 12.

27. The relative position of the second integrated control and the taskbar remains unchanged.

20. The method of claim 18.

28. a display module that displays a user interface (UI) of the application and a first integrated control of the application in response to an open command of the application in the augmented reality space, the first integrated control being located outside the UI, integrating multiple interaction functions of the application, and being displayed according to the UI; a control module that controls the application to execute an interaction function corresponding to an operation on the first integrated control in response to the operation. A control device for applications in an augmented reality space.

29. A processor and a memory, To carry out the method according to any one of claims 1 to 27, the memory stores a computer program; The processor is used to call and execute computer programs stored in the memory. Electronic devices.

30. storing a computer program that causes a computer to carry out the method according to any one of claims 1 to 27; A computer-readable storage medium.

31. A computer program is provided, The computer program, when executed by a processor, implements the method of any one of claims 1 to 27. Computer program products.

Images