Interaction Method, Device, Equipment and Storage Medium for Virtual Objects

Abstract

According to an embodiment of the present disclosure, there are provided an interaction method, apparatus, device, and storage medium for virtual objects. The method includes: detecting a gesture event for a presented scene, the scene including a plurality of virtual objects; selecting a plurality of interaction targets related to the detected gesture event from the plurality of virtual objects; and causing the plurality of interaction targets to respond to the gesture event while maintaining the relative positions between the plurality of interaction targets. In this way, the object interaction method can be made more diverse, the interaction efficiency can be improved, and the user experience can be enhanced.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and particularly to an interaction method, apparatus, device, and computer-readable storage medium for virtual objects. Background Art

[0002] Augmented Reality (AR) technology is a technology that combines virtual information with the real world. In the process of applying AR technology, virtual objects can be superimposed on the pictures in the real world and presented in the AR scene. In this way, the images that appear in the user's field of view include both the pictures of the real world and virtual objects, enabling the user to see both virtual objects and the real world at the same time. In addition, the user can interact with the virtual objects presented in the AR scene through gestures. For example, when multiple virtual objects are presented in the AR scene, the user can drag, scale, rotate, etc. a single virtual object by selecting it. Summary of the Invention

[0003] In a first aspect of the present disclosure, there is provided an interaction method for virtual objects. The method includes: detecting a gesture event for a presented scene, the scene including a plurality of virtual objects; selecting a plurality of interaction targets related to the gesture event from the plurality of virtual objects; and causing the plurality of interaction targets to respond to the gesture event while maintaining the relative positions between the plurality of interaction targets. In this way, the object interaction method can be made more diverse, the interaction efficiency can be improved, and the user experience can be enhanced.

[0004] In a second aspect of the present disclosure, there is provided an interaction apparatus for virtual objects. The apparatus includes: a gesture detection module configured to detect a gesture event for a presented scene, the scene including a plurality of virtual objects; a target selection module configured to select a plurality of interaction targets related to the gesture event from the plurality of virtual objects; and a gesture response module configured to cause the plurality of interaction targets to respond to the gesture event while maintaining the relative positions between the plurality of interaction targets. In this way, the object interaction method can be made more diverse, the interaction efficiency can be improved, and the user experience can be enhanced.

[0005] In a third aspect of the present disclosure, there is provided an electronic device. The device includes at least one processing unit; and at least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to execute the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, there is provided a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the content described in the present invention content section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0010] Figure 2 A schematic diagram showing a virtual object interaction architecture according to some embodiments of the present disclosure;

[0011] Figure 3 A flowchart showing an interaction process for virtual objects according to an embodiment of the present disclosure;

[0012] Figure 4 A schematic diagram showing a gesture event for a scene according to some embodiments of the present disclosure;

[0013] Figures 5A to 5D Shows Figure 4 A schematic diagram showing the response of a virtual object in the shown scene to a gesture event;

[0014] Figure 6 A schematic diagram showing the positions of multiple interaction targets and their reference points in a coordinate system;

[0015] Figure 7 A schematic diagram showing a user selection of a group of virtual objects among multiple virtual objects;

[0016] Figure 8 A schematic diagram showing the grouping of multiple virtual objects presented in a scene;

[0017] Figure 9 A schematic diagram showing a scene according to some embodiments of the present disclosure;

[0018] Figure 10 A block diagram showing an interaction device for virtual objects according to some embodiments of the present disclosure; and

[0019] Figure 11 A block diagram showing a device capable of implementing multiple embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not used to limit the protection scope of the present disclosure.

[0021] In the description of the embodiments of the present disclosure, the term "including" and its like shall be understood as an open inclusion, that is, "including but not limited to". The term "based on" shall be understood as "at least partially based on". The term "one embodiment" or "the embodiment" shall be understood as "at least one embodiment". The term "some embodiments" shall be understood as "at least some embodiments". There may also be other explicit and implicit definitions hereinafter.

[0022] As briefly mentioned above, in the current field of virtual object interaction, when multiple virtual objects are presented in a scene, the user can only interact with a single virtual object and cannot interact with multiple virtual objects simultaneously. It can be seen that this conventional interaction method is single and inefficient, affecting the user experience.

[0023] Embodiments of the present disclosure propose an interaction solution for virtual objects. In this solution, for multiple virtual objects in the presented scene, multiple interaction targets related to gesture events can be selected, and these interaction targets can be interacted with simultaneously while maintaining the relative positions between the individual interaction targets, such as placing, dragging, scaling, rotating these interaction targets, etc. In this way, the object interaction method can be made more diverse, the interaction efficiency can be improved, and the user experience can be enhanced.

[0024] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. In this example environment 100, a scene 150 is presented at or by the terminal device 110 to the user 130. The scene 150 includes objects 154, 1531, 1532, 1533, and 1534. The user 130 can interact with at least some of the objects 154, 1531, 1532, 1533, and 1534 through a predetermined gesture.

[0025] In some embodiments, as Figure 1As shown, the scene 150 can be an AR scene. The object 154 is a representation of a real object in the real world (a chair in this example) in the scene 150, such as an image of the real object or other forms of representation. For the sake of discussion only, the object 154 is also referred to as a 3D object in this article. In some embodiments, the object 154 is interactive. For example, the size, orientation, display pixels, etc. of the object 154 can change in response to the input of the user 130. In some embodiments, the object 154 is non-interactive and is only presented in the scene 150 for the user to view.

[0026] The objects 1531, 1532, 1533, and 1534 are virtual objects, specifically AR objects in this example. The objects 1531, 1532, 1533, and 1534 are respectively used to represent cylinders and cubes that can be placed on or near the chair. The objects 1531, 1532, 1533, and 1534 are interactive. For example, the size, orientation, color, etc. of the objects 1531, 1532, 1533, and 1534 can change in response to the input of the user 130.

[0027] It should be understood that the scene 150 is only exemplary and is not intended to limit the scope of the present disclosure. The scene 150 may include more or fewer objects, or may include other elements, such as user interface (UI) elements. In some embodiments, the scene 150 can be a Virtual Reality (VR) scene. The VR scene includes a plurality of virtual objects. The user can interact with the virtual objects presented in the VR scene.

[0028] The terminal device 110 can be any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, gaming devices, wearable devices, Personal Communication System (PCS) devices, personal navigation devices, Personal Digital Assistant (PDA), audio / video players, digital cameras / cameras, positioning devices, television receivers, radio broadcast receivers, e-book devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 is also capable of supporting any type of interface for the user (such as a "wearable" circuit, etc.).

[0029] An engine 120 is installed in the terminal device 110. The engine 120 is used to drive the rendering of the scene 150. In some examples, the engine 120 can be an AR game engine, and correspondingly, the scene 150 can be an AR game scene. In some embodiments, the engine 120 can be part of a content sharing application that can provide services related to multimedia content consumption to the user 130, including browsing, commenting, forwarding, creating (e.g., shooting and / or editing), publishing, etc. of multimedia content. Correspondingly, the scene 150 can be an AR content creation scene.

[0030] It should be understood that the structure and functions of the environment 100 are described only for exemplary purposes, without implying any limitation to the scope of the present disclosure. The terminal device 110 can include any suitable structure and functions to implement virtual object interaction according to the embodiments of the present disclosure.

[0031] Figure 2 A schematic diagram of a virtual object interaction architecture 200 according to some embodiments of the present disclosure is shown. As Figure 2 shown, the architecture 200 includes a plurality of virtual objects 253-1 to 253-m, which are also collectively or individually referred to as virtual objects 253, where m is a positive integer greater than or equal to 2. Each virtual object 253 includes an interaction component and a collider. The collider defines the boundary of the corresponding virtual object in the presented scene 150. For example, the collider of the virtual object 253-1 defines the boundary of the virtual object 253-1 in the scene 150. The interaction component is used to change the rendering state of the virtual object in the scene 150, such as size, position, orientation, etc., under the control of the gesture system 203.

[0032] The object camera 202 is a virtual camera for monitoring the virtual object 253. When a gesture event 201 is received, the object camera 202 or a raycaster associated with the object camera 202 can emit rays. By monitoring whether the rays intersect with each collider, it can be determined whether the gesture event 201 hits the corresponding virtual object. Subsequently, the gesture system 203 can change the rendering state of the virtual object in the scene 150 according to whether the gesture event 201 hits the virtual object.

[0033] The gesture event 201 can include placing, dragging, scaling, rotating, etc. The gesture event 201 can also include other types of gestures, and the embodiments of the present disclosure do not limit this.

[0034] In some embodiments, in Figure 1 the shown scene 150, UI elements 205 (or referred to as UI controls) can also be presented. In such an embodiment, as Figure 2As shown, the architecture 200 further includes a UI camera 204. The UI camera 204 is a virtual camera for monitoring UI elements 205. When a gesture event 201 is received, the UI camera 204 or a raycaster associated with the UI camera 204 can emit rays. By monitoring whether the rays intersect with the UI elements 205, it can be determined whether the gesture event 201 hits the UI elements 205. Subsequently, depending on whether the gesture event 201 hits the UI elements 205, the functions corresponding to the UI elements 205 can be triggered or the pages corresponding to the UI elements 205 can be presented.

[0035] The interaction components and colliders of the object camera 202, gesture system 203, UI camera 204, and virtual objects 253 in the architecture 200 can be implemented in the engine 120. Figure 1 The virtual objects 1531, 1532, 1533, 1534 shown in can be part of multiple virtual objects 253-1 to 253-m. It should be understood that the architecture 200 described herein is merely exemplary and is not intended to limit the scope of the present disclosure. Any suitable functions and structures can be adopted to implement the virtual object interaction according to the embodiments of the present disclosure.

[0036] Some example embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0037] Figure 3 A flowchart of an interaction process 300 for virtual objects according to an embodiment of the present disclosure is shown. The process 300 can be implemented at the terminal device 110. For ease of discussion, the process 300 will be described with reference to Figure 1 the environment 100 and Figure 2 the architecture 200.

[0038] In block 310, the terminal device 110 detects a gesture event for the scene 150. Figure 4 A schematic diagram of a gesture event 401 for the scene 150 according to some embodiments of the present disclosure is shown. The gesture event 401 can be regarded as a specific implementation of the gesture event 201. As Figure 4 shown, the scene 150 includes a 3D object 154 and multiple virtual objects 1531, 1532, 1533, and 1534. In some examples, the scene 150 can be presented on the screen of the terminal device 110. The user 130 can initiate the gesture event 401 by touching the screen. In other examples, the scene 150 can be presented in the real-world space. The user 130 can initiate the gesture event 401 by making gestures in the space. The gesture event 401 detected by the terminal device 110 can be related to multiple virtual objects among the virtual objects 1531, 1532, 1533, and 1534, so that these virtual objects can respond to the gesture event 401.

[0039] At block 320, the terminal device 110 selects multiple interaction targets related to the gesture event from among multiple virtual objects 1531, 1532, 1533, and 1534. The terminal device 110 can adopt any suitable strategy to select the interaction targets.

[0040] In some embodiments, if it is determined that the gesture event does not hit any of the virtual objects among the multiple virtual objects 1531, 1532, 1533, and 1534, the terminal device 110 determines these virtual objects as interaction targets. The gesture event not hitting any virtual object in the scene 150 means that the gesture event is used to operate on all the virtual objects in the scene simultaneously. In this case, the terminal device 110 selects all the virtual objects in the scene 150 as interaction targets. As Figure 4 shown, the gesture event 401 does not hit any of the virtual objects 1531, 1532, 1533, and 1534, which means that the gesture event 401 is used to operate on all the virtual objects in the scene 150. In this case, the virtual objects 1531, 1532, 1533, and 1534 are simultaneously determined as interaction targets.

[0041] In some embodiments, the terminal device 110 can determine the interaction targets based on user selection for the virtual objects 1531, 1532, 1533, and 1534. Such embodiments will be described below with reference to Figure 7 and Figure 8 to describe such embodiments.

[0042] At block 330, while maintaining the relative positions among the multiple interaction targets, the terminal device 110 causes the multiple interaction targets to respond to the gesture event. In other words, during the process of the multiple interaction targets responding to the gesture event, the relative positions among the respective interaction targets are maintained. Figures 5A to 5D Illustrated is Figure 4 a schematic diagram of an example response of the virtual objects 1531, 1532, 1533, and 1534 in the scene 150 shown in

[0043] Figure 5A to the gesture event 401. Figure 4 and Figure 5A , in the case where the gesture event 401 is a drag gesture, the terminal device 110 can simultaneously drag the virtual objects 1531, 1532, 1533, and 1534 from Figure 4 the position shown in

[0044] Figure 5B to the position shown in Figure 5. During the dragging process, the relative positions among the virtual objects 1531, 1532, 1533, and 1534 are maintained.

[0044] Figure 5B The example response shown is object scaling. In combination withFigure 4 And Figure 5B , when the gesture event 401 is a pinch gesture, the terminal device 110 can simultaneously scale the virtual objects 1531, 1532, 1533, and 1534 from the state shown in Figure 4 to the state shown in FIG. 5. During the scaling process, the relative positions among the virtual objects 1531, 1532, 1533, and 1534 are maintained. Figure 5B Object shrinking is shown as an example in

[0045] Figure 5C The example response shown in Figure 4 and Figure 5C , when the gesture event 401 is a rotation gesture, the terminal device 110 can simultaneously rotate the virtual objects 1531, 1532, 1533, and 1534 from the orientation shown in Figure 4 to the orientation shown in FIG. 5. During the rotation process, the relative positions among the virtual objects 1531, 1532, 1533, and 1534 are maintained.

[0046] Figure 5D The example response shown in Figure 5D is object placement. For ease of understanding, the positions where the virtual objects 1531, 1532, 1533, and 1534 are located before being placed are shown in dashed lines in

[0047] If the terminal device 110 determines that the gesture event involves a placement operation, it determines the reference points of the multiple interaction targets and the positions in the scene hit by the gesture event. Based on the reference points and the determined positions, the terminal device 110 can determine the respective target positions of the multiple interaction targets in the scene. Subsequently, the terminal device 110 can place the multiple interaction targets to the target positions respectively.

[0048] As Figure 5D shown, the reference points of the virtual objects 1531, 1532, 1533, and 1534 before placement are their center points C, and the click position of the gesture event 401 in the scene 150 is point P. Point P is the object placement point. The terminal device 110 can determine the placement positions of the virtual objects 1531, 1532, 1533, and 1534 according to the coordinates of point C and point P. The specific process is as follows.

[0049] Combined with Figure 6 describe how to determine the coordinates of the center point C of the interaction target. Figure 6 shows a schematic diagram of the positions of multiple interaction targets and their center points C in the coordinate system. Figure 6Shows interaction targets 601-1, 601-2, 601-3, 601-4... 601-n, which are also collectively referred to as interaction target 601, where n is a positive integer greater than or equal to 2. The coordinates of each interaction target 601 in the coordinate system are X1, X2,... Xn respectively. The coordinate Xc of the center point C of these interaction targets can be calculated by Equation 1.

[0050]

[0051] Specifically, according to Equation 1, the average value of the coordinates X1, X2,... Xn of each interaction target 601 can be calculated to obtain the coordinate Xc of the center point C, so as to represent the position of the center point C.

[0052] Continue to refer to Figure 5D . In Figure 5D , the interaction targets are four, namely virtual objects 1531, 1532, 1533 and 1534, and the coordinates are X1, X2, X3 and X4 respectively. Therefore, Figure 5D the coordinate Xc of the center point C shown in

[0053] The terminal device 110 determines the position of the scene 150 currently clicked by the gesture event 401. For example, light can be emitted by the object camera 202 shown in Figure 2 to determine the intersection point of the light and the scene 150 as point P, and the coordinate Xp of point P is obtained.

[0054] Figure 5D The target positions of the respective interaction targets shown in

[0055] X′ i =(X i -Xc)+Xp Equation 2.

[0056] Specifically, according to Equation 2, the vector offset Xi - Xc of the coordinate Xi of each virtual object relative to the coordinate Xc of the center point C can be calculated first. Subsequently, each vector offset Xi - Xc can be added to the coordinate Xp of point P to obtain the coordinate Xi' of the target position of each interaction target.

[0057] The terminal device 110 can move each interaction target to the target position represented by the coordinate Xi' respectively. In this way, the relative positions between the interaction targets are ensured to be maintained during the object placement process, without causing all these interaction targets to be concentrated at the point P hit by the gesture event.

[0058] It should be understood that although the principle of the present disclosure is described above by taking the center point C of multiple interaction targets as a reference point, other points can also be used as reference points to implement the placement of multiple virtual objects. For example, any point in the presented scene can be selected as the reference point, and its coordinates are denoted as Xref. Subsequently, the vector offset Xi - Xref of the coordinates Xi of each virtual object relative to the coordinates Xref of the reference point can be calculated. Based on this vector offset, the coordinates Xi' of the target positions of each interaction target can be calculated by Equation 3 similarly to Equation 2.

[0059] X′ i =(X i -X ref ) + Xp Equation 3.

[0060] Subsequently, the terminal device 110 can move each interaction target to the target position represented by the coordinates Xi'.

[0061] As described above with reference to block 320, in some embodiments, at block 320, the terminal device 110 can determine interaction targets based on user selection. Further, in some embodiments, in response to receiving a user selection of a set of virtual objects among multiple virtual objects, the terminal device 110 can determine the set of virtual objects as multiple interaction targets. In other words, in one interaction, it may be only necessary to operate on a set of virtual objects in the scene 150, and there is no need to operate on other virtual objects. In this case, the terminal device 110 can receive the user's selection of the set of virtual objects, thereby determining the set of objects as interaction targets. In such an embodiment, the gesture event can follow immediately after the user selection. That is, the user 130 can first select a set of virtual objects, and the most recent interaction thereafter is only performed on the selected set of virtual objects.

[0062] Figure 7 A schematic diagram showing a user selection of a set of virtual objects among multiple virtual objects is shown. As Figure 7 shown, the virtual objects 1532, 1533, and 1534 in the scene 150 can be selected using the selection box 710. For example, the user 130 can provide the selection box 710 before initiating the gesture event 201. The virtual objects included in the selection box 710 are used as interaction targets. In this example, at Figure 3 the block 330 shown, while maintaining the relative positions between the set of virtual objects, the terminal device 110 causes the virtual objects 1532, 1533, 1534 in the selection box 710 to respond to the gesture event 201. For example, the terminal device 110 causes the set of virtual objects 1532, 1533, 1534 to be combined with Figures 5A to 5DRespond to the gesture event 201 in a manner similar to the described manner, such as performing object dragging, object scaling, object rotation, object placement, etc. The specific interaction process will not be elaborated here.

[0063] It should be understood that in other embodiments, a group of virtual objects among multiple virtual objects can be selected in other ways, such as by clicking. A group of virtual objects can be formed by clicking on the virtual objects 1532, 1533, and 1534 in the scene 150 one by one, and used as the interaction target for the subsequent gesture event. The scope of the present disclosure is not limited in terms of how to select virtual objects.

[0064] In some embodiments, the terminal device 110 can group multiple virtual objects based on the user's selection of the multiple virtual objects. With such grouping, the user can interact with the grouped virtual objects. Figure 8 A schematic diagram showing the grouping of multiple virtual objects presented in the scene is shown. As Figure 8 shown, the terminal device 110 can divide the virtual objects 1531, 1532, 1533, and 1534 into a first group 810 and a second group 820 based on the user's selection of the multiple virtual objects. The terminal device 110 can determine the group to which the virtual object targeted by the gesture event 201 belongs, and determine the virtual objects in the determined group as the interaction target. For example, if the virtual object targeted by the gesture event 201 is the virtual object 1533, since the group to which the virtual object 1533 belongs is the second group 1522, the virtual objects 1533 and 1534 in the second group 820 are determined as the interaction target. In Figure 3 the box 330 shown, while maintaining the relative positions between the virtual objects 1533 and 1534 in the second group 1522, the terminal device 110 causes the virtual objects 1533 and 1534 to respond to the gesture event 201. For example, the terminal device 110 causes the virtual objects 1533 and 1534 to respond to the gesture event 201 in a manner similar to the manner described in conjunction with Figures 5A to 5D such as performing object dragging, object scaling, object rotation, object placement, etc. The specific interaction process will not be elaborated here.

[0065] In some embodiments, in addition to virtual objects, the scene presented by the terminal device 110 may also include UI elements. These UI elements may overlap with the virtual objects. Figure 9 A schematic diagram of a gesture event for a scene 950 according to some embodiments of the present disclosure is shown. As Figure 9As shown, compared with scenario 150, in addition to objects 154, 1531, 1532, 1533, and 1534, scenario 950 also includes UI elements 151 and 152. UI element 151 is a setting control for setting parameters of various aspects of scenario 150. UI element 152 is a music control for controlling the music playback in scenario 150. User 130 can set scenario 150 and control music playback through UI elements 151 and 152, so UI elements 151 and 152 are interactive. User 130 initiated gesture event 901, which can be regarded as a specific implementation of gesture event 201.

[0066] In this embodiment, at block 320, the terminal device 110 determines whether the gesture event involves (e.g., hits) a UI element in the scenario, and if it is determined that the gesture event does not involve (e.g., does not hit) a UI element, selects multiple interaction targets from multiple virtual objects. In other words, if the gesture event involves a UI element, the terminal device 110 will execute the function corresponding to the UI element or present the page corresponding to the UI element, rather than selecting an interaction target from the virtual objects.

[0067] As Figure 9 shown, if the terminal device 110 determines that the gesture event 901 does not involve UI elements 151 or 152, multiple interaction targets are selected from virtual objects 1531, 1532, 1533, and 1534. The specific selection method can be carried out in a manner similar to the manner described with reference to Figures 4 to 8 and will not be elaborated here. On the contrary, if the terminal device 110 determines that the gesture event involves UI elements 151 or 152, the terminal device 110 will execute the function corresponding to UI elements 151 or 152 or present the page corresponding to UI elements 151 or 152, rather than selecting an interaction target from virtual objects 1531, 1532, 1533, and 1534.

[0068] In this embodiment, the mutual exclusion between UI response and virtual object response is achieved. In this way, interaction confusion can be avoided and the user experience can be further improved.

[0069] In some embodiments, if the terminal device 110 detects a gesture event for a single virtual object among multiple virtual objects, the terminal device 110 can cause the single virtual object to respond to the gesture event. In this way, each object among multiple virtual objects can still be operated independently.

[0070] Although the principles of the present disclosure have been described above in conjunction with AR scenarios, it should be understood that the embodiments of the present disclosure can equally be used for the interaction of virtual objects in VR scenarios or other scenarios. The scope of the present disclosure is not limited in this regard.

[0071] Figure 10 FIG. 1 shows a schematic structural block diagram of an interaction device 1000 for virtual objects according to certain embodiments of the present disclosure. The device 1000 may be implemented as or included in a terminal device 110. Each module / component in the device 1000 may be implemented by hardware, software, firmware, or any combination thereof.

[0072] As shown in the figure, the device 1000 includes: a gesture detection module 1010 configured to detect gesture events for a presented scene, the scene including a plurality of virtual objects; a target selection module 1020 configured to select a plurality of interaction targets related to the gesture events from the plurality of virtual objects; and a gesture response module 1030 configured to cause the plurality of interaction targets to respond to the gesture events while maintaining the relative positions between the plurality of interaction targets.

[0073] In some embodiments, the target selection module 1020 is further configured to: if it is determined that the gesture event does not hit a virtual object among the plurality of virtual objects, determine the plurality of virtual objects as the plurality of interaction targets.

[0074] In some embodiments, the target selection module 1020 is further configured to: in response to receiving a user selection of a set of virtual objects among the plurality of virtual objects, determine the set of virtual objects as the plurality of interaction targets.

[0075] In some embodiments, the target selection module 1020 is further configured to: group the plurality of virtual objects based on a user selection of the plurality of virtual objects; determine the group to which the virtual object targeted by the gesture event belongs; and determine the virtual objects in the determined group as the plurality of interaction targets.

[0076] In some embodiments, the gesture response module 1030 is further configured to: if it is determined that the gesture event involves a placement operation, determine a reference point of the plurality of interaction targets and a position in the scene hit by the gesture event; based on the reference point and the position, determine respective target positions of the plurality of interaction targets in the scene; and cause the plurality of interaction targets to be placed at the target positions respectively.

[0077] In some embodiments, the target selection module 1020 is further configured to: determine whether the gesture event involves a user interface (UI) element in the scene; and if it is determined that the gesture event does not involve a UI element, select a plurality of interaction targets from the plurality of virtual objects.

[0078] In some embodiments, the scene includes an augmented reality scene, and the virtual objects include augmented reality objects.

[0079] Figure 11 FIG. 2 shows a block diagram of an electronic device 1100 in which one or more embodiments of the present disclosure may be implemented. It should be understood thatFigure 11 The illustrated electronic device 1100 is merely exemplary and should not constitute any limitation to the functions and scope of the embodiments described herein. Figure 11 The illustrated electronic device 1100 can be used to implement Figure 1 the terminal device 110.

[0080] As Figure 11 illustrated, the electronic device 1100 is in the form of a general-purpose computing device. The components of the electronic device 1100 can include, but are not limited to, one or more processors or processing units 1110, a memory 1120, a storage device 1130, one or more communication units 1140, one or more input devices 1150, and one or more output devices 1160. The processing unit 1110 can be an actual or virtual processor and is capable of performing various processes according to the programs stored in the memory 1120. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing ability of the electronic device 1100.

[0081] The electronic device 1100 generally includes multiple computer storage media. Such media can be any accessible media that can be obtained by the electronic device 1100, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 1120 can be a volatile memory (such as registers, caches, random access memory (RAM)), a non-volatile memory (such as read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 1130 can be a removable or non-removable medium and can include machine-readable media, such as a flash drive, a magnetic disk, or any other medium that can be used to store information and / or data (such as training data for training) and can be accessed within the electronic device 1100.

[0082] The electronic device 1100 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 11 it, a disk drive for reading from or writing to a removable, non-volatile magnetic disk (such as a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk can be provided. In these cases, each drive can be connected to a bus (not shown) by one or more data media interfaces. The memory 1120 can include a computer program product 1125, which has one or more program modules that are configured to execute the various methods or actions of the various embodiments of the present disclosure.

[0083] The communication unit 1140 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of the electronic device 1100 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating via a communication connection. Thus, the electronic device 1100 can operate in a networked environment using a logical connection to one or more other servers, network personal computers (PCs), or another network node.

[0084] The input device 1150 can be one or more input devices such as a mouse, keyboard, trackball, etc. The output device 1160 can be one or more output devices such as a display, speaker, printer, etc. The electronic device 1100 can also communicate with one or more external devices (not shown) as needed via the communication unit 1140, where the external devices are such as storage devices, display devices, etc., communicate with one or more devices that enable a user to interact with the electronic device 1100, or communicate with any device that enables the electronic device 1100 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0085] According to an exemplary implementation of the present disclosure, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, where the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, there is also provided a computer program product that is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0086] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0087] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, when executed by the processing unit of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable medium storing the instructions comprises a manufacture, including instructions for implementing various aspects of the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0088] The computer-readable program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to produce a computer-implemented process such that the instructions executed on the computer, other programmable data processing apparatus, or other devices implement the functions / acts specified in one or more boxes of the flowchart and / or block diagram.

[0089] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or combinations of special-purpose hardware and computer instructions.

[0090] The implementations of the present disclosure have been described above. The description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best explain the principles of the implementations, the practical application, or improvements made to the technology in the marketplace, or to enable other ordinary skill in the art to understand the implementations disclosed herein.

Claims

1. An interaction method for virtual objects, comprising: Detecting a gesture event for a presented scene, the scene including a plurality of virtual objects; Determining whether the gesture event involves a user interface (UI) element in the scene; If it is determined that the gesture event does not involve the UI element, selecting a plurality of interaction targets related to the gesture event from the plurality of virtual objects; And While maintaining the relative positions between the plurality of interaction targets, simultaneously interacting with the plurality of interaction targets, enabling the plurality of interaction targets to respond to the gesture event to change the presentation states of the plurality of interaction targets in the scene, Wherein enabling the plurality of interaction targets to respond to the gesture event includes: If it is determined that the gesture event involves a placement operation, determining a reference point of the plurality of interaction targets and a position in the scene hit by the gesture event; Based on the reference point and the position, determining respective target positions of the plurality of interaction targets in the scene; and Placing the plurality of interaction targets at the target positions respectively.

2. The method according to claim 1, wherein selecting the plurality of interaction targets includes: If it is determined that the gesture event does not hit a virtual object among the plurality of virtual objects, determining the plurality of virtual objects as the plurality of interaction targets.

3. The method according to claim 1, wherein selecting the plurality of interaction targets includes: In response to receiving a user selection of a group of virtual objects among the plurality of virtual objects, determining the group of virtual objects as the plurality of interaction targets.

4. The method according to claim 1, wherein selecting the plurality of interaction targets includes: Grouping the plurality of virtual objects based on a user selection of the plurality of virtual objects; Determining a group to which the virtual object targeted by the gesture event belongs; And Determining the virtual objects in the determined group as the plurality of interaction targets.

5. The method according to any one of claims 1 to 4, wherein the scene includes an augmented reality scene, and the virtual objects include augmented reality objects.

6. An interaction device for virtual objects, comprising: A gesture detection module configured to detect a gesture event for a presented scene, the scene including a plurality of virtual objects; A target selection module configured to determine whether the gesture event involves a user interface (UI) element in the scene; If it is determined that the gesture event does not involve the UI element, selecting a plurality of interaction targets related to the gesture event from the plurality of virtual objects; And The gesture response module is configured to simultaneously interact with the multiple interaction targets while maintaining the relative positions among the multiple interaction targets, enabling the multiple interaction targets to respond to the gesture event to change the presentation states of the multiple interaction targets in the scene. If it is determined that the gesture event involves a placement operation, determine the reference points of the multiple interaction targets and the position in the scene hit by the gesture event; based on the reference points and the position, determine the respective target positions of the multiple interaction targets in the scene; and place the multiple interaction targets at the target positions respectively.

7. An electronic device, comprising: At least one processing unit; And At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the device to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having stored thereon a computer program, which when executed by a processor implements the method according to any one of claims 1 to 5.

Images