Virtual object interaction method and device, equipment and medium

Abstract

The invention provides a virtual object interaction method and device, equipment and a medium. The method comprises the steps that a three-dimensional environment generated by a computer is displayed on a display component; presenting a first virtual object in the three-dimensional environment; displaying a second virtual object between the first virtual object and the user; detecting first input information of a user through input equipment, and displaying operation prompt information corresponding to the second virtual object; second input information of the user is detected through the input device, and when the second input information meets the preset condition, the operation prompt information is changed, so that the operation prompt information corresponds to the first virtual object. According to the method and the device, the user can interact with the shielded virtual object, so that the interaction interestingness and the interaction diversity of the user and the virtual object can be increased, and the interaction experience of the user and the virtual object is improved.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of human-computer interaction technologies, and in particular, to a virtual object interaction method, apparatus, device, and medium. Background Art

[0002] With the wide application of Extended Reality (XR) technologies, users can enter different virtual spaces through XR devices and interact with virtual objects in the virtual spaces.

[0003] Currently, when a user interacts with a virtual object in a virtual space, the user usually adopts Figure 1a controller interaction as shown in Figure 1b or gesture interaction as shown in Figure 1c to select a target virtual object and interact with it. However, when there is an occlusion relationship among multiple virtual objects in the user's field of view in the virtual space, such as virtual object C is occluded by virtual object A and virtual object D is occluded by virtual object B in

[0004] the user can only interact with the unoccluded virtual objects through gesture interaction or controller interaction, and cannot interact with the occluded virtual objects, resulting in a poor interaction experience between the user and the virtual objects.

[0005] In a first aspect, embodiments of the present application provide a virtual object interaction method, which is applied to an electronic device. The electronic device includes a display component and one or more input devices. The method includes:

[0006] Displaying a computer-generated three-dimensional environment on the display component;

[0007] Presenting a first virtual object in the three-dimensional environment;

[0008] Displaying a second virtual object between the first virtual object and the user;

[0009] Detecting first input information of the user via the input device and displaying operation prompt information corresponding to the second virtual object;

[0010] Detecting second input information of the user via the input device. When the second input information meets a predetermined condition, changing the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0011] Second aspect, an embodiment of the present application provides a virtual object interaction device configured in an electronic device, where the electronic device includes a display component and one or more input devices, including:

[0012] A display module, configured to display a computer-generated three-dimensional environment on the display component; present a first virtual object within the three-dimensional environment; display a second virtual object between the first virtual object and the user; detect first input information of the user via the input device, and display operation prompt information corresponding to the second virtual object;

[0013] A change module, configured to detect second input information of the user via the input device, and when the second input information meets a predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0014] Third aspect, an embodiment of the present application provides an electronic device, including:

[0015] A processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the virtual object interaction method as described in the first aspect embodiment or its various implementation manners.

[0016] Fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a computer program, where the computer program enables a computer to execute the virtual object interaction method as described in the first aspect embodiment or its various implementation manners.

[0017] Fifth aspect, an embodiment of the present application provides a computer program product containing program instructions, which, when the program instructions run on an electronic device, enable the electronic device to execute the virtual object interaction method as described in the first aspect embodiment or its various implementation manners.

[0018] The technical solution disclosed in the embodiment of the present application displays a computer-generated three-dimensional environment on the display component of the electronic device, presents a first virtual object within the three-dimensional environment, displays a second virtual object between the first virtual object and the user, detects first input information of the user via the input device, displays operation prompt information corresponding to the second virtual object, detects second input information of the user via the input device, and when the second input information meets a predetermined condition, changes the operation prompt information so that the operation prompt information corresponds to the first virtual object, enabling the user to interact with the occluded virtual object, thereby increasing the interaction interest and interaction diversity between the user and the virtual object and enhancing the interaction experience between the user and the virtual object. Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0020] Figure 1a It is a schematic diagram of interacting with any virtual object in a virtual space by using a controller interaction method in the related art;

[0021] Figure 1b It is a schematic diagram of interacting with any virtual object in a virtual space by using a gesture interaction method in the related art;

[0022] Figure 1c It is a top view of virtual object C being occluded by virtual object A and virtual object D being occluded by virtual object B in a virtual space;

[0023] Figure 2 It is a schematic flowchart of a virtual object interaction method provided by an embodiment of the present application;

[0024] Figure 3 It is a schematic flowchart of another virtual object interaction method provided by an embodiment of the present application;

[0025] Figure 4a It is a side view of a first virtual object partially occluding a second virtual object provided by an embodiment of the present application;

[0026] Figure 4b It is a side view of a first virtual object completely occluding a second virtual object provided by an embodiment of the present application;

[0027] Figure 5a It is a schematic diagram of a ray clone mode provided by an embodiment of the present application;

[0028] Figure 5b It is a schematic diagram of a ray splitting mode provided by an embodiment of the present application;

[0029] Figure 6 It is a schematic diagram of outputting a prompt message provided by an embodiment of the present application;

[0030] Figure 7 It is a schematic diagram of adjusting the display form of virtual objects with an occlusion relationship provided by an embodiment of the present application;

[0031] Figure 8 It is a schematic flowchart of yet another virtual object interaction method provided by an embodiment of the present application;

[0032] Figure 9Schematic diagram of a user interacting with any virtual object using a multimodal interaction method provided by an embodiment of the present application;

[0033] Figure 10 Schematic flowchart of another virtual object interaction method provided by an embodiment of the present application;

[0034] Figure 11 Schematic block diagram of a virtual object interaction device provided by an embodiment of the present application;

[0035] Figure 12 Schematic block diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0036] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. According to the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server including a series of steps or units does not necessarily have to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0038] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or solutions. Exactly speaking, using words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0039] In the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more, that is, at least two. "At least one" means one or more.

[0040] To facilitate the understanding of the embodiments of the present application, before describing each embodiment of the present application, some concepts involved in all embodiments of the present application are first explained as follows:

[0041] 1) Virtual Reality (VR for short), a technology for creating and experiencing virtual worlds, which determines the generation of a virtual environment. It is a multi-source information (the virtual reality mentioned in this article includes at least visual perception, and may also include auditory perception, tactile perception, motion perception, and even taste perception, olfactory perception, etc.). It realizes the integration of a virtual environment, an interactive three-dimensional dynamic visual scene, and the simulation of entity behavior, enabling users to immerse themselves in the simulated virtual reality environment and realizing applications in various virtual environments such as maps, games, videos, education, medical care, simulation, collaborative training, sales, assisting manufacturing, maintenance, and repair.

[0042] 2) Virtual Reality device (VR device), a terminal for realizing virtual reality effects, usually provided in the form of glasses, head-mounted displays (HMD for short), or contact lenses for realizing visual perception and other forms of perception. Of course, the form of the virtual reality device is not limited to this, and it can be further miniaturized or enlarged according to actual needs.

[0043] Optionally, the virtual reality devices described in the embodiments of the present application may include but are not limited to the following types:

[0044] 2.1) PC-based Virtual Reality (PCVR) device, which uses the PC to perform relevant calculations and data output for virtual reality functions. The external PC-based virtual reality device realizes the virtual reality effect using the data output by the PC.

[0045] 2.2) Mobile virtual reality device, which supports setting a mobile terminal (such as a smart phone) in various ways (such as a head-mounted display with a dedicated card slot). Through a wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations for virtual reality functions and outputs data to the mobile virtual reality device. For example, watch virtual reality videos through the APP of the mobile terminal.

[0046] 2.3) All-in-one virtual reality device, which has a processor for performing relevant calculations for virtual functions, thus having independent virtual reality input and output functions and does not need to be connected to a PC or a mobile terminal, with a high degree of freedom of use.

[0047] 3) Augmented Reality (AR): A technology that, during the process of a camera capturing an image, calculates in real time the camera pose parameters of the camera in the real world (or three-dimensional world, physical world), and adds virtual elements to the image captured by the camera according to the camera pose parameters. The virtual elements include, but are not limited to: images, videos, and 3D models. The goal of AR technology is to interact by overlaying the virtual world on the real world on the screen.

[0048] 4) Mixed Reality (MR): A simulated scene that integrates sensory inputs created by a computer (such as virtual objects) with sensory inputs from a physical set or their representations. In some MR scenes, the sensory inputs created by the computer can adapt to changes in the sensory inputs from the physical set. Additionally, some electronic systems for presenting MR scenes can monitor the orientation and / or position relative to the physical set so that virtual objects can interact with real objects (i.e., physical elements from the physical set or their representations). For example, the system can monitor movement so that a virtual plant appears stationary relative to a physical building.

[0049] 5) Extended Reality (XR) refers to all real and virtual combined environments and human-computer interactions generated by computer technology and wearable devices, which includes various forms such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR).

[0050] 6) A virtual scene is a virtual scene displayed (or provided) when an application runs on an electronic device. The virtual scene can be a simulation environment of the real world, a semi-simulated and semi-fictional virtual scene, or a purely fictional virtual scene. The virtual scene can be any one of a two-dimensional virtual scene, a 2.5D virtual scene, or a three-dimensional virtual scene. The embodiments of the present application do not limit the dimension of the virtual scene. For example, the virtual scene can include the sky, land, ocean, etc. The land can include environmental elements such as deserts, cities, etc. Users can control virtual objects to move in the virtual scene. It should be understood that the above virtual scene can also be referred to as a virtual space.

[0051] 7) A virtual object is an object that interacts in a virtual scene, is controlled by a user or a robot program (such as an artificial intelligence-based robot program), and can be stationary, move, and perform various behaviors in the virtual scene, such as various characters in a game.

[0052] When a user interacts with virtual objects presented in a virtual space, the user usually selects a target virtual object through controller interaction or gesture interaction and conducts various interactions with the target virtual object. When there is an occlusion relationship among multiple virtual objects in the virtual space within the user's field of view, the user can only interact with the unoccluded virtual objects through gesture interaction or controller interaction, and cannot interact with the occluded virtual objects, resulting in a poor interaction experience between the user and the virtual objects.

[0053] To solve the above technical problems, the inventive concept of this application is: when a user interacts with virtual objects having an occlusion relationship in a virtual space, the input device detects whether the input information of the user meets a predetermined condition. When the predetermined condition is met, the operation prompt information displayed on the second virtual object is changed so that the operation prompt information corresponds to the first virtual object, enabling the user to interact with the occluded virtual object, thereby enhancing the interaction experience between the user and the virtual objects.

[0054] The technical solutions of this application will be described in detail below through some embodiments. The embodiments described below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0055] Figure 2 It is a flowchart of a virtual object interaction method provided by an embodiment of this application. The virtual object interaction method provided by an embodiment of this application can be executed by a virtual object interaction device. The virtual object interaction device can be composed of hardware and / or software and can be integrated into an electronic device. In this application, the electronic device is various terminal devices including a display component and one or more input devices, such as VR devices, MR devices, or XR devices, etc. It should be understood that the above electronic device can also be referred to as a user device or a user apparatus, etc., and there is no limitation on the electronic device here. For the convenience of describing the technical solutions provided by this application, the following will take the electronic device as an XR device as an example for detailed description.

[0056] As Figure 2 shown, the method includes the following steps:

[0057] S101, display a computer-generated three-dimensional environment on the display component.

[0058] In this application, a display is provided in the display component, and the display is used to receive computer-generated three-dimensional environment data to display a virtual three-dimensional environment to the user based on the three-dimensional environment data.

[0059] The above display can be a single display or two displays. That is, one display can be set for the user's two eyes, or one display can be set for the user's left eye and right eye respectively. This application does not make any restrictions on this.

[0060] Among them, the computer-generated three-dimensional environmental data can be a virtual three-dimensional environment generated by computer simulation.

[0061] Moreover, the virtual three-dimensional environment displayed to the user through the display in the display component can be understood as a virtual scene or virtual space.

[0062] S102, present a first virtual object in the three-dimensional environment.

[0063] S103, display a second virtual object between the first virtual object and the user.

[0064] To facilitate the user to interact with the three-dimensional environment displayed by the XR device, at least one virtual object will be set in the three-dimensional environment displayed by the display component in the XR device, so that the user can interact with the three-dimensional environment based on the virtual object. In this application, the virtual objects presented in the three-dimensional environment can be, but are not limited to: virtual characters, virtual panels, virtual interfaces, virtual controls, or virtual icons, etc.

[0065] Among them, the above-mentioned first virtual object and second virtual object can be two different virtual objects of the same type, or two different virtual objects of different types, and this application does not make any restrictions on this.

[0066] Optionally, when there are multiple virtual objects in the three-dimensional environment, it is possible that one virtual object is blocked by another virtual object. That is, when the second virtual object is displayed between the user and the first virtual object, the second virtual object will block the first virtual object, for details, see Figure 1c .

[0067] S104, detect the first input information of the user via the input device, and display the operation prompt information corresponding to the second virtual object.

[0068] In this application, the input device can be at least one external device communicatively connected to the XR device, such as a handle, a mouse, a touchpad, and a microphone, etc., or it can also be at least one sensor on the XR device, such as a gaze tracking camera for tracking the user's line of sight, a gesture acquisition camera for collecting the user's gestures, and an inertial measurement unit (IMU) for obtaining the user's head pose data, etc.

[0069] Correspondingly, each of the above input devices can correspond to an interaction method. For example, the handle corresponds to the handle interaction method, the microphone corresponds to the voice interaction method, the gaze tracking camera corresponds to the eye movement interaction method, the IMU corresponds to the head interaction, and the gesture acquisition camera corresponds to the gesture interaction, and so on.

[0070] That is, the user can interact with virtual objects in a three-dimensional scene by means of an input device.

[0071] Generally, when it is necessary to interact with any virtual object in a three-dimensional environment, the user can send first input information for interacting with the virtual object to the XR device by means of any of the above input devices, so that the XR device displays corresponding operation prompt information on the virtual object based on the first input information, and then reminds the user that they can interact with the virtual object through the operation prompt information. Among them, the operation prompt information can be operation prompt text, visual indication information, etc., as long as it can play a prompting role, and the present application does not impose any restrictions on the specific display form of the operation prompt information.

[0072] However, when a second virtual object is displayed in front of the first virtual object displayed in the three-dimensional environment, that is, when the first virtual object is blocked by the second virtual object, if the user wants to interact with the first virtual object by means of any input device, they can only interact with the second virtual object located in front of the first virtual object based on the first input information, that is, only the operation prompt information corresponding to the second virtual object can be displayed, thus preventing the user from achieving the purpose of interacting with the first virtual object.

[0073] S105. Detect the second input information of the user via the input device. When the second input information meets a predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0074] In order to be able to interact with the blocked first virtual object normally, the user can adjust the interaction method and send second input information for interacting with the first virtual object to the XR device based on the adjusted interaction method. Furthermore, when the XR device determines that the second input information of the user meets the predetermined condition for interacting with the first virtual object, it can change the operation prompt information corresponding to the second virtual object to display the operation prompt information corresponding to the first virtual object, so that the user can interact with the blocked first virtual object based on the operation prompt information corresponding to the first virtual object.

[0075] In the present application, the second input information refers to information different from the first input information, such as the new first input information obtained by adjusting the first input information, or the input information different from the first input information added on the basis of the first input information, or the input information different from the first input information added on the basis of the new first input information, etc., and the present application does not impose any restrictions on this.

[0076] Moreover, the above-mentioned predetermined conditions are related to the user's interaction intention, that is, the predetermined conditions can be flexibly set based on the user's interaction intention. For example, when the user's interaction intention is to interact with the first virtual object, then the predetermined condition is set based on the intention that the user needs to interact with the first virtual object. For example, the interaction score between the user and the first virtual object is greater than the interaction score between the user and the second virtual object, etc. The present application does not impose any restrictions on the setting of the predetermined conditions.

[0077] It can be understood that in the present application, when there are virtual objects that are mutually occluded in the three-dimensional environment presented to the user, if the user needs to interact with the occluded virtual object, the input information that can interact with the unoccluded virtual object is adjusted, and the occluded virtual object is interacted with based on the adjusted input information, so that the user can interact with the occluded virtual object and improve the user's interaction experience with the virtual object.

[0078] The technical solution disclosed in the embodiments of the present application displays a computer-generated three-dimensional environment on the display component of the electronic device, presents a first virtual object in the three-dimensional environment, displays a second virtual object between the first virtual object and the user, detects the user's first input information via the input device, displays operation prompt information corresponding to the second virtual object, detects the user's second input information via the input device, and when the second input information meets the predetermined conditions, changes the operation prompt information so that the operation prompt information corresponds to the first virtual object, enabling the user to interact with the occluded virtual object, thereby increasing the interaction interest and interaction diversity between the user and the virtual object and enhancing the user's interaction experience with the virtual object.

[0079] Based on the foregoing embodiments, considering that the user's second input information can be the adjusted first input information; or, input information different from the first input information is added on the basis of the first input information; or, input information different from the first input information is added on the basis of the adjusted first input information. Therefore, the following is combined with Figure 3 For the case where the user's second input information is the adjusted first input information, and the interaction process with the first virtual object based on the adjusted first input information is specifically described.

[0080] As Figure 3 shown, the method may include the following steps:

[0081] S201, display a computer-generated three-dimensional environment on the display component.

[0082] S202, present a first virtual object in the three-dimensional environment.

[0083] S203, display a second virtual object between the first virtual object and the user.

[0084] S204. Detect the user's first input information via an input device and display operation prompt information corresponding to the second virtual object.

[0085] S205. Detect the user's second input information via an input device. If the second input information detected via the input device is the adjusted first input information, then determine a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object according to the adjusted first input information.

[0086] In this application, the adjusted first input information can be understood as the new first input information obtained by changing the first input information. Exemplarily, assume that the first input information is that the user emits an interaction ray from position 1 to any virtual object through an input device such as a handle. Then the adjusted first input information can be that the user moves the handle from position 1 to position 2 and emits an interaction ray to the virtual object at position 2.

[0087] In some alternative embodiments, the input device on the XR device will detect in real time whether the user has input information. When detecting an input operation of the user having input information, obtain the user's input information and send the user's input information to the processing module of the XR device, so as to compare the latest obtained input information (the second input information) with the previously obtained input information (the first input information) through the processing module to determine whether the latest obtained input information is consistent with the previously obtained input information. If the input device determines that the latest obtained input information and the previously obtained input information are two different input information in the same interaction mode, then determine that the latest obtained input information is the adjusted first input information. At this time, the XR device can determine a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object based on the adjusted first input information.

[0088] In this application, determining a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object based on the adjusted first input information may include the following implementation manners:

[0089] The first implementation manner may include the following steps:

[0090] Step 11. Determine a first distance between the visual indication information corresponding to the first virtual object and the center point position of the first virtual object, and a second distance between the visual indication information corresponding to the second virtual object and the center point position of the second virtual object.

[0091] Step 12. Determine the first intention score according to the first distance and determine the second intention score according to the second distance.

[0092] The visual indication information is the operation prompt information. In the present application, the visual indication information can be understood as information that can be recognized by the user's eyes, and can be selected as graphics with different color attributes and / or graphics with different transmission frequency attributes, such as cursors of different colors or cursors with different flashing frequencies, etc. The present application does not impose any restrictions on this. The shape of the cursor can be circular, cross-shaped, or hand-shaped, etc.

[0093] Considering that the occlusion relationship between virtual objects in a three-dimensional environment includes: partial occlusion and complete occlusion. Partial occlusion should be understood as the second virtual object occluding a small area of ​​the first virtual object, such as Figure 4a As shown; complete occlusion should be understood as the second virtual object occluding most of the area of ​​the first virtual object or even almost completely occluding it, as shown in Figure 4b shown.

[0094] Then, when the user interacts with a virtual object in an occlusion relationship by means of a handle or gesture, the visual indication information projected onto the virtual object by the ray emitted by the handle or gesture may include the following two situations:

[0095] In case 1, when it is determined that the second virtual object partially blocks the first virtual object, the ray clone mode is used to project the ray emitted by the handle or gesture onto the first virtual object and the second virtual object, see Figure 5a shown.

[0096] In case 2, when it is determined that the second virtual object completely blocks the first virtual object, the ray splitting mode is used to project the ray emitted by the handle or gesture to the first virtual object and the second virtual object, see Figure 5b shown.

[0097] To determine whether the second virtual object partially or completely obscures the first virtual object, the display area in which the first virtual object and the second virtual object are displayed in the three-dimensional environment can be obtained through the XR device, and the occlusion relationship between the first virtual object and the second virtual object can be determined based on the display area. The specific implementation process can be referred to the prior art, and will not be elaborated on here.

[0098] Considering that the user projects a ray into the three-dimensional environment through an interactive method such as a handle, the ray will intersect with the first virtual object and the second virtual object that have an occlusion relationship, and each intersection will display a visual indication information on the corresponding virtual object. Therefore, the present application can obtain the intersection positions of the above-mentioned ray with the first virtual object and the second virtual object respectively, and determine the intersection position as the location of the sight indication information.

[0099] Furthermore, by determining a first distance between the center point position of the first virtual object and the position where the visual indication information corresponding to the first virtual object is located, and determining a second distance between the center point position of the second virtual object and the position where the visual indication information corresponding to the second virtual object is located. Then, the first distance is determined as the first intention score for the user to interact with the first virtual object, and the second distance is determined as the second intention score for the user to interact with the second virtual object.

[0100] Among them, determining the first distance between the center point position of the first virtual object and the position where the visual indication information corresponding to the first virtual object is located, and determining the second distance between the center point position of the second virtual object and the position where the visual indication information corresponding to the second virtual object is located, specifically, it is to determine the distance between the center point position of the virtual object and the center point position of the visual indication information, and the two-point distance calculation formula can be used to calculate the above-mentioned first distance and the above-mentioned second distance respectively.

[0101] The second implementation manner may include the following steps:

[0102] Step 21, determine the display position of the operation prompt information corresponding to the adjusted first input information.

[0103] Step 22, determine the first intention score and the second intention score according to the display position of the operation prompt information.

[0104] When the first virtual object in the three-dimensional environment is blocked by the second virtual object, the user cannot directly interact with the blocked first virtual object through any interaction method. At this time, the user can move their own position or other ways to bypass the second virtual object, so that the user can see the first virtual object to be interacted with. Furthermore, then send the second input information (adjusted first input information) for interacting with the first virtual object to the XR device through any interaction method, so that the XR device can display the operation prompt information corresponding to the first virtual object based on the second input information.

[0105] That is to say, when the first virtual object that the user wants to interact with is blocked by the second virtual object, the user moves or bypasses the second virtual object in other ways to see the first virtual object, and then directly interacts with the first virtual object.

[0106] Since after bypassing the second virtual object, the user can directly see the first virtual object, the operation prompt information corresponding to the adjusted first input information can be directly displayed on the first virtual object, rather than on the second virtual object. Therefore, the present application can determine the first intention score and the second intention score according to the display position of the operation prompt information corresponding to the adjusted first input information.

[0107] In some embodiments, when the operation prompt information corresponding to the adjusted first input information is located on the first virtual object, the first intention score corresponding to the first virtual object can be determined to be 1, while there is no operation prompt information displayed on the second virtual object, so the first intention score corresponding to the second virtual object can be determined to be 0.

[0108] S206. Determine whether the second input information meets a predetermined condition according to the first intention score and the second intention score. If it meets the predetermined condition, execute S207; otherwise, display the operation prompt information corresponding to the second virtual object.

[0109] In some alternative embodiments, by comparing the first intention score and the second intention score, when the first intention score is greater than the second intention score, it indicates that the condition for the user to interact with the first virtual object is met, and at this time, it is determined that the second input information meets the predetermined condition; when the first intention score is less than or equal to the second intention score, it indicates that the condition for the user to interact with the first virtual object is not met, and at this time, it is determined that the second input information does not meet the predetermined condition.

[0110] That is to say, in this application, the predetermined condition is determined based on the virtual object that the user actually needs to interact with. For example, when the user wants to interact with the occluded first virtual object, the predetermined condition is set such that the user's input information meets the rules for the user to interact with the first virtual object.

[0111] In some alternative embodiments, in order to help the user interact with the first virtual object to be interacted with, when it is determined in this application that the first intention score is equal to the second intention score, a prompt message can also be output to the user, so that the user can independently select the target interaction object to interact with from the first virtual object and the second virtual object based on the prompt message.

[0112] In this application, the prompt message can be in the form of voice and / or text, etc. Among them, when the prompt message is a text prompt message, it can be displayed in the vicinity of the first virtual object and / or the second virtual object, or in the blank area within the three-dimensional scene.

[0113] Exemplarily, as Figure 6 shown, assuming that the first intention score corresponding to the first virtual object Y1 is equal to the second intention score corresponding to the second virtual object Y2, then a text prompt floating window of "Please select the target interaction object to interact with" is displayed near the first virtual object Y1, and the floating window also includes: a control for the first virtual object Y1 and a control for the second virtual object Y2, so that the user can select the control for the first virtual object Y1 to select the target interaction object.

[0114] S207. When the second input information meets the predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0115] Optionally, the operation prompt information corresponding to the first virtual object can be adjusted from the first display form to the second display form, so that the user can determine that the first virtual object has been selected based on the adjusted operation prompt information, and then various interactions can be performed with the first virtual object.

[0116] Exemplarily, assume that there is an occlusion relationship between virtual object X1 and virtual object X2, and virtual object X1 is partially occluded by virtual object X2. Then, when it is determined that virtual object X1 is the object to be interacted with that the user wants to interact with, virtual object X1 and the cursor on virtual object X1 can be highlighted, and virtual object X2 and the cursor on virtual object X2 can be blurred. Refer to Figure 7 ... Or, virtual object X1 and the cursor on virtual object X1 can also be highlighted, virtual object X2 can be blurred, and the cursor is not displayed on virtual object X2, so that the user can intuitively see that virtual object X1 is the object to be interacted with, and thus interact with virtual object X1. It should be understood that the above cursor is a form of manifestation of the operation prompt information as visual indication information.

[0117] The technical solution disclosed in the embodiments of the present application displays a computer-generated three-dimensional environment on the display component of the electronic device, presents a first virtual object in the three-dimensional environment, displays a second virtual object between the first virtual object and the user, detects the first input information of the user via the input device, displays the operation prompt information corresponding to the second virtual object, detects the second input information of the user via the input device, and when the second input information meets a predetermined condition, changes the operation prompt information so that the operation prompt information corresponds to the first virtual object, enabling the user to interact with the occluded virtual object, thereby increasing the interaction interest and interaction diversity between the user and the virtual object, and enhancing the interaction experience between the user and the virtual object.

[0118] Next, in combination with Figure 8 ..., when the second input information of the user is input information different from the first input information added on the basis of the first input information, the process of interacting with the first virtual object based on the second input information will be specifically described.

[0119] As Figure 8 shown, the method may include the following steps:

[0120] S301, display a computer-generated three-dimensional environment on the display component.

[0121] S302, present a first virtual object in the three-dimensional environment.

[0122] S303, display a second virtual object between the first virtual object and the user.

[0123] S304. Detect the first input information of the user via the input device, and display the operation prompt information corresponding to the second virtual object.

[0124] S305. Detect the second input information of the user through the input device. If the second input information of the user detected via the input device is the first input information and the third input information, then determine the first interaction score between the user and the first virtual object and the second interaction score between the user and the second virtual object according to the first input information and the third input information.

[0125] In this application, the type of the third input information is different from that of the first input information. For example, if the type of the first input information is the ray emitted by the handle, then the third input information can be the user's line of sight corresponding to the eye movement interaction method of the user. Another example is that if the type of the first input information is gesture interaction, then the third input information can be the user's head interaction method, etc.

[0126] In some optional embodiments, considering that there are multiple ways to interact with virtual objects, when the user interacts with the occluded first virtual object, multiple interaction methods can also be superimposed to send the second input information to the XR device, so as to achieve the purpose that when the first virtual object is occluded by the second virtual object, the user can interact with the first virtual object without moving or using other methods to bypass the second virtual object that occludes the first virtual object, thereby improving the user's interaction experience with the virtual object.

[0127] As an optional implementation method, the input device on the XR device can detect in real time whether the user has input information. When detecting an input operation of the user with input information, obtain the input information of the user, and send the input information to the processing module of the XR device, so as to compare the latest obtained input information (the second input information) with the previously obtained input information (the first input information) through the processing module to determine whether the latest obtained input information is consistent with the previously obtained input information. If the input device determines that the latest obtained input information includes the input information under other interaction methods on the basis of including the same input information as the previously obtained one, then determine that the latest obtained input information is the first input information + the third input information. At this time, the XR device can determine the first interaction score between the user and the first virtual object and the second interaction score between the user and the second virtual object based on the first input information and the third input information.

[0128] In some optional embodiments, this application determines the first interaction score between the user and the first virtual object and the second interaction score between the user and the second virtual object according to the first input information and the third input information, which may include the following steps:

[0129] Step 31, determine the weight value of the first input information, the weight value of the third input information, the weight value of the first virtual object, and the weight value of the second virtual object.

[0130] In this application, the weight values corresponding to each virtual object and each input information interaction method can be pre-configured in the XR device. Among them, the weight value of each interaction method can be understood as the weight value of the input information corresponding to each interaction method.

[0131] Moreover, the above virtual object weight values can be flexibly set according to the attributes of the virtual objects. The attributes of the virtual objects in this application may include but are not limited to: the display position of the virtual object, the content of the virtual object itself, and the practicality of the virtual object.

[0132] The weight values of the above interaction methods can be flexibly set according to the priority or attributes of the interaction methods, and this application does not impose any restrictions on this.

[0133] Furthermore, in order to meet the requirements of different application scenarios, different weight values can be set for the first virtual object, the second virtual object, and each interaction method according to the application attributes of each application scenario, and the weight values also support update operations.

[0134] For example, when the application scenario is a live broadcast scenario, and the interaction methods include: gesture interaction, eye movement interaction, voice interaction, and head interaction, then the weight value of the first virtual object in the live broadcast scenario can be set to 1, the weight value of the second virtual object can be set to 0.5, and the following weight values can be set for each interaction method: the weight value of gesture interaction is 1, the weight value of voice interaction is 0.8, the weight value of eye movement interaction is 0.6, and the weight value of head interaction is 0.4. Another example, when the application scenario is an office scenario, and the interaction methods include: handle interaction, eye movement interaction, voice interaction, and head interaction, then the weight value of the first virtual object in the office scenario can be set to 1, the weight value of the second virtual object can be set to 0.8, and the following weight values can be set for each interaction method: the weight value of eye movement interaction is 1, the weight value of handle interaction is 0.9, the weight value of head interaction is 0.8, and the weight value of voice interaction is 0.7, and so on.

[0135] That is to say, when this application obtains the weight value of the first virtual object, the weight value of the second virtual object, the weight value of the first input information, and the weight value of the third input information, it first determines the content scene type presented in the three-dimensional environment, and then obtains the weight values of each virtual object corresponding to the scene type and the weight values of the interaction methods corresponding to the first input information and the weight values of the interaction methods corresponding to the third input information from the pre-configured weight value list according to the scene type.

[0136] Step 32, determine the first interaction score according to the weight value of the first input information, the weight value of the third input information, and the weight value of the first virtual object.

[0137] Step 33: Determine a second interaction score according to the first input information weight value, the third input information weight value, and the second virtual object weight value.

[0138] After obtaining the first input information weight value, the third input information weight value, the first virtual object weight value, and the second virtual object weight value, the present application can calculate a first interaction score between the user and the first virtual object based on the first input information weight value, the third input information weight value, and the first virtual object weight value, and calculate a second interaction score between the user and the second virtual object based on the first input information weight value, the third input information weight value, and the second virtual object weight value.

[0139] Exemplarily, as Figure 9 shown, assuming that the interaction methods corresponding to the second input information for the first virtual object include: eye movement interaction and gesture interaction, and the interaction methods corresponding to the second input information for the second virtual object include: gesture interaction. Then, when the first virtual object weight value is 0.5, the second virtual object weight value is 1, the eye movement interaction weight value is 1, and the gesture interaction weight value is 0.8, the calculated first interaction score corresponding to the first virtual object is: (1 * 0.5) + (1 * 1) + (1 * 0.8) = 2.3, and the second interaction score corresponding to the second virtual object is: (1 * 1) + (1 * 0.8) = 1.8.

[0140] In some alternative embodiments, considering that during the transmission of rays corresponding to handle interaction or gesture interaction, etc., the signal energy will be lost as the distance increases. Therefore, the present application can also set different weight values for rays with different ray transmission distances. For example, the weight value of a ray intersecting a nearby virtual object is 1, and the weight value of a ray intersecting a distant virtual object is 0.5.

[0141] Furthermore, when calculating the first interaction score corresponding to the first virtual object and the second interaction score corresponding to the second virtual object, the weight value of the ray can also be considered, so as to further improve the accuracy and reliability of the user selecting the first virtual object as the target interaction object from virtual objects with an occlusion relationship.

[0142] It should be noted that the execution order of the above step 32 and step 33 can be to execute step 32 first and then step 33, or execute step 33 first and then step 32, or execute step 32 and step 33 synchronously. The present application does not impose any restrictions on this.

[0143] S306. Determine whether the second input information meets a predetermined condition based on the first interaction score and the second interaction score. If the predetermined condition is met, execute S307; otherwise, display operation prompt information corresponding to the second virtual object.

[0144] In some alternative embodiments, by comparing the first interaction score and the second interaction score, when the first interaction score is greater than the second interaction score, it indicates that the condition for the user to interact with the first virtual object is met, and at this time, it is determined that the second input information meets the predetermined condition; when the first interaction score is less than or equal to the second interaction score, it indicates that the condition for the user to interact with the first virtual object is not met, and at this time, it is determined that the second input information does not meet the predetermined condition.

[0145] S307. When the second input information meets the predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0146] The technical solution disclosed in the embodiments of the present application displays a computer-generated three-dimensional environment on the display component of the electronic device, presents a first virtual object in the three-dimensional environment, displays a second virtual object between the first virtual object and the user, detects the first input information of the user via the input device, displays operation prompt information corresponding to the second virtual object, detects the second input information of the user via the input device, and when the second input information meets the predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object, enabling the user to interact with the occluded virtual object, thereby increasing the interaction interest and interaction diversity between the user and the virtual object and enhancing the user's interaction experience with the virtual object. In addition, the present application adds other interaction methods on the basis of the unimodal interaction method to provide a multimodal interaction method to guide the user to interact with the occluded virtual object, meeting the user's usage requirement for directly interacting with the occluded virtual object.

[0147] The following combines Figure 10 to specifically describe the process of interacting with the first virtual object based on the second input information of the user when the second input information is the input information different from the first input information added on the basis of the adjusted first input information.

[0148] As Figure 10 shown, the method may include the following steps:

[0149] S401. Display a computer-generated three-dimensional environment on the display component.

[0150] S402. Present a first virtual object in the three-dimensional environment.

[0151] S403. Display a second virtual object between the first virtual object and the user.

[0152] S404. Detect the user's first input information via the input device and display operation prompt information corresponding to the second virtual object.

[0153] S405. Detect the user's second input information via the input device. If the second input information detected via the input device is the adjusted first input information and the third input information, then determine the third interaction score between the user and the first virtual object and the fourth interaction score between the user and the second virtual object according to the adjusted first input information and the third input information.

[0154] In some alternative embodiments, the input device on the XR device can detect in real time whether the user has input information. When detecting an input operation of the user having input information, obtain the user's input information and send the input information to the processing module of the XR device, so as to compare the latest obtained input information (the second input information) with the previously obtained input information (the first input information) through the processing module to determine whether the latest obtained input information is consistent with the previously obtained input information. If the input device determines that the latest obtained input information includes, in addition to the adjusted first input information, input information under other interaction methods, then determine that the latest obtained input information is the adjusted first input information + the third input information. At this time, the XR device can determine the third interaction score between the user and the first virtual object and the fourth interaction score between the user and the second virtual object based on the adjusted first input information and the third input information.

[0155] In some alternative embodiments, the present application determines the third interaction score between the user and the first virtual object and the fourth interaction score between the user and the second virtual object according to the adjusted first input information and the third input information, which may include the following steps:

[0156] Step 41. Determine the weight value of the adjusted first input information, the weight value of the third input information, the weight value of the first virtual object, and the weight value of the second virtual object.

[0157] Step 42. Determine the third interaction score according to the weight value of the first input information, the weight value of the third input information, and the weight value of the first virtual object.

[0158] Step 43. Determine the second interaction score according to the weight value of the first input information, the weight value of the third input information, and the weight value of the second virtual object.

[0159] Among them, the implementation principles of steps 41 to 43 are the same as or similar to those of steps 31 to 33 in the foregoing embodiments. The specific implementation process can refer to the foregoing embodiments, and no further limitation is imposed herein.

[0160] S406. Determine whether the second input information meets a predetermined condition according to the third interaction score and the fourth interaction score. If the predetermined condition is met, execute S407; otherwise, display operation prompt information corresponding to the second virtual object.

[0161] In some alternative embodiments, by comparing the third interaction score and the fourth interaction score, when the third interaction score is greater than the fourth interaction score, it indicates that the condition for the user to interact with the first virtual object is met, and at this time, it is determined that the second input information meets the predetermined condition; when the third interaction score is less than or equal to the fourth interaction score, it indicates that the condition for the user to interact with the first virtual object is not met, and at this time, it is determined that the second input information does not meet the predetermined condition.

[0162] S407. When the second input information meets the predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0163] The technical solution disclosed in the embodiments of the present application displays a computer-generated three-dimensional environment on the display component of the electronic device, presents a first virtual object in the three-dimensional environment, displays a second virtual object between the first virtual object and the user, detects the first input information of the user via the input device, displays operation prompt information corresponding to the second virtual object, detects the second input information of the user via the input device, and when the second input information meets the predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object, enabling the user to interact with the occluded virtual object, thereby increasing the interactivity and diversity of the interaction between the user and the virtual object and enhancing the interaction experience between the user and the virtual object. In addition, the present application adds other interaction methods on the basis of the single-modal interaction method to provide a multi-modal interaction method to guide the user to interact with the occluded virtual object, meeting the user's usage requirement for directly interacting with the occluded virtual object.

[0164] The following refers to the attached Figure 11 , and describes a virtual object interaction device proposed in the embodiments of the present application. Figure 11 FIG. is a schematic block diagram of a virtual object interaction device provided in an embodiment of the present application. In the present application, the virtual object interaction device is configured in an electronic device, and the electronic device includes a display component and one or more input devices.

[0165] As Figure 11 shown, the virtual object interaction device 500 includes: a display module 510 and a change module 520.

[0166] Among them, a display module 510 is configured to display a computer-generated three-dimensional environment on a display component; present a first virtual object within the three-dimensional environment; display a second virtual object between the first virtual object and a user; detect first input information of the user via the input device, and display operation prompt information corresponding to the second virtual object;

[0167] A change module 520 is configured to detect second input information of the user via the input device, and when the second input information meets a predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

[0168] In an optional implementation manner of the embodiment of the present application, the second input information includes:

[0169] The adjusted first input information; or,

[0170] The first input information and third input information; or,

[0171] The adjusted first input information and the third input information;

[0172] Among them, the third input information is of a different type from the first input information.

[0173] In an optional implementation manner of the embodiment of the present application, the predetermined condition is related to the interaction intention of the user.

[0174] In an optional implementation manner of the embodiment of the present application, the change module 502 includes:

[0175] A first determination unit is configured to, if the second input information of the user detected via the input device is the adjusted first input information, determine a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object according to the adjusted first input information;

[0176] A second determination unit is configured to determine whether the second input information meets the predetermined condition according to the first intention score and the second intention score;

[0177] A change unit is configured to change the operation prompt information when the second input information meets the predetermined condition.

[0178] In an alternative implementation of the embodiment of the present application, when the operation prompt information is visual indication information, the first determination unit is specifically configured to: determine a first distance between the visual indication information corresponding to the first virtual object and the center point position of the first virtual object, and a second distance between the visual indication information corresponding to the second virtual object and the center point position of the second virtual object; determine the first intention score according to the first distance, and determine the second intention score according to the second distance.

[0179] In an alternative implementation of the embodiment of the present application, when the operation prompt information is visual indication information, the first determination unit is specifically configured to: determine the display position of the operation prompt information corresponding to the adjusted first input information; determine the first intention score and the second intention score according to the display position of the operation prompt information.

[0180] In an alternative implementation of the embodiment of the present application, the second determination unit is specifically configured to: when the first intention score is greater than the second intention score, determine that the second input information meets a predetermined condition; when the first intention score is less than or equal to the second intention score, determine that the second input information does not meet the predetermined condition.

[0181] In an alternative implementation of the embodiment of the present application, the change unit is specifically configured to: adjust the operation prompt information corresponding to the first virtual object from a first display form to a second display form.

[0182] In an alternative implementation of the embodiment of the present application, the first determination unit is further configured to if it is detected via the input device that the second input information of the user is the first input information and the third input information, then determine a first interaction score between the user and the first virtual object and a second interaction score between the user and the second virtual object according to the first input information and the third input information;

[0183] The second determination unit is further configured to determine whether the second input information meets a predetermined condition according to the first interaction score and the second interaction score;

[0184] The change unit is further configured to change the operation prompt information when the second input information meets a predetermined condition.

[0185] An alternative implementation of the embodiment of the present application, the first determination unit is further configured to determine a first input information weight value, a third input information weight value, a first virtual object weight value, and a second virtual object weight value; determine the first interaction score according to the first input information weight value, the third input information weight value, and the first virtual object weight value; determine the second interaction score according to the first input information weight value, the third input information weight value, and the second virtual object weight value.

[0186] An alternative implementation of the embodiment of the present application, the second determination unit is further configured to determine that the second input information meets a predetermined condition when the first interaction score is greater than the second interaction score; determine that the second input information does not meet the predetermined condition when the first interaction score is less than or equal to the second interaction score.

[0187] An alternative implementation of the embodiment of the present application, the first determination unit is further configured to, if it is detected via the input device that the second input information of the user is the adjusted first input information and the third input information, determine a third interaction score between the user and the first virtual object and a fourth interaction score between the user and the second virtual object according to the adjusted first input information and the third input information;

[0188] The second determination unit is further configured to determine whether the second input information meets a predetermined condition according to the third interaction score and the fourth interaction score;

[0189] The change unit is further configured to change the operation prompt information when the second input information meets a predetermined condition.

[0190] It should be understood that the device embodiment and the foregoing method embodiment can correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, it will not be elaborated here. Specifically, Figure 11 the shown device 500 can execute Figure 2 the corresponding method embodiment, and the foregoing and other operations and / or functions of each module in the device 500 are respectively for implementing Figure 2 the corresponding processes in each method, and for the sake of brevity, it will not be elaborated here.

[0191] In the foregoing, the apparatus 500 according to the embodiments of the present application has been described from the perspective of functional modules with reference to the accompanying drawings. It should be understood that these functional modules can be implemented in the form of hardware, or in the form of instructions in software, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments of the first aspect in the embodiments of the present application can be completed by the integrated logic circuit in the hardware in the processor and / or instructions in the form of software. The steps of the method of the first aspect disclosed in the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps in the method embodiments of the first aspect described above.

[0192] Figure 12 FIG. is a schematic block diagram of an electronic device provided by an embodiment of the present application. As Figure 12 shown, the electronic device 600 may include:

[0193] A memory 610 and a processor 620. The memory 610 is used to store a computer program and transmit the program code to the processor 620. In other words, the processor 620 can call and run the computer program from the memory 610 to implement the virtual object interaction method in the embodiments of the present application.

[0194] For example, the processor 620 can be used to execute the virtual object interaction method embodiments according to the instructions in the computer program.

[0195] In some embodiments of the present application, the processor 320 may include, but is not limited to:

[0196] A general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and the like.

[0197] In some embodiments of the present application, the memory 610 includes, but is not limited to:

[0198] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be Read-Only Memory (ROM), Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. The volatile memory can be Random Access Memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double DataRate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0199] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 610 and executed by the processor 620 to complete the virtual object interaction method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0200] As Figure 12 shown, the electronic device 600 may further include:

[0201] A transceiver 630, which may be connected to the processor 620 or the memory 610.

[0202] Among them, the processor 620 may control the transceiver 630 to communicate with other devices. Specifically, it may send information or data to other devices, or receive information or data sent by other devices. The transceiver 630 may include a transmitter and a receiver. The transceiver 630 may further include an antenna, and the number of antennas may be one or more.

[0203] It should be understood that the various components in the electronic device are connected through a bus system. Among them, the bus system includes not only a data bus, but also a power bus, a control bus, and a status signal bus.

[0204] The present application also provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a computer, the computer can execute the virtual object interaction method in the above method embodiments.

[0205] The embodiments of the present application also provide a computer program product containing program instructions. When the program instructions run on an electronic device, the electronic device is enabled to execute the virtual object interaction method in the above method embodiments.

[0206] When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, etc. that integrates one or more available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0207] Those of ordinary skill in the art can realize that the modules and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or by a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0208] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or modules can be in electrical, mechanical, or other forms.

[0209] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, in each embodiment of this application, the functional modules can be integrated in a processing module, or each module can exist physically alone, or two or more modules can be integrated in one module.

[0210] In the embodiments of this application, the term "module" or "unit" refers to a computer program with a predetermined function or a part of a computer program, which works together with other related parts to achieve a predetermined goal, and can be fully or partially implemented by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit that includes the function of that module or unit.

[0211] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A virtual object interaction method, characterized in that, Applied to an electronic device, the electronic device including a display component and one or more input devices, the method comprising: Displaying a computer-generated three-dimensional environment on the display component; Presenting a first virtual object within the three-dimensional environment; Displaying a second virtual object between the first virtual object and the user; Detecting first input information of the user via the input device and displaying operation prompt information corresponding to the second virtual object; Detecting second input information of the user via the input device, and when the second input information meets a predetermined condition, changing the operation prompt information so that the operation prompt information corresponds to the first virtual object.

2. The method according to claim 1, characterized in that, The second input information includes: Adjusted first input information; or, The first input information and third input information; or, The adjusted first input information and the third input information; wherein the third input information is of a different type from the first input information.

3. The method according to claim 2, characterized in that, The predetermined condition is related to the interaction intention of the user.

4. The method according to claim 3, characterized in that, The detecting the second input information of the user via the input device and, when the second input information meets a predetermined condition, changing the operation prompt information includes: If the second input information of the user detected via the input device is the adjusted first input information, determining a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object according to the adjusted first input information; Determining whether the second input information meets the predetermined condition according to the first intention score and the second intention score; When the second input information meets the predetermined condition, changing the operation prompt information.

5. The method according to claim 4, characterized in that, When the operation prompt information is visual indication information, the determining a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object according to the adjusted first input information includes: Determining a first distance between visual indication information corresponding to the first virtual object and the center point position of the first virtual object, and a second distance between visual indication information corresponding to the second virtual object and the center point position of the second virtual object; Determining the first intention score according to the first distance and determining the second intention score according to the second distance.

6. The method according to claim 4, characterized in that, When the operation prompt information is visual indication information, the determining a first intention score for the user to interact with the first virtual object and a second intention score for the user to interact with the second virtual object according to the adjusted first input information includes: Determining the display position of the operation prompt information corresponding to the adjusted first input information; Determining the first intention score and the second intention score according to the display position of the operation prompt information.

7. The method according to claim 4, characterized in that, The determining whether the second input information meets the predetermined condition according to the first intention score and the second intention score includes: When the first intention score is greater than the second intention score, determining that the second input information meets the predetermined condition; When the first intention score is less than or equal to the second intention score, it is determined that the second input information does not meet the predetermined condition.

8. The method according to claim 4, characterized in that, When the second input information meets the predetermined condition, changing the operation prompt information includes: Adjusting the operation prompt information corresponding to the first virtual object from a first display form to a second display form.

9. The method according to claim 3, characterized in that, Detecting the second input information of the user via the input device, and when the second input information meets the predetermined condition, changing the operation prompt information includes: If the second input information of the user detected via the input device is the first input information and the third input information, then according to the first input information and the third input information, determine a first interaction score between the user and the first virtual object and a second interaction score between the user and the second virtual object; According to the first interaction score and the second interaction score, determine whether the second input information meets the predetermined condition; When the second input information meets the predetermined condition, change the operation prompt information.

10. The method according to claim 9, wherein The determining the first interaction score between the user and the first virtual object and the second interaction score between the user and the second virtual object according to the first input information and the third input information includes: Determine a first input information weight value, a third input information weight value, a first virtual object weight value, and a second virtual object weight value; According to the first input information weight value, the third input information weight value, and the first virtual object weight value, determine the first interaction score; According to the first input information weight value, the third input information weight value, and the second virtual object weight value, determine the second interaction score.

11. The method according to claim 9, wherein The determining whether the second input information meets the predetermined condition according to the first interaction score and the second interaction score includes: When the first interaction score is greater than the second interaction score, determine that the second input information meets the predetermined condition; When the first interaction score is less than or equal to the second interaction score, determine that the second input information does not meet the predetermined condition.

12. The method according to claim 3, wherein Detecting the second input information of the user via the input device, and when the second input information meets the predetermined condition, changing the operation prompt information includes: If the second input information of the user detected via the input device is the adjusted first input information and the third input information, then according to the adjusted first input information and the third input information, determine a third interaction score between the user and the first virtual object and a fourth interaction score between the user and the second virtual object; According to the third interaction score and the fourth interaction score, determine whether the second input information meets the predetermined condition; When the second input information meets the predetermined condition, change the operation prompt information.

13. A virtual object interaction device, wherein Configured in an electronic device, the electronic device includes a display component and one or more input devices, including: A display module, configured to display a computer-generated three-dimensional environment on a display component; present a first virtual object within the three-dimensional environment; display a second virtual object between the first virtual object and a user; detect first input information of the user via the input device, and display operation prompt information corresponding to the second virtual object; A change module, configured to detect second input information of the user via the input device, and when the second input information meets a predetermined condition, change the operation prompt information so that the operation prompt information corresponds to the first virtual object.

14. An electronic device, wherein Comprising: A processor and a memory, where the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the virtual object interaction method according to any one of claims 1 to 12.

15. A computer-readable storage medium, wherein For storing a computer program, which causes a computer to execute the virtual object interaction method according to any one of claims 1 to 12.

Images