Human-computer interaction method and apparatus, device and medium

Abstract

Provided in the present application are a human-computer interaction method and apparatus, a device and a medium. The method comprises: when an interaction mode triggered by a user in a virtual space is a preset interaction mode, controlling, according to the preset interaction mode, a first visual indication identifier to move, so as to select a target interaction object from at least one interaction object in the virtual space; in response to a first input operation of an input device, controlling the first visual indication identifier to interact with the target interaction object; in response to a movement operation of the input device, switching the first visual indication identifier to a second visual indication identifier; and, in response to a second input operation of the input device, controlling the second visual indication identifier to interact with the target interaction object. The present application can enrich human-computer interaction modes and improve human-computer interaction efficiency, thereby satisfying interaction requirements in different scenarios.

Description

Human-computer interaction methods, devices, equipment and media

[0001] This application claims priority to Chinese Patent Application No. 202410512870.0, filed on April 26, 2024, entitled "Human-Computer Interaction Method, Apparatus, Device and Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of human-computer interaction technology, and in particular to a human-computer interaction method, apparatus, device, and medium. Background Technology

[0003] With the development of electronic and computer technologies, various electronic devices have brought convenience to people's daily lives. For example, extended reality (XR) devices can provide users with a variety of virtual scenes, allowing users to immerse themselves in the virtual scene and interact with objects within it.

[0004] Currently, the main ways users interact with objects in virtual scenes include eye-tracking interaction, gesture interaction, controller interaction, and voice interaction. While these methods can meet basic interaction needs, they are difficult to meet the interaction requirements in complex environments or specific scenarios, and thus have certain limitations. Summary of the Invention

[0005] This application provides a human-computer interaction method, apparatus, device, and medium, which can enrich human-computer interaction methods, improve human-computer interaction efficiency, and thus meet the interaction needs in different scenarios.

[0006] In a first aspect, embodiments of this application provide a human-computer interaction method, comprising: when a user triggers an interaction mode in a virtual space that is a preset interaction mode, controlling a first visual indicator to move according to the preset interaction mode to select a target interaction object from at least one interaction object in the virtual space; responding to a first input operation of an input device, controlling the first visual indicator to interact with the target interaction object; responding to a movement operation of the input device, switching the first visual indicator to a second visual indicator; and responding to a second input operation of the input device, controlling the second visual indicator to interact with the target interaction object.

[0007] Secondly, embodiments of this application provide a human-computer interaction device, comprising: a movement control module, configured to control a first visual indicator to move according to a preset interaction mode triggered by a user in a virtual space, so as to select a target interactive object from at least one interactive object in the virtual space; a first interaction module, configured to control the first visual indicator to interact with the target interactive object in response to a first input operation of an input device; a switching control module, configured to switch the first visual indicator to a second visual indicator in response to a movement operation of the input device; and a second interaction module, configured to control the second visual indicator to interact with the target interactive object in response to a second input operation of the input device.

[0008] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to perform the human-computer interaction method as described in the first aspect embodiment or its various implementations.

[0009] Fourthly, embodiments of this application provide a computer-readable storage medium for storing a computer program that causes a computer to perform the human-computer interaction method as described in the first aspect embodiment or its various implementations.

[0010] Fifthly, embodiments of this application provide a computer program product containing program instructions that, when executed on an electronic device, cause the electronic device to perform the human-computer interaction method as described in the first aspect embodiment or its various implementations. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of this application;

[0013] Figure 2 is a flowchart of a human-computer interaction method provided in an embodiment of this application;

[0014] Figure 3a is a schematic diagram of a virtual panel provided in an embodiment of this application;

[0015] Figure 3b is a schematic diagram of a three-dimensional object in a virtual panel provided in an embodiment of this application;

[0016] Figure 4a is a schematic diagram of displaying a second visual indicator on a target interactive object according to an embodiment of this application;

[0017] Figure 4b is a schematic diagram of another method for displaying a second visual indicator on a target interactive object according to an embodiment of this application;

[0018] Figure 4c is a schematic diagram of determining a target interactive object from multiple interactive objects and determining the display position of a second visual indicator on the target interactive object according to an embodiment of this application;

[0019] Figure 5 is a flowchart of another human-computer interaction method provided in an embodiment of this application;

[0020] Figure 6 is a flowchart of another human-computer interaction method provided in an embodiment of this application;

[0021] Figure 7 is a schematic diagram of moving a second visual indicator out of the user's field of vision and moving a second visual indicator into the user's field of vision, according to an embodiment of this application.

[0022] Figure 8 is a schematic diagram of displaying visual cues to a user according to an embodiment of this application;

[0023] Figure 9 is a schematic diagram of interactive processing corresponding to changes in the user's field of view, provided in an embodiment of this application.

[0024] Figure 10 is a top view of the relative positional relationship between an interactive object and a user in a virtual space, provided in an embodiment of this application;

[0025] Figure 11 is a schematic block diagram of a human-computer interaction device provided in an embodiment of this application; and

[0026] Figure 12 is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0029] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or solution described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0030] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more, that is, at least two. "At least one" means one or more.

[0031] To facilitate understanding of the embodiments of this application, before describing the various embodiments of this application, some concepts involved in all embodiments of this application will be appropriately explained as follows.

[0032] 1) Virtual Reality (VR) is a technology for creating and experiencing virtual worlds. It defines and generates a virtual environment, which is a multi-source information (virtual reality, as mentioned in this article, includes at least visual perception, and may also include auditory perception, tactile perception, motion perception, and even taste and smell perception). It achieves a fusion of interactive, three-dimensional dynamic visual scenes and simulation of physical behavior within a virtual environment, immersing users in the simulated virtual reality environment. This enables applications in various virtual environments such as maps, games, videos, education, healthcare, simulation, collaborative training, sales, assisted manufacturing, maintenance, and repair.

[0033] 2) VR devices are terminals that enable virtual reality effects. They are typically available in the form of glasses, head-mounted displays (HMDs), or contact lenses to achieve visual perception and other forms of perception. Of course, VR devices are not limited to these forms and can be further miniaturized or enlarged depending on actual needs.

[0034] Optionally, the VR devices described in the embodiments of this application may include, but are not limited to, the following types.

[0035] 2.1) PC-based virtual reality (PCVR) devices utilize a PC for calculations and data output related to virtual reality functions. External PC-based virtual reality devices use the data output from the PC to achieve virtual reality effects.

[0036] 2.2) Mobile virtual reality devices support setting up mobile terminals (such as smartphones) in various ways (such as head-mounted displays with dedicated card slots). Through wired or wireless connection with the mobile terminal, the mobile terminal performs relevant calculations for virtual reality functions and outputs data to the mobile virtual reality device, such as watching virtual reality videos through the mobile terminal's APP.

[0037] 2.3) All-in-one virtual reality devices have processors for performing virtual functions, thus having independent virtual reality input and output functions. They do not need to be connected to a PC or mobile terminal, and have a high degree of freedom of use.

[0038] 3) Augmented Reality (AR): A technology that calculates the camera's pose parameters in the real world (or 3D world, real world) in real time during image acquisition, and adds virtual elements to the captured images based on these parameters. Virtual elements include, but are not limited to, images, videos, and 3D models. The goal of AR technology is to overlay the virtual world onto the real world on a screen for interactive experiences.

[0039] 4) Mixed Reality (MR): By presenting virtual scene information within a real-world setting, an interactive feedback loop is established between the real world, the virtual world, and the user to enhance the realism of the user experience. For example, a simulated scene integrates computer-created sensory input (e.g., virtual objects) with sensory input or its representation from a physical setting. In some MR scenes, the computer-created sensory input can adapt to changes in sensory input from the physical setting. Additionally, some electronic systems used to present MR scenes can monitor orientation and / or position relative to the physical setting, enabling virtual objects to interact with real objects (i.e., physical elements or their representations from the physical setting). For example, the system can monitor motion so that virtual plants appear stationary relative to physical buildings.

[0040] 5) XR refers to the combination of reality and virtuality through computers to create a virtual environment that allows for human-computer interaction. XR is also a collective term for various technologies such as VR, AR, and MR. By integrating the visual interaction technologies of these three technologies, it brings users a sense of immersion that allows for seamless transitions between the virtual and real worlds.

[0041] 6) A virtual scene is a virtual scene displayed (or provided) by an application while running on an electronic device. This virtual scene can be a simulation of the real world, a semi-simulated / semi-fictional virtual scene, or a purely fictional virtual scene. A virtual scene can be any of a two-dimensional, 2.5-dimensional, or three-dimensional virtual scene; this application does not limit the dimension of the virtual scene. For example, a virtual scene may include the sky, land, ocean, etc., and the land may include environmental elements such as deserts and cities. Users can control virtual objects to move within this virtual scene. It should be understood that the above-mentioned virtual scene can also be referred to as a virtual space.

[0042] 7) Virtual objects are objects that interact in a virtual scene. They are controlled by the user or a robot program (e.g., an AI-based robot program) and can remain still, move, and perform various behaviors in the virtual scene, such as various characters in a game.

[0043] Currently, user interaction with objects in virtual environments primarily relies on eye-tracking, gestures, controllers, and voice commands. While these methods can meet basic interaction needs, they are insufficient for complex environments or specific scenarios, exhibiting certain limitations.

[0044] To address the aforementioned technical problems, the inventive concept of this application is as follows: A user triggers a preset interaction method within a virtual space to control the movement of a first visual indicator based on this preset interaction method, thereby selecting a target interactive object within the virtual space using the first visual indicator. When the input device remains stationary, a first input operation is sent using the input device, and the first visual indicator is controlled to interact with the target interactive object based on the first input operation. When the input device moves, the first visual indicator is switched to a second visual indicator. Then, when the input device sends a second input operation, the second visual indicator is controlled to interact with the target interactive object based on the second input operation. Thus, when the input device remains stationary, a target interactive object can be selected by controlling the first visual indicator corresponding to the preset interaction method. After selecting the target interactive object, the first visual indicator is controlled by the first input operation sent from the stationary input device to interact with the target interactive object. When the input device moves, the first visual indicator is switched to the second visual indicator corresponding to the input device, and then the second visual indicator is controlled by the second input operation sent from the input device to interact with the target interactive object. This can enrich human-computer interaction methods, improve human-computer interaction efficiency, meet the interaction needs in different scenarios, and thus bring users a better user experience.

[0045] The technical solutions of this application will be described in detail below through some embodiments. The embodiments described below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0046] Figure 1 is a schematic diagram of an application scenario provided by an embodiment of this application. As shown in Figure 1, the application scenario may include a first terminal device 110 and an input device 120. The first terminal device 110 and the input device 120 can establish a connection through wired or wireless connection to realize signaling interaction.

[0047] The wired connection can be that the input device 120 is connected to the first terminal device 110 via a connecting cable. The wireless connection can be established through a local area network, such as a Wi-Fi network, or Bluetooth.

[0048] In some alternative embodiments, the first terminal device 110 can be any electronic product that can provide a virtual space for the user, such as a VR device, AR device, MR device, or XR device. In this application, the first terminal device 110 is preferably a head-mounted display (HMD) that provides a virtual space.

[0049] In some alternative embodiments, the input device 120 described above can be a device with input functionality, such as a mouse and a touchpad.

[0050] In one application scenario, the first terminal device 110 can display a virtual space to the user, allowing the user to control a first visual indicator corresponding to a preset interaction method to locate the object to be interacted with within the virtual space. Then, by sending interaction commands through the stationary input device 120, the user controls the first visual indicator to interact with the object. In some implementations, if the input device 120 moves after locating the object, the first visual indicator is switched to a second visual indicator corresponding to the input device 120. The user can then send interaction commands through the input device 120 to control the second visual indicator to interact with the object. Thus, based on the existing interaction methods of the first terminal device 110, the user can perform more complex interactions with the object through the input device 120, thereby enriching human-computer interaction methods, improving human-computer interaction efficiency, and meeting the interaction needs in different scenarios.

[0051] It should be understood that the first terminal device 110 and input device 120 shown in Figure 1 above are only schematic. The actual number of first terminal devices 110 and input devices 120 may vary depending on actual usage needs. This application does not impose any restrictions on this.

[0052] After introducing the application scenarios of the technical solution of this application, the technical solution of this application will be described in detail below.

[0053] Figure 2 is a flowchart of a human-computer interaction method provided in an embodiment of this application. The human-computer interaction method provided in this embodiment can be executed by a human-computer interaction device. This human-computer interaction device can consist of hardware and / or software and can be integrated into an electronic device. Specifically, the electronic device in this application is the first terminal device shown in Figure 1.

[0054] As shown in Figure 2, the method includes the following steps.

[0055] At S101, when the user triggers an interaction method in the virtual space that is a preset interaction method, the first visual indicator is moved according to the preset interaction method to select a target interaction object from at least one interaction object in the virtual space.

[0056] The aforementioned virtual space can be understood as a virtual scene that combines the real and virtual worlds and allows for human-computer interaction, constructed by the first terminal device when the user uses it, or it can be a virtual scene selected from multiple virtual scenes provided by the first terminal device. This application does not impose any restrictions on the virtual space.

[0057] For example, the virtual scene mentioned above can be a tropical rainforest, a desert, or an ancient-style corridor, etc., and can be selected or constructed according to the user's needs.

[0058] At least one interactive object within the aforementioned virtual space may be a virtual object, which may include, but is not limited to, virtual characters, virtual props, virtual interfaces, virtual controls, and other virtual objects. For example, when the virtual object is a virtual panel, the virtual panel may be as shown in Figure 3a. When the virtual object is a three-dimensional object within the virtual panel, the three-dimensional object may be as shown in Figure 3b.

[0059] Considering that the first terminal device has a see-through function, the virtual space can also include real objects from the real environment. These real objects can be interfaces, controls, icons, applications, and images displayed by the second terminal device that communicates with the first terminal device. This application does not impose specific limitations on the objects within the virtual space.

[0060] That is, at least one interactive object in the virtual space in this application includes virtual objects and / or real objects.

[0061] It is understood that the virtual objects and / or real objects in the virtual space in this application can be 2D objects or 3D objects.

[0062] In this application, the second terminal device that establishes a connection with the first terminal device can be a desktop computer, a laptop computer, or other terminal devices with a display interface. This application does not impose any restrictions on the type of the second terminal device.

[0063] In some alternative embodiments, the connection between the first terminal device and the second terminal device may be established in the following ways.

[0064] The first method is to establish a connection based on a wireless communication network.

[0065] Optionally, by connecting the first terminal device and the second terminal device to the same local area network, it can be determined that the first terminal device and the second terminal device have established a connection when the first terminal device and the second terminal device are connected to the same local area network.

[0066] In this application, the local area network can be, but is not limited to, Wi-Fi networks and Zigbee networks.

[0067] The second method is to establish connections based on distance.

[0068] Optionally, by bringing the first terminal device and the second terminal device infinitely close, when the distance between the first terminal device and the second terminal device is less than a preset distance value, it is determined that the first terminal device and the second terminal device have established a connection.

[0069] In this application, the preset distance value can be flexibly set according to the device's recognition accuracy. For example, when the recognition accuracy is low, the preset distance value can be set larger; when the recognition accuracy is high, the preset distance value can be set smaller. This application does not impose any restrictions on this. The aforementioned device can be understood as a first terminal device and / or a second terminal device.

[0070] In other words, both the first and second terminal devices can establish a normal communication connection whether or not there is a network connection. Therefore, when using the first terminal device, the user can enable its perspective function and interact with visual indicators and real objects displayed on the second terminal device using an input device, such as clicking, dragging, and zooming. The visual indicators can be understood as cursors.

[0071] It should be understood that the see-through function refers to the use of the environmental camera on the first terminal device to capture images of the user's real environment when the user is wearing the first terminal device, and then display these images to the user through the display interface. This allows the user to see the real-time environment outside the first terminal device, enabling the user to perceive the external environment while using the first terminal device, thereby increasing the sustainability of the user experience.

[0072] In some optional embodiments, after a user uses a first terminal device and enters any virtual space, they can see various virtual objects and / or various real objects presented within the virtual space. Then, the user manipulates a first visual indicator to move within the virtual space using a preset interaction method to filter for objects that need to be interacted with. When the first visual indicator stops on a particular interactive object, it indicates that the user may need to interact with that object, at which point the interactive object can be identified as the target interactive object. The interactive object can be any virtual object or any real object.

[0073] In this application, the preset interaction method can be any traditional interaction method used by the user when interacting with the first terminal device. Specifically, the preset interaction method is eye-tracking interaction, gesture interaction, controller interaction, voice interaction, or head interaction. Eye-tracking interaction specifically refers to eye-tracking interaction, where the first visual indicator moves with the user's gaze to achieve human-computer interaction based on the user's eye movement data. Gesture interaction involves the first visual indicator moving with the user's gesture, thereby determining the gesture based on the user's gesture image and controlling the first visual indicator to perform the corresponding interactive operation. Controller interaction can be understood as using a handle, wristband, hand controller, or other device to control the first visual indicator for human-computer interaction. Voice interaction involves recognizing the user's voice commands to control the movement of the first visual indicator and execute the corresponding interactive action. Head-tracking interaction specifically involves controlling the first visual indicator based on the user's head movement data to achieve human-computer interaction.

[0074] At S102, in response to a first input operation of the input device, the first visual indicator is controlled to interact with the target interactive object.

[0075] In this embodiment of the application, the input device is a mouse and / or a touchpad.

[0076] In addition, controlling the interaction between the first visual indicator and the target interactive object can include, but is not limited to: click interaction, tap interaction, long press interaction, up and down swipe interaction, left and right swipe interaction, etc.

[0077] In some alternative embodiments, when the input device is a mouse, the user can send an input operation such as "open" by pressing the right mouse button, so that the first terminal device controls the first visual indicator to open or trigger the target interactive object according to the click operation, so as to realize the click interaction with the target interactive object.

[0078] In some alternative embodiments, when the input device is a touchpad, the user can control the first visual indicator to scroll the target interactive object by swiping up, down, left, and right on the touchpad, so as to achieve up, down, left, and right swiping interaction with the target interactive object.

[0079] It should be understood that the first input operation sent by the aforementioned input device is an interactive operation command sent by the user using the input device when the input device is not being moved.

[0080] At S103, in response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator.

[0081] In some alternative embodiments, after selecting the target interactive object based on the first visual indicator, the user can switch the first visual indicator to a second visual indicator by moving the input device, thereby gaining control over interacting with the target interactive object through the input device.

[0082] In other words, when a user controls the first visual indicator using a preset interaction method, the right to interact with the virtual space is based on this preset method. However, when the user moves the input device to switch the first visual indicator to a second visual indicator, the right to interact with the virtual space shifts from the preset interaction method to the input device itself. This allows the user to control the second visual indicator's movement within the virtual space and interact with target objects via the input device. The initial position of the second visual indicator follows the display position of the first visual indicator.

[0083] It should be noted that when the second visual indicator interacts with the target interactive object, the target interactive object can be the original target interactive object located by the first visual indicator, or it can be a new target interactive object reselected from the virtual space by controlling the second visual indicator to move. This application does not impose any restrictions on this.

[0084] In some alternative embodiments, considering that the coordinate system of the input device is different from the coordinate systems of the first terminal device and the second terminal device, the coordinate system of the input device is located on a horizontal plane, while the coordinate systems of the first terminal device and the second terminal device are located on a vertical plane perpendicular to the horizontal plane. The horizontal plane is a plane parallel to the ground.

[0085] Therefore, when switching the first visual indicator on the target interactive object to a second visual indicator based on the movement operation of the input device, this application can first determine the display position of the second visual indicator based on the position information of the input device, and then display the second visual indicator at the display position of the second visual indicator while hiding the first visual indicator. The specific implementation process may include the following steps.

[0086] Step 1: Obtain the location information of the input device and generate a proxy point in the display interface of the first terminal device based on the location information of the input device.

[0087] In this context, the proxy point can be understood as the location of the input device's location information displayed on the screen of the first terminal device.

[0088] Step 2: Generate a ray based on the virtual camera and agent point in the first terminal device.

[0089] In this application, a virtual camera can be understood as a first-person virtual camera generated with the user as the target, and the virtual camera can display virtual scene images based on the user's field of view.

[0090] In some alternative embodiments, generating a ray can be done by taking the location of the virtual camera as the starting point and the direction towards the location of the proxy point as the direction, generating a ray that includes the location of the virtual camera and the location of the proxy point.

[0091] Step 3: Determine the intersection of the ray and the target interactive object as the display position of the second visual indicator, display the second visual indicator at the display position, and hide the first visual indicator on the target interactive object.

[0092] For example, as shown in Figure 4a, when the target interactive object is a two-dimensional panel, a proxy point can be generated in the display interface of the first terminal device based on the position information of the input device. Then, the intersection of the ray generated based on the virtual camera and the proxy point with the two-dimensional panel is determined as the mapping point of the proxy point. This mapping point is then determined as the display position of the second visual indicator. Subsequently, the second visual indicator is displayed at this position, while the first visual indicator is hidden.

[0093] For example, as shown in Figure 4b, when the target interactive object is a three-dimensional object in a two-dimensional panel, a proxy point can be generated in the display interface of the first terminal device based on the position information of the input device. Then, the collision point between the ray generated based on the virtual camera and the proxy point and the three-dimensional object is determined as the display position of the second visual indicator. Subsequently, the second visual indicator is displayed at this position, while the first visual indicator is hidden.

[0094] In some alternative embodiments, considering that there may be other interactive objects in front of and behind the target interactive object, when determining the display position of the second visual indicator based on the position information of the input device, the display position of the second visual indicator can be determined based on the state of each interactive object and the ray distance when the ray intersects with each interactive object.

[0095] Optionally, the display position of the second visual indicator is determined based on the state of each interactive object and the distance of the ray when it intersects with each interactive object. This can be achieved by determining the state score of each interactive object and the distance score of each ray, and then calculating the total score for each interactive object based on the state score and distance score. The interactive object with the highest total score is then identified as the target interactive object. Finally, the point where the ray intersects with the target interactive object is determined as the display position of the second visual indicator.

[0096] In this application, the states of interactive objects include: an active state and an inactive state. The score for the active state can be set to 1, and the score for the inactive state can be set to 0. Furthermore, the distance score for the ray distance can be set such that the closer the ray is, the higher the score, and vice versa. It should be understood that the scores for the interactive object states and the ray distance scores in this application are adjustable parameters and can be flexibly set according to actual application needs. The above is merely an illustrative example and does not constitute a specific limitation on this application.

[0097] For example, as shown in Figure 4c, assume two two-dimensional panels, panel XX1 and panel XX2, are displayed in the virtual space. The score for an active interactive object can be set to 1, and the score for an inactive interactive object can be set to 0. When panel XX1 is active and panel XX2 is inactive, the state score of panel XX1 is 1, and the state score of panel XX2 is 0. When the distance between panel XX1 and the ray is less than the distance between panel XX2 and the ray, the distance score of panel XX1 is determined to be 1, and the distance score of panel XX2 is determined to be 0.5. Based on the state score and the distance score, the total score of panel XX1 is calculated as 1 + 1 = 2, and the total score of panel XX2 is 0 + 0.5 = 0.5. Since 2 > 0.5, panel XX1 is determined to be the target interactive object that the user wants to interact with. The intersection point of the ray and panel XX1 is then determined as the display position of the second visual indicator, allowing the second visual indicator to be displayed at that position.

[0098] It should be noted that when displaying the second visual indicator on the target interactive object, the display angle of the second visual indicator can optionally be adjusted according to the display angle of the target interactive object, so as to unify the display angle of the second visual indicator with that of the target interactive object, thereby enhancing the sense of coordination between the second visual indicator and the target interactive object.

[0099] In this application, the input device is a mouse and / or a touchpad. Furthermore, the movement operation of the input device specifically refers to the following: when the input device is a mouse, the movement operation is a mouse movement operation; when the input device is a touchpad, the movement operation is a touch movement on the touchpad, i.e., a touch movement made by the user's finger on the touchpad.

[0100] Furthermore, in this application, the first visual indicator and the second visual indicator specifically refer to visual indicators with different display forms. For example, the first visual indicator may be a circular visual indicator or a hand-shaped visual indicator, while the second visual indicator may be a cross-shaped visual indicator or an arrow-shaped visual indicator. That is, as long as the display forms of the first visual indicator and the second visual indicator are different, this application does not impose any restrictions on the display forms of the first visual indicator and the second visual indicator.

[0101] Considering that users' hands may tremble when operating the input device, or that accidental movement might trigger the device's movement, leading to an incorrect switch from the first visual indicator to the second visual indicator, this application, upon detecting movement of the input device, first determines whether the movement distance is greater than the trigger distance for switching the first visual indicator to the second. If the movement distance is greater than the trigger distance, it indicates that the user needs to interact with the interactive object through the input device, and in this case, the first visual indicator is switched to the second visual indicator. If the movement distance is less than or equal to the trigger distance, it indicates that the user may have moved the input device due to accidental operation or hand tremor, rather than the user needing to interact with the interactive object through the input device. In this case, the first visual indicator continues to be displayed, and no switching operation is performed on the first visual indicator.

[0102] In this application, the trigger distance can be flexibly set according to the interaction sensitivity. When a higher interaction sensitivity is required, the trigger distance can be set smaller. When a lower interaction sensitivity is required, the trigger distance can be set larger. This application does not impose any restrictions in this regard.

[0103] It should be noted that the execution order of the above steps S102 and S103 can be either S102 first and then S103, or S103 first and then S102. This application does not impose specific restrictions on this.

[0104] At S104, in response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object.

[0105] In this application, controlling the interaction between the second visual indicator and the interactive object may include, but is not limited to, drag-and-drop interaction, scaling interaction, rotation interaction, editing interaction, and other interactions. This application does not impose specific limitations on these interactions.

[0106] As an example, assuming the input device is a mouse, the user can drag the interactive object from its current display position to the position where the mouse is when the user releases the left mouse button by pressing the left mouse button and dragging the mouse.

[0107] As another example, assuming the input device is a mouse, the user can right-click the mouse to display the functional interface associated with the interactive object, such as viewing properties, refreshing, and editing the name.

[0108] As another example, assuming the input device is a touchpad, the user can slide their finger on the touchpad to control the second visual indicator to move onto the rotation control. Then, the user can tap the touchpad twice to trigger the rotation control to perform a rotation operation on the interactive object.

[0109] It should be noted that the target interactive object in this application can be associated with various interactive controls. In addition to rotation controls, it can also include zoom in controls, zoom out controls, edit controls, setting controls, closing controls, and so on. This application does not impose any restrictions on this.

[0110] In some alternative embodiments, this application can also enable other interactions such as virtual object creation, character input, and interaction with virtual space based on the input device. This provides users with a wealth of creative tools, facilitating the creation and editing of virtual objects, thereby increasing user creative freedom and meeting user application needs in different scenarios.

[0111] In some alternative embodiments, this application can also allow for individual operation of the second visual indicator based on the input device. For example, when the input device is a mouse, the user can click the left button twice consecutively to gain individual control over the second visual indicator, thereby allowing the user to interact with the second visual indicator through the input device, such as moving the second visual indicator, changing the display form of the second visual indicator, and performing other operations.

[0112] Understandably, when a user selects a target interactive object in a virtual space using traditional interaction methods, this application tracks a first visual indicator corresponding to the traditional interaction method. When the first visual indicator rests on an interactive object in the virtual space, it determines that the user wants to interact with that object. Furthermore, the user can send a first input operation to a first terminal device via an input device, causing the first terminal device to control the first visual indicator to interact with the target interactive object based on the first input operation. Additionally, the user can move the input device connected to the first terminal device to switch the first visual indicator corresponding to the traditional interaction method to a second visual indicator corresponding to the input device. The user can then manipulate the second visual indicator to interact with the interactive object in various ways. This allows for interaction navigation using traditional interaction methods such as eye movement, head movement, gestures, controllers, and voice, with interaction operations triggered by the input device to interact with interactive objects in the virtual space, and for interaction navigation and operation to be achieved solely through the input device. This can enrich human-computer interaction methods, meet the interaction needs in different scenarios, reduce the learning cost for users to use input devices, and improve the ease of operation for users using input devices.

[0113] The technical solution disclosed in this application discloses that when a user triggers a preset interaction method in a virtual space, a first visual indicator is moved according to the preset interaction method to select a target interactive object from at least one interactive object in the virtual space. Then, in response to a first input device, the first visual indicator is controlled to interact with the target interactive object. In response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator. Furthermore, in response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object. This application, by introducing an external input device on top of traditional interaction methods, allows interaction with interactive objects using an external input device, enriching human-computer interaction methods, improving human-computer interaction efficiency, and thus meeting the interaction needs in different scenarios.

[0114] Based on the foregoing embodiments, considering that after switching the first visual indicator to the second visual indicator, the user may need to change the target interaction object, or other factors may prevent the user from inputting a second input operation using the second visual indicator to interact with the target interaction object. Therefore, after switching the first visual indicator to the second visual indicator, this application can monitor the length of time the input device is in a state without any input and determine whether the time length meets the visual indicator switching condition. When the visual indicator switching condition is met, the second visual indicator is switched back to the first visual indicator, so that the user can continue to operate the first visual indicator to reselect the target interaction object based on the preset interaction method. The above-mentioned visual indicator switching process provided by the embodiments of this application will be specifically described below with reference to Figure 5.

[0115] As shown in Figure 5, the method may include the following steps.

[0116] At S201, when the user triggers an interaction method in the virtual space that is a preset interaction method, the first visual indicator is moved according to the preset interaction method to select a target interaction object from at least one interaction object in the virtual space.

[0117] At S202, in response to a first input operation of the input device, the first visual indicator is controlled to interact with the target interactive object.

[0118] At S203, in response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator.

[0119] At S204, it is determined that the duration of the input device being inactive is greater than the first duration, and the second visual indicator is switched to the first visual indicator.

[0120] The first duration is used to determine whether the second visual indicator needs to be switched to the first visual indicator. The specific duration can be flexibly set according to the switching requirements. For example, the first duration can be set to 2 seconds or 3 seconds.

[0121] In some optional embodiments, after switching the first visual indicator to the second visual indicator, the first terminal device can monitor in real time whether there is any input operation on the input device. When it is determined that there is no input operation on the input device, it is determined that the input device is in an inactive state. Then, the duration of the inactive state of the input device is counted and compared with a first duration. If the duration is longer than the first duration, it means that the visual indicator switching condition is met, and at this time, the second visual indicator is switched to the first visual indicator. If the duration is less than or equal to the first duration, it means that the visual indicator switching condition is not met, and at this time, the duration of the inactive state of the input device is counted again until the duration is longer than the first duration or an input operation is detected on the input device.

[0122] Optionally, when switching the second visual indicator to the first visual indicator, the following implementation methods may be included.

[0123] The first implementation method is to dim the display of the second visual indicator. When the transparency of the second visual indicator is less than a preset value, the second visual indicator is switched to the first visual indicator.

[0124] The preset values ​​can be flexibly set according to the needs of faded display, such as transparency reaching 60%, 65% or 70%, etc., and this application does not impose any restrictions on this.

[0125] In other words, when switching the second visual indicator back to the first visual indicator, the transparency of the second visual indicator can be continuously reduced, making the second visual indicator gradually transparent. When the transparency of the second visual indicator reaches a preset value, the second visual indicator is directly switched back to the first visual indicator to complete the visual indicator switching operation.

[0126] The second implementation method is to fade out the second visual indicator. When the fade-out duration of the second visual indicator is longer than the second duration, the second visual indicator is switched to the first visual indicator.

[0127] The second duration can be flexibly set according to the actual visual indication display requirements, such as 3 seconds, 5 seconds, etc., and this application does not impose any restrictions on it.

[0128] In other words, when switching from a second-view indicator to a first-view indicator, this application uses a faded display of the second-view indicator to subtly remind the user that the second-view indicator is about to switch back to the first-view indicator. This avoids the problem of users not having time to take remedial measures when directly switching from the second-view indicator to the first-view indicator, resulting in users having to frequently move their input devices to gain interaction with the virtual space and leading to a poor user experience.

[0129] In this application, when the second visual indicator is switched to the first visual indicator, the tracking data corresponding to the preset interaction method can be obtained, and the display position of the first visual indicator can be determined based on the tracking data, and then the first visual indicator can be displayed at the display position.

[0130] Considering that the preset interaction methods are eye-tracking interaction, gesture interaction, controller interaction, voice interaction, or head interaction, the tracking data corresponding to the preset interaction methods can be eye-tracking data for eye-tracking interaction, gesture data for gesture interaction, controller position data for controller interaction, voice data for voice interaction, or head position data for head interaction, etc.

[0131] Accordingly, the display position of the first visual indicator is determined based on tracking data, and the user's gaze direction can be determined based on eye-tracking data. Then, based on the user's gaze direction, the point where the user's gaze falls in the virtual space is determined, and this point is used as the display position of the first visual indicator. Alternatively, the intersection of a ray emanating from the user's real hand and the virtual space can be determined, and this intersection point is used as the display position of the first visual indicator. Alternatively, the intersection of a ray emanating from the controller and the virtual space can be determined, and this intersection point is used as the display position of the first visual indicator. Alternatively, speech recognition can be performed on the speech data, and the recognized location can be used as the display position of the first visual indicator. Alternatively, the intersection of a ray emanating from the user's head and the virtual space can be determined, and this intersection point is used as the display position of the first visual indicator.

[0132] In some alternative embodiments, after switching the second visual indicator back to the first visual indicator, if another movement operation of the input device is detected, this application can display the second visual indicator at the center of the display interface of the first terminal device. Alternatively, the display position of the second visual indicator can be determined in virtual space based on the location information of the input device. The second visual indicator is then displayed at that position so that the user can interact with the target interactive object based on the second visual indicator.

[0133] The technical solution disclosed in this application discloses that when the user triggers a preset interaction method in a virtual space, a first visual indicator is moved according to the preset interaction method to select a target interactive object from at least one interactive object in the virtual space. Then, in response to a first input device, the first visual indicator is controlled to interact with the target interactive object. In response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator. Furthermore, in response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object. This application introduces an external input device on top of traditional interaction methods, allowing interaction with interactive objects via an external input device. This enriches human-computer interaction methods, improves human-computer interaction efficiency, and thus meets interaction needs in different scenarios. Furthermore, after the second visual indicator is displayed, if the input device remains inactive for an extended period, the second visual indicator is switched back to the first visual indicator, allowing the user to continue operating the first visual indicator based on the preset interaction method to reselect the target interactive object, thereby meeting different user interaction needs and improving the user experience.

[0134] In one optional implementation scenario, since the preset interaction method is eye-tracking interaction, gesture interaction, controller interaction, voice interaction, or head interaction, the human-computer interaction method provided in this application embodiment will be further explained and described with eye-tracking interaction as the preset interaction method. As shown in Figure 6, the method may include the following steps.

[0135] At S301, when the user triggers an eye-tracking interaction in the virtual space, the position of the user's gaze point in the virtual space is determined.

[0136] At point S302, the user's field of vision within the virtual space is determined with the gaze point as the center.

[0137] At S303, when an interactive object exists within the user's visual range, the interactive object is identified as the target interactive object, and a first visual indicator is displayed on the target interactive object.

[0138] In this application, the gaze point position can be understood as the focal position of the user's gaze on any interactive object in the virtual space.

[0139] When a user interacts with an object, their gaze will be focused on a specific point on that object. Therefore, this application can use an eye-tracking camera on a first terminal device to capture real-time images of the user's eyes and analyze and process these images to obtain the user's eye movement data. Then, based on the eye movement data, the user's gaze direction is determined, and based on that gaze direction, the user's gaze point is determined. When the user's gaze point is located on a specific interactive object, that object can be identified as the target interactive object the user wants to interact with.

[0140] In some optional embodiments, considering that the human eye's visual field includes a horizontal visual field and a vertical visual field, the horizontal visual field is generally 30 degrees to the left and 30 degrees to the right, that is, the horizontal visual field range is generally 30 degrees to the left and 30 degrees to the right with the center point of both eyes as the origin, which can be represented as [-30°, 30°]. The vertical visual field is generally 25 degrees upward and 35 degrees downward, that is, the vertical visual field range is generally 25 degrees upward and 35 degrees downward with the center point of both eyes as the origin, which can be represented as [-35°, 25°].

[0141] Therefore, the virtual scene displayed to the user by the first terminal device is usually only visible to the user within their field of vision. Based on this, this application can determine the user's field of vision centered on the gaze point in the virtual space, and then display a first visual indicator on the interactive object (i.e., the target interactive object) within the user's field of vision, so that the user can always see the first visual indicator and the target interactive object located by the first visual indicator under the best field of vision.

[0142] At S304, in response to a first input operation of the input device, the first visual indicator is controlled to interact with the target interactive object.

[0143] At S305, in response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator.

[0144] At S306, in response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object.

[0145] The implementation principles of S304 and S306 are the same as those of S102 to S104 in the previous embodiments. For details, please refer to the previous embodiments. They will not be elaborated further here.

[0146] In some optional embodiments, when a user controls the movement of the second visual indicator via an input device, this application can visually enhance the second visual indicator to make it easier for the user to see, avoiding the need to spend a lot of time searching for it. In this application, visual enhancement of the second visual indicator may include, but is not limited to, enlarging or thickening the indicator.

[0147] Considering that when a user controls the movement of the second visual indicator via an input device, the indicator may move out of the user's field of vision. Since the user may not be able to see the second visual indicator accurately and promptly when it leaves the user's field of vision, this application calculates the distance between the second visual indicator and the boundary of the user's field of vision in real time when the user controls its movement via the input device, and compares this distance with a first distance. Then, based on the comparison result, it is determined whether the second visual indicator needs to be subjected to any processing such as snapping to it. The first distance is an adjustable parameter and can be flexibly set according to actual application needs, such as 0.5 meters or 0.1 meters, etc., without any limitations here.

[0148] Optionally, when the distance between the second visual indicator and the boundary of the user's field of vision is less than a first distance, it is determined that the user may have accidentally moved the second visual indicator to the boundary of the user's field of vision. In this case, the second visual indicator is snapped to the boundary where the distance is less than the first distance. Subsequently, regardless of how the user moves the second visual indicator out of the user's field of vision, it is controlled to snap to the boundary where the distance is less than the first distance. This avoids problems caused by moving the second visual indicator out of the user's field of vision.

[0149] Alternatively, the second visual indicator can be attached to a boundary less than the first distance while simultaneously displaying a notification to the user, indicating whether the second visual indicator needs to be moved out of the current field of view. Simultaneously, movement actions on the second visual indicator are detected, and its display is controlled based on the detection results. If the user needs to move the second visual indicator out of their field of view, they can again control it to move in the direction away from the user's field of view via an input device. When the first terminal device calculates that the distance between the second visual indicator and the boundary of the user's field of view decreases again and is less than the second distance, or when the time interval between the current movement action and the previous movement action is less than a preset interval, the second visual indicator is controlled to move out of the user's field of view. If the user does not need to move the second visual indicator out of their field of view, they can control it to move towards the center of their field of view via an input device to prevent it from leaving the user's field of view.

[0150] In addition, when the distance between the second visual indicator and the boundary of the user's field of vision is greater than or equal to the first distance, it indicates that the user is manipulating the second visual indicator to move within the user's field of vision, and no intervention operation is performed on the second visual indicator.

[0151] The first distance is greater than the second distance, and the second distance is an adjustable parameter that can be flexibly set according to actual application needs. For example, the second distance can be set to 0.05 meters.

[0152] The above preset time interval is an adjustable parameter, which can be flexibly set according to the actual application needs. For example, the preset time can be set to 2 seconds or 3 seconds.

[0153] That is, this application also includes, when a first visual indicator on the target interactive object is switched to a second visual indicator, and the distance between the second visual indicator and the boundary of the user's field of vision is less than a first distance, attaching the second visual indicator to the boundary. The boundary is the boundary with the smallest distance to the second visual indicator. Movement operations on the second visual indicator are detected, and the display of the second visual indicator is controlled based on the detection result.

[0154] The control of the display of the second visual indicator based on the detection results includes: when the detection results determine that the distance between the moved second visual indicator and the boundary is less than a second distance, controlling the second visual indicator to leave the user's field of vision, where the second distance is less than a first distance. Alternatively, when the detection results determine that the time interval between the movement operation of the moved second visual indicator and the previous movement operation is less than a preset interval, controlling the second visual indicator to leave the user's field of vision.

[0155] Furthermore, once the secondary visual indicator leaves the user's field of vision, if the user wants to bring it back into their field of vision, they can directly move it back in using the input device. This achieves an "easy-to-enter, difficult-to-leave" effect for the secondary visual indicator within the user's field of vision. By first attaching the indicator to the boundary when it leaves the user's view, it serves as a reminder and helps the user avoid accidentally moving the indicator out of their sight.

[0156] For example, the process of moving the second visual indicator out of the user's field of vision and moving the moved second visual indicator back into the user's field of vision can be illustrated in Figure 7.

[0157] In some optional embodiments, after the user moves the second visual indicator out of the user's field of vision via the input device, in order to facilitate the user's quick and accurate retrieval of the moved second visual indicator, this application can also display a prompt message for the second visual indicator at the boundary of the user's field of vision when the second visual indicator left, so as to remind the user of the direction of the current location of the second visual indicator. For example, as shown in FIG8, an arrow combined with text information can be used to remind the user of the direction of the current location of the second visual indicator, so that the user can quickly and accurately find and operate the moved second visual indicator based on this visual reminder.

[0158] In practical applications, the input device can be a mouse, a touchpad, or both. When this application controls the second visual indicator to interact with the target interactive object in the virtual space based on the input operation of the input device, the number and type of the input device can be determined first, and then different methods can be used to interact with the interactive object based on the input device. Specifically, the following situations may be included.

[0159] In scenario one, if the number of input devices is determined to be one, and the type is a mouse or touchpad, and the second input operation entered by the user through the input device is a selection operation, then when the user's field of vision changes and the target interactive object is no longer within the changed field of vision, the second visual indicator will continue to perform a selection interaction event on the target interactive object based on the selection operation.

[0160] For example, as shown in Figure 9, assuming the user's field of vision is the dashed area and the input device is a mouse, when the user presses the left mouse button to control the second visual indicator to select the target interactive object YY, the user then turns their head, changing their field of vision to the solid area. If the user keeps pressing the left mouse button, even if the target interactive object YY is not currently within the changed user's field of vision, the first terminal device still retains the selection event process for the target interactive object YY. Therefore, the user can move the mouse while holding down the left mouse button to control the second visual indicator to achieve click and drag interaction events for the target interactive object.

[0161] In other words, after the user establishes an event process with the target interactive object by controlling the second visual indicator through the input device, this application retains the interactive event corresponding to triggering the event process, regardless of whether the target interactive object is within the user's field of vision.

[0162] Scenario 2: If the number of input devices is determined to be two, namely a mouse and a touchpad, and the second input operation of the mouse or touchpad is a selection operation, then when the user's field of view changes and the target interactive object is no longer within the changed user's field of view, the selection interaction event for the target interactive object continues based on the selection operation. The second visual indicator is then redisplayed within the changed user's field of view. Based on the input operation of the touchpad or mouse, the redisplayed second visual indicator is controlled to interact with the new target interactive object.

[0163] For example, assuming the user's field of vision is a dotted area and the input devices are a mouse and a touchpad, when the user presses the left mouse button to control the second visual indicator to select the target interactive object YY, the user then turns their head, changing the user's field of vision to a solid area. If the user keeps pressing the left mouse button, even if the target interactive object YY is not currently within the changed user's field of vision, the first terminal device still retains the selection event process for the target interactive object YY. Therefore, the user can move the mouse while holding down the left mouse button to achieve click-and-drag interaction events for the target interactive object YY. Because the target interactive object YY is not currently within the user's field of vision, the user may not see it, so the second visual indicator may not be displayed on the target interactive object YY. Since the first terminal device is also connected to a touchpad, other interactive objects may exist within the changed field of vision. In this case, the user can use the touchpad to manipulate the second visual indicator displayed within the changed user's field of vision to interact with any new target interactive object DD within the changed field of vision.

[0164] In other words, after a user establishes an event process with a target interactive object through the first input device, this application continues to retain the interactive event corresponding to triggering the event process even when the target interactive object is not within the user's field of vision. Furthermore, when the user's field of vision changes, the user can continue to use the second input device to manipulate the second visual indicator displayed in the changed field of vision and interact with other interactive objects in the changed field of vision.

[0165] In some alternative embodiments, considering that the virtual space is a three-dimensional space and that the space is assumed to be a spherical structure, the interactive objects within the virtual space are displayed in a way that always faces the user directly, as shown in Figure 10. In Figure 10, the user is at the center, and the objects within the virtual space surround the user.

[0166] The technical solution disclosed in this application discloses that when a user triggers a preset interaction method in a virtual space, a first visual indicator is moved according to the preset interaction method to select a target interactive object from at least one interactive object in the virtual space. Then, in response to a first input device, the first visual indicator is controlled to interact with the target interactive object. In response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator. Furthermore, in response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object. This application, by introducing an external input device on top of traditional interaction methods, allows interaction with interactive objects using an external input device, enriching human-computer interaction methods, improving human-computer interaction efficiency, and thus meeting the interaction needs in different scenarios.

[0167] The following description refers to Figure 11, which illustrates a human-computer interaction device according to an embodiment of this application. Figure 11 is a schematic block diagram of a human-computer interaction device provided in an embodiment of this application.

[0168] As shown in Figure 11, the human-computer interaction device 400 includes: a movement control module 410, a first interaction module 420, a switching control module 430, and a second interaction module 440. The movement control module 410 is used to control the movement of a first visual indicator according to a preset interaction mode triggered by the user in the virtual space, so as to select a target interactive object from at least one interactive object in the virtual space. The first interaction module 420 is used to control the first visual indicator to interact with the target interactive object in response to a first input operation of the input device. The switching control module 430 is used to switch the first visual indicator to a second visual indicator in response to a movement operation of the input device. The second interaction module 440 is used to control the second visual indicator to interact with the target interactive object in response to a second input operation of the input device.

[0169] In an optional implementation of this application, the device 400 further includes: a switching module, configured to determine that the duration of the inactive state of the input device is greater than a first duration, and switch the second visual indicator to the first visual indicator.

[0170] In one optional implementation of this application, the switching module is specifically used to: dim the display of the second visual indicator; when the transparency of the second visual indicator is less than a preset value, switch the second visual indicator to the first visual indicator; or, dim the display of the second visual indicator; when the dimming duration of the second visual indicator is greater than a second duration, switch the second visual indicator to the first visual indicator.

[0171] In one optional implementation of this application, the input device is a mouse and / or a touchpad.

[0172] In one optional implementation of this application, the preset interaction method is one of eye-tracking interaction, gesture interaction, controller interaction, voice interaction, and head interaction.

[0173] In an optional implementation of this application embodiment, when the preset interaction method is eye-tracking interaction, the device 400 further includes: a determining module, used to determine the position of the user's gaze point in the virtual space; and to determine the user's field of vision range in the virtual space with the gaze point position as the center; correspondingly, the motion control module 410 is specifically used to: when there is an interactive object in the user's visual range, determine the interactive object as the target interactive object, and display the first visual indicator on the target interactive object.

[0174] In an optional implementation of this application, the device 400 further includes: a processing module, configured to, when a first visual indicator on the target interactive object is switched to a second visual indicator, and the distance between the second visual indicator and the boundary of the user's field of vision is less than a first distance, attach the second visual indicator to the boundary, wherein the boundary is the boundary with the smallest distance to the second visual indicator; and a detection module, configured to detect a movement operation on the second visual indicator and control the display of the second visual indicator based on the detection result.

[0175] In one optional implementation of this application, the control module is specifically configured to control the second visual indicator to leave the user's field of vision when the distance between the moved second visual indicator and the boundary is less than a second distance, based on the detection result; or, when the time interval between the moving operation of the moved second visual indicator and the previous moving operation is less than a preset interval, the control module is configured to control the second visual indicator to leave the user's field of vision.

[0176] In one optional implementation of this application, the device 400 includes: a prompting module, configured to display prompting information for the second visual indicator at the boundary of the user's field of vision, wherein the boundary is the boundary at which the second visual indicator leaves.

[0177] In one optional implementation of this application, the second interaction module 440 is specifically configured to: if the input device is a mouse or a touchpad, and the second input operation is a selection operation, then when the user's field of view changes and the target interactive object is no longer within the changed user's field of view, continue to control the second visual indicator to perform a selection interaction event on the interactive object based on the selection operation; if the input device is the mouse and the touchpad, and the second input operation of the mouse or the touchpad is a selection operation, then when the user's field of view changes and the target interactive object is no longer within the changed user's field of view, continue to perform a selection interaction event on the target interactive object based on the selection operation, and redisplay the second visual indicator within the changed user's field of view, and control the redisplaying second visual indicator to interact with the new target interactive object based on the third input operation of the touchpad or the mouse.

[0178] It should be understood that the device embodiments and the foregoing method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device 400 shown in FIG11 can execute the method embodiment corresponding to FIG2, and the foregoing and other operations and / or functions of each module in the device 400 are respectively to implement the corresponding processes in each method in FIG2. For the sake of brevity, they will not be repeated here.

[0179] The apparatus 400 of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the first aspect method embodiment in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the first aspect method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the first aspect method embodiment described above.

[0180] The technical solution disclosed in this application discloses that when a user triggers a preset interaction method in a virtual space, a first visual indicator is moved according to the preset interaction method to select a target interactive object from at least one interactive object in the virtual space. Then, in response to a first input device, the first visual indicator is controlled to interact with the target interactive object. In response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator. Furthermore, in response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object. This application, by introducing an external input device on top of traditional interaction methods, allows interaction with interactive objects using an external input device, enriching human-computer interaction methods, improving human-computer interaction efficiency, and thus meeting the interaction needs in different scenarios.

[0181] Figure 12 is a schematic block diagram of an electronic device provided in an embodiment of this application. As shown in Figure 12, the electronic device 500 may include a memory 510 and a processor 520. The memory 510 is used to store computer programs and transfer the program code to the processor 520. In other words, the processor 520 can call and run the computer program from the memory 510 to implement the human-computer interaction method in the embodiment of this application.

[0182] For example, the processor 520 can be used to execute the above-described human-computer interaction method embodiments according to instructions in the computer program.

[0183] In some embodiments of this application, the processor 520 may include, but is not limited to: a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0184] In some embodiments of this application, the memory 510 includes, but is not limited to, volatile memory and / or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0185] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 510 and executed by the processor 520 to complete the human-computer interaction method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.

[0186] As shown in Figure 12, the electronic device 500 may further include a transceiver 530, which can be connected to the processor 520 or the memory 510. The processor 520 can control the transceiver 530 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 530 may include a transmitter and a receiver. The transceiver 530 may further include an antenna, and the number of antennas may be one or more.

[0187] It should be understood that the various components in the electronic device are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0188] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the human-computer interaction method of the above-described method embodiments.

[0189] This application also provides a computer program product containing program instructions, which, when executed on an electronic device, cause the electronic device to perform the human-computer interaction method described in the above method embodiments.

[0190] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0191] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0192] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0193] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0194] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0195] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A human-computer interaction method, comprising: When the user triggers an interaction in the virtual space using a preset interaction method, the first visual indicator is moved according to the preset interaction method to select a target interaction object from at least one interaction object in the virtual space. In response to a first input operation from the input device, the first visual indicator is controlled to interact with the target interactive object; In response to a movement operation of the input device, the first visual indicator is switched to a second visual indicator; as well as In response to a second input operation of the input device, the second visual indicator is controlled to interact with the target interactive object.

2. The method according to claim 1, wherein the method further comprises: When it is determined that the duration of the inactive state of the input device is greater than the first duration, the second visual indicator is switched to the first visual indicator.

3. The method according to claim 2, wherein switching the second visual indicator to the first visual indicator comprises: The second visual indicator is displayed in a faded manner. When the transparency of the second visual indicator is less than a preset value, the second visual indicator is switched to the first visual indicator. or, The second visual indicator is faded out. When the faded out duration of the second visual indicator is longer than the second duration, the second visual indicator is switched to the first visual indicator.

4. The method according to claim 1, wherein the input device is a mouse and / or a touchpad.

5. The method according to claim 1, wherein the preset interaction method is one of eye-tracking interaction, gesture interaction, controller interaction, voice interaction, and head interaction.

6. The method according to claim 5, wherein when the preset interaction mode is eye-tracking interaction, the method further includes: Determine the user's gaze point location within the virtual space; The user's field of vision within the virtual space is determined with the gaze point location as the center. The method of controlling the movement of the first visual indicator according to the preset interaction method to select a target interactive object from at least one interactive object in the virtual space includes: When an interactive object exists within the user's visual range, the interactive object is identified as the target interactive object, and the first visual indicator is displayed on the target interactive object.

7. The method according to claim 6, wherein the method further comprises: When the first visual indicator on the target interactive object is switched to the second visual indicator, and the distance between the second visual indicator and the boundary of the user's field of vision is less than the first distance, the second visual indicator is attached to the boundary, wherein the boundary is the boundary with the smallest distance from the second visual indicator. Detect movement operations in response to the second visual indicator; as well as The display of the second visual indicator is controlled based on the detection results.

8. The method of claim 7, wherein controlling the display of the second visual indicator based on the detection result comprises: When the detection result determines that the distance between the moved second visual indicator and the boundary is less than a second distance, the second visual indicator is controlled to leave the user's field of vision, where the second distance is less than the first distance; or... If the time interval between the second visual indicator's movement and the previous movement is less than a preset interval, based on the detection results, the second visual indicator is controlled to leave the user's field of vision.

9. The method of claim 8, wherein the method further comprises: A prompt message for the second visual indicator is displayed at the boundary of the user's field of vision, the boundary being the boundary at which the second visual indicator leaves.

10. The method of claim 6, wherein controlling the second visual indicator to interact with the target interactive object in response to a second input operation of the input device comprises: If the input device is a mouse or touchpad, and the second input operation is a selection operation, then when the user's field of view changes and the target interactive object is no longer within the changed user's field of view, the second visual indicator continues to perform a selection interaction event on the interactive object based on the selection operation. If the input device is the mouse and the touchpad, and the second input operation of the mouse or the touchpad is a selection operation, then when the user's field of view changes and the target interactive object is no longer within the changed user's field of view, the target interactive object is still selected based on the selection operation, and the second visual indicator is redisplayed within the changed user's field of view. Based on the third input operation of the touchpad or the mouse, the redisplayed second visual indicator is controlled to interact with the new target interactive object.

11. A human-computer interaction device, comprising: The movement control module is used to control the movement of a first visual indicator according to the preset interaction method triggered by the user in the virtual space, so as to select a target interaction object from at least one interaction object in the virtual space. The first interaction module is used to respond to a first input operation of the input device and control the first visual indicator to interact with the target interactive object; A switching control module is used to switch the first visual indicator to a second visual indicator in response to a movement operation of the input device; as well as The second interaction module is used to respond to a second input operation of the input device and control the second visual indicator to interact with the target interactive object.

12. An electronic device, comprising: A processor and a memory, the memory being used to store a computer program, the processor being used to invoke and run the computer program stored in the memory to perform the human-computer interaction method as described in any one of claims 1 to 10.

13. A computer-readable storage medium for storing a computer program that causes a computer to perform the human-computer interaction method as described in any one of claims 1 to 10.

14. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the human-computer interaction method as described in any one of claims 1 to 10.

Images