Interaction method and electronic equipment

Abstract

The invention provides an interaction method and electronic equipment. The method comprises the following steps: acquiring spatial position data between an operation body and a virtual object; determining an interaction stage of the operation body and the virtual object based on the spatial position data; and for the interaction stage, the virtual object is rendered, so that the virtual object presents different visual display effects along with the change of the interaction stage, and the different visual display effects have different visual feature types.

Description

Technical Field

[0001] This application relates to human-computer interaction technology, and more particularly to an interaction method and electronic device. Background Technology

[0002] With the continuous development of extended reality technology, users' interactive needs in virtual or augmented reality environments are increasing. Current extended reality interaction solutions (such as those based on color changes, sound cues, or haptic feedback) have at least the following significant limitations in practical applications: color feedback is easily affected by ambient light; sound feedback is less effective in noisy environments or when facing hearing-impaired users; and haptic feedback is difficult to apply to scenarios such as contactless gesture interaction. These limitations affect the universality of interaction and the consistency of the user experience. Summary of the Invention

[0003] This application provides an interaction method and an electronic device.

[0004] The technical solution of this application embodiment is implemented as follows: This application provides an interaction method, including: Obtain spatial position data between the operating entity and the virtual object; Based on the spatial location data, the interaction stage between the operator and the virtual object is determined; For each interaction stage, the virtual object is rendered so that it presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types.

[0005] This application provides an electronic device, including: Memory is used to store executable instructions or computer programs. When a processor executes computer-executable instructions or computer programs stored in the memory, it performs the following steps: Obtain spatial position data between the operating entity and the virtual object; Based on the spatial location data, the interaction stage between the operator and the virtual object is determined; For each interaction stage, the virtual object is rendered so that it presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types. Attached Figure Description

[0006] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0007] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0008] Figure 1 This is one of the flowcharts illustrating an interaction method provided in an embodiment of this application; Figure 2 This is a second flowchart illustrating an interaction method provided in an embodiment of this application; Figure 3 This is a third flowchart illustrating an interaction method provided in an embodiment of this application; Figure 4 This is a fourth flowchart illustrating an interaction method provided in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the visual display effect without interactive input, provided in an embodiment of this application. Figure 6 This is a schematic diagram illustrating a visual display effect for close-up interaction provided in an embodiment of this application; Figure 7 This is a schematic diagram illustrating the visual display effect of touch interaction provided in an embodiment of this application; Figure 8 This is a schematic diagram illustrating the visual display effect of a press-interaction provided in an embodiment of this application; Figure 9 This is a schematic diagram illustrating the changes in visual display effects during a complete interactive process, as provided in an embodiment of this application. Figure 10 This is a schematic diagram of the structure of an interactive device provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0010] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0011] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0012] The terms "first," "second," and "third" appearing in the embodiments of this application do not have specific meanings (such as no order distinction, nor do they indicate a special limitation on the number of devices in the embodiments of this application), but are merely for the purpose of clearly describing the embodiments of this application and do not constitute any limitation on the embodiments of this application. The term "multiple" appearing in the embodiments of this application refers to two or more integers.

[0013] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies or terms of the embodiments of this application are described below. The following related technologies or related terms are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0014] Extended Reality (XR) is a collective term for technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), used to create interactive experiences between users and virtual environments. XR technology can provide users with immersive multi-sensory experiences, including visual, auditory, and tactile sensations, through head-mounted displays or other interactive devices.

[0015] In XR interactions, users often fail to accurately perceive the actual location of virtual objects due to a lack of clear distance feedback, leading to unnatural interactions or even misoperations. Current solutions, such as color changes, sound feedback, or vibration feedback, can provide some level of interaction cues, but they are limited by ambient lighting, noise interference, or usage scenarios, making it difficult to meet diverse interaction needs. In particular, color changes are difficult to notice in bright outdoor light; sound feedback is easily masked in noisy environments; and vibration feedback is unsuitable for gesture interactions.

[0016] To address at least one of the aforementioned problems in related technologies, this application provides an interaction method. By acquiring spatial position data between the operator and the virtual object, the method dynamically identifies the interaction stage between them and renders visual display effects with corresponding visual characteristics based on different interaction stages. This allows users to receive clear visual feedback at different interaction stages, effectively enhancing their perception of the distance to virtual objects. Furthermore, by employing multiple types of visual feedback instead of the traditional single-sensory feedback mechanism, this application reduces the requirements for ambient lighting and sound conditions, thereby enhancing adaptability, stability, and scene compatibility in different physical environments. It also naturally supports contactless interaction, expanding the user coverage, enhancing the user's perception of the distance to virtual objects, and improving the naturalness and accuracy of the interaction.

[0017] It should be noted that the interaction methods provided in the embodiments of this application can be executed by an XR device, wherein the XR device can be a head-mounted display device, a gesture tracking device, or a mixed reality terminal.

[0018] Figure 1 This is one of the flowcharts illustrating an interaction method provided in an embodiment of this application; for example... Figure 1 As shown, the method may include the following steps 101 to 103: Step 101: Obtain the spatial position data between the operating body and the virtual object.

[0019] It's important to clarify that the "operating body" refers to the body part the user interacts with in the virtual environment; this can be a hand, finger, or other interactive controller. A virtual object is a 3D model or graphical element in virtual space that can be perceived and interacted with by the operating body, such as virtual buttons, icons, or other 3D model elements. Spatial position data describes the relative position of the operating body and the virtual object in 3D space, and can include parameters such as XYZ axis coordinates, distance values, and direction angles. For example, when a user approaches a virtual object, the system acquires the spatial position data of the user's hand relative to the virtual object. Spatial position data is the fundamental input for subsequent interaction stage determination and rendering of corresponding visual feedback.

[0020] In some embodiments, spatial location data can be collected by devices such as sensors, cameras, depth cameras, and lidar, and analyzed and processed in real time by relevant algorithms (such as AI algorithms).

[0021] Step 102: Based on the spatial location data, determine the interaction stage between the operating entity and the virtual object.

[0022] It should be noted that the interaction phase represents the interaction state between the operator and the virtual object within different distance ranges, and each interaction phase corresponds to a different visual display effect. Dividing the interaction phases helps to achieve more refined interaction control and optimize the user experience.

[0023] In this embodiment, the interaction stage between the operator and the virtual object can be dynamically determined based on the spatial position data between them, so as to adjust the rendering method of the virtual object according to the interaction stage.

[0024] For example, when the user is more than 1 meter away from the virtual object, it is determined that the user has not yet entered the interactive range of the virtual object; when the user is less than or equal to 1 meter away, it is determined that the user is approaching the virtual object; when the user is less than or equal to 0.5 meters away, it is determined that the user is touching the virtual object; and when the user is less than or equal to 0.1 meters away, it is determined that the user is pressing the virtual object. Each interaction stage represents a specific interactive relationship between the user and the virtual object and guides the system to provide corresponding visual feedback.

[0025] Step 103: For the current interaction stage, render the virtual object so that the virtual object presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types.

[0026] It should be noted that visual feature types refer to the different visual representations used when rendering virtual objects, including but not limited to contour lighting effects, shadow effects, and positional movement effects. These visual features are used to convey visual cues to the user about the relationship between the manipulated object and the virtual object, enhancing the user's depth perception and interactive feedback. Visual display effects refer to the changes in the appearance of the virtual object at different stages of interaction, such as the virtual object's edges emitting bright light effects, the virtual object's surface displaying visible shadow effects, changes in the virtual object's spatial structure, or relative displacement of different parts of the virtual object.

[0027] In some embodiments, for different interaction stages, the virtual object can be rendered based on different visual feature types using the Unity engine or other graphics rendering engines, so that the virtual object presents different visual display effects as the interaction stage changes.

[0028] The interaction method provided in this application dynamically identifies the interaction stage by acquiring spatial position data between the operator and the virtual object, and renders visual display effects with corresponding visual characteristics based on different interaction stages. This allows users to receive clear visual feedback at different interaction stages, effectively enhancing their ability to perceive the distance of virtual objects. Furthermore, by employing multiple types of visual feedback instead of the traditional single-sensory feedback mechanism, this application reduces the requirements for ambient lighting and sound conditions, thereby enhancing adaptability, stability, and scene compatibility in different physical environments. It also naturally supports contactless interaction, expanding the user coverage, enhancing users' ability to perceive the distance of virtual objects, and improving the naturalness and accuracy of the interaction.

[0029] In some embodiments, the interaction phase includes a first interaction phase, wherein the first interaction phase characterizes the operator entering a first adjacent region of the virtual object; When the current interaction stage is the first interaction stage, rendering the virtual object for the current interaction stage includes: The virtual object is rendered based on the first visual feature type, so that the virtual object presents a visual display effect with a target outline.

[0030] It should be noted that the first interaction stage represents the entry of the operating object (such as the user's hand) into the first adjacent area of ​​the virtual object. This first adjacent area can be adaptively set based on the actual application, for example, it can be set to a range within 1 meter of the operating object. By defining the first interaction stage, early visual feedback can be provided even before the operating object directly touches it, thereby enhancing the user's perception of the virtual object's spatial location, reducing accidental operations, and improving the naturalness of the interaction.

[0031] It should be noted that the first visual feature type refers to a specific rendering method used to change the visual display effect or appearance of virtual objects during the first interaction phase, indicating that the distance between the operator and the virtual object is decreasing, or that the operator is approaching the virtual object. There is a close relationship between the first interaction phase and the first visual feature type. When the operator and the virtual object enter the first interaction phase, the system can automatically activate the visual feedback mechanism for the first interaction phase, that is, switch to the rendering mode of the first visual feature type, thereby ensuring that the user can receive timely prompts regarding the interaction status of the first interaction phase.

[0032] It should be noted that the target outline is a visual representation that enhances the boundary information of virtual objects. The target outline helps users understand the shape and position of virtual objects and avoids misoperation caused by visual blur.

[0033] Understandably, by using the first visual feature type to render virtual objects in the first interaction stage, the user's perception of the virtual object's presence can be significantly improved, especially in environments with strong light, where good visibility can be maintained.

[0034] For example, Figure 2 This is a second flowchart illustrating an interaction method provided in an embodiment of this application; as shown... Figure 2 As shown, the method may include the following steps 201 to 203: Step 201: Obtain the spatial position data between the operating entity and the virtual object; Step 202: Based on the spatial location data, determine the interaction stage between the operating entity and the virtual object; Step 203: If the current interaction stage is the first interaction stage, render the virtual object based on the first visual feature type so that the virtual object presents a visual display effect with a target outline.

[0035] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0036] In this embodiment, when the operator enters the first adjacent area of ​​the virtual object, i.e., when the two are in the first interaction stage, the virtual object is rendered based on the first visual feature type, so that it presents a visual display effect with an enhanced target outline. This method enhances the boundary recognizability of the virtual object, intuitively guides the user to perceive the current interaction state, and thus improves the user's ability to perceive the spatial distance of the virtual object.

[0037] In some embodiments, rendering the virtual object based on a first visual feature type, such that the virtual object presents a visual display effect with a target outline, includes: The edge region of the virtual object is rendered based on the first visual feature type, so that the virtual object presents a visual display effect with a glowing outline; the visual salience of the glowing outline is higher than the visual salience of the surface of the virtual object itself.

[0038] It should be noted that the edge region refers to the boundary part of a virtual object. In 3D modeling, the edge region of a virtual object is usually composed of the outer vertices of polygons.

[0039] Understandably, applying special rendering techniques to the edge areas of virtual objects can highlight their outlines, ensuring good visibility even against complex backgrounds or in bright lighting conditions.

[0040] It should be noted that glowing outlines are a visual effect that, by increasing brightness and color saturation, makes the edges of virtual objects appear to emit a glowing effect, thereby enhancing the three-dimensionality and presence of the virtual object. Visual display effects are image information generated and presented on XR devices using computer graphics technology, including but not limited to color, lighting, and shape changes. By changing the rendering method of the edge areas of virtual objects, more attractive and guiding visual feedback can be achieved.

[0041] It's important to note that visual salience refers to the ability of a visual element to attract attention within the overall image, and it's related to factors such as brightness, contrast, and color saturation. Compared to the surface of the virtual object, the glowing outline has higher values ​​in brightness, contrast, and color saturation, making it more prominent. For example, the glowing outline can be drawn using high-brightness, high-contrast colors (such as yellow and white), while the surface of the virtual object can be drawn using softer colors (such as gray and blue), thus creating a clear visual hierarchy in the image.

[0042] It is understandable that the virtual object's surface refers to the main body of the virtual object, that is, all areas except for the edge areas.

[0043] In this embodiment, by rendering the edge region of the virtual object based on a first visual feature type, the virtual object presents a visual display effect with a glowing outline, and the visual salience of the glowing outline is higher than that of the virtual object's surface. This effectively improves the recognizability and boundary clarity of the virtual object under different lighting conditions. This method, without relying on sound or tactile feedback, enhances the user's ability to perceive the presence and location of virtual objects in complex environments (especially in outdoor bright light scenes), thereby improving the reliability of interaction and the immersive experience.

[0044] In some embodiments, the interaction phase further includes a second interaction phase, the second interaction phase representing the operator entering a second neighboring region of the virtual object; the second neighboring region is closer to the virtual object than the first neighboring region; When the current interaction stage is the second interaction stage, rendering the virtual object for the current interaction stage includes: The virtual object is rendered based on the second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the manipulation object.

[0045] It should be noted that the second interaction stage represents the operation object (such as the user's hand) entering the second proximity area of ​​the virtual object, and this second proximity area is closer to the virtual object than the first proximity area. In other words, the advancement of the interaction stage corresponds to the operation object gradually approaching the virtual object in space. The second proximity area can be adaptively set based on actual applications, for example, it can be set to a range within 0.5 meters of the virtual object. Compared to the first proximity area (such as within 1 meter), the second proximity area represents a more refined level of interaction.

[0046] It should be noted that the second visual feature type refers to a specific rendering method used to render virtual objects during the second interaction phase. The second visual feature type differs from the first visual feature type to distinguish between different interaction phases. The second visual feature type aims to enhance the user's perception of the relative positional relationship between the virtual object and the manipulated object, manifested as the projection of the manipulated object's (e.g., a hand) shadow onto the surface of the virtual object. The shadow effect is an intuitive visual feedback mechanism that helps users judge the distance and direction between the manipulated object and the virtual object, improving the naturalness and accuracy of the interaction.

[0047] In some embodiments, when it is determined that the operator has entered the second adjacent area of ​​the virtual object, the system generates a shadow on the surface of the virtual object that corresponds to the shape of the operator. For example, in a museum XR display, when a user's hand approaches the virtual object, a shadow effect matching the outline of the hand appears on the virtual object, indicating to the user that they have entered the triggerable area, further enhancing the realism of the user interaction.

[0048] For example, Figure 3 This is a flowchart illustrating an interaction method provided in an embodiment of this application; as shown below. Figure 3 As shown, the method may include the following steps 301 to 303: Step 301: Obtain the spatial position data between the operating entity and the virtual object; Step 302: Based on the spatial location data, determine the interaction stage between the operating entity and the virtual object; Step 303: If the current interaction stage is the second interaction stage, render the virtual object based on the second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the operating object.

[0049] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0050] In this embodiment, when the operator enters the second adjacent area of ​​the virtual object, i.e., when the two are in the second interaction stage, the virtual object is rendered based on the second visual feature type, so that the surface of the virtual object presents a shadow effect corresponding to the shape of the operator. This method not only effectively improves the user's perception of the spatial distance of the virtual object, but also enhances interaction guidance, helping the user to better understand the current interaction state, thereby improving the user experience.

[0051] In some embodiments, rendering the virtual object based on a second visual feature type, such that the surface of the virtual object body presents a shadow effect corresponding to the shape of the manipulator, includes: Based on the spatial location data and specific light source direction parameters, the projection area of ​​the operating body on the surface of the virtual object is determined; The projection area is rendered based on the second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the manipulator; the shadow effect changes with the change of the spatial position data.

[0052] It should be noted that specific light source direction parameters refer to the light source angle information preset or dynamically calculated by the system, used to simulate the direction and intensity of light projection in the real world, thereby generating natural shadow effects. The projection area refers to the shadow coverage area generated on the surface of the virtual object based on spatial location data and light source direction parameters. The projection area is a graphic area formed by projecting the outline of the manipulated object onto the surface of the virtual object according to the light source direction. The projection area can be point-like, line-like, or area-like, depending on the shape and position of the manipulated object.

[0053] For example, when a user's finger approaches a virtual button, the system can calculate and render the shadow effect of the user's finger on the surface of the virtual button in real time based on the spatial position of the user's finger and specific light source direction parameters. The shadow effect can dynamically change as the user's finger moves, allowing the user to clearly see the contact point between the finger and the virtual object.

[0054] In this embodiment, by combining spatial location data with specific light source direction parameters, the projected area on the surface of the virtual object is calculated and dynamically rendered in real time, generating a shadow effect that updates as the shape and position of the manipulated object changes. This method intuitively maps the spatial relationship of the manipulated object into a visual shadow feedback, significantly enhancing the user's perception of the distance and relative position of the virtual object, thereby effectively improving the accuracy, naturalness, and user experience of the interaction.

[0055] In some embodiments, the interaction phase further includes a third interaction phase, wherein the third interaction phase represents the operator entering a third neighboring region of the virtual object; the third neighboring region is closer to the virtual object than the second neighboring region; When the current interaction stage is the third interaction stage, rendering the virtual object for the current interaction stage includes: The virtual object is rendered based on the third visual feature type, so that the virtual object presents a visual display effect of target spatial structure and / or relative displacement of different parts of the virtual object.

[0056] It should be noted that the third interaction stage represents the entry of the operating object (such as the user's hand) into the third proximity area of ​​the virtual object, and this third proximity area is closer to the virtual object than the second proximity area. The third proximity area can be adaptively set based on actual applications, for example, it can be set to a range within 0.1 meters of the virtual object. Compared to the second proximity area (such as within 0.5 meters), the third proximity area represents a more refined level of interaction. Compared to the second proximity area, the third proximity area places greater emphasis on high-precision interactive feedback and an immersive experience.

[0057] It should be noted that the third visual feature type refers to a specific rendering method used to render virtual objects in the third interaction stage. The third visual feature type differs from the first and second visual feature types to distinguish between different interaction stages. The third visual feature type provides more intuitive operation prompts by altering the spatial structure or movement trajectory of the virtual object. For example, when the object enters the third adjacent area, the virtual object can split into two parts. One part remains in place and serves as a support surface for the virtual object, while the other part can pop forward in the form of an aperture, creating a visual feedback of being pressed.

[0058] It should be noted that the target spatial structure refers to the three-dimensional spatial layout of the virtual object in the third interaction stage. The target spatial structure can be dynamically adjusted according to the user's behavior to enhance the realism of the interaction. For example, in the third interaction stage, the virtual object can transform from its original planar structure into a three-dimensional structure with depth and hierarchy.

[0059] It should be noted that the visual effect of relative displacement refers to the dynamic movement or deformation effect between different parts within a virtual object. The visual effect of relative displacement can be used to simulate collisions, compressions, or bounces in the physical world, thereby enhancing the user's perception of the force and direction of the interaction. For example, when a user presses down on a virtual button, the surface of the virtual button can deform, accompanied by the upper aperture of the virtual button moving forward. This process creates the illusion of pressing down forcefully.

[0060] For example, Figure 4 This is a fourth flowchart illustrating an interaction method provided in an embodiment of this application; as shown below. Figure 4 As shown, the method may include the following steps 401 to 403: Step 401: Obtain the spatial position data between the operating entity and the virtual object; Step 402: Based on the spatial location data, determine the interaction stage between the operating entity and the virtual object; Step 403: If the current interaction stage is the third interaction stage, render the virtual object based on the third visual feature type, so that the virtual object presents a visual display effect of target spatial structure and / or relative displacement of different parts of the virtual object.

[0061] It should be noted that the descriptions of the same steps and contents as in other embodiments in this embodiment can be found in the descriptions in other embodiments, and will not be repeated here.

[0062] In this embodiment, when the operator enters the third adjacent region of the virtual object, i.e., when the two are in the third interaction stage, the virtual object is rendered based on the third visual feature type, so that the virtual object presents a visual display effect of relative displacement of the target spatial structure and / or different parts of the virtual object. This method enhances the realism and immersion of the interaction and significantly improves the naturalness of the interaction.

[0063] In some embodiments, rendering the virtual object based on a third visual feature type, such that the virtual object presents a visual display effect of a target spatial structure and / or relative displacement of different parts of the virtual object, includes: Based on the third visual feature type, the virtual object is rendered as a combination of at least two visual objects; In response to a pressing operation applied to the virtual object by the operating body, the at least two visual objects are controlled to produce a relative displacement, so that the virtual object presents a pressed visual display effect.

[0064] In this embodiment, the third visual feature type is a visual representation method used to enhance the user's perception of the depth of virtual objects. It combines visual elements such as spatial structure changes and displacement to assist the user in judging the interaction distance. For example, in a museum XR display project, when the user's fingertip approaches a virtual button, the virtual button can be split into two independent visual objects: one is a base surface, and the other is an aperture. The base surface and the aperture form a combination, thereby enhancing the tactile feedback and visual hierarchy during the interaction process.

[0065] It should be noted that visual objects refer to the graphical elements or sub-components used to construct virtual objects in a virtual environment. Visual objects can exist independently or be combined together through logical relationships to form a complete virtual object. A composite object refers to the overall structure formed by combining multiple visual objects according to certain spatial relationships or logical rules. For example, when a virtual button is pressed, it can be decomposed into two visual objects: a base surface and an aperture. The base surface and the aperture undergo relative displacement when the user applies pressure, simulating the visual effect of the virtual button being pressed. Relative displacement refers to the change in position between two visual objects in three-dimensional space. A pressing operation refers to the pressure action applied by the operator to the virtual object, that is, the process of the operator contacting the virtual object itself.

[0066] For example, when a user's fingertip approaches and presses a virtual button, the system detects this action and triggers a corresponding visual feedback mechanism. This causes a relative displacement between two visual objects within the virtual button (such as the base surface and the aperture), resulting in the virtual button appearing as if it has been pressed.

[0067] In this embodiment, a virtual object is rendered as a combination of at least two visual objects based on a third visual feature type. When a pressing operation is detected, the relative displacement between these visual objects is controlled to simulate the dynamic visual effect of the virtual object deforming under pressure. This method effectively enhances the user's spatial perception and operational feedback of the virtual object by providing intuitive and physically consistent visual feedback, thereby improving the accuracy, immersion, and overall interaction quality.

[0068] In some embodiments, the system can also output an audio prompt indicating that the virtual object has been pressed in response to a pressing operation applied to the virtual object by the operator. For example, when a user presses a virtual button, the system can simultaneously play a short "click" sound or electronic sound effect to simulate the auditory feedback of a physical button being pressed; if the pressing operation is accompanied by deformation of the virtual object or displacement of a component, a slight material squeezing sound or elastic component rebound sound effect can also be played to enhance the multi-sensory realism of the interaction and the sense of operation confirmation.

[0069] In some embodiments, the at least two visual objects include a first visual object and a second visual object, wherein the first visual object is configured with a collider for interacting with the operating body during the pressing operation; the step of controlling the at least two visual objects to generate relative displacement in response to the pressing operation applied by the operating body to the virtual object, so that the virtual object presents a pressed visual display effect, includes: In response to the pressing operation applied by the operating body to the first visual object, the operating body is constrained to the surface of the first visual object by the collision body, and the second visual object is controlled to generate a displacement relative to the first visual object in the opposite direction of the pressing operation, so that the virtual object presents a pressed visual display effect.

[0070] It's important to note that a collider is a virtual boundary or shape used in three-dimensional space to simulate physical collision behavior. A collider can be a geometric shape (such as a cube, sphere, or capsule) or a complex mesh shape generated from the model's outline. The function of a collider is to detect whether a manipulated object touches a virtual object and provide a feedback mechanism, enabling the system to determine whether the object has actually made contact. For example, in an XR environment, a collider ensures that the manipulated object does not directly penetrate the virtual object, thereby enhancing realism and interaction accuracy.

[0071] Understandably, the first visual object is part of the virtual object, serving as a key interface element for user perception and operation. The first visual object receives input from the action (such as pressing) and achieves physical interaction through colliders. The second visual object is another part of the virtual object, used in conjunction with the first visual object to jointly construct the complete visual presentation. During a pressing operation, the displacement of the second visual object enhances the visual feedback of the pressing action. The motion logic of the second visual object depends on the state changes of the first visual object; a close linkage exists between the two.

[0072] In this embodiment, when the operator applies a pressing operation to the first visual object, the system can use a collider to restrict the operator to the surface of the first visual object, preventing the operator from penetrating or deviating. This restriction mechanism enhances the realism of the interaction, allowing the user to feel that the virtual object has a physical presence, rather than just being a graphical representation. Simultaneously, the second visual object will shift in the opposite direction of the pressing direction, creating a noticeable deformation effect, making the entire virtual object exhibit a dynamic change under pressure. This method avoids the problem of the operator passing through the virtual object in traditional interactions, improving the user's immersion and interactive experience.

[0073] In this embodiment, the virtual object is split into a first visual object with a collider and a displaceable second visual object. When the operator presses down, the collider provides physical constraints and drives the first and second visual objects to produce relative displacement, thereby simulating the visual effect of the virtual object being pressed and deformed. This method effectively prevents the operator from penetrating the virtual object, enhancing the physical realism of the interaction and the sense of confirmation of the operation, while significantly improving the user's depth perception, interaction naturalness, and operation accuracy in the XR environment.

[0074] In some embodiments, the method further includes: When the current interaction stage is the third interaction stage, the virtual object is controlled to move closer to the operating body, so that the spatial position of the virtual object is aligned with the spatial position of the operating body.

[0075] In this embodiment, the virtual object is controlled to move closer to the operating body in the third interaction stage. That is, the system actively adjusts the spatial position of the virtual object so that the spatial position of the virtual object is aligned with the spatial position of the operating body, thereby ensuring that the operating body can accurately perform subsequent operations.

[0076] It's important to note that spatial alignment refers to aligning the spatial position of a virtual object with the coordinates of the manipulated object (such as a user's finger) in three-dimensional space to achieve precise interaction. Spatial alignment can be achieved through spatial tracking technologies (such as hand skeleton tracking and depth cameras) to ensure that the virtual object always follows the real-time changes of the manipulated object. Once spatial alignment is complete, the manipulated object can directly act on the virtual object, significantly enhancing the realism and naturalness of the interaction.

[0077] In this embodiment, by introducing an active adsorption and spatial alignment mechanism for virtual objects in the third interaction stage, the virtual objects are dynamically adjusted to precisely match the operating object as it is about to complete the operation. This strategy effectively reduces positioning errors during precise user operations, thereby optimizing the interaction process and significantly improving the accuracy, efficiency, and overall naturalness of the interaction.

[0078] In some embodiments, determining the interaction stage between the operator and the virtual object based on the spatial location data includes: Acquire perceptual data to characterize the interactive intent of the operator; If, based on the perceived data, it is determined that the interaction intention of the operator is directed towards the virtual object, the interaction stage between the operator and the virtual object is determined based on the spatial location data.

[0079] It should be noted that interaction intent refers to the subjective purpose or tendency of a user to interact with a virtual object, as expressed through the user's interaction interface.

[0080] In some embodiments, the sensed data may include, but is not limited to, the following types: Eye-tracking data: screen coordinates or 3D spatial coordinates of the user's gaze point are obtained through eye-tracking sensors; Manipulator posture data: Manipulator posture (such as hand joint angles) acquired through a depth camera or inertial measurement unit (IMU).

[0081] In this embodiment of the application, after determining that the interaction intention of the operator is clearly directed to the virtual object based on the perception data, the current interaction stage (e.g., the first interaction stage (>1 meter), the second interaction stage (≤0.5 meter), or the third interaction stage (≤0.1 meter)) is accurately determined based on the currently acquired spatial location data (e.g., the three-dimensional distance and relative angle between the operator and the virtual object) and according to the preset distance threshold or spatial relationship rules.

[0082] For example, in a museum AR guided tour application, if a user's hand approaches a virtual button but their gaze is focused on another virtual button, the system will not trigger the interactive feedback of that virtual button due to the mismatch of intents. Conversely, if the user simultaneously looks at the "Bronze Cauldron" button and moves their hand toward it, the system will trigger feedback in stages based on the change in hand distance: within 1 meter, the edge of the virtual button glows; within 0.5 meters, the surface of the virtual button displays a hand shadow; and within 0.1 meters, when a press is detected, the virtual button deforms and provides tactile simulation, thereby achieving precise intent-driven interaction.

[0083] It is understood that by introducing interaction intent perception as a pre-filter condition, the embodiments of this application make the judgment in the interaction stage more in line with the user's subjective intention, significantly reduce false responses caused by unintentional approach, and improve the accuracy and intelligence of the interaction. Especially in scenarios where multiple virtual objects are densely arranged, it can effectively optimize the management of interaction focus and improve the user experience.

[0084] The following examples illustrate possible implementation schemes of the interaction method described in one or more of the above embodiments.

[0085] This solution, applied to a museum XR display project, uses different visual display formats at varying distances, combined with visual and auditory feedback, to enable users to more clearly perceive the actual distance and accurately trigger virtual buttons. The solution designs visually distinguishable button states, including a default state, a proximity state (approaching on the XYZ axes), a hover or touch state, a pressed state, and a resume or release state. The specific implementation of the solution can include the following four interaction stages: (1) Mid-field interaction (>1m distance range): If interaction is not initiated, the virtual button will display its default state. Figure 5 As shown, the virtual button is in its original default state.

[0086] (2) Mid-range interaction (≤1m distance range): The AI-assisted perception system analyzes user interaction behavior in real time to obtain distance data in the dynamic environment. When the user's eye focuses and the user's current gesture points to the same virtual button, the virtual button changes from its default state to a closer state and displays an edge-glowing animation to indicate that the user has approached the virtual button. For example... Figure 6 As shown, the virtual button emits a glowing animation at its edge.

[0087] (3) Near-field interaction (≤0.5m distance range): The system uses AI-assisted perception to calculate the shadow position of the user's hand on the virtual button, thereby obtaining precise position information of the user's hand relative to the virtual button. During this process, the system displays a visual hand shadow effect on the virtual button to enhance the user's perception of the gesture position. For example... Figure 7 As shown, the virtual button surface displays a hand shadow effect.

[0088] (4) Near-field interaction (≤0.1m distance range). The system uses an algorithm to automatically attach the virtual button to the user's finger, allowing the user to trigger the virtual button by pressing it with their finger (Z-press direction). The system renders the virtual button in a pressed state and plays a shape-changing animation effect, that is, the button model is separated into two parts—the face and the aura. The system processes these two parts separately. The face model implements obstacle avoidance functionality through the Unity engine (e.g., adding a collider to the face model mesh), while the aura in the virtual button model moves forward and pops up with the finger pressing direction (Z-press), such as... Figure 8 As shown in the image. The system will also emit an audio prompt to indicate that the button has been pressed.

[0089] Figure 9 This is a schematic diagram illustrating the visual display effect changes of a complete interactive process provided in an embodiment of this application, such as... Figure 9 As shown, the virtual button presents different visual effects (default effect, outline highlighting effect, hand shadow effect, and button deformation effect) under different interaction states (default state, proximity state, touch state, and press state).

[0090] Understandably, this solution triggers differentiated visual feedback (such as outline highlighting, hand shadows, and button deformation) at different interaction distances, combined with auditory and tactile cues, enabling users to clearly perceive the position and distance of virtual buttons. The shape changes of virtual buttons not only effectively avoid the visual problem of user gestures penetrating virtual buttons, but also make the interaction process more in line with natural operational expectations, thereby improving the overall accuracy, naturalness, and immersion of interaction in the virtual environment.

[0091] It should be noted that this solution can also be applied to more XR scenarios that require precise interaction, such as education, healthcare, and industrial simulation, to help users understand the relationship between virtual objects and real space more intuitively.

[0092] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0093] Figure 10 This is a schematic diagram of the structure of an interactive device provided in an embodiment of this application, such as... Figure 10 As shown, the interactive device 1000 includes: an acquisition module 1010, an interaction module 1020, and a rendering module 1030; wherein: Module 1010 is used to obtain spatial position data between the operating body and the virtual object; The interaction module 1020 is used to determine the interaction stage between the operating body and the virtual object based on the spatial location data. The rendering module 1030 is used to render the virtual object according to the current interaction stage, so that the virtual object presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types.

[0094] In some embodiments, the interaction phase includes a first interaction phase, which represents the operator entering a first adjacent region of the virtual object; when the current interaction phase is the first interaction phase, the rendering module 1030 is further configured to: render the virtual object based on a first visual feature type, so that the virtual object presents a visual display effect with a target outline.

[0095] In some embodiments, the rendering module 1030 is further configured to: render the edge region of the virtual object based on a first visual feature type, so that the virtual object presents a visual display effect with a glowing outline; the visual salience of the glowing outline is higher than the visual salience of the surface of the virtual object body.

[0096] In some embodiments, the interaction phase further includes a second interaction phase, the second interaction phase representing that the operator enters a second neighboring region of the virtual object; the second neighboring region is closer to the virtual object than the first neighboring region; when the current interaction phase is the second interaction phase, the rendering module 1030 is further configured to: render the virtual object based on a second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the operator.

[0097] In some embodiments, the rendering module 1030 is further configured to: determine the projection area of ​​the operation body on the surface of the virtual object body based on the spatial position data and specific light source direction parameters; render the projection area based on a second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the operation body; the shadow effect changes with the change of the spatial position data.

[0098] In some embodiments, the interaction phase further includes a third interaction phase, wherein the third interaction phase represents the operator entering a third neighboring region of the virtual object; the third neighboring region is closer to the virtual object than the second neighboring region; when the current interaction phase is the third interaction phase, the rendering module 1030 is further configured to: render the virtual object based on a third visual feature type, so that the virtual object presents a visual display effect of target spatial structure and / or relative displacement of different parts of the virtual object.

[0099] In some embodiments, the rendering module 1030 is further configured to: render the virtual object as a combination of at least two visual objects based on a third visual feature type; and control the at least two visual objects to produce relative displacement in response to a pressing operation applied to the virtual object by the operator, so that the virtual object presents a pressed visual display effect.

[0100] In some embodiments, the at least two visual objects include a first visual object and a second visual object. The first visual object is configured with a collider for interacting with the operator during the pressing operation. The rendering module 1030 is further configured to: in response to a pressing operation applied by the operator to the first visual object, use the collider to restrict the operator to the surface of the first visual object, and control the second visual object to generate a displacement relative to the first visual object in the opposite direction of the pressing operation, so that the virtual object presents a pressed visual display effect.

[0101] In some embodiments, the rendering module 1030 is further configured to: control the virtual object to move closer to the operating body when the current interaction stage is the third interaction stage, so that the spatial position of the virtual object is aligned with the spatial position of the operating body.

[0102] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0103] Figure 11This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 11 The illustrated electronic device 1100 includes a processor 1101, which can retrieve and execute computer-executable instructions or computer programs from memory to perform the following steps: Obtain spatial position data between the operating entity and the virtual object; Based on the spatial location data, the interaction stage between the operator and the virtual object is determined; For each interaction stage, the virtual object is rendered so that it presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types.

[0104] Optionally, such as Figure 11 As shown, the electronic device 1100 may further include a memory 1102. The processor 1101 can retrieve and run computer programs from the memory 1102 to implement the methods described in the embodiments of this application.

[0105] The memory 1102 can be a separate device independent of the processor 1101, or it can be integrated into the processor 1101.

[0106] Optionally, such as Figure 11 As shown, the electronic device 1100 may also include a transceiver 1103, and the processor 1101 may control the transceiver 1103 to communicate with other devices. Specifically, it may send information or data to other devices or receive information or data sent by other devices.

[0107] The transceiver 1103 may include a transmitter and a receiver. The transceiver 1103 may further include an antenna, and the number of antennas may be one or more.

[0108] This application also provides a computer-readable storage medium for storing computer programs.

[0109] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of this application, and the computer program causes the processor or electronic device to perform the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0110] This application also provides a computer program product, including computer program instructions.

[0111] Optionally, the computer program product can be applied to the electronic device in the embodiments of this application, and the computer program instructions cause the processor or electronic device to execute the various methods of the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0112] This application also provides a computer program.

[0113] Optionally, the computer program can be applied to the electronic device in the embodiments of this application. When the computer program runs on the processor or electronic device, it causes the processor or electronic device to execute the various methods of the embodiments of this application. For the sake of brevity, it will not be described in detail here.

[0114] It should be noted that the descriptions of the devices, storage media, computer program products, and computer program embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the device, storage media, computer program products, and computer program embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0115] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0116] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0117] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0118] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0119] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0120] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0121] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0122] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0123] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a wearable device, mobile phone, tablet computer, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An interaction method, comprising: Obtain spatial position data between the operating entity and the virtual object; Based on the spatial location data, the interaction stage between the operator and the virtual object is determined; For each interaction stage, the virtual object is rendered so that it presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types.

2. The method according to claim 1, wherein the interaction phase includes a first interaction phase, the first interaction phase representing the operator entering a first adjacent region of the virtual object; When the current interaction stage is the first interaction stage, rendering the virtual object for the current interaction stage includes: The virtual object is rendered based on the first visual feature type, so that the virtual object presents a visual display effect with a target outline.

3. The method according to claim 2, wherein rendering the virtual object based on the first visual feature type, so that the virtual object presents a visual display effect with a target outline, includes: The edge region of the virtual object is rendered based on the first visual feature type, so that the virtual object presents a visual display effect with a glowing outline; the visual salience of the glowing outline is higher than the visual salience of the surface of the virtual object itself.

4. The method according to claim 2, wherein the interaction phase further comprises a second interaction phase, the second interaction phase representing that the operator enters a second neighboring region of the virtual object; the second neighboring region is closer to the virtual object than the first neighboring region; When the current interaction stage is the second interaction stage, rendering the virtual object for the current interaction stage includes: The virtual object is rendered based on the second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the manipulation object.

5. The method according to claim 4, wherein rendering the virtual object based on a second visual feature type, such that the surface of the virtual object body presents a shadow effect corresponding to the shape of the manipulation object, includes: Based on the spatial location data and specific light source direction parameters, the projection area of ​​the operating body on the surface of the virtual object is determined; The projection area is rendered based on the second visual feature type, so that the surface of the virtual object body presents a shadow effect corresponding to the shape of the manipulator; the shadow effect changes with the change of the spatial position data.

6. The method according to claim 4, wherein the interaction phase further comprises a third interaction phase, the third interaction phase representing that the operator enters a third neighboring region of the virtual object; the third neighboring region is closer to the virtual object than the second neighboring region; When the current interaction stage is the third interaction stage, rendering the virtual object for the current interaction stage includes: The virtual object is rendered based on the third visual feature type, so that the virtual object presents a visual display effect of target spatial structure and / or relative displacement of different parts of the virtual object.

7. The method according to claim 6, wherein rendering the virtual object based on a third visual feature type, such that the virtual object presents a visual display effect of a target spatial structure and / or different parts of the virtual object undergoing relative displacement, comprises: Based on the third visual feature type, the virtual object is rendered as a combination of at least two visual objects; In response to a pressing operation applied to the virtual object by the operating body, the at least two visual objects are controlled to produce a relative displacement, so that the virtual object presents a pressed visual display effect.

8. The method according to claim 7, wherein the at least two visual objects include a first visual object and a second visual object, the first visual object being configured with a collider for interacting with the operating body during the pressing operation; the step of controlling the at least two visual objects to generate relative displacement in response to the pressing operation applied by the operating body to the virtual object, so that the virtual object presents a pressed visual display effect, includes: In response to the pressing operation applied by the operating body to the first visual object, the operating body is constrained to the surface of the first visual object by the collision body, and the second visual object is controlled to generate a displacement relative to the first visual object in the opposite direction of the pressing operation, so that the virtual object presents a pressed visual display effect.

9. The method according to any one of claims 6 to 8, further comprising: When the current interaction stage is the third interaction stage, the virtual object is controlled to move closer to the operating body, so that the spatial position of the virtual object is aligned with the spatial position of the operating body.

10. An electronic device, comprising: Memory is used to store executable instructions or computer programs. When a processor executes executable data instructions or computer programs stored in the memory, it performs the following steps: Obtain spatial position data between the operating entity and the virtual object; Based on the spatial location data, the interaction stage between the operator and the virtual object is determined; For each interaction stage, the virtual object is rendered so that it presents different visual display effects as the interaction stage changes, and the different visual display effects have different visual feature types.

Images