Space interaction method and device and electronic equipment

Abstract

The invention discloses a space interaction method and device and electronic equipment, and belongs to the technical field of interaction.The method comprises the steps that eye movement information of a user is obtained; under the condition that the eye movement information indicates that the eye movement position is located in the first operation space, in response to a first input of the user to the target object, executing a first operation on the target object in the first operation space; wherein the target object is displayed in the first operation space.

Description

Technical Field

[0001] This application belongs to the field of interaction technologies, and particularly relates to a spatial interaction method, apparatus, and electronic device. Background Art

[0002] With the continuous development of technology, the interaction between users and electronic devices not only exists within the original operation space where the manipulable object is located, but also enables cross-space interaction of the manipulable object. However, in actual application scenarios, there are cases where the user intends to perform relevant operations on the manipulable object at the edge position of the original operation space, but due to deviations generated when the user drags the manipulable object, the manipulable object is accidentally moved outside the original operation space, resulting in cross-space interaction, that is, the interaction result at this time conflicts with the user's intention.

[0003] It can be seen that there is a risk of misoperation in current spatial interaction, making it difficult to meet the actual interaction needs of users. Summary of the Invention

[0004] The purpose of the embodiments of this application is to provide a spatial interaction method, apparatus, and electronic device, which can reduce the risk of misoperation in spatial interaction, thereby improving the consistency between the interaction result and the user's interaction intention, and further better meeting the actual interaction needs of users.

[0005] In a first aspect, the embodiments of this application provide a spatial interaction method, which includes:

[0006] Obtain the eye movement information of the user;

[0007] When the eye movement information indicates that the eye movement position is within the first operation space, in response to the user's first input to the target object, perform a first operation on the target object within the first operation space;

[0008] Wherein, the target object is displayed within the first operation space.

[0009] In a second aspect, the embodiments of this application provide a spatial interaction apparatus, which includes:

[0010] A first acquisition module, configured to obtain the eye movement information of the user;

[0011] A first interaction module, configured to, when the eye movement information indicates that the eye movement position is within the first operation space, in response to the user's first input to the target object, perform a first operation on the target object within the first operation space;

[0012] Wherein, the target object is displayed within the first operation space.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory. The memory stores a program or instructions that can run on the processor. When the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instructions are stored. When the program or instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0015] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instructions to implement the method described in the first aspect.

[0016] In a sixth aspect, an embodiment of the present application provides a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0017] In the embodiment of the present application, eye movement information of a user can be obtained; when the eye movement information indicates that the eye movement position is within a first operation space, in response to a first input of the user to a target object displayed in the first operation space, a first operation is performed on the target object within the first operation space. In this way, the user interaction intention is identified through the eye movement information of the user. If the eye movement position is within the first operation space, it can be considered that the user interaction intention is in the first operation space. At this time, the operation of the user on the target object can be locked within the first operation space, reducing the risk of misoperation in spatial interaction, thereby improving the consistency between the interaction result and the user interaction intention, and further better meeting the actual interaction needs of the user. Description of the Drawings

[0018] Figure 1 is a flowchart of the spatial interaction method provided by an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of eye movement information in the spatial interaction method provided by an embodiment of the present application;

[0020] Figure 3 is one of the schematic diagrams of the first operation space and the target object in the spatial interaction method provided by an embodiment of the present application;

[0021] Figure 4 is Figure 3 one of the operation schematic diagrams of the target object in

[0022] Figure 5 is Figure 3 the second operation schematic diagram of the target object in

[0023] Figure 6 It is Figure 3 Figure 3 of the operation schematic diagram of the target object in

[0024] Figure 7 It is Figure 2 of the schematic diagram of the first operation space and the target object in the spatial interaction method provided by the embodiment of the present application;

[0025] Figure 8 It is Figure 7 Figure 1 of the operation schematic diagram of the target object in

[0026] Figure 9 It is Figure 7 Figure 2 of the operation schematic diagram of the target object in

[0027] Figure 10 It is Figure 7 Figure 3 of the operation schematic diagram of the target object in

[0028] Figure 11 It is the structural schematic diagram of the spatial interaction device provided by the embodiment of the present application;

[0029] Figure 12 It is the structural schematic diagram of the electronic device provided by the embodiment of the present application;

[0030] Figure 13 It is the hardware structural schematic diagram of the electronic device provided by the embodiment of the present application. Detailed implementation manners

[0031] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0032] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually of the same category, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims means at least one of the connected objects, and the character " / ", generally represents an "or" relationship between the associated objects before and after.

[0033] Next, the spatial interaction method provided by the embodiment of the present application will be described in detail in conjunction with the accompanying drawings, through specific embodiments and their application scenarios.

[0034] Figure 1 It is a schematic flowchart of the spatial interaction method provided by an embodiment of the present application. The spatial interaction method may include:

[0035] Step 101, obtain the eye movement information of the user.

[0036] In step 101, the eye movement information of the user can be obtained. Among them, the eye movement information may include the eye movement position and eye movement speed of the user, etc.

[0037] Step 102, when the eye movement information indicates that the eye movement position is within the first operation space, in response to the first input of the user on the target object, perform a first operation on the target object within the first operation space; wherein, the target object is displayed in the first operation space.

[0038] In step 102, the first operation space may be a planar window or a three-dimensional space. The target object may be an operable object within the first operation space. For example, the target object may be a tab page within a planar window, and the user can drag the tab page to a specified position within the planar window to open the page of the tab page. The target object may also be a chess piece within a three-dimensional space, and the user can move the chess piece to play chess within the three-dimensional space.

[0039] It can be understood that the user can perform relevant operations on the target object through the first input. Among them, the target object can only move the display position in response to the first input. For example, the user can move the display position of the target object to the position expected by the user through the first input. The target object can also perform other relevant functions while moving the display position in response to the first input. For example, the user can drag the tab page to move the tab page, and when releasing the hand, open the page corresponding to the tab page at the corresponding position.

[0040] The first input may include a drag input and may also include an input where the user quickly clicks along the expected movement route, which is not specifically limited here.

[0041] As Figure 2 shown, if the eye movement information indicates that the eye movement position is within the first operation space 210, it can be considered at this time that the user's interaction intention is within the first operation space 210. In other words, at this time, regardless of whether the target object 211 is dragged out of the first operation space 210 by the user, a first operation is performed on the target object 211 within the first operation space 210 in response to the first input.

[0042] It can be understood that if the user drags the target object 211 out of the first operation space 210 at this time, it can be considered as a user's misoperation. The user's true intention may be to perform a first operation on the target object 211 in the edge area of the first operation space 210. Therefore, the operation of the user on the target object 211 is locked within the first operation space 210.

[0043] In some examples, such as Figure 3 and Figure 4 shown, taking the first operation space 210 as a plane window and the target object 211 as a tab page as an example, in response to the user's drag input on the tab page, the display position of the tab page moves. When the user releases the hand, the page corresponding to the tab page is opened within the current plane window.

[0044] In other examples, such as Figure 7 and Figure 8 shown, taking the first operation space 210 as a three-dimensional space and the target object 211 as a chess piece, and there is a corresponding chessboard set in this three-dimensional space as an example, in response to the user's drag input on the chess piece, the display position of the chess piece can move within this three-dimensional space so that the chess piece reaches different positions on the chessboard.

[0045] In the embodiments of the present application, the spatial interaction method can obtain the user's eye movement information; when the eye movement information indicates that the eye movement position is within the first operation space, in response to the user's first input on the target object displayed in the first operation space, perform a first operation on the target object within the first operation space. In this way, the user's interaction intention is recognized through the user's eye movement information. If the eye movement position is within the first operation space, it can be considered that the user's interaction intention is within the first operation space. At this time, the operation of the user on the target object can be locked within the first operation space, reducing the risk of misoperation in spatial interaction, thereby improving the consistency between the interaction result and the user's interaction intention, and further better meeting the actual interaction needs of the user.

[0046] In some embodiments, after obtaining the user's eye movement information, the method further includes:

[0047] When the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is less than or equal to a preset eye movement threshold, in response to the user's first input on the target object, determine the movement parameters of the target object, where the movement parameters include at least one of the movement speed and the movement acceleration;

[0048] When the movement parameters are less than or equal to a preset movement threshold, in response to the first input, perform a first operation on the target object within the first operation space.

[0049] In this embodiment, if the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is less than or equal to a preset eye movement threshold. For example, when the user's eye movement speed is less than or equal to 5° / s, there may be two situations at this time: one is that the user's attention is not concentrated, and the other is that the user has an interaction intention outside the first operation space. At this time, it is necessary to further determine which situation it belongs to by the movement parameters of the target object in response to the first input.

[0050] The movement parameters may include at least one of the movement speed and the movement acceleration. In the case where the movement parameters are less than or equal to a preset movement threshold, in response to the first input, a first operation is performed on the target object within the first operation space.

[0051] For example, in the case where the movement speed of the user dragging the target object is less than or equal to a preset speed threshold, or the movement acceleration is less than or equal to a preset acceleration threshold, it can be considered that the user's eye movement outside the first operation space may be caused by inattention, and the user's interaction intention outside the first operation space is not strong. At this time, the user's operation on the target object can be locked within the first operation space. Among them, the preset speed threshold may include 1 to 5 cm / s, such as it can be set to 5 cm / s, and the preset acceleration threshold can be set to 0.1 m / s². There is no specific limitation here.

[0052] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is slow, the interaction intention of the user can be recognized by the movement parameters of the target object in response to the first input. If the movement parameters are less than or equal to the preset movement threshold, it can be considered that the user's interaction intention outside the first operation space is not strong. At this time, the user's operation on the target object can be locked within the first operation space, further reducing the risk of spatial interaction misoperation and better meeting the actual interaction needs of the user.

[0053] In some embodiments, after determining the movement parameters of the target object, the method further includes:

[0054] In the case where the movement parameters are greater than the preset movement threshold, in response to the first input, determine the real-time display position of the target object;

[0055] In the case where at least part of the real-time display position is outside the first operation space, in response to the first input, perform a first operation on the target object outside the first operation space.

[0056] In this embodiment, when the moving speed of the target object dragged by the user is greater than the preset speed threshold or the moving acceleration is greater than the preset acceleration threshold, it can be considered that the user has a strong interaction intention outside the first operation space. At this time, the real-time display position of the target object in response to the first input can be determined, and based on the real-time display position, it can be judged whether the user's operation on the target object is within the first operation space or outside the first operation space.

[0057] If the real-time display position is still within the first operation space, then in response to the first input, a first operation can be performed on the target object within the first operation space.

[0058] For example, as Figure 4 shown, if the user drags a tab page and the real-time display position of the tab page when released is still within the current plane window, then the page corresponding to the tab page is opened within the current plane window. Another example, as Figure 8 shown, the user drags a chess piece and the real-time display position of the chess piece when released is still within the original three-dimensional space, then at this time the chess piece remains fixed at the corresponding position on the three-dimensional space chessboard.

[0059] If at least part of the real-time display position is outside the first operation space, since the user has a strong interaction intention outside the first operation space at this time, it can be considered that the operation of the user moving the target object outside the first operation space is not a misoperation. Then, in response to the first input, a first operation can be performed on the target object outside the first operation space.

[0060] In some examples, as Figure 5 shown, taking the first operation space 210 as a plane window and the target object 211 as a tab page as an example, after the user drags the tab page out of the original plane window, a new window can be generated at the release position, and the page corresponding to the tab page is opened in the new window.

[0061] In other examples, as Figure 9 shown, taking the first operation space 210 as a three-dimensional space, the target object 211 as a chess piece, and there is a corresponding chessboard set in the three-dimensional space as an example, after the user drags the chess piece out of the original three-dimensional space, the real-time display position of the chess piece can be outside the three-dimensional space.

[0062] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is slow, the target object can identify the user's interaction intention in response to the movement parameter of the first input. If the movement parameter is greater than the preset movement threshold, it can be considered that the user has a relatively strong interaction intention outside the first operation space. At this time, if at least part of the real-time display position of the target object is outside the first operation space, the first operation can be performed on the target object outside the first operation space to improve the consistency between the interaction result and the user's interaction intention and better meet the user's actual interaction needs.

[0063] In some embodiments, after obtaining the user's eye movement information, the method further includes:

[0064] When the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is greater than the preset eye movement threshold, in response to the user's first input to the target object, determine the real-time display position of the target object;

[0065] When at least part of the real-time display position is outside the first operation space, in response to the first input, perform the first operation on the target object outside the first operation space.

[0066] In this embodiment, if the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is greater than the preset eye movement threshold. For example, when the user's eye movement speed is greater than 5° / second, there is no need to further identify the user's interaction intention through the movement parameter of the target object in response to the first input, and it can be directly considered that the user's interaction intention outside the first operation space is relatively strong. At this time, the real-time display position of the target object in response to the first input can be determined, and based on the real-time display position, it can be judged whether the user's operation on the target object is within the first operation space or outside the first operation space.

[0067] As mentioned above, at this time, if the real-time display position is still within the first operation space, the first operation can be performed on the target object within the first operation space in response to the first input.

[0068] If at least part of the real-time display position is outside the first operation space, since the user's interaction intention outside the first operation space is relatively strong at this time, it can be considered that the user's operation of moving the target object outside the first operation space is not a misoperation. Then, the first operation can be performed on the target object outside the first operation space in response to the first input.

[0069] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is fast, it can be considered that the user has a relatively strong interaction intention outside the first operation space. At this time, if at least part of the real-time display position of the target object is outside the first operation space, the first operation can be performed on the target object outside the first operation space to improve the consistency between the interaction result and the user's interaction intention and better meet the user's actual interaction needs.

[0070] In some embodiments, the method further includes:

[0071] Obtaining a plurality of historical interaction operations of the user;

[0072] Determining, among the plurality of historical interaction operations, the first number of times of performing a first operation on a target object within a first operation space and the second number of times of performing the first operation on the target object outside the first operation space;

[0073] When the first number of times meets a preset condition, in response to a first input from the user, performing the first operation on the target object within the first operation space;

[0074] When the second number of times meets a preset condition, in response to the first input, determining the real-time display position of the target object;

[0075] When at least a part of the real-time display position is outside the first operation space, in response to the first input, performing the first operation on the target object outside the first operation space.

[0076] In this embodiment, the interaction intention of the current user operation can also be identified by judging the situation of the user's historical interaction operations.

[0077] A plurality of historical interaction operations of the user can be obtained. Each historical interaction operation can include an interaction result, and the interaction result can indicate whether the operation of the user on the target object is within the first operation space or outside the first operation space.

[0078] The number of times of performing the first operation on the target object within the first operation space can be counted to obtain the first number of times, and the number of times of performing the first operation on the target object outside the first operation space can be counted to obtain the second number of times.

[0079] The sum of the first number of times and the second number of times can be the total number of times. It can be understood that the total number of times can be equal to the number of the obtained plurality of historical interaction operations.

[0080] When the first number of times meets a preset condition, in response to a first input from the user, performing the first operation on the target object within the first operation space. Exemplarily, if the first number of times is greater than a preset number threshold, or if the ratio between the first number of times and the total number of times is greater than a preset ratio, it can be considered that the first number of times meets the preset condition. For example, if the proportion of the first number of times is greater than 80%, it can be considered that the interaction intention of the current user may also be within the first operation space, and then in response to the first input, the first operation can be performed on the target object within the first operation space.

[0081] When the second number meets the preset condition, in response to the first input, determine the real-time display position of the target object; when at least part of the real-time display position is outside the first operation space, in response to the first input, perform a first operation on the target object outside the first operation space. By way of example, if the second number is greater than the preset number threshold, or if the ratio between the second number and the total number is greater than the preset ratio, it can be considered that the second number meets the preset condition. For example, if the proportion of the second number is greater than 80%, it can be considered that the current interaction intention of the user may also be outside the first operation space. At this time, determine the real-time display position of the target object in response to the first input. When at least part of the real-time display position is outside the first operation space, in response to the first input, perform a first operation on the target object outside the first operation space.

[0082] In this way, through the proportion of the number of times of performing the first operation on the target object within the first operation space and the proportion of the number of times of performing the first operation on the target object outside the first operation space in multiple historical interaction operations, the current interaction intention of the user can be identified, and interaction operations can be performed based on this interaction intention to improve the consistency between the interaction result and the user's interaction intention.

[0083] In some embodiments, after performing the first operation on the target object outside the first operation space in response to the first input, the method further includes:

[0084] When the eye movement information indicates that the eye movement position is within the second operation space and the distance between the projection of the target object and the second operation space is less than the preset distance threshold, perform the first operation on the target object within the second operation space.

[0085] In this embodiment, if the eye movement information indicates that the eye movement position is within the second operation space, and the user drags the target object out of the first operation space, and at this time the distance between the target object and the second operation space is less than the preset distance threshold, the target object can be directly adsorbed into the second operation space, and the first operation can be performed on the target object within the second operation space.

[0086] It can be understood that the distance between the projection of the target object and the second operation space can refer to the minimum straight-line distance between the center point of the projection of the target object and the boundary of the second operation space, or the minimum straight-line distance between the edge of the projection of the target object and the boundary of the second operation space, or the minimum straight-line distance between the center point of the projection of the target object and the center point of the second operation space. No specific limitation is made here.

[0087] In some examples, such as Figure 6As shown, taking the first operation space 210 as a plane window and the target object 211 as a tab page as an example, after the user drags the tab page out of the original plane window, if the distance between the projection of the tab page and the second operation space 220 is less than a preset distance threshold, for example, less than 100 dmm, the tab can be adsorbed into the second operation space 220. The page corresponding to the tab page can be opened within the second operation space 220.

[0088] In some other examples, such as Figure 10 As shown, taking the first operation space 210 as a three-dimensional space and the target object 211 as a chess piece, and taking the corresponding chessboard is set in this three-dimensional space as an example, after the user drags the chess piece out of the original three-dimensional space, if the distance between the projection of the chess piece and the second operation space 220 is less than the preset distance threshold, the chess piece can be adsorbed into the second operation space 220 so that the chess piece can reach different positions on the chessboard within the second operation space.

[0089] In this way, when the eye movement position is within the second operation space, it can be considered that the user's interaction intention is in the second operation space. At this time, if the target object leaves the first operation space and the projection is close to the second operation space, it can be directly adsorbed into the second operation space, so as to simplify the operation process of the user moving the target object to the second operation space in combination with the user's interaction intention, improving the accuracy and convenience of the interaction.

[0090] In the space interaction method provided by the embodiments of the present application, the execution subject can be a space interaction device. In the embodiments of the present application, taking the space interaction device executing the space interaction method as an example, the space interaction device provided by the embodiments of the present application is described.

[0091] Such as Figure 11 As shown, the space interaction device 1100 may include:

[0092] A first acquisition module 1101, configured to acquire the user's eye movement information;

[0093] A first interaction module 1102, configured to, when the eye movement information indicates that the eye movement position is within the first operation space, in response to a first input of the user on the target object, perform a first operation on the target object within the first operation space;

[0094] Wherein, the target object is displayed in the first operation space.

[0095] In this way, the user's interaction intention is identified through the user's eye movement information. If the eye movement position is within the first operation space, it can be considered that the user's interaction intention is in the first operation space. At this time, the operation of the user on the target object can be locked within the first operation space, reducing the risk of misoperation in space interaction, thereby improving the consistency between the interaction result and the user's interaction intention, and further better meeting the actual interaction needs of the user.

[0096] In some embodiments, the spatial interaction device 1100 may further include:

[0097] A first determination module, configured to determine movement parameters of a target object in response to a first input from a user when the eye movement information indicates that the eye movement position is outside a first operation space and the eye movement speed is less than or equal to a preset eye movement threshold, where the movement parameters include at least one of a movement speed and a movement acceleration;

[0098] A second interaction module, configured to perform a first operation on the target object within the first operation space in response to the first input when the movement parameters are less than or equal to a preset movement threshold.

[0099] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is slow, the movement parameters of the target object in response to the first input can be used to identify the user's interaction intention. If the movement parameters are less than or equal to the preset movement threshold, it can be considered that the user's interaction intention outside the first operation space is not strong. At this time, the operation of the user on the target object can be locked within the first operation space, further reducing the risk of spatial interaction misoperation and better meeting the actual interaction needs of the user.

[0100] In some embodiments, the spatial interaction device 1100 may further include:

[0101] A second determination module, configured to determine the real-time display position of the target object in response to the first input when the movement parameters are greater than the preset movement threshold;

[0102] A third interaction module, configured to perform a first operation on the target object outside the first operation space in response to the first input when at least part of the real-time display position is outside the first operation space.

[0103] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is slow, the movement parameters of the target object in response to the first input can be used to identify the user's interaction intention. If the movement parameters are greater than the preset movement threshold, it can be considered that the user has a relatively strong interaction intention outside the first operation space. At this time, if at least part of the real-time display position of the target object is outside the first operation space, a first operation can be performed on the target object outside the first operation space to improve the consistency between the interaction result and the user's interaction intention and better meet the actual interaction needs of the user.

[0104] In some embodiments, the spatial interaction device 1100 may further include:

[0105] A third determination module, configured to determine a real-time display position of a target object in response to a first input from a user when the eye movement information indicates that the eye movement position is outside a first operation space and the eye movement speed is greater than a preset eye movement threshold;

[0106] A fourth interaction module, configured to perform a first operation on the target object outside the first operation space in response to the first input when at least a part of the real-time display position is outside the first operation space.

[0107] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is relatively fast, it can be considered that the user has a relatively strong intention to interact outside the first operation space. At this time, if at least a part of the real-time display position of the target object is outside the first operation space, the first operation can be performed on the target object outside the first operation space to improve the consistency between the interaction result and the user's interaction intention, and better meet the actual interaction needs of the user.

[0108] In some embodiments, the spatial interaction device 1100 may further include:

[0109] A second acquisition module, configured to acquire multiple historical interaction operations of the user;

[0110] A statistics module, configured to determine a first number of times of performing the first operation on the target object within the first operation space and a second number of times of performing the first operation on the target object outside the first operation space among the multiple historical interaction operations;

[0111] A fifth interaction module, configured to perform the first operation on the target object within the first operation space in response to the first input from the user when the first number of times meets a preset condition;

[0112] A fourth determination module, configured to determine the real-time display position of the target object in response to the first input when the second number of times meets a preset condition;

[0113] A sixth interaction module, configured to perform the first operation on the target object outside the first operation space in response to the first input when at least a part of the real-time display position is outside the first operation space.

[0114] In this way, the current interaction intention of the user can be identified by the proportion of the number of times of performing the first operation on the target object within the first operation space and the proportion of the number of times of performing the first operation on the target object outside the first operation space among the multiple historical interaction operations, and the interaction operation is performed based on this interaction intention to improve the consistency between the interaction result and the user's interaction intention.

[0115] In some embodiments, the spatial interaction device 1100 may further include:

[0116] The seventh interaction module is configured to, after performing a first operation on a target object outside a first operation space in response to a first input, perform the first operation on the target object in a second operation space when the eye movement information indicates that the eye movement position is within the second operation space and the distance between the projection of the target object and the second operation space is less than a preset distance threshold.

[0117] In this way, when the eye movement position is within the second operation space, it can be considered that the user's interaction intention is in the second operation space. At this time, if the target object leaves the first operation space and the projection is close to the second operation space, it can be directly adsorbed into the second operation space, so as to simplify the operation process of the user moving the target object to the second operation space in combination with the user's interaction intention, improving the accuracy and convenience of the interaction.

[0118] The spatial interaction device in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than terminals. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a palm computer, a vehicle-mounted electronic device, a Mobile Internet Device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and can also be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present application do not make specific limitations.

[0119] The spatial interaction device in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems. The embodiments of the present application do not make specific limitations.

[0120] The spatial interaction device provided by the embodiments of the present application can implement Figures 1 to 10 each process implemented by the method embodiments. To avoid repetition, it will not be elaborated here.

[0121] Optionally, as Figure 12As shown in the figure, an embodiment of the present application further provides an electronic device 1200, which includes a processor 1201 and a memory 1202. A program or instruction that can run on the processor 1201 is stored on the memory 1202. When the program or instruction is executed by the processor 1201, it implements each step of the above-mentioned embodiment of the spatial interaction method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0122] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0123] Figure 13 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application.

[0124] The electronic device 1300 includes, but is not limited to: a radio frequency unit 1301, a network module 1302, an audio output unit 1303, an input unit 1304, a sensor 1305, a display unit 1306, a user input unit 1307, an interface unit 1308, a memory 1309, and a processor 1310 and other components.

[0125] Those skilled in the art can understand that the electronic device 1300 may further include a power source (such as a battery) for supplying power to each component. The power source can be logically connected to the processor 1310 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 13 The structure of the electronic device shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements, which will not be elaborated here.

[0126] Among them, the processor 1310 can be used to:

[0127] Obtain the eye movement information of the user;

[0128] In the case that the eye movement information indicates that the eye movement position is within the first operation space, in response to the user's first input to the target object, perform a first operation on the target object within the first operation space;

[0129] Among them, the target object is displayed in the first operation space.

[0130] In this way, the user's interaction intention is recognized through the user's eye movement information. If the eye movement position is within the first operation space, it can be considered that the user's interaction intention is in the first operation space. At this time, the operation of the user on the target object can be locked within the first operation space, reducing the risk of misoperation in spatial interaction, thereby improving the consistency between the interaction result and the user's interaction intention, and further better meeting the actual interaction needs of the user.

[0131] In some embodiments, the processor 1310 may also be used for:

[0132] When the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is less than or equal to a preset eye movement threshold, in response to a first input from the user to a target object, determine the movement parameters of the target object, where the movement parameters include at least one of a movement speed and a movement acceleration;

[0133] When the movement parameters are less than or equal to a preset movement threshold, in response to the first input, perform a first operation on the target object within the first operation space.

[0134] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is slow, the target object can identify the user's interaction intention based on the movement parameters in response to the first input. If the movement parameters are less than or equal to the preset movement threshold, it can be considered that the user's interaction intention outside the first operation space is not strong. At this time, the operation of the user on the target object can be locked within the first operation space, further reducing the risk of spatial interaction misoperation and better meeting the actual interaction needs of the user.

[0135] In some embodiments, the processor 1310 may also be used for:

[0136] When the movement parameters are greater than a preset movement threshold, in response to the first input, determine the real-time display position of the target object;

[0137] When at least part of the real-time display position is outside the first operation space, in response to the first input, perform a first operation on the target object outside the first operation space.

[0138] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is slow, the target object can identify the user's interaction intention based on the movement parameters in response to the first input. If the movement parameters are greater than the preset movement threshold, it can be considered that the user has a stronger interaction intention outside the first operation space. At this time, if at least part of the real-time display position of the target object is outside the first operation space, a first operation can be performed on the target object outside the first operation space to improve the consistency between the interaction result and the user's interaction intention and better meet the actual interaction needs of the user.

[0139] In some embodiments, the processor 1310 may also be used for:

[0140] When the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is greater than a preset eye movement threshold, in response to a first input from the user to a target object, determine the real-time display position of the target object;

[0141] When at least part of the real-time display position of the target object is outside the first operation space, in response to a first input, a first operation is performed on the target object outside the first operation space.

[0142] In this way, when the user's eye movement position is outside the first operation space and the eye movement speed is relatively fast, it can be considered that the user has a relatively strong intention to interact outside the first operation space. At this time, if at least part of the real-time display position of the target object is outside the first operation space, a first operation can be performed on the target object outside the first operation space to improve the consistency between the interaction result and the user's interaction intention and better meet the user's actual interaction needs.

[0143] In some embodiments, the processor 1310 may further be configured to:

[0144] Obtain a plurality of historical interaction operations of the user;

[0145] Determine the first number of times of performing the first operation on the target object within the first operation space and the second number of times of performing the first operation on the target object outside the first operation space among the plurality of historical interaction operations;

[0146] When the first number of times meets a preset condition, in response to the user's first input to the target object, perform the first operation on the target object within the first operation space;

[0147] When the second number of times meets a preset condition, in response to the first input, determine the real-time display position of the target object;

[0148] When at least part of the real-time display position is outside the first operation space, in response to the first input, perform the first operation on the target object outside the first operation space.

[0149] In this way, the proportion of the number of times of performing the first operation on the target object within the first operation space and the proportion of the number of times of performing the first operation on the target object outside the first operation space among the plurality of historical interaction operations can be used to identify the user's current interaction intention, and interaction operations are performed based on this interaction intention to improve the consistency between the interaction result and the user's interaction intention.

[0150] In some embodiments, the processor 1310 may further be configured to:

[0151] When the eye movement information indicates that the eye movement position is within the second operation space and the distance between the projection of the target object and the second operation space is less than a preset distance threshold, perform the first operation on the target object within the second operation space.

[0152] In this way, when the eye movement position is within the second operation space, it can be considered that the user's interaction intention is in the second operation space. At this time, if the target object leaves the first operation space and the projection is relatively close to the second operation space, it can be directly adsorbed into the second operation space, so as to simplify the operation process of the user moving the target object to the second operation space in combination with the user's interaction intention, improving the accuracy and convenience of the interaction.

[0153] It should be understood that in the embodiments of the present application, the input unit 1304 may include a Graphics Processing Unit (GPU) 13041 and a microphone 13042. The graphics processor 13041 processes the image data of static pictures or videos obtained by an image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 1306 may include a display panel 13061, and the display panel 13061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 1307 includes at least one of a touch panel 13071 and other input devices 13072. The touch panel 13071 is also referred to as a touch screen. The touch panel 13071 may include two parts: a touch detection device and a toucher. The other input devices 13072 may include, but are not limited to, a physical keyboard, function keys (such as volume keys, power keys, etc.), a trackball, a mouse, and a joystick, which will not be elaborated here.

[0154] The memory 1309 can be used to store software programs and various data. The memory 1309 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1309 may include a volatile memory or a non-volatile memory, or the memory 1309 may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synch link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1309 in the embodiments of the present application includes, but is not limited to, these and any other suitable types of memories.

[0155] The processor 1310 may include one or more processing units; optionally, the processor 1310 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1310.

[0156] The embodiments of the present application further provide a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above-mentioned embodiment of the spatial interaction method and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0157] Among them, the processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory ROM, random access memory RAM, magnetic disks, or optical discs, etc.

[0158] Another embodiment of the present application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement each process of the above embodiment of the spatial interaction method, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0159] It should be understood that the chip mentioned in the embodiments of the present application may also be referred to as a system-on-chip, system chip, chip system, or system-on-chip, etc.

[0160] The embodiments of the present application provide a computer program product. The program product is stored in a storage medium and is executed by at least one processor to implement each process of the above embodiment of the spatial interaction method, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0161] It should be noted that in this article, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0162] Through the description of the above embodiments, those skilled in the art can clearly understand that the above method of the embodiment can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence or the part that contributes to the prior art, can be embodied in the form of a computer software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0163] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific implementation manners. The above specific implementation manners are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them belong to the protection scope of the present application.

Claims

1. A spatial interaction method, characterized in that, The method includes: Obtaining the eye movement information of the user; When the eye movement information indicates that the eye movement position is within the first operation space, in response to the first input of the user to the target object, performing a first operation on the target object within the first operation space; Wherein, the target object is displayed within the first operation space.

2. The method according to claim 1, wherein After obtaining the eye movement information of the user, the method further includes: When the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is less than or equal to a preset eye movement threshold, in response to the first input of the user to the target object, determining the movement parameter of the target object, where the movement parameter includes at least one of movement speed and movement acceleration; When the movement parameter is less than or equal to a movement threshold, in response to the first input, performing a first operation on the target object within the first operation space.

3. The method according to claim 2, wherein After determining the movement parameter of the target object, the method further includes: When the movement parameter is greater than the movement threshold, in response to the first input, determining the real-time display position of the target object; When at least part of the real-time display position is outside the first operation space, in response to the first input, performing a first operation on the target object outside the first operation space.

4. The method according to claim 1, characterized in that, After obtaining the eye movement information of the user, the method further includes: When the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is greater than a preset eye movement threshold, in response to the first input of the user to the target object, determining the real-time display position of the target object; When at least part of the real-time display position is outside the first operation space, in response to the first input, performing a first operation on the target object outside the first operation space.

5. The method according to claim 1, wherein The method further includes: Obtaining multiple historical interaction operations of the user; Determining, among the multiple historical interaction operations, the first number of times of performing a first operation on the target object within the first operation space and the second number of times of performing a first operation on the target object outside the first operation space; When the first number of times meets a preset condition, in response to the first input of the user to the target object, performing a first operation on the target object within the first operation space; When the second number of times meets the preset condition, in response to the first input, determining the real-time display position of the target object; When at least part of the real-time display position is outside the first operation space, in response to the first input, performing a first operation on the target object outside the first operation space.

6. The method according to any one of claims 3 to 5, characterized in that After performing, in response to the first input, a first operation on the target object outside the first operation space, the method further includes: When the eye movement information indicates that the eye movement position is within the second operation space and the distance between the projection of the target object and the second operation space is less than a preset distance threshold, performing a first operation on the target object within the second operation space.

7. A spatial interaction device, characterized in that, The device includes: A first acquisition module, configured to acquire the user's eye movement information; A first interaction module, configured to, when the eye movement information indicates that the eye movement position is within a first operation space, in response to a first input of the user to a target object, perform a first operation on the target object within the first operation space; wherein the target object is displayed within the first operation space.

8. The device according to claim 7, wherein The apparatus further includes: A first determination module, configured to, when the eye movement information indicates that the eye movement position is outside the first operation space and the eye movement speed is less than or equal to a preset eye movement threshold, in response to a first input of the user to a target object, determine a movement parameter of the target object in response to the first input, where the movement parameter includes at least one of a movement speed and a movement acceleration; A second determination module, configured to, when the movement parameter is greater than a movement threshold, in response to the first input, determine a real-time display position of the target object; A third interaction module, configured to, when at least a part of the real-time display position is outside the first operation space, in response to the first input, perform a first operation on the target object outside the first operation space.

9. The device according to claim 8, characterized in that, The apparatus further includes: A seventh interaction module, configured to, after performing a first operation on the target object outside the first operation space in response to the first input, when the eye movement information indicates that the eye movement position is within a second operation space and the distance between a projection of the target object and the second operation space is less than a preset distance threshold, perform a first operation on the target object within the second operation space.

10. An electronic device, characterized in that, It includes a processor and a memory, where the memory stores a program or instruction that can run on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1-6 are implemented.

Images