Object interaction method for three-dimensional space, and display device

Abstract

The present disclosure provides an object interaction method for a three-dimensional space, and a display device, used for constraining an interaction behavior of an interaction object, and effectively ameliorating the false triggering problem generated in a three-dimensional space interaction process. The method comprises: collecting interaction information of interaction between an object and a display content in an interaction three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction behavior generated between the object and the display content; extracting, from the interaction information, key point information of the object in different interaction stages, wherein the key point information is used for reflecting a behavior feature of the interaction behavior; and on the basis of the key point information, determining whether the interaction behavior of the object is valid.

Description

A three-dimensional space object interaction method and display device Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a method and display device for object interaction in a three-dimensional space. Background Art

[0002] Traditional displays are flat and typically use touch for interaction. VR / AR devices provide users with a full-field interactive 3D space, typically using gestures and joints for interaction. Glasses-free 3D displays provide users with a fixed depth of space, with a fixed position and size.

[0003] When a user performs gesture interaction with displayed content in an interactive three-dimensional space, false triggering is prone to occur because the displayed content presents many interactive elements at the same time.

[0004] Summary of the Invention

[0005] In a first aspect, an embodiment of the present disclosure provides a method for object interaction in a three-dimensional space, the method comprising:

[0006] Collecting interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction between the object and the display content;

[0007] Extracting key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavioral characteristics of the interaction behavior;

[0008] Determine whether the interactive behavior of the object is valid according to the key point information.

[0009] As an optional implementation manner, the interaction information includes at least two of the following:

[0010] A first parameter set of the object in an interaction preparation stage, wherein the first parameter set includes start hovering information of the object;

[0011] a second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and a changing behavior of the object during the interaction;

[0012] The third parameter set of the object at the end of the interaction includes the retraction trajectory information of the object after the collision ends.

[0013] As an optional implementation manner, key point information of the object at different interaction stages is extracted from the interaction information:

[0014] Extracting at least one of a start hovering position and a start hovering duration from the first parameter set;

[0015] Extracting at least one of the following from the second parameter set: a starting collision position, a depth change during the collision process, a position of a minimum depth during the collision process, a position of a maximum depth during the collision process, an ending collision position, an ending movement position, a collision duration, a movement direction, a movement speed, and a change in behavior of the object at the minimum depth; wherein the depth change indicates a change in the distance of the object from the display screen during the collision process;

[0016] An end hovering position and an interaction duration are extracted from the third parameter set, wherein the end hovering position is a position of the object in a retraction trajectory after the collision ends that is closest to the start hovering position.

[0017] As an optional implementation manner, determining whether the interactive behavior of the object is valid according to the key point information includes:

[0018] determining an interaction category of the interaction behavior of the object according to the interaction information;

[0019] Extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information;

[0020] Whether the interactive behavior of the object is valid is determined based on whether the extracted key point information meets the constraint conditions corresponding to the interaction category.

[0021] As an optional implementation manner, determining whether the interactive behavior of the object is valid based on whether the extracted key point information satisfies the constraint conditions corresponding to the interaction category includes:

[0022] Determining a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for positionally constraining the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for temporally constraining the interactive behavior from a temporal dimension;

[0023] Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter meet a constraint condition.

[0024] As an optional implementation, if the interaction category includes collision interaction, the extracted key point information includes:

[0025] Start hovering position and start hovering duration;

[0026] At least two of the following: the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration;

[0027] End hover position and interaction duration.

[0028] As an optional implementation, if the collision interaction is a single click or a long press, whether the interaction behavior of the object is valid is determined in the following manner:

[0029] determining a collision distance according to the starting collision position and the ending collision position;

[0030] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0031] If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all meet the single-click or long-press constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0032] As an optional implementation, if the collision interaction is a double-click, whether the interaction behavior of the object is valid is determined in the following manner:

[0033] determining a collision distance according to the starting collision position and the ending collision position;

[0034] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0035] Determine the double-click distance based on the minimum depth of movement during the collision process of different movement trajectories;

[0036] If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time, and the interaction time all meet the double-click constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0037] As an optional implementation, if the collision interaction is zooming, whether the interactive behavior of the object is valid is determined in the following manner:

[0038] Determining, based on an interactive behavior corresponding to a finger of the object, a first start hovering position, a first start collision position, a first farthest position moved during a collision process, a first depth change during the collision process, and a first start hovering duration;

[0039] Determining, based on an interaction behavior corresponding to another finger of the object, a second start hovering position, a second start collision position, a second farthest position moved during the collision process, a second depth change during the collision process, and a second start hovering duration;

[0040] determining a hovering distance according to the first start hovering position and the second start hovering position;

[0041] determining a collision distance according to the first collision start position and the second collision start position;

[0042] determining a distance between two fingers according to the first farthest position and the second farthest position;

[0043] If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, and the second start hovering duration all meet the scaling constraints, then it is determined that the interactive behavior of the object is valid.

[0044] As an optional implementation, if the interaction category includes sliding interaction, the extracted key point information includes:

[0045] Start hovering position, start hovering duration; movement direction, movement speed; end movement position and interaction duration.

[0046] As an optional implementation, whether the interactive behavior of the object is valid is determined in the following manner:

[0047] Determine a sliding distance according to the starting hovering position and the ending moving position;

[0048] If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraints of the sliding interaction, then it is determined that the interactive behavior of the object is valid.

[0049] As an optional implementation, if the interaction category includes grab-and-release interaction, the extracted key point information includes:

[0050] The duration of the hovering start; the changing behavior of the object at the minimum depth of movement during the collision process; the duration of the interaction.

[0051] As an optional implementation, whether the interactive behavior of the object is valid is determined in the following manner:

[0052] If the interaction duration, the start hovering duration, and the object's changing behavior all meet the constraints of the grab-and-release interaction, then the object's interactive behavior is determined to be valid.

[0053] In a second aspect, an embodiment of the present disclosure provides a display device, including a display screen and a controller:

[0054] The display screen is used to display content in the interactive three-dimensional space;

[0055] The controller is configured to perform:

[0056] Collecting interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction between the object and the display content;

[0057] Extracting key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavioral characteristics of the interaction behavior;

[0058] Determine whether the interactive behavior of the object is valid according to the key point information.

[0059] As an optional implementation manner, the interaction information includes at least two of the following:

[0060] A first parameter set of the object in an interaction preparation stage, wherein the first parameter set includes start hovering information of the object;

[0061] a second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and a changing behavior of the object during the interaction;

[0062] The third parameter set of the object at the end of the interaction includes the retraction trajectory information of the object after the collision ends.

[0063] As an optional implementation manner, the controller is specifically configured to execute:

[0064] Extracting at least one of a start hovering position and a start hovering duration from the first parameter set;

[0065] Extracting at least one of the following from the second parameter set: a starting collision position, a depth change during the collision process, a position of a minimum depth during the collision process, a position of a maximum depth during the collision process, an ending collision position, an ending movement position, a collision duration, a movement direction, a movement speed, and a change in behavior of the object at the minimum depth; wherein the depth change indicates a change in the distance of the object from the display screen during the collision process;

[0066] An end hovering position and an interaction duration are extracted from the third parameter set, wherein the end hovering position is a position of the object in a retraction trajectory after the collision ends that is closest to the start hovering position.

[0067] As an optional implementation manner, the controller is specifically configured to execute:

[0068] determining an interaction category of the interaction behavior of the object according to the interaction information;

[0069] Extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information;

[0070] Whether the interactive behavior of the object is valid is determined based on whether the extracted key point information meets the constraint conditions corresponding to the interaction category.

[0071] As an optional implementation manner, the controller is specifically configured to execute:

[0072] Determining a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for positionally constraining the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for temporally constraining the interactive behavior from a temporal dimension;

[0073] Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter meet a constraint condition.

[0074] As an optional implementation, if the interaction category includes collision interaction, the extracted key point information includes:

[0075] Start hovering position and start hovering duration;

[0076] At least two of the following: the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration;

[0077] End hover position and interaction duration.

[0078] As an optional implementation, if the collision interaction is a single click or a long press, the controller is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0079] determining a collision distance according to the starting collision position and the ending collision position;

[0080] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0081] If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all meet the single-click or long-press constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0082] As an optional implementation, if the collision interaction is a double-click, the controller is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0083] determining a collision distance according to the starting collision position and the ending collision position;

[0084] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0085] Determine the double-click distance based on the minimum depth of movement during the collision process of different movement trajectories;

[0086] If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time, and the interaction time all meet the double-click constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0087] As an optional implementation, if the collision interaction is zooming, the controller is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0088] Determining, based on an interactive behavior corresponding to a finger of the object, a first start hovering position, a first start collision position, a first farthest position moved during a collision process, a first depth change during the collision process, and a first start hovering duration;

[0089] Determining, based on an interaction behavior corresponding to another finger of the object, a second start hovering position, a second start collision position, a second farthest position moved during the collision process, a second depth change during the collision process, and a second start hovering duration;

[0090] determining a hovering distance according to the first start hovering position and the second start hovering position;

[0091] determining a collision distance according to the first collision start position and the second collision start position;

[0092] determining a distance between two fingers according to the first farthest position and the second farthest position;

[0093] If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, and the second start hovering duration all meet the scaling constraints, then it is determined that the interactive behavior of the object is valid.

[0094] As an optional implementation, if the interaction category includes sliding interaction, the extracted key point information includes:

[0095] Start hovering position, start hovering duration; movement direction, movement speed; end movement position and interaction duration.

[0096] As an optional implementation manner, the controller is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0097] Determine a sliding distance according to the starting hovering position and the ending moving position;

[0098] If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraints of the sliding interaction, then it is determined that the interactive behavior of the object is valid.

[0099] As an optional implementation, if the interaction category includes grab-and-release interaction, the extracted key point information includes:

[0100] The duration of the hovering start; the changing behavior of the object at the minimum depth of movement during the collision process; the duration of the interaction.

[0101] As an optional implementation manner, the controller is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0102] If the interaction duration, the start hovering duration, and the object's changing behavior all meet the constraints of the grab-and-release interaction, then the object's interactive behavior is determined to be valid.

[0103] In a third aspect, an embodiment of the present disclosure further provides an electronic device, including a processor and a memory, wherein the memory is configured to store a program executable by the processor, and the processor is configured to read the program in the memory and perform the following steps:

[0104] Collecting interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction between the object and the display content;

[0105] Extracting key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavioral characteristics of the interaction behavior;

[0106] Determine whether the interactive behavior of the object is valid according to the key point information.

[0107] In a fourth aspect, an embodiment of the present disclosure further provides a device for object interaction in a three-dimensional space, the device comprising:

[0108] An information collection unit, configured to collect interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of the interaction between the object and the display content;

[0109] an information extraction unit, configured to extract key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect behavioral characteristics of the interaction behavior;

[0110] The validity determination unit is configured to determine whether the interactive behavior of the object is valid according to the key point information.

[0111] In a fifth aspect, an embodiment of the present disclosure further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, it is used to implement the steps of the method described in the first aspect above.

[0112] These and other aspects of the present disclosure will become more readily apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0113] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0114] FIG1 is a schematic diagram of an interactive scenario with gestures as objects provided by an embodiment of the present disclosure;

[0115] FIG2A-FIG2B are schematic diagrams of an error interaction feedback provided by an embodiment of the present disclosure;

[0116] FIG3 is a flowchart showing a specific implementation of a method for object interaction in a three-dimensional space according to an embodiment of the present disclosure;

[0117] FIG4 is a schematic diagram of an interaction in which an object is a gesture, provided by an embodiment of the present disclosure;

[0118] FIG5 is a schematic diagram of an interaction trajectory of a single-click gesture provided by an embodiment of the present disclosure;

[0119] FIG6 is a schematic diagram of an interaction trajectory of a long press gesture provided by an embodiment of the present disclosure;

[0120] FIG7 is a schematic diagram of an interaction trajectory of a double-click gesture provided by an embodiment of the present disclosure;

[0121] FIG8 is a schematic diagram of an interaction trajectory of a zoom gesture provided by an embodiment of the present disclosure;

[0122] FIG9 is a schematic diagram of an interaction trajectory of a sliding gesture provided by an embodiment of the present disclosure;

[0123] FIG10 is a schematic diagram of an interaction trajectory of a grab and release gesture provided by an embodiment of the present disclosure;

[0124] FIG11 is a schematic diagram of a naked-eye 3D display device provided by an embodiment of the present disclosure;

[0125] FIG12 is a schematic diagram of a display device provided by an embodiment of the present disclosure;

[0126] FIG13 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure;

[0127] FIG14 is a schematic diagram of a three-dimensional object interaction device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0128] To make the objectives, technical solutions, and advantages of the present disclosure more clear, the present disclosure will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only a portion of the embodiments of the present disclosure, rather than all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present disclosure without creative effort are intended to fall within the scope of protection of the present disclosure.

[0129] In the embodiments of the present disclosure, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0130] The application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Persons skilled in the art will appreciate that, as new application scenarios emerge, the technical solutions provided by the embodiments of the present disclosure will also be applicable to similar technical problems. In the description of the present disclosure, unless otherwise specified, "multiple" means two or more.

[0131] Currently, implementing interactive functions based on gesture recognition technology is a major research hotspot, with applications in fields such as VR / AR and naked-eye 3D displays. Its core is to collect user gesture information through sensor devices such as cameras, identify gestures through relevant recognition and classification algorithms, and assign different semantic information to different gestures to achieve different interactive functions. Traditional displays are flat displays and generally use touch interaction. Devices such as VR / AR provide users with an interactive three-dimensional space with a full field of view, and generally use gesture joints for interaction. Unlike the above two, naked-eye 3D displays provide users with an interactive three-dimensional space of a certain depth. The position and size of this space are fixed. When users use their bare hands for gesture interaction, they tend to interact through simple gestures such as clicking, grasping, and sliding, which is more in line with the user's daily operating habits.

[0132] It's important to note that enabling interaction between gestures and 3D content requires the coordinated implementation of both. At the content level, common, easy-to-understand controls must be provided. These controls have pre-set behaviors or functions upon triggering. Interactive feedback, such as texture, color, brightness, and sound, is provided before, during, and after the controls are triggered to alert the user to the current interaction. Furthermore, and crucially, the controls themselves contain positional information. Only when the gesture's location coincides with the control's location is the two considered to have collided, triggering the control's interaction. Regarding content, the project primarily develops 2D multi-layer interface interaction applications and applications that interact with 3D models. At the gesture level, different interaction effects can be achieved by analyzing the user's gesture location, type, motion, and speed, combined with information such as the content's location. Common gestures include single-tap, long-press, double-tap, zoom, swipe, grab, and release. These gestures can be distinguished by different parameters, such as gesture category, collision duration, collision count, and number of collision points.

[0133] First, a brief description of the interactive scene based on the interactive stereoscopic space involved in this embodiment is provided. As shown in FIG1 , this embodiment provides a schematic diagram of an interactive scene with gestures as the object. Taking a desktop-level 32-inch glasses-free 3D display device as an example, the interaction effect between bare hands and displayed content is debugged. The screen size of the glasses-free 3D display device is w×h (width×height) = 70cm×40cm. Considering that the arm length of an adult is 65cm, and the user is viewing from 65cm in front of the screen, gesture interaction requires the gesture to be 30cm away from the eye to avoid blocking the line of sight. Therefore, the size of the interactive stereoscopic space can be set to: 70cm×40cm×35cm. In implementation, multiple cameras can be integrated into the screen frame to capture the position and movement of gestures in the screen's interactive stereoscopic space to achieve bare-hand interaction. By adjusting the camera shooting angle, the field of view of the multiple cameras can fully cover the interactive stereoscopic space, allowing the user to freely interact with the 3D content using their bare hands.

[0134] For example, when users use gestures to interact with a 2D multi-layer interface or 3D model components in an interactive 3D space, the content will simultaneously present a large number of interactive elements, distributed at different positions or depths in the interactive 3D space. The following are possible interaction issues:

[0135] Problem 1) If the user wants to interact with the content elements inside, the gesture will collide with the outer layer during the process of extending, thus triggering the wrong interaction effect.

[0136] When a user actually interacts, they typically extend their hand into the interactive 3D space from the front. However, for interactions with 2D multi-layer interfaces or 3D model components, multiple interactive elements are presented simultaneously in terms of content at the same time. These elements are distributed at different positions or depths. Therefore, during interaction, the user might want to interact with an element within the space, but during the gesture's movement, the user might collide with an element on the outer layer, triggering an incorrect interaction. As shown in Figure 2A, this embodiment provides a schematic diagram of incorrect interaction feedback. For example, a user wants to interact with content element 2, but their gesture collides with content element 4 during movement. This triggers incorrect interaction feedback for content element 4.

[0137] Problem 2) When gestures are randomly or swung back and forth in the interactive 3D space, they may collide with the content or trigger a sliding gesture, thereby triggering incorrect interaction effects.

[0138] During actual interaction, a gesture may randomly swing in the interactive 3D space, without the user intending to interact with a specific content element. This may result in a collision between the gesture and the content, triggering an incorrect interaction. Alternatively, a gesture randomly swinging in the space may trigger a sliding gesture, thereby triggering an interactive effect that switches content. This is also incorrect. As shown in Figure 2B, this embodiment provides a schematic diagram of incorrect interaction feedback. In the process of randomly swinging or swinging back and forth in the interactive 3D space, a gesture collides with content element 4, triggering interactive feedback for content element 4. This is incorrect and does not meet the user's interaction expectations.

[0139] In order to solve the problem of users accidentally touching during gesture interaction, this embodiment provides a three-dimensional object interaction method. By extracting key point information from the behavioral trajectory of the object's interactive behavior, it is determined whether the interactive behavior is valid, and the interaction process is analyzed and refined. It can determine whether the interactive behavior is valid based on the different characteristics of the interactive behavior generated at different interaction stages. Taking the object as an example, different gesture interaction processes are analyzed and refined. By setting corresponding parameters for different gesture interaction processes to impose conditional constraints, it is possible to prevent false triggering during the gesture interaction process and ensure the accuracy of gesture interaction. The object interaction method provided in this embodiment has the idea of ​​determining valid interaction as follows: constraining parameters such as the interaction position and interaction time corresponding to different interaction stages, thereby determining valid interaction and preventing the phenomenon of accidentally touching gestures during the interaction process.

[0140] As shown in FIG3 , the specific implementation process of a three-dimensional space object interaction method provided by this embodiment is as follows:

[0141] Step 300: Collect interaction information between an object and display content in an interactive 3D space, wherein the interaction information represents a behavior trajectory of the interaction between the object and the display content;

[0142] Optionally, the interactive stereoscopic space in this embodiment is determined based on the stereoscopic space that the display device can display. During implementation, the position and size of the interactive stereoscopic space can also be set according to the actual application scenario. For example, when the actual operation scene is the stereoscopic space of a naked-eye 3D display screen, a smaller interactive stereoscopic space can be set based on the fact that users can achieve interaction within a smaller range through gestures. When the actual operation scene is an AR / VR interaction scene, a larger interactive stereoscopic space can be set to facilitate all-round interaction.

[0143] Optionally, the objects in this embodiment include, but are not limited to, fingers, palms, hands, joints, arms, legs, feet, and other manipulable body parts. The objects in this embodiment include, but are not limited to, humans, robots, animals, and other manipulable objects. This embodiment does not impose any further limitations on this.

[0144] Optionally, the display content in this embodiment is 3D content or multi-layer 2D content. It should be noted that when the display content is a 2D plane, the 2D plane actually has a certain thickness when displayed, rather than a completely two-dimensional plane. It can restore the real scene as much as possible and provide users with a better interactive experience.

[0145] In some embodiments, the interaction information in this embodiment represents a behavior trajectory corresponding to the interaction behavior generated by the object and the displayed content during the interaction, including but not limited to at least two of the following:

[0146] Item 1: a first parameter set of the object in the interaction preparation stage, wherein the first parameter set includes the start hovering information of the object.

[0147] Optionally, the start hovering information in this embodiment includes but is not limited to at least one of a start hovering position, a start hovering time, and a start hovering duration.

[0148] Optionally, when the starting hovering position is located in the interactive three-dimensional space and the starting hovering duration is less than a preset value, the interactive behavior of the object in the interaction preparation stage is valid.

[0149] In implementation, hovering refers to the state when an object enters the interactive three-dimensional space before interaction but does not touch the displayed content. Usually, the hovering position should be in the interactive three-dimensional space, and the hovering duration should be consistent with the user's interaction habits.

[0150] Item 2. A second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and changing behavior of the object during the interaction process.

[0151] In implementation, collision refers to a state generated when an object contacts the display content while operating the display content.

[0152] Optionally, in this embodiment, the collision information between the object and the displayed content refers to parameter information during the collision between the object and the displayed content, including but not limited to any one or more of the following:

[0153] The starting collision position; the depth change during the collision process; the position of the minimum depth of movement during the collision process; the farthest position of movement during the collision process; the ending collision position; the ending movement position; the collision duration; the movement direction; the movement speed; the changing behavior of the object at the minimum depth.

[0154] The depth change indicates the change in the distance from the object to the display screen during the collision process. The smaller the depth, the smaller the distance, and the greater the depth, the greater the distance.

[0155] Item 3: a third parameter set of the object at the end of the interaction, wherein the third parameter set includes the retraction trajectory information of the object after the collision ends.

[0156] Optionally, the withdrawal trajectory information in this embodiment includes but is not limited to at least one of end hovering information and interaction duration.

[0157] The end hovering information includes but is not limited to at least one of an end hovering position and an end hovering time.

[0158] Step 301: extract key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavioral characteristics of the interaction behavior;

[0159] Optionally, the time and location at which the interactive behavior is generated in different interaction stages are different.

[0160] In some embodiments, this embodiment extracts key point information at different interaction stages from the behavioral trajectory of the object's interactive behavior, including key point information in a first parameter set at the interaction preparation stage, a second parameter set at the interaction occurrence stage, and a third parameter set at the interaction end stage. The extracted key point information is not limited to any one or more of the following:

[0161] Extracting at least one of a start hovering position, a start hovering time, and a start hovering duration from the first parameter set;

[0162] Extracting at least one of the following from the second parameter set: a starting collision position, a depth change during the collision process, a position of a minimum depth during the collision process, a position of a maximum depth during the collision process, an ending collision position, an ending movement position, a collision duration, a movement direction, a movement speed, and a change in behavior of the object at the minimum depth; wherein the depth change indicates a change in the distance of the object from the display screen during the collision process;

[0163] Extracting an end hovering position and an interaction duration from the third parameter set, wherein the end hovering position is the position of the object in the retraction trajectory after the collision that is closest to the start hovering position. The interaction duration represents the duration of a complete interaction process between the object and the displayed content.

[0164] It should be noted that, taking the hand as an example, the behavioral characteristics of the interactive behavior in the gesture interaction process are analyzed and refined as follows:

[0165] (1) The first parameter set in the interaction preparation phase;

[0166] In practice, the interaction preparation phase typically involves hovering. Two parameters are extracted during this phase: the start hover position, the start hover time, and the start hover duration. The start hover position and start hover time are used to prepare for subsequent judgments, while the start hover duration is used to determine whether the user is preparing to interact with the displayed content or simply waving around in the interactive space. Generally, the start hover position is required to be within the interactive 3D space, and the start hover duration must conform to the user's interaction habits.

[0167] (2) the second parameter set during the interaction occurrence phase;

[0168] In practice, key point information collected during the interaction phase includes any one or more of the following:

[0169] 2a. When the gesture collides with the displayed content, the starting collision position is extracted to prepare for the judgment in the subsequent steps.

[0170] 2b. When the gesture collides with the displayed content, at a certain moment, the collision point (interaction point) between the gesture and the displayed content reaches the minimum depth. At this time, the position of the minimum depth of the gesture movement during the collision process is extracted to prepare for the judgment in the subsequent steps.

[0171] 2c. When the gesture continuously collides with the displayed content, extract the collision duration. This parameter can determine whether the user performed a single-click gesture or a long-press gesture. The collision duration threshold must conform to user interaction habits.

[0172] 2d. If another collision is detected while the gesture and the displayed content are still in continuous contact, the starting collision position of the second collision is extracted. If the distance between the starting collision positions of the two collisions is large, the interaction is considered valid and a zoom gesture can be considered. The zoom factor is proportional to the change in the distance between the two starting collision positions.

[0173] 2e. When the interaction behavior is a single collision, when the gesture and the displayed content end the collision, the end collision position is extracted. This parameter can be compared with the start collision position when the collision occurs. When the distance between the two positions is within a certain range, the interaction is considered valid. Generally, when the distance between the start collision position and the end collision position is small, it is considered a single-click or long-press gesture. For a double-click gesture, there is at most one collision point at a certain moment, and there are two collisions within a period of time, that is, two start collision positions when the collision occurs, and two end collision positions when the collision ends. When the distance between any two of these four positions is within a certain range, the double-click interaction is considered valid. In addition, there are two positions where the depth of the gesture interaction point reaches the minimum depth. When the distance between these two positions is within a certain range, the double-click interaction is considered valid.

[0174] 2f. During the gesture interaction process, the movement speed and direction of the gesture are extracted, which can be used in conjunction with the gesture hovering position and the start hovering time. When the gesture hovers at a certain position and moves in a certain direction with a speed greater than a certain threshold, it can be considered as a sliding gesture. Generally, the start hovering position is required to be within the interaction space and does not collide with any displayed content. The start hovering duration and movement speed meet certain requirements. In addition, the movement direction is left-right or up-down. It can be considered as a left-right sliding gesture or an up-down sliding gesture. Left-right sliding can be designed to switch display content or scenes, and up-down sliding can be designed to use gestures to call out menu options and other functions.

[0175] (3) the third parameter set at the end of the interaction;

[0176] During implementation, when the gesture is withdrawn after the collision with the displayed content ends, the motion trajectory of the gesture is tracked. When the distance between the last hovering position in the motion trajectory (i.e., the end hovering position) and the start hovering position is within a certain range, the interaction is considered valid.

[0177] During implementation, the interaction duration can also be extracted, i.e., the duration of a complete gesture interaction. This timing starts at the first hovering point and ends when the final interaction gesture is determined to be complete. Generally, a gesture interaction is considered valid if the interaction duration is within a certain timeframe. However, zoom gestures are not subject to this restriction; the zooming process can take longer. After zooming the content, if the hovering time at any interaction point exceeds a threshold, the zoom gesture interaction ends, the content is zoomed, and the two interaction points stop colliding.

[0178] Step 302: Determine whether the interactive behavior of the object is valid based on the key point information.

[0179] Optionally, the effectiveness of the interaction behavior is determined by extracting key point information related to the behavioral characteristics of the interaction behavior. Specifically, the effectiveness is determined through the following steps:

[0180] (1) determining the interaction category of the interaction behavior of the object according to the interaction information;

[0181] (2) extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information;

[0182] (3) Determine whether the object's interactive behavior is valid based on whether the extracted key point information satisfies the constraint conditions corresponding to the interaction category.

[0183] In some embodiments, whether the interactive behavior of the object is valid may also be determined by:

[0184] Determining a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for positionally constraining the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for temporally constraining the interactive behavior from a temporal dimension;

[0185] Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter meet a constraint condition.

[0186] Optionally, the first behavior parameter includes but is not limited to: collision distance, hovering distance, depth change during the collision process, double-click distance, two-finger distance, sliding distance, etc.

[0187] Among them, when the interactive behavior includes single-click, long press, and double-click, the collision distance is the distance between the starting collision position and the ending collision position; the hovering distance is the distance between the starting hovering position and the ending hovering position; the double-click distance is the distance between the positions of the minimum depth moved in the collision process of different movement trajectories.

[0188] When the interactive behavior includes zooming, the hovering distance is the distance between the first starting hovering position and the second starting hovering position, the collision distance is the distance between the first starting collision position and the second starting collision position, and the two-finger distance is the distance between the first farthest position and the second farthest position. According to the interactive behavior corresponding to one finger of the object, the first starting hovering position, the first starting collision position and the first farthest position moved during the collision process are determined, and according to the interactive behavior corresponding to another finger of the object, the second starting hovering position, the second starting collision position and the second farthest position moved during the collision process are determined.

[0189] When the interaction includes sliding, the sliding distance is the distance between the start hover position and the end hover position.

[0190] The second behavior parameter includes but is not limited to: interaction duration, start hovering duration, collision duration, movement speed, movement direction, change behavior of the object at the minimum depth of movement during the collision process, etc.

[0191] During implementation, the following seven commonly used gestures are used as examples to illustrate the process of determining a gesture as a valid interaction, so as to prevent accidental touches during gesture interaction, thereby generating erroneous feedback. First, the algorithm divides gesture categories into single-finger gestures, five-finger natural stretch gestures, and five-finger natural bend gestures, and relies on these three gestures to extract the coordinates of the interaction points. As shown in Figure 4, this embodiment provides an interaction diagram in which the object is a gesture, wherein the single-finger gesture extracts the fingertip area as the interaction point, the five-finger natural stretch gesture extracts the fingertip area and the palm area as the interaction point, and the five-finger natural bend gesture extracts the gesture center area as the interaction point.

[0192] In some embodiments, if the interaction category includes collision interaction, the extracted key point information includes:

[0193] Start hovering position, start hovering time, start hovering duration;

[0194] At least two of the following: the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration;

[0195] End the hovering position and end the hovering time.

[0196] In some embodiments, if the collision interaction is a single click or a long press, whether the interaction behavior of the object is valid is determined as follows:

[0197] determining a collision distance according to the starting collision position and the ending collision position;

[0198] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0199] Determine the interaction duration based on the start and end hover times;

[0200] If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all meet the single-click or long-press constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0201] In practice, taking a single-click gesture as an example, as shown in FIG5 , this embodiment provides a schematic diagram of an interaction trajectory of a single-click gesture, and the interaction process is as follows:

[0202] The gesture hovers at position 1, the coordinate of the starting hovering position is p1, and the starting hovering duration is t1;

[0203] A collision occurs at position 2, and the coordinates of the collision start position are p2; the coordinates of position 3, the minimum depth of movement during the collision process, are p3. The gesture starts to turn back from position 3 and ends at position 4, with the coordinates of the collision end position being p4. The collision duration from position 2 to position 4 is t2; during the gesture retraction process, the trajectory point closest to position 1 is position 5, that is, the coordinates of the hovering end position are p5.

[0204] In practice, the conditions for preventing accidental touch gestures are as follows:

[0205] In terms of distance, the depth from p1 to p3 is required to be reduced, the depth from p3 to p5 is restored, the straight-line distance from p1 to p5 is less than 1.8 cm, and the straight-line distance from p2 to p4 is less than 1 cm;

[0206] In terms of time, the hovering time t1 is required to be greater than 0.3s and the collision time t2 is required to be less than 0.8s.

[0207] When all the above conditions are met and the interaction duration for a complete interaction is less than 2 seconds, the single-click gesture is considered valid.

[0208] In implementation, taking a long press gesture as an example, as shown in FIG6 , this embodiment provides a schematic diagram of an interaction trajectory of a long press gesture, and the interaction process is as follows:

[0209] The gesture hovers at position 1, the coordinate of the starting hovering position is p1, and the starting hovering duration is t1;

[0210] A collision occurs at position 2, and the coordinates of the collision start position are p2; the coordinates of position 3, the minimum depth of movement during the collision process, are p3. The gesture starts to turn back from position 3 and ends at position 4, with the coordinates of the collision end position being p4. The collision duration from position 2 to position 4 is t2; during the gesture retraction process, the trajectory point closest to position 1 is position 5, that is, the coordinates of the hovering end position are p5.

[0211] In practice, the conditions for preventing accidental touch gestures are as follows:

[0212] In terms of distance, the depth from p1 to p3 is required to be reduced, the depth from p3 to p5 is restored, the straight-line distance from p1 to p5 is less than 1.8 cm, and the straight-line distance from p2 to p4 is less than 1 cm;

[0213] In terms of time, the hovering time t1 is required to be greater than 0.3s and the collision time t2 is required to be greater than 0.8s.

[0214] When all the above conditions are met, and the interaction duration for a complete interaction is less than 2 seconds, the long press gesture is considered valid. In practice, the entire process is almost identical to the single-click gesture, except that the collision duration t2 is longer.

[0215] In some embodiments, if the collision interaction is a double-click, whether the interaction behavior of the object is valid is determined as follows:

[0216] determining a collision distance according to the starting collision position and the ending collision position;

[0217] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0218] Determine the double-click distance based on the minimum depth of movement during the collision process of different movement trajectories;

[0219] If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time, and the interaction time all meet the double-click constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0220] In practice, taking the double-click gesture as an example, as shown in FIG7 , this embodiment provides a schematic diagram of the interaction trajectory of the double-click gesture. The interaction process is as follows:

[0221] The gesture hovers at position 1. The coordinates of the first hovering position are p1 and the hovering duration is t1.

[0222] A collision occurs at position 2, and the coordinates of the first collision start position are p2; the coordinates of position 3, the minimum depth of movement during the first collision, are p3. The gesture starts to turn back from position 3 and ends the collision at position 4. The coordinates of the first collision end position are p4, and the collision duration from position 2 to position 4 is t2;

[0223] During the gesture's retraction process, the gesture trajectory turns back again at position 5, and the coordinates of the hovering position are p5. The gesture produces a second collision at position 6, and the coordinates of the collision starting position are p6. The coordinates of position 7, the minimum depth of movement during the first collision, are p7. The gesture turns back again from position 7 and ends the collision at position 8. The coordinates of the collision ending position are p8. The collision duration from position 6 to position 8 is t3.

[0224] During the gesture retraction process, the point closest to position 1 is position 9, and the coordinate of the end hovering position is p9.

[0225] The conditions for preventing accidental touch gestures are:

[0226] In terms of distance, the depth from p1 to p3 is required to be reduced, the depth from p3 to p5 is restored, the depth from p5 to p7 is required to be reduced, and the depth from p7 to p9 is restored. The straight-line distance between any two points among p1, p5, and p9 is required to be less than 1.8 cm, the straight-line distance between any two points among p2, p4, p6, and p8 is required to be less than 1 cm, and the straight-line distance between p3 and p7 is required to be less than 1 cm.

[0227] In terms of timing, the hovering duration t1 must be greater than 0.3s, and the collision durations t2 and t3 must be less than 0.8s. If all of these conditions are met and the total duration of a complete interaction is less than 2s, the double-click gesture is considered valid.

[0228] In some embodiments, if the collision interaction is zooming, whether the interactive behavior of the object is valid is determined as follows:

[0229] Determining, based on an interactive behavior corresponding to a finger of the object, a first starting hovering position, a first starting collision position, a first farthest position moved during a collision process, a first depth change during the collision process, a first starting hovering duration, and a first farthest hovering duration after moving to the first farthest position;

[0230] Determining, based on an interaction behavior corresponding to another finger of the object, a second starting hovering position, a second starting collision position, a second farthest position moved during the collision process, a second depth change during the collision process, a second starting hovering duration, and a second farthest hovering duration after moving to the second farthest position;

[0231] determining a hovering distance according to the first start hovering position and the second start hovering position;

[0232] determining a collision distance according to the first collision start position and the second collision start position;

[0233] determining a distance between two fingers according to the first farthest position and the second farthest position;

[0234] If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, the second start hovering duration, the first farthest hovering duration and the second farthest hovering duration all meet the scaling constraints, then it is determined that the interactive behavior of the object is valid.

[0235] In practice, taking the zoom gesture as an example, as shown in FIG8 , this embodiment provides a schematic diagram of the interaction trajectory of the zoom gesture, and the interaction process is as follows:

[0236] The gesture hovers at position 1. The coordinates of the first hovering position are p1 and the hovering duration is t1.

[0237] A collision occurs at position 2, and the coordinates of the first start collision position are p2; the coordinates of position 3, the minimum depth of movement during the first collision, are p3; during the period of continuous collision between the gesture and the content, a second collision between the gesture and the content is detected, and the coordinates of the second start collision position are p5, and the coordinates of position 6, the minimum depth of movement during the second collision, are p6; at this time, the scaling factor of the content is proportional to the distance from p3 to p6. If the coordinates of the gesture moving from position 3 to the first farthest position 7 during the first collision are p7, and the coordinates of the gesture moving from position 6 to the second farthest position 10 during the second collision are p10, then the change in distance is ||(p7–p10)–(p3–p6)||, and the content is scaled based on this; after the scaling is completed, the two gestures hover at positions 7 and 10 for a duration of t2; the two gestures end the collision and leave the content area in turn.

[0238] The conditions for preventing accidental touch gestures are:

[0239] In terms of distance, the depths from p1 to p3 and p4 to p6 are required to be reduced, the depths from p7 to p9 and p10 to p12 are required to be restored, and the straight-line distances from p1 to p4, p2 to p5, and p3 to p6 are required to be greater than 2 cm.

[0240] In terms of time, t1 and t2 must be greater than 0.3s and 0.3s, respectively. If both conditions are met, the zoom gesture is considered valid.

[0241] In some embodiments, if the interaction category includes sliding interaction, the extracted key point information includes:

[0242] Start hovering position and start hovering duration;

[0243] Movement direction and speed;

[0244] End move location and interaction duration.

[0245] In some embodiments, if the interaction category includes a sliding interaction, whether the interaction behavior of the object is valid is determined as follows:

[0246] Determine a sliding distance according to the starting hovering position and the ending moving position;

[0247] If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraints of the sliding interaction, then it is determined that the interactive behavior of the object is valid.

[0248] In practice, taking a sliding gesture as an example, as shown in FIG9 , this embodiment provides a schematic diagram of an interaction trajectory of a sliding gesture, and the interaction process is as follows:

[0249] The gesture hovers at position 1, the coordinate of the starting hovering position is p1, and the starting hovering duration is t1;

[0250] The gesture moves quickly from position 1 to position 2, the coordinate of the end movement position is p2, and the movement time is t2; from the main view direction, the horizontal movement distance of the gesture position is x, and the vertical movement distance is y.

[0251] The conditions for preventing accidental touch gestures are:

[0252] In terms of position, the distance between p1 and p2 is required to be greater than 35cm;

[0253] In terms of timing, the hovering duration t1 must be greater than 0.3s and the movement duration must be less than 0.14s, meaning the gesture's speed must be greater than 2.5m / s. If all of these conditions are met and the total duration of a complete interaction is less than 2s, the swipe gesture is considered valid. Whether it's an up-down or left-right swipe is determined based on x and y. If the x:y ratio is greater than 2, it's considered a left-right swipe; if the x:y ratio is less than 0.5, it's considered an up-down swipe.

[0254] In some embodiments, if the interaction category includes a grab-and-release interaction, the extracted key point information includes:

[0255] The duration of the hovering start; the changing behavior of the object at the minimum depth of movement during the collision process; the duration of the interaction.

[0256] In some embodiments, if the interaction category includes a grab-and-release interaction, whether the interaction behavior of the object is valid is determined as follows:

[0257] If the interaction duration, the start hovering duration, and the object's changing behavior all meet the constraints of the grab-and-release interaction, then the object's interactive behavior is determined to be valid.

[0258] In practice, as shown in FIG10 , this embodiment provides a schematic diagram of an interaction trajectory of a grab and release gesture. Taking the grab gesture as an example, the interaction process is as follows:

[0259] The gesture is a five-finger natural stretch gesture, hovering at position 1, the coordinates of the starting hovering position are p1, and the starting hovering duration is t1;

[0260] The gesture moves forward and a collision occurs at position 2. The coordinate of position 3, the minimum depth of movement during the collision, is p3.

[0261] The conditions for preventing accidental touch gestures are:

[0262] The initial hovering duration t1 is greater than 0.3 seconds, and at position 3, the gesture changes from naturally extending five fingers to naturally bending five fingers. If all of the above conditions are met and the total duration of a complete interaction is less than 2 seconds, the grab gesture is considered valid.

[0263] The interaction process using the release gesture as an example is as follows:

[0264] After the model element is captured, the gesture is bound to the corresponding content and hovered at position 1. The coordinates of the starting hover position are p1 and the starting hover duration is t1.

[0265] The content follows the gesture, moving from position 3 to the target position 4. When the gesture changes from naturally bending the five fingers to naturally extending the five fingers at position 4, the gesture is unbound from the corresponding content, and the content no longer follows the gesture. The gesture moves from position 4 to position 6, the gesture depth is restored, and the collision ends at position 5, completing the release operation.

[0266] The conditions for preventing accidental touch gestures are:

[0267] The initial hovering duration t1 is greater than 0.3 seconds, and at position 4, the gesture changes from naturally bending the five fingers to naturally extending the five fingers. If all of the above conditions are met and the total duration of a complete interaction is less than 2 seconds, the grab gesture is considered valid.

[0268] The three-dimensional object interaction method provided in this embodiment analyzes and refines the gesture interaction process. By constraining relevant parameters such as hover time, collision duration, collision position, number of collisions, collision distance, gesture movement speed, and movement direction, it prevents false triggering during the gesture interaction process and ensures the accuracy of gesture interaction.

[0269] Example 2: Based on the same inventive concept, the embodiment of the present disclosure also provides a display device. Since the display device is the display device in the method in the embodiment of the present disclosure, and the principle of solving the problem by the display device is similar to that of the method, the implementation of the display device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0270] It should be noted that gesture interaction, as the most natural way of interaction for humans, has irreplaceable advantages over other ways of interaction. The display device provided in this embodiment includes but is not limited to a naked-eye 3D display device. Multiple cameras can be integrated at the screen frame to capture the position and movement of gestures in the interactive three-dimensional space of the screen, thereby enabling bare hands to interact with 3D scenes and content. As shown in Figure 11, a schematic diagram of a naked-eye 3D display device provided by this embodiment, in which multiple cameras are integrated at the screen frame, four cameras are integrated in the figure, respectively located at the four vertices of the screen frame, for collecting the gestures of the object in the interactive three-dimensional space. The display device in this embodiment does not require the user to carry a handle or wear data gloves to achieve bare-hand or bare-finger interaction, which has high user acceptance and also enhances the value of the screen.

[0271] Optionally, multiple cameras can be integrated into the screen frame to capture the position and movement of gestures in the interactive 3D space of the screen, enabling bare-hand interaction. For example, five ToF cameras can be integrated, with one ToF camera set at each of the four top corners of the screen and one ToF camera set at the center below the screen. By adjusting the camera shooting angle, the field of view of the five cameras can completely cover the interactive 3D space, allowing users to freely interact with 3D content using their bare hands. Among them, ToF (Time of Flight) is a distance measurement method that calculates the distance between the two by measuring the "flight time" of ultrasonic / microwave / light signals between the emitter and the reflector. The ToF camera in this embodiment refers to a ToF sensor that emits infrared light or laser. The light generated will bounce off any object and return to the sensor. Based on the time difference between the light emission and the light reflection from the object and returning to the sensor, the sensor can measure the distance between the object and the sensor, thereby capturing the position of the user's gesture in the interactive 3D space. The camera captures the gesture image to recognize the gesture action, and ultimately obtains the position and action of the user's gesture during interaction, enabling bare-hand interaction with 3D content.

[0272] As shown in FIG12 , the display device includes a display screen 1200 and a controller 1201 :

[0273] The display screen 1200 is used to display content in an interactive three-dimensional space;

[0274] The controller 1201 is configured to perform the following steps:

[0275] Collecting interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction between the object and the display content;

[0276] Extracting key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavioral characteristics of the interaction behavior;

[0277] Determine whether the interactive behavior of the object is valid according to the key point information.

[0278] As an optional implementation manner, the interaction information includes at least two of the following:

[0279] A first parameter set of the object in an interaction preparation stage, wherein the first parameter set includes start hovering information of the object;

[0280] a second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and a changing behavior of the object during the interaction;

[0281] The third parameter set of the object at the end of the interaction includes the retraction trajectory information of the object after the collision ends.

[0282] As an optional implementation manner, the controller 1201 is specifically configured to execute:

[0283] Extracting at least one of a start hovering position and a start hovering duration from the first parameter set;

[0284] Extracting at least one of the following from the second parameter set: a starting collision position, a depth change during the collision process, a position of a minimum depth during the collision process, a position of a maximum depth during the collision process, an ending collision position, an ending movement position, a collision duration, a movement direction, a movement speed, and a change in behavior of the object at the minimum depth; wherein the depth change indicates a change in the distance of the object from the display screen during the collision process;

[0285] An end hovering position and an interaction duration are extracted from the third parameter set, wherein the end hovering position is a position of the object in a retraction trajectory after the collision ends that is closest to the start hovering position.

[0286] As an optional implementation manner, the controller 1201 is specifically configured to execute:

[0287] determining an interaction category of the interaction behavior of the object according to the interaction information;

[0288] Extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information;

[0289] Whether the interactive behavior of the object is valid is determined based on whether the extracted key point information meets the constraint conditions corresponding to the interaction category.

[0290] As an optional implementation manner, the controller 1201 is specifically configured to execute:

[0291] Determining a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for positionally constraining the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for temporally constraining the interactive behavior from a temporal dimension;

[0292] Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter meet a constraint condition.

[0293] As an optional implementation, if the interaction category includes collision interaction, the extracted key point information includes:

[0294] Start hovering position and start hovering duration;

[0295] At least two of the following: the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration;

[0296] End hover position and interaction duration.

[0297] As an optional implementation, if the collision interaction is a single click or a long press, the controller 1201 is specifically configured to determine whether the interaction behavior of the object is valid in the following manner:

[0298] determining a collision distance according to the starting collision position and the ending collision position;

[0299] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0300] If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all meet the single-click or long-press constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0301] As an optional implementation, if the collision interaction is a double-click, the controller 1201 is specifically configured to determine whether the interaction behavior of the object is valid in the following manner:

[0302] determining a collision distance according to the starting collision position and the ending collision position;

[0303] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0304] Determine the double-click distance based on the minimum depth of movement during the collision process of different movement trajectories;

[0305] If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time, and the interaction time all meet the double-click constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0306] As an optional implementation, if the collision interaction is zooming, the controller 1201 is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0307] Determining, based on an interactive behavior corresponding to a finger of the object, a first start hovering position, a first start collision position, a first farthest position moved during a collision process, a first depth change during the collision process, and a first start hovering duration;

[0308] Determining, based on an interaction behavior corresponding to another finger of the object, a second start hovering position, a second start collision position, a second farthest position moved during the collision process, a second depth change during the collision process, and a second start hovering duration;

[0309] determining a hovering distance according to the first start hovering position and the second start hovering position;

[0310] determining a collision distance according to the first collision start position and the second collision start position;

[0311] determining a distance between two fingers according to the first farthest position and the second farthest position;

[0312] If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, and the second start hovering duration all meet the scaling constraints, then it is determined that the interactive behavior of the object is valid.

[0313] As an optional implementation, if the interaction category includes sliding interaction, the extracted key point information includes:

[0314] Start hovering position, start hovering duration; movement direction, movement speed; end movement position and interaction duration.

[0315] As an optional implementation manner, the controller 1201 is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0316] Determine a sliding distance according to the starting hovering position and the ending moving position;

[0317] If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraints of the sliding interaction, then it is determined that the interactive behavior of the object is valid.

[0318] As an optional implementation, if the interaction category includes grab-and-release interaction, the extracted key point information includes:

[0319] The duration of the hovering start; the changing behavior of the object at the minimum depth of movement during the collision process; the duration of the interaction.

[0320] As an optional implementation manner, the controller 1201 is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0321] If the interaction duration, the start hovering duration, and the object's changing behavior all meet the constraints of the grab-and-release interaction, then the object's interactive behavior is determined to be valid.

[0322] Example 3. Based on the same inventive concept, the embodiment of the present disclosure also provides an electronic device. Since the electronic device is the electronic device in the method in the embodiment of the present disclosure, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0323] As shown in FIG13 , the electronic device includes a processor 1300 and a memory 1301 . The memory 1301 is used to store programs executable by the processor 1300 . The processor 1300 is used to read the programs in the memory 1301 and perform the following steps:

[0324] Collecting interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction between the object and the display content;

[0325] Extracting key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavioral characteristics of the interaction behavior;

[0326] Determine whether the interactive behavior of the object is valid according to the key point information.

[0327] As an optional implementation manner, the interaction information includes at least two of the following:

[0328] A first parameter set of the object in an interaction preparation stage, wherein the first parameter set includes start hovering information of the object;

[0329] a second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and a changing behavior of the object during the interaction;

[0330] The third parameter set of the object at the end of the interaction includes the retraction trajectory information of the object after the collision ends.

[0331] As an optional implementation manner, the processor 1300 is specifically configured to execute:

[0332] Extracting at least one of a start hovering position and a start hovering duration from the first parameter set;

[0333] Extracting at least one of the following from the second parameter set: a starting collision position, a depth change during the collision process, a position of a minimum depth during the collision process, a position of a maximum depth during the collision process, an ending collision position, an ending movement position, a collision duration, a movement direction, a movement speed, and a change in behavior of the object at the minimum depth; wherein the depth change indicates a change in the distance of the object from the display screen during the collision process;

[0334] An end hovering position and an interaction duration are extracted from the third parameter set, wherein the end hovering position is a position of the object in a retraction trajectory after the collision ends that is closest to the start hovering position.

[0335] As an optional implementation manner, the processor 1300 is specifically configured to execute:

[0336] determining an interaction category of the interaction behavior of the object according to the interaction information;

[0337] Extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information;

[0338] Whether the interactive behavior of the object is valid is determined based on whether the extracted key point information meets the constraint conditions corresponding to the interaction category.

[0339] As an optional implementation manner, the processor 1300 is specifically configured to execute:

[0340] Determining a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for positionally constraining the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for temporally constraining the interactive behavior from a temporal dimension;

[0341] Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter meet a constraint condition.

[0342] As an optional implementation, if the interaction category includes collision interaction, the extracted key point information includes:

[0343] Start hovering position and start hovering duration;

[0344] At least two of the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration;

[0345] End hover position and interaction duration.

[0346] As an optional implementation, if the collision interaction is a single click or a long press, the processor 1300 is specifically configured to determine whether the interaction behavior of the object is valid in the following manner:

[0347] determining a collision distance according to the starting collision position and the ending collision position;

[0348] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0349] If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all meet the single-click or long-press constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0350] As an optional implementation, if the collision interaction is a double-click, the processor 1300 is specifically configured to determine whether the interaction behavior of the object is valid in the following manner:

[0351] determining a collision distance according to the starting collision position and the ending collision position;

[0352] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0353] Determine the double-click distance based on the minimum depth of movement during the collision process of different movement trajectories;

[0354] If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time, and the interaction time all meet the double-click constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0355] As an optional implementation, if the collision interaction is zooming, the processor 1300 is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0356] Determining, based on an interactive behavior corresponding to a finger of the object, a first start hovering position, a first start collision position, a first farthest position moved during a collision process, a first depth change during the collision process, and a first start hovering duration;

[0357] Determining, based on an interaction behavior corresponding to another finger of the object, a second start hovering position, a second start collision position, a second farthest position moved during the collision process, a second depth change during the collision process, and a second start hovering duration;

[0358] determining a hovering distance according to the first start hovering position and the second start hovering position;

[0359] determining a collision distance according to the first collision start position and the second collision start position;

[0360] determining a distance between two fingers according to the first farthest position and the second farthest position;

[0361] If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, and the second start hovering duration all meet the scaling constraints, then it is determined that the interactive behavior of the object is valid.

[0362] As an optional implementation, if the interaction category includes sliding interaction, the extracted key point information includes:

[0363] Start hovering position, start hovering duration; movement direction, movement speed; end movement position and interaction duration.

[0364] As an optional implementation manner, the processor 1300 is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0365] Determine a sliding distance according to the starting hovering position and the ending moving position;

[0366] If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraints of the sliding interaction, then it is determined that the interactive behavior of the object is valid.

[0367] As an optional implementation, if the interaction category includes grab-and-release interaction, the extracted key point information includes:

[0368] The duration of the hovering start; the changing behavior of the object at the minimum depth of movement during the collision process; the duration of the interaction.

[0369] As an optional implementation manner, the processor 1300 is specifically configured to determine whether the interactive behavior of the object is valid in the following manner:

[0370] If the interaction duration, the start hovering duration, and the object's changing behavior all meet the constraints of the grab-and-release interaction, then the object's interactive behavior is determined to be valid.

[0371] Example 4. Based on the same inventive concept, the embodiment of the present disclosure also provides an object interaction device in a three-dimensional space. Since the device is the device in the method in the embodiment of the present disclosure, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0372] As shown in FIG14 , the device includes:

[0373] An information collection unit 1400 is configured to collect interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of the interaction between the object and the display content;

[0374] An information extraction unit 1401 is configured to extract key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavior characteristics of the interaction behavior;

[0375] The validity determining unit 1402 is configured to determine whether the interactive behavior of the object is valid according to the key point information.

[0376] As an optional implementation manner, the interaction information includes at least two of the following:

[0377] A first parameter set of the object in an interaction preparation stage, wherein the first parameter set includes start hovering information of the object;

[0378] a second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and a changing behavior of the object during the interaction;

[0379] The third parameter set of the object at the end of the interaction includes the retraction trajectory information of the object after the collision ends.

[0380] As an optional implementation manner, the information extraction unit 1401 is specifically configured to:

[0381] Extracting at least one of a start hovering position and a start hovering duration from the first parameter set;

[0382] Extracting at least one of the following from the second parameter set: a starting collision position, a depth change during the collision process, a position of a minimum depth during the collision process, a position of a maximum depth during the collision process, an ending collision position, an ending movement position, a collision duration, a movement direction, a movement speed, and a change in behavior of the object at the minimum depth; wherein the depth change indicates a change in the distance of the object from the display screen during the collision process;

[0383] An end hovering position and an interaction duration are extracted from the third parameter set, wherein the end hovering position is a position of the object in a retraction trajectory after the collision ends that is closest to the start hovering position.

[0384] As an optional implementation manner, the validity determining unit 1402 is specifically configured to:

[0385] determining an interaction category of the interaction behavior of the object according to the interaction information;

[0386] Extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information;

[0387] Whether the interactive behavior of the object is valid is determined based on whether the extracted key point information meets the constraint conditions corresponding to the interaction category.

[0388] As an optional implementation manner, the validity determining unit 1402 is specifically configured to:

[0389] Determining a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for positionally constraining the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for temporally constraining the interactive behavior from a temporal dimension;

[0390] Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter meet a constraint condition.

[0391] As an optional implementation, if the interaction category includes collision interaction, the extracted key point information includes:

[0392] Start hovering position and start hovering duration;

[0393] At least two of the following: the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration;

[0394] End hover position and interaction duration.

[0395] As an optional implementation, if the collision interaction is a single click or a long press, the validity determination unit 1402 is specifically configured to determine whether the interaction behavior of the object is valid in the following manner:

[0396] determining a collision distance according to the starting collision position and the ending collision position;

[0397] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0398] If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all meet the single-click or long-press constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0399] As an optional implementation, if the collision interaction is a double-click, the validity determining unit 1402 is specifically configured to determine whether the interaction behavior of the object is valid by:

[0400] determining a collision distance according to the starting collision position and the ending collision position;

[0401] Determine the hovering distance according to the starting hovering position and the ending hovering position;

[0402] Determine the double-click distance based on the minimum depth of movement during the collision process of different movement trajectories;

[0403] If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time, and the interaction time all meet the double-click constraint conditions, then it is determined that the interaction behavior of the object is valid.

[0404] As an optional implementation, if the collision-type interaction is zooming, the validity determining unit 1402 is specifically configured to determine whether the interactive behavior of the object is valid by:

[0405] Determining, based on an interactive behavior corresponding to a finger of the object, a first start hovering position, a first start collision position, a first farthest position moved during a collision process, a first depth change during the collision process, and a first start hovering duration;

[0406] Determining, based on an interaction behavior corresponding to another finger of the object, a second start hovering position, a second start collision position, a second farthest position moved during the collision process, a second depth change during the collision process, and a second start hovering duration;

[0407] determining a hovering distance according to the first start hovering position and the second start hovering position;

[0408] determining a collision distance according to the first collision start position and the second collision start position;

[0409] determining a distance between two fingers according to the first farthest position and the second farthest position;

[0410] If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, and the second start hovering duration all meet the scaling constraints, then it is determined that the interactive behavior of the object is valid.

[0411] As an optional implementation, if the interaction category includes sliding interaction, the extracted key point information includes:

[0412] Start hovering position, start hovering duration; movement direction, movement speed; end movement position and interaction duration.

[0413] As an optional implementation manner, the validity determining unit 1402 is specifically configured to determine whether the interactive behavior of the object is valid by:

[0414] Determine a sliding distance according to the starting hovering position and the ending moving position;

[0415] If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraints of the sliding interaction, then it is determined that the interactive behavior of the object is valid.

[0416] As an optional implementation, if the interaction category includes grab-and-release interaction, the extracted key point information includes:

[0417] The duration of the hovering start; the changing behavior of the object at the minimum depth of movement during the collision process; the duration of the interaction.

[0418] As an optional implementation manner, the validity determining unit 1402 is specifically configured to determine whether the interactive behavior of the object is valid by:

[0419] If the interaction duration, the start hovering duration, and the object's changing behavior all meet the constraints of the grab-and-release interaction, then the object's interactive behavior is determined to be valid.

[0420] Based on the same inventive concept, embodiments of the present disclosure provide a computer storage medium comprising computer program code. When executed on a computer, the computer program code causes the computer to execute any of the three-dimensional object interaction methods discussed above. Because the principles underlying the problems solved by the computer storage medium are similar to those of the three-dimensional object interaction methods, the implementation of the computer storage medium can be referenced to the implementation of the methods, and any repetitions will not be repeated.

[0421] In a specific implementation process, computer storage media may include: Universal Serial Bus Flash Drive (USB), mobile hard disk, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0422] Based on the same inventive concept, embodiments of the present disclosure further provide a computer program product comprising computer program code that, when executed on a computer, causes the computer to execute any of the three-dimensional object interaction methods discussed above. Because the principles underlying the problems solved by the aforementioned computer program products are similar to those of the three-dimensional object interaction methods, the implementation of the aforementioned computer program products can be referenced to the implementation of the methods, and any repetitions will not be repeated.

[0423] The computer program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0424] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0425] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0426] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0427] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0428] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A method for object interaction in a three-dimensional space, wherein: The method includes: Collecting interaction information between an object and display content in an interactive three-dimensional space, wherein the interaction information represents a behavior trajectory of an interaction between the object and the display content; Extracting key point information of the object at different interaction stages from the interaction information, wherein the key point information is used to reflect the behavior characteristics of the interaction behavior; Whether the interactive behavior of the object is valid is determined according to the key point information.

2. The method according to claim 1, wherein: The interaction information includes at least two of the following: A first parameter set of the object in the interaction preparation stage, wherein the first parameter set includes start hovering information of the object; a second parameter set of the object during the interaction phase, the second parameter set including collision information between the object and the displayed content, a moving direction and a moving speed of the object relative to the displayed content, and a changing behavior of the object during the interaction process; The third parameter set of the object at the end of the interaction includes the retraction trajectory information of the object after the collision ends.

3. The method according to claim 2, wherein: Extract key point information of the object at different interaction stages from the interaction information: Extract at least one of a start hovering position and a start hovering duration from the first parameter set; Extracting at least one of the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, the ending movement position, the collision duration, the moving direction, the moving speed, and the changing behavior of the object at the minimum depth from the second parameter set; wherein the depth change indicates the change in the distance of the object from the display screen during the collision process; The end hovering position and the interaction duration are extracted from the third parameter set, wherein the end hovering position is the position of the object in the retraction trajectory after the collision ends that is closest to the start hovering position.

4. The method according to any one of claims 1 to 3, wherein: The determining whether the interactive behavior of the object is valid according to the key point information includes: Determining an interaction category of the interaction behavior of the object according to the interaction information; Extracting key point information of different interaction stages corresponding to the interaction categories from the interaction information; Whether the interactive behavior of the object is valid is determined according to whether the extracted key point information satisfies the constraint conditions corresponding to the interactive category.

5. The method according to claim 4, wherein: The determining whether the interactive behavior of the object is valid according to whether the extracted key point information satisfies the constraint condition corresponding to the interactive category includes: Determine a first behavior parameter and a second behavior parameter according to the key point information, wherein the first behavior parameter represents a parameter for performing position constraints on the interactive behavior from a spatial dimension, and the second behavior parameter represents a parameter for performing time constraints on the interactive behavior from a time dimension; Whether the interactive behavior of the object is valid is determined according to whether the first behavior parameter and the second behavior parameter satisfy a constraint condition.

6. The method according to claim 4 or 5, wherein: If the interaction category includes collision interaction, the extracted key point information includes: Start hovering position and start hovering duration; At least two of the starting collision position, the depth change during the collision process, the position of the minimum depth during the collision process, the farthest position during the collision process, the ending collision position, and the collision duration; End hover position and interaction duration.

7. The method according to claim 6, wherein: If the collision interaction is a single click or a long press, whether the interaction behavior of the object is valid is determined in the following manner: determining a collision distance according to the start collision position and the end collision position; Determine the hovering distance according to the starting hovering position and the ending hovering position; If the depth change of the collision process, the collision distance, the hovering distance, the start hovering time, the collision time, and the interaction time all satisfy the single-click or long-press constraint conditions, it is determined that the interaction behavior of the object is valid.

8. The method according to claim 6, wherein: If the collision interaction is a double click, whether the interaction behavior of the object is valid is determined in the following manner: determining a collision distance according to the start collision position and the end collision position; Determine the hovering distance according to the starting hovering position and the ending hovering position; Determine the double-click distance according to the position of the minimum depth of movement in the collision process of different moving trajectories; If the depth change of the collision process, the collision distance, the hovering distance, the double-click distance, the start hovering time, the collision time and the interaction time all meet the double-click constraint conditions, it is determined that the interaction behavior of the object is valid.

9. The method according to claim 6, wherein: If the collision interaction is scaling, then whether the interaction behavior of the object is valid is determined in the following manner: Determine, according to an interactive behavior corresponding to a finger of the object, a first start hovering position, a first start collision position, a first farthest position moved during a collision process, a first depth change during the collision process, and a first start hovering duration; Determine, according to the interaction behavior corresponding to another finger of the object, a second start hovering position, a second start collision position, a second farthest position moved during the collision process, a second depth change during the collision process, and a second start hovering duration; determining a hovering distance according to the first start hovering position and the second start hovering position; determining a collision distance according to the first collision start position and the second collision start position; Determine the distance between two fingers according to the first farthest position and the second farthest position; If the hovering distance, the collision distance, the two-finger distance, the first depth change, the second depth change, the first start hovering duration, and the second start hovering duration all satisfy the scaling constraint conditions, it is determined that the interactive behavior of the object is valid.

10. The method according to claim 4 or 5, wherein: If the interaction category includes sliding interaction, the extracted key point information includes: Start hovering position, start hovering duration; moving direction, moving speed; end moving position and interaction duration.

11. The method according to claim 10, wherein: Determine whether the interaction behavior of the object is valid by: Determine a sliding distance according to the starting hovering position and the ending moving position; If the moving direction, the moving speed, the sliding distance, the interaction duration, and the start hovering duration all satisfy the constraint conditions of the sliding interaction, it is determined that the interaction behavior of the object is valid.

12. The method according to claim 4 or 5, wherein: If the interaction category includes grab-and-release interaction, the extracted key point information includes: Start hovering duration; the object's changing behavior at the minimum depth of movement during collision; interaction duration.

13. The method according to claim 12, wherein: Determine whether the interaction behavior of the object is valid by: If the interaction duration, the start hovering duration, and the change behavior of the object all meet the constraint conditions of the grab-and-release interaction, it is determined that the interaction behavior of the object is valid.

14. A display device, wherein: Including display and controller: The display screen is used to display content in an interactive three-dimensional space; The controller is configured to execute the steps of the method according to any one of claims 1 to 13.

15. An electronic device, wherein: The electronic device comprises a processor and a memory, wherein the memory is used to store a program executable by the processor, and the processor is used to read the program in the memory and execute the steps of any one of the methods of claims 1 to 13.

16. A computer storage medium having a computer program stored thereon, wherein: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images