Low latency pose estimation system utilizing user interaction based priority

Abstract

An interaction system comprises a controller. The controller is configured to receive spatial data of a position of a user in front of an interaction display, determine a skeletal model of the user based on the spatial data, and track the skeletal model of the user when a position and / or motion of a feature of the skeletal model matches a predetermined trigger condition for initiating a user input in a display coordinate system. The controller is further configured to generate a mapping region defining an interaction space within a region in front of the user when it has been determined that the trigger condition is matched, map the mapping region to the display coordinate system, and track the skeletal model to determine a subsequent input coordinate within the mapping region and an associated display coordinate in the display coordinate system of the interaction display when the user provides a user input.

Description

Technical Field

[0001] This technology relates to the field of human-computer interaction, specifically to systems and methods for tracking and interpreting the movements and gestures of a user in front of an interactive display. This technology is particularly relevant to interactive presentations in which users interact with digital content through their body movements and gestures. Background Technology

[0002] Interactive applications (such as those used to provide input to digital whiteboards) typically require real-time tracking of user movements and gestures to provide an immersive and intuitive user experience. A common approach to achieving this is to compute and track a skeletal model of the user based on real-time camera data. These skeletal models represent the position and movement of various body parts (such as hands, head, and limbs) related to the user's interaction with the system.

[0003] However, calculating skeleton models based on real-time camera data involves complex algorithms and computations, requiring powerful processing capabilities. This is particularly challenging for low-power computing systems, which are designed to consume minimal energy and typically have limited processing power. Consequently, these systems may struggle to efficiently process real-time camera data and calculate skeleton models, leading to long latency in interactive applications. This delay between user input and system response negatively impacts user experience and engagement.

[0004] Existing technology systems typically involve gaming applications that require significant computing resources.

[0005] Therefore, there is a need for an improved method and system for computing and tracking skeleton models in interactive applications, particularly suitable for low-power computing systems, which can reduce latency and computational load, especially when providing input to digital whiteboards or similar interactive systems. Summary of the Invention

[0006] According to a first aspect of this disclosure, an interactive system is provided, comprising a controller. The controller is configured to: receive spatial data of a user's position in front of an interactive display; determine a skeleton model of the user based on the spatial data; and track the user's skeleton model when the determined position and / or movement of features of the skeleton model matches a predetermined trigger condition for initiating user input in the display coordinate system of the interactive display. Upon determination that the trigger condition is matched, the controller generates a mapping region defining an interactive space within an area in front of the user. The controller maps the mapping region to the display coordinate system, whereby input coordinates within the mapping region have associated display coordinates in the display coordinate system. The controller then tracks the user's skeleton model to determine subsequent input coordinates within the mapping region and associated display coordinates in the display coordinate system of the interactive display when the user provides user input. The controller is configured to track the skeleton model in a first mode at a first frame rate and in a second mode at a second frame rate higher than the first frame rate. The controller is configured to track the skeleton model in the second mode when the trigger condition is determined to match. This allows for reduced computational load because the skeleton model is tracked at an increased frame rate in the second mode only when the user triggers the trigger condition, rather than being tracked continuously. This feature reduces interaction latency when the user actively provides input, where the need for low latency is paramount.

[0007] Optionally, in some examples, the controller is configured to track the skeleton model in a first mode until a match to the trigger condition is determined, after which the controller is configured to track the skeleton model in a second mode. This feature enables a smoother user experience by providing a transition between the two modes while reducing computational load.

[0008] Optionally, in some examples, the controller is configured to track the skeleton model in a second mode until the detected input feature providing the input coordinates moves out of the mapped region, after which the controller is configured to track the skeleton model in the first mode. This approach achieves efficient use of computational resources because higher frame rate tracking is only used when necessary.

[0009] Optionally, in some examples, the controller is configured to: determine the skeleton models of multiple users located in front of the interactive display, and, in a second mode, track a subset of the skeleton models of the identified active user subset once matching trigger conditions have been determined for each user in the active user subset. This feature enables multiple users to interact with the system simultaneously with less computational resource requirements, while still maintaining a high level of responsiveness to active users.

[0010] Optionally, in some examples, the frame rate in the first mode is based on the number of users in an identified subset of inactive users who have not yet been determined to match the triggering condition. Because the frame rate is adjusted based on the number of inactive users, this feature achieves efficient use of computing resources.

[0011] Optionally, in some examples, the controller is configured to: in the first mode, determine the corresponding skeleton model for one user within a subset of inactive users, in a cyclical manner, advancing one frame at a time. This feature achieves efficient use of computational resources while also determining the location of inactive users within a complete loop.

[0012] Optionally, in some examples, the controller is configured to track the skeleton model frame by frame in a second mode. This feature enables a high level of responsiveness when the user actively provides input.

[0013] Optionally, in some examples, the controller is configured to track a subset of skeleton models of an identified pre-active user subset in a third mode. The controller is configured to track the skeleton models at a third frame rate in the third mode, where the third frame rate is higher than the first frame rate but lower than the second frame rate. This feature results in more responsive interactions for users who are about to become active, thus providing a smoother user experience.

[0014] Optionally, in some examples, the controller is configured to continuously receive spatial data, once the triggering condition has been determined to match, to track the user's skeletal model and the user's associated motion within the mapped area over a period of time. The controller is configured to determine the input coordinates corresponding to the motion and generate control signals to display the output corresponding to the motion in a display coordinate system. This feature enables continuous tracking of user motion, thereby providing a more natural and intuitive interaction with the system.

[0015] Optionally, in some examples, the controller is configured to generate a control signal to stop displaying the output in the display coordinate system when the detected input feature, providing the input coordinates, moves out of the mapped area. This feature provides a clear end to the user input, thus offering a more intuitive interaction with the system.

[0016] Optionally, in some examples, the controller is configured to detect multiple features of the skeleton model and / or combinations of their motions, such that a matching trigger condition is determined once the combination of features matches a predetermined user command used to trigger user input in the display coordinate system. This feature can reduce the occurrence of false triggers of user input.

[0017] Optionally, in some examples, user input is triggered based on a combination of features detected from spatial data, which includes at least a first feature of a skeleton model associated with the user's body and a second feature associated with an input object such as a stylus. This feature can reduce the occurrence of false triggers of user input while enabling a wider range of user inputs.

[0018] Optionally, in some examples, user input is triggered based on detecting feature combinations in a defined order, and / or based on detecting feature combinations that satisfy a defined spatial relationship. This feature can further reduce the occurrence of false triggers of user input.

[0019] According to a second aspect of this disclosure, a method for an interactive system is provided. The method includes: receiving spatial data of a user's position in front of an interactive display; determining a skeleton model of the user based on the spatial data; and tracking the skeleton model when the determined position and / or motion of features of the skeleton model matches a predetermined trigger condition for initiating user input in a display coordinate system of the interactive display. When the matching trigger condition has been determined, the method includes: generating a mapping region, wherein the mapping region defines an interactive space within a region in front of the user; mapping the mapping region to a display coordinate system, whereby input coordinates within the mapping region have associated display coordinates in the display coordinate system; and tracking the user's skeleton model to determine subsequent input coordinates within the mapping region and associated display coordinates in the display coordinate system of the interactive display when the user provides user input. The method includes tracking the skeleton model in a first mode at a first frame rate and in a second mode at a second frame rate, the second frame rate being higher than the first frame rate. The method includes: tracking the skeleton model in the second mode when the matching trigger condition has been determined. This method reduces computational load because the skeleton model is tracked only when the user triggers the trigger condition, rather than being tracked continuously. Attached Figure Description

[0020] The example will be explained in more detail below with reference to the accompanying drawings.

[0021] Figure 1 It is an interactive system 100 based on the example of this disclosure.

[0022] Figure 2 An example of tracking a skeleton model using a three-frame 0-2 image sequence is shown in the interactive system 100, illustrating the identification of active and inactive user subsets and their corresponding skeleton models. The active user subset is tracked frame by frame, while the skeleton model of the inactive user subset is cycled through every three frames, thus enabling computation at a lower frame rate.

[0023] Figure 3The diagram shows a sequence of three frames, with each frame indicating which skeleton models were computed in that image. Here, each skeleton model is computed in every single frame, resulting in a very high computational load and poor frame rates and latency experienced by the interactive user.

[0024] Figure 4 A typical skeleton model of a user is shown, where each key point is marked with a unique ID.

[0025] Figure 5 A method for an interactive system, based on an example of this disclosure, is described. Detailed Implementation

[0026] The specific embodiments described below provide information and examples of the disclosed technology, and the details provided are sufficient to enable those skilled in the art to practice this disclosure.

[0027] Figure 1 An interactive system 100 according to an example of this disclosure is shown. The interactive system 100 includes a controller 101 configured to receive spatial data of the user's position in front of an interactive display 201. The controller 101 determines a skeleton model 103 of the user based on the spatial data, and when the skeleton model 103 has features 106 (see...) Figure 2 and Figure 4 When the position and / or movement of the user (as shown in the example) matches a predetermined trigger condition for initiating user input in the display coordinate system 203 of the interactive display 201, the controller 101 tracks the skeleton model 103. Once a matching trigger condition has been determined, the controller 101 generates a mapping region 104 that defines an interaction space 105 within an area A in front of the user. The mapping region 104 is mapped to the display coordinate system 203, whereby input coordinates (xi, yi) within the mapping region 104 have associated display coordinates (xd, yd) in the display coordinate system 203. The controller 101 is configured to track the user's skeleton model 103 to determine subsequent input coordinates (xi, yi) within the mapping region and associated display coordinates (xd, yd) in the display coordinate system 203 of the interactive display 201 when the user provides user input.

[0028] Controller 101 is configured to track skeleton model 103 at a first frame rate in a first mode and at a second frame rate in a second mode. The second frame rate is higher than the first frame rate. As further described below, controller 101 is configured to track skeleton model 103 in the second mode when a matching trigger condition has been determined.

[0029] Figure 2An example of using a three-frame (0-2) image sequence to track skeleton models in an interactive system 100 is shown, illustrating the identification of an active user subset 205 and an inactive user subset 206 and their corresponding skeleton models. Users marked by short dashed bounding boxes are identified as interacting with system 100 because, in this example, feature 106 of skeleton model 103 matches the trigger condition. The skeleton model 103 of the triggering user is tracked in the second mode at an associated second frame rate, while the remaining users are tracked in the first mode at a first frame rate, which is lower than the second frame rate. Therefore, the triggering user can be identified as the active user subset 205, and the associated skeleton model 103 is computed and tracked in each frame (0-2). The remaining users marked by long dashed bounding boxes can be identified as the inactive user subset 206, and their skeleton models are polled and traversed in a three-frame cycle, thus being computed at a lower frame rate.

[0030] Figure 3 The image shows a sequence of three frames, with each frame indicating which skeleton models were computed in that image. In this example, computed every skeleton model in every frame, resulting in a very high computational load and poor frame rates and latency for the interactive user.

[0031] Figure 4 A typical skeleton model of a user is shown, where each key point is labeled with a unique ID. The skeleton model represents the location of user body parts (such as hands, head, or other body parts related to user and system interaction).

[0032] Figure 5 A method 300 for an interactive system 100 according to an example of this disclosure is described. Method 300 includes: receiving 301 spatial data of a user's position in front of an interactive display 201; determining 302 a skeleton model 103 of the user based on the spatial data; tracking 303 the skeleton model when the position and / or movement of features 106 of the skeleton model matches a predetermined trigger condition for initiating user input in a display coordinate system 203 of the interactive display; generating 304 a mapping region 104; mapping the mapping region to the display coordinate system; and tracking 306 the user's skeleton model to determine subsequent input coordinates (xi, yi) within the mapping region and associated display coordinates (xd, yd) in the display coordinate system of the interactive display when the user provides user input. Method 300 includes: tracking 307 the skeleton model at a first frame rate in a first mode and at a second frame rate in a second mode, the second frame rate being higher than the first frame rate. Method 300 further includes: tracking 308 the skeleton model in the second mode when a matching trigger condition has been determined. Therefore, method 300 provides the advantages described in conjunction with the interactive system 100. This method 300 provides a low-latency interactive experience for active users, while reducing computational load by tracking inactive users at a lower frame rate.

[0033] 1. Details of the Interaction System The interactive system 100 enables convenient interaction between the user and the display. The system 100 tracks the user's movement and translates it into input for the display 201. The system 100 is responsive and efficient, reducing computational load by tracking the movement of the user actively interacting with the display 201 with higher accuracy. This is achieved using a controller 101, which receives spatial data of the user's position and creates a skeleton model 103 of the user. Subsequently, the controller 101 tracks the skeleton model when the position and / or movement of features 106 of the skeleton model match predetermined triggering conditions for initiating user input in the display coordinate system 203 of the interactive display 201.

[0034] 1.1 Receive spatial data and determine the skeleton model In some implementations, controller 101 is a component of interaction system 100. Controller 101 is configured to receive spatial data of the user's position in front of interactive display 201. This spatial data can be obtained from various sources (e.g., a camera or sensor 107) and may include information about the user's position, orientation, and movement. Based on this spatial data, controller 101 determines a skeletal model 103 of the user. Skeletal model 103 is a representation of the user's body, including key points such as the head, hands, and other body parts relevant to the user's interaction with the system.

[0035] 1.2 Trigger-based skeleton tracking model The controller 101 tracks the skeleton model when the position and / or movement of the skeleton model feature 106 matches a predetermined trigger condition for initiating user input in the display coordinate system 203 of the interactive display 201.

[0036] 1.3 Generating and Mapping the Interaction Space Controller 101 is also configured to generate a mapping region 104, which defines an interaction space 105 within region A in front of the user, such as... Figure 1 As illustrated schematically. The mapping area is then mapped to the display coordinate system 203, whereby the input coordinates (xi, yi) within the mapping area 104 have associated display coordinates (xd, yd) in the display coordinate system 203. The controller 101 tracks the user's skeleton model 103 and monitors the user's movement within the interaction space 105. As the user moves, the controller 101 determines the associated display coordinates (xd, yd) within the mapping area 104 and in the display coordinate system 203. This achieves the conversion of the user's movement within the interaction space 105 into input to the interactive display 201. The mapping area 104 is designed to be flexible and adaptable, allowing the interaction space 105 to be adjusted based on the user's position and movement.

[0037] In some examples, interactive display 201 is a component of interactive system 100. Interactive display 201 can be any type of display capable of receiving and processing user input. In some embodiments, interactive display 201 can be a touch-sensitive display capable of detecting the presence and location of a touch within a display area. In other embodiments, interactive display 201 can be a non-touch-sensitive display used in conjunction with other input devices, such as a stylus or mouse. Interactive display 201 is communicatively connected to controller 101.

[0038] 1.4 Triggering Features of Skeleton Model Trigger feature 106 is a combination of one or more parts of skeleton model 103 used to determine when the position and / or movement of feature 106 matches a predetermined trigger condition for initiating user input in display coordinate system 203 of interactive display 201. Trigger feature 106 can relate to any part of skeleton model 103, such as a user's hand, arm, head, or other body part associated with interaction between the user and system 100. For brevity, this disclosure refers to this part or more parts of skeleton model 103 as feature 106. For example, feature 106 can trigger the trigger condition when a combination of multiple parts of skeleton model 103 is placed in a specific relative position relative to each other. For example, a user expecting to provide input can raise their arm, and the trigger condition can be triggered when the user's left or right forearm and upper arm are identified as being in a defined relative position (e.g., within a defined angular interval) relative to each other. In other examples, the trigger condition can be satisfied when the forearm and / or upper arm are in a defined position relative to the user's head (e.g., within a defined distance from the user's head). In other examples, the trigger condition can be met when trigger feature 106 follows a defined movement path. For example, a user can raise their arm and then move the arm relative to their body, such as extending it away from their body, to trigger the trigger condition.

[0039] Therefore, the interactive system 100 can be used under different triggering conditions. As discussed, triggering conditions can be specific locations, such as the specific relative positions or movements of different parts of the skeleton model 103, or combinations of positions or movements of multiple parts of the skeleton model 103. The controller 101 tracks the skeleton model and monitors the position and / or movement of the feature 106. When the feature 106 matches a predetermined triggering condition, the controller 101 initiates user input in the display coordinate system 203 and, as further described below, tracks the skeleton model 103 of the active user with higher accuracy than the skeleton model of an inactive user, thereby reducing computational load.

[0040] 1.5 Frame Rate Adjustment Based on User Activity Controller 101 is configured to track skeleton model 103 at a first frame rate in a first mode and at a second frame rate in a second mode, where the second frame rate is higher than the first frame rate. When a matching trigger condition has been determined, controller 101 tracks skeleton model 103 in the second mode, for example... Figure 2 As shown, features 106 of the user's skeleton model 103 are matched with the triggering conditions. This enables system 100 to provide a low-latency interactive experience for active users while reducing computational load by tracking inactive users at a lower frame rate.

[0041] Controller 101 can be configured to track skeleton model 103 in a first mode until a match to the trigger condition is determined, after which controller 101 is configured to track skeleton model 103 in a second mode. This saves computational resources when the user does not actively provide input. Controller 101 can also be configured to track the skeleton model in the second mode until the detected input feature 202 providing input coordinates (xi, yi) moves out of the mapping region 104, after which controller 101 is configured to track skeleton model 103 in the first mode. Input feature 202 can be the user's hand or finger, or a stylus. Therefore, when the user exits the input mode, controller 101 can return to the first mode for tracking at a lower resolution.

[0042] The controller 101 can be configured to: determine the skeleton models 103 of multiple users located in front of the interactive display 201, and, in a second mode, track the skeleton model subset of the identified active user subset 205 when the triggering conditions have been determined for each user in the active user subset 205. Figure 2 An example is shown where one user in a group of four has been identified as an active user subset 205 because the position and / or action of feature 106 of the skeleton model 103 of the aforementioned user matches the triggering condition. The active user subset 205 is tracked in a second mode, and for... Figure 2 In each frame of the example, the skeleton model 103 of the users in the active user subset 205 is calculated. Therefore, the controller 101 can be configured to track the skeleton model 103 frame by frame in the second mode.

[0043] exist Figure 2 In the example, the skeleton model of the inactive user subset 206 is traversed in a three-frame cycle, thus being computed at a lower frame rate. Therefore, the frame rate in the first mode can be based on the number of users in the identified inactive user subset 206 for whom a matching trigger condition has not yet been determined. Controller 101 can be configured to: in the first mode, in a cyclical manner, advance one frame at a time, determine the corresponding skeleton model for one user in the inactive user subset 206, such as... Figure 2 As shown in the example.

[0044] Controller 101 can be configured to track a subset of skeleton models of an identified pre-active user subset in a third mode. Controller 101 can be configured to track the skeleton models at a third frame rate in the third mode. The third frame rate is higher than the first frame rate but lower than the second frame rate. This feature provides more responsive interactions for users about to become active, resulting in a smoother user experience.

[0045] Controller 101 can be configured to continuously receive the spatial data once a matching trigger condition has been determined, to track the user's skeleton model 103 and the user's associated motion within the mapped area over a period of time. Therefore, controller 101 can be configured to determine the input coordinates (xi, yi) associated with the motion and generate control signals to display the motion output 204 in the display coordinate system (xd, yd), such as... Figure 1 As shown. Controller 101 can be configured to generate a control signal to stop displaying the output in the display coordinate system when the detected input feature 202, which provides input coordinates (xi, yi), moves out of the mapping region 104.

[0046] The controller 101 can be configured to detect multiple features of the skeleton model 103 and / or combinations of their motions, such that a matching trigger condition is determined once the combination of features matches a predetermined user command for triggering user input in the display coordinate system 203. This feature reduces the occurrence of false triggering of user input. User input can be triggered based on detecting a combination of features from spatial data, the combination of features including at least a first feature 106 of the skeleton model associated with the user's body and a second feature associated with an input object (e.g., a stylus). User input can be triggered based on detecting a combination of features in a defined order, and / or based on detecting a combination of features that satisfies a defined spatial relationship.

[0047] In some configurations, the interactive system 100 can be applied to video conferencing systems, home entertainment systems (such as televisions), or digital whiteboards. A digital whiteboard is an interactive display with touch sensors that allows users to write or draw on the display using their fingers or a stylus. The interactive system 100 can enhance the user experience of the digital whiteboard by providing a low-latency interactive experience and reducing computational load. The controller 101 receives spatial data of the user's position in front of the digital whiteboard and determines the user's skeleton model 103. Subsequently, the controller 101 tracks the skeleton model when the position and / or movement of features 106 of the skeleton model matches a predetermined trigger condition for initiating user input in the display coordinate system 203 of the digital whiteboard. The controller 101 generates a mapping region 104 that defines an interaction space 105 within an area A in front of the user. The mapping region is then mapped to the display coordinate system 203, whereby input coordinates (xi, yi) within the mapping region have associated display coordinates (xd, yd) in the display coordinate system 203. This enables the conversion of user movement within the interaction space 105 into input for the digital whiteboard, providing a seamless and intuitive user experience. The controller 101 is also configured to track the skeleton model at a first frame rate in a first mode and at a second frame rate in a second mode, where the second frame rate is higher than the first frame rate. When a matching trigger condition has been determined, the controller 101 tracks the skeleton model in the second mode. This allows the system 100 to provide a low-latency interactive experience for active users while reducing computational load by tracking inactive users at a lower frame rate. This application of the interaction system 100 can enhance the user experience of the digital whiteboard and touch-sensing device by providing a more responsive and intuitive interactive experience.

[0048] The terminology used herein is for descriptive purposes only and is not intended to limit this disclosure. As used herein, the singular forms “a” (or “an”) and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items. It should also be understood that the terms “comprising” and / or “including” as used herein specify the presence of the stated features, elements, actions, steps, operations, components, and / or parts, but do not preclude the presence or addition of one or more other features, elements, actions, steps, operations, components, and / or groups thereof.

[0049] It should be understood that although the terms first, second, etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this disclosure, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0050] Relative terms (e.g., "below," "above," "upper," "lower," "horizontal," or "vertical") may be used herein to describe the relationship between one element and another, as illustrated in the figures. It should be understood that these terms, and those discussed above, are intended to cover different orientations of the device other than those depicted in the figures. It should be understood that when an element is referred to as "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements present. Conversely, when an element is referred to as "directly connected" or "directly coupled" to another element, there are no intermediate elements present.

[0051] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It should also be understood that terms used herein shall be interpreted as having the same meaning as in the context of this specification and related art, and shall not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0052] It should be understood that this disclosure is not limited to the aspects shown above and in the accompanying drawings; rather, those skilled in the art will recognize that many changes and modifications can be made within the scope of this disclosure and the appended claims. Aspects have been disclosed in the drawings and description for illustrative purposes only and not for limiting purposes, and the scope of this disclosure is set forth in the appended claims.

Claims

1. An interactive system (100), comprising: Controller (101), the controller is configured to: Receive spatial data of the user's position in front of the interactive display (201); Based on the spatial data, the user's skeleton model is determined (103). When the position and / or movement of the feature (106) of the skeleton model is determined to match a predetermined trigger condition for initiating user input in the display coordinate system (203) of the interactive display, the skeleton model of the user is tracked. The controller is further configured to, when a match has been determined to be found: A mapping region (104) is generated, wherein the mapping region defines the interaction space (105) within the area (A) in front of the user. The mapping region is mapped to the display coordinate system, whereby the input coordinates (xi, yi) within the mapping region have associated display coordinates (xd, yd) in the display coordinate system (203); and Track the user's skeleton model to determine subsequent input coordinates (xi, yi) within the mapped area and associated display coordinates (xd, yd) in the display coordinate system of the interactive display when the user provides the user input; The controller is configured to track the skeleton model at a first frame rate in a first mode and at a second frame rate in a second mode, wherein the second frame rate is higher than the first frame rate, and wherein the controller is configured to track the skeleton model in the second mode when the trigger condition has been determined to match.

2. The interactive system according to claim 1, wherein, The controller is configured to track the skeleton model in the first mode until a match to the trigger condition is determined, and then the controller is configured to track the skeleton model in the second mode.

3. The interactive system according to claim 1 or 2, wherein, The controller is configured to track the skeleton model in the second mode until the detected input feature (202) providing the input coordinates (xi,yi) moves out of the mapping region, and then the controller is configured to track the skeleton model in the first mode.

4. The interactive system according to any one of claims 1 to 3, wherein, The controller is configured to: Determine the skeletal models of multiple users located in front of the interactive display; and Once the triggering conditions have been determined for each user in the active user subset (205), the skeleton model subset of the identified active user subset is tracked in the second mode.

5. The interactive system according to any one of claims 1 to 4, wherein, The frame rate in the first mode is based on the number of users in the identified subset (206) of inactive users who have not yet been determined to match the triggering condition.

6. The interactive system according to claim 5, wherein, The controller is configured to: in the first mode, in a cyclical manner, determine the corresponding skeleton model for one user within the subset of inactive users for each frame advanced.

7. The interactive system according to any one of claims 1 to 6, wherein, The controller is configured to track the skeleton model frame by frame in the second mode.

8. The interactive system according to any one of claims 1 to 7, wherein, The controller is configured to track a subset of skeleton models of an identified subset of pre-active users in a third mode, wherein the controller is configured to track the skeleton models at a third frame rate in the third mode, wherein the third frame rate is higher than the first frame rate and lower than the second frame rate.

9. The interactive system according to any one of claims 1 to 8, wherein, The controller is configured to continuously receive the spatial data when the triggering condition has been determined to match, so as to track the user's skeleton model and the user's associated motion within the mapped area over a period of time. The controller is configured to: determine the input coordinates (xi, yi) associated with the motion and generate a control signal to display the output of the motion (204) in the display coordinate system (xd, yd).

10. The interactive system according to claim 9, wherein, The controller is configured to generate a control signal to stop displaying the output in the display coordinate system when the detected input feature (202) providing the input coordinates (xi,yi) moves out of the mapping area.

11. The interactive system according to any one of claims 1 to 10, wherein, The controller is configured to detect a plurality of the features of the skeleton model and / or combinations of their motions, such that a matching trigger condition is determined once the combination of the features matches a predetermined user command for triggering user input in the display coordinate system.

12. The interactive system according to claim 11, wherein, The user input is triggered based on the detection of the following feature combination from the spatial data, the feature combination including at least a first feature (106) of the skeleton model associated with the user's body and a second feature associated with an input object such as a stylus.

13. The interactive system according to claim 11 or 12, wherein, The user input is triggered based on the detection of feature combinations in a defined order, and / or based on the detection of feature combinations that satisfy a defined spatial relationship.

14. A method (300) for an interactive system (100), comprising: Receive spatial data of the user's position in front of the interactive display (201) (301); Based on the spatial data, determine (302) the user's skeleton model (103); When the position and / or movement of the feature (106) of the skeleton model (103) is determined to match a predetermined trigger condition for initiating user input in the display coordinate system (203) of the interactive display, the skeleton model (103) is tracked (303). Wherein, when the triggering condition has been determined to match, the method includes: Generate (304) a mapping region (104), wherein the mapping region defines an interaction space (105) within a region (A) in front of the user. The mapping region (305) is mapped to the display coordinate system, whereby the input coordinates (xi, yi) within the mapping region have associated display coordinates (xd, yd) in the display coordinate system (203); and Tracking (306) the skeleton model of the user to determine subsequent input coordinates (xi, yi) within the mapped area and associated display coordinates (xd, yd) in the display coordinate system of the interactive display when the user provides the user input; wherein, the method further includes: The skeleton model is tracked in a first mode at a first frame rate and in a second mode at a second frame rate, where the second frame rate is higher than the first frame rate; and Once the triggering condition has been determined to match, the skeleton model is tracked (308) in the second mode.

Images