Method and apparatus for user control of applications and corresponding devices

Abstract

The present disclosure relates to a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by an input element obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by an input element obtained from a group or one or more tracked users, the method comprising: - tracking an action of one or more users selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate users, the selection request being captured by at least one sensor coupled to the electronic device; - determining at least one input element suitable for controlling the electronic device according to the tracked action. It also relates to corresponding devices, electronic components, systems, computer-readable program products, and storage media.

Description

Technical Field

[0001] The present disclosure relates to the technical field of user control of electronic devices suitable for use in a multi-user environment. Corresponding applications, devices, electronic components, systems, computer-readable program products, and computer-readable storage media are described. Background Art

[0002] Many electronic devices can be coupled to user tracking components to obtain information about the user, thereby obtaining real-time information. In the present disclosure, the user tracking component includes, for example, a sensor (wearable or non-wearable) that obtains information about the shape, posture, position and / or movement of the user or one or more parts of his or her body, such as a depth camera or a chrominance camera. Based on the information obtained by the sensor, some of those electronic devices may also be at least partially controlled by (one or more) users. In fact, the tracking mechanism helps enable a natural interactive experience from the user's perspective. For example, in the field of virtual reality (VR) or augmented reality (AR), tracking the user's movement can allow the user to manipulate virtual objects of a virtual scene or navigate in a 3D scene without using auxiliary devices (such as remote control devices).

[0003] However, user tracking in multi-user environments (such as collaborative games or collaboration tools) causes some problems because the sensors that tracking relies on can obtain information about multiple users concurrently.

[0004] Some solutions propose to consider only actions performed by all users simultaneously, or to track only joint actions of users (such as overlapping hands of two users in a 2-user environment). However, all these solutions do not allow users to conveniently interact with applications in a multi-user environment.

[0005] Therefore, a solution is needed to enhance the user experience of applications that are at least partially controlled by user tracking. Summary of the invention

[0006] The present principles address at least one of the above-mentioned disadvantages to be addressed by proposing a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, the current mode of the electronic device and at least one tracked user being selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0007] According to some embodiments, the present disclosure relates particularly to a method implemented in an electronic device, wherein the electronic device is adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method comprising:

[0008] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0009] - determining at least one input element suitable for controlling the electronic device based on the tracked motion.

[0010] According to another aspect, the present disclosure also relates to an electronic device adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, the electronic device comprising at least one processor adapted to:

[0011] A current mode of the electronic device and at least one tracked user is selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0012] According to some embodiments, the present disclosure relates particularly to an electronic device adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the electronic device comprising at least one processor adapted to:

[0013] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0014] - determining at least one input element suitable for controlling the electronic device based on the tracked motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present disclosure may be better understood, and other specific features and advantages may emerge, upon reading the following description, which refers to the accompanying drawings, in which:

[0016] Figure 1 An example of a tracking system according to an exemplary embodiment of the present disclosure is shown;

[0017] Figure 2 illustrates an example functional structure of an electronic device suitable for implementing at least some embodiments of the methods of the present disclosure;

[0018] Figure 3 depicts an example overview of user interaction with a tracking system in at least some embodiments of the present disclosure;

[0019] Figure 4 illustrates a state machine of an example selection mechanism that may be used when implementing some embodiments of the methods of the present disclosure;

[0020] Figure 5 A state machine illustrating an example user state diagram that may be used when implementing some embodiments of the methods of the present disclosure;

[0021] Figure 6 An overview of some example embodiments of the disclosed methods is depicted.

[0022] It should be noted that the drawings illustrate example embodiments and that the embodiments of the present disclosure are not limited to the embodiments shown. For example, it is obvious to those skilled in the art that the embodiments of the present disclosure may also be implemented in a Figures 1 to 6 A more complex tracking system is implemented in . DETAILED DESCRIPTION

[0023] The present disclosure relates to the technical field of electronic devices or corresponding applications or methods, adapted to be at least partially controlled by at least one physical action of one or more users (such as concurrent users), the at least one physical action being captured by one or more sensing devices (or sensors) coupled to the electronic device and triggering processing of the electronic device. For example, such a physical action can be a gesture or a sequence of gestures, a sound or a sequence of sounds, or a combination thereof, generated by one or more users.

[0024] Unlike prior art solutions, at least some embodiments of the present disclosure may allow selection between multi-user mode and single-user mode and dynamic selection of the user that will interact with the system and therefore be tracked.

[0025] The term "processing" will be understood herein in its larger context and encompass one or more different operations and, for example, modifying the rendering of content on a display device (e.g., a display of a television) coupled to an electronic device. The rendering of content may vary depending on the nature of the content and may include, for example, displaying a visual portion of the content or playing an audio portion of the content. "Visual portion" should be understood as a video, one or more image sequences and / or one or more still images, or a combination thereof.

[0026] In an example use case where the electronic device is adapted to process VR or AR content comprising at least one virtual object or scene, processing triggered by an action of a user may, for example, initiate and / or modify rendering of such a virtual object or scene.

[0027] In at least one embodiment of the present disclosure, the electronic device may be configured as a "single-user" (or "single user") mode in which only the actions of a single user are tracked, or a "multi-user" (or "multiple users") mode in which the actions of multiple users may be tracked (e.g., in which several actions of several users may be tracked simultaneously).

[0028] In single-user mode, only one person can control the interaction at a time. For example, in some embodiments, once a user is selected to control the application, other users cannot interfere. Such embodiments may, for example, allow a single user to navigate in a 3D scene and / or thereby change the rendering of the scene, with other (one or more) people located near the selected user being only allowed to view the 3D scene without changing it.

[0029] In multi-user mode, two or more users can interact with the electronic device at the same time. For example, such an embodiment may be suitable for use cases where the electronic device is executing a collaborative application such as a game, e.g., a 3D puzzle, where players are considered a team and must work together to achieve a common goal (e.g., putting elements together to form an object or drawing a shared object or scene). Multi-user mode may also be suitable for situations where multiple users can act independently (or almost independently), such as when different users operate different objects in the same VR or AR scene.

[0030] Figure 1 Illustrated is a tracking system 100 that includes one or more electronic devices suitable for implementing at least one embodiment of the present disclosure.

[0031] exist Figure 1 In the embodiment of the present invention, the tracking system 100 is located in a home environment. or Interface), some electronic devices of the system can exchange data within a communication network (e.g., a local area network (LAN)).

[0032] The tracking system may, for example, include one or several gateways 110 providing access to at least one other communication network, such as a wide area network (WAN). For example, the gateway (GTW) 110 may provide access to a broadcast network or a broadband network, such as the Internet. For example, the GTW 110 may receive data from a broadcast network or receive and / or transmit data to a broadband network. Thus, at least some of the electronic devices of the system 100 may receive and / or transmit data, such as audiovisual data, from those other communication networks via the gateway 110.

[0033] The tracking system 100 may also include at least one other electronic device adapted to process (and, for example, render) audio and / or visual signals. Such electronic devices may include a set-top box (STB) 140, a tablet 120, a PC, a smartphone, a Blu-ray player, a television 160, an audio speaker 150, and / or a display 130. All of these exemplary devices may be connected to the GTW 110 wirelessly or via a wired connection.

[0034] Figure 2 An example functional structure of an electronic device 200 that can implement the application is described. The electronic device 200 can be, for example, suitable for implementing the method 600 of the present disclosure in at least some embodiments thereof, which will be described later. The electronic device can be, for example, the above combined Figure 1 One of the exemplary electronic devices introduced. It should be noted that Figure 2 The elements, modules, or blocks of the functional structure shown in FIG. 2 may be implemented by software components of the electronic device 200 and / or hardware components of the electronic device 200 . Figure 2 At least some of the elements shown in may include at least one software component and at least one hardware component of processing device 200. Depending on the embodiment, several of these elements may be part of the same physical device or may belong to different, separate, physical devices.

[0035] exist Figure 2 In an example embodiment of the electronic device 200, the electronic device 200 may include different devices (or modules) connected together via a communication bus 20 (such as a control, data and address bus, which may also carry timer signals). For example, it may include one or several microprocessors 21 (or central processing units (CPUs)), ROM (or "read only memory") 25, RAM (or "random access memory") 26 and (one or more) wired and / or wireless communication interfaces 27, 28 providing access to LAN and / or WAN networks. For example, at least one of the communication interfaces 27, 28 of the electronic device 200 may be adapted to receive one or more audio and / or video content or streams.

[0036] exist Figure 2In the example embodiment of the present invention, the electronic device 200 may also include at least one input / output (I / O) module 24 (such as a tactile display, switch, LED, button, etc.). For example, the electronic device may include (or more generally be coupled to) at least one acquisition module 242 suitable for detecting user body movements, such as an audio and / or video acquisition device, such as a camera, a microphone, and / or a position and / or motion tracking device (such as a GPS module, a rotating mouse, a gyroscope, an accelerometer).

[0037] exist Figure 2 In an example embodiment of the present invention, one or more acquisition modules are adapted to detect gestures of a user and detect movements of a user, such as hand movements.

[0038] In the illustrated embodiment, the electronic device may include (or more generally be coupled to) a content rendering module 244, such as a television or tablet computer, adapted to render audio and / or visual content in one or more formats. The content rendering module may include a display and / or a speaker. In some embodiments, this content rendering unit may be optional.

[0039] Each of the mentioned memories may comprise at least one register, ie a memory area of ​​low capacity (small amount of binary data) or high capacity (capable of storing an entire program or all or part of data representing data to be calculated or displayed).

[0040] When the electronic device 200 is powered on, at least one microprocessor 21 loads program instructions 260 into registers of the RAM 26 via a power module 22, such as a power module connected to a power line network, for example, to perform the processes required for at least one embodiment of the method 600 described below and execute the program instructions.

[0041] In any embodiment of the present disclosure, the electronic device 200 of the system may be suitable for implementing the method 600 of the present disclosure. Figure 2 In the example embodiment shown in , the electronic device is suitable for being configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, and the microprocessor 21 may be configured to:

[0042] - selecting a current mode of the electronic device and at least one tracked user according to at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0043] In some embodiments, the microprocessor 21 may be configured to:

[0044] - tracking body movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0045] - determining at least one input element suitable for controlling said electronic device based on said tracked physical movement.

[0046] Combination Figure 2 The depicted structure is for example purposes only. It is noted that the electronic device 200 may also include additional capabilities even though shown as provided by separate devices. In addition, at least some embodiments of the present disclosure may be implemented at least in part in at least one electronic device that is externally coupled to another device.

[0047] An example embodiment of the method 600 of the present disclosure is described below with respect to Figure 4 , 5 and 6 for details. Figure 6 An overview of an example embodiment of method 600 is shown, Figure 4 illustrates a state machine suitable for use by a method for selecting one or more users authorized to control an electronic device, and Figure 5 The diagram shows the state of tracking a user staying in the sensing area of ​​the tracking system. In the example embodiment shown, the method Figure 2 The software application may be implemented in a software application executed in the electronic device 200. The software application may be started when the electronic device 200 is powered on or at a later stage.

[0048] In a specific embodiment, the electronic device can be configured in a single-user mode or a multi-user mode. The single-user mode and the multi-user mode are respectively associated with a first and a second motion mode, which respectively correspond to one or more motions that can be captured by at least one sensor of the tracking system. The detection of the first and the second motion mode is interpreted as a user request to be selected to control the electronic device and to configure the device in the corresponding mode, respectively. The corresponding detected user action will be referred to as a "selection request" (more precisely, a single-user selection request or a multi-user selection request, depending on the motion mode) hereinafter.

[0049] exist Figure 4 and 5 In an example embodiment, the first and second motion modes are registered gesture modes, for example, the first motion mode corresponds to "single hand raised", and the second motion mode corresponds to "both hands raised", and the detected motion mode is the "arm raised" gesture.

[0050] When the application is started, or when there is no user in the acquisition area of ​​the sensor, the device has not yet been controlled by a given user and is therefore configured in a default mode. In some embodiments, the default mode can be a single-user mode or a multi-user mode (e.g., selecting (one or more) default users to control the electronic device, such as the first user detected by the tracking module of the electronic device once the application is started, or the user who is currently closest to the electronic device). In other embodiments, the default mode can be another mode, such as Figure 4 The "idle" (or "waiting for control request") mode (idle state 410) shown in FIG. Figure 4 and 6 In an example embodiment of the present invention, the electronic device remains in the "idle" mode until an action corresponding to the first and / or second action mode is detected. When the first action mode is detected 610 while the device is configured in the "idle" mode, the method includes configuring 620 the electronic device in "single user mode (one user selection state 420)". Selecting 630 a user at the origin of the detected action as a single user to control the application, and the method includes tracking the actions performed by the selected user in order to detect an event to be used as an input element for controlling 640 the device.

[0051] Similarly, when the second action mode 612 is detected when the device is configured in the "idle" mode (idle state 410), the method includes configuring the electronic device in the "multi-user mode" (multi-user selection state 430). Figure 6 As shown, the method comprises constituting a group of users authorized to control the device by selecting 630 users. More precisely, the user who is the origin of the detected motion corresponding to the first or second motion mode, respectively, is selected to belong to the group when configuring the device in single-user or multi-user mode.

[0052] In multi-user mode, additional users can be added to the group later. Figure 4 and 6In the embodiment shown in , selection 630 is performed during a limited time period. For example, the selection period can start when the device is configured as a multi-user mode (multi-user selection state 430). Depending on the embodiment, the selection period can be a period of a given duration, such as a constant duration, such as when a timer initiated at the beginning of the selection period reaches its end (or zero value) with a certain amount of time (or in other words, when the amount of time set on the timer at the beginning of the selection period passes), the period ends, or the selection period can be a period of variable duration ended by a specific operation on the device (such as a specific action of a user (e.g., a selected user or a user acting as a supervisor)), or when a given number of users (such as the maximum number of users selected at the same time) is currently selected. For example, a specific action can correspond to a given action mode known from an electronic device. Depending on the embodiment, a given action mode can be a first action mode, a second action mode and / or an action mode different from the first and / or second action mode. Similarly, the selection period can start from a specific event (such as a selection of the first user, or a command of a supervisor user, or an automatic event triggered by a timer, a specific state of an electronic device, and / or a specific state of a software application executed on an electronic device).

[0053] During the selection period, in the multi-user mode (multi-user selection state 430), when a selection request (multi-user request state 460) (either as a single-user selection request or a multi-user selection request) is performed by a user who does not belong to the selected user group, at least one other user is selected and added to the user group of the control application. Figure 4 and 5 In the illustrated example embodiment, when executed during a selection period, the selection of additional users is not subject to the confirmation (e.g., approval) or rejection of the selected user (or otherwise referred to as a member of the group of selected users). In other embodiments, on the contrary, the selection of additional users may be conditional, depending on the confirmation or rejection of at least one selected user. Depending on the embodiment, the confirmation and / or rejection of at least one selected user may be explicit or implicit. For example, a specific action mode may be requested to confirm (or reject), and the non-occurrence of a specific action mode may be interpreted by the electronic device as a rejection (or confirmation) of at least one selected user. Depending on the embodiment, a specific action mode may be a first action mode, a second action mode, and / or an action mode different from the first and / or second action mode.

[0054] Depending on the embodiment, once the selection period ends (selection state 450), no additional users may be selected, or the selection of additional users may be limited, similar to the description above regarding the unended selection period. Figure 4 and 5Once the selection period ends in multi-user mode, when a selection request performed by another user who does not belong to the group of the selected user is detected (multi-user request state 460), the processing of the request can be postponed during a given time period. If a specific action mode corresponding to a rejection is detected from a user in the group of users controlling the application during a given time period (the first and / or second mode in the example embodiment), then the selection request of the other user can be ignored. If no other specific action mode (e.g., regarded as representing a rejection) is detected from the group of users that have already controlled the application during a given time period, then the selection request of the other user can be considered. For example, if the selection request of the other user is a single-user selection request (one user request state 440) (a first mode is detected), then the method may include configuring the electronic device in single-user mode (one user selection state 420), and the other user is selected as a single user controlling the application in single-user mode. If the selection request of the other user is a multi-user selection request (a second mode is detected), then the method may include iterating the formation of the group of users for controlling the application, and the (one or more) users who previously controlled the application and were previously part of the user group are removed from the group (in other words, deselected). More precisely, a new user group may be formed to control the application, which new group includes the "additional" users and other members added during a selection period similar to that already described.

[0055] The selection performed in the single-user mode can similarly depend on whether the selection period has ended. However, in the single-user mode (one user selection state 420), once a single user is selected, the selection period can be considered to be over (selection 450). Therefore, when a single-user selection request is detected from a user other than the user currently selected for controlling the device (one user request state 440), the processing of the selection request can be postponed during a given time period (which may be the same or different time period compared to the postponed time period of the multi-user mode introduced above). This time period can be obtained, for example, via the configuration file of the application (and can be selected by the provider of the application or one of its users). For example, similar to what has been described about the multi-user mode, the selection request of other users can be ignored according to the confirmation or rejection of the single user that has been selected within a given time period. For example, if another specific action mode (e.g., regarded as a representative of rejection) originating from the single user that has been selected for controlling the application is detected, then the selection request of another user can be ignored. If no other action mode from the selected single user is detected during this time period, then the device can stay in the single-user mode, and another user at the source of the selection request is newly selected as the single user of the control application.

[0056] In some embodiments, if one of the currently selected (one or more) users performs a selection request when another selection request without another user action is processed or postponed, the selection request can be ignored. In other embodiments, once a user has been selected, the first and / or second modes can be used as other input elements. For example, the first and / or second modes can be used as confirmation or rejection that the selected user is deselected, or a member of the selected user group requests to deselect all users currently selected for controlling the application. In a variant, the third mode can be associated with an idle (or "waiting for control request") mode. In such an embodiment, the method may include configuring the device to an "idle" mode (therefore being regarded as a request to deselect the currently selected (one or more) users) when an action corresponding to the third mode is detected. When there is no user in the area sensed by the sensor, the application can also automatically return to an idle or "waiting for control" mode.

[0057] It should be noted that the method may also include applying some rules or outputting some recommendations to avoid (or solve) some problems. For example, if several selection requests are detected at the same time, all selection requests may be ignored, a visual and / or audio message may be output on the user interface, or only one selection request (e.g., selected according to the user's priority, or by default or randomly) may be accepted.

[0058] As shown above, at least some embodiments of the present disclosure may therefore help one or several users interact together more conveniently, as compared to some prior art solutions, such as when selecting (one or more) users to control an electronic device (and / or when requested to assign control to another user other than the selected user).

[0059] The embodiments detailed above include tracking one or more users (e.g., a user near the electronic device, such as a user facing the display (in embodiments involving a display for AR and / or VR scenarios), or a user in front of a television (when the electronic device 200 is a television)).

[0060] Depending on the embodiment, tracking can be performed via different tracking components. For example, the tracking component of the electronic device 200 may include an acquisition module 242, which includes at least one wearable sensor (such as clothing or equipment) suitable for providing its position (or geolocation) to the electronic device, or at least one camera, such as a color and / or depth camera suitable for at least partially reconstructing a human skeleton.

[0061] like Figure 5 As shown in , tracking can provide, for example, a list of detected users, each detected user associated with a unique identifier.

[0062] In some embodiments, for example, in an embodiment where the user automatically deselects (untracked state 510) when leaving a scene covered by a tracking device, different unique identifiers may be assigned to the user in time. It is noteworthy that the unique identifier assigned to the user may change when leaving and then reentering a scene covered by a tracking device. In other embodiments, the user may be assigned the same identifier (also referred to as the user's signature), particularly when leaving and then reentering the scene. This identifier or signature may be associated with, for example, at least one sensor worn by the user. The signature may also be based on at least a portion of the user's skeleton (e.g., a joint of a skeletal portion for calculating a signature) or the user's face (e.g., using facial recognition technology). Such a signature may be obtained, for example, via dedicated software and / or a hardware module coupled to an electronic device.

[0063] In some embodiments, tracking can include, when a new user is detected during tracking (tracked state 520), comparing its signature with user signatures known to the tracking system (e.g., the signature is part of the information already stored about some users). If the signature already exists, then the stored user information will be updated with the data provided by tracking. Otherwise, the new user's information is created.

[0064] In order to distinguish users, the method may include storing and accessing information about at least one user detected by (one or more) sensors, such as a selected user (selected state 540) and a tracked user (tracked state 520) in a storage module (such as a database) of the electronic device.

[0065] Information about a user can include:

[0066] an identifier of the user (e.g. an identifier assigned by tracking, thanks to e.g. a user tracking module, or given by a signature module) and uniquely identifying the user (e.g. an identifier valid only while the user is being tracked, or an identifier that remains constant across several tracking sessions with at least one interruption);

[0067] A dataset, given by the user tracker module and containing joint positions of at least a part of the user's skeleton. It is updated at every frame or at another frequency depending on the tracker.

[0068] Selection and / or tracking indicators, representing the user's current selection state (e.g. Figure 4 and 5 shown).

[0069] Tracking may also include analyzing the body movements of the user(s) in order to identify gesture patterns known to the tracking system (such as the first, second and / or third patterns that have been introduced). The analysis may include analyzing the user's gestures and / or words via, for example, a natural interaction module and / or analyzing the gestures of the tracked user or a part of the user (such as the position of his hands and / or head) via, for example, a gesture detection module. For example, in Figure 3 , 4 In the example embodiments of and 5, the gesture detection module can be adapted to detect 2 or 3 gestures: one arm raised (can be the left hand and / or the right hand), above the head, and two arms raised. In some embodiments, the method can require the gesture to be maintained for a period of time, such as a few seconds (e.g., 2, 3, 4 seconds) for verification.

[0070] Depending on the embodiment, and for example depending on whether the detected user is the selected user, the user's entire anatomy may be tracked, or tracking may be limited to only a portion of the user's anatomy, such as a portion of the user's skeleton. Such embodiments may therefore help save processing costs involved in tracking by the tracking system. For example, in embodiments where the tracking device(s) include a camera, tracking may be of a reduced viewing area around the detected user (tracked state 520) and / or the selected user (selected state 540), (the entire scene is tracked only when no user is selected) or around a portion of a user (such as a user's hand). Figure 3 , 4 In the exemplary embodiments of and 5, tracking can be limited to the 3D position of the user's arms or the user's hands, or can be performed only for the 3D position of the user's hands and the user's head. It should be noted that the actions of unselected users are tracked only for detecting a selection request (selection state 530 (and therefore for detecting some specific action mode)), while the actions of selected users can be tracked to detect input elements for controlling 640 the electronic device. Therefore, different action modes can be tracked for selected users than for users that have not yet been selected. Therefore, tracking can be based on the user's "selection state" (reference Figure 5 ) are different.

[0071] In some embodiments, all information about users whose tracked components are lost (or in other words, no longer detected by the sensor) can be deleted from the database. In other embodiments, at least a portion of the information about users whose tracked components are lost can be at least temporarily retained by the tracking system. For example, in some embodiments, in some cases, the user may be considered likely to be detected again soon, so its information is kept by the tracking system for a non-empty duration (e.g., a duration of less than 5, 10, or 20 minutes). If the user was selected before being lost, it will remain in this state when it comes back. In an embodiment where (one or more) tracking devices include an RGBD camera, if the user disappears and the last position of the tracked portion of its skeleton is close to the center of the camera's field of view, for example, it can be assumed that the user will reappear soon because the camera may have lost the user due to overlap problems. Moreover, the tracking of the user (control of the electronic device) can be interfered with by the presence of other users moving near and around him / her, resulting in full or partial occlusion of the tracked user.

[0072] The present disclosure is described with respect to example sensors, such as wearable sensor(s), cameras, microphone(s), gyroscope(s), accelerometer(s), or GPS modules(s). Of course, as will be appreciated by those skilled in the art, the present disclosure may also be applied to tracking systems using other tracking devices other than those referenced in the present disclosure.

[0073] As one skilled in the art will appreciate, aspects of the present principles may be embodied as a system, apparatus, method, signal or computer-readable product or medium.

[0074] For example, the present disclosure relates to a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method comprising:

[0075] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0076] - determining at least one input element suitable for controlling the electronic device based on the tracked motion.

[0077] According to at least one embodiment of the present disclosure, the method includes changing a rendering of a virtual scene output on a rendering device coupled to the electronic device according to the determined input element.

[0078] According to at least one embodiment of the present disclosure, by considering the similarity criteria between the first action mode associated with the single-user mode and the second action mode associated with the multi-user mode, respectively, the selection request is identified as a request to configure the electronic device in the single-user mode or is identified as a request to configure the electronic device in the multi-user mode.

[0079] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request of a candidate user when the electronic device is configured in the single-user mode, the first user is currently selected as the single tracked user, selecting the candidate user to be tracked instead of the first user, and the selection is performed based on a confirmation of the first user.

[0080] According to at least one embodiment of the present disclosure, the method includes, during a selection period, after receiving a selection request from a candidate user to configure the electronic device in the single-user mode, when the electronic device has been configured in the multi-user mode, configuring the electronic device in the single-user mode and selecting the candidate user as the single tracked user.

[0081] According to at least one embodiment of the present disclosure, the selection period ends after a limited period of time.

[0082] According to at least one embodiment of the present disclosure, the method includes, during the selection period, after receiving a selection request from a candidate user to configure the electronic device in the multi-user mode, when the electronic device has been configured in the multi-user mode, adding the candidate user to the group requested by the tracked user.

[0083] According to at least one embodiment of the present disclosure, the method includes ending the selection period after capturing at least one end command of at least one first user of the group of tracked users.

[0084] According to at least one embodiment of the present disclosure, the command is identified by considering the occurrence of a third motion pattern in the tracked motion of the first user.

[0085] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request of a candidate user for configuring the electronic device in the multi-user mode, when the electronic device is configured in the multi-user mode and the selection period has ended, adding the candidate user to the selected tracked users according to a confirmation of a first user in the group of tracked users.

[0086] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user for configuring the electronic device to the single-user mode, when the electronic device is configured in the multi-user mode and the selection period has ended, configuring the electronic device in the single-user mode and selecting the candidate user as the single tracked user according to a confirmation of a first user in the group of tracked users.

[0087] According to at least one embodiment of the present disclosure, the confirmation is identified by considering the occurrence of a fourth motion pattern in the tracked motion of the first user.

[0088] In accordance with at least one embodiment of the present disclosure, the method includes determining a signature that uniquely identifies one of the tracked users.

[0089] Moreover, the present disclosure relates to a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, the current mode of the electronic device and at least one tracked user being selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0090] According to at least one embodiment of the present disclosure, the method may include changing the rendering of a virtual scene output on a rendering device coupled to the electronic device according to the obtained at least one input element.

[0091] According to at least one embodiment of the present disclosure, by considering the similarity criteria between the first action mode associated with the single-user mode and the second action mode associated with the multi-user mode, respectively, the selection request is identified as a request to configure the electronic device in the single-user mode or is identified as a request to configure the electronic device in the multi-user mode.

[0092] According to at least one embodiment of the present disclosure, the method includes, when the electronic device is in the single-user mode, ending the selection period after selecting a first user for controlling the electronic device.

[0093] According to at least one embodiment of the present disclosure, the method includes, when the electronic device is configured in the single-user mode and is controlled by tracking a first user, upon receiving a selection request of a candidate user, selecting the candidate user to be tracked for controlling the electronic device instead of the first user, the selection being performed based on confirmation and / or rejection of the first user.

[0094] According to at least one embodiment of the present disclosure, the method includes ending the selection period after capturing at least one end command of at least one selected user.

[0095] According to at least one embodiment of the present disclosure, the command is identified by considering the occurrence of a third motion pattern in the tracked motion of the first user.

[0096] According to at least one embodiment of the present disclosure, the method includes, when the electronic device has been configured in the multi-user mode, upon receiving a selection request of a candidate user during the selection period, adding the candidate user to the group of tracked users based on confirmation and / or rejection of a first user in the group of tracked users.

[0097] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user for configuring the electronic device in the single-user mode, when the selection period has ended, configuring the electronic device in the single-user mode, the candidate user being a single user who controls the electronic device.

[0098] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user for configuring the electronic device in the multi-user mode, when the selection period has ended, configuring the electronic device in the multi-user mode, the electronic device is controlled by a new group of tracked users, including the candidate user, and starting a new selection period for adding other users to the new group.

[0099] According to at least one embodiment of the present disclosure, the configuring is performed after obtaining confirmation that at least one of the tracked users currently controls the electronic device.

[0100] According to at least one embodiment of the present disclosure, the confirmation and / or rejection is identified by considering the occurrence of a fourth motion pattern in the tracked motion of the first user.

[0101] In accordance with at least one embodiment of the present disclosure, the method includes determining a capturable signature that uniquely identifies one of the tracked users.

[0102] According to at least one embodiment of the present disclosure, the first, second, third and / or fourth motion modes belong to the group consisting of:

[0103] - at least one sound sequence;

[0104] - at least one movement of at least one physical object;

[0105] - at least one movement of the human body;

[0106] - at least one posture of the human body;

[0107] - at least one pose of the physical object;

[0108] - at least one orientation of the physical object;

[0109] - combinations thereof.

[0110] According to at least one embodiment of the present disclosure, the at least one sensor belongs to the group consisting of:

[0111] - Colorimetric camera;

[0112] - Depth camera;

[0113] -microphone;

[0114] - Gyroscope;

[0115] - accelerometer;

[0116] -GPS module.

[0117] According to another aspect, the present disclosure relates to an electronic device comprising at least one processor or at least one processing circuit, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the electronic device comprising at least one processor adapted to:

[0118] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0119] - determining at least one input element suitable for controlling the electronic device based on the tracked motion.

[0120] The present disclosure also relates to an electronic device adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, the electronic device comprising at least one processor adapted to:

[0121] A current mode of the electronic device and at least one tracked user is selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0122] Although not explicitly described, the above-mentioned electronic device of the present disclosure may be suitable for executing any embodiment of the above-mentioned method of the present disclosure.

[0123] According to another aspect, the present disclosure relates to a communication system comprising one or more electronic devices adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the one or more electronic devices comprising at least one processor or at least one processing circuit adapted to:

[0124] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0125] - determining at least one input element suitable for controlling the electronic device based on the tracked motion.

[0126] The present disclosure also relates to a communication system, the communication system comprising one or more electronic devices adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, the one or more electronic devices comprising at least one processor or at least one processing circuit adapted to:

[0127] A current mode of the electronic device and at least one tracked user is selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0128] Although not explicitly described, the present embodiments related to methods or to corresponding electronic devices or communication systems may be employed in any combination or sub-combination.

[0129] According to another aspect, the present disclosure relates to a non-transitory computer-readable program product comprising program code instructions which, when the non-transitory computer-readable program product is executed by a computer, perform any embodiment of the method of the present disclosure.

[0130] For example, at least one embodiment of the present disclosure is directed to a non-transitory computer-readable program product including program code instructions, which, when executed by a computer, execute a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method comprising:

[0131] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0132] - determining at least one input element suitable for controlling the electronic device based on the tracked motion.

[0133] At least one embodiment of the present disclosure is directed to a non-transitory computer-readable program product including program code instructions, which, when executed by a computer, perform a method implemented in an electronic device adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the current mode of the electronic device and at least one tracked user being selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0134] According to another aspect, the present disclosure relates to a computer-readable storage medium carrying a non-transitory software program, the software program comprising program code instructions that, when the non-transitory software program is executed by a computer, perform any embodiment of the method of the present disclosure.

[0135] According to an embodiment of the present disclosure, a computer-readable storage medium tangibly implements a non-transitory software program, which includes program code instructions executable by a computer. When the non-transitory software program is executed by a computer, the program code instructions perform a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method comprising:

[0136] - tracking the movements of at least one user, the at least one tracked user being selected among candidate users according to a current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0137] - determining at least one input element suitable for controlling the electronic device based on the tracked motion.

[0138] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium carries a non-transitory software program including program code instructions executable by a computer. When the software program is executed by a computer, the program code instructions execute a method implemented in an electronic device, the electronic device being suitable for being configured in a single-user mode in which the electronic device is controlled by an input element obtained by tracking a single user, and in a multi-user mode in which the electronic device is controlled by an input element obtained by tracking a group or one or more users, wherein a current mode of the electronic device and at least one tracked user is selected based on at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0139] As will be appreciated by those skilled in the art, various aspects of the present principles may be implemented as systems, methods, or computer-readable media. Thus, various aspects of the present disclosure may take the form of hardware embodiments, software embodiments (including firmware, resident software, microcode, etc.), or embodiments of combined software and hardware aspects, which may generally be referred to herein as "circuits," "modules," or "systems." In addition, various aspects of the present principles may take the form of computer-readable storage media. Any combination of one or more computer-readable storage media may be utilized.

[0140] Computer-readable storage media may take the form of a computer-readable program product that is implemented in one or more computer-readable media and has computer-readable program code implemented thereon that can be executed by a computer. Given the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, computer-readable storage media as used herein are considered to be non-transitory storage media. Computer-readable storage media may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination of the foregoing.

[0141] It should be recognized that although the following provides more specific examples of computer-readable storage media to which the present principles can be applied, as one of ordinary skill in the art will readily recognize, this is merely an illustrative and non-exhaustive list: portable computer floppy disks, hard disks, read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] Thus, for example, those skilled in the art will recognize that the block diagrams presented herein represent conceptual views of illustrative system components and / or circuits of some embodiments of the present principles. Similarly, it will be recognized that any flow charts, flow diagrams, state transition diagrams, pseudo-codes, etc. represent various processes that can be substantially represented in a computer-readable storage medium and thus executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0143] It is also pointed out that at least some of the embodiments may be combined together even if not explicitly disclosed above.

[0144] The present disclosure is not merely an abstract concept of identifying users and assigning users to certain modes that can be accomplished by the human mind. Instead, the present disclosure includes a practical application of transforming a sequence of sounds or a spatial relationship / orientation of an object (such as a part of a human body) into an operating mode of a machine and / or a request to control the machine, selecting a user to be tracked by at least one sensor coupled to the machine, and controlling the machine according to an input element determined from the actions of the tracked user. For example, the practical application can modify the rendering of a virtual scene by a rendering device according to the determined input element.

Claims

1. An electronic device adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by captured physical movements of at least one tracked input element obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by captured physical movements of tracked input elements obtained by tracking two or more users manipulating different tracked input elements, wherein user tracking is performed using one or more sensing devices coupled to the electronic device, Wherein the tracked input element corresponds to a tracked physical object or a part of a user's body, the electronic device comprises at least one processor adapted to: selecting at least one tracked candidate user and a mode of the electronic device to be in the single-user mode or the multi-user mode based on at least one selection request captured by the one or more sensing devices as performed by the at least one tracked candidate user, the electronic device being characterized in that the selection request of the candidate user is captured during a selection period of limited duration; and Features The at least one processor is further adapted to, when the mode of the electronic device is the multi-user mode and upon receiving a selection request of a candidate user during the selection period, add the candidate user to the group of tracked users according to a confirmation and / or rejection of a first user of the group of tracked users.

2. The electronic device of claim 1, wherein the at least one processor is adapted to modify a rendering of a virtual scene output on a rendering device coupled to the electronic device based on the obtained at least one tracked input element.

3. An electronic device as claimed in claim 1, wherein the selection request is identified as a request to configure the electronic device in the single-user mode or as a request to configure the electronic device in the multi-user mode by considering a similarity criterion between a first action mode associated with the single-user mode and a second action mode associated with the multi-user mode, respectively. 4 . The electronic device of claim 1 , wherein the at least one processor is adapted to, when the mode of the electronic device is the single-user mode, end the selection period after selecting a first user for controlling the electronic device.

5. The electronic device of claim 1, the at least one processor being adapted to end the selection period upon capturing at least one end command of at least one selected user. 6 . The electronic device of claim 5 , wherein the end command is identified by considering the occurrence of a third motion pattern in the tracked motion of the first user.

7. The electronic device of claim 1, wherein the at least one processor is adapted to, upon receiving a selection request from a candidate user for configuring the electronic device in the single-user mode, configure the electronic device in the single-user mode when the selection period has ended, wherein the candidate user is the single user controlling the electronic device.

8. An electronic device as described in claim 1, wherein the at least one processor is adapted to: after receiving a selection request from a candidate user for configuring the electronic device in the multi-user mode, when the selection period has ended, configure the electronic device in the multi-user mode, the electronic device is controlled by a new group of tracked users, including the candidate user, and start a new selection period for adding other users to the new group.

9. The electronic device of claim 8, wherein the configuring is performed after obtaining confirmation that at least one of the tracked users currently controls the electronic device.

10. The electronic device of claim 5, wherein the confirmation and / or rejection is identified by considering the occurrence of a fourth motion pattern in the tracked motion of the first user.

11. The electronic device of claim 1, the at least one processor adapted to determine a capturable signature that uniquely identifies one of the tracked users.

12. A method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by captured physical movements of at least one tracked input element obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by captured physical movements of tracked input elements obtained by tracking two or more users manipulating different tracked input elements, wherein user tracking is performed using one or more sensing devices coupled to the electronic device, wherein the tracked input element corresponds to a tracked physical object or part of a user's body, The method comprises: selecting at least one tracked candidate user and a mode of the electronic device to be in the single-user mode or the multi-user mode based on at least one selection request captured by the one or more sensing devices as performed by the at least one tracked candidate user, The method is characterized in that a selection request of a candidate user is captured during a selection period of limited duration; and Features The method further includes, when the mode of the electronic device is the multi-user mode and upon receiving a selection request of a candidate user during the selection period, adding the candidate user to the group of tracked users according to confirmation and / or rejection of a first user of the group of tracked users.

Images