Electronic device and method executed in electronic device

Abstract

The invention relates to an electronic device and a method executed in the electronic device. An electronic device: wherein the electronic device is adapted to be controlled, at least in part, by a capture action of a tracked input element obtained by tracking a group of tracked users comprising two or more users using one or more sensing devices coupled to the electronic device, an electronic device includes a tracked input element corresponding to a tracked physical object or a portion of a user's body or audio input from a user, the electronic device including at least one processor configured to receive a selection request of a tracked candidate user during a selection period of a limited duration, the tracked candidate user is added to the group of tracked users in accordance with an acknowledgement of a first user of the group of tracked users, the selection period ending when a user of the group of tracked users captures a given action, the selection request being performed by the tracked candidate user and captured by the one or more sensing devices.

Description

[0001] This application is a divisional application of the invention patent application with the application number 202080028796.5, the application date of March 20, 2020, and the invention title of "Method and Apparatus for User Control of Applications and Corresponding Devices". Technical Field

[0002] The present disclosure relates to the technical field of user control of electronic devices suitable for use in a multi-user environment. Corresponding applications, devices, electronic components, systems, computer-readable program products, and computer-readable storage media are described. Background Art

[0003] Many electronic devices can be coupled to user tracking components to obtain information about the user, thereby obtaining real-time information. In the present disclosure, the user tracking components include, for example, sensors (wearable or non-wearable) that obtain information about the shape, posture, position, and / or movement of the user or one or more parts of their body, such as a depth camera or a chromaticity camera. Based on the information obtained by the sensors, some of those electronic devices can also be at least partially controlled by the user(s). In fact, the tracking mechanism helps to enable a natural interaction experience from the user's perspective. For example, in the field of virtual reality (VR) or augmented reality (AR), tracking the user's movement can allow the user to manipulate virtual objects in a virtual scene or navigate in a 3D scene without using an auxiliary device (such as a remote control device).

[0004] However, user tracking in a multi-user environment (such as a collaborative game or a collaborative tool) can cause some problems because the sensors on which the tracking depends can concurrently obtain information about multiple users.

[0005] Some solutions propose to only consider the actions performed simultaneously by all users, or only track the joint actions of the users (such as the hands of two users overlapping in a 2-user environment). However, all of these solutions do not allow users to conveniently interact with applications in a multi-user environment.

[0006] Therefore, a solution is needed to enhance the user experience of applications that are at least partially controlled by user tracking. Summary of the Invention

[0007] Embodiments of the present invention may include an electronic device: wherein, the electronic device is adapted to be controlled at least in part by a capture action of a tracked input element obtained by tracking a group of tracked users including two or more users by using one or more sensing devices coupled to the electronic device, wherein, the tracked input element corresponds to a tracked physical object or a part of a user's body or an audio input from the user, the electronic device includes at least one processor, and the at least one processor is configured to: when a selection request of a tracked candidate user is received during a selection period of a finite duration, add the tracked candidate user to the group of tracked users according to the confirmation of a first user in the group of tracked users, the selection period ends when a user in the group of tracked users captures a given action, and the selection request is executed by the tracked candidate user and captured by the one or more sensing devices.

[0008] Another embodiment of the present invention may include a method executed in an electronic device: wherein, the electronic device is adapted to be controlled at least in part by a capture action of a tracked input element obtained by tracking a group of tracked users including two or more users by using one or more sensing devices coupled to the electronic device, wherein, the tracked input element corresponds to a tracked physical object or a part of a user's body or an audio input from the user, the electronic device includes at least one processor, and the at least one processor is configured to: when a selection request of a tracked candidate user is received during a selection period of a finite duration, add the tracked candidate user to the group of tracked users according to the confirmation of a first user in the group of tracked users, the selection period ends when a user in the group of tracked users captures a given action, and the selection request is executed by the tracked candidate user and captured by the one or more sensing devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The present disclosure may be better understood after reading the following description, and other specific features and advantages may occur, and this description refers to the accompanying drawings, wherein:

[0010] Figure 1 An example of a tracking system according to an exemplary embodiment of the present disclosure is shown;

[0011] Figure 2 An example functional structure of an electronic device suitable for implementing at least some embodiments of the method of the present disclosure is illustrated;

[0012] Figure 3 An example overview of the interaction between a user and a tracking system in at least some embodiments of the present disclosure is depicted;

[0013] Figure 4Illustrates a state machine of an example selection mechanism that can be used when implementing some embodiments of the method of the present disclosure;

[0014] Figure 5 Illustrates a state machine of an example user state diagram that can be used when implementing some embodiments of the method of the present disclosure;

[0015] Figure 6 Depicts an overview of some example embodiments of the method of the present disclosure.

[0016] It should be noted that the drawings illustrate example embodiments and the embodiments of the present disclosure are not limited to the illustrated embodiments. For example, it will be apparent to those skilled in the art that the embodiments of the present disclosure can also be implemented in a more complex tracking system than Figures 1 to 6 that shown herein. Detailed Description

[0017] The present disclosure relates to the technical field of electronic devices or corresponding applications or methods, and is adapted to be at least partially controlled by at least one physical action of one or more users (such as concurrent users), the at least one physical action being captured by one or more sensing devices (or sensors) coupled to the electronic device and triggering the processing of the electronic device. For example, such physical actions can be a gesture or a sequence of gestures, a sound or a sequence of sounds, or a combination thereof, generated by one or more users.

[0018] Different from the prior art solutions, at least some embodiments of the present disclosure can allow selection between a multi-user mode and a single-user mode and dynamically select the user who will interact with the system and thus will be tracked.

[0019] The term "processing" will be understood herein in its broader scope and encompasses one or more different operations, and for example, modifying the rendering of content on a display device (such as the display of a television set) coupled to the electronic device. The rendering of the content can vary according to the nature of the content and can include, for example, displaying the visual part of the content or playing the audio part of the content. "Visual part" should be understood as a video, one or more sequences of images and / or one or more still images, or a combination thereof.

[0020] In an example use case where the electronic device is adapted to process VR or AR content including at least one virtual object or scene, the processing triggered by the user's action can, for example, initiate and / or modify the rendering of such virtual object or scene.

[0021] In at least one embodiment of the present disclosure, the electronic device can be configured in a "single-user" (or "single user") mode where only the actions of a single user are tracked, or a "multi-user" (or "multiple users") mode where the actions of multiple users can be tracked (for example, where several actions of several users can be tracked simultaneously).

[0022] In single-user mode, only one person can control the interaction at a time. For example, in some embodiments, once a user is selected to control the application, other users cannot interfere. Such embodiments can, for example, allow a single user to navigate in a 3D scene and / or thus change the rendering of the scene, and (one or more) others located near the selected user are only allowed to view the 3D scene without changing it.

[0023] In multi-user mode, two or more users can interact with the electronic device simultaneously. For example, such embodiments can be suitable for use cases where the electronic device is executing a collaborative application (such as a game, for example, a 3D puzzle where players are considered a team and must collaborate together to achieve a common goal (such as putting elements together to form an object or drawing a shared object or scene)). Multi-user mode can also be suitable for situations where multiple users can act independently (or almost independently), such as when different users operate different objects in the same VR or AR scene.

[0024] Figure 1 The tracking system 100 is illustrated, which includes one or more electronic devices suitable for implementing at least one embodiment of the present disclosure.

[0025] In Figure 1 embodiments, the tracking system 100 is located in a home environment. Due to a wired communication interface (such as an Ethernet interface) or a wireless communication interface (such as a WIFI® or Bluetooth® interface), some electronic devices of the system can exchange data within a communication network (for example, a local area network (LAN)).

[0026] The tracking system can, for example, include one or several gateways 110 that provide access to at least one other communication network (such as a wide area network (WAN)). For example, the gateway (GTW) 110 can provide access to a broadcast network or a broadband network (such as the Internet). For example, the GTW 110 can receive data from a broadcast network or receive and / or transmit data to / from a broadband network. Thus, at least some electronic devices of the system 100 can receive and / or transmit data from / to those other communication networks via the gateway 110, such as audiovisual data.

[0027] The tracking system 100 can also include at least one other electronic device suitable for processing (and for example, rendering) audio and / or visual signals. Such electronic devices can include a set-top box (STB) 140, a tablet 120, a PC, a smart phone, a Blu-ray player, a television 160, an audio speaker 150, and / or a display 130. All of these exemplary devices can be connected to the GTW 110 wirelessly or through a wired connection.

[0028] Figure 2Describes an example functional structure of an electronic device 200 that can implement an application. The electronic device 200 can be adapted, for example, in at least some of its embodiments to implement the method 600 of the present disclosure, which will be described later. The electronic device can be, for example, one of the exemplary electronic devices described above in connection with Figure 1 It should be noted that Figure 2 The elements, modules, or blocks of the functional structure shown in can be implemented by software components of the electronic device 200 and / or hardware components of the electronic device 200. Figure 2 At least some of the elements shown in can include at least one software component and at least one hardware component of the processing device 200. Depending on the embodiment, several of these elements can be part of the same physical device or can belong to different, separate, physical devices.

[0029] In Figure 2 In an example embodiment, the electronic device 200 can include different devices (or modules) that are connected together via a communication bus 20 (such as a control, data, and address bus, which can also carry timer signals). For example, it can include one or several microprocessors 21 (or central processing units (CPUs)), ROM (or "read-only memory") 25, RAM (or "random access memory") 26, and one or more wired and / or wireless communication interfaces 27, 28 that provide access to a LAN and / or WAN network. For example, at least one of the communication interfaces 27, 28 of the electronic device 200 can be adapted to receive one or more audio and / or video contents or streams.

[0030] In Figure 2 In an example embodiment, the electronic device 200 can further include at least one input / output (I / O) module 24 (such as a tactile display, switch, LED, button, etc.). For example, the electronic device can include (or more generally be coupled to) at least one acquisition module 242 adapted to detect a user's body movements, such as audio and / or video acquisition devices, such as cameras, microphones, and / or position and / or motion tracking devices (such as GPS modules, rotary mice, gyroscopes, accelerometers).

[0031] In Figure 2 In an example embodiment, one or more acquisition modules are adapted to detect a user's posture and detect a user's movement, such as hand movement.

[0032] In the illustrated embodiment, the electronic device can include (or more generally be coupled to) a content rendering module 244 adapted to render audio and / or visual content in one or more formats, such as a television or a tablet computer. The content rendering module can include a display and / or speakers. In some embodiments, this content rendering unit can be optional.

[0033] Each of the memories mentioned may include at least one register, i.e., a memory area of low capacity (a small amount of binary data) or high capacity (capable of storing an entire program or all or part of the data representing the data to be calculated or displayed).

[0034] When the electronic device 200 is powered on, via the power module 22, such as a power module connected to a power line network, at least one microprocessor 21 loads the program instructions 260 into the registers of the RAM 26, for example, performs the processes required for at least one embodiment of the method 600 described below and executes the program instructions.

[0035] In any embodiment of the present disclosure, the electronic device 200 of the system may be adapted to implement the method 600 of the present disclosure. In Figure 2 the example embodiment shown, the electronic device is adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, and the microprocessor 21 may be configured for:

[0036] - Select the current mode of the electronic device and at least one tracked user according to at least one selection request, the selection request of the candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0037] In some embodiments, the microprocessor 21 may be configured for:

[0038] - Track the body movements of at least one user, at least one tracked user being selected among the candidate users according to the current mode of the electronic device and according to the selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0039] - Determine at least one input element suitable for controlling the electronic device according to the tracked physical movements.

[0040] In combination with Figure 2 The structures depicted are for illustrative purposes only. It should be noted that even though shown as being provided by separate devices, the electronic device 200 may also include additional capabilities. Furthermore, at least some embodiments of the present disclosure may be implemented at least partially in at least one electronic device externally coupled to another device.

[0041] Example embodiments of the method 600 of the present disclosure are described in detail below with respect to Figure 4 、 Figure 5 and Figure 6 are elaborated. Figure 6 shows an overview of an example embodiment of the method 600,Figure 4 illustrates a state machine suitable for use by a method for selecting one or more users authorized to control an electronic device, and Figure 5 illustrates the states of a user while staying in the sensing area of a tracking system. In the illustrated example embodiment, the method is implemented in a software application executed in a device 200 of Figure 2 . The software application may be started when the electronic device 200 is powered on or at a later stage.

[0042] In a specific embodiment, the electronic device may be configured in a single-user mode or a multi-user mode. The single-user mode and the multi-user mode are respectively associated with first and second action modes, and the first and second action modes respectively correspond to one or more actions that can be captured by at least one sensor of the tracking system. The detections of the first and second action modes are respectively interpreted as a user request being selected to control the electronic device and configuring the device in the corresponding mode. The corresponding detected user actions will hereinafter be referred to as "selection requests" (more precisely, depending on the action mode, a single-user selection request or a multi-user selection request).

[0043] In Figure 4 and Figure 5 's example embodiment, the first and second action modes are registered gesture modes. For example, the first action mode corresponds to "raising one hand", while the second action mode corresponds to "raising both hands", and the detected action mode is the "arm raising" posture.

[0044] At the start of the application, or when there is no user in the acquisition area of the sensor, the device is not yet controlled by a given user and is thus configured in a default mode. In some embodiments, the default mode may be a single-user mode or a multi-user mode (e.g., selecting (one or more) default users to control the electronic device, such as the first user detected by the tracking module of the electronic device once the application is started, or the user who is currently the closest to the electronic device). In other embodiments, the default mode may be another mode, such as Figure 4 's "idle" (or "waiting for a control request") mode (idle state 410) as shown. In Figure 4 and Figure 6 's example embodiment, the electronic device remains in the "idle" mode until an action corresponding to the first and / or second action mode is detected. When the first action mode 610 is detected while the device is configured in the "idle" mode, the method includes configuring 620 the electronic device in the "single-user mode (one user selection state 420)". The user at the origin of the detected action is selected 630 as the single user of the control application, and the method includes tracking the actions performed by the selected user in order to detect events to be used as input elements for controlling 640 the device.

[0045] Similarly, when a second mode of operation 612 is detected while the device is configured in an "idle" mode (idle state 410), the method includes configuring the electronic device in a "multi-user mode" (multi-user selection state 430). As Figure 6 shown, the method includes constituting a group of users authorized to control the device by selecting 630 users. More precisely, the user who is the origin of the detected action corresponding to the first or second mode of operation, respectively, is selected to belong to the group when configuring the device in single-user or multi-user mode.

[0046] In multi-user mode, other users can be added to the group at a later time. In Figure 4 and Figure 6 the embodiment shown, the selection 630 is performed during a limited time period. For example, the selection period can start when the device is configured in multi-user mode (multi-user selection state 430). Depending on the embodiment, the selection period can be a period of a given duration, such as a constant duration, e.g., when a timer initiated at the start of the selection period reaches its end (or zero value) by a certain amount of time (or in other words, when the amount of time set on the timer at the start of the selection period has elapsed), or the selection period can be a variable-duration period ended by a specific operation on the device (such as a specific action of a user ((e.g., the selected user or a user acting as a supervisor))), or when a given number of users are currently selected (such as the maximum number of users selected simultaneously). For example, the specific action can correspond to a given action pattern known to the electronic device. Depending on the embodiment, the given action pattern can be the first mode of operation, the second mode of operation, and / or an action pattern different from the first and / or second mode of operation. Similarly, the selection period can start from a specific event (such as the selection of the first user, or a command from a supervisor user, or an automatic event triggered by a timer, a specific state of the electronic device, and / or a specific state of a software application executed on the electronic device).

[0047] During the selection period, in multi-user mode (multi-user selection state 430), when a selection request (multi-user request state 460) (either a single-user selection request or a multi-user selection request) is executed by a user who does not belong to the selected group of users, at least one other user is selected and added to the group of users of the control application. In Figure 4 and Figure 5In the illustrated example embodiment, when executed during a selection period, the selection of additional users is not restricted by the confirmation (such as approval) or rejection of the selected user (or otherwise referred to as a member of the group of selected users). In other embodiments, conversely, the selection of additional users may be conditional depending on the confirmation or rejection of at least one selected user. Depending on the embodiment, the confirmation and / or rejection of at least one selected user may be explicit or implicit. For example, a specific action pattern may be requested for confirmation (or rejection), and the non-occurrence of the specific action pattern may be interpreted by the electronic device as a rejection (or confirmation) of at least one selected user. Depending on the embodiment, the specific action pattern may be a first action pattern, a second action pattern, and / or an action pattern different from the first and / or second action patterns.

[0048] Depending on the embodiment, once the selection period ends (selection state 450), no other users can be selected, or the selection of additional users may be restricted, similar to the description above for an unended selection period. For example, according to Figure 4 and Figure 5 , once the selection period ends in multi-user mode, when a selection request (multi-user request state 460) executed by an additional user who does not belong to the group of selected users is detected, the processing of the request may be postponed during a given time period. If a user from the group of users of the control application and corresponding to a rejection specific action pattern (the first and / or second pattern in the example embodiment) is detected during the given time period, then the selection request of the additional user may be ignored. If no other specific action pattern (e.g., regarded as representing a rejection) from the group of users of the already controlled application is detected during the given time period, then the selection request of the additional user may be considered. For example, if the selection request of the additional user is a single-user selection request (one user request state 440) (the first pattern is detected), then the method may include configuring the electronic device to single-user mode (one user selection state 420), and the additional user is selected as the single user to control the application in single-user mode. If the selection request of the additional user is a multi-user selection request (the second pattern is detected), then the method may include iterating the composition of the group of users for the control application, and the (one or more) users of the previous controlled application who were previously part of the user group are removed from the group (in other words, deselected). More precisely, a new user group may be formed to control the application, and the new group includes the "additional" user and other members added during a selection period similar to the one already described.

[0049] Selections made in single - user mode can similarly depend on whether the selection period has ended. However, in single - user mode (where one user is selected, state 420), once a single user is selected, the selection period can be considered ended (selection 450). Thus, when a single - user selection request (one - user request state 440) is detected from a user other than the currently selected user for controlling the device, the processing of the selection request can be postponed for a given period of time (which may be the same or different compared to the postponement period for the multi - user mode described above). This period of time can be obtained, for example, via the application's configuration file (and can be selected by the provider of the application or one of its users). For example, similar to what has been described regarding multi - user mode, selection requests from other users can be ignored based on the confirmation or rejection of the single user already selected within the given period of time. For example, if another specific action pattern (e.g., one that is considered a rejection) from the single user already selected for controlling the application is detected, then the selection request from another user can be ignored. If no additional action pattern from the selected single user is detected during this period of time, then the device can remain in single - user mode and the other user at the origin of the selection request is newly selected as the single user controlling the application.

[0050] In some embodiments, if one of the currently selected user(s) makes a selection request while another selection request from a different user is being processed or postponed without additional user action, then the selection request can be ignored. In other embodiments, once a user has been selected, the first and / or second mode can be used as other input elements. For example, the first and / or second mode can be used as confirmation or rejection for deselecting the selected user, or as a request from a member of the selected group of users to deselect all currently selected users for controlling the application. In a variant, a third mode can be associated with an idle (or "waiting for control request") mode. In such an embodiment, the method can include configuring the device to an "idle" mode (and thus considered a request to deselect the currently selected user(s)) when an action corresponding to the third mode is detected. The application can also automatically return to the idle or "waiting for control" mode when no user is detected within the area sensed by the sensor.

[0051] It should be noted that the method can also include applying some rules or outputting some recommendations to avoid (or resolve) some problems. For example, if several selection requests are detected simultaneously, then all selection requests can be ignored, a visual and / or audio message can be output on the user interface, or only one selection request (e.g., selected according to the user's priority, or by default or randomly) can be accepted.

[0052] As shown above, compared to some prior art solutions, at least some embodiments of the present disclosure can thus help one or several users to interact together more conveniently, for example when selecting (one or more) users to control an electronic device (and / or when requested to assign control to another user instead of the selected user).

[0053] The embodiments detailed above include tracking one or more users (e.g., users near the electronic device, such as users facing a display (in embodiments of a display involving AR and / or VR scenarios), or users in front of a television set (when the electronic device 200 is a television set)).

[0054] Depending on the embodiment, the tracking can operate via different tracking components. For example, the tracking component of the electronic device 200 can include an acquisition module 242, which includes at least one wearable sensor (such as clothing or a device) adapted to provide the electronic device with its location (or geolocation), or at least one camera, such as a color and / or depth camera adapted to at least partially reconstruct the human skeleton.

[0055] As Figure 5 shown, the tracking can provide, for example, a list of detected users, each detected user associated with a unique identifier.

[0056] In some embodiments, for example, in embodiments where a user is automatically deselected (untracked state 510) when leaving the scene covered by the tracked device, different unique identifiers can be assigned to the user over time. Notably, the unique identifier assigned to a user can change when leaving and then re-entering the scene covered by the tracking device. In other embodiments, the user can be assigned the same identifier (also referred to as the user's signature), especially when leaving and then re-entering the scene. This identifier or signature can be associated with, for example, at least one sensor worn by the user. The signature can also be based on at least a part of the user's skeleton (e.g., the joints of the skeletal part used to calculate the signature) or the user's face (e.g., using facial recognition technology). Such a signature can be obtained, for example, via a dedicated software and / or hardware module coupled to the electronic device.

[0057] In some embodiments, the tracking can include, when a new user is detected during tracking (tracked state 520), comparing its signature with the signatures of users known to the tracking system (e.g., the signature is part of the information already stored about some users). If the signature already exists, then the stored user information is updated with the data provided by the tracking. Otherwise, information about the new user is created.

[0058] To distinguish users, the method may include storing and accessing in a storage module (such as a database) of an electronic device information about at least one user (such as a selected user (selected state 540) and a tracked user (tracked state 520)) detected by one or more sensors.

[0059] Information about a user may include:

[0060] • An identifier of the user (such as an identifier assigned by tracking, attributed to, for example, a user tracking module, or given by a signature module), which uniquely identifies the user (such as an identifier that is only valid when the user is being tracked, or an identifier that remains unchanged during several tracking sessions with at least one interruption period);

[0061] • A data set, given by a user tracker module and containing joint positions of at least a part of the user's skeleton. It is updated at each frame or at another frequency depending on the tracker.

[0062] • A selection and / or tracking indicator, representing the current selection state of the user (such as Figure 4 and Figure 5 as shown).

[0063] Tracking may also include analyzing the body movements of one or more users to identify pose patterns known to the tracking system (such as the first, second, and / or third patterns that have been introduced). The analysis may include analyzing the user's gestures and / or words via, for example, a natural interaction module and / or analyzing the pose of the tracked user or a part of the user (such as the position of their hand and / or head) via, for example, a pose detection module. For example, in the example embodiments of Figure 3 , Figure 4 and Figure 5 , the pose detection module may be adapted to detect 2 or 3 poses: one arm raised (which can be the left and / or right hand), above the head, and both arms raised. In some embodiments, the method may require the pose to be held for a period of time, such as a few seconds (for example, 2, 3, 4 seconds), for verification.

[0064] Depending on the embodiment, and for example depending on whether the detected user is the selected user, the entire human body of the user may be tracked, or the tracking may be limited to a part of the user's human body, such as a part of the user's skeleton. Such embodiments can thus help save the processing costs involved in the tracking of the tracking system. For example, in embodiments where one or more tracking devices include a camera, the tracking may be a reduced visible area around the detected user (tracked state 520) and / or the selected user (selected state 540) (tracking the entire scene only when no user is selected), or around a part of the user (such as the user's hand). In Figure 3 ,Figure 4 and Figure 5 In exemplary embodiments of Figure 4 and Figure 5 , tracking may be limited to the 3D position of the user's arm or the user's hand, or may be performed only for the 3D positions of the user's hand and the user's head. It should be noted that the actions of unselected users are tracked only for detection selection requests (selection state 530 (and thus for detecting some specific action patterns)), while the actions of selected users may be tracked to detect input elements for controlling the electronic device 640. Thus, different action patterns may be tracked for selected users rather than for unselected users. Thus, tracking may vary according to the user's "selection state" (refer to Figure 5 ).

[0065] In some embodiments, all information about a user whose tracked part is lost (or in other words, is no longer detected by the sensor) may be deleted from the database. In other embodiments, at least a portion of the information about a user whose tracked part is lost may be held by the tracking system at least temporarily. For example, in some embodiments, in some cases, a user may be considered likely to be detected again soon, so its information is held by the tracking system for a non - zero duration (e.g., a duration less than 5, 10, or 20 minutes). If the user was selected before being lost, then it will remain in that state when it comes back. In embodiments where the (one or more) tracking devices include an RGBD camera, if the user disappears and the last position of the tracked part of their skeleton is close to the center of the camera's field of view, for example, it may be assumed that the user will reappear soon because the camera may have lost the user due to an occlusion problem. Also, the tracking of the user (for controlling the electronic device) may be disrupted by the presence of other users moving nearby and around, resulting in the complete or partial occlusion of the tracked user.

[0066] This disclosure is described with respect to example sensors such as (one or more) wearable sensors, cameras, (one or more) microphones, (one or more) gyroscopes, (one or more) accelerometers, or (one or more) global positioning system (GPS) modules. Of course, as will be understood by those skilled in the art, this disclosure may also be applied to tracking systems using other tracking devices than those cited in this disclosure.

[0067] As will be recognized by those skilled in the art, aspects of this principle may be implemented as a system, device, method, signal, or computer - readable product or medium.

[0068] For example, the present disclosure relates to a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method comprising:

[0069] - Tracking the actions of at least one user, the at least one tracked user being selected among candidate users according to the current mode of the electronic device and according to a selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0070] - Determining at least one input element suitable for controlling the electronic device based on the tracked actions.

[0071] According to at least one embodiment of the present disclosure, the method includes changing the rendering of a virtual scene output on a rendering device coupled to the electronic device according to the determined input element.

[0072] According to at least one embodiment of the present disclosure, by separately considering similarity criteria between a first action mode associated with the single-user mode and a second action mode associated with the multi-user mode, the selection request is identified as a request to configure the electronic device in the single-user mode or is identified as a request to configure the electronic device in the multi-user mode.

[0073] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user when the electronic device is configured in the single-user mode, the first user being currently selected as the single tracked user, selecting the candidate user being tracked instead of the first user, the selection being performed based on confirmation by the first user.

[0074] According to at least one embodiment of the present disclosure, the method includes, during a selection period, after receiving a selection request from a candidate user to configure the electronic device in the single-user mode, when the electronic device has been configured in the multi-user mode, configuring the electronic device in the single-user mode and selecting the candidate user as the single tracked user.

[0075] According to at least one embodiment of the present disclosure, the selection period ends after a limited period of time.

[0076] According to at least one embodiment of the present disclosure, the method includes, during the selection period, after receiving a selection request from a candidate user to configure the electronic device in the multi-user mode, when the electronic device has been configured in the multi-user mode, adding the candidate user to the group requested by the tracked user.

[0077] According to at least one embodiment of the present disclosure, the method includes ending the selection period after capturing at least one end command of at least one first user in the group of tracked users.

[0078] According to at least one embodiment of the present disclosure, the command is identified by considering the occurrence of a third action mode in the tracked action of the first user.

[0079] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user to configure the electronic device in the multi-user mode, when the electronic device is configured in the multi-user mode and the selection period has ended, adding the candidate user to the selected tracked users according to the confirmation of the first user in the group of tracked users.

[0080] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user to configure the electronic device in the single-user mode, when the electronic device is configured in the multi-user mode and the selection period has ended, configuring the electronic device in the single-user mode according to the confirmation of the first user in the group of tracked users and selecting the candidate user as the single tracked user.

[0081] According to at least one embodiment of the present disclosure, the confirmation is identified by considering the occurrence of a fourth action mode in the tracked action of the first user.

[0082] According to at least one embodiment of the present disclosure, the method includes determining a signature that uniquely identifies one of the tracked users.

[0083] Moreover, the present disclosure relates to a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users, the current mode of the electronic device and at least one tracked user being selected according to at least one selection request, the selection request of the candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0084] According to at least one embodiment of the present disclosure, the method may include changing a rendering of a virtual scene output on a rendering device coupled to the electronic device according to the at least one obtained input element.

[0085] According to at least one embodiment of the present disclosure, by separately considering similarity criteria between a first action mode associated with the single-user mode and a second action mode associated with the multi-user mode, the selection request is identified as a request to configure the electronic device in the single-user mode or as a request to configure the electronic device in the multi-user mode.

[0086] According to at least one embodiment of the present disclosure, the method includes, when the electronic device is in the single-user mode, ending the selection period after selecting a first user to control the electronic device.

[0087] According to at least one embodiment of the present disclosure, the method includes, when the electronic device is configured in the single-user mode and is controlled by tracking a first user, after receiving a selection request from a candidate user, selecting the candidate user to be tracked for controlling the electronic device instead of the first user, and the selection is performed according to the confirmation and / or rejection of the first user.

[0088] According to at least one embodiment of the present disclosure, the method includes ending the selection period after capturing at least one end command of at least one selected user.

[0089] According to at least one embodiment of the present disclosure, the command is identified by considering the occurrence of a third action mode in the tracked actions of the first user.

[0090] According to at least one embodiment of the present disclosure, the method includes, when the electronic device has been configured in the multi-user mode, after receiving a selection request from a candidate user during the selection period, adding the candidate user to the group of tracked users according to the confirmation and / or rejection of the first user in the group of tracked users.

[0091] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user to configure the electronic device in the single-user mode, when the selection period has ended, configuring the electronic device in the single-user mode, and the candidate user is the single user controlling the electronic device.

[0092] According to at least one embodiment of the present disclosure, the method includes, after receiving a selection request from a candidate user for configuring the electronic device in the multi-user mode, when the selection period has ended, configuring the electronic device in the multi-user mode, the electronic device being controlled by a new group of tracked users, including the candidate user, and starting a new selection period for adding other users to the new group.

[0093] According to at least one embodiment of the present disclosure, the configuration is performed after obtaining confirmation that at least one of the tracked users currently controls the electronic device.

[0094] According to at least one embodiment of the present disclosure, the confirmation and / or rejection is identified by considering the occurrence of a fourth action pattern in the tracked actions of the first user.

[0095] According to at least one embodiment of the present disclosure, the method includes determining a capturable signature that uniquely identifies one of the tracked users.

[0096] According to at least one embodiment of the present disclosure, the first, second, third, and / or fourth action patterns belong to a group including the following:

[0097] - at least one sound sequence;

[0098] - at least one movement of at least one physical object;

[0099] - at least one movement of the human body;

[0100] - at least one pose of the human body;

[0101] - at least one pose of a physical object;

[0102] - at least one orientation of a physical object;

[0103] - a combination thereof.

[0104] According to at least one embodiment of the present disclosure, the at least one sensor belongs to a group including the following:

[0105] - a color camera;

[0106] - a depth camera;

[0107] - a microphone;

[0108] - a gyroscope;

[0109] - an accelerometer;

[0110] - a GPS module.

[0111] According to another aspect, the present disclosure relates to an electronic device that includes at least one processor or at least one processing circuit. The electronic device is adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users. The electronic device includes at least one processor adapted to:

[0112] - Track the actions of at least one user, where at least one tracked user is selected among candidate users according to the current mode of the electronic device and according to a selection request of the candidate user, and the selection request is captured by at least one sensor coupled to the electronic device;

[0113] - Determine at least one input element suitable for controlling the electronic device based on the tracked actions.

[0114] The present disclosure also relates to an electronic device that is adapted to be configured in a single-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is at least partially controlled by input elements obtained by tracking one or more users. The electronic device includes at least one processor adapted to:

[0115] Select the current mode of the electronic device and at least one tracked user according to at least one selection request, where the selection request of the candidate user is captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0116] Although not explicitly described, the above electronic device of the present disclosure may be adapted to perform any embodiment of the above method of the present disclosure.

[0117] According to another aspect, the present disclosure relates to a communication system that includes one or more electronic devices. The electronic device is adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users. The one or more electronic devices include at least one processor or at least one processing circuit adapted to:

[0118] - Track the actions of at least one user, where at least one tracked user is selected among candidate users according to the current mode of the electronic device and according to a selection request of the candidate user, and the selection request is captured by at least one sensor coupled to the electronic device;

[0119] - Determine at least one input element suitable for controlling the electronic device based on the tracked actions.

[0120] The present disclosure also relates to a communication system, the communication system including one or more electronic devices, the electronic devices being adapted to be configured in a single-user mode in which the electronic devices are at least partially controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic devices are at least partially controlled by input elements obtained by tracking one or more users, the one or more electronic devices including at least one processor or at least one processing circuit, being adapted to:

[0121] Select a current mode of the electronic device and at least one tracked user according to at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0122] Although not explicitly described, the present embodiments related to the method or to the corresponding electronic device or communication system can be adopted in any combination or sub-combination.

[0123] According to another aspect, the present disclosure relates to a non-transitory computer-readable program product including program code instructions, which, when executed by a computer, cause the program code instructions to execute any embodiment of the method of the present disclosure.

[0124] For example, at least one embodiment of the present disclosure relates to a non-transitory computer-readable program product including program code instructions, which, when executed by a computer, cause the program code instructions to execute a method implemented in an electronic device, the electronic device being adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method including:

[0125] - Tracking actions of at least one user, the at least one tracked user being selected among candidate users according to the current mode of the electronic device and according to the selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0126] - Determining at least one input element suitable for controlling the electronic device according to the tracked actions.

[0127] At least one embodiment of the present disclosure relates to a non-transitory computer-readable program product including program code instructions which, when executed by a computer, execute a method implemented in an electronic device adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the current mode of the electronic device and at least one tracked user being selected according to at least one selection request, the selection request of a candidate user being captured by at least one sensor coupled to the electronic device during a selection period of limited duration.

[0128] According to another aspect, the present disclosure relates to a computer-readable storage medium carrying a non-transitory software program including program code instructions which, when executed by a computer, execute any embodiment of the method of the present disclosure.

[0129] According to an embodiment of the present disclosure, the computer-readable storage medium tangibly embodies a non-transitory software program including program code instructions executable by a computer which, when executed by the computer, execute a method implemented in an electronic device adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained from a single tracked user, and in a multi-user mode in which the electronic device is controlled by input elements obtained from a group or one or more tracked users, the method comprising:

[0130] - Tracking the actions of at least one user, the at least one tracked user being selected among candidate users according to the current mode of the electronic device and according to the selection request of the candidate user, the selection request being captured by at least one sensor coupled to the electronic device;

[0131] - Determining at least one input element suitable for controlling the electronic device according to the tracked actions.

[0132] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium carries a non-transitory software program including program code instructions executable by a computer. When the software program is executed by the computer, the program code instructions execute a method implemented in an electronic device. The electronic device is adapted to be configured in a single-user mode in which the electronic device is controlled by input elements obtained by tracking a single user, and in a multi-user mode in which the electronic device is controlled by input elements obtained by tracking a group or one or more users. The current mode of the electronic device and at least one tracked user is selected according to at least one selection request. The selection request of a candidate user is captured by at least one sensor coupled to the electronic device during a selection period of a limited duration.

[0133] As those skilled in the art will recognize, aspects of the present principles may be implemented as a system, method, or computer-readable medium. Accordingly, aspects of the present disclosure may take the form of hardware embodiments, software embodiments (including firmware, resident software, microcode, etc.), or embodiments combining software and hardware aspects, which aspects may generally be referred to herein as “circuits,” “modules,” or “systems.” Additionally, aspects of the present principles may take the form of a computer-readable storage medium. Any combination of one or more computer-readable storage media may be utilized.

[0134] A computer-readable storage medium may take the form of a computer-readable program product implemented in one or more computer-readable media and having computer-readable program code executable by a computer implemented thereon. Given the inherent ability to store information therein and the inherent ability to provide information retrieval therefrom, the computer-readable storage medium used herein is considered a non-transitory storage medium. A computer-readable storage medium may be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[0135] It should be recognized that while the following provides more specific examples of computer-readable storage media to which the present principles may be applied, it is merely an illustrative and not an exhaustive list as readily recognized by those of ordinary skill in the art: portable computer floppy disks, hard disks, read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] Thus, for example, those skilled in the art will recognize that the block diagrams presented herein represent a conceptual view of illustrative system components and / or circuitry of some embodiments of the present principles. Similarly, it will be recognized that any flowcharts, flow diagrams, state transition diagrams, pseudocode, etc., represent various processes that may be substantially represented in a computer-readable storage medium and thus executed by a computer or processor, whether or not such computer or processor is explicitly shown.

[0137] It should also be noted that, even if not explicitly disclosed above, at least some embodiments may be combined together.

[0138] The present disclosure is not merely an abstract concept of recognizing a user and assigning the user to certain modes that can be accomplished by human thinking. Instead, the present disclosure includes the actual application of transforming a sound sequence or the spatial relationship / orientation of an object (such as a part of the human body) into an operating mode of a machine and / or a request to control the machine, selecting at least one user to be tracked by a sensor coupled to the machine, and controlling the machine based on input elements determined from the actions of the tracked user. For example, the actual application may modify the rendering of a virtual scene by a rendering device according to the determined input elements.

Claims

1. An electronic device: Among them, The electronic device is adapted to be controlled at least in part by a capture action of a tracked input element obtained by tracking a group of tracked users including two or more users by using one or more sensing devices coupled to the electronic device. Wherein, the tracked input element corresponds to a tracked physical object or a part of a user's body or an audio input from the user. The electronic device includes at least one processor, and the at least one processor is configured to: When a selection request of a tracked candidate user is received during a selection period of a finite duration, add the tracked candidate user to the group of tracked users according to the confirmation of the first user in the group of tracked users, the selection period ends when a user in the group of tracked users captures a given action, and the selection request is executed by the tracked candidate user and captured by the one or more sensing devices.

2. The electronic device according to claim 1, wherein, The at least one processor is further configured to: when a reset request of a user is received after the selection period has ended, configure the electronic device to be controlled by a new group of tracked users including the candidate user, and start a new selection period to add additional users to the new group.

3. The electronic device according to claim 1, wherein, The at least one processor is configured to determine a capturable signature that uniquely identifies one of the tracked users.

4. The electronic device according to claim 1, wherein, The selection period ends when a specific action of a user in the group of tracked users is received.

5. The electronic device according to claim 1, wherein, When a given number of users have been added to the group of tracked users, the selection period ends.

6. The electronic device according to claim 1, wherein, The selection period starts at a specific event.

7. A method executed in an electronic device: Among them, The electronic device is adapted to be controlled at least in part by a capture action of a tracked input element obtained by tracking a group of tracked users including two or more users by using one or more sensing devices coupled to the electronic device. Wherein, the tracked input element corresponds to a tracked physical object or a part of a user's body or an audio input from the user. The electronic device includes at least one processor, and the at least one processor is configured to: When a selection request of a tracked candidate user is received during a selection period of a finite duration, add the tracked candidate user to the group of tracked users according to the confirmation of the first user in the group of tracked users, the selection period ends when a user in the group of tracked users captures a given action, and the selection request is executed by the tracked candidate user and captured by the one or more sensing devices.

8. The method according to claim 7, further comprising: When a reset request of a user is received after the selection period has ended, configure the electronic device to be controlled by a new group of tracked users including the candidate user; And Start a new selection period to add additional users to the new group.

9. The method according to claim 7, further comprising determining a capturable signature that uniquely identifies one of the tracked users.

10. The method according to claim 7, wherein, The selection period ends when a specific action of a user in the group of tracked users is received.

11. The method according to claim 7, wherein, The selection period ends when a given number of users have been added to the group of tracked users.

12. The method according to claim 7, wherein The selection period starts at a specific event.

Images