Relative posture measurements using electronic devices
Patent Information
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- LEGO AS
- Filing Date
- 2024-01-25
- Publication Date
- 2026-04-29
AI Technical Summary
Existing systems for determining the pose of electronic devices, particularly in interactive toys, face challenges related to complexity, computational load, memory requirements, responsiveness, power consumption, and scalability, often requiring additional devices like cameras or base stations that complicate user interaction and increase manufacturing costs.
A system of cooperating electronic devices with a sensing arrangement that includes transmitters, receivers, and processors for generating and measuring electromagnetic fields, allowing wireless communication and coordinated sensing actions to determine relative poses without direct line of sight, using electromagnetic coils for efficient and modular operation.
The system provides reliable, responsive, and scalable pose determination with low power consumption and minimal communication overhead, supporting flexible addition and removal of devices, and integration of multiple functionalities in a small form factor, enhancing user experience.
Description
[0001] The present disclosure relates in one aspect to a system of multiple cooperating electronic devices, the electronic devices comprising a sensing arrangement being adapted for sensing a pose of one or more of the electronic devices relative to another one or more of the electronic devices.BACKGROUND
[0002] Systems of electronic devices are used for a variety of applications, in particular as modular, wireless interactive toys.
[0003] In many situations, e.g. in the context of providing interactive play involving multiple toy elements, it is desirable to provide the electronic devices with the capability to determine their poses relative to each other.
[0004] In this context it is generally desirable to avoid the need for additional devices, such as cameras, base stations and / or the like, because such devices may affect the play value, may complicate the user interaction with the devices and / or increase the complexity or manufacturing cost of the system.
[0005] WO 2020 / 156719 discloses a modular toy system that comprises a plurality of separate electronic toy modules. Each electronic toy module comprises a function device operable to perform a user-perceptible function and a control circuit for controlling the function device. At least a first electronic toy module of the plurality of electronic toy modules comprises a first control circuit, a first function device and a sensor system configured for contactless detection of respective coordinate values of at least two coordinates, each coordinate being indicative of a position or an orientation of at least a second electronic toy module of the plurality of electronic toy modules relative to the first electronic toy module. The first control circuit is configured to control operation of the first function device based on one or more of the detected coordinate values.
[0006] US 2014 / 030955 A1 describes a device that has a body with moving parts attached to it. Inductance coils are attached to the inside of the body and the moving parts.
[0007] The device is equipped with a means of measurement of mutual induction between the coils connected with a computing means designed for determination of mutual position of the specified inductance coils based on mutual induction values, and the computing means is connected to a means designed for creating effects perceivable by the user based on information on mutual position of inductance coils.
[0008] In the context of an interactive toy system, children or other users of the toy system may arrange, and repeatedly re-arrange, multiple wireless interactive toy modules relative to one another, including in configurations not originally foreseen. For example, when the wireless interactive toys are wireless interactive toy construction elements, a child or other user may construct a variety of different toy construction models, which may include multiple wireless interactive toys. Moreover, the user may move the toy modules relative to one another in abrupt and often unforeseeable ways. It is therefore beneficial, when an interactive toy system can seamlessly, and without need for complicated user configuration, cope with a large variety of situations while ensuring proper functioning of the toy system. In particular, a pose estimation performed by the wireless interactive toys should be performed reliably and efficiently.
[0009] Electromagnetic pose detection offers a practical and cost-effective mechanism for such pose determination, which has the potential of providing a large working range without requiring line of sight between the electronic devices. Electromagnetic pose detection may be based on inductive coupling between generator and sensor coils, where a generic system may have three-axis sensor and generator coils. An example of such a mechanism is disclosed in W. Kim, J. Song and F. C. Park, "Closed-Form Position and Orientation Estimation for a Three-Axis Electromagnetic Tracking System," in IEEE Transactions on Industrial Electronics, vol. 65, no. 5, pp. 4331-4337, May 2018, DOI: 10.1109 / TIE.2017.2760244.
[0010] However, prior art mechanisms for determining pose disregard challenges of complexity, computational load, and memory requirements that arise when increasing the number of interacting electronic devices detecting their mutual poses.
[0011] Furthermore, the pursuit of a stimulating and satisfying user experience may lead to strict requirements on the responsiveness of the electronic devices to user interactions. Severe constraints on size, cost, and power consumption may apply as well. For example, in the context of modular, interactive toys, when the user manipulates and moves about two or more toy modules, the user normally expects the toy modules to react promptly and in a predictable manner that is consistent with the movements of the toy modules relative to each other. A lagging or incorrect pose estimation may result in a confusing and unsatisfactory user experience. At the same time constraints on the size and shape may result directly from the requirement that the toys can be handled playfully by the user, their modularity, and a compatibility with existing modular toys.
[0012] It is generally desirable to provide an electronic device and a system of electronic devices, in particular interactive wireless toys that are robust and operate reliably.
[0013] It is further desirable to provide an electronic device and a system of interactive electronic devices that operate with low lag, i.e. a fast response to user interactions.
[0014] It is further desirable to provide a scalable system of interactive electronic devices that allows for flexibly adding and removing devices to the system in an efficient and reliable manner. Furthermore, it may be desirable to perform such addition and removal without affecting the overall functioning of the system.
[0015] It is further desirable to provide an electronic device and a system of electronic devices that operate with low power consumption.
[0016] It is further desirable to provide an electronic device and a system of electronic devices that require little communications overhead.
[0017] It is further desirable to provide a system of electronic devices that can easily be operated by a user, in particular a child, with little or no need for user configuration.
[0018] It is further desirable to provide an electronic device and a system of electronic devices that can be manufactured at low cost.
[0019] It is further desirable to provide an electronic device that has a small form factor, e.g. small enough to be easily grasped and held by a small child's hand.
[0020] It is further desirable to provide a system of electronic devices that is scalable and can cope with a varying number of electronic devices.
[0021] Against this background, despite previous efforts, it remains desirable to provide an electronic device and a system of such devices that solve one or more of the above problems and / or other problems, and / or that have other benefits, or that at least provide an alternative to existing solutions.SUMMARY
[0022] According to a broad aspect of the disclosure, a system comprises a plurality of cooperating electronic devices, wherein each of the cooperating electronic devices comprises: a sensing component of a sensing arrangement; a transceiver adapted for wireless communication with other electronic devices; and a processor operationally coupled to the transceiver and to the sensing component; wherein each of the cooperating electronic devices is configured to share with the other cooperating electronic devices a parameter request for a physical parameter of one or more of the cooperating electronic devices relative to one or more of the other cooperating electronic devices; to cooperate with one or more of the other electronic devices to operate the sensing components of the cooperating electronic devices to obtain measurement data indicative of the requested physical parameters, and to process the measurement data to determine the requested physical parameter. According to a particular aspect, the requested physical parameter includes a pose of a first electronic device relative to a second electronic device. The requested physical parameter may thus include information about a pose of a first electronic device relative to a second electronic device. Furthermore, the position of the first electronic device relative to the second electronic device may include a distance and / or a direction. Preferably in some embodiments, the direction is a disambiguated direction as discussed elsewhere herein. According to a particular aspect, the sensing arrangement comprises: a transmitter for generating a stimulation field; a receiver for measuring one or more local properties of a stimulation field; and a processing module configured to compute a pose of one or more of the electronic devices relative to another one or more of the electronic devices, based on the measured one or more local properties of the stimulation field. The measured one or more local properties are physical properties of the stimulation field generated by the transmitter. The term 'local' as used herein in the context of measuring physical properties refers to the respective location of the receiver where the measurement is performed. In a further particular aspect, the generated stimulation field is an alternating electromagnetic field. In a further particular aspect, the generated electromagnetic field is picked up at the receiver by inductive coupling.
[0023] According to a preferred embodiment, a system comprises a plurality of electronic devices, wherein each of the electronic devices comprises: a sensing component of a spatial sensing arrangement; a transceiver adapted for wireless communication with the other electronic devices; and a processor operationally coupled to the transceiver and to the sensing component; wherein the electronic devices are configured to share a request for a pose of a first electronic device relative to a second electronic device; and to cooperate to operate the sensing components of one or more of the electronic devices to obtain measurement data and to process the measurement data to determine the requested relative pose.
[0024] According to some embodiments, the electronic devices are configured to share one or more requests for pose information, each pose request including a request for pose information of a respective electronic device relative to a respective other electronic device; and to cooperate to operate the sensing components of one or more of the electronic devices to obtain measurement data and to process the measurement data to determine the pose information according to the shared requests. Each pose request may include a request for pose information of a respective electronic device relative to a respective other electronic device. For example, a first request may include a request for pose information of a first electronic device relative to a second electronic device, and a second request may include a request for pose information of a third electronic device relative to a fourth electronic device.
[0025] The sensing component of each of the electronic devices is operable to perform sensing actions in response to control signals from the processor. Such sensing actions may include any or all of the steps necessary for performing the measurements required to fulfil a request for spatial measurement data for determining a relative pose of a first electronic device relative to a second electronic device. Depending on the sensing component present in a given electronic device, the electronic device may perform at least a portion of such sensing actions. A complete measurement typically involves two or more electronic devices, each contributing by operating their respective sensing component so as to perform, in combination, all the necessary sensing actions to fulfil the spatial data request. The sensing components of the electronic devices thus form a cooperative sensing arrangement, wherein the sensing arrangement is operable to sense respective poses of one or more of the electronic devices relative to another one or more of the electronic devices. Cooperation occurs through wireless communication using the transceiver of the electronic devices, and is mediated by the respective processors.
[0026] The cooperative sensing arrangement is structured as a service provided by the cooperating electronic devices to each of the cooperating electronic devices that may request spatial data. The cooperative sensing arrangement may be controlled by a service module, wherein the service module is configured to receive requests for spatial data as an input, to handle the acquisition of the requested spatial data, and providing the spatial data as an output to fulfil the requests. To facilitate the cooperation, the processor of an electronic device shares a request for spatial data that arises at said electronic device through wireless communication with the other electronic devices. The system then coordinates the sensing actions required to fulfil the request with the cooperating electronic devices. The electronic devices cooperate to obtain measurement data indicative of the requested pose or pose information, and to process the measurement to determine the requested pose or pose information. Obtaining the measurements may involve operating one, and typically more than one, or all of the cooperating electronic devices.
[0027] Advantages of a system according to the disclosure comprising a spatial sensing arrangement for cooperating electronic devices that is provided as a service to the cooperating electronic devices include the following. An improved performance predictability in terms of responsiveness and timeliness of spatial sensing may be achieved already with only two cooperating electronic devices, due to the coordination of sensing tasks. The disclosure facilitates the possibility to control and optimize resource use, e.g. in respect of power usage, and / or for avoiding interference. For example, each of two electronic devices may concurrently request spatial data relative to the other one of the two electronic devices. A reduction in the use of resources can then be achieved, since only one measurement may be needed that can be shared. Furthermore, the best sensing components of the cooperating electronic devices can be selected and assigned for each sensing action, e.g. the device with the best battery status may be selected for generating a stimulation field that can be measured by one or more of the other electronic device(s). The disclosure is particularly advantageous in the case of many requests, where it allows for control and coordination of sensing actions, and further allows optimizing required sensing actions. The disclosure also provides scalability of the system where many devices can be handled in an efficient manner, yet allowing the use of processors with a relatively low processing power as compared to systems requiring a smart device, or a central hub with relatively powerful processors. The disclosure also supports modularity and flexibility in case an electronic device is added to or removed from the group of cooperating electronic devices. Furthermore, the disclosure also supports stability in case a master device selected tasked among the cooperating electronic devices for tasks of coordination is removed or drops out of the group of cooperating devices. Further advantages are also achieved in the case that certain components of the electronic devices are put to multiple uses for different, potentially conflicting functionalities, e.g. for tag reading and / or for receiving wireless charging energy, where spatial sensing provided as a service allows for control and coordination of actions to be performed to accomplish / support these further uses, thereby providing a satisfying user experience. The disclosure thus also contributes to ensuring reliable and responsive operation of the electronic devices with multiple functionalities that are integrated in a small form factor. Also further potentially interfering or conflicting functions or operations that may be performed by one or more of the cooperating devices can be taken into account. Thereby, the disclosure is also particularly useful for supporting a high level of multiple functionalities integrated in cooperating electronic devices with a small form factor. These and further advantages are also achieved and further enhanced by further advantageous embodiments as disclosed in the following.
[0028] Further according to some embodiments, cooperation of the electronic devices includes generating a schedule for performing sensing actions, and to operate the sensing components to perform the sensing actions consistent with the schedule. Furthermore, cooperation of the electronic devices includes obtaining the measurement results from the sensing components operated consistent with the schedule, and processing the measurement data to determine spatial data according to the shared requests. As mentioned above, the cooperative sensing arrangement is structured as a service provided by the cooperating electronic devices to each of the cooperating electronic devices that may request spatial data. When a need for spatial data arises, the need is submitted to the service as a request shared among the cooperating electronic devices. The service identifies sensing actions, which are then performed in a timely coordinated manner by the cooperating electronic devices to fulfil the request.
[0029] Advantageously, coordination is implemented in a schedule shared among and followed by the cooperating electronic devices. The cooperating electronic devices then perform sensing actions consistent with the schedule and according to the shared requests. Further requests for spatial data from any of the cooperating electronic devices may already exist, and may be shared among the cooperating devices. Such requests may be added to a pool of requests to the service, e.g. in a list. The service may then determine the sensing actions to be performed to fulfil the multiple requests at hand, and furthermore coordinate execution of the sensing actions to fulfil the requests. Once executed, the sensing actions provide measurement results, which are processed to determine the requested spatial data. Processing of the measurement data may be performed in a processing module receiving measurement data from one or more of the sensing components performing the sensing actions. The processing module may be implemented in any of the processors of the cooperating electronic devices, and may conceivably even be implemented in a distributed manner. Typically, however, the measurement results are also directly processed in the processor of the electronic device where the actual measurement producing the measurement result is performed. The service may then provide the determined spatial data to the one or more requesting electronic devices. Furthermore, the spatial data may be shared among all cooperating electronic devices. The requested spatial data may thus be provided as a service and may be implemented as a service module on any of the cooperating electronic devices. Conceivably, the service module even may be implemented in a distributed manner. Typically, the service is implemented in all of the cooperating electronic devices, and one of the cooperating devices is assigned the role of a master device. The service is thus typically operated on a master device that has been selected among the cooperating electronic devices beforehand, or which is selected dynamically, based on pre-determined selection criteria. Cooperation may thus include selecting one of the cooperating electronic devices as a master device, wherein the master device is configured to coordinate the sensing actions among the cooperating electronic devices so as to fulfil one or more requests for determining respective poses of one or more of the plurality of electronic devices relative to another one or more of the electronic devices.
[0030] Further according to some embodiments, the spatial sensing arrangement comprises: one or more transmitters for generating a stimulation field; one or more receivers for measuring one or more local properties of the stimulation field; and one or more processing modules configured to compute a pose of one or more of the electronic devices relative to another one or more of the electronic devices, based on the measured one or more local properties of the stimulation field.
[0031] Using a stimulation field for spatial sensing has the advantage that the spatial measurements can be performed by the cooperating electronic devices, without the need for a direct line of sight between the electronic devices. Furthermore, it turns out that spatial sensing using a stimulation field can be integrated in the cooperating electronic devices with a small form factor and using processors with relatively low data processing power, thus facilitating a flexible and modular system without the need of external spatial sensing arrangements, such as triangulation beacons, camera sensing, or the like. This is particularly useful, e.g. when the electronic devices are combined in structures where the line of sight may easily be obstructed, such as when the electronic devices are wireless interactive toy construction elements forming part of one or more toy constructing models constructed from toy construction elements of a toy construction system. External sensing arrangements may also be undesirable or even prohibitive for an unexperienced user group as they may require particular set-up considerations to make them work.
[0032] Further according to some embodiments, the sensing components of each of the electronic devices comprises one or more of: a transmitter adapted to generate a stimulation field; a receiver adapted to measure a local property of a stimulation field generated by another electronic device; and a processing module configured to compute a pose of the electronic device relative to another one of the electronic devices, based on the measured local property of the stimulation field generated by the other electronic device.
[0033] By adapting each of the cooperating electronic devices to be operable as a transmitter, as a receiver, and as a processor for determining relative pose information, a high modularity and flexibility for combining electronic devices to cooperate in an interactive user experience is achieved.
[0034] Further according to some embodiments, operation of the sensing components further includes operating only one transmitter of the cooperating electronic devices at a time. Thereby, the issue of interference between measurements may be reduced to avoiding interference between stimulation fields of cooperating electronic devices, thereby simplifying the task of handling many simultaneous spatial parameter requests.
[0035] Further according to some embodiments, operation of the sensing components further includes operating one or more receivers together with the transmitter with a time overlap. Thereby an efficient use of the stimulation field is achieved, where multiple measurements can be performed simultaneously at different locations using the same stimulation field.
[0036] Further according to some embodiments, one or more, preferably each, of the cooperating electronic devices comprises one or more electromagnetic coils, wherein the one or more electromagnetic coils are operable as a transmitter adapted to generate an alternating electromagnetic stimulation field. Using an electromagnetic coil arrangement as a spatial sensing component, here for generating a stimulation field, turns out to be an efficient and space saving technique to obtain reliable measurement data for fulfilling requests for determining the respective poses of one or more of a plurality of cooperating electronic devices relative to one or more other ones of the plurality of cooperating electronic devices. Surprisingly, it furthermore turns out that the coils may also be put to use for other tasks than relative pose measurements, such as nearfield wireless tag reading and wireless charging of the electronic devices. Thereby, a space efficient and multifunctional sensing component is provided allowing for a large variety of spatial sensing tasks, and furthermore facilitate integrating a high level of functionality within an extremely restrictive form factor, such as within the dimensions of a modular toy construction element. Advantageously, such dimensions are linear dimensions of less than 5 cm, less than 4 cm, less than 3 cm, or less than 2cm, and at least 0.5 cm, or at least 1 cm. Advantageously, these ranges of dimensions are applicable to the dimensions of a bounding box of the housing of one or more of the electronic devices in three orthogonal directions. According to some embodiments, the electronic devices may have a housing with a bounding box volume of less than 30 cm 3< , less than 20 cm 3< , preferably less than 15 cm 3< , less than 12 cm 3< , less than 10 cm 3< , or even less than 8 cm 3< .
[0037] Further embodiments and advantages of electromagnetic spatial probing and of coil arrangements are discussed in detail further below.
[0038] Further according to some embodiments, one or more, preferably each, of the electronic devices comprises one or more electromagnetic coils, wherein the one or more electromagnetic coils are operable as a receiver adapted to pick up the generated alternating electromagnetic stimulation field and to measure one or more local properties of the alternating electromagnetic stimulation field. As mentioned above, using an electromagnetic coil arrangement as a spatial sensing component turns out to be an efficient and space saving technique to obtain reliable measurement data for fulfilling requests for determining the pose of one or more of a plurality of cooperating electronic devices relative to one or more other ones of the plurality of cooperating electronic devices. By also using the electromagnetic coil arrangement as a sensing component for measuring, the modularity, versatility and flexibility in use of the cooperating electronic devices is further enhanced, without a significant penalty in footprint, e.g. in respect of harshly restrictive constraints on the form factor of the electronic devices, such as for elements of a modular toy construction system.
[0039] Further according to some embodiments, the system is further configured to initiate a spatial parameter request in one or more of the electronic devices in response to a trigger event. A request for spatial parameters may arise in any one or more, or all, of the cooperating electronic devices, and is serviced by the electronic devices in cooperation. By way of example only, the request may stem from an interaction with one or more of the cooperating electronic devices, from programmed instructions carried out at one or more of the electronic devices, or similar events. Any such events may then trigger a request for determining respective poses of one or more of the electronic devices relative to one or more of the other cooperating electronic devices. Initiating such requests at one or more, or all of the cooperating electronic devices has the advantage that the system can flexibly cater to a large variety of use scenarios, which may require discrete and / or continued tracking of spatial parameters, and which may involve dynamically configuration and / or reconfiguration of the cooperating electronic devices during such uses. This is particularly pronounced in a play environment, where the electronic devices are interactive toys, and in particular where the electronic devices are toy construction elements of a modular toy construction system. Fulfilling these requests in cooperation as a service synergistically adds to this flexibility and modularity.
[0040] The following non-exhaustive list of trigger events gives advantageous examples where initiating a spatial parameter request is useful, and can be seen to support a large variety of use scenarios.
[0041] Further according to some embodiments, the trigger event is one or more of: a user interaction with one or more of the electronic devices; a programmed event occurring in a user interaction module, such as a game event in a game implemented on the cooperating electronic devices; a change in operational mode of one or more of the electronic devices; addition of one or more electronic devices to the system; removal of one or more electronic devices from the system; grouping of multiple of the electronic devices in a group of cooperating electronic devices; addition of one or more electronic devices to a group of cooperating electronic devices; removal of one or more electronic devices from of a group of cooperating electronic devices.
[0042] Further according to some embodiments, the system is further configured to cause generation of a user-perceptible output at one or more of the interacting electronic devices, based on the determined spatial parameters. The spatial parameters are indicative of the spatial relations between the electronic devices. By generating user-perceptible output according to the determined spatial parameters, the user-perceptible output may thus be linked to the spatial relations between the electronic devices. Furthermore, as mentioned above, the spatial parameter requests may result from a user-interaction. The user-perceptible output may thus further be linked to a user interaction affecting the spatial relations between the cooperating electronic devices. Thereby, a highly intuitive and illustrative interactive user-experience can be created, where one or more spatial relations between cooperating electronic devices are linked to the user experience. The user experience may become even more vivid, when the spatial relations between the cooperating electronic devices are determined and / or even tracked in a continued manner, and wherein corresponding user-perceptible output is dynamically generated. Synergistically, the enhanced user experience is further facilitated by a responsive system with low lag, i.e. by reducing the latency between a change in the spatial relations between the electronic devices, and the user-perceptible output generated based on these spatial relations. As already discussed elsewhere herein, this is achieved by controlling the electronic devices to handle the spatial parameter requests in a cooperating manner, where requests are fulfilled as a service.
[0043] Further according to some embodiments, one or more, preferably all of the electronic devices are wireless interactive toys. Thereby a highly scalable and flexible interactive toy system is provided allowing to create play experiences based on the spatial relations between cooperating wireless interactive toys.
[0044] Further according to some embodiments, one or more, preferably all of the wireless interactive toys are or comprise a wireless interactive toy construction element. Thereby, a modular and highly scalable interactive toy construction system is provided allowing to create play experiences based on spatial relations between cooperating wireless interactive toy construction elements and toy construction models constructed using such toy construction elements. As further detailed elsewhere herein, advantageously the toy construction elements are modular construction elements of a modular toy construction system with coupling members for the releasable connection of the toy construction elements to form toy construction models.
[0045] Further according to some embodiments, the electronic devices are further configured to form a wireless toy network with other ones of the plurality of electronic devices of the interactive toy system. As also discussed elsewhere herein, the wireless toy network may be implemented using any suitable wireless network infrastructure, such as a wireless local area network infrastructure, or preferably a wireless personal network infrastructure. Advantageously, the wireless toy network is 'open' in the sense that compatible electronic devices, and in particular wireless interactive toys, such as wireless interactive toy construction elements can freely join and / or leave the wireless toy network. Further advantageously, the wireless toy network is open to compatible electronic devices and any compatible further devices related to the system. Such further related devices may be adapted to provide supporting functions or otherwise interact with the system, e.g. to facilitate and / or to configure operation of the system. Such further related devices are expressly configured to be compatible with the wireless toy network, and thus to have access to the wireless toy network. Examples for such further related devices include one or more of charging devices, smart phones, tablet computers, PCs, and similar devices. Preferably, the network is at the same time 'selective' or 'exclusive' in the sense that the network is only accessible to compatible electronic devices and any compatible further devices related to the system, so as to mitigate interference with other devices that may operate on the same wireless network infrastructure and that are not part of the plurality of cooperating electronic devices or that are not adapted to support operation of the system or otherwise interact with the system. Thereby, an improved reliability and responsive performance of the system is achieved without jeopardizing the modularity and flexibility of the system.
[0046] Further according to some embodiments, wireless communication among the electronic devices is by broadcasting messages in a network of cooperating electronic devices and selectively receiving messages from cooperating electronic devices in the network. Thereby, a further improved reliability and responsive performance of the system is achieved that furthermore supports the highly modular and flexible nature of the system.
[0047] According to a further aspect, the present disclosure relates to an electronic device that comprises: one or more electromagnetic coils for measuring at least one magnetic field generated by one or more electromagnetic coils of another electronic device; a processing unit for computing a pose of the electronic device relative to said other electronic device.
[0048] According to various embodiments of the electronic device, at least one of the one or more electromagnetic coils is configured for measuring gradient information of said measured at least one magnetic field; and wherein the processing unit is configured to compute the pose based on the at least one measured magnetic field and based on the measured gradient information.
[0049] The inventors have realised that a pose estimation based on measured gradient information in addition to the measured magnetic field provides a reliable pose estimation that resolves the pose ambiguities of prior art methods.
[0050] The one or more electromagnetic coils of the electronic device measure the magnetic field and the gradient information at least at one location of the electronic device, where the magnetic field is generated by an electromagnetic coil of said other electronic device, in particular by an electromagnetic coil located within said other electronic device. Said other electronic device may sequentially generate magnetic fields by multiple coils, which may be oriented differently relative to each other; hence multiple fields may be generated and the electronic device may thus measure multiple fields.
[0051] The measured magnetic field may represent one or more measured components indicative of the strength of the magnetic field, i.e. one or more measured magnetic field components. The measured magnetic field component may include phase information about a time varying magnetic field and / or information about the direction of the magnetic field. For the purpose of the present description the magnetic field component may be expressed as a component of a magnetic field intensity vector H or as a component of a magnetic flux density vector B, or otherwise. The field measurements may provide information about components of the magnetic field intensity vector or of the magnetic flux density vector along one or more directions. The measured magnetic field thus refers to the measured values indicative of the magnetic field components measured by the electronic device.
[0052] The gradient information may include any information indicative of the spatial variation of the magnetic field at the location of the electronic device, e.g. the spatial variation within the electronic device. The gradient information may include information about a gradient vector of the magnetic field, or other information indicative of the spatial variation of the magnetic field within the electronic device. The gradient information may include direction information about a direction of the spatial gradient of the magnetic field, e.g. of the magnetic field intensity vector or of the magnetic flux density vector or of components thereof, at the location of the electronic device. The direction information may e.g. be expressed as a sign or other indication of the direction of the gradient component along the axial directions of the respective one or more electromagnetic coils, or otherwise. For example, the direction information may indicate whether the component of the gradient vector along the coil axis is positive or negative. The gradient information may further include information about the magnitude of the gradient, e.g. the magnitudes along the respective coil axes.
[0053] In some embodiments, the electronic device includes more than one coil, such as three or more coils. Preferably, each of the three coils defines a coil axis and the three coils are arranged such that their respective coil axes are linearly independent, e.g. orthogonal to one another. Accordingly, the electronic device may perform pose determinations in more than one dimension, in particular in three dimensions. The pose determination may be a determination of six degrees of freedom, in particular three directional degrees of freedom and three orientation degrees of freedom.
[0054] For the purpose of the present disclosure, the terms pose and pose information are intended to refer to information about the position and / or the orientation of the electronic device relative to the other electronic device. Furthermore, the term pose determination is intended to refer to the determination of such information. The information about the position may include a direction and / or a distance between the electronic devices. The direction may be expressed as a direction of a position vector of the electronic device relative to a coordinate system of the other electronic device, or otherwise. The orientation may be expressed as a rotation of a coordinate system of the electronic device relative to a coordinate system of the other electronic device, or otherwise. It will be appreciated that, in some situations, the electronic device may only need partial pose information, e.g. only information about the distance between the electronic devices. In such situations, the electronic device may compute partial pose information. In other situations, the electronic device may require complete pose information, i.e. information about the distance, the direction and the orientation. In such situations, the electronic device may thus compute the complete pose.
[0055] As will be described in greater detail herein, the pose determination based solely on magnetic field component, including the directions of the magnetic field vectors, results in an ambiguous pose. In particular, the ambiguity affects the determination of the direction between the electronic device and the other electronic device, as the computation results in two solutions for the direction between the devices that are both consistent with the measured magnetic fields. The gradient information may be used to resolve said ambiguity in the determination of the direction. Accordingly, in some embodiments, the computation of the partial or complete pose comprises a computation of at least a relative direction between the other electronic device and the electronic device, wherein the computation of the direction is based on the at least one measured magnetic field component and on the measured gradient information.
[0056] The gradient information may be obtained in a variety of ways. For example, one or more of the electromagnetic coils for measuring the at least one magnetic field may be configured to provide magnetic field measurements at respective locations relative to the electronic device, or to otherwise obtain a measurement indicative of a variation of the magnetic field, e.g. a variation along a direction relative to the one or more electromagnetic coils, e.g. along an axial direction of one or more of the electromagnetic coils.
[0057] To this end, the electronic device may include at least one set of coils that are arranged in series with each other. The coils of the set of coils may be arranged coaxial with each other or otherwise. The set of coils may be a pair of two coils or include more than two coils. The set of coils may define at least two measurement nodes for measuring respective electric signals, e.g. one measurement node associated with each of the set of coils, or otherwise, Alternatively or additionally, the electronic device may include at least one coil that defines at least two measurement nodes at respective locations along the coil, e.g. at one end of the coil and at a centre of the coil. The coil portions may thus be considered to be two series-connected coil portions of a coil. The measurement nodes may be associated with respective coil portions of the at least one coil.
[0058] In some embodiments, the measurement of the gradient may thus involve more than one field measurements, which may be performed concurrently or sequentially.
[0059] The electronic device may comprise a magnetic field measurement circuit configured to measure electric signals, e.g. voltages, at respective ones of the measurement nodes associated with respective coils of a set of coils and / or with respective coil portions of a single coil. In some embodiments, the magnetic field measurement circuit is configured to measure a differential voltage between two pairs of measurement nodes. One of the measurement nodes may be common for both measurements. In some embodiments, the magnetic field measurement circuit is configured to measure the electric signal at the measurement nodes relative to a suitable reference, such as ground. The measurement nodes may be positioned at different points (taps) along the coil, e.g. at one end of the coil, or at any point along its length.
[0060] As discussed above, in some embodiments, the measurement circuit comprises two or more coils which are connected in series, optionally with one or more additional electronic components in-series between them. In this embodiment, one or more measurement nodes may be located at nodes between different pairs of coils, and / or at the end of the series of coils.
[0061] In one embodiment, one node of the circuit is a reference node, which may be grounded, and the measurement circuit is configured to measure the differential voltage between a first measurement node and the reference node and between a second measurement node and the reference node.
[0062] The first measurement node may be located at one end of the coil or at one end of a set of series-connected coils, while the second measurement node may be positioned at a location along the coil between the opposite ends of the coil or at a location between two series-connected coils. For example, the second measurement node may be a central measurement node positioned centrally between the ends of the coil. The second measurement node may be formed as a coil tap, e.g. a centre tap. Accordingly, the electronic device may not only measure the overall magnetic field but also variations of the magnetic field along the coil axis or variations between series-connected coils, thus obtaining gradient information indicative of whether the component of the magnetic field along the axis of the coil is larger at one end of the coil (or at one coil of a set of coils) than at the other, opposite end of the coil (or at another coil of the set of coils). It will be appreciated that other measurement arrangements may be used to obtain the gradient information.
[0063] According to some embodiments, the set of coils comprises a set of two coils, which may be spaced apart from each other, such as axially spaced apart from each other. The two coils may be arranged coaxially with each other. The two coils of the set are preferably arranged such that the electromotive forces induced in the respective coils of the set are different in the two ambiguous poses, in particular such that the difference between the magnetic fields at the two coils depends on, which of the two ambiguous poses the electronic device is in. To this end, the two coils may be arranged in series with each other. The two coils may be spaced apart from each other with a clearance between the two coils, e.g. a clearance of more than 1 mm, such as more than 2 mm. For example, the coils may be arranged proximally to respective opposite sides of a housing of the electronic device, thereby maximizing the distance between the coils. The coils making up a set of coils may have the same inductance or otherwise be identical, or they may be different from each other. A measurement node may be located between the two coils, e.g. along a wire or circuitry that electrically connects the two spaced-apart coils. Accordingly, reliable gradient information may be obtained.
[0064] In some embodiments, the one or more electromagnetic coils comprises three sets, in particular three pairs, of coils. Each set may comprise two spaced apart coils connected in series. In some embodiments, each of the three sets of coils defines a coil axis and the three sets of coils are arranged such that their respective coil axes are linearly independent, e.g. orthogonal to one another. Generally, some embodiments of the electronic device are configured to perform three magnetic field measurements and three gradient measurements for each magnetic field (e.g. for each of three magnetic fields) generated by the other electronic device.
[0065] Prior art methods for computing pose information based on measured magnetic fields suffer from an ambiguity of the determined pose. In particular, they result in two solutions, in particular two solutions where the electronic device has diametrically opposite (antipodal) positions relative to the other electronic device, which generates the at least one magnetic field. In some embodiments of the electronic device disclosed herein, the additional gradient information is used to disambiguate between the two solutions. Accordingly, in some embodiments, the processing unit is configured to compute pose information indicative of two candidate poses and to select, based at least on the measured gradient information, one of said candidate poses as a computed pose. The computation of the candidate poses may be performed based, in particular solely based, on the measured magnetic field componets, including the relative directions of the magnetic field vector and / or relative phase information, while the selection of one of the candidate poses may be based on the gradient information.
[0066] In some embodiments, the processing unit is further configured to compute one or more confidence values associated with the computed pose, e.g. with the selected candidate pose, thereby allowing downstream processing steps that utilize the pose information to base any actions on the confidence that the determined pose is accurate. In particular, the downstream processing may base one or more actions on the computed pose and on the computed one or more confidence values. This may e.g. be useful when the determined pose abruptly changes. In such a situation, the downstream process may take a decision as to whether to rely on the information about the determined abrupt change or whether to ignore it as a likely result of noisy measurements. The one or more confidence values may include e.g. an accuracy measure of an aspect of the pose, e.g. of the distance and / or the one or more confidence values may include a confidence value indicative of the confidence that the correct candidate pose has been selected.
[0067] In some embodiments, the processing unit is configured to repeatedly perform pose estimation resulting in a temporal sequence of computed poses, thereby allowing the electronic device to track the relative pose over time and, hence, allowing the electronic device to react to changes in the relative pose. In some embodiments, the processing unit is configured to base the computation of a current pose of the temporal sequence of computed poses on a current set of one or more measurements of the at least one magnetic field and on one or more previous poses of the temporal sequence of poses. Accordingly, the processing unit may implement a tracking mechanism that uses one or more previously determined poses as an input to the computation of the current pose measurement, e.g. by applying a suitable filter or otherwise. This allows the process to avoid erroneous determinations of abrupt changes of the pose. In some embodiments, the processing unit is configured to perform the computation of the two current candidate poses based on the current measurement(s) of the at least one magnetic field (e.g. field components, including relative phases and / or field directions) and on one or more previous poses. Alternatively or additionally, the processing unit may be configured to perform the selection of one of the two current candidate poses, which result from one or more current measurements of the at least one magnetic field (e.g. field components along one or more directions, including phase information), based at least on a current measurement of the gradient information and on one or more previous poses of the temporal sequence of poses. Accordingly, the processing unit may implement a tracking mechanism that uses one or more previously determined poses as an input to the disambiguation of the current pose measurement.
[0068] The computation of the current candidate poses may further be based on respective confidence values associated with the current and / or one or more previous candidate poses. Similarly, the selection of one of the two current candidate poses may be further based on respective confidence values associated with the current and / or one or more previous selected candidate poses.
[0069] According to some embodiments, the processing unit comprises a data storage device having stored thereon a set of one or more device-specific calibration parameters, and wherein the processing unit is configured to compute the candidate poses and / or to select one of said candidate poses further based on said stored set of one or more device-specific calibration parameters. The one or more device-specific calibration parameters may include one or more calibration parameters that are specific to the electronic device. Alternatively or additionally, the one or more device-specific calibration parameters may include one or more calibration parameters that are specific to said other electronic device. To this end, the electronic device may receive said one or more device-specific calibration parameters of the other electronic device from said other electronic device. For example, the other electronic device may communicate, e.g. broadcast, its one or more device-specific calibration parameters wirelessly, in particular via a wireless communications network. Examples of device specific calibration parameters may include field calibration values associated with the transmit strength of the magnetic field transmitted by the other electronic device and / or calibration parameters associated with the receive sensitivity for measuring the magnetic field by the electronic device.
[0070] In some embodiments, the electronic device comprises a wireless communications interface configured for wireless communication via a wireless communications network and wherein the electronic device is configured to wirelessly communicate with one or more other electronic devices via said wireless communications network. Accordingly, the electronic device may receive calibration parameters and / or other information from e.g. the other electronic device. Examples of other information may include a timing schedule indicative of a time when the other electronic device generates the magnetic field and / or when the electronic device is to measure the magnetic field(s) generated by the other electronic device. The timing schedule may be a shared timing schedule, shared between multiple electronic devices as described herein.
[0071] The present disclosure relates to different aspects including the electronic device described above and in the following, corresponding apparatus, systems, methods, and / or products, each yielding one or more of the benefits and advantages described in connection with one or more of the other aspects, and each having one or more embodiments corresponding to the embodiments described in connection with one or more of the other aspects and / or disclosed in the appended claims.
[0072] In particular, according to one aspect, disclosed herein are embodiments of a system of such electronic devices. In some embodiments, some or all electronic devices of the system are configured to perform corresponding pose estimation as disclosed herein.
[0073] In particular, disclosed herein are embodiments of a system of electronic devices, the system comprising a plurality of electronic devices, the plurality of electronic devices comprising at least one receiver electronic device and at least one transmitter electronic device, the receiver electronic device being an electronic device as defined in any one of the embodiments disclosed herein, wherein the transmitter electronic device comprises one or more electromagnetic coils for generating one or more magnetic fields, wherein at least one of the one or more electromagnetic coils of the receiver electronic device is configured for measuring at least one of the one or more magnetic fields generated by said transmitter electronic device and for measuring gradient information of said measured at least one magnetic field, and wherein the processing unit of the receiver electronic device is configured to compute a pose of the receiver electronic device relative to said transmitter electronic device based on the measured at least one magnetic field and based on the measured gradient information. For the purpose of the present description, the term transmitter electronic device is intended to refer to an electronic device that generates the one or more magnetic fields via the one or more electromagnetic coils of the transmitter electronic device. The generation and emission of the magnetic field by the transmitter electronic device is thus also referred to as transmission of the magnetic field.
[0074] In some embodiments, one or more of the electronic devices may only be operable as receiver electronic devices. Alternatively or additionally, one or more of the electronic devices may only be operable as transmitter electronic devices. Yet alternatively or additionally, one or more of the electronic devices may be selectively operable as transmitter electronic device or as receiver electronic device. For example, the processing unit of the electronic device may control the electronic device to operate as a transmitter electronic device or as a receiving operating device, e.g. alternatingly, according to a timing schedule, or otherwise. In some embodiments, each of the electronic devices may be selectively operable as a transmitter electronic device or as a receiver electronic device, thereby making the system scalable and flexible.
[0075] Each electronic device may be individually movable within a space, independently from the other electronic devices of the system. In some embodiments, the system is an interactive toy system and one or more, such as all, of the electronic devices are wireless interactive toys. In particular, the interactive toy system may be an interactive toy construction system and each wireless interactive toy is or comprises a wireless interactive toy construction element. For example, the or each wireless interactive toy may be an individual one of the interactive toy construction elements, or a toy construction model constructed from two or more toy construction elements of the toy construction systems, including at least one interactive toy construction element.
[0076] In the context of an interactive toy system, the user may move around the wireless interactive toys as part of a play experience. The wireless interactive toy may be configured to detect the user's interactions with components of the interactive toy system and react to the detected interactions by providing user-perceptible outputs, e.g. including audible, visual and / or haptic outputs. In embodiments where the wireless interactive toy is a toy construction element of a toy construction system, or is constructed from a plurality of toy construction elements, the user may repeatedly attach and / or detach the wireless interactive toy construction elements to / from one another and / or to / from other toy construction elements of the toy construction system so as to construct and deconstruct different spatial structures. Spatial structures constructed from two or more toy construction elements will also be referred to as toy construction models. They may repeatedly be constructed and disassembled without destroying the toy construction elements from which they are constructed.
[0077] It is therefore highly desirable that the wireless interactive toys have a short response time to changes in the relative positions of the wireless interactive toys, such that the user perceives an immediate detectable reaction when the user moves the components of the toy system relative to each other. Accordingly, it is desirable that the wireless interactive toys perform the pose estimation operations reliably and at a sufficiently high rate to facilitate a fast response time.
[0078] The magnetic pose estimation operation may have a pose estimation range, i.e. a maximum range at which one electronic device may still determine a relative pose of another electronic device. For example, the pose estimation range may be at least 1 m, such as at least 2 m, such as at least 5 m. However, other embodiments may have different ranges. Yet further, in some embodiments, the effective ranges for different aspect of the pose determination (e.g. the distance, direction, orientation, disambiguation), may be different from each other.
[0079] In some embodiments, the electronic devices may be configured to perform the pose estimation operation in a cooperative manner. To this end, one or more of the electronic devices, such as each electronic device, may be configured to output at least one magnetic field that may be sensed by one or more of the other electronic devices. Moreover, one or more of the electronic devices, such as each electronic device, may be configured to sense at least one magnetic field output by another one of the electronic devices and to compute at least its relative pose relative to the other electronic device based at least on the sensed magnetic field. The relative pose may include a relative position and, optionally, a relative orientation of the electronic devices.
[0080] The electronic devices may share information about their computed poses via the wireless communication interface. For example, an electronic device may i) make measurements of the one or more magnetic fields generated by another electronic device, and then processes those measurements to compute an estimated pose, and then ii) share this computed estimated pose with other electronic devices using a wireless communication network. Then iii) these other electronic devices may use these pose estimates to help them determine their own pose.
[0081] Generally, the magnetic field or fields generated by the transmitter electronic device may be time-varying magnetic fields having a predetermined frequency. In some embodiments, at least a first electronic device of the plurality of electronic devices is configured to measure ambient electromagnetic signals at a time where the at least one transmitter electronic device are not generating a magnetic field (in particular when no transmitter electronic device of the plurality of electronic devices is generating a magnetic field); wherein the first electronic device comprises a processing unit configured to select a target frequency based on the measured ambient electromagnetic signals and to communicate the selected target frequency to the other electronic devices of said plurality of electronic devices; and wherein the transmitter electronic device is configured to generate the magnetic field as a time-varying magnetic field at said communicated target frequency and wherein the at least one receiver electronic device is configured to measure the magnetic field at said communicated target frequency.
[0082] Each electronic device may include a transmitter or other signal generator coupled to the one or more electromagnetic coils for causing the one or more electromagnetic coils to generate the magnetic field. The electronic device may include a receiver circuit coupled to the one or more electromagnetic coils for sensing a magnetic field generated by another electronic device. The transmitter and receiver may be formed as separate circuits or as a single combined circuit, e.g. as a transceiver circuit. The transmitter and / or receiver circuit may be implemented at least in part by the processing unit and / or at least in part by one or more separate electronic circuits.
[0083] The electronic devices may use the sensed pose information to detect user interactions with the electronic devices and to determine interactive responses to the detected user interactions, e.g. as part of a play experience executed by the electronic devices.
[0084] In some embodiments, at least a first receiver electronic device is configured to: measure magnetic fields generated by a first and a second transmitter electronic device of the plurality of electronic devices; receive disambiguated pose information about a relative pose of the first and second transmitter electronic devices relative to each other; compute respective disambiguated poses of the first receiver electronic device relative to said first and second transmitter electronic devices based at least on the measured magnetic fields and on the received disambiguated pose information. Accordingly, the disambiguation information created by one electronic device may be used by another electronic device for disambiguating its own relative pose, i.e. disambiguation information may be propagated within the system of electronic devices. Accordingly, the reliability of the pose determination is increased, which may be particularly useful when some electronic devices are so far apart from each other that the measurement of the gradient information becomes relatively noisy.
[0085] In some embodiments, the at least one transmitter electronic device is configured to obtain, e.g. to receive, retrieve from internal memory or determine, one or more field calibration values of the generated at least one magnetic field and to communicate the obtained one or more field calibration values to the at least one receiver electronic device; and wherein the processing unit of the at least one receiver electronic device is configured to compute the pose of the receiver electronic device relative to the transmitter electronic device based on at least the received one or more field calibration values. The field calibration values may be static values that are determined at manufacturing time and stored in the electronic device. Alternatively or additionally, the field calibration values may include dynamic values that may depend on factors like the battery level of the electronic device, operating temperature and / or the like. Such dynamic values may be determined by the electronic device during operation.
[0086] In some embodiments, the at least one receiver electronic device is configured to communicate a computed pose of the receiver electronic device to the transmitter electronic device (relative to which the receiver electronic device has computed its pose) and / or to one or more other electronic devices of the plurality of electronic devices. Accordingly, the plurality of electronic devices may cooperatively measure relative pose information and share the measured information with each other, thereby allowing the system to determine relative poses of multiple electronic devices in an efficient manner. This in turn allows the measurements to be made relatively frequently.
[0087] In some embodiments, the electronic devices are configured to wirelessly communicate with each other via a wireless communications network. Accordingly, in some embodiments, each electronic device comprises a wireless communications interface configured for wireless communication via a wireless communications network and wherein the electronic devices are configured to wirelessly communicate with each other via said wireless communications network. The wireless communication may be based on a suitable radio-frequency communication technology, e.g. using the Bluetooth protocol, Bluetooth Low-energy, ZigBee, Wifi, and / or the like. In some embodiments, the wireless communication may comprise broadcasting one or more messages by one or more of the electronic devices via the wireless communications network to the other electronic devices of the plurality of electronic devices. The wireless communications interface may comprise a radio transceiver, e.g. a Bluetooth chip or circuit, and an associated antenna, in particular an antenna different from the one or more electromagnetic coils.
[0088] The wireless communication between the electronic devices may be a short-range wireless communication having a communications range of e.g. less than 100 m, such as less than 10 m. Nevertheless, the communications range is preferably larger than the effective range of the pose sensing operation. In some embodiments, the communications range may be at least 2 m, such as at least 3 m, such as at least 5 m.
[0089] The electronic devices may use the wireless communications network to coordinate operation of the system of electronic devices. For example, the plurality of electronic devices may be configured to share, via said wireless communications network, calibration parameters and / or a timing schedule for scheduling the generation of magnetic fields by one or more respective transmitter electronic devices. Alternatively or additionally, the plurality of electronic devices may be configured to share measured distance information and / or further pose information, optionally including confidence information about the shared pose information, via the wireless communication.
[0090] The wireless communications network may be an open network in the sense that electronic devices may join and leave the network at any time, i.e. such that the number of electronic devices within the communications network may vary over time, optionally up to a predetermined maximum number of electronic devices. The wireless communications network may be closed in the sense that only electronic devices of the system of electronic devices may join and communicate via the wireless communications network. The latter may e.g. be enforced by requiring a predetermined identifier, address, key, encryption scheme and / or the like.
[0091] The timing schedule may be shared by at least a subset, or even all, of the electronic devices, e.g. by all electronic devices currently communicating with each other via the same wireless communications network. In some embodiments, the plurality of electronic devices comprise a scheduler device configured to create the shared timing schedule and to distribute / broadcast the shared timing schedule to the other electronic devices of the plurality of electronic devices, e.g. via said wireless communications network. In particular, in some embodiments, at least one of the electronic devices is configured to create a timing schedule and to communicate the created timing schedule to the at least one receiver electronic device and / or the at least one transmitter electronic device; the at least one transmitter electronic device may be configured to temporarily generate the magnetic field according to the communicated timing schedule; and the at least one receiver electronic device may be configured to measure the generated magnetic field based on the communicated timing schedule.
[0092] It will be appreciated that the shared timing schedule may be created in a variety of ways. For example, the schedule may be predetermined and / or static, or it may be dynamic, e.g. adapted to the number of electronic devices currently participating in the wireless communications network, adapted to a currently executed play experience, etc.
[0093] In some embodiments, the shared timing schedule includes reserved times for performing one or more different types of contactless sensing operations. For example, in some embodiments, the shared timing schedule includes respective times reserved for wireless tag sensing operations and respective reserved times for performing the magnetic pose sensing operations.
[0094] Sharing the timing schedule among the electronic devices allows each electronic device to perform cooperative contactless sensing operations, in particular the pose measurements described herein, in a synchronized manner.
[0095] The processing unit of the electronic device may be implemented as any suitable processing circuitry or device, e.g. as one or more suitably programmed microprocessor or other processing device, e.g. as one or more ASICs, one or more microcontrollers, and / or one or more other suitable processing units, or combinations thereof.
[0096] In some embodiments, the electronic device is a wireless interactive toy, in particular a wireless interactive toy construction element. Accordingly, according to another aspect, the present disclosure relates to a toy construction system comprising a plurality of wireless interactive toy construction elements disclosed herein. The toy construction system may include additional toy construction elements, including toy construction elements that do not perform any electronic functions, such as conventional toy construction elements.
[0097] Each toy construction element of the toy construction system and, in particular, each wireless interactive toy construction element, may comprise coupling members configured to engage coupling members of other toy construction elements of the toy construction system so as to detachably attach the toy construction elements to each other. To this end, the coupling members may utilize different coupling mechanisms, e.g. based on frictional engagement of the coupling members with each other, based on screws, plug-and-socket connections or other forms of mating engagements of cooperating coupling members.
[0098] Hence, toy construction elements that have been interconnected with each other by means of the coupling members can again be disconnected from each other such that they can be interconnected again with each other or with other toy construction elements of the system, e.g. so as to form a different spatial structure. In some embodiments, the toy construction elements are provided with a first and a second type of coupling members, such as coupling pegs and peg-receiving recesses for frictionally engaging the pegs, or other pairs of mating or otherwise complementary coupling members configured to engage each other so as to form a physical connection. One type of coupling members may be located on one side, e.g. the top side, of the toy construction element while another, complementary type of coupling members may be located on an opposite side, e.g. the bottom side, of the toy construction element. In some embodiments, the toy construction elements include pegs extending from the top face of the toy construction element and corresponding peg-receiving cavities extending into the bottom face of the toy construction element for frictionally engaging the pegs by a suitable clamping force.
[0099] Generally, the toy construction system may impose limitations on the degrees of freedom of how the toy construction elements may be attached to each other, e.g. by limiting the possible relative positions and / or orientations at which they can be attached to each other. These limitations may facilitate the detection of relative poses of electronic toy construction elements within a toy construction model. To this end, the coupling members may be positioned on grid points of a regular grid; in particular, the coupling members of the toy construction elements may be arranged such that the coupling members of a set of mutually interconnected toy construction elements are positioned on grid points of a three-dimensional regular grid. The dimensions of the toy construction elements may be defined as integer multiples of a unit length defined by the regular grid. It will be understood that a three-dimensional grid may be defined by a single unit length, by two unit lengths, e.g. one unit length applicable in two spatial dimensions while the other unit length is applicable in the third spatial dimension. Yet alternatively, the three-dimensional grid may define three unit lengths, one for each spatial dimension.
[0100] In some embodiments, the toy construction elements are made from plastics material, e.g. thermoplastic polymers, or from another suitable material. The toy construction elements and, in particular, at least a housing of the wireless interactive toy construction elements may e.g. be made by an injection moulding process or by another suitable manufacturing process.
[0101] Each wireless interactive toy construction element may comprise a housing. Each wireless interactive toy construction element may include a function device and / or a sensor system accommodated within said housing. Each wireless interactive toy construction element may include a processing unit accommodated within said housing. Each wireless interactive toy construction element may include said one or more electromagnetic coils within said housing. Each wireless interactive toy construction element may include a wireless communications interface accommodated within said housing. Each wireless interactive toy construction element may include a power source, e.g. a battery, accommodated within said housing, the power source may be rechargeable. The housing may be box-shaped or have a different suitable shape. The housing may define a top face and a bottom face, opposite the top face. At least some of the coupling members may extend from the top face. The housing may further comprise one or more side faces extending between the top and bottom faces. In some embodiments all wireless interactive toy construction elements are configured to be interchangeably and detachably connectable to other toy construction elements of the toy construction system.
[0102] Embodiments of the system described herein provide a distributed control for coordinating the contactless sensing operations. The distributed control requires minimal or even no user-configuration, is robust, reliable and yet efficient.BRIEF DESCRIPTION OF THE DRAWINGS
[0103] Embodiments of the various aspects disclosed herein will be described in more detail in connection with the appended drawings, in which FIG. 1shows an example of an electronic device in the form of a wireless interactive toy construction element; FIG. 2shows a schematic block diagram of an example of an electronic device formed as a wireless interactive toy construction element; FIG. 3illustrates two wireless interactive toy construction elements; FIG. 4schematically shows an exploded view of an example of an electronic device in the form of a wireless interactive toy construction element; FIG. 5illustrates a schematic view of an embodiment of an electronic device; FIG. 6schematically illustrates a magnetic timing sequence; FIG. 7shows a schematic flow diagram of an example of a pose measurement process; FIG. 8schematically illustrates a system of two electronic devices with two candidate poses; FIGs. 9A-Bschematically illustrate an electromagnetic coil with a central measurement node; FIGs. 10A-Billustrate an example of pose tracking; FIG. 11schematically illustrates another example of pose tracking, including disambiguation tracking; FIG. 12illustrates an example of how such received pose information may be utilised by an electronic device; FIG. 13schematically illustrates a system of cooperating electronic devices; FIG. 14shows a schematic flow diagram of handling spatial parameter requests as a service by cooperating electronic devices. DETAILED DESCRIPTION
[0104] Various aspects and embodiments of an interactive toy system comprising a plurality of interactive wireless toys will now be described with reference to toy construction elements in the form of bricks. In this particular and corresponding embodiments, the electronic devices are each formed as a respective wireless interactive toy construction element, which each has a housing that is generally shaped as an orthogonal polyhedron with flat side faces and having coupling members extending from its upper surface and a cavity extending into its bottom surface. However other shapes and sizes of wireless interactive toy construction elements may be used, e.g. box-shaped or tile-shaped toy construction elements of different dimensions and with different numbers of coupling members. Moreover, while the brick-shape has proven to be particularly useful, the various aspects disclosed herein may be applied to other forms of electronic devices, including other forms of toy construction elements for use in play applications, educational applications, and / or the like.
[0105] FIG. 1 shows an example of an electronic device in the form of a wireless interactive toy construction element, generally designated 100. On the left-hand side of FIG. 1 the wireless interactive toy construction element is shown with its top surface visible and, on the right-hand side, the wireless interactive toy construction element is shown with its bottom side visible. In particular, the wireless interactive toy construction element comprises a generally box-shaped housing 101 with coupling pegs 104 extending from its top surface and with a cavity extending into the element from the bottom. The cavity is defined by side walls 102 and by a central, downwardly extending tube 103. The coupling pegs of another toy construction element can be received in the cavity in a frictional engagement, e.g. as disclosed in US 3 005 282. The construction elements shown in the remaining figures have this known type of coupling members in the form of cooperating pegs and cavities. However, other types of coupling members may also be used in addition to, or instead of, the pegs and cavities. The coupling pegs are arranged across the top surface in a square planar grid, i.e. defining orthogonal directions along which sequences of coupling pegs are arranged. The distance between neighboring coupling pegs is uniform and equal in both directions. This or similar arrangements of coupling members at coupling locations defining a regular planar grid allow the toy construction elements to be interconnected in a discrete number of positions and orientations relative to each other, in particular at right angles with respect to each other. In an assembled toy construction model, the coupling members of multiple toy construction elements may thus be located on grid points of a three-dimensional grid defined relative to the toy construction model.
[0106] In some embodiments, the housing 101 of the wireless interactive toy construction element is made from plastics material, e.g. thermoplastic polymers, or from another suitable material. The housing may e.g. be made by an injection molding process or by another suitable manufacturing process.
[0107] The wireless interactive toy construction element 100 may include a function device 105 accommodated within the housing 101 of the wireless interactive toy construction element. Generally, a function device may be any suitable device for performing a function, such as a function that provides a user-perceptible effect, such as a visible and / or audible effect. Examples of function devices may include any suitable mechanical and / or electrical device, arrangement, and / or circuitry adapted to perform one or more mechanical and / or electrical functions.
[0108] Examples of a mechanical function include driving a rotatable output shaft, winding-up a string or a chain which enables pulling an object closer to a toy module, moving a hinged part of the wireless interactive toy construction element, etc. The mechanical function may thus enable opening or closing a door, ejecting an object, rotating a turntable, moving a linear actuator, etc. Such mechanical motions can be driven by an electric motor.
[0109] Examples of an electrical function include emitting constant or blinking light, activating several lamps in a predetermined sequence, emitting audible sound such as beep, alarm, bell, siren, voice message, music, synthetic sound, natural or imitated sound simulating and / or stimulating play activities, playback of a sound, and / or other audio content, etc.
[0110] Accordingly, the function device may be selected from a motor, a light source (e.g. one or more LEDs), a sound source (e.g. a loudspeaker). The function wireless toy construction element may include more than one function device. In some embodiments the system includes different types of wireless interactive toy construction elements comprising, respectively, different types of function devices.
[0111] The wireless interactive toy construction element 100 includes a sensor system 108 comprising one or more sensors accommodated within the housing 101 of the wireless interactive toy construction element. The sensor system 108 may include one or more sensors, e.g. including the magnetic pose sensor described herein and / or one or more other sensors, e.g. a linear or rotary encoder, a light detector, and a sound detector (e.g. a microphone), an accelerometer, and / or the like.
[0112] FIG. 2 shows a schematic block diagram of an example of an electronic device formed as a wireless interactive toy construction element, generally designated 100, e.g. of the wireless interactive toy construction element shown in FIG. 1.
[0113] The wireless interactive toy construction element 100 comprises a housing 101 defining a top face which is provided with coupling members 104, all as described above with reference to FIG.1. The wireless interactive toy construction element 100 further comprises a processing unit 209 and one or more electromagnetic coils 207, all accommodated within the housing 101. The wireless interactive toy construction element may further comprise, also accommodated within the housing 101, one or more additional sensors 108 and / or one or more function devices 105, e.g. as described in connection with the sensor system and function device of the embodiment of FIG. 1 above. The wireless interactive toy construction element 100 further comprises a rechargeable battery 210 or other rechargeable power source, and a wireless communications interface 211, also accommodated within the housing 101.
[0114] Each electromagnetic coil 207 defines a coil axis around which the electromagnetic coil extends. In this example, the electromagnetic coils 207 are arranged such that one coil is arranged with its coil axis extending out of the top face of the housing while another other coil is arranged with its coil axis extending out of one of the side faces of the housing. A third electromagnetic coil (not explicitly shown in FIG. 2) may be arranged with its coil axis extending out of another side face of the housing, e.g. such that the axes of all three coils are oriented orthogonal to each other or at least linearly independent from each other. In this manner, the wireless interactive toy construction element may use the electromagnetic coils to detect pose coordinates of another wireless interactive toy construction element having similar arrangements of electromagnetic coils as described herein. It will be appreciated, however, that other arrangements of coils are possible. In some embodiments, the wireless interactive toy may use one or more of the electromagnetic coils to perform additional functions, e.g. to detect, identify and / or capture data from a wireless tag. Yet further, the wireless interactive toy construction element may receive electrical energy via one or more of the electromagnetic coils 207 for charging the battery 210, which in turn powers the processing unit 209, the function device 105 and the wireless communications interface 211.
[0115] The electromagnetic coils 207 are operable to detect the pose of the wireless interactive toy construction element 100 relative to another wireless interactive toy construction element. To this end, the wireless interactive toy construction element 100 may be configured to use the electromagnetic coils 207 to measure the components (along one or more directions, including a relative phase) of one or more magnetic fields generated by corresponding coils of another wireless interactive toy construction element of the system. In this manner, based on measurements by the electromagnetic coils 207, the wireless interactive toy construction element may detect its own relative pose relative to another wireless interactive toy construction element. Moreover, by monitoring the pose over time, the wireless interactive toy construction element may detect whether the wireless interactive toy construction elements move or are stationary relative to each other. In this manner, the wireless interactive toy construction element may determine which other wireless interactive toy construction elements are mutually interconnected within a coherent toy construction model. The wireless interactive toy construction element may further be configured to energize one or more of the electromagnetic coils to allow one or more other wireless interactive toy construction elements to measure the the thus generated one or more magnetic fields and to base a corresponding pose determination on these measurements.
[0116] The one or more function devices 105 may include a light source, e.g. an LED, a loudspeaker, a motor, and / or another function device operable to perform a user-perceivable function.
[0117] The one or more sensors 108 may include a light sensor, a sound sensor, a rotational encoder, an accelerometer, a gyro, and / or any other suitable sensor, e.g. as described in connection with FIG. 1.
[0118] The wireless communications interface 211 may comprise a radio-frequency transceiver and an associated antenna. In some embodiments, the wireless communications interface may comprise a Bluetooth chip or circuit or another form of radio-frequency transceiver adapted for communication via a wireless communications network, e.g. using known low-power, short-range wireless networking technology, such as Bluetooth / Bluetooth Low Energy, ZigBee, Z-Wave, or a similar wireless technology for low-power personal area data networking in compliance with a standardized protocol. The wireless communications interface 211 may be operable for two-way communication with other wireless interactive toy construction elements of the system. Accordingly, the wireless interactive toy construction element may be operable to communicate its identity and / or operational characteristics, calibration parameters, scheduling information, pose information and / or the like.
[0119] The processing unit 209 may comprise one or more microcontrollers, one or more microprocessors, one or more ASICs, and / or one or more other suitable processing units, or combinations thereof. The processing unit 209 may be configured to control the functional behavior of the wireless interactive toy construction element. The wireless interactive toy construction element may be provided, e.g. pre-programmed, with a default behaviour, e.g. with default executable instructions stored in a memory of the wireless interactive toy construction element and executable by the processing unit of the wireless interactive toy construction element. The default executable instructions may define a set of predetermined rules for reacting to external stimuli as sensed by the sensor system, in particular for reacting to detected poses relative to other wireless interactive toy construction elements or to detected changes in such poses. In some embodiments, the behaviour of the wireless interactive toy construction element may be programmed or configured by the user, e.g. by receiving program data and / or configuration parameters. To this end, the wireless interactive toy construction element may receive program and / or control data and / or configuration parameters from a computer or from another external electronic device, e.g. directly or via another toy module of the system. An external electronic device may e.g. be or comprise a desktop computer, a tablet computer, a smartphone, a laptop computer, or another programmable computing device. Alternatively or additionally, the wireless interactive toy construction element may capture program and / or control data and / or configuration parameters from a wireless tag, e.g. from an RFID tag, or from other data storage devices. For example, the wireless interactive toy construction element may be operable to read out such wireless tag or other data storage device in a contactless manner as described herein.
[0120] FIG. 3 illustrates two wireless interactive toy construction elements 300A and 300B, respectively, e.g. wireless interactive toy construction elements of the type shown in FIGs. 1 and 2. The wireless interactive toy construction elements 300A and 300B may include the same type of function device or different types of function devices. The wireless interactive toy construction element 300A is operable to detect the pose of the wireless interactive toy construction element 300A relative to an internal coordinate system of the wireless interactive toy construction element 300B. The relative position may be defined as a 3D displacement 312 of an internal coordinate system 311A of the wireless interactive toy construction element 300A relative to an internal coordinate system 311B of the wireless interactive toy construction element 300B. The relative orientation may be defined in different ways, e.g. by a quaternion or by three angles defining the rotation of the internal coordinate system 311A of the wireless interactive toy construction element 300A relative to the internal coordinate system 311B of the wireless interactive toy construction element 300B, or otherwise. The angles may thus describe a pitch, yaw and roll, respectively, of the element 300A relative to the element 300B. Similarly, wireless interactive toy construction element 300B may detect its own corresponding pose relative to element 300A.
[0121] One or both of the wireless interactive toy construction elements 300A and 300B may be operable, at predetermined times, to activate their one or more electromagnetic coils to emit one or more magnetic fields. When wireless interactive toy construction element 300B emits a magnetic field, wireless interactive toy construction element 300A may use its respective electromagnetic coils to receive and sense the emitted magnetic field and to perform magnetic field measurements. Based on the measurements of the emitted magnetic fields, the receiving wireless interactive toy construction element 300A may compute its own pose relative to the currently wireless interactive toy construction element 300B that currently generates the magnetic field, as described herein. While FIG. 3 only illustrates two wireless interactive toy construction elements, it will be appreciated that a system may include more than two wireless interactive toy construction elements. When one of the wireless interactive toy construction elements emits a magnetic field, the other wireless interactive toy construction elements may use their respective electromagnetic coils to receive and sense the emitted magnetic field and to perform respective magnetic field measurements and pose determinations.
[0122] It will be appreciated that a three-dimensional positioning may require that each wireless interactive toy element includes three coils that are preferably arranged orthogonally relative to each other.
[0123] The wireless interactive toy construction elements 300A-B may further be operable to communicate with each other via a wireless communications network, e.g. by broadcasting messages to all other wireless interactive toy construction elements of the system or otherwise. When the wireless interactive toy construction elements are operable to communicate with each other, they may share a timing schedule defining which wireless interactive toy construction elements are to emit a magnetic field at what times. The wireless interactive toy construction elements may further exchange information about their identity and / or the type of function device they include and / or about their respective operational states, calibration parameters, the measurements they desire to make, etc.
[0124] The wireless interactive toy construction elements may broadcast their determined poses via the wireless communications network to the remaining wireless interactive toy construction elements, including to the wireless interactive toy construction element that is currently transmitting a magnetic field. The role of the transmitting wireless toy construction element (i.e. the role of the wireless toy construction element currently emitting the one or more magnetic fields) may selectively be assigned to respective ones of the wireless interactive toy construction elements, e.g. cyclically or according to another timing schedule, or otherwise. The timing schedule may be created by a selected one of the wireless interactive toy construction elements, also referred as the scheduler device. The scheduler device may broadcast the created timing schedule to the other wireless interactive toy construction elements or share the created schedule with the other wireless interactive toy construction elements in another manner. Accordingly, after a number of wireless interactive toy construction elements have generated respective magnetic fields, the wireless interactive toy construction elements are capable of obtaining distance and, optionally even directional and / or orientational pose information relative to some, if not all, of the other wireless interactive toy construction elements of the system.
[0125] The thus obtained pose information may be used by the wireless interactive toy construction elements in a variety of ways. For example, based on their relative distance and orientation, the wireless interactive toy construction element 300A may determine whether it is physically connected to wireless interactive toy construction element 300B, e.g. as part of the same coherent toy construction model - or at least whether it is likely to be thus interconnected. To this end, the wireless interactive toy construction element 300A may determine whether the measured pose is consistent with the limitations imposed by the toy construction system. Alternatively, or additionally, the wireless interactive toy construction element may monitor the relative pose over a period of time. If the relative pose remains constant, the wireless interactive toy construction element may determine that the elements are indeed physically interconnected.
[0126] In particular, the wireless interactive toy construction element 300A may monitor the pose relative to element 300B during a period where wireless toy construction element 300A detects changes of its own motion, e.g. using an IMU or accelerometer of the wireless interactive toy construction element. If the relative pose between the elements 300A and 300B remains unchanged during such detected motion, one or each of the elements may determine that it is physically interconnected with the respective other element.
[0127] Alternatively, or additionally, one or both of the wireless interactive toy construction elements may implement a play experience based at least in part on the detected pose, e.g. based on detected proximities, detected relative movements, orientations, and / or the like.
[0128] FIG. 4 shows an exploded view of an example of an electronic device in the form of a wireless interactive toy construction element, generally designated 100. The wireless interactive toy construction element 100 is similar to the embodiments of FIGs. 1 and 2, in that it comprises a housing, a processing unit and one or more electromagnetic coils, all accommodated within the housing. As will be described below, the wireless interactive toy construction element 100 further comprises additional sensors and function devices, a rechargeable battery 210 (or other rechargeable power source) and a wireless communications interface 211, all as described in connection with one or more of the previous embodiments.
[0129] In the example of FIG. 4 the housing of the wireless interactive toy construction element 100 comprises an upwardly open bottom housing member 101A and a lid member 101B. The lid member 101B is configured to be connected to the bottom housing member 101A and, when connected, to close the opening of the bottom housing member, such that the bottom housing member and the lid member together form a housing accommodating the remaining components of the wireless interactive toy construction element. The lid member may be fastened to the bottom housing member in any suitable manner, e.g. by a snap fit connection, by welding, gluing or otherwise.
[0130] In some embodiments, the bottom housing member 101A and the lid member 101B are made from plastics material, e.g. thermoplastic polymers, or from another suitable material. The housing may e.g. be made by an injection molding process or by another suitable manufacturing process. In the present example, the lid member is made from a transparent material.
[0131] The housing is box-shaped as described in connection with FIG. 1. The top face of the lid member includes coupling pegs 104 and the bottom face of the bottom housing member comprises corresponding cavities (not shown in FIG. 4), also as described in connection with FIG. 1.
[0132] The wireless interactive toy construction element 100 includes a rechargeable battery 210 (or other rechargeable power source) accommodated in the bottom housing member 101A.
[0133] The wireless interactive toy construction element 100 includes a support frame 420, a printed circuit board (PCB) 430 and three pairs of electromagnetic coils 207X, 207Y and 207Z, respectively. The PCB 430 is attached to the support frame 420 and the coils are wound around the frame 420 such that the support frame, the coils and the PCB form a core assembly 42 accommodated inside the bottom housing member 101A.
[0134] The wireless interactive toy construction element 100 further comprises an antenna 413 and a loudspeaker 415, both mounted to the lid member 101B so as to form a top assembly 41. The top assembly 41 further comprises a coloured insert 412 sandwiched between the lid member 101B and the antenna 413. The top assembly 41 further comprises an opaque insert 414 sandwiched between the antenna 413 and the loudspeaker 415. The inserts 412 and 414 provide vents to facilitate highfrequency sound from the loudspeaker to pass, while obscuring the antenna and the other electronics components from view through the transparent lid member 101B. The PCB 430 is electrically connected to the antenna 413, the loudspeaker 415, the rechargeable battery 210, and to the coils of the coil pairs 207X, 207Y and 207Z. The PCB 430 further comprises a number of electronic components. In particular, the PCB comprises a processing unit 209, LEDs 431A and 431B, a side-firing LED 438, a microphone 432, a coil interface circuit 433, a memory 435, an accelerometer 436 and a wireless communications circuit 437. The top assembly further comprises an acoustic seal 416 positioned on top of the microphone 432.
[0135] The processing unit 209 may be implemented as a single processing unit or as a combination of multiple components, e.g. as an ASIC and a microprocessor, or otherwise. The processing unit 209 is coupled to the coils of the coil pairs 207X, 207Y and 207Z via the coil interface circuit 433. The coil interface circuit 433 may be implemented as a single unit or as multiple components. At least part of the coil interface circuit may be implemented by an ASIC, e.g. an ASIC that also implements a part of the processing unit 209. The processing unit 209 is further coupled to the remaining electronic components and implements suitable drivers and / or other control functionality for operating the respective electronic components. The wireless communications circuit 437 is coupled to the antenna 413 and to the processing unit 209 for implementing a suitable wireless communications protocol, such as based on Bluetooth Low Energy (BLE) or otherwise. The communications circuit 437, or at least a part thereof, may be integrated with the processing unit 209 as schematically illustrated in FIG. 4.
[0136] The LEDs 431A and 431B are arranged at respective opposite ends of the PCB such that the light from the LEDs is visible through the transparent lid member 101B. The side-firing LED 438 is arranged at one end of the PCB such that the light from the side-firing LED is output through a side portion 1011 of the transparent lid member 101B.
[0137] As will be described in more detail below, the electromagnetic coils are arranged as three pairs 207X, 207Y and 207Z of respective coils such that the coils of each pair are arranged coaxially with each other and orthogonal to the coils of the other pairs. In particular, coil pair 207X comprises coils 207XA and 207XB, while coil pair 207Y comprises coils 207YA and 207YB, and coil pair 207Z comprises coils 207ZA and 207ZB. The coils of each coil pair are arranged spaced apart from each other, proximal to respective opposite sides of the box-shaped housing. The coils of each coil pair are electrically connected in series with each other, in particular such that they are cumulatively coupled.
[0138] FIG. 5 illustrates a schematic view of an embodiment of an electronic device. The electronic device, generally designated 100 may be a wireless interactive toy construction element, e.g. as described in connection with any of FIGs. 1, 2 or 4. Alternatively, the electronic device may be a different type of electronic device.
[0139] The electronic device 100 comprises one or more electromagnetic coils for measuring at least one magnetic field generated by an electromagnetic coil of another electronic device. In the example of FIG. 5, the electronic device comprises three pairs 207X, 207Y and 207Z of electromagnetic coils, each comprising two series-connected coils. In particular, one pair of coils 207X comprises coils 207XA and 207XB, while another pair of coils 207Y comprises coils 207YA and 207YB, and a third pair of coils 207Z comprises coils 207ZA and 207ZB.
[0140] The electronic device 100 further comprises a processing unit 209 for computing a pose of the electronic device relative to said other electronic device.
[0141] Each of the pairs of electromagnetic coils is configured for measuring gradient information of said measured magnetic field and the processing unit is configured to compute the pose based on the measurement(s) of the magnetic field and based on the measured gradient information.
[0142] To this end, each of the pairs of electromagnetic coils defines two measurement nodes at respective locations along the series-connected pair of coils. In FIG. 5, one of the measurement nodes of each coil pair is designated 501X, 501Y and 501Z, respectively, while the other measurement node of each coil pair is designated 502X, 502Y and 502Z, respectively. In particular, measurement nodes 501X, 501Y and 501Z are arranged at one end of the corresponding coil pair. Each coil pair further defines an additional measurement node 502X, 502Y, 502Z, respectively, at the center between the two coils of the corresponding pair. The electronic device 100 comprises a magnetic field measurement circuit 4332 configured to measure electric signals at respective ones of the measurement nodes of each of the coil pairs, in particular between each of the measurement nodes of each coil pair and a reference node and / or to ground. To this end, in the example of FIG. 5, the nodes 503-X, 503-Y and 503-Z may be connected to ground, e.g. via suitable capacitors or otherwise..
[0143] The electronic device 100 comprises a coil interface circuit 433 electrically coupled to the pairs of electromagnetic coils 207X, 207Y and 207Z and electrically coupled to the processing unit 209. The coil interface circuit 433 comprises the magnetic field measurement circuit 4332 and a magnetic field generation circuit 4331. While illustrated as separate blocks in FIG. 5, it will be appreciated that the coil interface circuit and the processing unit may partly or completely be integrated with each other. For example, a part of the magnetic field measurement circuit 4332 and / or of the magnetic field generation circuit 4331 may be implemented by the processing unit 209, or otherwise.
[0144] The magnetic field generation circuit 4331 may be configured to apply a transmission signal to the measurement node 501X, 501Y, 501Z, respectively, of one or more of the electromagnetic coil pairs so as to cause the corresponding one or more coils to create a magnetic field that can be measured by another electronic device.
[0145] In particular, the processing unit 209 may selectively operate the electronic device in transmission mode or in receiving mode. When operated in transmission mode, the electronic device may control the electromagnetic coils to generate one or more magnetic fields so as to allow other electronic devices to perform magnetic field measurements. When operated in receiving mode, the electronic device may measure electric signals (e.g. voltages) at respective ones of the measurement nodes of each of the coil pairs so as to perform magnetic field measurements of at least one magnetic field generated by another electronic device.
[0146] In particular, when operated in transmission mode, the electronic device may sequentially activate its electromagnetic coils one pair at a time so as to transmit magnetic fields, in particular a sequence of distinct magnetic fields. When operated in receiving mode, the electronic device may concurrently or sequentially probe its pairs of electromagnetic coils so as to measure magnetic field components along respective directions. A possible timing sequence for the transmission and measurement cycles will be described in connection with FIG. 6.
[0147] FIG. 6 schematically illustrates a magnetic timing sequence. In particular, the upper part of FIG. 6 shows an example of a transmission sequence of a transmitter electronic device, in particular an electronic device operating in transmission mode. The transmitter electronic device generates magnetic fields during a transmission cycle 200. During the transmission cycle, the transmitter electronic device sequentially activates its electromagnetic coils (or coil pairs) one at a time. For example, during an initial transmission time slot 101, the transmitter electronic device may activate its electromagnetic coil pair 207X. During a subsequent transmission time slot 102, the transmitter electronic device may activate its electromagnetic coil pair 207Y. During a third transmission time slot 103, the transmitter electronic device may activate its electromagnetic coil pair 207Z. It will be appreciated, however, that the transmitter electronic device may activate its electromagnetic coils in a different order.
[0148] The lower part of FIG. 6 shows an example of a reception sequence of a receiver electronic device, in particular an electronic device operating in receiving mode. The receiver electronic device synchronizes its measurements with the transmission cycle of the transmitter electronic device. To this end, the electronic devices may use wireless communication via a wireless communications network to synchronize the magnetic field measurements. For example, the electronic devices may establish a shared timing schedule that defines which electronic device is to operate in transmission mode at what time and / or they may synchronise their respective clocks.
[0149] The shared timing schedule may be used in a cyclic manner by repeating the schedule multiple times, e.g. until a new shared timing schedule has been created and distributed to the electronic devices. Alternatively, new schedules may be created and distributed such that a new schedule is used by the electronic devices each time a current timing schedule has been completed. Creating new timing schedules may allow the system to adapt the timing schedule to new situations, e.g. to electronic devices being turned OFF or otherwise leaving the wireless communication network, or to new electronic devices joining the wireless communications network, or by a change in the needed rate of pose sensing operations of one or more of the electronic devices. For the purpose of executing the timing schedule, the electronic devices may perform a suitable synchronization scheme, e.g. using broadcast signals of the wireless communications network. The shared timing schedule assigns reserved times as reserved time intervals for respective operations.
[0150] The shared timing schedule reserves pose sensing times, and assigns a transmitter role to a respective electronic device for each of the reserved pose sensing times. The upper portion of FIG. 6 illustrates one of such reserved pose sensing times. During a reserved pose sensing time, the electronic device designated as transmitter electronic device emits a sequence of magnetic fields to allow the remaining electronic devices to operate as receiver electronic devices and perform pose sensing operations of their respective poses relative to the currently transmitting electronic device. Accordingly, the pose sensing times are times reserved for cooperative magnetic sensing operations, i.e. magnetic sensing operations which involve a synchronized and concurrent operation of more than one, in particular all, electronic devices of the system. The times reserved for cooperative sensing operations thus serve to synchronize these operations and to assign different roles, in particular the transmitter role and, optionally, the receiver roles, to the respective electronic devices. In some embodiments, the duration of the pose sensing times may be selected long enough to allow the receiver electronic devices to perform the magnetic field measurements. In some embodiments the duration of the pose sensing times are between 5 ms and 20 ms, such as about 10 ms. The pose sensing times may be grouped such that they are assigned as a sequence of consecutive time intervals, during which respective electronic devices are designated as transmitter electronic devices, so as to allow different electronic devices to act as transmitter electronic devices and allow for an improved overall determination of the spatial configuration of the electronic devices. The pose sensing times, or groups thereof, may be sufficiently spaced apart to allow for reserved times for individual sensing operations, in particular tag sensing operations, to be allocated between them.
[0151] It will further be appreciated that the shared timing schedule may include alternative or additional times, e.g. for alternative or additional operations. It will further be appreciated that the shared timing schedule may be created in a variety of ways, e.g. as a static schedule or dynamically, e.g. adaptively depending on the specific situation.
[0152] In any event, during each of the transmission time slots 101, 102 and 103, respectively, the receiver electronic device may perform magnetic field measurements with each of its electromagnetic coils. For example, during transmission time slot 101, the receiver electronic device may initially measure the magnetic field during a reception time slot 301 using one of its coils or coil pairs, e.g. coil pair 207X, followed by a measurement during reception time slot 302 using another one of the coils or coil pairs, e.g. coil pair 207Y, and finally followed by a measurement during reception time slot 303 using the third one of the coils or coil pairs, e.g. coil pair 207Z. Again, it will be appreciated that the receiver electronic device may perform the measurements in a different order. During each of the subsequent transmission time slots 102 and 103, the receiver electronic device may again cycle through its coils or coil pairs and perform respective measurements. Accordingly, the entire measurement cycle has a duration 200 of at least three times the duration 100 of a transmission time slot. Each reception time slot may thus have a duration 300 of about a third of the duration 100 of a transmission time slot. Optionally, the reception time slots are slightly shorter than one third of the duration of the transmission time slots so as to allow for a buffer to account for possible de-synchronisations or clock drifts between the transmitter and the receiver electronic devices.
[0153] During each reception time slot, the receiver electronic device may obtain signals at the respective measurement nodes, as will be described in more detail below. Accordingly, for each of the three electromagnetic coils of the transmitter electronic device, the receiver electronic device performs field measurements using each of the electromagnetic coils of the receiver electronic device. During each field measurement for a given transmitting coil or transmitting coil pair and a given receiving coil or receiving coil pair, the receiver electronic device may measure the signal at the corresponding end node 501X, 501Y or 501Z of the receiving coil or coil pair and it may measure the signal at the center node 502X, 502Y or 502Z of said receiving coil or coil pair. Each signal measurement involves a measurement of the signal magnitude and phase. Preferably, for each receiving coil or coil pair, the measurements at the end nodes 501X, 501Y and 501Z, respectively, and the measurement at the corresponding mid-nodes 502X, 502Y, 502Z, respectively, are performed concurrently, thereby obtaining a more accurate phase determination. It will be appreciated that the receiver electronic device may perform each signal measurement multiple times so as to increase the overall signal to noise ratio.
[0154] The receiver electronic device may then determine the relative pose of the receiver electronic device relative to the transmitting electronic device based on the thus obtained signals.
[0155] FIG. 7 shows a schematic flow diagram of an example of a pose measurement process. The process may be performed by the processing unit of the receiver electronic device.
[0156] In initial step S1, the process performs field estimations of the measured magnetic field components along the coil axes of the respective coils or coil pairs. To this end the process performs phase coherent field measurements to estimate the respective field measurements. For each transmitting coil or coil pair and for each measurement node of each receiving coil or coil pair, the measurement process yields a number of, e.g. between 2 and 20, complex measurements. For each transmitting coil or coil pair and for each measurement node of each receiving coil or coil pair, the process processes these signal measurements to yield: an amplitude estimation, a phase estimation and, optionally, an estimate of the frequency drift between the local oscillators of the receiving and transmitter electronic devices.
[0157] To this end, for each transmitting coil or coil pair, and for each measurement node of each receiving coil or coil pair, the process may initially combine the number of measurements into one. The process may then perform a de-rotation in the complex plane and a time-align of the measurements by making use of an estimate of the frequency error between the transmitter electronic device and the receiver electronic device to obtain amplitude and phase estimates of the magnetic field. The process may optionally adjust the estimated amplitudes and / or phases for run-time variations of the electronic devices and or for manufacturing variations between different electronic devices that may influence the magnetic field generation of the transmitter electronic device and / or the field measurement of the receiver electronic device. Examples of run-time variations that may be compensated for include the current operating temperature of the receiving and / or transmitting brick, the battery voltage of the receiving and / or transmitter electronic device, and / or gain settings of the measurement circuit. Examples of static compensation factors may include one or more device-specific calibration parameters of the transmitting and / or receiver electronic device to compensate for possible variations between components used for manufacturing the respective devices. Such calibration parameters may e.g. be established by performing calibration measurements at manufacturing time and or based on suitable theoretic models. It will be appreciated that other embodiments may perform compensations for alternative or additional factors, or they may omit such compensations. For the purpose of applying compensations for run-time or static properties of the transmitter electronic device, the electronic devices may exchange information about their static and / or varying properties via the wireless communications network.
[0158] In step S2, the process performs a pose estimation to compute information about an estimated pose based on the field measurements obtained from the signals at the respective end nodes of the respective coils or coil pairs and / or from the signals at the mid nodes. The pose estimation may be based on any suitable pose estimation algorithm known as such in the art, e.g. the algorithm described in W. Kim, J. Song and F. C. Park, "Closed-Form Position and Orientation Estimation for a Three-Axis Electromagnetic Tracking System," in IEEE Transactions on Industrial Electronics, vol. 65, no. 5, pp. 4331-4337, May 2018, doi: 10.1109 / TIE.2017.2760244.
[0159] The pose estimation based on the magnetic field measurements inherently results in ambiguous determination. In particular, the pose estimation results in two possible solutions that are both consistent with the measured magnetic fields (e.g. the field components, including their relative phases).
[0160] This is illustrated in FIG. 8, which schematically illustrates a system of two electronic devices 300A-B, respectively, e.g. the wireless interactive toy construction elements of FIG. 3. The electronic devices 300A-B may be electronic devices as described in connection with FIGs. 4 or 5. Each electronic device comprises three electromagnetic coils or three pairs of electromagnetic coils as described herein. In the example of FIG. 8, the electronic device 300A is performing a determination of its pose relative to electronic device 300B. Accordingly, electronic device 300B has transmitted respective magnetic fields via its respective electromagnetic coils and electronic device 300A has performed magnetic field measurements using its electromagnetic coils and performed a pose estimation of step S2 above. The pose estimation results in two solutions, namely the (correct) pose of device 300A relative to device 300B and a second solution, illustrated by the pose of device 300A relative to a (spurious) device designated by reference numeral 301B and shown in phantom in FIG. 8. Electronic device 300A cannot determine which of the two solutions corresponds to the actual pose of electronic device 300A based only on the measured magnetic field components using the three coils or coil pairs.
[0161] Again referring to FIG. 7, and with continued reference to FIG. 8, in order to disambiguate the pose estimation and determine an unambiguous pose, the process proceeds at step S3 and performs a pose disambiguation based on an estimate of the gradient of the magnetic field along the coil axes of the respective coils of the receiver electronic device. This approach is based on the realization that the two ambiguous poses of device 300A relative to device positions 300B and 301B result in different field gradients at the receiver electronic device 300A, i.e. a measurement of the gradient can be used to distinguish between the two possible solutions for the correct pose.
[0162] At least some embodiments disclosed herein use axially spaced apart coil segments of the receiving coils to perform at least an estimate of the magnetic field gradient. To this end, the receiver electronic device 300A measures signals at the central measurement nodes of its electromagnetic coil pairs.
[0163] An example of the use of the central measurement nodes of the receiving electromagnetic coil pairs for the pose disambiguation is illustrated in FIGs. 9A-B. Each of FIGs. 9A-B shows one of the pairs of receiving coils, in this example coil pair 207Z, of an embodiment of the electronic device, e.g. of electronic device 300A of FIGs. 3 or 8. As described above, the coil pair 207Z comprises two coils 207ZA and 207ZB, respectively, that are arranged coaxially to each other and axially spaced apart from each other. They are series-coupled to each other and define measurement nodes 501Z at one end of the coil pair 207Z, and an additional central measurement node 502Z arranged between the coils 207ZA and 207ZB. The processing unit 209 of the electronic device is configured to measure a first signal at the end node 501Z and a second signal at the central measurement node 502Z. In the example of FIGs. 9A-B, a reference node 503Z is connected to ground via a capacitor, and the processing unit is connected to end node 501Z and to central node 502Z to measure respective signals at these points relative to ground.
[0164] As discussed above, the measured signal, e.g. the measured voltage signal, at the end node 501Z is used in step S2 to compute the magnetic field component along the corresponding axis. In particular, the measured signal at the end node 501Z is indicative of the sum of the induced electromotive forces on both coils 207ZA and 207ZB. This signal is therefore useful for determining the magnetic field component B z along the coil axis, in this example the z-axis. The determined magnetic field components along the respective axes of all three coils are used to compute the ambiguous pose determined in step S2.
[0165] The measured signal e.g. the measured voltage signal, at the central node 502Z, in combination with the measured signal at end node 501Z, is indicative of the difference of the induced electromotive forces between the coils 207ZA and 207ZB. This is illustrated in FIGs. 9A and 9B. In FIG. 9A, the induced electromotive force is larger in the left coil 207ZA while, in FIG. 9B, the induced electromotive force is larger in the right coil 207ZB.
[0166] Hence, by measuring the amplitude and phase of the voltage at the end node and the central node, the processing unit 209 obtains sufficient information to estimate the field component along the coil axis and the first spatial derivative of the field component along the coil axis, e.g. B z and dB z / dz in the example of FIGs. 9A-B.
[0167] Whilst the magnetic fields measured for the two ambiguous poses of device 300A are identical, the signs of the spatial derivatives are not. It will be appreciated that the electronic device may perform corresponding measurements for the x- and y-directions using the other coil pairs.
[0168] Accordingly, while again referring to FIG. 7, at step S3, for each of the two ambiguous poses of device 300A that result from the pose estimation of step S2, the process calculates the corresponding expected gradient signals. The process then selects the pose whose measured gradient signals most closely match the computed expected gradient signals. Preferably, the measurements at the end nodes 501X-Z and at the corresponding centre nodes 502X-Z, respectively, are performed concurrently so as to obtain an accurate phase comparison.
[0169] When the measurements are performed with three receiving coils (or coil pairs) and when each receiving coil (or coil pair) performs respective measurements for three transmitting coils (or coil pairs) of the transmitter electronic device, the process may determine respective decision boundaries in the complex plane for each receiving coil or each pair of receiving coils. When, for a given receiving coil (or coil pair) and for a given transmitting coil (or coil pair), the measured signal lies on one side of the decision boundary, the measurement for the corresponding coils is indicative for one pose of the two ambiguous poses. Conversely, when the measured signal lies on the opposite side of the decision boundary, the measurement for said coils is indicative for the other pose of the two ambiguous poses. The process may further determine a confidence value for each pair of receiving and transmitting coils (or coil pairs), e.g. based on the distance of the measured signal from the decision boundary.
[0170] The process may then compute an accumulated / combined decision and corresponding confidence value based on the individual confidence values.
[0171] In some embodiments, the electronic device may repeatedly perform magnetic field measurements, e.g. at regular or irregular intervals, triggered by play events, upon request from another electronic device and / or the like. In some embodiments, the process may use information from previous magnetic field measurements for determining a current pose. To this end, in step S4, the process may perform pose tracking.
[0172] The pose tracking may comprise a tracking of the ambiguous pose as determined in step S2 and / or a tracking of the disambiguated pose based on the decision taken at step S3.
[0173] The pose tracking of the ambiguous pose may comprise the tracking of one or more pose parameters individually or as a combined multi-dimensional tracking. The pose tracking may apply one or more filters to the measured poses, thereby using past pose estimations to improve this pose estimation. The one or more filters may include one or more of the following filters: a filter for the determined range, a filter for the determined direction, a filter for the determined orientation.
[0174] In one embodiment, the process may compute a distribution of the prediction of the pose or for one or more coordinates of the pose. The computation of the distribution may be based on the most recent previous estimate and on the elapsed time since the most recent previous estimate. The computation may further be based on assumed or previously determined estimates of the typical speed of movement of the relative poses and / or on a determined or assumed uncertainty of the previous pose. The uncertainty may e.g. be determined from an estimated uncertainty of the field measurement process.
[0175] An example of the pose tracking will be illustrated with reference to the detected range, i.e. the detected distance to the other electronic device. FIGs. 10A-B illustrate an example of pose tracking. In particular, FIG. 10A schematically illustrates the determined distribution 901 of the most recent previous range, e.g. based on the determined previous range and a corresponding estimated measurement uncertainty or confidence.
[0176] FIG. 10B shows the corresponding predicted distribution 902 at a subsequent time, which in this example has a larger variance that the previous distribution, due to the additional uncertainty resulting from the lapsed time since the previous measurement. As illustrated in FIG. 10B, in addition to the larger variance, the mean of the predicted distribution may also change.
[0177] The process may further use the current range computed in step S2 from the current field measurements, and determine an associated uncertainty. The process may thus determine a current range distribution 903, e.g. as a Gaussian or other predetermined distribution around the currently measured range and having a variance determined by the uncertainty associated with the currently measured range. The process may then determine a resulting range distribution 904 from the predicted range distribution 902 and from the currently measured range distribution 903. It will be appreciated that the process may employ other filtering schemes. It will further be appreciated that the process may use a similar or another filtering schemes for the direction and / or orientation and / or the determined ambiguity state. It will be appreciated that the process may employ a suitable weighting between the previous estimate and the current measurement, e.g. based on the time elapsed and / or based on the estimated quality of the current and / or previous measurement. In some embodiments, the process implements a Kalman filter or another suitable filter.
[0178] Alternatively or additionally, the process may also perform a filtering of the disambiguation state, e.g. as illustrated in FIG. 11. To this end, the process may, based on the previous pose estimate define a "forward" direction (e.g. the pose closest to the previously measured pose, in FIG. 11 represented by the binary value +1) and a "reverse" direction (in FIG. 11 represented by the binary value -1). During subsequent pose estimations, the process may determine subsequent disambiguated directions with the associated confidence values and perform a suitable filter, e.g. a Kalman filter, Bayesian filter, a moving average or the like to determine a filtered disambiguation direction.
[0179] In FIG. 11, the individual disambiguated directions and associated confidence values at respective times are illustrated as dots 11, each representing a confidence value C between +1 (corresponding to a forward direction with 100% confidence) and -1 (corresponding to a reverse direction with 100% confidence). The filtered, or tracked, direction is illustrated as a line 12. As above, it will be appreciated that the process may employ a suitable weighting between the previous estimate and the current measurement, e.g. based on the time elapsed and / or based on the estimated quality of the current and / or previous measurement.
[0180] It will be appreciated that some embodiments of the pose estimation process may use alternative or additional methods for estimating the pose based on the magnetic measurements.
[0181] For example, in some embodiments, the process may use additional information for performing the pose disambiguation, e.g. information about the relative poses of other electronic devices. To this end, the electronic devices may exchange information about their own relative poses via a wireless communications network, e.g. by broadcasting information about their own determined poses relative to one or more other electronic devices. An electronic device may then utilize that information when performing its own pose estimation, e.g. to improve the pose disambiguation.
[0182] FIG. 12 illustrates an example of how such received pose information may be utilised by an electronic device. In particular, FIG. 12 illustrates an electronic device 300A that is to determine its relative pose relative to two other electronic devices 300B and 300C. As discussed above, the pose estimation based on measured magnetic field components results in an ambiguous pose determination, i.e. the electronic device 300A cannot determine, based on the magnetic field components alone, whether the two other electronic devices are positioned in their positions 300B and 300C or in the alternative positions and orientations 301B and 301C. The electronic device 300A may use a gradiometry disambiguation to determine the correct pose, e.g. as described above. In some situations, this disambiguation based on gradiometry may be associated with some uncertainty, e.g. when the distance between the electronic devices is large.
[0183] However, if one of the electronic devices 300B and 300C already has determined the relative pose of devices 300B and 300C relative to each other, said electronic device may communicate this information to electronic device 300A. Electronic device 300A may then use that information to perform its own disambiguation, since only one of the ambiguous poses is consistent with the relative pose of electronic devices 300B and 300C, relative to each other. It will be appreciated that there are other scenarios and / or approaches which may be used to reduce the number of possible ambiguous spatial configurations of a plurality of electronic devices when incomplete relative pose information is exchanged among the plurality of electronic devices.
[0184] Accordingly, the disambiguated pose of two electronic devices, e.g. electronic devices 300B and 300C of FIG. 12, can be used to disambiguate ambiguous relative pose measurements of one or more additional electronic devices, e.g. of electronic device 300A of FIG. 12.
[0185] This propagation of disambiguation information may be used to improve the robustness of pose estimation and or to extend the effective range of disambiguated pose estimation, or to enable an electronic device, which does not have the capability of measuring field gradients, to determine its unambiguous pose.
[0186] Generally, the magnetic fields used for the purpose of pose measurements may be time-varying magnetic fields that vary at a suitable frequency. Many electronic devices are relatively small and have limited battery capacity. Accordingly, the magnetic fields used for pose estimation should preferably be relatively weak, which in turn involves the risk of interference from spurious emitters, e.g. from switch-mode power supplies of consumer products, and radio transmitters.
[0187] In some embodiments, the electronic devices may dynamically select the frequency of the magnetic fields, e.g. from a predetermined set of discrete frequencies or otherwise within a predetermined frequency band, so as to reduce unintended interference.
[0188] Therefore, in some embodiments, at least one of the electronic devices of the system of electronic devices may be configured to measure ambient signals at the currently selected frequency whilst none of the other electronic devices of the system, e.g. of the electronic devices communicating via an instance of the wireless communications network, are performing magnetics operations in the relevant frequency range. If the electronic device that performs the measurement measures an ambient signal above a predetermined threshold, the electronic device may perform measurements at one or more alternative frequencies, e.g. at each of a discrete set of predetermined candidate frequencies. The electronic device may then select the frequency where the lowest ambient signal strengths have been measured and communicate information indicative of the selected frequency to the other electronic devices of the system via the wireless communications network. In some embodiments, the other electronic devices of the system may also report detected interference to the electronic device that has been assigned the task of selecting the frequency, e.g. via a status message.
[0189] Embodiments of the electromagnetic pose measurements disclosed herein allow pose estimation covering 6 degrees of freedom, i.e. position and orientation. Embodiments of the pose measurement have been found to be accurate to within a few millimeters, without having to rely on additional sensors, such as additional coils, that would take up space in a small electronic device and cause additional manufacturing costs. In particular, the process does not rely on expensive components such as cameras or base stations. The process does not rely on specific user behavior, e.g. rely on relative movements of the electronic devices, positioning the devices within line of sight from another, or the like. The process is scalable, does not require user-calibration, is resilient to typical environmental perturbations, and has a relatively low complexity. Moreover, the required hardware is small enough to fit into electronic devices suitable for being held in the hands of small children.
[0190] While various aspects have mainly been described with reference to toy construction systems, it will be appreciated that they may also be embodied by other types of electronic devices.
[0191] Referring to Figs.13 and 14 in the following, handling of spatial parameters requests according to one embodiment of a system comprising a plurality of cooperating electronic devices is discussed.
[0192] Fig.13 shows a system comprising a plurality of electronic devices 100A-C, 100X, each electronic device 100A-C, 100X having a sensing component T / R of a spatial sensing arrangement; a transceiver adapted for wireless communication with the other electronic devices (not shown); and a processor (not shown) that is operationally coupled to the transceiver and to the sensing component. The sensing component T / R 1, T / R 2, T / R 3, T / R n of each of the electronic devices 100A-C, 100X comprises both transmitters T and receivers R, as well as a processing module implemented in the processor. Advantageously, as discussed in detail above, the transmitters T comprise electromagnetic coils and driving circuitry for the generation of an alternating electromagnetic stimulation field. Further advantageously, the receivers R comprise electromagnetic coils for picking up an alternating electromagnetic stimulation field and measurement circuitry that is adapted to measuring one or more local properties of a stimulation field. Further advantageously, the receiver R uses the same electromagnetic coil arrangement as the transmitter T. The measurement results are passed to the processing module, and the processing module is configured to compute a partial or full pose {P;O} of one or more of the electronic devices 100A-C, 100X relative to another one or more of the electronic devices 100A-C, 100X, based on the measured one or more local properties of the stimulation field.
[0193] In the example shown in Fig.13 multiple electronic devices 100A-C, here three, are shown to join a wireless network of cooperating devices, such as a wireless toy network, through which they can exchange messages to cooperate with each other to provide an interactive user experience 130 to one or more users 99 of the electronic devices 100A-C as indicated by block arrows 9. Advantageously, wireless communication among the cooperating electronic devices 100A-C is by broadcasting messages in the network of cooperating electronic devices, e.g. labelled with a group marker, and selectively receiving messages from cooperating electronic devices 100A-C in the network, e.g. selecting messages by identifying the group marker.
[0194] The wireless network may be implemented on any suitable wireless communication infrastructure, such as a local area network or a personal area network like Bluetooth, BLE, or similar. Advantageously, the network is open exclusively to compatible electronic devices, i.e. the wireless network is "OPEN" in the sense that compatible devices 100X may freely join and leave the wireless toy network. Furthermore, the term 'compatible' may also refer to further related devices (not shown in Fig.13) that are specifically configured to participate in the dedicated wireless network for cooperating electronic devices. Such further related devices may be adapted to provide supporting functions or otherwise interact with the system, e.g. to facilitate and / or to configure operation of the system. Such further related devices are expressly configured to be compatible with the wireless toy network, and thus to have access to the wireless toy network. Examples for such further related devices include one or more of charging devices, smart phones, tablet computers, PCs, and similar devices.
[0195] In the example shown in Fig.13, one or more further electronic devices 100X may join (or leave) to cooperate (or to stop cooperating) with the group of cooperating electronic devices 100A-C, thus modifying the interactive user experience 130 as indicated by the solid arrow 10. Here, the term 'compatible electronic devices' is understood such that the electronic devices are specifically configured to participate in a dedicated wireless network for cooperating electronic devices, which may in turn be implemented on a known type of network infrastructure. Advantageously, also the further electronic devices 100X have a sensing component T / R n, which for the sake of flexibility and modularity preferably also is adapted to emit a stimulation field, to measure local properties of a stimulation field, and to compute pose information {P;O} of the electronic devices relative to another one or more of the electronic devices 100X, based on measured one or more local properties of a stimulation field as discussed above.
[0196] Users 99 can interact with the cooperating electronic devices 100A-C in an interactive user experience 130, such as in playful interactions like construction play, role play, and / or game play. As discussed throughout the disclosure, spatial sensing components allow the cooperating electronic devices 100A-C to determine poses of one or more of the electronic devices 100A-C relative to one or more of the other electronic devices 100A-C. The cooperating electronic devices 100A-C may thus provide a rich user experience 130 utilising the spatial relations between the cooperating electronic devices 100A-C. To that end, the user experience 130 may require one or more relative poses {P; O} ij to be determined, where i and j are integer indices denoting one of a plurality of cooperating electronic devices, before generating a user-perceptible output or defining an interactive behaviour based on the relative poses.
[0197] For example, as seen in Fig.13, the user experience may require a first pose {P; O} 21 of a second electronic device 100B relative to a first electronic device 100A. The user experience may further require a second pose {P; O} 31 of a third electronic device 100C relative to the first electronic device 100A, and a third pose {P; O} 32 of the third electronic device 100C relative to the second electronic device 100B. The required pose {P; O} ij may be each time any of a relative position P and / or orientation O of one of the electronic devices 100A-C relative to another one of the electronic devices 100A-C. Furthermore, a relative position P may be any of a distance and / or direction of one of the electronic devices 100A-C relative to another one of the electronic devices 100A-C. Furthermore, a relative orientation O may be any one or more of angular parameters describing the orientation of one of the electronic devices 100A-C relative to another one of the electronic devices 100A-C. As such, a relative pose may be any one or more of a set of spatial parameters describing the six degrees of freedom of a position and orientation of one of the electronic devices 100A-C relative to another one of the electronic devices 100A-C.
[0198] Conceivably, the user experience might then be programmed to sequentially perform three pairwise measurements, respectively, in an attempt to satisfy all requests one by one: between the first and second electronic devices 100A and 100B to determine the first pose {P; O} 21; between the first and third electronic devices 100A and 100C to determine the second pose {P; O} 31, and between the second and third electronic devices 100B and 100C to determine the third pose {P; O} 32.
[0199] While the above-mentioned sequential measurement scheme may be feasible for very few cooperating electronic devices, such an approach may quickly become prohibitive for a scalability of the system to larger numbers of cooperating electronic devices. However, as discussed in the present disclosure, the inventors have realized that a number of significant advantages are achieved by another scheme for determining the required poses to provide an improved user experience.
[0200] With reference to Figs. 13 and 14, by way of example, the scheme includes providing spatial parameters indicative of relative poses between cooperating electronic devices as a cooperative system service to the cooperating electronic devices. Notably, a significant reduction in the use of shared resources is thereby achieved, leading to, among others, improved responsiveness, reliability, and / or reduced power usage. Furthermore, an improved integration with other functions such as wireless tag reading and / or wireless charging of a rechargeable power source in the electronic devices, as well as user-perceptible function components in a relatively small form factor is thereby facilitated.
[0201] Fig. 14 shows schematically operation of an embodiment of a system comprising a plurality of cooperating electronic devices, such as wireless interactive toys, in particular a toy construction system comprising a plurality of wireless interactive toy construction elements for providing an interactive user experience based on one or more poses of one or more of the cooperating electronic devices relative to one or more other ones of the cooperating electronic devices, and a method of controlling the same. Further referring to the example of the system illustrated in Fig.13, operation of the system may include the following steps.
[0202] At 131, a programmed interactive user experience 130 controlling the interactive operation of a plurality of cooperating electronic devices 100A-C may initiate one or more requests for spatial parameters, each request being indicative of a pose of one electronic device 100A-C relative to another one 100A-C. The one or more spatial parameter requests may be initiated responsive to respective trigger events occurring in the course of the operation. Examples for such trigger events may include, but are not limited to one or more of: a user interaction with one or more of the electronic devices; a programmed event occurring in a user interaction module, such as a game event in a game implemented on the cooperating electronic devices; a change in operational mode of one or more of the electronic devices; addition of one or more electronic devices to the system; removal of one or more electronic devices from the system; grouping of multiple of the electronic devices in a group of cooperating electronic devices; addition of one or more electronic devices to a group of cooperating electronic devices; removal of one or more electronic devices from of a group of cooperating electronic devices, or the like.
[0203] The user experience 130 passes the one or more spatial parameter requests to a programmed system service 140 at arrow 34, where the spatial parameter requests are processed by the electronic devices in cooperation to return the requested spatial parameters as a service at arrow 43 to the programmed user experience 130. Based on the spatial parameters determined by the system service 140, the user experience 130 may then produce a corresponding user-perceptible output and / or define a corresponding interactive behaviour of the system at 132.
[0204] The system service 140 may share the received spatial parameter requests among the cooperating electronic devices through wireless communication at 141. At 142, the spatial parameter requests are converted into measurement requests. In particular, the conversion may include identifying for each measurement request the sensing actions to be performed by sensing components of the cooperating electronic devices in order to fulfil the spatial parameter requests. Furthermore, the conversion preferably also includes optimizing the sensing actions in respect of the entirety of available spatial parameter requests, or at least in respect of a portion of the available spatial parameter requests. Optimization is typically performed in respect of a predetermined set of criteria and / or optimization goals directed to reducing the usage of resources for measuring and / or increasing the quality of accuracy and / or precision of the measurement results. Typically, optimization is done with a goal to reduce the total number of sensing operations required to fulfil at least a portion, preferably all of the spatial parameter requests. Furthermore, the optimization may include identifying the sensing actions that minimizes power usage of the electronic devices and / or assigning sensing operations to electronic devices depending on a power status of the devices and an expected power usage for a given sensing action. Furthermore, the optimization may include assigning sensing actions to electronic devices according to an estimation of the usefulness of the electronic devices for contributing to a given spatial parameter request, where usefulness may include considerations of a measurement history, knowledge of a previous placement of an electronic device relative to other ones of the electronic devices, the presence of the required sensing components adapted to perform a given sensing action, a battery status, details of the request, and the like. The details of the request may include specifying if the spatial parameters should be determined in a direct measurement performed at a requesting electronic device relative to a further one of the electronic devices, an indirect measurement performed at the further one of the electronic devices relative to the requesting device, or by inference involving knowledge about spatial relations of further ones of the electronic devices.
[0205] For example, in the course of the optimization, certain spatial parameter requests may be identified and removed from the list of spatial parameter requests as being redundant. For example, a request for a pose of a first electronic device relative to a second electronic device and a corresponding request for a pose of the second relative to the first may be fulfilled by a single measurement between the first and the second electronic devices and sharing the results. In the case of a measurement requiring emitting a stimulation field at one electronic device, and measuring the stimulation field at a different electronic device, the service may identify one of the first and second electronic devices as the transmitter and the other one of the first and second electronic devices as the receiver, e.g. based on further criteria such as a battery status (emitting a stimulation field requiring more remaining battery power than measuring a stimulation field) and / or consideration of further spatial parameter requests. In the above-mentioned example of Fig.13, the optimization may identify that only two measurements are required to fulfil the three pending spatial parameter requests {P; O} 21, {P; O} 31, {P; O} 32, since a third one may be inferred from the two other measurements. Furthermore, a most recent relative pose history may suggest that the second electronic device 100B is currently placed somewhere between the first and third electronic devices 100A, 100C. The second electronic device may therefore be assigned the sensing action of emitting a stimulation field, and each of the first and third electronic devices may be assigned the sensing action of measuring local properties of the stimulation field while it is emitted. Fulfilling the three spatial parameter requests {P; O} 21, {P; O} 31, {P; O} 32 of Fig.13 as a service thus only requires a single stimulation field emission by one of the electronic devices, which can be simultaneously measured at two different locations by the two remaining electronic devices. Already in the relatively simple scenario of Fig.13, the resource usage for fulfilling the three pending spatial parameter requests as a service is thereby reduced by roughly a factor of three as compared to a sequential scheme of performing a respective measurement for each of the three requests.
[0206] Once the sensing actions that are required to fulfil the spatial parameter requests are identified, the service may at 143 generate a schedule for performing sensing actions. Typically, only sensing actions requiring shared resources in a contentious manner are scheduled. In the case of electromagnetic probing of relative poses between electronic devices, generation and emission of an electromagnetic stimulation field may be considered to be contentious, as it may lead to undesired perturbation and interference between measurements if multiple stimulation fields are emitted simultaneously. Therefore, the schedule generated at 143 may be a time domain schedule assigning respective time slots for emitting a stimulation field to each of the electronic devices requesting to do so according to the sensing actions assigned to them. The schedule may each time cover a predetermined time frame or period of time and may be repeated in a continued manner until it is regenerated, e.g. in response to a change in spatial parameter requests.
[0207] At 144, the schedule is distributed to the cooperating electronic devices.
[0208] At 145, the electronic devices may operate their respective sensing components consistent with the schedule in order to collect the corresponding measurement results. The electronic devices assigned the role of a stimulation field emitter are thus operated at their respective time slots allocated to them, in order to perform the sensing action of generating a stimulation field at the allocated time. Any of the electronic devices assigned the role of stimulation field receivers are also aware of the stimulation field emission schedule, and are thus each time operated at the correct time slots for the sensing action of measuring local properties of the stimulation field. As mentioned before, multiple receivers may be operated in the same stimulation field emission time slot to obtain respective measurement results for the same stimulation field at different electronic devices.
[0209] At 146, the requested spatial parameters are computed based on the obtained measurement results, wherein precision, detail, and accuracy of the computation may depend on the available measurement result, and the details of the spatial parameter request. Computation may thus include, for example, determining a simple presence, if a front or back of another electronic faces towards the requesting electronic device, a rough estimation of a distance and / or orientation, any one or more of the parameters describing the relative pose of two electronic devices with respect to each other, or all details of the actual relative pose.
[0210] At 147, the determined spatial parameters are shared among the cooperating electronic devices, and passed at arrow 43 from the system service 140 to the user experience 130, where the determined spatial parameters are used to determine and generate a user-perceptible output and / or to define an interactive behaviour at 132 as already mentioned above. Sharing and distribution of information as required to facilitate the cooperation is performed using the transceivers, by wireless data communication of the cooperating electronic devices with each other through a wireless network formed by the electronic devices. Electronic devices may even dynamically join and leave the cooperation. Coordination, optimisation, and scheduling is typically performed in one of the cooperating devices selected as a master device. Advantageously, the above system service scheme may be implemented in the processors of each of the cooperating devices thereby allowing to flexibly distribute cooperative tasks, and to assign and to re-assign a master role, if necessary.
[0211] The skilled person will understand that implementation of a system service fulfilling requests for relative spatial parameters among a plurality of cooperating electronic devices as disclosed herein may be structured in many different ways, and is not limited to the specific embodiment as illustrated in Fig.14 and described herein with reference to Fig.13. For example, the number of cooperating electronic devices may vary significantly, due to the dynamic scalability of the system. Furthermore, the separation into a user experience module 130 and a system service module 140 as shown in Fig.14 may also be modified. For example, the spatial parameter requests may be converted into a measurement request already in a modified user experience module, before passing the requests to a modified system service, which is configured to fulfil the measurement requests in a cooperative manner and to return the corresponding measurement results to the modified user experience, where the requested spatial parameters are determined based on the measurement results received from the system service. Alternatively, there may be no distinct separation into a user experience module and a service module as seen in Fig.14. Instead the programmed instructions, which control the cooperation of the cooperating electronic devices may directly implement the cooperative service for determining spatial parameters indicative of one or more relative poses of one or more electronic devices relative on or more other electronic devices in a control software of each of the cooperating devices.
[0212] Furthermore the above-mentioned scheduling may also take into account other contentious uses of shared resources or potential interference from the simultaneous operation of other components in or around the cooperating electronic devices, of which the cooperating electronic devices are informed / aware, where time slots may be reserved and assigned to take account of such potentially contentious or interfering operations. Examples for further scheduling may include reserving time slots for one or more of wireless nearfield reading of data stored on tags or other storage elements, operating an electromagnetically driven actuator, such as a loudspeaker or a motor, activating a further wireless communication component, power switching and the like. In some embodiments, even wireless charging of a rechargeable power source arranged in one or more of the cooperating electronic devices may be scheduled accordingly, e.g. because the wireless charging field would potentially interfere with any of the electromagnetic sensing operations, and / or because the charging circuit may share hardware, such as parts of the coil arrangement with the electromagnetic sensing components.
Claims
1. A system comprising a plurality of cooperating electronic devices (100A-C, 100X), wherein each of the electronic devices (100A-C, 100X) comprises: - a sensing component (T / R 1, T / R 2, T / R 3, T / R n) of a spatial sensing arrangement; - a transceiver adapted for wireless communication with the other electronic devices (100A-C, 100X); and - a processor operationally coupled to the transceiver and to the sensing component; wherein the electronic devices (100A-C, 100X) are configured: - to share a request for a pose ( {P;O}ij ) of a first electronic device (100A-C, 100X) relative to a second electronic device (100A-C, 100X); and - to cooperate to operate the sensing components (T / R 1, T / R 2, T / R 3, T / R n) of one or more of the electronic devices (100A-C, 100X) to obtain measurement data and to process the measurement data to determine the requested pose ( {P;O}ij ).
2. The system according to claim 1, wherein cooperation of the electronic devices (100A-C, 100X) further includes generating a schedule for performing sensing actions, and to operate the sensing components (T / R 1, T / R 2, T / R 3, T / R n) to perform the sensing actions consistent with the schedule.
3. The system according to any one of claim 1 or claim 2, wherein the spatial sensing arrangement comprises: - one or more transmitters (T) for generating a stimulation field; - one or more receivers (R) for measuring one or more local properties of the stimulation field; and - one or more processing modules configured to compute a pose ( {P;O}ij ) of one or more of the electronic devices relative to another one or more of the electronic devices (100A-C, 100X), based on the measured one or more local properties of the stimulation field.
4. The system according to any one of the preceding claims, wherein the sensing components (T / R 1, T / R 2, T / R 3, T / R n) of each of the electronic devices (100A-C, 100X) comprises one or more of: - a transmitter (T) adapted to generate a stimulation field; - a receiver (R) adapted to measure a local property of a stimulation field generated by another electronic device (100A-C, 100X); and - a processing module configured to compute a pose ( {P;O}ij ) of the electronic device (100A-C, 100X) relative to another one of the electronic devices (100A-C, 100X), based on the stimulation field generated by the other electronic device.
5. The system according to any one of claim 3 or claim 4, wherein operation of the sensing components (T / R 1, T / R 2, T / R 3, T / R n) further includes operating only one transmitter (T) of the cooperating electronic devices (100A-C, 100X) at a time.
6. The system according to claim 5, wherein operation of the sensing components (T / R 1, T / R 2, T / R 3, T / R n) further includes operating one or more receivers (R) together with the transmitter (T) with a time overlap.
7. The system according to any one of the preceding claims, wherein one or more, preferably each, of the cooperating electronic devices (100A-C, 100X) comprises one or more electromagnetic coils, wherein the one or more electromagnetic coils are operable as a transmitter (T) adapted to generate an alternating electromagnetic stimulation field.
8. The system according to any one of the preceding claims, wherein one or more, preferably each, of the electronic devices (100A-C, 100X) comprises one or more electromagnetic coils, wherein the one or more electromagnetic coils are operable as a receiver (R) adapted to measure one or more local properties of an alternating electromagnetic stimulation field.
9. The system according to any one of the preceding claims, wherein the system is further configured to initiate a spatial parameter request in one or more of the electronic devices (100A-C, 100X) in response to a trigger event.
10. The system according to claim 9, wherein the trigger event is one or more of: - a user interaction with one or more of the electronic devices (100A-C, 100X); - a programmed event occurring in a user interaction module, such as a game event in a game implemented on the cooperating electronic devices (100A-C, 100X); - a change in operational mode of one or more of the electronic devices (100A-C, 100X); - addition of one or more electronic devices (100A-C, 100X) to the system; - removal of one or more electronic devices (100A-C, 100X) from the system; - grouping of multiple of the electronic devices (100A-C, 100X) in a group of cooperating electronic devices; - addition of one or more electronic devices (100A-C, 100X) to a group of cooperating electronic devices (100A-C, 100X); - removal of one or more electronic devices (100A-C, 100X) from of a group of cooperating electronic devices (100A-C, 100X).
11. The system according to any one of the preceding claims, wherein the system is further configured to cause generation of a user-perceptible output at one or more of the interacting electronic devices (100A-C, 100X), based on the determined spatial parameters.
12. The system according to any one of the preceding claims, wherein one or more, preferably all of the electronic devices (100A-C, 100X) are wireless interactive toys.
13. The system according to claim 12, wherein one or more, preferably all of the wireless interactive toys are or comprise a wireless interactive toy construction element.
14. The system according to any one of the preceding claims, wherein the electronic devices (100A-C, 100X) are further configured to form a wireless toy network with other ones of the plurality of electronic devices (100A-C, 100X) of the interactive toy system.
15. The system according to any one of the preceding claims, wherein wireless communication among the electronic devices (100A-C, 100X) is by broadcasting messages in a network of cooperating electronic devices (100A-C, 100X) and selectively receiving messages from cooperating electronic devices (100A-C, 100X) in the network.