SENSOR-BASED CONTEXT MANAGEMENT

Abstract

According to an exemplary aspect of the present invention, there is provided an apparatus for identifying user activity, comprising: - a memory configured to store data from a sensor of the first type relating to an activity; - at least one processing core configured to: Compiling a message based at least in part on the data from the sensor of the first type, Causing the message to be transmitted from the device to a server external to the device, Causing a response to the message to be received in the device from the server as a machine-readable instruction comprising at least two machine-readable characteristics, wherein each of the at least two characteristics characterizes sensor data produced during a predefined activity type, wherein the at least two machine-readable characteristics comprise reference data specific to a context in which the device operates, and Deriving an estimated activity type using the reference data specific to a context based at least in part on sensor data by comparing the sensor data to the reference data specific to a context.

Description

TECHNICAL FIELD

[0001] The present invention relates to the identification of user activity based on sensor information. BACKGROUND

[0002] User sessions, such as training sessions, can be recorded, for example, in notebooks, spreadsheets, or other suitable media. Recorded training sessions enable more systematic training, and progress toward set goals can be evaluated and tracked using the resulting recordings. Such recordings can be retained or stored for future reference, for example, to evaluate an individual's progress as a result of training. An activity session may include a training session or another type of session.

[0003] Personal devices, such as smart watches, smartphones, or smart jewelry, can be configured to produce recorded sessions of user activity. Such recorded sessions may be useful for managing physical exercise, child safety, or in professional applications. Recorded sessions, or more generally, sensor-based activity management, may be of various types, such as running, walking, skiing, canoeing, hiking, or providing assistance to seniors. For example, document DE 102014107571 A1 discloses a method comprising receiving input from a sensor at a mobile unit. Based on the sensor input, a processor determines whether a user of the mobile unit is engaged in a particular physical activity.A control setting is made on the mobile device to provide content during a period during which the relevant physical activity is detected. Furthermore, document US 2013 / 0289932 A1 describes a method for configuring a motion sensor comprising an accelerometer and a gyroscope, a processor unit, a memory unit, and a program selection. At least one motion parameter from one or more output signals of the accelerometer and / or gyroscope can be determined by the processor unit using a selected signal processing program.

[0004] Recorded sessions can be viewed on a personal computer (PC), for example, recordings can be copied from the personal device to the PC. Files on a PC can be protected with passwords and / or encryption, for example.

[0005] Personal devices may be equipped with sensors that can be used, for example, to determine a position of the personal device. A satellite position sensor may, for example, receive position information from a satellite constellation and deduce the location of the personal device. A recorded training session may include a route determined by repeatedly determining the position of the personal device during the training session. Such a route can be viewed later using, for example, a PC.

[0006] As an alternative to a satellite position sensor, a personal device may also be configured to determine its position, for example, using a cellular network-based determination method, wherein a cellular network is used to assist in determining the position. A cell maintaining the personal device's connection to the cellular network may, for example, have a known position, thereby providing a position estimate of the personal device based on the connection and a finite geographic extent of the cell. SUMMARY OF THE INVENTION

[0007] The invention is defined by the features of the independent claims. Some specific embodiments are defined in the dependent claims.

[0008] According to a first aspect of the present invention, there is provided an apparatus comprising a memory configured to store data from a sensor of the first type, at least one processing core configured to compile a message based at least in part on the data from the sensor of the first type, to cause the message to be transmitted from the apparatus, to cause a machine-readable instruction to be received at the apparatus, and to derive an estimated activity type using the machine-readable instruction based at least in part on sensor data.

[0009] Various embodiments of the first aspect may include at least one feature from the following list: • the machine-readable instruction comprises at least one of the following: an executable program, an executable script, and a set of at least two machine-readable characteristics, each of the characteristics characterizing sensor data produced during a predefined activity type • the at least one processing core is configured to derive the estimated activity type at least in part by comparing the data of the sensor of the first type or a processed form of the data of the sensor of the first type with reference data using the machine-readable instruction • the data from the sensor of the first type includes acceleration sensor data • the memory is further configured to store data from a sensor of the second type, and the at least one processing core is configured to derive the estimated activity type using the machine-readable instruction based at least in part on data from the sensor of the second type • the data of the second type sensor is of a different type than the data of the first type sensor • the data from the second type of sensor comprises at least one of sound sensor data, microphone-derived data and vibration sensor data • the at least one processing core is configured to derive the estimated activity type at least in part by comparing the second type sensor data or a processed form of the second type sensor data with reference data, wherein the reference data comprises reference data of a first type and a second type • the at least one processing core is configured to present the estimated activity type to a user for verification • the at least one processing core is configured to cause the memory to store the estimated activity type and a second estimated activity type in a sequence of estimated activity types • the at least one processing core is configured to cause the memory to delete the machine-readable instruction in response to a determination that an activity session has ended.

[0010] According to a second aspect of the present invention, there is provided a device comprising at least one processing core, at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to cause the device, with the at least one processing core, to receive at least one message from a user device, the message comprising information characterizing data from a sensor of the first type, to determine an activity context based at least in part on the data from the sensor of the first type, and to transmit to the user device a machine-readable instruction configured to cause the activity type determination in the activity context.

[0011] According to a third aspect of the present invention, a method is provided comprising storing data of a sensor of the first type in a device, compiling a message based at least in part on the data of the sensor of the first type, causing the message to be transmitted from the device, causing a machine-readable instruction to be received in the device, and deriving an estimated activity type based at least in part on sensor data using the machine-readable instruction.

[0012] Various embodiments of the third aspect may comprise at least one feature corresponding to a feature in the preceding enumeration list mentioned in connection with the first aspect.

[0013] According to a fourth aspect of the present invention, a method is provided, comprising receiving a message from a user device, the message comprising information characterizing data of a sensor of the first type, determining an activity context based at least in part on the data of the sensor of the first type, and transmitting to the user device a machine-readable instruction configured to effect activity type determination in the activity context.

[0014] According to a fifth aspect of the present invention, there is provided an apparatus comprising means for storing data from a sensor of the first type in a device, means for compiling a message based at least in part on the data from the sensor of the first type, means for causing the message to be transmitted from the device, means for causing a machine-readable instruction to be received in the device, and means for deriving an estimated activity type based at least in part on sensor data using the machine-readable instruction.

[0015] According to a sixth aspect of the present invention, there is provided a non-transitory computer-readable medium having stored thereon a set of computer-readable instructions that, when executed by at least one processor, cause a device to store at least data from a sensor of the first type, compile a message based at least in part on the data from the sensor of the first type, cause the message to be transmitted by the device, cause a machine-readable instruction to be received at the device, and derive, using the machine-readable instruction, an estimated activity type based at least in part on sensor data.

[0016] According to a seventh aspect of the present invention, a computer program is provided which is configured to bring about the performance of a method according to at least one of the third and fourth aspects. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 illustrates an example system according to at least some embodiments of the present invention; Fig. 2 illustrates an exemplary time sequence with multiple sensors; Fig. Figure 2B illustrates a second exemplary time sequence with multiple sensors; Fig. 3 illustrates an example device that may support at least some embodiments of the present invention; Fig. 4 illustrates signaling according to at least some embodiments of the present invention, and Fig. 5 is a flow diagram of a method according to at least some embodiments of the present invention. EMBODIMENTS

[0017] Using sensor data to determine an activity type increases the usability of personal devices. For example, a user may suffer from partial disabilities that make it difficult to use the device without assistance. Using sensor data from more than one sensor can further increase the accuracy of the activity type estimate. To enable a device of a user with reduced abilities to determine an activity type, a machine-readable instruction can be selected and provided to the user device by a back-end server. The selection can be based on sensor data collected from the user device, such that the machine-readable instruction enables the user device to derive an estimated activity type in the user context in which the user device is located.The derivation is thus partially performed in the back-end server, allowing the user device to provide a good estimate with only limited processor, power, and / or memory resources.

[0018] Fig. 1 illustrates an example system according to at least some embodiments of the present invention. The system includes device 110, which may, for example, comprise a smart watch, digital watch, smartphone, phablet device (smartlet), tablet device, or other suitable device type. Device 110 may include a display, which may, for example, comprise a touchscreen display. This display may be of limited size. Device 110 may, for example, be powered by a rechargeable battery. An example of a limited-size display is an arm-worn display.

[0019] The device 110 may be communicatively coupled to a communication network. Fig. 1, device 110 is coupled to a base station 120 via wireless connection 112, for example. Base station 120 may comprise a cellular or non-cellular base station, where a non-cellular base station may be referred to as an access point. Examples of cellular technologies include Wideband Code Division Multiple Access (WCDMA) and Long Term Evolution (LTE), while examples of non-cellular technologies include wireless local area network (WLAN) and Worldwide Interopability For Microwave Access (WiMAX). Base station 120 may be coupled to network node 130 via connection 123. Connection 123 may, for example, be a wired connection. Network node 130 may, for example, comprise a controller or a gateway device. Network node 130 may interface to network 140, which may, for example, comprise the Internet or an enterprise network, via connection 134.Network 140 may be coupled to other networks via connection 141. In some embodiments, device 110 is not configured to couple to base station 120. Network 140 may, for example, include or be communicatively coupled to a back-end server.

[0020] Device 110 may be configured to receive satellite position information from satellite constellation 150 via satellite link 151. The satellite constellation may include, for example, the Global Positioning System (GPS) or the Galileo constellation. Satellite constellation 150 may include more than one satellite, although for clarity, Fig. 1, only one satellite is shown. Receiving the position information via satellite link 151 may similarly include receiving data from more than one satellite.

[0021] Alternatively or in addition to receiving data via a satellite constellation, device 110 may obtain position information by interacting with a network that includes base station 120. For example, cellular networks may use various methods to track a device's position, such as trilateration, multilateration, or position tracking based on the identity of a base station with which connectivity is available or in progress. Similarly, a non-cellular base station or access point may know its own position and provide it to device 110, allowing device 110 to track its position even within the communication range of that access point.

[0022] For example, device 110 may be configured to obtain a current time from satellite constellation 150, base station 120, or by querying it from a user. After device 110 has the current time and an estimate of its position, device 110 may, for example, consult a lookup table to determine, for example, the time remaining until sunset or sunrise. Device 110 may similarly obtain knowledge of the time of year.

[0023] Device 110 may include or be coupled to at least one sensor, such as an acceleration sensor, humidity sensor, temperature sensor, heart rate sensor, or blood oxygen level sensor. Device 110 may be configured to produce and store sensor data using at least one sensor, for example, in a time sequence comprising a plurality of samples taken in a temporal sequence.

[0024] Device 110 may be configured to provide an activity session. An activity session may be associated with an activity type. Examples of activity types include rowing, paddling, cycling, jogging, walking, hunting, swimming, and paragliding. An activity session, in a simple form, may comprise device 110 storing sensor data produced by sensors included in device 110 or in another device with which device 110 is associated or paired. An activity session may be determined to have started and ended at specific times, such that the determination occurs subsequent to or concurrent with the starting and / or ending. In other words, device 110 may store sensor data to enable subsequent identification of activity sessions based at least in part on the stored sensor data.

[0025] An activity session in device 110 may enhance a benefit a user may receive from the activity. For example, if the activity involves outdoor exercise, the activity session may provide a recording of the activity session. Device 110 may, in some embodiments, provide the user with contextual information. Such contextual information may include, for example, locally relevant weather information received via, for example, base station 120. Such contextual information may include at least one of the following: a rain warning, a temperature warning, an indication of time until sunset, an indication of a nearby service relevant to the activity, a security warning, an indication of nearby users, and an indication of a nearby location where multiple other users have taken photographs.Contextual information can be presented during an activity session.

[0026] A recording of an activity session may include information about at least one of the following: a distance traveled during the activity session, a metabolic rate or metabolic effect of the activity session, a time the activity session lasted, an amount of energy expended during the activity session, a sound recording obtained during the activity session, and an elevation profile map along the length of the distance traveled during the activity session. For example, a distance may be determined based on location information. Metabolic effect and expended energy may be determined based at least in part on information related to the user to which device 110 has access. A recording may be stored on device 110, an auxiliary device, or a server or cloud storage service for data.A recording stored on a server or cloud may be encrypted before transmission to the server or cloud to protect the user's privacy. A recording may be produced even if the user has not indicated that an activity session has been started, because a start and end of an activity session can be determined after the session has ended, for example, based at least in part on sensor data.

[0027] Device 110 may have access to a backhaul communication link to provide information related to ongoing activity. For example, search and rescue services may be able to access information about joggers in a particular area of a forest to enable their rescue if, for example, a chemical spill renders the forest unsafe for human use. Other users may alternatively or additionally be enabled to receive information about ongoing activity sessions. Such other users may, for example, be preconfigured as users authorized to receive such information, with non-authorized users not being provided with such information. As a specific example, users on a friends list may be able to receive information about an ongoing activity session.The friends list can be maintained, for example, in a social media service. Information about ongoing activity sessions can be provided by device 110, for example, as periodic updates.

[0028] After an activity has ended, sensor data may be stored in device 110 or in memory accessible to device 110. The stored sensor data may be stored as a time sequence spanning the activity session, as well as time preceding and / or following the activity session. The start and end times of the activity session may be selected by the user from the time sequence. The start and end points may be preselected by device 110 or a PC for a user to accept or reject, with the preselection based on changes in the time sequence. For example, if accelerometer data begins to indicate more active movements from device 110 in the time sequence, a start point of an activity session may be preselected. Such a change may correspond, for example, to a time in the time sequence when the user stopped driving and began jogging.A phase in the time sequence where the more active movements end can similarly be preselected as the end point of the activity session.

[0029] Sensor data may include information from more than one sensor, where the more than one sensor may include sensors of at least two of its own types. Sensor data may include, for example, accelerometer data and barometric pressure sensor data. Other examples are sound volume sensor data, humidity sensor data, and electromagnetic sensor data. Sensor data from a sensor of a first type may generally be referred to as first-type sensor data, and sensor data from a sensor of a second type may be referred to as second-type sensor data. A type of sensor may be defined by a physical property that a sensor can measure according to its configuration. For example, all temperature sensors may be considered temperature-type sensors, regardless of the physical principle used in the temperature sensor to measure temperature.

[0030] Preselecting the start and / or end points in the time sequence may involve detecting that sensor data characteristics from more than one sensor type change at approximately the same time point. Using more than one type of sensor data may increase the accuracy of preselecting the start and / or end points if the sensors in question are affected by the activity performed during the activity session.

[0031] An activity type may be determined based at least in part on the sensor data. This determination may occur when the activity occurs or afterward when the sensor data is analyzed. The activity type may be determined, for example, by device 110 or a PC that has access to the sensor data, or a server that is given access to the sensor data. If a server is given access to the sensor data, the sensor data may be anonymized. Determining the activity type may comprise comparing the sensor data with reference data. The reference data may comprise reference datasets, each reference dataset associated with an activity type. The determination may comprise determining the reference dataset that most closely resembles the sensor data, for example, in a least squares sense. Alternatively to the sensor data itself, a processed form of the sensor data may be compared with the reference data.The processed form may, for example, comprise a frequency spectrum obtained from the sensor data. The processed form may, for example, comprise a set of local minima and / or maxima from the temporal sequence of the sensor data. The determined activity type may be selected as the activity type associated with the reference dataset that most closely resembles the processed or original sensor data.

[0032] Different activity types may be associated with different characteristic frequencies. For example, if the user was running, accelerometer data may reflect a higher characteristic frequency compared to walking. Thus, in some embodiments, determining the activity type may be based at least in part on deciding which reference data set has a characteristic frequency that most closely matches a characteristic frequency of a portion of the sensor-derived information in the time sequence under consideration. Accelerometer data may be used alternatively or additionally to determine a characteristic movement amplitude.

[0033] If device 110 is configured to store a time sequence of more than one type of sensor data, multiple sensor data types may be used to determine the activity type. The reference data may comprise reference data sets that are multi-sensory in nature, such that each reference data set comprises data that can be compared to each available sensor data type. For example, if device 110 is configured to compile a time sequence of acceleration and sound sensor data types, the reference data may comprise reference data sets, each reference data set corresponding to an activity type, each reference data set comprising data that can be compared to the acceleration data and data that can be compared to the sound data.The determined activity type may be determined as the activity type associated with the multi-sensory reference dataset that most closely matches the sensor data stored by device 110. In turn, original sensor data or processed sensor data may be compared to the reference datasets. For example, if device 110 comprises a smartphone, it may comprise multiple sensors to accomplish the smartphone function. Examples of such sensors include microphones to enable voice calls and cameras to enable video calls. Further, in some cases, a radio receiver may be configurable to measure the properties of electric or magnetic fields. Device 110 may comprise a radio receiver, and generally, device 110 is equipped with wireless communication capability.

[0034] A first example of multisensory activity type determination is hunting, where device 110 stores data from a first type of sensor, including accelerometer data, and data from a second type of sensor, including sound data. The reference data would comprise a hunting reference dataset, which would comprise acceleration reference data and sound reference data to enable comparison with sensor data stored by device 110. Hunting may include periods of low sound and low acceleration, and interspersed combinations of loud, short sound and high-frequency, low-amplitude acceleration, corresponding to a gunshot and impact.

[0035] A second example of multi-sensory activity type detection is swimming, where device 110 stores data from a first-type sensor, including humidity sensor data, and data from a second-type sensor, including magnetic field data from a compass sensor. The reference data would include a swimming reference dataset, which would include humidity reference data and magnetic field reference data to enable comparison with sensor data stored by device 110. Swimming may involve high humidity, as it involves immersion in water, and elliptical movements of an arm to which device 110 may be attached, which may be detectable as periodically varying magnetic field data. In other words, the direction of the Earth's magnetic field from the viewpoint of the magnetic compass sensor may vary periodically over time.

[0036] A determined or inferred activity type may be considered an estimated activity type until the user confirms that the determination is correct. In some embodiments, a few, for example, two or three, most likely activity types may be presented to the user so that the user can select the correct activity type. Using two or more types of sensor data increases the probability that the estimated activity type is correct.

[0037] A context process may be used to derive an estimated activity type based on sensor data. A context process may include first determining a context in which the sensor data has been produced. The context process may include, for example, using the sensor data to determine the context, such as a user context, and then deriving an activity type within that context. For example, a context may include outdoor activity, and deriving an estimated activity type may include first determining, based on the sensor data, that the user is in an outdoor context, selecting a machine-readable instruction for outdoor context, and using the machine-readable instruction to differentiate between different activity types in the outdoor context, such as jogging and orienteering.As another example, a context may include indoor activity, and deriving an estimated activity type may include first determining based on the sensor data that the user is in an indoor context, selecting a machine-readable instruction for indoor context, and using the machine-readable instruction to differentiate between different activity types in the indoor context, such as 100-meter dashes and wrestling.

[0038] The machine-readable instruction may, for example, comprise a script, such as an executable or compilable script, an executable computer program, a software plug-in, or a non-executable computer-readable descriptor that enables device 110 to differentiate between at least two activity types within the determined context. The machine-readable instruction may include information about which type(s) of sensor data, and in which format, are to be used in deriving the activity type using the machine-readable instruction.

[0039] Determining an outdoor context may include determining that the sensor data indicates a wide range of geographic movement, thereby indicating that the user roamed outdoors. Determining an indoor context may include determining that the sensor data indicates a narrower range of geographic movement, thereby indicating that the user stayed within a small area during the activity session. If temperature-type sensor data is available, a lower temperature may be associated with an outdoor activity, and a higher temperature may be associated with an indoor activity. Temperature may be particularly indicative of this if the user is located in a geographic area where winter, fall, or spring conditions cause an outdoor temperature to be lower than an indoor temperature.The geographical area may be available in location data.

[0040] In some embodiments, deriving an estimated activity type is therefore a two-phase process, first determining a context based on the sensor data and then deriving an estimated activity type within that context using a machine-readable instruction specific to that context. Selecting the context and / or the activity type within the context may include comparing sensor data or processed sensor data with reference data. The two-phase process may use two types of reference data: context-type reference data and activity-type reference data, respectively.

[0041] The context process may adaptively learn how to more accurately determine contexts and / or activity types based on previous activity sessions recorded from a plurality of users. The determination of context may be based on context-type reference data, wherein the context-type reference data is adaptively updated, for example, depending on the previous sessions recorded from the plurality of users. Adapting the context-type reference data may take place in a server, wherein the server is configured, for example, to provide updated context-type reference data to a device such as device 110 or a PC associated therewith. A server may have access to information and high processing capabilities due to the plurality of users and may therefore, for example, be more capable of updating the context-type reference data than device 110.

[0042] The two-phase process described above may be performed in a distributed manner, wherein a user device, such as device 110, initially receives sensor data of at least one, and in some embodiments, at least two, types. This sensor data is used to compile a message, the message comprising the sensor data at least in part in a raw or processed format, which message is transmitted from the user device to a back-end server. The server may use the sensor data in the message to determine a context in which device 110 appears to be located. This determination may, for example, be based on reference data. The server may then provide the user device with a machine-readable instruction to thereby enable the user device to infer an activity type within the context. This inferring may also be based at least in part on reference data.The reference data used in the server does not have to be the same as the reference data used in the user device.

[0043] The selection of the machine-readable instruction in the server may be based on capabilities of device 110 in addition to the sensor data. The machine-readable instruction may be selected so that sensor data device 110 can produce it, in particular, in a manner that can be accepted as input for the machine-readable instruction. To enable this selection, the message transmitted from device 110 to the server may include an indication related to device 110. Examples of such an indication include a model and manufacturer of device 110, a serial number of device 110, and an indication of the sensor types disposed in device 110. Device 110 may provide the indication to the server in a different message, as an alternative to including it in the same message.

[0044] An advantage of the distributed two-phase process is that the user device does not need to be able to detect a wide range of potential activity types with different characteristics. The size of the reference data used in the user device can be reduced, for example, by performing context detection in a server. The machine-readable instruction can at least partially comprise the reference data used in the user device. Since the size of the reference data can thus be reduced, the user device can be built with less memory, and / or the derivation of the activity type can consume less memory in the user device.

[0045] The machine-readable instruction may enable the detection of events within the context. One example is detecting the number of shots fired while hunting. Other examples include the number of times a slope has been skied down, the number of laps run around a course, the number of golf shots played, and, where applicable, the initial velocities of golf balls immediately after a shot. Other examples of detecting events include detecting the number of steps taken while running or strokes taken while swimming.

[0046] Although an activity type is generally selectable within a context, an activity type itself can be viewed as a context within which further selection may be possible. For example, an initial context may include indoor activity, where an activity type may be identified as a water sport. Within this activity type, swimming can be inferred as an activity type. Breaststroke can further be inferred as an activity type, where swimming is viewed as a context. Although the terms "context" and "activity type" are used here for convenience, what is relevant is the hierarchical relationship between the two.

[0047] A user interface of device 110 may be modified based on the context or activity type. For example, if the activity type is inferred to be hunting, a user interface may display the number of shots detected. If swimming is used, as another example, the number of laps or distance traveled may be displayed. In some embodiments, the user interface adaptations behave hierarchically, such that, for example, an outdoor user interface is initially activated in response to a determination that the current context is an outdoor context. Once an activity type has been identified within the context, the user interface may be further adapted to serve that particular activity type.In some embodiments, device 110 is configured to receive user interface information from a server to enable device 110 to adapt to a large number of situations without requiring all of those user interface adaptations to have been previously stored in device 110.

[0048] In some embodiments, a server may transmit a request message to device 110, wherein the request message is configured to cause device 110 to perform at least one measurement, such as collecting sensor data, and return a result of the measurement to the server. In this way, the server may be enabled to detect what is happening in the environment of device 110.

[0049] The machine-readable instructions may be adapted by the server. For example, a user who first receives a device 110 may initially be provided with machine-readable instructions reflecting an average user population in response to messages sent by device 110. As the user engages in activity sessions, the machine-readable instructions may subsequently be adapted to more accurately reflect usage by that particular user. For example, the length of the limb may influence periodic characteristics of sensor data collected while the user is swimming. To enable adaptation, the server may request sensor data from device 110, for example, periodically, and compare the thus-obtained sensor data with the machine-readable instructions to fine-tune the instructions for future use with that particular user.

[0050] Fig. Figure 2 illustrates an example time series with multiple sensors. The upper axis, 201, represents a humidity sensor time series 210, while the lower axis, 202, represents a time series 220 of the deviation from magnetic north from an axis of device 110.

[0051] Humidity time sequence 210 shows an initial period of low humidity followed by a rapid increase in humidity, which then remains at a relatively constant elevated level before beginning to decrease at a slower rate than the increase as device 110 dries.

[0052] The time sequence 220 of the magnetic deviation shows an initial erroneous sequence of deviation changes due to movement of the user as he operates a lock in a locker room, for example, followed by a period of approximately periodic movements before the erroneous sequence begins again. The wavelength of the periodically repeating movement is in Fig. 2 exaggerated to make the illustration clearer.

[0053] A swimming activity type can be determined as an estimated activity type starting at point 203 and ending at point 205 of the time sequence, based on a comparison with a reference dataset contained in a machine-readable instruction linked to, for example, a water sports context. Swimming can be linked via the reference dataset as an activity type with simultaneously high humidity and periodic movements.

[0054] Fig. Figure 2B illustrates a second example time sequence with multiple sensors. Similar numbers show Fig. 2B similar elements as in Fig. 2. In contrast to Fig. 2 will not be one, but two activity sessions in the time sequence of Fig. 2B. It has been determined that a cycling session begins at starting point 207 and ends at point 203, where the swimming session begins. The composite activity session can thus, for example, relate to a triathlon. During cycling, humidity remains low, and the magnetic variation changes only slowly, for example, when the user is cycling in a velodrome.

[0055] Fig. Figure 3 illustrates an example device that may support at least some embodiments of the present invention. Shown is device 300, which may include, for example, a mobile communication device, such as mobile device 110 of Fig. 1 or Fig. 2. Included in device 300 is processor 310, which may, for example, comprise a single-core or multi-core processor, where a single-core processor comprises one processing core and a multi-core processor comprises more than one processing core. Processor 310 may comprise more than one processor. A processing core may, for example, comprise a Cortex-A8 processing core manufactured by ARM Holdings or a Steamroller processing core manufactured by Advanced Micro Devices Corporation. Processor 310 may comprise at least one Qualcomm Snapdragon and / or Intel Atom processor. Processor 310 may comprise at least one application-specific integrated circuit, ASIC. Processor 310 may comprise at least one field-programmable gate array, FPGA. Processor 310 may be means for performing method steps in device 300. Processor 310 may be configured, at least in part, by computer instructions to perform actions.

[0056] Device 300 may include memory 320. Memory 320 may include random access memory (RAM) and / or persistent storage. Memory 320 may include at least one RAM chip. Memory 320 may include, for example, solid-state, magnetic, optical, and / or holographic memory. Memory 320 may be at least partially accessible to processor 310. Memory 320 may be at least partially included within processor 310. Memory 320 may be means for storing information. Memory 320 may include computer instructions that processor 310 is configured to execute.If computer instructions configured to cause processor 310 to perform certain actions are stored in memory 320, and device 300 as a whole is configured to run under the direction of processor 310 using computer instructions from memory 320, processor 310 and / or its at least one processing core may be considered configured to perform the certain actions. Memory 320 may be at least partially contained within processor 310. Memory 320 may be at least partially external to device 300 but accessible to device 300.

[0057] Device 300 may include a transmitter 330. Device 300 may include a receiver 340. Transmitter 330 and receiver 340 may be configured to transmit and receive information, respectively, according to at least one cellular or non-cellular standard. Transmitter 330 may include more than one transmitter. Receiver 340 may include more than one receiver. Transmitter 330 and / or receiver 340 may be configured to operate, for example, according to the Global System for Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), IS-95, wireless local area network (WLAN), Ethernet, and / or Worldwide Interoperability for Microwave Access (WiMAX) standards.

[0058] Device 300 may include a near-field communication (NFC) transceiver 350. NFC transceiver 350 may support at least one NFC technology, such as NFC, Bluetooth, Wi-Fi, or similar technologies.

[0059] Device 300 may include a user interface 360. UI 360 may include at least one of a display, a keyboard, a touchscreen, a vibrator arranged to signal a user by causing device 300 to vibrate, a speaker, and a microphone. A user may be able to operate device 300 using UI 360, for example, to manage activity sessions.

[0060] Device 300 may include, or be arranged to include, a user identity module 370. User identity module 370 may include, for example, a subscriber identity (SIM) card installable in device 300. User identity module 370 may include information identifying a subscription of a user of device 300. User identity module 370 may include cryptographic information useful to verify the identity of a user of device 300 and / or to facilitate encryption of communicated information and billing of the user of device 300 for communications conducted via device 300.

[0061] Processor 310 may be equipped with a transmitter arranged to output information from processor 310 via electrical lines located in device 300 to other devices included in device 300. Such a transmitter may comprise a serial bus transmitter arranged, for example, to output information via at least one electrical line to memory 320 for storage therein. The transmitter may comprise a parallel bus transmitter as an alternative to a serial bus. Processor 310 may likewise comprise a receiver arranged to receive information in processor 310 via electrical lines located in device 300 from other devices included in device 300.Such a receiver may comprise a serial bus transmitter, which is arranged, for example, to receive information via at least one electrical line from receiver 340 for processing in processor 310. As an alternative to a serial bus, the receiver may comprise a parallel bus receiver.

[0062] Device 300 may further comprise devices that are Fig. 3 are not illustrated. For example, when device 300 comprises a smartphone, it may include at least one digital camera. Some devices 300 may include a rear-facing camera and a front-facing camera, where the rear-facing camera may be for digital photography and the front-facing camera may be for video telephony. Device 300 may include a fingerprint sensor that at least partially authenticates a user of device 300. Device 300 lacks at least one device described above in some embodiments. For example, some devices 300 may lack an NFC transceiver 350 and / or a user identity module 370.

[0063] Processor 310, memory 320, transmitter 330, receiver 340, NFC transceiver 350, UI 360, and / or user identity module 370 may be interconnected by electrical wiring internal to device 300 in many different ways. For example, each of said devices may be separately connected to a master bus internal to device 300 to allow the devices to exchange information. However, as one skilled in the art will appreciate, this is only an example and depends on the embodiment. Various ways of interconnecting at least two of said devices may be chosen without departing from the scope of the present invention.

[0064] Fig. Figure 4 illustrates signaling according to at least some embodiments of the present invention. On the vertical axes, on the left side, device 110 of Fig. 1 and a server SRV on the right. Time progresses from top to bottom. In phase 410, device 110 initially receives sensor data from at least one, and in some embodiments, from at least two, sensors. The sensor data may include data from a sensor of the first type and, in some embodiments, also data from a sensor of the second type. The sensor or sensors may, for example, be contained in device 110. The sensor data may be stored in time sequences, for example, with a sampling frequency of 1 Hz, 10 Hz, 1 kHz, or indeed a different sampling interval. The sampling interval does not have to be the same as in the data from the sensor of the first type and the data from the sensor of the second type.

[0065] Phase 410 may include one or more activity sessions of at least one activity type. If multiple activity sessions are present, they may be of the same activity type or different activity types. The user need not indicate to device 110 that activity sessions are in progress, at least in some embodiments. Device 110 may identify activity types or sessions during phase 410, but need not do so in all embodiments. The time series compiled during phase 410 may last, for example, 10 or 24 hours. The time series may, as a specific example, be from the previous time at which sensor data was downloaded from device 110 to another device, such as personal computer PC1.

[0066] The sensor data is provided, at least in part, to Server SRV in raw or processed format in phase 420. This phase may further include providing Server SRV with optional activity and / or event reference data. The provision may, for example, occur via Base Station 120. The time sequences may be encrypted during download to protect user privacy.

[0067] Server SRV may determine a context and an associated machine-readable instruction based at least in part on the sensor data in the message from phase 420 in phase 430. If activity and / or event reference data is provided in phase 420, that data may be used in phase 430.

[0068] The machine-readable instruction determined in phase 430 is provided to device 110 in phase 440, enabling inference of an estimated activity type within that context based on sensor data in phase 450. The inference of phase 450 may be based on sensor data included in the message of phase 420, or device 110 may acquire new sensor data and use it with the machine-readable instruction in the inference of phase 450.

[0069] Fig. 5 is a flow diagram of a method according to at least some embodiments of the present invention. The phases of the illustrated method may be performed, for example, in device 110, an auxiliary device, or a PC, or in a control device configured to control its function when implanted therein.

[0070] Phase 510 includes storing data from a sensor of the first type in a device. Phase 520 includes compiling a message based at least in part on the data from the sensor of the first type. Phase 530 includes causing the message to be transmitted from the device. Phase 540 includes causing a machine-readable instruction to be received at the device. Finally, phase 550 includes deriving an estimated activity type using the machine-readable instruction based at least in part on sensor data. The message of phase 510 may include activity and / or event reference data.

[0071] It should be understood that embodiments of the disclosed invention are not limited to the specific structures, method steps, or materials disclosed herein, but extend to their equivalents, as will be known to those skilled in the relevant arts. It should also be understood that the terminology used herein is used for the purpose of describing particular embodiments only and is not intended to be limiting.

[0072] References in this specification to "one (1) embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the teachings. The appearances of the phrases "in one (1) embodiment" or "in an embodiment" in various places in this specification are not necessarily all referring to the same embodiment. When a numerical value is referred to by a term such as about or substantially, the exact numerical value is also disclosed.

[0073] For convenience, many items, structural elements, compositional elements, and / or materials may be presented herein in a common list. However, these lists should be viewed as if each item of the list is individually identified as a separate and distinct item. Therefore, no individual item of this list should be considered a de facto equivalent to any other item of the same list simply because they are presented in a common group, unless the contrary is specified. Moreover, various embodiments and examples of the present invention may be recited herein, along with alternatives for their various components. It should be understood that such embodiments, examples, and alternatives should not be construed as de facto equivalents to one another, but should be viewed as separate and autonomous representations of the present invention.

[0074] The described features, structures, or characteristics may further be combined in any suitable manner in one or more embodiments. In the foregoing description, numerous specific details are provided, such as example lengths, widths, shapes, etc., in order to provide a thorough understanding of embodiments of the invention. However, one skilled in the art will recognize that the invention may be practiced without one or more of the specific details, or with different methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.

[0075] While the foregoing examples illustrate the principles of the present invention in one or more specific applications, it will be apparent to those skilled in the art that numerous modifications in form, usage, and details of implementation may be made without claiming the inventive faculty and without departing from the principles and concepts of the invention. Accordingly, it is not intended that the invention be limited except as by the claims described below.

[0076] The verbs "comprise" and "include" are used in this document as open limitations that neither exclude nor require the presence of other unmentioned features. The features mentioned in the dependent claims can be freely combined with each other unless explicitly stated otherwise. It should also be noted that the use of "a," i.e., a singular form, in this document does not exclude a plural form. INDUSTRIAL APPLICABILITY

[0077] At least some embodiments of the present invention find industrial application in facilitating the analysis of sensor data. LIST OF ACRONYMS GPS Global Positioning System LTE Long Term Evolution NFC near-field communication WCDMA Wideband Code Division Multiple Access WiMAX Worldwide Interoperability For Microwave Access WLAN Wireless local area network

[0078] LIST OF REFERENCE SYMBOLS 110 device 120 base station 130 network nodes 140 Network 150 satellite constellation 201, 202 axes in Fig. 2 203, 205, 207 Endpoints of the activity session in Fig. 2 and Fig. 2B 210, 220 Time sequences of the sensor data in the Fig. 2 and Fig. 2B 310-370 in Fig. 3 illustrated structure 410-430 phases of the procedure of Fig. 4 510-530 Phases of the procedure of Fig. 5

Claims

[1] Device for identifying a user activity, comprising: - a memory configured to store data from a sensor of the first type relating to an activity; - at least one processing core configured to: Compiling a message based at least in part on the data from the sensor of the first type, Causing the message to be transmitted from the device to a server external to the device, Causing a response to the message to be received in the device from the server as a machine-readable instruction comprising at least two machine-readable characteristics, wherein each of the at least two characteristics characterizes sensor data produced during a predefined activity type, wherein the at least two machine-readable characteristics comprise reference data specific to a context in which the device operates, and Deriving an estimated activity type using the reference data specific to a context based at least in part on sensor data by comparing the sensor data to the reference data specific to a context. [2] The apparatus of claim 1, wherein the machine-readable instruction comprises at least one of the following: an executable program and an executable script. [3] The device of claim 1 or 2, wherein the at least one processing core is configured to derive the estimated activity type at least in part by comparing the first type sensor data or a processed form of the first type sensor data with reference data specific to a context in which the device operates. [4] The apparatus of any one of claims 1 to 3, wherein the data of the first type of sensor comprises acceleration sensor data. [5] The apparatus of any one of claims 1 to 4, wherein the memory is further configured to store data from a second type of sensor, and wherein the at least one processing core is configured to derive the estimated activity type using the machine-readable instruction based at least in part on data from the second type of sensor. [6] The apparatus of claim 4, wherein the data of the second type sensor is of a different type than the data of the first type sensor. [7] The apparatus of any one of claims 5 to 6, wherein the second type sensor data comprises at least one of sound sensor data, microphone derived data, and vibration sensor data. [8] The apparatus of any one of claims 5 to 7, wherein the at least one processing core is configured to derive the estimated activity type at least in part by comparing the second type sensor data or a processed form of the second type sensor data with reference data, the reference data comprising first type and second type reference data. [9] The apparatus of any one of claims 1 to 8, wherein the at least one processing core is configured to present the estimated activity type to a user for verification. [10] The apparatus of any one of claims 1 to 9, wherein the at least one processing core is configured to cause the memory to store the estimated activity type and a second estimated activity type in a sequence of estimated activity types. [11] The apparatus of any of claims 1 to 10, wherein the at least one processing core is configured to cause the memory to delete the machine-readable instruction in response to a determination that an activity session has ended. [12] A server comprising at least one processing core, at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to cause the server to perform at least the following with the at least one processing core: - receiving a message from a user device, the message comprising information characterising data from the sensor of the first type relating to an activity; - determining an activity context of the user device based at least in part on the data from the sensor of the first type, and - Transmitting a response to a user terminal comprising a machine-readable instruction comprising at least two machine-readable characteristics, wherein each of the at least two machine-readable characteristics characterizes sensor data produced during a predefined activity type, wherein the at least two machine-readable characteristics comprise reference data specific to a context in which the device operates, and wherein the machine-readable instruction is configured to cause determination of the activity type in the activity context of the user device. [13] A method performed in a device for identifying a user activity, comprising: - storing data from a sensor of the first type relating to an activity in a device; - compiling a message based at least in part on the data from the sensor of the first type; - causing the message to be transmitted from the device to a server external to the device; - causing a response to the message to be received in the device from the server as a machine-readable instruction comprising at least two machine-readable characteristics, each of the at least two machine-readable characteristics characterizing sensor data produced during a predefined activity type, the at least two machine-readable characteristics comprising reference data specific to a context in which the device operates, and - Deriving an estimated activity type using the machine-readable instruction based at least in part on sensor data. [14] The method of claim 13, wherein the machine-readable instruction comprises at least one of the following: an executable program and an executable script. [15] The method of claim 13 or 14, wherein the estimated activity type is derived at least in part by comparing the data of the first type of sensor or a processed form of the data of the first type of sensor with reference data. [16] The method of any one of claims 13 to 15, wherein the data from the first type of sensor comprises acceleration sensor data. [17] The method of any one of claims 13 to 16, further comprising storing data from a second type of sensor, and wherein the estimated activity type is derived using the machine-readable instruction based at least in part on data from the second type of sensor. [18] The method of claim 17, wherein the data of the second type sensor is of a different type than the data of the first type sensor. [19] The method of any one of claims 17 to 18, wherein the second type of sensor data comprises at least one of sound sensor data, microphone derived data, and vibration sensor data. [20] The method of any one of claims 17 to 19, wherein the estimated activity type is derived at least in part by comparing the second type sensor data or a processed form of the second type sensor data with reference data, the reference data comprising first type and second type reference data. [21] The method of any one of claims 13 to 20, further comprising presenting the estimated activity type to a user for verification. [22] The method of any one of claims 13 to 21, further comprising storing the estimated activity type and a second estimated activity type in a sequence of estimated activity types. [23] The method of any of claims 13 to 22, further comprising deleting the machine-readable instruction in response to a determination that an activity session has ended. [24] Method performed in a server, comprising: - receiving a message from a user device, the message comprising information characterising data from the sensor of the first type relating to an activity; - determining an activity context of the user device based at least in part on the data from the sensor of the first type, and - Transmitting a machine-readable instruction to a user device comprising at least two machine-readable characteristics, wherein each of the at least two machine-readable characteristics characterizes sensor data produced during a predefined activity type, wherein the at least two machine-readable characteristics comprise reference data specific to a context in which the device operates, and wherein the machine-readable instruction is configured to cause determination of the activity type in the activity context of the user device. [25] A non-transitory computer-readable medium having stored thereon a set of computer-readable instructions which, when executed by at least one processor, cause a device to at least: - stores data from a sensor of the first type relating to an activity; - compiles a message based at least in part on the data from the sensor of the first type; - causes the message to be transmitted from the device to a server external to the device; - causes a response to the message to be received in the device from the server as a machine-readable instruction comprising at least two machine-readable characteristics, each of the at least two machine-readable characteristics characterizing sensor data produced during a predefined activity type, the at least two machine-readable characteristics comprising reference data specific to a context in which the device operates, and - derives an estimated activity type using the machine-readable instruction based at least in part on sensor data. [26] A computer program configured to effect the performance of a method according to at least one of claims 13 to 24.

Images