Machine segmentation of sensor measurements and derivatives in virtual motion testing.

Abstract

The user device may machine segment the sensor readings by determining a context window. The context window may include a start point and an end point. The context window may be used to define a period of time within which the sensor readings are segmented.

Description

[Background technology]

[0001] A wearable user device, such as a wristwatch, may use one or more sensors to sense data representing the wearer's physiological signals. In some cases, particular sensors may be used (or configured with different sampling rates) when the wearer performs a given motion or series of motions requested by the wearable user device. During this time, the sensor data collected may have varying relevance to the given motion or series of motions. Summary of the Invention

[0002] Various embodiments are described, including systems, methods, and devices for segmenting signal data associated with a virtual clinical exam.

[0003] One or more computer systems can be configured to perform specific operations or actions by installing software, firmware, hardware, or a combination thereof on the system that, when operated, causes the system to perform the action. One or more computer programs can be configured to perform specific operations or actions by including instructions that, when executed by a data processing device, cause the device to perform the action. One general aspect includes a computer-implemented method for utilizing a wearable user device to access test information identifying (i) a first timing indicator associated with a first time, (ii) a second timing indicator associated with a second time, and (iii) a virtual movement test type of a virtual movement test. The method also includes accessing, by the wearable user device, signal data acquired by the wearable user device during a time period bounded by the first time and the second time. The method also includes determining, by the wearable user device and based on the virtual movement test type, a first signal data type for segmenting the signal data and first signal data of the first signal data type output by a first sensor of the wearable user device during the time period. The method also includes determining, by the wearable user device, a context window within the time period by at least selecting a historical signal profile of the first signal data type, the historical signal profile being derived from a previous occurrence of the virtual athletic test. Determining the context window also includes comparing the first signal data to the historical signal profile to identify a third time corresponding to a start of the context window and a fourth time corresponding to an end of the context window. The method also includes segmenting, by the wearable user device, a portion of the signal data received during the context window. The method also includes generating, by the wearable user device, a virtual athletic test data package based on the portion of the signal data and the test information.Other embodiments of this aspect include corresponding computer systems, devices, and computer programs stored on one or more computer storage devices, each configured to perform the actions of the method.

[0004] Another general aspect includes a computer-implemented method of receiving, at an input device of a wearable user device, a first user input identifying a start point of a first time period during which a virtual movement test will be performed. The method also includes receiving, at an input device of the wearable user device, a second user input identifying an end point of the first time period. The method also includes accessing, by the wearable user device and based on the virtual movement test, first signal data output by a first sensor of the wearable user device during the first time period. The method also includes determining, by the wearable user device, a context window within the first time period based on the first signal data and a virtual movement test type associated with the virtual movement test, the context window defining a second time period within the first time period. The method also includes determining, by the wearable user device, second signal data output by a second sensor of the wearable user device during the second time period. The method also includes associating, by the wearable user device, the second signal data with the virtual movement test. Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs stored on one or more computer storage devices, each configured to perform the actions of the method. [Brief explanation of the drawings]

[0005] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate one or more specific embodiments and, together with the description, serve to explain the principles and implementations of the particular embodiments.

[0006] [Figure 1]FIG. 1 illustrates an exemplary system including a wearable device for use in implementing techniques relating to segmenting sensor data for virtual athletic testing, according to at least one embodiment. [Figure 2] FIG. 2 shows a block diagram and corresponding flowchart illustrating the process of segmenting sensor data for a virtual motion test, according to at least one embodiment. [Figure 3] FIG. 3 shows an example flowchart illustrating a process involved in implementing techniques for segmenting sensor data for a virtual motion test, according to at least one embodiment. [Figure 4] FIG. 4 illustrates a diagram including example sensors and segmented sensor data, according to at least one embodiment. [Figure 5] FIG. 5 illustrates a diagram including example sensors and segmented sensor data for a single device, according to at least one example. [Figure 6] FIG. 6 illustrates a diagram including example sensors and segmented sensor data for a single device, according to at least one example. [Figure 7] FIG. 7 illustrates a detailed view of a portion of the diagram shown in FIG. 4, according to at least one embodiment. [Figure 8] FIG. 8 illustrates a diagram including example sensors and segmented sensor data for different devices, according to at least one embodiment. [Figure 9] FIG. 9 shows an example flowchart illustrating a process for implementing techniques for segmenting sensor data for a virtual motion test, according to at least one embodiment. [Figure 10] FIG. 10 shows an exemplary flowchart illustrating a process for implementing techniques for segmenting sensor data for a virtual motion test, according to at least one embodiment. [Figure 11] FIG. 11 illustrates an exemplary architecture for implementing techniques for segmenting sensor data for virtual motion testing, according to at least one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0007] Examples are described herein in the context of segmenting sensor data collected by a wearable user device while performing a virtual athletic test on the wearable user device. Those skilled in the art will appreciate that the following description is exemplary only and is not intended to be limiting in any way. For example, the techniques described herein can be used to segment sensor data collected during different types of structured tests, activities, and / or unstructured time periods, and in some examples may be implemented on non-wearable user devices. Reference will now be made in detail to implementations of the examples illustrated in the accompanying drawings. The same reference labels will be used throughout the drawings and the following description to refer to the same or similar items.

[0008] For purposes of clarity, not all of the routine features of the embodiments described herein are shown and described. In the development of any such actual implementation, it will be understood that numerous implementation-specific decisions will, of course, need to be made to achieve the developer's specific goals, such as compliance with application- and business-related constraints, and that these specific goals will vary from implementation to implementation and from developer to developer.

[0009] Parkinson's disease and other neurological disorders can cause motor symptoms. Traditionally, trained clinicians perform motor tests in a clinic or at a patient's home to help determine whether a patient's symptoms are related to a specific movement disorder, such as Parkinson's disease, and to track the progression of such disorders. For example, during the physical component of a Parkinson's disease test, a clinician looks for tremor (e.g., repetitive movements caused by involuntary muscle contractions), rigidity (e.g., stiffness in the arms or legs), bradykinesia or akinesia (e.g., slowness and / or lack of movement during normal tasks), and postural instability (e.g., problems with natural balance). In some examples, at least a portion of the test may be based on the Unified Parkinson's Disease Rating Scale (UPDRS). Using the wearable devices described herein, patients can perform these (and other types of) tests at home without medical supervision. The wearable device includes logic to direct the patient's activity, which in some examples may require different types of movement or immobility. As described in detail herein, the wearable device includes multiple sensors that collect sensor data as the patient performs these activities. This sensor data can then be processed to derive the patient's physiological signals and identify findings that are the same as or similar to those made by a physician during a consultation.

[0010] Sensors may be configured to sample at different rates depending on whether activity is being recorded. In some examples, the patient may indicate when the activity begins and ends. Sensors may collect a large amount of sensor data during this period, such as by increasing the sampling rate, increasing the sampling resolution, increasing the number of sensing channels, using more active sensors, etc. However, it has been observed that the portion of data collected during this period is more probative of certain important metrics related to the test than other periods. For example, one test may test how well a patient can hold their hands in their lap without moving them while sitting. To begin, the patient may select a “Start” button on the user interface of the wearable device. If the patient is not already sitting, they must sit down and then move their hands to their lap. During this “preparation” time, sensors may sample a large amount of data, which, in some examples, may not necessarily be probative of how well the patient will perform on the current test. Therefore, it is desirable to identify the point in time when the data indicates that the patient has finished preparation and is actually performing the test. The techniques described herein are adapted to determine this actual start time and actual end time. The window of time during which the patient is actually performing the test may be referred to herein as the context window and has a start point and an end point. The context window may be determined using the historical signal profile for the first type of sensor and the sensor data (e.g., the first sensor measurements and derived values) collected by the first type of sensor. Once the context window is defined using sensor data from one sensor (or in some examples, more sensors), the context window can be used to machine segment the portions of the sensor data collected by the device's various sensors that are most important to the test. Machine segmentation may include segmenting the sensor data in an automatic manner.Other sensor data collected before and after the context window may be ignored, or at least given less weight, in the analysis of how well the patient performed on the test. Similar techniques can be applied to sensors housed on different devices that may not be time-synchronized with the sensors on the wearable device. Additionally, applying the techniques described herein to sensors on different devices may enable time alignment between sensor data collected using the other device and the wearable device.

[0011] In certain examples, a patient is provided with a wearable device, such as a wristwatch, as part of a disease progression program. The wristwatch may include a set of sensors configured to track the patient's various movements, heart rate, etc., and software for conducting various virtual exercise tests. The virtual exercise tests may be accessed upon user request, and / or the wristwatch may suggest an appropriate time to conduct the test. In either case, the patient may select a button (e.g., a physical button or a graphical user interface (“GUI”) element) to begin the test and the same or a different button to end it. The wearable device may generate timestamps indicating the start and end of the test, which may be associated with a test identifier (e.g., an identifier that uniquely identifies the type of test) and a session identifier (e.g., an identifier that uniquely identifies the session in which the test was conducted). Additionally, during the test, the wearable device may instruct multiple sensors to collect sensor data, which may be obtained in the form of sensor measurements and derived values ​​and / or user-input data. The sensor measurements and other collected sensor data may be in the form of signal data. After or during the test, using the techniques described herein, the wearable device may determine a context window representing a portion of the test during which signal data indicates that the patient is performing an activity related to the test. To do this, the wearable device selects sensor data collected during the test from a first sensor and compares the sensor data to historical signal profiles for that test type and the same type of sensor. The results of this comparison, which may be performed heuristically and / or using a machine learning model, may include a start point of the context window and, in some examples, an end point of the context window. The start and end points of the context window may be expressed as timestamps, which can then be used to segment the portion of sensor data from the first sensor and the output from other sensors on the wearable device.Once the sensor data is segmented, the wearable device uses the segmented signal data and information about the test to generate a virtual exercise test data package. This virtual exercise test data package may be shared with a remote server for further processing and / or may be shared with the patient's physician. In some examples, the output may also be used to adjust the operation of the sensors during future tests.

[0012] This illustrative example is provided to introduce the reader to the general subject matter discussed herein, and the present disclosure is not limited to this example. The following sections describe various additional non-limiting examples of techniques related to segmenting signal data collected during a virtual movement test.

[0013] The technology described herein enables one or more technical improvements to the computer that performs the virtual exercise test. For example, the sampling rate of sensors in future tests may be adjusted based on information learned during the analyzed test, thereby conserving battery power in the portable user device. Additionally, the approach described herein can improve the authenticity of data collected during a test, and a context window may be determined by the output from one sensor and applied to the output from all other sensors, avoiding the tedious, processor-intensive process of post-processing signal alignment for all sensor data. This approach conserves computing resources on the user device, allowing these resources to be used for other purposes, such as processing sensor data, updating user interfaces, etc. Instead of sending sensor data to a remote server for processing, the data is processed by the wearable device at least until a data package is generated, which may be securely encrypted and shared with the remote server, thus protecting patient privacy.

[0014] Turning now to the figures, Figure 1 shows an exemplary system including a user device for use in implementing techniques related to segmenting sensor data for virtual athletic testing, according to at least one embodiment. The system 100 includes a user device 102, such as a wearable user device, that may communicate with various other devices and systems via one or more networks 104.

[0015] The embodiments described herein may take the form of, be incorporated into, or operate in conjunction with a suitable wearable electronic device, such as, for example, a device that may be worn on a user's wrist and secured by a band. The device may have a variety of functions, including, but not limited to, keeping time, monitoring a user's physiological signals and providing health-related information based at least in part on those signals, communicating (wired or wirelessly) with other electronic devices (which may be different types of devices with different functions), providing alerts to the user (which may include audio, tactile, visual, and / or other sensory output, any or all of which may be synchronized with each other), visually depicting data on a display, collecting data from one or more sensors that may be used to initiate, control, or modify the operation of the device, determining the location of touch on the surface of the device and / or the amount of force applied to the device and using either or both as inputs, receiving audio input to control one or more functions, receiving tactile input to control one or more functions, etc.

[0016] As shown in Figure 1, the user device 102 includes one or more processor units 106 configured to access a memory 108 having instructions stored thereon. The processor units 106 of Figure 1 may be implemented as any electronic device capable of processing, receiving, or transmitting data or instructions. For example, the processor units 106 may include one or more microprocessors, central processing units (CPUs), application specific integrated circuits (ASICs), digital signal processors (DSPs), or combinations of such devices. As used herein, the term "processor" is meant to encompass a single processor or processing unit, multiple processors, multiple processing units, or any other suitably configured single or multiple computing elements.

[0017] Memory 108 may include removable and / or non-removable elements, both of which are examples of non-transitory computer-readable storage media. For example, non-transitory computer-readable storage media may include volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Memory 108 is an example of a non-transitory computer storage medium. Additional types of computer storage media that may be present in user device 102 include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital video disk (DVD) or other optical storage device, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other medium that can be used to store desired information and that can be accessed by user device 102. Combinations of any of the above should also be included within the scope of non-transitory computer-readable storage media. Alternatively, computer-readable communication media may include computer-readable instructions, program modules, or other data transmitted within a data signal such as a carrier wave or other transmission. However, as used herein, computer-readable storage media does not include computer-readable communication media.

[0018] In addition to storing computer-executable instructions, the memory 108 may be configured to store historical sensor data profiles. The historical sensor data profiles may identify configuration settings for operating the sensors of the user device 102 for a particular type of test. In some examples, the historical sensor data profiles may be generated using historical data collected from other occurrences of tests performed on other users in controlled or uncontrolled environments. Machine learning techniques may be applied to the historical data to construct the profiles. The historical sensor data profiles may include a tagged start point of the actual test and a tagged end point of the actual test, which may differ from the start and end points identified by the user. The historical data profiles may be received by the user device 102 from a server computer or other external device that has access to sensor data for multiple users.

[0019] The instructions or computer program may be configured to perform one or more of the operations or functions described in connection with the user device 102. For example, the instructions may be configured to control or coordinate the operation of various components of the device, including, but not limited to, a display 110, one or more input / output (I / O) components 112, one or more communication channels 114, one or more motion sensors 116, one or more environmental sensors 118, one or more biosensors 120, a speaker 122, a microphone 124, a battery 126, and / or one or more haptic feedback devices 128.

[0020] The display 110 may be configured to display information via one or more graphical user interfaces and may also function as an input component, for example, a touch screen. Messages regarding the performance of tests may be presented on the display 110 using the processor unit 106.

[0021] The I / O component 112 may include a touchscreen display, as described, and may also include one or more physical buttons, knobs, and the like located in any suitable position relative to the bezel of the user device 102. In some embodiments, the I / O component 112 may be located on a band of the user device 102.

[0022] The communication channel 114 may include one or more antennas and / or one or more network radios that enable communication between the user device 102 and one or more other external sensors 130, other electronic devices such as smartphones or tablets, other wearable electronic devices, or external computing systems such as desktop computers or network-connected servers. In some examples, the communication channel 114 may enable the user device 102 to pair with a primary device such as a smartphone. The pairing may be via Bluetooth or Bluetooth Low Energy (“BLE”), near-field communication (“NFC”), or other suitable network protocols and may enable some persistent data sharing. For example, data from the user device 102 may be periodically streamed and / or shared with the smartphone, which may process the data and / or share it with a server. In some examples, the user device 102 may be configured to communicate directly with a server over any suitable network, e.g., the Internet, a cellular network, etc.

[0023] The sensors of the user device 102 may generally be divided into three categories, including motion sensors 116, environmental sensors 118, and biosensors 120. As described herein, references to one or more "sensors" may include one or more sensors from any one and / or more of the three categories of sensors. In some examples, sensors may be implemented as hardware elements and / or in software.

[0024] Generally, the motion sensor 116 may be configured to measure acceleration and rotational forces along three axes. Examples of motion sensors include an accelerometer, a gravity sensor, a gyroscope, a rotation vector sensor, a movement detection sensor, a pedometer sensor, a global positioning system (GPS) sensor, and / or any other suitable sensor. Motion sensors may be useful for monitoring device movement, such as tilt, vibration, rotation, or shaking. The movement may be a reflection of direct user input (e.g., a user steering a car in a game or a user controlling a ball in a game), but may also be a reflection of the physical environment in which the device is located (e.g., a car moving with a driver). In the first case, the motion sensor may monitor movement relative to the device's frame of reference or your application's frame of reference; in the second case, the motion sensor may monitor movement relative to the world's frame of reference. Motion sensors themselves are not typically used to monitor the device's location, but can be used in conjunction with other sensors, such as an Earth's magnetic field sensor, to determine the device's location relative to the world's frame of reference. The motion sensors 116 may return a multidimensional array of sensor values ​​for each event when the sensors are active. For example, during a single sensor event, an accelerometer may return acceleration force data for three coordinate axes, and a gyroscope may return rotation rate data for three coordinate axes.

[0025] In general, the environmental sensor 118 may be configured to measure environmental parameters such as temperature and pressure, lighting, and humidity. The environmental sensor 118 may also be configured to measure the physical location of the device. Examples of the environmental sensor 118 may include a barometer, a light meter, a thermometer, a direction sensor, a magnetometer, a global positioning system (GPS) sensor, and any other suitable sensor. The environmental sensor 118 may be used to monitor relative ambient humidity, lighting, ambient air pressure, and ambient temperature near the user device 102. In some examples, the environmental sensor 118 may return a multidimensional array of sensor values ​​for each sensor event, or a single sensor value for each data event. For example, temperature (°C) or pressure (hPa). Also, unlike the motion sensor 116 and the biosensor 120, which may require high-pass or low-pass filtering, the environmental sensor 118 may not typically require any data filtering or processing.

[0026] The environmental sensors 118 may also be useful for determining the device's physical location in a world frame of reference. For example, an Earth magnetic field sensor may be used in combination with an accelerometer to determine the user device's 102 position relative to the magnetic north pole. These sensors may also be used to determine the user device's 102 orientation in a portion of a frame of reference (e.g., within a software application). The Earth magnetic field sensor and accelerometer may return a multidimensional array of sensor values ​​for each sensor event. For example, the Earth magnetic field sensor may provide Earth magnetic field strength values ​​for each of three coordinate axes during a single sensor event. Similarly, the accelerometer sensor may measure the acceleration applied to the user device 102 during a sensor event. A proximity sensor may provide a single value for each sensor event.

[0027] In general, biosensor 120 may be configured to measure biometric signals of a wearer of user device 102, such as, for example, heart rate, blood oxygen level, sweating, skin temperature, etc. Examples of biosensor 120 may include heart rate sensors (e.g., photoplethysmography (PPG) sensors, electrocardiogram (ECG) sensors, electroencephalogram (EEG) sensors, etc.), pulse oximeters, moisture sensors, thermometers, and any other suitable sensors. Biosensor 120 may return a multidimensional array of sensor values ​​and / or, depending on the sensor, may return a single value.

[0028] The acoustic elements, such as the speaker 122 and microphone 124, may share a port in the housing of the user device 102 or may include dedicated ports. The speaker 122 may include drive electronics or circuitry and may be configured to generate an audible sound or acoustic signal in response to a command or input. Similarly, the microphone 124 may also include drive electronics or circuitry and be configured to receive an audible sound or acoustic signal in response to a command or input. The speaker 122 and microphone 124 may be acoustically coupled to a port or opening in the case that may allow the passage of acoustic energy but prevent the ingress of liquids and other debris.

[0029] The battery 126 may include any suitable device for providing power to the user device 102. In some examples, the battery 126 may be rechargeable or single-use. In some examples, the battery 126 may be configured for inductive (e.g., air) or near-field charging.

[0030] The haptic device 128 may be configured to provide haptic feedback to the wearer of the user device 102. For example, alerts, commands, and the like may be communicated to the wearer using the speaker 122, the display 110, and / or the haptic device 128.

[0031] The external sensors 130(1)-130(n) may be any suitable sensors, such as motion sensors 116, environmental sensors 118, and / or biosensors 120, embodied in any suitable device. For example, the sensors 130 may be incorporated into other user devices, which may be single-purpose or multipurpose. For example, a heart rate sensor may be incorporated into a chest band used to capture heart rate data at the same time that the user device 102 captures sensor data. In other examples, position sensors may be incorporated into devices and worn in different locations on a human user. In this example, the position sensors may be used to track the position of body parts (e.g., hands, arms, legs, feet, head, torso, etc.). Any of the sensor data obtained from the external sensors 130 may be used to implement the techniques described herein.

[0032] 2 illustrates a system 202 and corresponding flowchart illustrating a process 200 for segmenting sensor data for a virtual athletic test, according to at least one embodiment. The system 202 includes a service provider 204 and a user device 206. FIG. 2 illustrates certain operations performed by the user device 206 as it relates to information for segmenting the sensor data. The user device 206 is one embodiment of the user device 102 introduced previously.

[0033] As described in further detail herein, the service provider 204 may be any suitable computing device (e.g., a personal computer, a handheld device, a server computer, a server cluster, a virtual computer) configured to execute computer-executable instructions to perform operations as described herein. The computing device may be remote from the user device 206. As described herein, the user device 206 is any suitable portable electronic device (e.g., a wearable device, a handheld device, an implantable device) configured to execute computer-executable instructions to perform operations as described herein. The user device 206 includes one or more on-board sensors 208. The sensors 208 are examples of the sensors 116-120 described herein.

[0034] The service provider 204 and the user device 206 may be in network communication via any suitable network, such as the Internet, a cellular network, etc. In some examples, the user device 206 may be in intermittent network communication with the service provider 204. For example, the network communication may enable the transfer of data (e.g., virtual test data packages, calibration information) that may be used by the service provider 204 to share with interested parties and to improve the management of tests on the user device 206 and other user devices 206. In some examples, the user device 206 is in network communication with the service provider 204 via a primary device. For example, the user device 206 may be a wearable device, such as a wristwatch, as shown. In this example, the primary device may be a smartphone that connects to the wearable device via a first network connection (e.g., Bluetooth) and to the service provider 204 via a second network connection (e.g., cellular). However, in some examples, the user device 206 may include appropriate components that enable the user device 206 to communicate directly with the service provider 204.

[0035] 2 provides an overview of how the system 202 may be used to segment sensor data for a virtual athletic test. The process 200 may begin at block 210 by the user device 206 accessing test information. The test information may be generated by the user device 206 during or after the virtual athletic test is performed. The test information may indicate characteristics of the virtual athletic test, such as the type of virtual athletic test, tasks associated with that type of virtual athletic test, user- or system-provided timestamps identifying the start and end points of the test, user-provided feedback regarding the test, and other information regarding the test. In some examples, the user device 206 accesses the test information from memory of the user device 206.

[0036] At block 212, the user device 206 accesses sensor data 214 associated with the test information and acquired by a sensor 208(1) (e.g., one of the sensors 208). In some embodiments, the sensor data 214 may have been collected during the test identified by the test information accessed in block 210. In some embodiments, the sensor data 214 may be processed (e.g., filtered, digitized, packetized, etc.) by the sensor generating the sensor data 214. In some embodiments, the sensor 208 provides the sensor data 214 without any processing. Logic on the user device 206 may control the operation of the sensor 208 with respect to collecting data during the test. All of the sensors 208 may be time-aligned because they are all on the same device (e.g., the user device 206) and are thereby aligned with the same internal clock (e.g., the clock of the user device 206). If this is not the case, techniques described herein may be used to time-align the sensor data in addition to segmenting it.

[0037] At block 216, the user device 206 determines a context window 218 within the period for which the sensor data was collected. As illustrated in test information block 220, the test may include a start point 222 and an end point 224 indicated by the test information accessed in block 210, as well as sensor data output from two or more sensors 208(1) and 208(2) that was at least partially acquired in block 212. Block 216 includes using the sensor data output from sensor 208(1) to determine a start point 226 of the context window, and in some examples, an end point 228 of the context window 218. Various approaches for implementing block 216 are described herein. In some examples, this may include comparing the sensor data output by sensor 208(1) to a historical signal profile of the sensor data output by sensor 208(1).

[0038] At block 230, the user device 206 segments a portion of the signal data received during the context window 218. This may include using the start point 226 and end point 228 to define a period of time that includes sensor data that is perhaps more interesting than data acquired outside the context window 218 (e.g., between start point 222 and start point 226, and between end point 228 and end point 224). Segmenting the portion of the signal data may include using the context window 218 to not only segment the data output by sensor 208(1), but also segment the data output by sensor 208(2). In this manner, a context window determined using sensor data output by one sensor 208 of the user device 206 may be applied to sensor data output by any number of other sensors 208 of the user device 206.

[0039] At block 232, the user device 206 generates adjustment information for adjusting one or more sensors 208. This may include using the determined start point 226 and end point 228 as control points for determining when to adjust the sampling rate and / or when to turn the sensor 208 on and off during future tests (e.g., future sensor events). The adjustment information may include control information for controlling operation of the one or more sensors 208 during future sensor events. In some examples, the adjustment information includes updated parameter values ​​for the sensor 208. This may include offsets, calibrations, etc. In some examples, the process 200 may also include the user device 206 adjusting the sensor 208 using the adjustment information.

[0040] At block 234, the user device 206 generates a virtual exercise test data package 236, which may include test information, segmented sensor data, and, in some examples, a context window. In some examples, the virtual exercise test data package 236 may include other information related to the virtual exercise test. For example, images, video, text, etc. may be bundled with the virtual exercise test data package. In some examples, the segmented sensor data and information defining the context window may be identified by the user device 206 and shared with the service provider 204 over a network, such as the network 104, as described herein. The virtual exercise test data package 236 may be usable by the user device 206 and / or the service provider 204 to evaluate how the user performed on the test. In some examples, the service provider 204 may share aspects of the virtual test data package 236 with other users, such as a medical professional overseeing the administration of the virtual test.

[0041] 3, 9, and 10 show example flow diagrams illustrating processes 300, 900, and 1000 according to at least some embodiments. These processes, and any other processes described herein (e.g., process 200), are illustrated as logical flow diagrams, each operation of which represents a sequence of operations that may be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, operations may represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to perform a process.

[0042] Additionally, some, any, or all of the processes described herein may be performed under the control of one or more computer systems configured with specific executable instructions, or may be performed by hardware, as code (e.g., executable instructions, one or more computer programs, one or more applications) collectively executed on one or more processors, or any combination thereof. As noted above, code may be stored on a non-transitory computer-readable storage medium, for example, in the form of a computer program including a plurality of instructions executable by one or more processors.

[0043] FIG. 3 shows an example flowchart illustrating a process 300 for implementing a technique for segmenting sensor data for a virtual exercise test, according to at least one embodiment. FIGS. 4-7 show diagrams including various example sensors, example segmented sensor data, and various devices. FIGS. 4-7 are introduced with respect to FIG. 3. Process 300 is performed by user device 102 (FIG. 1). Process 300 specifically addresses various approaches to segmenting sensor data, according to various embodiments.

[0044] Process 300 begins at block 302 by the user device 102 determining a starting point for a period of a particular type of virtual clinical exam. This may include determining the starting point based on exam information such as those described in FIG. 2 at 210, 212, and 216. This may include using a timestamp corresponding to a user input at the user device 206.

[0045] At block 304, the process 300 includes the user device 102 accessing historical sensor data profiles associated with a particular type of virtual test and the sensors used to collect sensor data during the virtual clinical test. This may include the user device 102 using a set of evaluation rules to determine which historical sensor data profiles are appropriate. In some examples, the historical sensor data profiles may be accessed from memory of the user device 102 and / or requested from an external computing system. The evaluation rules may define which profiles are appropriate for a particular test type. The historical sensor data profiles may be specific to the type of test (e.g., sitting and standing, hand movements, one-leg balance) and the type of sensor (e.g., accelerometer, gyroscope, heart rate monitor, etc.).

[0046] At block 306, the process 300 includes the user device 102 aligning the start of the time period with a corresponding start of the historical sensor data profile. This may include using a time-stamped time of the start of the time period (e.g., when the user requested to start a test or otherwise interacted with the user device 102) and the corresponding time in the historical sensor data profile. In some examples, the sensor data profile may have one or more other alignment points to ensure proper alignment with the time period. For example, certain values ​​(e.g., highest and / or lowest values) in the historical data profile may be tagged as alignment points and matched to the highest and / or lowest values ​​in the sensor data.

[0047] At block 308, the process 300 includes the user device 102 determining differences between a portion of the signal data and a portion of the historical sensor data profile. Because it can be assumed that the signal data does not perfectly match the historical sensor data profile, the user device 102 can compare the two and identify locations of small and / or large differences, for example, based on an average difference between corresponding data points below a threshold, a set of evaluation rules, or an overall similarity between the different profiles. For example, the evaluation rules may indicate that for a particular test type and for this particular sensor, the user device 102 should expect to observe large signal fluctuations during "warm-up time" and low signal fluctuations during the actual test. The historical signal profile may represent an averaged or learned representation of these fluctuations.

[0048] At block 310, the process 300 includes the user device 102 using the historical signal profile to determine whether the difference is within some threshold. A small difference may indicate that a portion of the signal data is consistent with the historical signal profile. If the difference is too large, the process 300 may return to block 308 to continue determining the difference. If the difference is within the threshold, the process 300 may continue to block 312.

[0049] At block 312, the process 300 includes the user device 102 determining the start of the context window when the difference is within the threshold. In this example, because the difference at that time is within the threshold, the start of the historical signal profile may be set as the start of the context window. In some examples, other locations along the historical signal profile may be compared to identify the end and / or different start of the context window. Similar threshold comparisons may be performed at these selected locations to determine whether other points are consistent with the historical signal profile. For example, the sensor data may be divided into several equal time intervals (e.g., 10), and the sensor value at the start of each interval may be compared to the corresponding value at the same time in the historical profile. The difference between the values ​​at these points may be compared to determine whether the end of the context window is likely within one of the intervals. Once this is determined, the interval may be divided into smaller intervals, and the process may be repeated to identify similarities over these shorter periods. In some examples, the threshold difference may be smaller the second time through the process. This approach may be repeated a fixed number of times in succession until the difference between some aggregated values ​​is less than some difference threshold, or in some other suitable manner.

[0050] At block 314, the process 300 includes the user device 102 determining whether there are other sensors that can be used to determine the different context windows. If so, the process 300 returns to block 304 to access different historical sensor data profiles associated with the particular type of virtual test and different sensors that collect different sensor data during the clinical test. Once the different context windows are generated, the process 300 proceeds to block 316, where the process 300 includes the user device 102 determining an aggregate starting point using only the start point of the context window or the start point of the context window together with the start points of one or more different context windows. In some examples, determining the aggregate starting point may include defining the aggregate starting point as the same time as the start point determined at 312. In some examples, determining the aggregate starting point may include selecting the aggregate starting point from among a set of starting points that includes the start point of the context window and the start points of one or more different context windows. For example, such selection may be based on the sensor type used to define the context window and / or the test type. This may be possible because data output by certain sensors may be more reliable for determining the start point of a context window than others, and / or certain test types may have better defined start points (and end points) than other test types. In some examples, determining the aggregate start point may include averaging the time of the start point and other start points.

[0051] At block 318, the process 300 includes the user device 102 using the aggregate starting point to segment a portion of the sensor data collected by the user device 102. As described in FIG. 2, this may also be used to generate data packages and / or adjustment information.

[0052] FIG. 4 illustrates a diagram 400 including an example sensor 208(1) and segmented sensor data, according to at least one embodiment. Diagram 400 is an example of using a single sensor to segment that particular sensor's data. Diagram 400 includes a timeline 402 spanning t=0 to t=5 and a representation of sensor data 404 acquired by sensor 208(1) (e.g., one of sensors 116-120) during a period 406. The profile of sensor data 404 may vary at different times between t=0 and t=5. In some embodiments, the sensor's sampling rate may vary between t=0 and t=5. Diagram 400 also includes timestamps 408 and 410, corresponding to the user-input or machine-defined start and end points, respectively, of the virtual exercise test. For example, after being prompted by user device 102 to perform the virtual test, the user may provide a command to user device 102 at t=1 to begin the virtual exercise test, in the form of a user interface selection, a physical button, a voice command, etc. A timestamp 408 may be generated by the user device 206 and associated with the current sensor data 404. This may correspond to block 302 of FIG.

[0053] Between t=1 and t=2, the user may still be preparing for the test (e.g., getting ready, assuming a predetermined posture). Therefore, the data collected between t=1 and t=2 may not be very relevant to the actual results of the virtual exercise test. Therefore, blocks 304 through 312 may be performed to identify a context window 412 bounded by a window start point 414 and a window end point 416. As can be seen in the example sensor data 404, the window start point 414 and the window end point 416 may correspond to an inflection point in the data or other locations where a larger change in slope is observed. For example, between t=2 and t=3, the user may be doing their best to perform the virtual exercise test by, for example, sitting still and holding their hands on their lap. Therefore, during this time, the sensor 208(1) (e.g., an accelerometer) shows little movement. However, once the virtual exercise test ends, the sensor data 404 shows more variability (e.g., between t=3 and t=4). In some embodiments, the window end point 416 is not a determined value, but rather is matched to the end point 410, which may be user-defined by selecting to input on the user device that the test is finished, or the end point 410 may be auto-defined (e.g., the virtual test may run for a fixed period of time and may automatically end after the time has elapsed). The portion of the sensor data 404 within the context window 412 may be segmented from the other sensor data 404 and saved along with other information about the virtual exercise test (e.g., test type, sensor type, window start point, window end point), as described in block 318.

[0054] FIG. 5 illustrates a diagram 500 including exemplary sensors 208(1) and 208(2) of the same user device 102 and segmented sensor data, according to at least one embodiment. Diagram 500 is one example of using a first sensor 208(1) to segment that particular sensor's data and data obtained from a second sensor 208(2). Diagram 500 includes a timeline 502 and a representation of first sensor data 504 acquired by a first sensor 208(1) and sensor data 505 acquired by a second sensor 208(2), all during a period 506, spanning t=0 to t=5. The profiles of sensor data 504 and 505 may vary at different times between t=0 and t=5. In some embodiments, the sampling rates of sensors 208(1) and 208(2) may vary between t=0 and t=5. Diagram 500 also includes timestamps 508 and 510 corresponding to the user-inputted or machine-defined start and end points, respectively, of the virtual motor test. For example, after being prompted by user device 102 to perform the virtual motor test, the user may select a button on user device 102 at t=1 to begin the virtual motor test. Timestamp 508 may be generated by user device 102 and associated with sensor data 504 and 505 at that time. This may correspond to block 302 of FIG. 3. In FIG. 5, context window 512, bounded by window start point 514 and window end point 516, may be determined in a manner similar to that described with respect to context window 412 of FIG. 4. However, the difference in FIG. 5 lies in the segmentation step. In this example, sensor data 504 is used to generate context window 512, which is then used to segment sensor data 504 and sensor data 505 (acquired by sensor 208(2)). The portion of the sensor data 504 and 505 within the context window 512 may be segmented from the other sensor data 504 and 505 and saved along with other information about the virtual exercise test (e.g., test type, sensor type, window start point, window end point), as described in block 318.

[0055] FIG. 6 illustrates a diagram 600 including exemplary sensors 208(1) and 208(2) of the same user device 102 and segmented sensor data, according to at least one embodiment. Diagram 600 is an example in which a first sensor 208(1) is used to determine a first window start point 614 for a context window 612, and a second sensor 208(2) is used to determine a second window start point 615 for the context window 612. These different start points 614 and 615 can be used to generate an aggregate start point 618. A first window end point 616 and a second window end point 617 can also be used to generate an aggregate end point 620, as described herein. Diagram 600 includes a timeline 602 and a representation of first sensor data 604 acquired by a first sensor 208(1) and second sensor data 605 acquired by a second sensor 208(2), all during a period 606, spanning t=0 to t=9. The profiles of sensor data 604 and 605 may vary at different times from t=0 to t=9. In some examples, the sampling rate of sensors 208(1) and 208(2) may vary from t=0 to t=9. Diagram 600 also includes timestamps 608 and 610, which correspond to user-input or machine-defined start and end points, respectively, of the virtual movement test. For example, after being prompted by user device 102 to perform the virtual movement test, the user may select a button on user device 102 at t=1 to begin the virtual movement test. Timestamp 608 may be generated by user device 102 and associated with sensor data 604 and 605 at that time. This may correspond to block 302 of FIG. 3.

[0056] In FIG. 6 , first sensor data 604 acquired from first sensor 208(1) may be used to determine a first window start point 614 and a first window end point 616. Similarly, as described in block 314 of FIG. 3 , second sensor data 604 acquired from second sensor 208(2) may be used to determine a second start point 615 and a second end point 617. At this point, there are essentially two context windows, one for each of the two sensors 208(1) and 208(2). User device 102 may analyze the two context windows to identify a context window 612 bounded by an aggregate start point 618 and an aggregate end point 620. In this sense, the context window differs from context windows 412 and 512 because of the time shift. This may result in better overall signal data than simply using context window 412 or 512. FIG. 7 shows a detailed view of start region 622 for the calculation of aggregate start point 618 for context window 612. The segmentation of the sensor data 604 and 605 may be performed similarly to that described in the other figures.

[0057] FIG. 7 illustrates a diagram 700 showing a detailed view of start region 622 of diagram 600, according to at least one embodiment. In addition to showing the detailed view, diagram 700 illustrates an iterative process for refining the time associated with aggregate starting point 618, depicted as 618(1) at an old time and as 618(2) at a new time. Sensor data 604 and 605 are also illustrated in more detail to illustrate that the process of determining aggregate starting point 618 may include one or more iterative evaluations of sensor data 604 and 605. For example, as part of a first evaluation, aggregate starting point 618 was identified as 618(1). Process 300, including blocks 302-312, may be repeated for sensor data 604 and sensor data 605 over a time window spanning between t=2 and t=4 to again identify potential starting points. These new calculations can be used to generate offset values ​​for aggregate starting point 618(1) to arrive at aggregate starting point 618(2). In some embodiments, this may include choosing an intermediate time between the first start point 614 and the second start point 615. In some embodiments, determining the aggregate start point 618(2) may include defining the aggregate start point 618(2) as the same time as start point 614 or 615. In some embodiments, determining the aggregate start point 618(2) may include choosing the aggregate start point from among a set of start points including the start point of the context window 614 and the start points of one or more different context windows 612. For example, such a selection may be based on the sensor type used to define the context window and / or test type. This may be possible because data output by certain sensors may be more reliable for determining the start point of the context window than others and / or certain test types may have better defined start points (and end points) than other test types. In some embodiments, determining the aggregate start point may include averaging the time of the start point and the other start points.

[0058] FIG. 8 shows a diagram 800 including a first exemplary sensor 208(1) from user device 102 and a second sensor 208(3) from a different device, according to at least one embodiment. For example, second sensor 208(3) may be one of external sensors 130. Diagram 800 is an example of using first sensor 208(1) to generate a context window, time-aligning data for second sensor 208(3) with that of first sensor 208(1), and using the context window to segment the data from first sensor 208(1) and second sensor 208(3). Diagram 800 includes a timeline 802 and representations of first sensor data 804 acquired by first sensor 208(1) and second sensor data 805 acquired by second sensor 208(3), all during a period 806, spanning between t=0 and t=5. The profiles of sensor data 804 and 805 may vary at different times from t=0 to t=5. In some examples, the sampling rate of sensors 208(1) and 208(3) may vary from t=0 to t=5. Diagram 800 also includes timestamps 808 and 810, respectively, corresponding to user-input or machine-defined start and end points of the virtual exercise test. For example, after being prompted by user device 102 to perform the virtual exercise test, the user may select a button on user device 102 at t=1 to begin the virtual exercise test. Timestamp 808 may be generated by user device 102 and associated with sensor data 804 at that time. Because sensor data 805 is acquired by a different device, sensor data 805 may be streamed, shared, or otherwise transmitted to user device 102. Sensor data 804 and 805 may not be consistent because they were captured using devices and / or sensors with different internal clocks.

[0059] The process of generating the context window 812 may be performed as described elsewhere herein. Once the context window 812 is defined, the window start point 814, the window end point 816, the timestamps 808 and 810, and / or any other points of the first sensor data 804 may be compared to the second sensor data 805 based on the sensor type of the second sensor 208(3). This may reveal an offset 828 (e.g., an “X”) between the first sensor data 804 and the second sensor data 805. To account for the offset 828, the second sensor data 805 may be time-shifted until at least the start points of the identified windows match, as illustrated by the dashed version of the second sensor data 805. Once the context window 812 is defined, it may be used to segment the sensor data output by the first sensor 208(1), the second sensor 208(3), and any other sensors, as described elsewhere herein.

[0060] 9 shows an example flowchart illustrating a process 900 for implementing techniques for segmenting sensor data for a virtual athletic test, according to at least one embodiment. Process 900 is performed by user device 102 (FIG. 1). Process 900 specifically addresses various approaches to segmenting sensor data, according to various embodiments. In some embodiments, user device 102 is a wearable user device, such as a watch or other device described herein.

[0061] Process 900 begins at block 902 by the user device 102 accessing test information. The test information may identify (i) a first timing indicator (e.g., a first timestamp) associated with a first time, (ii) a second timing indicator (e.g., a second timestamp) associated with a second time, and (iii) a virtual motor test type of the virtual motor test. In some examples, the first and second timing indicators each include a data tag that includes a corresponding timestamp. The virtual motor test may include a series of tasks to assess the motor function of the wearer of the user device.

[0062] In some examples, process 900 may further include user device 102 generating test information as part of conducting the virtual motor test during the time period. In some examples, process 900 may further include user device 102 receiving a first user input indicating a start point of the virtual motor test, generating a first timing indicator in response to receiving the first user input and based on the first user input, receiving a second user input indicating an end point of the virtual motor test, and generating a second timing indicator in response to receiving the second user input and based on the second user input.

[0063] At block 904, the process 900 includes the user device 102 accessing signal data acquired during a period bounded by a first time and a second time. The user device 102 may acquire the signal data using one or more sensors. In some embodiments, the signal data may include signal data collected from multiple sensors on the user device.

[0064] At block 906, the process 900 includes the user device 102 determining a first signal data type for segmenting the signal data. This may be based on the virtual exercise test type identified in block 902. The signal data type (e.g., the first signal data type) may be defined by the sensor used. For example, an accelerometer may output accelerometer-type data. The first signal data of the first signal data type may be output by a first sensor of the wearable user device during the time period. The first sensor may include any one of the sensors described herein, such as a gyroscope, an accelerometer, a photoplethysmography (PPG) sensor, a heart rate sensor, etc.

[0065] At block 908, the process 900 includes the user device 102 determining a context window. The context window may be determined within a time period. In some examples, determining the context window may include selecting a historical signal profile of a first signal data type. The historical signal profile may be derived from a previous occurrence of the virtual exercise test. Determining the context window may also include comparing the first signal data to the historical signal profile to identify a third time corresponding to a start point of the context window and a fourth time corresponding to an end point of the context window. In some examples, the context window may include a start point and an end point. The start point of the context window may be associated with a third time that is later than the first time and earlier than the second time. The end point of the context window may be associated with a fourth time that is later than the third time and earlier than the second time. In some examples, comparing the first signal data to the historical signal profile may include accessing a set of evaluation rules associated with the virtual exercise test type and evaluating the first signal data according to the set of evaluation rules to identify the third time and the fourth time. In some examples, the set of evaluation rules may define signal characteristics that indicate a context window start point and a context window end point for a virtual motion test type.

[0066] In some examples, the start point and end point of the context window are a first start point and a first end point of the context window. In this example, process 900 may further include determining a second signal data type for segmenting the signal data by user device 102 and based on the virtual movement test type. The second signal data of the second signal data type may be output by a second sensor of the user device during the time period. In this example, block 908 may include accessing a different set of evaluation rules associated with the virtual movement test type and evaluating the second signal data according to the different set of evaluation rules to identify a second start point of the context window and a second end point of the context window. In this example, the set of evaluation rules may be associated with the first signal data type, and the different set of evaluation rules may be associated with the second signal data type.

[0067] In some embodiments, the first starting point may be different from the second starting point. In some embodiments, the process 900 may further include the user device 102 determining the actual starting point of the context window by performing one or more of: selecting the actual starting point based on an earlier occurrence of the first starting point or the second starting point; or selecting the actual starting point based on a comparison of a first signal difference measured between first signal data at the first starting point and a corresponding first time in the historical signal profile and a second signal difference measured between second signal data at the second starting point and a corresponding second time in the historical signal profile.

[0068] In some embodiments, the process 900 may further include the user device 102 determining a third timing indicator associated with the third time and a fourth timing indicator associated with the fourth time, and associating the third and fourth timing indicators with portions of the signal data.

[0069] In some examples, the user device 102 may include determining a second signal data type for segmenting the signal data based on the virtual exercise test type. The second signal data of the second signal data type may be output by a second sensor of the user device during the time period. In this example, determining 908 the context window may be further based at least in part on the second signal data.

[0070] At block 910, the process 900 includes the user device 102 segmenting a portion of the signal data received during the context window. In some embodiments, the portion of the signal data may include at least a portion of the first signal data. In some embodiments, the portion of the signal data may exclude the first signal data.

[0071] At block 912, the process 900 includes the user device 102 generating a virtual athletic test data package, which may be based on the portion of the signal data and the test information. In some examples, generating the virtual athletic test data package may include generating results of the virtual athletic test that include the portion of the signal data. In some examples, the process 900 may further include the user device 102 outputting a portion of the results by presenting the portion of the results on a display of the user device or transmitting the portion of the results to a remote computing device.

[0072] At block 914 , the process 900 includes the user device 102 transmitting the virtual athletic test data package to a remote server, such as the service provider 204 .

[0073] In some examples, process 900 further includes adjusting, by the user device 102, operation of the first sensor based on the context window during a subsequent virtual movement test of the first virtual movement test type. In some examples, the operation may include a sampling rate. In this example, adjusting the sampling rate based on the context window may include instructing the first sensor to capture data at a first sampling rate outside the context window and instructing the first sensor to capture data at a second sampling rate within the context window.

[0074] In some examples, the virtual movement test may be performed during a time period, in which example, associating the portion of the signal data with the virtual movement test may include tagging the portion of the signal data with a context window start point and a context window end point within the time period during which the virtual movement test is performed.

[0075] 10 shows an example flowchart illustrating a process 1000 for implementing techniques for segmenting sensor data for a virtual athletic test, according to at least one embodiment. Process 1000 is performed by user device 102 (FIG. 1). Process 1000 specifically addresses various approaches to segmenting sensor data, according to various embodiments. In some embodiments, user device 102 is a wearable user device, such as a wristwatch or other device described herein.

[0076] Process 1000 begins at block 1002 by the user device 102 receiving a first user input at an input device of the user device 102 identifying the start of a first time period during which the virtual exercise test will be conducted. The first input may be received at a graphical user interface, a physical button, or any other location.

[0077] At block 1004, the process 1000 includes receiving a second user input at an input device of the user device that identifies an end point of the first time period.

[0078] At block 1006, the process 1000 includes accessing, by the user device 102 and based on the virtual exercise test, first signal data output by a first sensor of the user device during a first time period.

[0079] At block 1006, the process 1000 includes determining, by the user device 102, a context window within a time period based on the first signal data and a virtual movement test type associated with the virtual movement test. The context window may define a second time period within the first time period. In some examples, determining the context window within a time period may include accessing a set of evaluation rules associated with the virtual movement test type and evaluating the first signal data according to the set of evaluation rules to identify a start point of the second time period and an end point of the second time period. The set of evaluation rules may define signal characteristics indicative of a start point of the second time period and an end point of the second time period for the virtual movement test type.

[0080] In some examples, determining the context window defining the second time period may further include accessing a different set of evaluation rules associated with the virtual exercise test type and evaluating a portion of the second signal data acquired during the first time period according to the different set of evaluation rules to identify a start point of the second time period and an end point of the second time period. In some examples, the set of evaluation rules may be associated with a first signal data type of the first signal data, and the different set of evaluation rules may be associated with a second signal data type of the second signal data.

[0081] At block 1008, process 1000 includes determining, by the user device 102, second signal data output by a second sensor of the user device during a second time period. In some examples, the first sensor and the second sensor may share a common characteristic (e.g., each may have the ability to track some aspect of movement). In some examples, the common characteristic may be an activity metric. In some examples, the first signal data is different from the second signal data.

[0082] At block 1010, the process 1000 includes associating, by the user device 102, the second signal data with the virtual movement test, which may include correlating and storing this data.

[0083] In some examples, the process 1000 may further include the user device 102 segmenting a portion of the first signal data output by the first sensor of the wearable user device during the second time period and associating the portion of the first signal data with the virtual exercise test.

[0084] 11 illustrates an exemplary architecture or environment 1100 configured to implement techniques for segmenting sensor data, according to at least one embodiment. For example, the architecture 1100 may enable data sharing between various entities of the architecture, at least some of which may be connected via one or more networks 1102, 1112. For example, the exemplary architecture 1100 may be configured to enable a user device 1106 (e.g., user device 102), a service provider 1104 (e.g., service provider 204, sometimes referred to herein as a remote server, a service provider computer, etc.), a health organization 1108, and any other sensors 1110 (e.g., sensors 116-120 and 130) to share information. In some embodiments, the service provider 1104, the user device 1106, the health organization 1108, and the sensors 1110(1)-1110(N) may be connected via one or more networks 1102 and / or 1112 (e.g., via Bluetooth, WiFi, the Internet, cellular, etc.). In architecture 1100, one or more users may use different user devices via one or more networks 1112 (or other networks) to manage, control, or otherwise utilize user device 1106. Additionally, in some embodiments, user device 1106, service provider 1104, and sensor 1110 may be configured or otherwise structured as a single device such that functionality described with respect to service provider 1104 can be performed by user device 1106, and vice versa.

[0085] In some embodiments, the networks 1102, 1112 may include any one or combination of many different types of networks, such as a cable network, the Internet, a wireless network, a cellular network, a satellite network, other private and / or public networks, or any combination thereof. While the illustrated embodiment depicts a user device 1106 accessing a service provider 1104 via the network 1102, the techniques described may equally apply if the user device 1106 interacts with the service provider 1104 via a landline telephone, a kiosk, or in any other manner. It is also noted that the techniques described may apply to other client / server configurations (e.g., set-top boxes) as well as non-client / server configurations (e.g., locally stored applications, peer-to-peer configurations).

[0086] As mentioned above, the user device 1106 may be configured to collect and / or manage user activity data potentially received from the sensor 1110. In some examples, the user device 1106 may be configured to provide the user's health, fitness, activity, and / or medical data to a third-party or first-party application (e.g., the service provider 1104). This data, in turn, may be used by the service provider 1104 to implement the techniques described herein.

[0087] The user device 1106 may be any type of computing device, such as, but not limited to, a mobile phone, a smartphone, a personal digital assistant (PDA), a wearable device (e.g., a ring, a watch, a necklace, a sticker, a belt, a shoe, a shoe accessory, a belt clip device), an implantable device, etc. In some examples, the user device 1106 may communicate with the service provider 1104, the sensor 1110, and / or a health authority via the networks 1102, 1112, or via other network connections.

[0088] The sensor 1110 may be a standalone sensor or may be integrated into one or more devices. In some embodiments, the sensor 1110 may be shared with the user device 1106 and collect sensor data relevant to implementing the techniques described herein. For example, the user device 1106 may be a primary user device 1106 (e.g., a smartphone), and the sensor 1110 may be a sensor device external to the user device 1106 that can share sensor data with the user device 1106. For example, the external sensor 1110 may share information with the user device 1106 over the network 1112 (e.g., via Bluetooth or other short-range wireless communication protocol). In some embodiments, the external sensor 1110 includes a network radio that enables it to communicate with the user device 1106 and / or the service provider 1104. The user device 1106 may include one or more applications for managing the remote sensor 1110. This may enable pairing with the sensor 1110, data reporting frequency, data processing of data from the sensor 1110, data reconciliation, etc.

[0089] The sensors 1110 may be attached to various parts of the human body (e.g., feet, legs, torso, arms, hands, neck, head, eyes) to collect various types of information, such as activity data, motion data, or heart rate data. The sensors 1110 may include accelerometers, respiration sensors, gyroscopes, PPG sensors, pulse oximeters, electrocardiogram (ECG) sensors, electromyography (EMG) sensors, electroencephalography (EEG) sensors, global positioning system (GPS) sensors, auditory sensors (e.g., microphones), ambient light sensors, barometric altimeters, electro-optical heart rate sensors, and any other suitable sensors designed to obtain patient physiological, health, and / or motion data.

[0090] In one exemplary configuration, the user device 1106 may include at least one memory 1114 and one or more processing units (or processors) 1116. The processor 1116 may be suitably implemented in hardware, computer-executable instructions, firmware, or a combination thereof. The computer-executable instructions or firmware implementation of the processor 1116 may include computer-executable or machine-executable instructions written in any suitable programming language to perform the various functions described. The user device 1106 may also include a geolocation device (e.g., a GPS device, etc.) for providing and / or recording geolocation information associated with the user device 1106. The user device 1106 also includes one or more sensors 1110(2), which may be of the same type as described with respect to the sensor 1110.

[0091] Depending on the configuration and type of user device 1106, memory 1114 may be volatile (such as random access memory (RAM)) and / or non-volatile (such as read-only memory (ROM), flash memory, etc.). Volatile memory as described herein may be referred to as RAM, but is suitable as any volatile memory that does not retain data stored therein once unplugged from a host and / or power source.

[0092] Both removable and non-removable memory 1114 are examples of non-transitory computer-readable storage media. For example, non-transitory computer-readable storage media may include volatile or non-volatile, removable or non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Memory 1114 is an example of a non-transitory computer-readable storage medium or non-transitory computer-readable storage device. Additional types of computer storage media that may be present in user device 1106 may include, but are not limited to, PRAM, SRAM, DRAM, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage, or any other medium that can be used to store desired information and that can be accessed by user device 1106. Combinations of any of the above should also be included within the scope of non-transitory computer-readable storage media. Alternatively, computer-readable communication media may include computer-readable instructions, program modules, or other data transmitted within a data signal such as a carrier wave or other transmission. However, as used herein, computer-readable storage media does not include computer-readable communication media.

[0093] Looking more specifically at the contents of memory 1114, memory 1114 may include an operating system 1120 and / or one or more application programs or services for implementing the features disclosed herein. User device 1106 also includes one or more machine learning models 1136, representing any suitable predictive models. The machine learning models 1136 may be utilized by user device 1106 to determine a context window, as described herein.

[0094] The service provider 1104 may also include memory 1124 that includes one or more application programs or services for implementing the features disclosed herein. In this manner, the techniques described herein may be implemented by any one or combination of computing devices (e.g., the user device 1106 and the service provider 1104).

[0095] The user device 1106 also includes a data store, including one or more databases or the like, for storing data such as sensor data 1126 and static data 1128. In some embodiments, the databases 1126 and 1128 may be accessed via a network service.

[0096] The service provider 1104 may also be any type of computing device, such as, but not limited to, a mobile phone, a smartphone, a PDA, a laptop computer, a desktop computer, a thin client device, a tablet computer, a wearable device, a server computer, or a virtual machine instance. In some examples, the service provider 1104 may be in communication with the user device 1106 and the health organization 1108 over the network 1102 or over other network connections.

[0097] In one exemplary configuration, service provider 1104 may include at least one memory 1130 and one or more processing units (or processors) 1132. Processor 1132 may be suitably implemented in hardware, computer-executable instructions, firmware, or a combination thereof. The computer-executable instructions or firmware implementation of processor 1132 may include computer-executable or machine-executable instructions written in any suitable programming language to perform the various functions described.

[0098] The memory 1130 may store program instructions that are loadable and executable on the processor 1132, as well as data generated during the execution of these programs. Depending on the configuration and type of the service provider 1104, the memory 1130 may be volatile (such as RAM) and / or non-volatile (such as ROM, flash memory, etc.). The volatile memory described herein may be referred to as RAM, but any volatile memory that does not retain data stored therein once unplugged from a host and / or power source is suitable. Both removable and non-removable memory 1130 are examples of non-transitory computer-readable storage media.

[0099] Looking more specifically at the contents of memory 1130, memory 1130 may include an operating system 1134 and / or one or more application programs or services for implementing the features disclosed herein.

[0100] Service provider 1104 also includes a data store, including one or more databases or the like, for storing data such as sensor data 1138 and static data 1140. In some embodiments, databases 1138 and 1140 may be accessed via a network service.

[0101] Turning now to health organization 1108, although depicted as a single entity, health organization 1108 may represent multiple health organizations. Health organization 1108 includes an EMR system 1148 that is accessed via dashboard 1146 (e.g., by a user using clinician user device 1142). In some examples, EMR system 1148 may include record storage 1144 and dashboard 1146. Record storage 1144 may be used to store health records of patients associated with health organization 1108. Dashboard 1146 may be used to read and write records in record storage 1144. In some examples, dashboard 1146 is used by a clinician to manage disease progression of a patient population, including the patient operating user device 102. Clinicians may operate clinician user device 1142 to interact with dashboard 1146 to view results of virtual exercise tests on an individual patient basis, a patient population basis, etc. In some examples, a clinician may use the dashboard 1146 to “push” tests to the user device 102 .

[0102] While the present subject matter has been described in detail with reference to specific embodiments thereof, it will be understood that those skilled in the art, upon achieving the foregoing understanding, may readily produce modifications, variations, and equivalents to such embodiments. Accordingly, the present disclosure is presented for purposes of illustration and not limitation, and does not preclude the inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to those skilled in the art. Indeed, the methods and systems described herein may be embodied in a variety of other forms, and further, various omissions, substitutions, and changes in the form of the methods and systems described herein may be made without departing from the spirit of the present disclosure. The appended claims and their equivalents are intended to cover such forms or modifications as are within the scope and spirit of the present disclosure.

[0103] Unless otherwise indicated, throughout the discussion herein, use of terms such as "processing," "computing," "calculating," "determining," and "identifying" will be understood to refer to the actions or processes of one or more computers or similar electronic computing devices that manipulate or transform data represented as physical electronic or magnetic quantities in the memory, registers, or other information storage, transmission, or display devices of the computing platform.

[0104] The systems or systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device can include any suitable arrangement of components that provides results conditioned on one or more inputs. Suitable computing devices range from general-purpose computing devices to special-purpose computing devices that implement one or more embodiments of the present subject matter, including general-purpose microprocessor-based computing systems that access stored software that programs or configures the computing system. Any suitable programming, scripting, or other type of language, or combination of languages, may be used to implement the teachings contained herein in software used to program or configure a computing device.

[0105] Embodiments of the methods disclosed herein may be implemented in operation of such a computing device. The order of the blocks presented in the above examples may be varied, e.g., the blocks may be rearranged, combined, and / or divided into sub-blocks. Certain blocks or processes may be performed in parallel.

[0106] Conditional language used herein, such as "can," "could," "might," "may," "for example," among others, is generally intended to convey that certain embodiments include certain features, elements, and / or steps, while other embodiments do not, unless otherwise stated or understood otherwise within the context of use. Thus, such conditional language is not generally intended to imply that features, elements, and / or steps are in any way required by one or more embodiments, or that one or more embodiments necessarily include logic for determining, with or without authorial input or prompting, whether these features, elements, and / or steps are included or should be implemented in any particular embodiment.

[0107] Disjunctive language, such as the phrase "at least one of X, Y, or Z," unless otherwise specified, is understood within the context to generally indicate that an item, value, etc. can be either X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not, and should not be, intended to imply that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0108] Use of the term "or" herein is intended to encompass an inclusive and exclusive OR condition. In other words, "A or B or C" includes any or all of the following alternative combinations, as appropriate for the particular use: A alone, B alone, C alone, A and B only, A and C only, B and C only, and all three of A, B, and C.

[0109] The use of terms such as "a," "an," and "the" and similar referents in the context of describing the disclosed embodiments (particularly in the context of the claims below) shall be construed to cover both the singular and the plural unless otherwise stated herein or clearly contradicted by context. Terms such as "comprising," "including," and "having" are synonymous and are used in an inclusive, open-ended manner and do not exclude additional elements, features, acts, operations, etc. Also, the term "or," when used to connect a list of elements, for example, is used in its inclusive (rather than exclusive) sense, such that "or" refers to one, some, or all of the elements in the list. The use of "adapted" or "configured" herein is intended as open, inclusive language that does not exclude devices adapted or configured to perform additional tasks or steps. The term "connected" should be construed as partially or wholly contained within, attached to, or joined together, even if there is something intervening. The recitation of ranges of values ​​herein is merely intended to serve as a shorthand method of referring individually to each separate value falling within that range, unless otherwise stated herein, and each separate value is incorporated herein as if it were individually recited herein. Additionally, the use of "based on" is meant to be open and inclusive in that a process, step, calculation, or other action "based on" one or more recited conditions or values ​​may, in fact, be based on additional conditions or values ​​beyond those recited. Similarly, the use of "based at least in part on" is meant to be open and inclusive in that a process, step, calculation, or other action "based at least in part on" one or more recited conditions or values ​​may, in fact, be based on additional conditions or values ​​beyond those recited. Headings, lists, and numbering contained herein are for ease of description only and are not intended to be limiting.

[0110] The various features and processes described above may be used independently of one another or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. In addition, in some implementations, certain method or process blocks may be omitted. The methods and processes described herein are also not limited to any particular order, and the associated blocks or states may be performed in other orders as appropriate. For example, the described blocks or states may be performed in an order other than that specifically disclosed, or multiple blocks or states may be combined into a single block or state. The example blocks or states may be performed sequentially, in parallel, or in some other manner. Blocks or states may be added to or removed from the embodiments of the present disclosure. Similarly, the example systems and components described herein may be configured differently from what is described. For example, elements may be added, removed, or rearranged compared to the disclosed embodiments.

[0111] All references cited herein, including publications, patent applications, and patents, are hereby incorporated by reference to the same extent as if each reference was individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

Claims

1. 1. A computer-implemented method comprising: accessing, by the wearable user device, (i) a first timing indicator associated with a first time, (ii) a second timing indicator associated with a second time, and (iii) test information identifying a virtual exercise test type of the virtual exercise test; accessing, by the wearable user device, signal data acquired by the wearable user device during a period bounded by the first time and the second time; determining, by the wearable user device and based on the virtual exercise test type, a first signal data type for segmenting the signal data, the first signal data of the first signal data type being output by a first sensor of the wearable user device during the time period; The wearable user device defines a context window within the period of time as at least: selecting a historical signal profile of the first signal data type, the historical signal profile being derived from a previous occurrence of the virtual movement test; and determining a fourth time corresponding to an end of the context window by comparing the first signal data to the historical signal profile to identify a third time corresponding to a start of the context window and a fourth time corresponding to an end of the context window; segmenting, by the wearable user device, a portion of the signal data received during the context window; generating, by the wearable user device, a virtual athletic test data package based on the portion of the signal data and the test information.

2. 2. The computer-implemented method of claim 1, further comprising adjusting, by the wearable user device, operation of the first sensor based on the context window during a subsequently performed virtual exercise test of the virtual exercise test type.

3. 3. The computer-implemented method of claim 2, wherein the operation includes a sampling rate, and wherein adjusting the sampling rate based on the context window includes instructing the first sensor to capture data at a first sampling rate outside the context window and instructing the first sensor to capture data at a second sampling rate within the context window.

4. comparing the first signal data to the historical signal profile; accessing a set of assessment rules associated with the virtual athletic test type; The computer-implemented method of claim 1 , further comprising: evaluating the first signal data according to the set of evaluation rules to identify the third time point and the fourth time.

5. The computer-implemented method of claim 4 , wherein the set of evaluation rules defines signal characteristics indicative of the start point of the context window and the end point of the context window for the virtual exercise test type.

6. the start point and the end point of the context window are a first start point and a first end point of the context window, and the method further includes determining, by the wearable user device and based on the virtual exercise test type, a second signal data type for segmenting the signal data, wherein second signal data of the second signal data type is output by a second sensor of the wearable user device during the time period, and determining the context window within the time period; accessing a different set of assessment rules associated with the virtual athletic test type; 5. The computer-implemented method of claim 4, further comprising evaluating the second signal data according to a different set of evaluation rules to identify a second start point of the context window and a second end point of the context window.

7. The computer-implemented method of claim 6 , wherein the set of evaluation rules is associated with the first signal data type and the different set of evaluation rules is associated with the second signal data type.

8. The first starting point is different from the second starting point, and the method further comprises: selecting the actual start point based on an earlier occurrence of the first start point or the second start point; and selecting the actual starting point based on a comparison of a first signal difference measured between the first signal data at the first starting point and a corresponding first time in the historical signal profile and a second signal difference measured between the second signal data at the second starting point and a corresponding second time in the historical signal profile.

9. 2. The computer-implemented method of claim 1, wherein the context window includes a start point and an end point, the start point of the context window being associated with a third time that is later than the first time and earlier than the second time, and the end point of the context window being associated with a fourth time that is later than the third time and earlier than the second time.

10. determining a third timing indicator associated with the third time and a fourth timing indicator associated with the fourth time; 10. The computer-implemented method of claim 9, further comprising associating the third and fourth timing indicators with the portion of the signal data.

11. The computer-implemented method of claim 1 , further comprising generating the test information as part of conducting the virtual exercise test during the time period.

12. receiving, at the wearable user device, a first user input indicating a starting point of the virtual exercise test; generating the first timing indicator in response to receiving the first user input and based on the first user input; receiving, at the wearable user device, a second user input indicating an endpoint of the virtual exercise test; 2. The computer-implemented method of claim 1, further comprising: generating the second timing indicator in response to receiving the second user input and based on the second user input.

13. The computer-implemented method of claim 1 , wherein the first and second timing indicators each include a data tag containing a corresponding timestamp.

14. The computer-implemented method of claim 1 , wherein the signal data comprises signal data collected from multiple sensors of the wearable user device.

15. 2. The computer-implemented method of claim 1, further comprising determining, by the wearable user device and based on the virtual exercise test type, a second signal data type for segmenting the signal data, wherein second signal data of the second signal data type is output by a second sensor of the wearable user device during the time period.

16. 16. The computer-implemented method of claim 15, wherein determining the context window is further based at least in part on the second signal data.

17. The computer-implemented method of claim 1 , wherein the portion of the signal data includes at least a portion of the first signal data.

18. The computer-implemented method of claim 1 , wherein the portion of the signal data excludes the first signal data.

19. The computer-implemented method of claim 1 , wherein the first sensor includes at least one of a gyroscope, an accelerometer, a photoplethysmography sensor, or a heart rate sensor.

20. The computer-implemented method of claim 1 , wherein the virtual motor test includes a series of tasks for assessing motor function of a wearer of the wearable user device.

21. generating results of the virtual exercise test that include the portion of the signal data; 10. The computer-implemented method of claim 1, further comprising: outputting a portion of the results, wherein outputting the portion of the results comprises at least one of presenting the portion of the results on a display of the wearable user device or transmitting the portion of the results to a remote computing device.

22. 2. The computer-implemented method of claim 1, wherein the virtual movement test is performed during the time period, and associating the portion of the signal data with the virtual movement test comprises tagging the portion of the signal data with a start point of the context window and an end point of the context window within the time period during which the virtual movement test is performed.

23. When executed by one or more processors of a wearable device, the wearable device is caused to: accessing, by the wearable user device, (i) a first timing indicator associated with a first time, (ii) a second timing indicator associated with a second time, and (iii) test information identifying a virtual exercise test type of the virtual exercise test; accessing, by the wearable user device, signal data acquired by the wearable user device during a period bounded by the first time and the second time; determining, by the wearable user device and based on the virtual exercise test type, a first signal data type for segmenting the signal data, the first signal data of the first signal data type being output by a first sensor of the wearable user device during the time period; The wearable user device defines a context window within the period of time as at least: selecting a historical signal profile of the first signal data type, the historical signal profile being derived from a previous occurrence of the virtual movement test; and determining a fourth time corresponding to an end of the context window by comparing the first signal data to the historical signal profile to identify a third time corresponding to a start of the context window and a fourth time corresponding to an end of the context window; segmenting, by the wearable user device, a portion of the signal data received during the context window; generating, by the wearable user device, a virtual athletic test data package based on the portion of the signal data and the test information.

24. a memory configured to store computer-executable instructions; one or more processors that access the memory and execute the computer-executable instructions to perform at least accessing test information identifying (i) a first timing indicator associated with a first time, (ii) a second timing indicator associated with a second time, and (iii) a virtual exercise test type of the virtual exercise test; accessing signal data acquired by the wearable user device during a time period bounded by the first time and the second time; determining a first signal data type for segmenting the signal data based on the virtual exercise test type, the first signal data of the first signal data type being output by a first sensor of the wearable user device during the time period; The context window within the period is at least selecting a historical signal profile of the first signal data type, the historical signal profile being derived from a previous occurrence of the virtual movement test; and comparing the first signal data to the historical signal profile to identify a third time corresponding to a start point of the context window and a fourth time corresponding to an end point of the context window. Segmenting, by the wearable user device, a portion of the signal data received during the context window; and and one or more processors configured to generate, by the wearable user device, a virtual athletic test data package based on the portion of the signal data and the test information.

25. 1. A computer-implemented method comprising: receiving, at an input device of the wearable user device, a first user input identifying a start point of a first time period during which the virtual exercise test will be performed; receiving a second user input at the input device of the wearable user device identifying an end point of the first time period; accessing, by the wearable user device and based on the virtual exercise test, first signal data output by a first sensor of the wearable user device during the first time period; determining, by the wearable user device, a context window within the first time period based on the first signal data and a virtual exercise test type associated with the virtual exercise test, the context window defining a second time period within the first time period; determining, by the wearable user device, second signal data output by a second sensor of the wearable user device during the second time period; and correlating, by the wearable user device, the second signal data with the virtual movement test.

26. 26. The computer-implemented method of claim 25, wherein the first sensor and the second sensor share a common characteristic.

27. 27. The computer-implemented method of claim 26, wherein the common characteristics include an activity metric.

28. 27. The computer-implemented method of claim 26, wherein the first signal data is different from the second signal data.

29. Segmenting a portion of the first signal data output by the first sensor of the wearable user device during the second time period; 26. The computer-implemented method of claim 25, further comprising associating the portion of the first signal data with the virtual movement test.

30. Determining the context window within the first time period comprises: accessing a set of assessment rules associated with the virtual athletic test type; and evaluating the first signal data according to the set of evaluation rules to identify a start point of the second time period and an end point of the second time period.

31. 31. The computer-implemented method of claim 30, wherein the set of evaluation rules defines, for the virtual exercise test type, signal characteristics indicative of the start point of the second time period and the end point of the second time period.

32. determining the context window defining the second period of time; accessing a different set of assessment rules associated with the virtual athletic test type; 31. The computer-implemented method of claim 30, further comprising: evaluating a portion of second signal data acquired during the first time period according to a different set of evaluation rules to identify the starting point of the second time period and the ending point of the second time period, wherein the set of evaluation rules is associated with a first signal data type of the first signal data and the different set of evaluation rules is associated with a second signal data type of the second signal data.

33. 26. The computer-implemented method of claim 25, wherein the context window corresponds to a time when a user performs the virtual motor test.

34. When executed by one or more processors of a wearable user device, the wearable user device is caused to: receiving, at an input device of the wearable user device, a first user input identifying a start point of a first time period during which a virtual exercise test will be conducted; receiving a second user input at the input device of the wearable user device identifying an end point of the first time period; accessing, by the wearable user device and based on the virtual exercise test, first signal data output by a first sensor of the wearable user device during the first time period; determining, by the wearable user device, a context window within the first time period based on the first signal data and a virtual exercise test type associated with the virtual exercise test, the context window defining a second time period within the first time period; determining, by the wearable user device, second signal data output by a second sensor of the wearable user device during the second time period; and associating, by the wearable user device, the second signal data with the virtual athletic test.

Images