System and method for maintaining data integrity in health analysis platform by assessing and modifying time-series outliers in filtered healthcare data

Abstract

To maintain data integrity in a health analysis platform.SOLUTION: A system and a method for (i) filtering existing healthcare data by determining or extracting a first subset of data sets so that the first subset is focused on a common health-related attribute and (ii) identifying an outlier data point in the first subset are disclosed. For instance, each of the first subset of data sets represents measurements of physiological parameter(s) of entities over time. After the first subset is determined on the basis of the common health-related attribute, (i) a respective rate of change of the measurements of the physiological parameter over time and (ii) whether the rate of change is greater than a threshold are determined. Thereafter, an outlier data point among the first subset is identified on the basis of the determination as to whether the rate of change is greater than the threshold. A data structure representing the outlier data point is generated and stored.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present specification generally relates to a system and method for maintaining data integrity in a health analytics platform by assessing and correcting time series outliers in filtered healthcare data. [Background technology]

[0002] Generally, a user's health can be assessed by measuring one or more physiological characteristics of the user and comparing the measured physiological characteristics to health criteria. For example, the health criteria can correspond to or be derived from existing healthcare data, including historical data, prior clinical trial data, real-time data, and other existing healthcare data. Thus, having accurately measured existing healthcare data can improve the quality of the assessment. Summary of the Invention

[0003] An embodiment according to the present disclosure includes a system for maintaining data integrity in a computational health analysis platform, the system including at least one processor and a memory subsystem communicatively coupled to the at least one processor. The memory subsystem stores instructions that, when executed by at least one processor, cause the at least one processor to perform operations including: (i) accessing one or more first data structures including a plurality of time series datasets relating to a plurality of entities and one or more health-related attributes of the plurality of entities, each of the time series datasets representing measurements of one or more physiological parameters of the plurality of entities over time; (ii) determining a first subset of the time series datasets based on the one or more health-related attributes; (iii) determining, for the first subset of the time series datasets, a rate of change of each of the measurements of the one or more physiological parameters over time and whether the rate of change is greater than a threshold; (iv) identifying one or more outlier data points from the first subset of the time series datasets based on the determination of whether the rate of change is greater than the threshold; (v) generating a second data structure representing the one or more outlier data points; and (vi) storing the second data structure in a hardware storage device.

[0004] An embodiment according to the present disclosure includes a method for (i) accessing, by an electronic device, one or more first data structures including a plurality of time series datasets relating to a plurality of entities and one or more health-related attributes of the plurality of entities, each of the time series datasets representing measurements of one or more physiological parameters of the plurality of entities over time; (ii) determining, by the electronic device, a first subset of the time series datasets based on the one or more health-related attributes of the plurality of entities; (iii) for the first subset of the time series datasets, determining, by the electronic device, a rate of change over time for each of the measurements of the one or more physiological parameters and whether the rate of change over time is greater than a threshold; (iv) identifying, by the electronic device, one or more outlier data points in the first subset of the time series datasets based on the determination of whether the rate of change over time is greater than a threshold; (v) generating, by the electronic device, a second data structure representing the one or more outlier data points; and (vi) storing, by the electronic device, the second data structure in a hardware storage device.

[0005] Other embodiments of this aspect include corresponding computer systems, devices, and computer programs stored on one or more computer storage devices, each configured to perform the operations or actions described herein. One or more computer systems can be configured to perform particular operations by having software, firmware, hardware, or other components that cause the system to perform or execute the operations during operation. Also, one or more computer programs can be configured to perform particular operations by including instructions that, when executed by a data processing device, cause the device to perform the operations.

[0006] The details of one or more embodiments of the subject matter herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram of an exemplary data integrity maintenance system.

[0008] [Figure 2] 1 is an example of software or algorithms utilized by an electronic device.

[0009] [Figure 3] FIG. 1 is a flowchart diagram of an exemplary process for identifying outlier data points in healthcare data.

[0010] [Figure 4] FIG. 1 is a flowchart diagram of an exemplary process for identifying and correcting outlier data points in healthcare data.

[0011] [Figure 5] FIG. 1 is a diagram of an exemplary computing system.

[0012] [Figure 6] 1 is a graph showing physiological parameter measurements versus time without outlier data points.

[0013] [Figure 7] 1 is a graph showing measurements of physiological parameters versus time with outlier data points;

[0014] Like reference numbers and designations in the various drawings indicate like elements. DETAILED DESCRIPTION OF THE INVENTION

[0015] Historical data, traditional clinical trial data, real-time data, and other existing healthcare data have many valid uses and can be used as reference data for health assessments, treatment diagnoses, treatment decisions, and medical research.

[0016] Thus, the accuracy of existing healthcare data is important in various aspects of healthcare, such as ensuring reliable health assessments, supporting effective treatment decisions, and facilitating advances in medical research. For example, such accurate healthcare data can be used as reference data for clinical trials to evaluate the efficacy and safety of treatments, for current health monitoring and diagnosis in assessing patient conditions, and for research purposes.

[0017] However, existing healthcare data can exhibit data inconsistencies or errors. For example, inconsistencies frequently occur in historical data, prior clinical trial data, real-time data, and other existing healthcare data due to a variety of reasons, including variations in methods, specimens, units of measurement, in vitro diagnostic devices, differences in data entry standards, human error, sensor inaccuracies, inconsistencies in data formats, and more.

[0018] Furthermore, if medical evaluations, treatment decisions, or medical research rely on existing medical data that exhibits data inconsistencies or errors, it may lead to misleading conclusions, reliability issues, inaccuracies, and / or reduced patient safety. Therefore, maintaining high-quality medical data can improve the quality, reliability, or accuracy of medical evaluations.

[0019] Implementations according to the present disclosure address the above data quality issues by at least [1] filtering existing healthcare data (e.g., historical data, prior clinical trial data, real-time data, and other existing healthcare data) by first determining or extracting a first subset of data from the existing healthcare data such that the first subset of data focuses on common health-related attributes, and [2] identifying outlier data points in the first subset of data.

[0020] For example, the data integrity maintenance system can filter such existing healthcare data by at least initially determining or extracting a first subset of time series data based on health-related attributes. For example, the first subset of time series data can be determined or extracted based on one or more health-related attributes, including disease indications, non-disease-indicating medical conditions, use of the same medication, the same medical treatment, or gender. The time series data represents measurements of one or more physiological parameters of an entity (e.g., an individual, patient, user, subject, etc.) over time. This determination or extraction of the first subset of time series data can be effective because it can adjust the range of physiological measurement data based on commonalities of health-related attributes that can be directly correlated with the physiological measurement data. Thus, by comparing physiological measurement data among first subsets of time series data that share health-related attributes, outlier data points or errors in the existing healthcare data can be more accurately and reliably detected.

[0021] For example, after a first subset of time-series data is determined or extracted based on health-related attributes, the data integrity maintenance system can identify outlier data points within the first subset of data. As an example, outlier data points within the first subset of time-series data are identified based on determining whether a rate of change in physiological parameter measurements (e.g., laboratory measurements) over time is greater than a threshold. In some embodiments, the threshold is determined based on (i) a mean value of the physiological parameter values ​​within the first subset over time and (ii) one or more standard deviations from the mean value. For example, the threshold may be configurable by a user.

[0022] For example, identified outlier data points can be determined to be errors (e.g., source errors, conversion errors) or valid extreme values, and / or such outlier data points can be corrected by correcting or deleting one or more of the physiological parameter measurements that correspond to the outlier data points. As an example, in clinical trial data, suspiciously large increases or decreases in the series of physiological parameter measurements for each patient can be flagged, but sudden increases or decreases in physiological parameter measurements for patients that are relatively consistent across the set of patients (within a first subset of the time series data) are not flagged because these values ​​are likely to be true measurements influenced by health-related attributes such as various treatments received during the clinical trial.

[0023] Thus, data quality can be improved based on (1) filtering existing healthcare data by initially determining or extracting a first subset of data from the existing healthcare data such that the first subset of data is tailored to or focused on common health-related attributes, (2) identifying outlier data points within the first subset of data, and / or (3) correcting the outlier data points. For example, data quality can be improved by preventing the processing of erroneous healthcare data.

[0024] Furthermore, improving the accuracy of existing healthcare data based on improved data quality can lead to reliable health assessments, effective treatment decisions, and the promotion of medical research. For example, such accurate healthcare data can be used as reference data for clinical trials to evaluate the effectiveness and safety of treatments, for current health monitoring and diagnosis to evaluate patient conditions, and for research purposes.

[0025] Furthermore, the embodiments described herein can also reduce the amount of computer resources consumed during computing of healthcare data. For example, when generating a health assessment, a computing system that encounters low-quality data may generate results with errors and / or inconsistencies that make it unusable. Thus, the computer system may reprocess the data multiple times (e.g., based on manual feedback from a user) until a satisfactory result is obtained. Such iterative operations may increase the amount of computing resources (e.g., CPU usage, computer memory resources, storage resources, etc.) consumed during the health assessment process. The embodiments described herein can be used to automatically identify and remove errors and / or inconsistencies in healthcare data, thereby [1] reducing the likelihood that healthcare data will be reprocessed due to low quality and [2] reducing the consumption of computer resources.

[0026] 1 illustrates an exemplary data integrity maintenance system 100 for filtering existing healthcare data, identifying outlier data points within the filtered healthcare data, and correcting the outlier data points. The data integrity maintenance system 100 may include an electronic device 120 and a sensor device 110 communicatively coupled to each other (e.g., via one or more wired or wireless communication links 150). Generally, the data integrity maintenance system 100 accesses a data structure (e.g., health-related data, such as existing healthcare data or lab test data, stored in a data store, such as a database module 122, or otherwise accessible to the electronic device 120, e.g., via a server) through a processing method according to an embodiment described herein and determines the data integrity (e.g., data quality) of the data structure. Furthermore, in some embodiments, the data integrity maintenance system 100 acquires sensor data about a user using the sensor device 110 and processes the sensor data using the electronic device 120 to determine one or more biomarkers indicative of the user's medical condition.

[0027] In general, electronic device 120 can include any number of devices configured to receive, process, and transmit data. Examples of electronic device 120 include client computing devices (e.g., desktop computers or notebook computers), server computing devices (e.g., server computers or cloud computing systems), mobile computing devices (e.g., cellular phones, smartphones, tablets, personal data assistants, notebook computers with network capabilities), wearable computing devices (e.g., smartphones or headsets), and other computing devices capable of receiving, processing, and transmitting data. In some implementations, electronic device 120 can include computing devices that operate using one or more operating systems (e.g., Microsoft Windows, Apple macOS, Linux, Unix, Google Android, and Apple iOS, etc.) and one or more architectures (e.g., x86, PowerPC, and ARM, etc.).

[0028] The sensor device 110 includes one or more sensors 112 configured to obtain measurements related to a user's physiology, user behavior, and / or other characteristics of the user. For example, the sensor device 110 can include or correspond to a wearable device (e.g., a smart watch), a smartphone, a medical monitoring system, laboratory equipment, etc. By way of example, the sensor device can include one or more sensors 112 configured to obtain physiological parameters including vital signs such as glucose levels, heart rate, blood pressure, respiratory rate, body temperature, etc. For example, the one or more sensors can be optical sensors (e.g., PPG), pulse pressure sensors (PP), pressure sensors, electrocardiograms (ECG), bioimpedance sensors, galvanic skin response sensors, intraocular pressure / contact sensors, accelerometers, gyroscopes, pressure sensors, acoustic sensors, electromechanical motion sensors, and / or electromagnetic sensors. Furthermore, for example, if the sensor device takes the form of laboratory equipment, it can also measure physiological parameters or perform blood tests such as analyzing blood glucose levels, cholesterol, and other biomarkers.

[0029] Additionally, the sensor device 110 includes a communication module 116 configured to transmit and / or receive data from the electronic device 120. By way of example, the communication module 116 may include one or more receivers, transmitters, and / or transceivers. In some implementations, the communication module 116 may communicate with the electronic device 120 via one or more wireless links (e.g., serial links, Ethernet links, etc.) and / or radio links (e.g., Wi-Fi links, Bluetooth links, etc.).

[0030] Generally, electronic device 120 is configured to receive sensor data (e.g., physiological parameter data such as clinical parameters) acquired by sensor device 110 and process the sensor data to determine one or more biomarkers indicative of a medical condition of the user. Further, electronic device 120 is configured to present information regarding the biomarkers and any other information to the user and / or other users (e.g., healthcare providers).

[0031] 1, electronic device 120 is depicted as a single component. However, in practice, electronic device 120 may be implemented in one or more computing devices (e.g., each computing device includes at least one processor, such as a microprocessor or microcontroller). As an example, electronic device 120 may be a single computing device, such as a single smartphone. As another example, electronic device 120 may include multiple computing devices connected via a network (e.g., the Internet, a local area network, etc.), and components of electronic device 120 may be maintained and operated on some or all of the computing devices. For example, electronic device 120 may include multiple computing devices, and components of electronic device 120 may be distributed across one or more of these computing devices.

[0032] Additionally, electronic device 120 is illustrated as a separate component from sensor device 110. However, electronic device 120 can be a separate component from sensor device 110, but electronic device 120 can also include, be coupled to, or be adjacent to (e.g., within a housing for) sensor device 110. For example, electronic device 120 can be a wearable device that includes, is coupled to, or is adjacent to sensor device 110.

[0033] 1, electronic device 120 includes database module 122, communication module 124, processing module 126, and user interface module 128. The operational modules may be provided as one or more computer-executable software modules, hardware modules, or the like. For example, one or more operational modules may be implemented as blocks of software code having instructions that cause one or more processors to perform the operations described herein. Additionally or alternatively, one or more of the operational modules may be implemented in an electronic circuit, such as, for example, a programmable logic circuit, a field programmable logic array (FPGA), or an application-specific integrated circuit (ASIC).

[0034] Communications module 124 is configured to transmit and / or receive data from sensor device 110. By way of example, communications module 124 may include one or more receivers, transmitters, and / or transceivers. In some embodiments, communications module 124 may communicate with sensor device 110 (e.g., via communications module 116) over one or more wired links (e.g., serial links, Ethernet links, etc.) and / or wireless links (e.g., Wi-Fi links, Bluetooth links, etc.).

[0035] The database module 122 maintains information related to the operation of the data integrity maintenance system 100 .

[0036] As an example, the database module 122 can store input data 122a that is used as input for determining one or more biomarkers indicative of a user's health. For example, the input data 122a can include at least a portion of the sensor data generated by the sensor device 110.

[0037] As another example, the database module 122 can store output data 122b generated by the electronic device 120. As an example, the output data 122b can include one or more metrics or biomarkers generated by the electronic device 120 based on the input data 122a.

[0038] Additionally, the database module 122 may store processing rules 122c that specify how the data in the database module 122 may be processed to perform the operations described herein.

[0039] As an example, the processing rules 122c may include one or more rules that specify how the input data 122a is formatted, parsed, and processed to determine one or more corresponding metrics or biomarkers for the user.

[0040] As another example, processing rules 122c may include one or more rules that specify the conditions under which data is presented to a user (eg, using user interface module 128) and how the data is presented.

[0041] As another example, processing rules 122c may include one or more rules that specify how data is stored for future retrieval and / or processing (eg, using database module 122).

[0042] Exemplary data processing techniques are described in further detail below.

[0043] The processing module 126 processes data stored or otherwise accessible on the electronic device 120. For example, the processing module 126 may be used to perform one or more of the operations described herein (e.g., by executing processing rules 122c on the input data 112a to generate output data 122b).

[0044] User interface module 128 is configured to present information to a user and / or receive input from a user. As an example, user interface module 128 may include one or more display devices (e.g., display screen, touch screen, etc.) configured to present a user interface (e.g., a graphical user interface, GUI) that allows a user to interact with electronic device 120 and / or sensor device 110. Exemplary interactions include viewing data, transmitting data from one component to another, and / or issuing commands to electronic device 120 and / or sensor device 110. Commands may include, for example, a user instructing one or more of electronic device 120 and / or sensor device 110 to perform a particular action or task. In some implementations, user interface module may also present information to a user audibly (e.g., using one or more speakers) and / or via haptic feedback (e.g., using one or more haptic generators, such as via generation).

[0045] In some implementations, software applications may be used to facilitate performance of the tasks described herein. As one example, the applications may be installed on electronic device 120. Furthermore, a user may interact with the applications to input data and / or commands to electronic device 120 and to review data generated by electronic device 120.

[0046] 2 is an exemplary implementation 200 of software or algorithms utilized by a processor-based electronic device (e.g., electronic device 120 of data integrity maintenance system 100 of FIG. 1 , a computing device (which may also be a server) of system 500 of FIG. 5 ). In particular, the software or algorithms are utilized by the electronic device to: [1] filter existing healthcare data by initially determining or extracting a first subset of data from the existing healthcare data such that the first subset of data is tuned to or focused on common health-related attributes; [2] identify outlier data points within the first subset of data; and / or [3] correct the outlier data points.

[0047] The exemplary implementation 200 illustrates a data store 210 and outlier detection software 220 .

[0048] The data store 210 may include or correspond to a data store of an electronic device (which may also be a server). For example, the data store 210 may be the database module 122 of the electronic device 120 and one or more storage devices 530 of a computing system (which may also be a server) of the system 500. The data store 210 may be in data communication with the electronic device (which may also be a server).

[0049] The data store 210 may include one or more first data structures 212. The first data structure 212 may include or correspond to existing healthcare data (e.g., historical data, prior clinical trial data, real-time data, and other existing healthcare data). For example, the first data structure 212 may include a time-series data set regarding an entity (e.g., an individual, a user, a patient, a subject, or the like). Each of the time-series data sets represents measurements of one or more physiological parameters 213 of the entity and one or more health-related attributes 214 of the entity. For example, the one or more physiological parameters may represent clinical parameters collected continuously at regular intervals. For example, the one or more physiological parameters may represent one or more vital signs, such as glucose level, heart rate, blood pressure, respiratory rate, body temperature, etc. For example, the one or more health-related attributes may include at least one of a disease indication, a medical condition other than a disease indication, the same medication use, the same medical treatment, gender, etc.

[0050] Furthermore, at least a portion of the data filtering or outlier detection can be implemented as respective software programs that can be executed on an electronic device. The software programs can be stored in a memory (e.g., database module 122 in FIG. 1 , memory 520, storage device 530 in FIG. 5 ), and can include machine-readable instructions that, when executed by a processor, cause the processor-based electronic device to perform the instructions of the software program. As shown, outlier detection software 220 can include data filtering tool 230 and / or outlier detection tool 240. In some implementations, outlier detection software 220 can include one or more tools. In some implementations, some of the tools can be combined, some of the tools can be divided into more tools, or a combination thereof. In some implementations, outlier detection software 220 can execute on a server (e.g., a computing device (which can also be a server) of system 500), or on both a server and an electronic device (e.g., it is assumed in this example that the electronic device does not take the form of a server).

[0051] In some embodiments, data filtering tool 230 may take the form of software separate from outlier detection software 220 and execute on a server, while outlier detection tool 240 may take the form of outlier detection software 220 and execute on an electronic device in data communication with the server.

[0052] The data filtering tool 230 can filter such existing healthcare data by at least initially determining or extracting a first subset of time series data based on one or more health-related attributes 214. For example, by way of illustration, entities corresponding to the first subset of time series data can correspond to a population group that takes or consumes the same medication or receives or has undergone the same medical procedure. For example, multiple entities in the first subset of time series data can correspond to a population group that exhibits the same disease indication.

[0053] The filtering process and examples can be seen in conjunction with the exemplary processes 300 and 400 of FIGS.

[0054] The outlier detection tool 240 may determine one or more outlier data points in a first subset of the time series data extracted from the first data structure 212. For example, for the first subset of the time series data set, the rate of change of each of the physiological parameter values ​​(e.g., measurements) over time may first be determined. For example, if the physiological parameter is a blood glucose level and the health-related attribute 214 is a particular medication taken by the patient (corresponding to the first subset of the time series data), the rate of change of each of the blood glucose levels over time would be determined.

[0055] After the rate of change over time of each of the physiological parameter values ​​(of the first subset of time series data) is determined, the outlier detection tool 240 can determine whether the respective rate of change over time is greater than a threshold. For example, the threshold can be determined based on (i) an average of the one or more physiological parameter values ​​(of the multiple entities of the first subset) over time and (ii) one or more standard deviations from the average. For example, by way of illustration, if the respective rate of change corresponds to a rate of change over time of a patient's blood glucose levels, the rate of change for multiple patients (corresponding to the first subset of time series data) can be determined, and each patient's rate of change can be compared to a threshold based on the average and standard deviation of the rate of change for the multiple patients.

[0056] After comparing each rate of change to a threshold, the outlier detection tool 240 may determine one or more outlier data points. For example, if each rate of change is greater than a threshold, the data point (e.g., the data point representing the respective physiological parameter value) corresponding to the respective rate of change may be determined as an outlier data point.

[0057] A more detailed process is described in the exemplary processes 300 and 400 of FIGS.

[0058] Example Process 3 is a flowchart diagram of an example process 300 for determining one or more outlier data points in healthcare data. In particular, example process 300 filters existing healthcare data by: [1] initially determining or extracting a first subset of data from the existing healthcare data such that the first subset of data focuses on common health-related attributes; and [2] identifying outlier data points within the first subset of data. Process 300 can be implemented by processor-based systems such as data integrity maintenance system 100 and system 500, and in combination with example implementation 200, as described in this disclosure.

[0059] At 302, one or more first data structures including time-series datasets are accessed. For example, a processor-based electronic device (e.g., electronic device 120) can be used to access the one or more first data structures. For example, the one or more first data structures can correspond to existing healthcare data (e.g., historical data, prior clinical trial data, real-time data, other existing healthcare data, etc.) stored in a data store (e.g., data store 210). For example, the data store can correspond to a memory of electronic device 120 or a storage device of a computing device (e.g., one or more storage devices 530 of the computing device of FIG. 5 including a server). The one or more first data structures include multiple time-series datasets related to multiple entities (e.g., individuals, users, patients, subjects, etc.) and one or more health-related attributes of the multiple entities. The time-series datasets represent measurements of one or more physiological parameters of the multiple entities over time. For example, the one or more physiological parameters represent one or more vital signs, such as glucose level, heart rate, blood pressure, respiratory rate, body temperature, etc.

[0060] At 304, a first subset of the time series data set is determined based on one or more health-related attributes. For example, the electronic device can extract or determine the first subset of time series data based on the one or more health-related attributes. For example, the one or more health-related attributes can include at least one of a disease indication, a medical condition other than a disease indication, use of the same medication, the same medical treatment, gender, etc. For example, the first subset of the time series data set can represent measurements of one or more physiological parameters of multiple entities that share the same health-related attribute. For example, the one or more physiological parameters of the first subset represent outcome variables measuring the effect of the same medication or the same medical treatment. For example, by way of illustration, the multiple entities corresponding to the first subset of time series data can correspond to a population taking or receiving the same medication or undergoing or having undergone the same medical treatment.

[0061] At 306, a rate of change over time for each of the physiological parameter values ​​(e.g., measurements) is determined for the first subset of the time series data set. For example, if the physiological parameter is blood glucose level and the health-related attribute is a particular medication that the patient (corresponding to the first subset of time series data) is taking or has taken, the rate of change over time for each of the blood glucose levels is determined. For example, for illustrative purposes, if the relationship between the physiological parameter value versus time is plotted in a graph, as shown by way of example in FIGS. 6 and 7, the rate of change would correspond to a change in slope of the data points. Thus, for example, a change in slope of the data points (as a rate of change) can be determined for each of the patients in the first subset of time series data.

[0062] After the respective rates of change of the physiological parameter values ​​(of the first subset of time-series data) over time are determined, the electronic device can determine whether the respective rates of change are greater than a threshold. For example, the threshold can be determined based on (i) an average of the one or more physiological parameter values ​​(of the multiple entities of the first subset) over time and (ii) one or more standard deviations from the average. For example, the threshold can correspond to a value representing three standard deviations from the average. For example, the threshold can be preset or configurable by a user. For example, the threshold can be changed or controlled based on user input to the electronic device.

[0063] For example, by way of illustration, the rates of change for multiple patients (corresponding to a first subset of the time series data) can be determined, where each rate of change corresponds to a patient's rate of change in blood glucose levels over time, and each patient's rate of change can be compared to a threshold based on the mean and standard deviation of the rates of change for the multiple patients.

[0064] At 308, one or more outlier data points are determined. For example, as described above, the electronic device may first compare each rate of change to a threshold value to determine whether each rate of change is greater than the threshold value. If each rate of change is greater than the threshold value, one or more data points (e.g., data points representing each physiological parameter measurement) corresponding to the respective rate of change may be determined as outlier data points.

[0065] For example, the rates of change of consecutive pairs (e.g., consecutive data pairs for the same time period) of physiological parameter values ​​(for the multiple entities corresponding to the first subset of time series data) can be compared to each other over a period of time, e.g., the difference between consecutive pairs of physiological parameter values ​​for each entity can be compared to a mean difference representing the average of the differences between consecutive paired values ​​for all entities.

[0066] For example, the time period can be preset or configurable by a user. For example, within the time period, a threshold can be determined based on (i) the mean difference between consecutive pairs of physiological parameter values ​​and (ii) one or more standard deviations from the mean difference. For example, the threshold can correspond to a value representing three standard deviations from the mean difference. For example, the threshold can be preset or configurable by a user. For example, the threshold can be set based on input by a user into the electronic device.

[0067] At 310, a second data structure representing the one or more outlier data points is generated. For example, the electronic device may format the data into a standardized data format, such as JavaScript Object Notation (JSON) format or Extensible Markup Language (XML) format. The standardized data format may be any other suitable data format suitable for storage in a data store or hardware storage device.

[0068] The second data structure is stored in a hardware storage device at 312. For example, the data structure may be stored in a data store (e.g., data store 210, memory of electronic device 120, one or more storage devices 530, etc.).

[0069] In some implementations, instead of or in addition to storing the data structure on a hardware storage device, the data structure may be output to a user interface (e.g., using user interface module 128). For example, if the data structure is to be output for user display without being stored, the data structure may be generated in another format suitable for display in step 310. For example, if the data structure is to be stored and then output for user display, the data structure may be converted from the stored format to the appropriate display format and then output.

[0070] In some embodiments, the second data structure can be provided or transmitted to a computational health analysis platform.

[0071] 4 is a flowchart diagram of an example process 400 for [1] identifying outlier data points in a first subset of data and [2] correcting the outlier data points. Process 400 can be implemented by a processor-based system, such as data integrity maintenance system 100 and system 500, in conjunction with example implementation 200 and example process 300 described in this disclosure.

[0072] At 402, the rate of change of each of the physiological parameter values ​​over time is determined for a first subset of the time series data set (from exemplary process 300). The technique used in step 402 may be the same as the technique used in step 306 of FIG. 3, so the technique will not be repeated here.

[0073] One or more outlier data points are determined at 404. The technique used in step 402 may be the same as the technique used in step 308 of Figure 3, so the technique will not be repeated here.

[0074] A data structure representing the one or more outlier data points is generated at 406. For example, the electronic device can format the data into a standardized data format suitable for display or can output the data for display in a user interface.

[0075] In some embodiments, the electronic device may [1] first format the data into a standardized data format suitable for storage on a data store or hardware storage device, and store the data structure on the hardware storage device as described in step 312 of FIG. 3 , and [2] convert the data structure from the storage format to an appropriate display format before output.

[0076] At 408, the one or more outlier data points are output (e.g., using the user interface module 128) for display on a user interface. For example, a graph representing a first subset of the time series data can be generated for display on a user interface. For example, a graph representing a relationship depicting physiological parameter values ​​versus time can be generated, as shown in the exemplary graphs of FIGS. 6 and 7. FIG. 6 shows a graph 600 illustrating a relationship between physiological parameter measurements (on the y-axis) and time (on the x-axis) without any outlier data points. The data points shown in FIG. 6 are considered to be within the normal range of physiological parameter value changes over time for an individual subject. Meanwhile, FIG. 7 shows a graph 700 illustrating a relationship between physiological parameter measurements (on the y-axis) and time (on the x-axis) with an outlier data point (shown as the peak data point at location 702). For example, the one or more outlier data points can be output for display in one or more colors.

[0077] In some embodiments, outlier data points may be reviewed and flagged as records or data points requiring review or verification.

[0078] At 410, the first subset of the time series data is modified. For example, the electronic device can modify the first subset of the time series data set based on a user instruction. For example, modifying the first subset of the time series data set can include correcting or deleting one or more measurements. For example, modifying the first subset of the time series data set can include designating one or more outlier data points as unusable data or data requiring modification. For example, modifying the first subset of the time series data can improve data integrity by (i) removing or isolating one or more outlier data points, or (ii) designating one or more outlier data points as unusable data or data requiring modification.

[0079] In some embodiments, the data structure including the modified first subset of the time-series data can be provided or transmitted to a computational health analysis platform.

[0080] Exemplary Computing System 5 illustrates an exemplary computing system according to an embodiment of the present disclosure. System 500 can be used for any of the operations described with respect to the various implementations discussed herein. System 500 can be included in, used by, communicate with, or correspond to electronic device 120. Furthermore, system 500 can be included in, used by, or communicate with sensor device 110. System 500 can include one or more processors 510, memory 520, one or more storage devices 530, and one or more input / output (I / O) devices 560 controllable via one or more I / O interfaces 540. The various components 510, 520, 530, 540, or 560 can be interconnected via at least one system bus 550, which can enable the transfer of data between the various modules and components of system 500.

[0081] The processor 510 may be configured to process instructions for execution within the system 500. The processor 510 may include a single-threaded processor, a multi-threaded processor, or both. The processor 510 may be configured to process instructions stored in the memory 520 or the storage device 530. The processor 510 may include hardware-based processors, each including one or more cores. The processor 510 may include a general-purpose processor, a special-purpose processor, or both.

[0082] Memory 520 may store information within system 500. In some implementations, memory 520 includes one or more computer-readable media. Memory 520 may include any number of volatile memory units, any number of non-volatile memory units, or both volatile and non-volatile memory units. Memory 520 may include read-only memory, random access memory, or both. In some examples, memory 520 may be employed as active or physical memory by one or more executing software modules.

[0083] Storage device 530 may be configured to provide (e.g., persistent) mass storage for system 500. In some implementations, storage device 530 may include one or more computer-readable media. For example, storage device 530 may include a floppy disk device, a hard disk device, an optical disk device, or a tape device. Storage device 530 may include read-only memory, random access memory, or both. Storage device 530 may include one or more of an internal hard disk drive, an external hard disk drive, or a removable drive.

[0084] Either or both of memory 520 or storage device 530 may include one or more computer-readable storage media (CRSM). The CRSM may include one or more of electronic storage media, magnetic storage media, optical storage media, magneto-optical storage media, quantum storage media, mechanical computer storage media, etc. The CRSM may provide storage of computer-readable instructions describing data structures, processes, applications, programs, other modules, or other data for operation of system 500. In some embodiments, the CRSM may include a data store that provides storage of computer-readable instructions or other information in a non-transitory format. The CRSM may be incorporated into system 500 or may be external to system 500. The CRSM may include read-only memory, random-access memory, or both. One or more CRSMs suitable for embodying computer program instructions and data may include any type of non-volatile memory, including, but not limited to, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. In some examples, processor 510 and memory 520 may be supplemented by, or incorporated in, one or more application-specific integrated circuits (ASICs).

[0085] System 500 may include one or more I / O devices 560. I / O devices 560 may include one or more input devices, such as a keyboard, mouse, pen, game controller, touch input device, audio input device (e.g., microphone), gesture input device, haptic input device, image or video imaging device (e.g., camera), or other device. In some examples, I / O devices 560 may also include one or more output devices, such as a display, LEDs, audio output device (e.g., speaker), printer, haptic output device, etc. I / O devices 560 may be physically incorporated into one or more computing devices of system 500 or may be external with respect to one or more computing devices of system 500.

[0086] The system 500 may include one or more I / O interfaces 540 to enable components or modules of the system 500 to control, interface with, or otherwise communicate with I / O devices 560. The I / O interfaces 540 may enable the transfer of information in, out of, or between components of the system 500 through serial, parallel, or other types of communication. For example, the I / O interfaces 540 may conform to a version of the RS-232 standard for serial ports or a version of the IEEE 1284 standard for parallel ports. As another example, the I / O interfaces 540 may be configured to provide connections via Universal Serial Bus (USB) or Ethernet. In some examples, the I / O interfaces 540 may be configured to provide serial connections conforming to a version of the IEEE 1394 standard.

[0087] I / O interface 540 may also include one or more network interfaces that enable communication between computing devices within system 500 or between system 500 and other networked computing systems. A network interface may include one or more network interface controllers (NICs) or other types of transceiver devices configured to send and receive communications over one or more networks using any network protocol.

[0088] The computing devices of system 500 can communicate with each other or with other computing devices using one or more networks. Such networks can include public networks such as the Internet, private networks such as institutional or personal intranets, or any combination of private and public networks. The networks can include any type of wired or wireless network, including, but not limited to, local area networks (LANs), wide area networks (WANs), wireless WANs (WWANs), wireless LANs (WLANs), mobile communication networks (e.g., 3G, 4G, Edge, etc.), etc. In some implementations, communications between computing devices can be encrypted or otherwise secured. For example, communications can employ one or more public encryption keys, ciphers, digital certificates, or other authentication information supported by security protocols such as the Secure Sockets Layer (SSL) or any version of the Transport Layer Security (TLS) protocol.

[0089] System 500 may include any number of computing devices of any type. Computing devices may include, but are not limited to, personal computers, smartphones, tablet computers, wearable computers, implant computers, mobile gaming devices, e-readers, automotive computers, desktop computers, laptop computers, notebook computers, gaming consoles, home entertainment devices, network devices, server computers, mainframe computers, distributed computing devices (e.g., cloud computing devices), microcomputers, systems-on-chips (SoCs), systems-in-packages (SiPs), etc. Examples herein may describe computing devices as physical devices, although implementations are not limited thereto. In some examples, a computing device may include one or more of a virtual computing environment, hypervisor, emulation, or virtual machine running on one or more physical computing devices. In some examples, two or more computing devices may comprise a cluster, cloud, farm, or other grouping of multiple devices that cooperate to provide load balancing, failover support, parallel processing capabilities, shared storage resources, shared network capabilities, or other aspects.

[0090] The term "configured" is used herein in connection with systems and computer program components. One or more computer systems configured to perform a particular operation or actions means that the system has software, firmware, hardware, or components installed in these computing systems that, when in operation, cause the system to perform the operation or actions. One or more computer programs configured to perform a particular operation or actions means that they contain instructions that, when executed by a data processing device, cause the device to perform the operation or actions.

[0091] Embodiments and functional operations of the subject matter described herein can be implemented in digital electronic circuitry, in tactilely embodied computer software or firmware, in computer hardware including the structures disclosed herein and their structural equivalents, or in one or more combinations thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by or to control the operation of a data processing apparatus. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random access memory or serial access memory device, or one or more combinations thereof. Alternatively, or in addition, the program instructions can be encoded in an artificially generated propagated signal, e.g., a mechanically generated electrical, optical, or electromagnetic signal, generated to encode information for transmission to a suitable receiver for execution by the data processing apparatus.

[0092] The term "data processing apparatus" means data processing hardware and includes any type of apparatus, device, and machine for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. An apparatus may also be or include special-purpose logic circuitry, such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). An apparatus may optionally include, in addition to hardware, code that creates an execution environment for a computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more of these computing systems.

[0093] A computer program may also be referred to or written as a program, software, software application, app, module, software module, script, or code, and may be written in any form of programming language, including compiled or interpreted, or declarative or procedural, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but need not, correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, a single file dedicated to the program in question, or multiple cooperating files, e.g., files that store one or more modules, subprograms, or portions of code. A computer program may be deployed to run on one computer, on multiple computers located at one site, or on multiple computers distributed across multiple sites and interconnected by a data communications network.

[0094] As used herein, the term "database" is used broadly to refer to any collection of data: the data need not be structured in any particular way, or even at all, and can be stored on storage devices in one or more locations. Thus, for example, an index database can contain multiple data collections, each of which can be organized and accessed in a different way.

[0095] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by special purpose logic circuitry, e.g., FPGAs or ASICs, or a combination of special purpose logic circuitry and one or more programmable computers.

[0096] A computer suitable for executing a computer program can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Generally, the central processing unit receives instructions and data from a read-only memory or a random-access memory, or both. The essential elements of a computer are a central processing unit for executing instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by, or incorporated in, special-purpose logic circuitry. Generally, a computer also includes one or more mass storage devices for storing data, e.g., magnetic disks, magneto-optical disks, optical disks, or both, operatively coupled to receive data from and / or transfer data to them. However, a computer need not have such devices. Furthermore, a computer may be incorporated in other devices, e.g., a mobile phone, a personal digital assistant (PDA), a portable audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage mechanism, e.g., a universal serial bus (USB) flash drive.

[0097] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0098] The term "memory subsystem" may include one or more memories, each of which may be a computer-readable medium. A memory subsystem may include a memory hardware unit (e.g., a hard drive or disk) that stores data or instructions in software form. Alternatively or in addition, a memory subsystem may include data or instructions that are hardwired to a processing circuit.

[0099] To provide for user interaction, embodiments of the subject matter described herein can be implemented on a computer having a display device, such as a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, or a mouse or trackball, through which the user can provide input to the computer. Other types of devices can also be used to provide for user interaction. For example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, including acoustic input, voice input, or tactile input. Furthermore, a computer can interact with a user by sending documents to and receiving documents from a device used by the user, for example, by sending a web page to a web browser on the user's device in response to a request received from the web browser. A computer can also interact with a user by sending text messages or other forms of messages to a personal device, such as a smartphone running a messaging application, and receiving a response message from the user.

[0100] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., a data server, or a middleware component, e.g., an application server, or a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with embodiments of the subject matter described herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), e.g., the Internet.

[0101] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers, each having a client-server relationship to the other. In some embodiments, a server sends data, such as an HTML page, to a user device for the purpose of displaying the data to and receiving user input from a user interacting with the user device, which functions as a client. Data generated at the user device, e.g., a result of a user interaction, can be received from the device by the server.

[0102] Although many specific embodiment details are described herein, these should not be construed as limitations on the scope of the invention or the claims, but rather as descriptions of features that may be unique to particular embodiments of a particular invention. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, while features may be described above as acting in particular combinations, and even initially claimed as such, one or more features from a claimed combination may, in some cases, be removed from the combination, and the claimed combination may be directed to subcombinations or variations of the subcombinations.

[0103] Similarly, although operations are depicted in the figures and claimed in a particular order, this should not be understood as requiring such operations to be performed in the particular order shown, or sequentially, or that all of the operations shown be performed, to achieve desirable results. In certain situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged in multiple software products.

[0104] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results. As an example, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous.

Claims

1. 1. A system for maintaining data integrity in a computational health analysis platform, comprising: at least one processor; a memory subsystem communicatively coupled to the at least one processor; Equipped with the memory subsystem stores instructions that, when executed by the at least one processor, cause the at least one processor to perform operations; The operation is accessing one or more first data structures, each of the first data structures comprising a plurality of time-series data sets relating to a plurality of entities, the time-series data sets comprising measurements of one or more physiological parameters of the plurality of entities over time and one or more health-related attributes of the plurality of entities; determining a first subset of the time series dataset based on one or more health-related attributes of the plurality of entities; determining a rate of change over time for each of the measurements of the one or more physiological parameters for a first subset of the time series data set, and determining whether the rate of change is greater than a threshold; identifying one or more outlier data points within the first subset of the time series data set based on a determination of whether the rate of change is greater than the threshold; and generating a second data structure representing the one or more outlier data points; storing the second data structure on a hardware storage device; Including, the system.

2. the operations include providing the second data structure to a computational health analysis platform; The system of claim 1 .

3. the one or more health-related attributes include at least one of a disease indication, a medical condition other than a disease indication, the same medication use, the same medical treatment, or gender; The system of claim 1 .

4. the one or more physiological parameters represent clinical parameters collected continuously at regular intervals; The system of claim 1 .

5. the one or more physiological parameters represent one or more vital signs; The system of claim 1 .

6. the one or more vital signs include at least one of glucose level, heart rate, blood pressure, respiratory rate, or body temperature; The system of claim 5.

7. a plurality of entities of the first subset corresponding to a population group exhibiting the same disease indication; The system of claim 1 .

8. the first subset of entities corresponds to a population group that is or has taken at least the same drug or is or has taken the same medical treatment, and the one or more physiological parameters of the first subset represent outcome variables measuring the effect of the same drug or the same medical treatment; The system of claim 1 .

9. the threshold is determined based on (i) an average value of measurements of one or more physiological parameters of the plurality of entities of the first subset over time and (ii) one or more standard deviations from the average value; The system of claim 1 .

10. the threshold corresponds to a value representing three standard deviations from the mean value; The system of claim 9.

11. identifying the one or more outlier data points includes comparing rates of change over time of successive pairs of measurements of one or more physiological parameters of the first subset of entities; The system of claim 1 .

12. the threshold is determined based on (i) the mean difference of the consecutive pairs and (ii) one or more standard deviations from the mean difference; The system of claim 11.

13. the threshold corresponds to a value representing three standard deviations from the mean difference; The system of claim 12.

14. the operations further include outputting the one or more outlier data points for display on a user interface. The system of claim 1 .

15. the operations further include modifying the threshold value based on user input to the user interface. The system of claim 14.

16. the operations further include generating a graph representing the first subset of the time series data for display on the user interface. The system of claim 14.

17. the one or more outlier data points are output for display in one or more colors.

17. The system of claim 16.

18. the operations further include modifying the time series data set based on a user instruction. The system of claim 1 .

19. Modifying the time series data set includes correcting or deleting one or more of the measurements.

20. The system of claim 18.

20. accessing the one or more first data structures includes accessing data collected from one or more wearable sensors; The system of claim 1 .

21. the operations further include modifying the first subset of the time series data based on the second data structure. The system of claim 1 .

22. modifying the first subset of time series data to improve data integrity by (i) removing or isolating the one or more outlier data points, or (ii) designating the one or more outlier data points as unusable data or data in need of modification; 22. The system of claim 21.

23. 1. A method comprising: accessing, by an electronic device, one or more first data structures including a plurality of time series data sets relating to a plurality of entities, each of the time series data sets representing measurements of one or more physiological parameters of the plurality of entities over time and one or more health-related attributes of the plurality of entities; determining, by the electronic device, a first subset of the time series data set based on one or more health-related attributes of the plurality of entities; determining, by the electronic device, a rate of change over time for each of the physiological parameter measurements for a first subset of the time series data set, and determining, by the electronic device, whether the rate of change is greater than a threshold; identifying, by the electronic device, one or more outlier data points from within the first subset of the time series data set based on determining whether the rate of change is greater than the threshold; generating, by the electronic device, a second data structure representing the one or more outlier data points; storing, by the electronic device, the second data structure on a hardware storage device; A method comprising:

24. providing, by the electronic device, the second data structure to a computational health analysis platform.

24. The method of claim 23.

25. the one or more health-related attributes include at least one of a disease indication, a medical condition other than a disease indication, the same medication use, the same medical treatment, or gender; 24. The method of claim 23.

26. the one or more physiological parameters represent one or more vital signs; 24. The method of claim 23.

27. the first subset of entities corresponds to a population group that is or has been taking the same drug or is or has been taking the same medical treatment, and the one or more physiological parameters of the first subset represent outcome variables measuring the effect of the same drug or medical treatment; 24. The method of claim 23.

28. the threshold value is determined based on (i) an average value of measurements of the one or more physiological parameters of a plurality of entities of the first subset over time, and (ii) one or more standard deviations from the average value; 24. The method of claim 23.

29. identifying the one or more outlier data points includes comparing rates of change over time of successive pairs of measurements of one or more physiological parameters of the first subset of entities; 24. The method of claim 23.

30. the threshold is determined based on (i) the mean difference of the consecutive pairs and (ii) one or more standard deviations from the mean difference; 30. The method of claim 29.

31. outputting, by the electronic device, the one or more outlier data points for display on a user interface.

24. The method of claim 23.

32. further comprising modifying the threshold value based on user input to the user interface.

32. The method of claim 31 .

33. generating, by the electronic device, a graph representing the first subset of the time series data for display on the user interface.

32. The method of claim 31 .

34. the one or more outlier data points are generated on the display in one or more colors; 32. The method of claim 31 .

35. and further comprising modifying, by the electronic device, the time series data set based on a user instruction.

24. The method of claim 23.

36. Modifying the time series data set includes correcting or deleting one or more of the measurements.

36. The method of claim 35.

37. One or more non-transitory computer readable media storing instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any of claims 23 to 36.

Images