Collaborative framework for improved regulatory decision making for measurement solutions

Abstract

In one aspect, a method of managing at least elements of a standardized digital solution for characterizing disease is presented. The method includes obtaining, at a processor, a central data structure. The method includes receiving a request for a new standardized digital solution. The method includes generating the new standardized digital solution by utilizing one or more elements of a previously approved standardized digital solution in the central data structure. The method includes receiving feedback from the supervisory collaborator, the feedback dedicated to the received input contributing to at least the elements of the new standardized digital solution. The method includes recording feedback received from the supervisory collaborator, wherein the received feedback includes a supervisory acceptance of at least the element of the new standardized digital solution.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of and priority to European Provisional Patent Application No. EP22382561.3, filed on June 10, 2022, and entitled “Collaborative Frameworks for Improving Regulatory Decision-Making of Measurement Solutions”, the entire disclosure of which is hereby incorporated by reference into this document in its entirety for all purposes. Technical Field

[0003] The presently disclosed subject matter generally relates to collaborative frameworks. In particular, the presently disclosed subject matter relates to a collaborative framework for improving regulatory decision making for measurement solutions. Background Art

[0004] Digital health technologies show high potential for real-world evidence data generation. Over the past decade, the number of clinical trials involving digital health technologies has shown a compound annual growth rate of 34.1%. However, multiple limitations have hampered the use of digital health technologies to date. Frequently cited limitations include: (1) lack of standardized processes, (2) concerns about how to select the most appropriate digital metrics, (3) how to collect, analyze, and interpret the captured real-world evidence, (4) difficulties in maintaining the integrity of solutions given changing technologies, (5) how to prepare supporting materials for regulatory submissions, and (6) lack of means to translate ideas into practical applications in clinical research and clinical care. Summary of the invention

[0005] Disclosed herein is a collaborative framework for achieving improved regulatory decision making (e.g., regulatory acceptance) for coordinated, digitally derived measures. For example, the methods disclosed herein can be used to achieve improved regulatory decision making for standardized digital solutions and / or individual elements of standardized digital solutions. In various examples of the present disclosure, such standardized digital solutions are valuable for characterizing diseases. Therefore, standardized digital solutions represent unified standards that can be provided to all parties (including regulators), so that all parties can characterize diseases in a standardized manner. Example standardized digital solutions include target solution profiles (TSPs) and digital measurement solutions (DMSs), as further described in detail herein. Additional standardized digital solutions include resources and / or elements of TSPs and DMSs. Generally, the methods disclosed herein relate to providing a collaborative platform that brings together various stakeholders to create standardized digital solutions.

[0006] On the one hand, a method for managing at least an element of a standardized digital solution for characterizing a disease is proposed. The method includes obtaining a central data structure at a processor. The central data structure includes compiled evidence of multiple elements of previously approved standardized digital solutions. The method includes receiving a user request for a new standardized digital solution at the processor. The method includes generating a new standardized digital solution at the processor by utilizing one or more elements of the previously approved standardized digital solution in the central data structure. The method includes receiving additional data for the new standardized digital solution from a user at the processor. The method includes updating the central data structure at the processor to incorporate the additional data. The method includes receiving feedback from a regulatory collaborator, the feedback being dedicated to the received input that contributes to at least an element of the new standardized digital solution. The method includes recording received feedback from a regulatory collaborator, wherein the received feedback includes regulatory acceptance of at least the element of the new standardized digital solution.

[0007] In another aspect, a device for managing at least elements of a standardized digital solution for characterizing a disease is presented. The device includes a processor and a memory communicatively coupled to the processor. The memory contains instructions for configuring the processor to obtain a central data structure. The central data structure includes compiled evidence of multiple elements of previously approved standardized digital solutions. The processor is configured to receive a user request for a new standardized digital solution. The processor is configured to generate a new standardized digital solution by utilizing one or more elements of the previously approved standardized digital solution in the central data structure. The processor is configured to receive additional data for the new standardized digital solution from a user. The processor is configured to update the central data structure to incorporate the additional data.

[0008] In one aspect, a method for digitally facilitating regulatory decision making for at least elements of a standardized digital solution for characterizing a disease is presented, the method comprising: providing a digital environment including multiple elements of a standardized digital solution, wherein the digital environment is accessible to multiple collaborators; obtaining a central data structure at a processor, wherein the central data structure includes compiled evidence of multiple elements of one or more previously approved standardized digital solutions; receiving a request for a new standardized digital solution at the processor; generating a new standardized digital solution at the processor by utilizing one or more of the multiple elements of the one or more previously approved standardized digital solutions in the central data structure; receiving additional data for the new standardized digital solution from a collaborator among the multiple collaborators at the processor; updating the central data structure at the processor to incorporate the additional data; receiving feedback from a regulatory collaborator at the processor, wherein the received feedback includes regulatory acceptance of at least elements of the new standardized digital solution; and digitally recording the received feedback from the regulatory collaborator in association with the central data structure.

[0009] In one aspect, one or more previously approved standardized digital solutions include measurement data collected via a medical device. In various examples of the present disclosure, the processor is further configured to generate multiple new standardized digital solutions for different patient populations. In various examples of the present disclosure, elements of one or more previously approved standardized digital solutions are included in one or more of a validation layer, a measurement instrument layer, or a definition layer of a central data structure.

[0010] In one aspect, generating a new standardized digital solution includes analyzing evidence data for one or more previously approved digital solutions and generating the new standardized digital solution based at least on an acceptability classification of the evidence data. In various examples of the present disclosure, the acceptability classification is determined based on one or more of the following: freshness of the evidence data, quantity of the evidence data, patient population of the evidence data, regulatory acceptance of the evidence data, clinical validation of the evidence data, or technical verification of the evidence data, content validity of the evidence data, construct validity of the evidence data, sensitivity to changes in the evidence data, or a combination thereof.

[0011] In one aspect, generating a new standardized digital solution is based on a medical condition or a diagnostic method. In various examples of the present disclosure, the new standardized digital solution includes a diagnostic-specific algorithm. In various examples of the present disclosure, the elements of the new standardized digital solution include any of the following: a measurable concept of an element of interest; a measurement method element; a raw data element; an algorithm element; a data set element; a technical validation element; an analytical validation element; a clinical validation element; or a regulatory element.

[0012] On the one hand, the method also includes generating a task within a digital environment by establishing standardized roles for two or more collaborators among a plurality of collaborators before receiving a collaborator request, wherein each of the two or more collaborators is an independent stakeholder in the task. In various examples of the present disclosure, establishing roles for the two or more collaborators includes assigning a primary role to a first collaborator among the two or more collaborators and assigning a secondary role to a second collaborator among the two or more collaborators, and wherein the primary role has additional permissions compared to the secondary role, wherein the permissions include the permission to access, view and / or use the results developed from the task. In various examples of the present disclosure, the collaborators among the two or more collaborators are assigned to any of the following roles: funder, observer, technology provider, or data partner. In various examples of the present disclosure, the method also includes receiving an agreement on one or more standards for the task from each of the two or more collaborators before receiving input from one or more of the two or more collaborators to define at least the element of the standardized digital solution, wherein the one or more standards for the task include the terms and conditions of the agreement.

[0013] In one aspect, the method further includes, prior to receiving an agreement from each of the two or more collaborators, generating a predefined template identifying one or more criteria for the task; and providing the predefined template to each of the two or more collaborators.

[0014] In one aspect, the new standardized digital solution is a digital measurement solution, and wherein the digital measurement solution includes a measurement definition that defines one or more concepts of interest related to a disease; a measurement instrument resource that is configured to transform measurements of interest captured according to the measurement definition into a data set that provides information for characterizing the disease, wherein the measurement instrument resource of the digital measurement solution is specific to the device used to capture the measurements of interest. In various examples of the present disclosure, the new standardized digital solution is a target solution profile that represents a common class of digital measurement solutions; and the target solution profile includes: a measurement definition that defines one or more concepts of interest related to a disease; a measurement instrument resource that is configured to transform data captured according to the measurement definition into a data set, the measurement instrument resource being device technology agnostic and therefore interchangeable between different target solution profiles.

[0015] On one hand, the digital environment is a collaborative platform built on the measurement stack model.

[0016] In one aspect, the received input includes one or more of the following types of information: technical feasibility information, patient perspective information, and clinical information.

[0017] In one aspect, the method further includes: generating an agreement based on one or more of: common interests, regulator collaborators, regulatory decisions, and established roles of two or more collaborators, wherein the agreement includes terms that include one or more of: task rules and collaborator permissions; providing the agreement to each of the two or more collaborators via a digital environment; and providing a framework for editing the terms for one or more collaborators via the digital environment.

[0018] In one aspect, the method further includes generating a dossier for use in seeking regulatory decision making, the dossier associated with at least an element of the new standardized measurement solution.

[0019] These and other aspects and features of non-limiting examples of the presently disclosed and claimed subject matter will become apparent to those skilled in the art upon reviewing the following description of specific non-limiting examples of the disclosed subject matter in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] These and other features, aspects and advantages of the present invention will become better understood from the following description and accompanying drawings.It should be noted that wherever practicable, similar or identical reference numerals may be used in the drawings and may indicate similar or identical functionality.

[0021] Figure 1 is a block diagram of a digital solution system according to an example of the present disclosure.

[0022] Figure 2A is an example measurement stack according to examples of the present disclosure.

[0023] Figure 2B is an example measurement stack showing various elements according to an example of the present disclosure.

[0024] Figure 2C is an example measurement stack of one or more elements forming a Target Solution Profile (TSP), a Target Instrumentation Profile (TIP), or a Target Component Profile (TCP) in accordance with the instructions of examples of the present disclosure.

[0025] Figure 2D An example target solution configuration file according to an example of the present disclosure is shown.

[0026] Figure 3 are exemplary apparatus for administering at least elements of a standardized digital solution for characterizing a disease.

[0027] Figure 4 Exemplary acceptability criteria determinations for various elements of a layer are shown.

[0028] Figure 5A An exemplary targeted solution profile for atopic dermatitis is depicted.

[0029] Figure 5B Depicted is an exemplary digital measurement solution for atopic dermatitis.

[0030] Figure 5C Depicted is an example of resource interchangeability of different digital measurement solutions for atopic dermatitis.

[0031] Fig. 6A An example target solution profile for pulmonary hypertension is depicted.

[0032] Figure 6B A first example digital measurement solution for pulmonary hypertension is depicted.

[0033] Figure 6C A second example digital measurement solution for pulmonary hypertension is depicted.

[0034] Fig.6D Depicts an example of repurposing measurement instrumentation resources for a digital measurement solution.

[0035] Figure 7 An example of a dynamic digital document is depicted.

[0036] Fig. 8A A high-level overview of an exemplary collaborative effort to develop a standardized solution is depicted.

[0037] Figure 8B An example process involving various parties for implementing a dynamic regulatory assessment of a standardized solution is depicted.

[0038] Figure 8C An example process involving various parties for implementing a dynamic regulatory assessment of a standardized solution is depicted.

[0039] Fig. 9 Depicted is an example flow chart for reusing and extending a digital measurement solution.

[0040] Fig.10 Depicted is an example of a decision tree.

[0041] Fig.11 Describes the user interface.

[0042] Fig.12 A spreadsheet showing cost savings is depicted.

[0043] Fig.13 Depicted is a computing system that can be implemented in any of the embodiments as described throughout this disclosure.

[0044] Fig.14 A first example of the acceptability of a digital measurement solution layer is depicted.

[0045] Fig.15 A second example of the acceptability of a digital measurement solution layer is depicted.

[0046] Fig.16 A third example of the acceptability of a digital measurement solution layer is depicted.

[0047] Fig.17 A fourth example of the acceptability of a digital measurement solution layer is depicted. DETAILED DESCRIPTION

[0048] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a comprehensive understanding of the subject matter disclosed in the present invention. However, it will be apparent that the subject matter disclosed in the present invention can be practiced without these specific details. As used herein, the words "exemplary" or "illustrative" mean "used as an example, instance, or illustration". Any specific implementation described herein as "exemplary" or "illustrative" is not necessarily to be understood as being preferred or advantageous over other specific implementations. All specific implementations described below are exemplary specific implementations provided to enable those skilled in the art to make or use examples of the present disclosure, and are not intended to limit the scope of the present disclosure as defined by the claims.

[0049] The terms "subject" or "patient" are used interchangeably and encompass cells, tissues, organisms, human or non-human, mammalian or non-mammal, male or female, whether in vivo, ex vivo, or in vitro.

[0050] The terms "disease" or "disorder" are used interchangeably and generally refer to a diseased state in a subject. Generally, standardized solutions, such as digital measurement solutions, are implemented to characterize a disease in a subject.

[0051] The phrase "measurement stack" refers to the organization of one or more resources consisting of elements. In a specific example of the present disclosure, the measurement stack consists of two or more resources. In a specific example of the present disclosure, the measurement stack consists of three or more resources. For example, the measurement stack includes measurement definition resources, measurement instrument resources, and evidence resources. The measurement stack provides the structure of a standardized solution (such as a target solution profile or a digital measurement solution).

[0052] The phrase "target solution profile" or "TSP" refers to a measurement stack in which generic descriptions are combined to provide profiles that are device technology agnostic (e.g., profiles that are independent of a specific hardware device and / or independent of a specific software). In various instances of the present disclosure, a target solution profile includes each of a measurement definition resource, a measurement instrument resource, and an evidence resource. In various instances of the present disclosure, the measurement instrument resource of a target solution profile describes a general method for capturing and transforming raw data of interest, but does not specify specific devices and algorithms for capturing and transforming raw data. A target solution profile represents a class of common digital measurement solutions. A target solution profile may specify performance requirements and / or standards such that the common class of digital measurement solutions represented by the target solution profile is evaluated and validated to function within the performance requirements and / or standards.

[0053] The phrase "digital measurement solution" or "DMS" refers to a specific digital solution built on the measurement stack. In various instances of the present disclosure, the DMS specifies all elements of a complete solution, which may include equipment, algorithms, external data, definitions, and / or evidence. For example, a digital measurement solution identifies a specific device or software used to capture raw data. In various instances, a digital measurement solution identifies a specific algorithm used to transform raw data into meaningful health data. Thus, implementing a digital measurement solution can be used to characterize a subject's disease.

[0054] The phrase "standardized solution" refers to a standard digital solution that can be used to characterize a disease. Examples of standardized solutions include digital measurement solutions and target solution profiles.

[0055] Any term not directly defined herein should be understood to have a meaning commonly associated with them, as understood in the field of the present disclosure. Certain terms are discussed herein to provide additional guidance to physicians when describing the composition, equipment, method, etc. of various aspects of the present disclosure and how to prepare or use them. It should be understood that the same thing can be described in more than one way. Therefore, alternative language and synonyms can be used for any one or more terms in the terms discussed herein. It is not important whether the term is elaborated or discussed in this article. Some synonyms or alternative methods, materials, etc. are provided. Unless explicitly stated, the narration of one or more synonyms or equivalents does not exclude the use of other synonyms or equivalents. The use of examples (including examples of terms) is for illustrative purposes only, and does not limit the scope and meaning of various aspects disclosed herein.

[0056] It must be noted that, as used in this specification, the singular forms "a," "an," "the," and "said" include plural referents unless the context clearly dictates otherwise.

[0057] All references, issued patents, and patent applications cited in the text of this specification are hereby incorporated by reference in their entirety for all purposes.

[0058] Digital Solution System

[0059] See now Figure 1 , a digital solution system 130 is presented. The digital solution system includes a resource module 140. The resource module 140 generates or obtains individual elements and then constructs a resource composed of two or more elements. The resource module 140 can store elements and / or resources in an element repository 170. Examples of elements include: 1) health aspects elements related to diseases, 2) hypothesis elements, 3) concept of interest elements that define measurable units of health aspects that inform diseases, 4) measurement method elements that define how to capture raw data, 5) raw data elements that specify raw data characteristics, 6) algorithm elements for implementing algorithms that transform raw data, 7) health data elements that describe meaningful interpretations of disease-related data, 8) analytical validation elements, 9) clinical validation elements, and 10) regulatory intelligence elements. Further descriptions of these example elements are included herein.

[0060] In various instances, the resource module 140 can organize a single element into a resource consisting of two or more elements. As an example, the resource module 140 can organize A) health aspects elements related to the disease, B) hypothesis elements, and C) concept elements of interest into resources, hereinafter referred to as measurement definition resources. As another example, the resource module 140 can organize A) measurement method elements that define how to capture raw data, B) raw data elements that specify raw data characteristics, C) algorithm elements for implementing algorithms that transform raw data, and D) health data elements that describe meaningful interpretations of disease-related data as resources into resources, hereinafter referred to as measuring instrument resources. As yet another example, the resource module 140 can organize A) analysis and verification elements, B) clinical verification elements, and C) regulatory intelligence elements into resources, hereinafter referred to as evidence resources. Further details of these example resources are described in this article.

[0061] In various examples, the resource module 140 generates elements and constructs resources by a de novo approach. For example, the resource module 140 identifies a specific disease, generates elements, and constructs resources that can be used to characterize a specific disease. In various examples, the resource module 140 can receive elements and / or resources from a third-party entity. In such examples, the third-party entity can be a resource developer who creates its own elements and / or resources and provides its own elements and / or resources to the digital solution system 130. Therefore, the resource module 140 can organize the elements received from the third-party entity into resources. In various examples, the resource module 140 can organize the fusion of the elements generated from scratch and the elements received from the third-party entity into resources.

[0062] The target solution profile module 145 generates a target solution profile (TSP) using elements and / or resources, which are, for example, elements and / or resources generated from scratch by the resource module 140, or elements and / or resources obtained by the resource module 140 from a third-party entity. In various instances, the TSP includes measurement definition resources, measurement instrument resources, and / or evidence resources. In a specific instance, the TSP includes each of the measurement definition resources, measurement instrument resources, and evidence resources. Generally speaking, the TSP represents such a measurement stack: wherein a general description is combined to provide a profile that is agnostic to device technology (e.g., a profile that is independent of a specific hardware device and independent of specific software). The general description is valuable for ensuring that the resources of the TSP can be easily interchanged. For example, the measurement instrument resources of the TSP can specify a class of devices for capturing measurement results. Examples of a class of devices include, but are not limited to, wearable devices (e.g., wrist-worn devices), wearable devices, image- and voice-based devices, contactless sensors, and sensor-based smart devices (e.g., scales, thermometers, and respirators).

[0063] In various examples, the TSP module 145 uses a method (e.g., bottom-up method) to build a TSP using a focused disease method. Here, the TSP is built by first identifying a disease or disease of interest. Therefore, for the purpose of characterizing a disease of interest, the elements of the TSP are assembled. In various examples, the TSP module 145 uses a method (e.g., top-down method) to build a TSP using a focused measurement instrument method. Here, the TSP is built by identifying available elements and resources (e.g., elements and resources stored in the element repository 170). This ensures that previously generated and / or verified elements and resources can be easily reused. Therefore, in various examples, building a TSP can involve reusing elements and resources from other TSPs, so that new elements and resources do not need to be generated. In a specific example, even in the scenario of developing other TSPs and new TSPs for different diseases, the measurement instrument resources of other TSPs can also be reused to build a new TSP. The TSP module 145 can store the generated TSP in the TSP repository 175. Further details of the example TSP are described in this article.

[0064] The digital measurement solution (DMS) module 150 builds one or more DMSs. In various instances, the DMS module 150 builds one or more DMSs by incorporating specific information into the TSP. Here, the TSP represents a class of solutions for one or more DMSs. For example, the DMS module 150 can incorporate specific device hardware into the elements of the TSP. Thus, the DMS specifies the specific equipment to be used to capture raw measurements. As another example, the DMS module 150 can incorporate specific algorithms into the elements of the TSP. Thus, the DMS specifies specific algorithms for transforming raw measurements into meaningful health data sets that can be interpreted as characterizing a disease.

[0065] In various examples, the DMS module 150 constructs two or more DMSs in a common category represented by the TSP. In various instances, the DMS module 150 constructs three or more DMSs, four or more DMSs, five or more DMSs, six or more DMSs, seven or more DMSs, eight or more DMSs, nine or more DMSs, ten or more DMSs, eleven or more DMSs, twelve or more DMSs, thirteen or more DMSs, fourteen or more DMSs, fifteen or more DMSs, sixteen or more DMSs, seventeen or more DMSs, eighteen or more DMSs, nineteen or more DMSs, twenty or more DMSs, twenty-five or more DMSs, fifty or more DMSs, one hundred or more DMSs, two hundred or more DMSs, three hundred or more DMSs, four hundred or more DMSs, five hundred or more DMSs, six hundred or more DMSs, seven hundred or more DMSs, eight hundred or more DMSs, nine hundred or more DMSs, or one thousand or more DMSs that belong to a common class represented by the TSP. The DMS module 150 may store the generated DMS in the DMS repository 180. Further details of an example DMS are described herein.

[0066] The qualification protocol module 155 executes qualification protocols that enable the quick opening of an upgraded DMS (e.g., in view of upgraded devices and / or upgraded software versions) by verifying comparability of results between multiple DMSs of a common category. For example, when a new device or a new software package is released, the new device or new software package can be incorporated into the updated or upgraded DMS. Here, the qualification protocol module 155 implements qualification protocols to verify the new DMS incorporating the new device or new software package. This ensures that the new DMS achieves comparable results to other DMSs of the same common category.

[0067] In various instances, a DMS that has been successfully verified using a qualification protocol can be deemed to have been successfully verified. For example, metadata associated with a successfully verified DMS can be annotated. For example, the metadata can identify the qualification protocol used, and the fact that the DMS was successfully verified. In various instances, the metadata including the annotations can be made available for inspection by a third party. Thus, a third party (such as a customer interested in using a DMS to characterize a disease) can select a DMS that has been successfully verified.

[0068] The disease characterization module 160 implements the DMS to characterize the disease. In various examples, the disease characterization module 160 can be implemented by a third party entity (e.g., Figure 1A) is adopted by a third-party entity 110 shown in A. For example, the third-party entity can be a customer who is interested in characterizing the disease. Therefore, the third-party entity can adopt the disease characterization module 160 to implement the selected DMS to characterize the disease. In various instances, the disease characterization module 160 can obtain the measurement results of interest. For example, the measurement results of interest can be raw data obtained according to the measurement method specified by the DMS. In addition, implementing the DMS involves transforming the measurement results of interest into meaningful health data using the algorithm specified in the DMS. The disease characterization module 160 can interpret the meaningful health data to characterize the disease.

[0069] The market module 165 implements a market for standardized solutions (e.g., DMS and TSP) and enables third-party entities to access DMS and TSP for their use. In various examples, the market module 165 provides an interface to the third-party entity that depicts the various DMS and TSP available for access. Such an interface can be organized as a directory for easy access.

[0070] In various instances, the market module 165 provides a catalog of TSPs that can be used to characterize various diseases. The market module 165 can receive a selection of one of the TSPs. For example, a third party can select a TSP for characterizing a disease of interest to the third party. In addition, the third party can select the TSP because the TSP includes specifications consistent with the capabilities of the third party. In one scenario, the market module 165 can provide the selected TSP to the third party. In one scenario, the market module 165 can identify one or more DMSs that belong to a public category represented by the selected TSP. Here, the market module 165 provides one or more DMSs in the public category to the third party.

[0071] In various examples, the market module 165 can provide recommendations to third parties that are accessing the market. For example, the market module 165 can provide recommendations to third parties that identify one or more components, one or more resources, one or more TSPs, or one or more DMSs. This can be useful for third parties that may need additional guidance on the best standardized solution that will meet their needs.

[0072] In various examples, the market module 165 receives suggestions for additional standardized solutions that would be valuable. For example, the market module 165 may receive suggestions for specific DMS or TSPs that are not present in the market from a third party. Such a third party may be a resource developer or a customer who has discovered a gap that cannot be met by the current offering of standardized solutions. For example, the suggestion may identify that the specifications of a specific device exceed the specifications of the available TSPs and DMSs. Therefore, the market module 165 may provide suggestions to any one of the resource module 140, the TSP module 145, and / or the DMS module 150 to generate additional standardized solutions that may be included in the market.

[0073] In various instances, the market module 165 provides a catalog of target solution profiles and receives a search query. For example, a third party presented with a catalog of target solution profiles can provide a search query for a specific element or resource in the target solution profile. In various instances, a third party provides a search query for a concept of interest or for a specific disease. The market module 165 queries available TSPs (e.g., TSPs stored in the target solution profile repository 175) based on the search query, and then returns a list of TSPs that satisfy the search query. For example, if the search query specifies a specific concept of interest, the market module 165 evaluates the elements of the TSP for the concept of interest that satisfies the search query. Therefore, the market module 165 can provide (e.g., to a third party) a list of TSPs that satisfy the search query.

[0074] Examples disclosed herein relate to building a TSP and a DMS, and implementing the TSP and the DMS to characterize a disease. Generally speaking, the TSP and the DMS are built on a measurement stack composed of one or more elements (also referred to herein as "layers"). That is, the measurement stack provides the structure or framework of the TSP or DMS. The elements and / or resources of the measurement stack can be generated and / or maintained by the resource module 140, as described above with reference to Figure 1 As described in B.

[0075] The purpose of the measurement stack is to fill the gaps mentioned previously, such as the lack of standardization and concerns about data collection, analysis, and interpretation. First, the measurement stack provides a standardized structure that represents a common way to describe a solution, thereby allowing standardization. Second, the measurement stack enables and enables coordination between multiple resources and components. Third, the measurement stack model will enable resources to be converted between diseases and use cases, thereby enabling reusability at the component level.

[0076] In various instances, a measurement stack includes one or more resources. Examples of resources include measurement definition resources, measurement instrument resources, or evidence resources. A resource refers to one or more elements of a stack. In various instances, a resource refers to two or more elements. In various instances, a resource refers to three or more elements. In various instances, a resource refers to three or more elements.

[0077] In various instances, the measurement definition resource includes two elements. In various instances, the measurement definition resource includes three elements. In various instances, the measurement definition resource includes four elements. In various instances, the measurement instrument resource includes two elements. In various instances, the measurement instrument resource includes three elements. In various instances, the measurement instrument resource includes four elements. In various instances, the evidence resource includes two elements. In various instances, the evidence resource includes three elements. In various instances, the evidence resource includes four elements.

[0078] In various instances, the measurement stack includes two resources. For example, the measurement stack may include a measurement definition resource associated with a specific disease, and a measurement instrument resource that describes the capture of data that can be used to characterize the disease. In a specific instance, the measurement stack includes three resources. For example, the measurement stack may include a measurement definition resource associated with a specific disease, a measurement instrument resource that describes the capture of data that can be used to characterize the disease, and an evidence resource for validating a meaningful data set for the disease.

[0079] In various examples, the elements of a resource are connected to each other. For example, an element of a resource is configured to communicate with at least one other element of the same resource. For example, within a resource, the elements are organized into layers, so that a first element is configured to communicate with a second element adjacent to the first element. This enables information to be transferred from one element to the next element.

[0080] In various instances, an element of a first resource is connected to an element of a second resource. Thus, an element of a first resource can communicate with an element of a second resource. As an example, within a measurement stack, a first resource can be located lower in the measurement stack relative to a second resource. Here, an element of a first resource can be connected to an element of a second resource, thereby enabling the first resource and the second resource to interface with each other.

[0081] In various examples, the resources of the measurement stack are ordered as follows (from bottom to top of the stack): 1) measurement definition resources and 2) measurement instrumentation resources. In a specific example, the resources of the measurement stack are ordered as follows (from bottom to top of the stack): 1) measurement definition resources, 2) measurement instrumentation resources, and evidence resources.

[0082] Example Measurement Stack

[0083] Examples disclosed herein relate to building a TSP and a DMS, and implementing the TSP and the DMS to characterize a disease. Generally speaking, the TSP and the DMS are built on a measurement stack composed of one or more elements (also referred to herein as "layers"). That is, the measurement stack provides the structure or framework of the TSP or DMS. The elements and / or resources of the measurement stack can be generated and / or maintained by the resource module 140, as described above with reference to Figure 1 B. Further details of the example measurement stack are described in International Application No. WO2022234126 filed on May 6, 2022, which is hereby incorporated by reference in its entirety.

[0084] The purpose of the measurement stack is to fill the gaps mentioned previously, such as the lack of standardization and concerns about data collection, analysis, and interpretation. First, the measurement stack provides a standardized structure that represents a common way to describe a solution, thereby allowing standardization. Second, the measurement stack enables and enables coordination between multiple resources and components. Third, the measurement stack model will enable resources to be converted between diseases and use cases, thereby enabling reusability at the component level.

[0085] In various instances, a measurement stack includes one or more resources. Examples of resources include measurement definition resources, measurement instrument resources, or evidence resources. A resource refers to one or more elements of a stack. In various instances, a resource refers to two or more elements. In various instances, a resource refers to three or more elements. In various instances, a resource refers to three or more elements.

[0086] In various instances, the measurement definition resource includes two elements. In various instances, the measurement definition resource includes three elements. In various instances, the measurement definition resource includes four elements. In various instances, the measurement instrument resource includes two elements. In various instances, the measurement instrument resource includes three elements. In various instances, the measurement instrument resource includes four elements. In various instances, the evidence resource includes two elements. In various instances, the evidence resource includes three elements. In various instances, the evidence resource includes four elements.

[0087] In various instances, the measurement stack includes two resources. For example, the measurement stack may include a measurement definition resource associated with a specific disease, and a measurement instrument resource that describes the capture of data that can be used to characterize the disease. In a specific instance, the measurement stack includes three resources. For example, the measurement stack may include a measurement definition resource associated with a specific disease, a measurement instrument resource that describes the capture of data that can be used to characterize the disease, and an evidence resource for validating a meaningful data set for the disease.

[0088] In various examples, the elements of a resource are connected to each other. For example, an element of a resource is configured to communicate with at least one other element of the same resource. For example, within a resource, the elements are organized into layers, so that a first element is configured to communicate with a second element adjacent to the first element. This enables information to be transferred from one element to the next element.

[0089] In various instances, an element of a first resource is connected to an element of a second resource. Thus, an element of a first resource can communicate with an element of a second resource. As an example, within a measurement stack, a first resource can be located lower in the measurement stack relative to a second resource. Here, an element of a first resource can be connected to an element of a second resource, thereby enabling the first resource and the second resource to interface with each other.

[0090] In various examples, the resources of the measurement stack are ordered as follows (from bottom to top of the stack): 1) measurement definition resources and 2) measurement instrumentation resources. In a specific example, the resources of the measurement stack are ordered as follows (from bottom to top of the stack): 1) measurement definition resources, 2) measurement instrumentation resources, and evidence resources.

[0091] See now Figure 2A , which shows an example measurement stack according to an example of the present disclosure. Figure 2A As shown, the example measurement stack includes: specific diseases (in Figure 2A including measurement definitions for Meaningful Aspects of Health (MAH) (referred to as “conditions” in Figure 2A tagged as "Definition" in ); instrumentation resources including components related to capturing data, algorithms, and data sets (in Figure 2A and a description of one or more validation evidence resources used to validate the generated dataset (in Figure 2A ). Exemplary conditions are further detailed in Table 1. Exemplary meaningful aspects of health (MAH) are detailed in Table 2.

[0092] like Figure 2A As shown, a specific disease is located at the bottom of the measurement stack. Here, the disease can control the generation or selection of one or more of the above resources in the measurement stack. For example, the disease controls the generation or selection of measurement definition resources.

[0093] Generally speaking, a measurement definition resource defines a measurable concept related to the disease. Thus, a measurable concept provides information for characterizing a disease (e.g., the presence of a disease, the severity of a disease, the progression of a disease, etc.). For example, for an atopic dermatitis condition, a measurement definition resource may define a concept related to atopic dermatitis as nighttime scratching. Thus, a subject's nighttime scratching may be measured to characterize the subject's disease (e.g., a higher amount of nighttime scratching may indicate a more severe atopic dermatitis than a lower amount of nighttime scratching). Examples of the various elements of a measurement definition resource are described in further detail herein.

[0094] The measuring instrument resource defines how to capture the measurement concepts related to the disease, and further defines how to transform the captured raw data into an interpretable and meaningful health data set. For example, the measuring instrument resource can describe the equipment specifications that affect the capture of the raw data. In addition, the measuring instrument resource can transform the raw data into a health data set that is more meaningful for a specific disease. A meaningful health data set can be a measurement result of a concept related to the disease (as described with respect to the measurement definition resource), or can be easily interpreted to obtain a measurement result of a concept related to the disease. Returning to the above-mentioned atopic dermatitis example, a meaningful health data set can include a measure of scratching at night (e.g., scratching events per hour, scratching duration per hour, total number of scratching events). Alternatively, a meaningful health data set can be a data set from which a measure of scratching at night (e.g., scratching events per hour, scratching duration per hour, total number of scratching events) can be easily extracted. Examples of the various elements of the measuring instrument resource are further described in detail herein.

[0095] The evidence resource includes one or more verifications that verify the data set generated by the measuring instrument resource. This ensures that the data set (e.g., a healthy data set) generated by the measuring instrument resource is accurate and can be used to accurately characterize the disease. Examples of verifications included in the evidence resource can include technical verification, analytical verification, and / or clinical verification. In various instances, the evidence resource includes two or more verifications. In various instances, the evidence resource includes three or more verifications. Generally speaking, performing the verification of the evidence resource ensures that the measurement result is accurate and can therefore be considered to meet the conditions for use and approval of clinical trials (e.g., as its criteria). Examples of the various elements of the evidence resource are described in further detail herein.

[0096] In various instances, different resources of a measurement stack are selected or generated specifically for a particular disease. For example, a measurement definition resource may describe concepts that are particularly relevant to the disease, and therefore, the measurement definition resource may be specific to the disease. In some instances, different resources in a measurement stack are interchangeable and thus may be used in measurement stacks for various diseases. For example, a measurement instrument resource may be interchangeable such that the measurement instrument resource may be included in a first measurement stack for a first disease, and may further be included in a second measurement stack for a second disease. In this way, interchangeable or reusable resources enable more efficient generation and construction of measurement stacks.

[0097] Figure 2B is an example measurement stack showing various elements according to an example of the present disclosure. Figure 2B As shown, the measurement stack includes three resources (e.g., measurement definition resources, measurement instrument resources, and evidence resources). Each of these three resources is composed of two or more elements. Specifically, the measurement definition resource includes: 1) health aspect element, 2) hypothesis element, and 3) concept of interest element. The measurement instrument resource includes: 1) measurement method element, 2) raw data element, 3) algorithm element, and 4) health data element. The evidence resource includes: 1) analysis verification element, and 2) clinical verification element. Figure 2B As shown, there are a total of 8 elements in the measurement stack.

[0098] In various examples, the measurement stack may be arranged differently to include additional elements or fewer elements. In various examples, although not shown, the measurement stack may also include a supervisory element, which is valuable for comparing supervisory experts to measurement endpoints. Such supervisory elements may be included in the evidence resource. Therefore, in such examples, there are a total of 9 elements in the measurement stack.

[0099] In various instances, the functionality of two or more components in a resource may be combined into a single component. Figure 2A 8 elements explicitly shown in FIG. For example, the hypothesis element and the concept of interest element can be combined into a single element. As another example, the health aspect element, the hypothesis element, and the concept of interest element can be combined into a single element.

[0100] First mention the condition (e.g. Figure 2B ) which may refer to any disease. Examples of disorders or diseases include atopic dermatitis, Parkinson's disease, Alzheimer's disease, chronic obstructive pulmonary disease (COPD), pulmonary arterial hypertension (PAH), asthma, retinal disease, major depressive disorder (MDD), or cancer. Additional examples of disorders are described herein and are shown in Table 1.

[0101] Meaningful Aspects of Health (MAH) (in Figure 2B A hypothesis (referred to as health aspects in the context of a disease) generally defines aspects that the disease is to improve. Examples of meaningful health aspects (MAHs) are shown in Table 2. A hypothesis refers to a way to improve the meaningful health aspects. For example, a hypothesis may involve an intervention (e.g., a therapeutic intervention, a surgical intervention, or a change in lifestyle) that predicts that the meaningful health aspects associated with the disease will improve. Generally speaking, improving meaningful health aspects is associated with improving the disease. In various instances, for a specific disease, there may be multiple available meaningful health aspects (e.g., multiple aspects that the disease is to improve).

[0102] The concept of interest describes a measurable unit that informs meaningful health aspects. For example, a concept of interest is a measurable unit that can be used to inform meaningful health aspects related to a disease, and therefore informs the severity of the disease. Examples of concepts of interest (COIs) are shown in Table 3. In various instances, the concept of interest describes a medical measurement of a disease (e.g., a measurable unit that a health care group will measure to determine the severity of the disease). For example, in the context of Parkinson's disease, the medical measurement of Parkinson's disease is tremor. Here, the amount of tremor can be a measure of the severity of the disease. In various instances, the concept of interest describes a measurable experience of an individual with the disease. Here, the measurable experience may not be a medically relevant unit of measurement, but it may still have a significant impact on patients suffering from the disease. In such instances, the concept of interest may be a symptom of the disease that the patient wants to change. For example, also in the context of Parkinson's disease, the measurable experience of an individual with Parkinson's disease may be sleep deprivation. Although sleep deprivation is not a medical unit of measurement for Parkinson's disease, it is still a measure that can provide information about the severity of the disease.

[0103] Referring next to the measurement method element, it generally describes a solution implemented to capture data of a concept of interest. For example, a solution of the measurement method element includes a hardware, software, or firmware solution. Example solutions of the measurement method element include sensors, devices such as computing devices, cellular devices, or wearable devices, and mobile applications. In various instances, the sensor can be built into a device such as a wearable device or a cellular device.

[0104] In various instances, the measurement method element identifies the specifications of the measurement method. For example, for a wearable device, the measurement method element identifies the operating specifications of the wearable device (e.g., the frequency or frequency range at which the device captures data (e.g., 10 Hz to 100 Hz), the time interval at which the device captures data (e.g., 24 hours a day, or in response to a command), the presence of one or more sensors at which the wearable device captures data, the storage capacity of the wearable device, and / or the estimated battery life). In various instances, the product employed by the measurement method uses algorithms to process data captured by (mobile) sensors to generate measures of behavioral function and / or physiological function. This includes novel feature measures and indicators whose underlying biological processes are not yet understood. As with other digital medicine products, these products may be characterized by a large amount of evidence supporting the quality, safety, and effectiveness indicated in their performance requirements.

[0105] Next, refer to the raw data, which represents the raw data set captured according to the specific method of the measurement method element. For example, if the measurement method element identifies a wearable device (and corresponding specifications), the raw data represents the data set captured by the wearable device according to these specifications. In various instances, the raw data itself does not provide meaningful data that can be interpreted. As a specific example, a raw file may include data captured at 10Hz to 100Hz acceleration. The data is captured in 3DSI units (XYZ gravity) and data is collected continuously for 28 days. In short, this example describes what kind of raw data is captured (acceleration in 3D SI units), its frequency (10Hz to 100Hz), and the amount of data captured (28 days).

[0106] Next, we refer to algorithms, which transform the raw data from the raw data element into meaningful data sets (e.g., meaningful health data related to the concept of interest in the measurement). Returning again to the atopic dermatitis example, the algorithm interprets the raw measurement device data captured during sleep and transforms the raw data into meaningful health data (e.g., scratching events). In some scenarios, algorithms are specific to a specific measurement method. Therefore, a specific algorithm in an algorithm element can only transform the raw data set results captured from a specific measurement method.

[0107] Next, refer to the health data element, which includes health data, also referred to herein as meaningful health data or meaningful health data sets. Health data is transformed from raw data by an algorithm, representing an interpretable data set that provides information for a specific concept of interest. Returning to the atopic dermatitis example again, health data may include or be easily interpreted as including any one of total sleep time, scratching events per hour, and the total number of scratching events. Here, health data is the result of an algorithm / other processing that converts "raw data" into its final health-related data. An example may include converting accelerometer data into a number of steps. There may be intermediate stages in the processing, such as identifying each severe symptom onset in a day can be a step, and then a further refinement is to calculate the average time of all of these. Both of these can be classified as health data.

[0108] Next, refer to the analytical validation element, which involves validating one or more other elements in the measurement stack. In various instances, the input to analytical validation includes elements of measurement definition resources and elements of measurement instrument resources. The output of analytical validation includes supporting evidence of the success or failure of validation of the corresponding solution that combines the elements of measurement definition resources and elements of measurement instrument resources. Generally speaking, a digital measurement solution is incomplete unless the results it generates are proven to be analytically valid to support clinical interpretation. During analytical validation, the digital measurement solution is exposed to a range of test conditions and procedural stresses to generate sample data, and the results are recorded for statistical analysis. These results validate or redefine the functional scope, outside of which the reliability of the measurement results may be questionable. Successful analytical validation will mean that solutions that meet this profile can support accurate labeling claims without unexpected risks and consequences.

[0109] In various instances, the analysis verification element can perform analysis verification of device specifications, algorithms, and health data outputs. In various instances, the analysis verification involves comparing the data with appropriate measurement standards. Example measurement standards for various diseases can be established by a third party or in the community. For example, example measurement standards for different diseases can be standards established by ICHOM Connect.

[0110] For example, analytical validation ensures that meaningful health data meets the necessary sensitivity requirements, specificity requirements, and / or reliability requirements. In various instances, the necessary sensitivity requirement is at least 50% sensitivity, at least 60% sensitivity, at least 70% sensitivity, at least 75% sensitivity, at least 80% sensitivity, at least 85% sensitivity, at least 90% sensitivity, at least 91% sensitivity, at least 92% sensitivity, at least 93% sensitivity, at least 94% sensitivity, at least 95% sensitivity, at least 96% sensitivity, at least 97% sensitivity, at least 98% sensitivity, or at least 99% sensitivity. In various instances, the necessary specificity requirement is at least 50% specificity, at least 60% specificity, at least 70% specificity, at least 75% specificity, at least 80% specificity, at least 85% specificity, at least 90% specificity, at least 91% specificity, at least 92% specificity, at least 93% specificity, at least 94% specificity, at least 95% specificity, at least 96% specificity, at least 97% specificity, at least 98% specificity, or at least 99% specificity. In various instances, the necessary reliability requirement is any of at least 50% reliability, at least 60% reliability, at least 70% reliability, at least 75% reliability, at least 80% reliability, at least 85% reliability, at least 90% reliability, at least 91% reliability, at least 92% reliability, at least 93% reliability, at least 94% reliability, at least 95% reliability, at least 96% reliability, at least 97% reliability, at least 98% reliability, or at least 99% reliability.

[0111] In various examples, analytical validation enables comparison of the digital solution provided by the measurement stack with a reference measure currently employed or previously developed to characterize a disease. For example, returning to the atopic dermatitis example, analytical validation may determine that the digital solution provided by the measurement stack is able to appropriately measure nocturnal scratching, based on appropriate reliability requirements, specificity requirements, and sensitivity requirements. Here, the digital solution may be comparable to or better than a reference measure (e.g., infrared observations to monitor nocturnal scratching).

[0112] In various instances, the analysis verification element includes analysis verification, in addition or alternatively, including technical verification. In various instances, technical verification verifies that the data set (e.g., raw data from the raw data element and health data from the health data element) is appropriate. As an example, technical verification can evaluate whether the captured raw data complies with the firmware protocol and / or software protocol dedicated to the device. As another example, technical verification can evaluate the location of the raw data captured by the measurement method according to the specifications identified in the measurement method. For example, these specifications can include battery life, data storage space, and available measurement frequency. Therefore, technical verification determines whether the raw data captured by the measurement method complies with these specifications. As an example, if the raw data set indicates that the data is captured at a frequency exceeding the specification identified in the measurement method, the technical verification can mark the problem, so that the verification process fails. Alternatively, if the raw data set indicates that the data is captured at a frequency within the specification identified in the measurement method, the technical verification can be considered successful. Additional details and examples of technical validation are described in Goldsack, JC et al., Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). npj Digit. Med., 3, 55 (2020), which is hereby incorporated by reference in its entirety.

[0113] Next, refer to the clinical validation element, which involves clinical validation of digital solutions. Clinical validation is the process of evaluating whether a measurement solution acceptably identifies, measures, or predicts a meaningful clinical, biological, physical, functional state, or experience in a specified use environment. It is valuable to understand what levels of accuracy, precision, and reliability are valuable for a solution to be useful in a specific clinical research environment. Clinical validation aims to obtain measurement results that have undergone verification and analytical validation steps and evaluate whether they can answer specific clinical questions. Generally speaking, a digital measurement solution is incomplete unless the results it generates are interpretable from a clinical perspective and are sufficiently related to meaningful health aspects of the disease. Here, the clinical validation element provides guidance for clinically interpreting measurement results.

[0114] For example, clinical validation can involve analyzing whether a digital solution identifies, measures, and predicts meaningful clinical, biological, physical, functional states, or experiences associated with a disease.

[0115] As an example of clinical validation, a guideline can be included that identifies temperature measurements with increments of 0.00001% as being irrelevant to clinical decision making. In addition, the temperature standard can be identified as 0.1 degree increments. In various instances, clinical validation is an in vivo validation performed in a specific target population. Therefore, clinical validation means checking whether the measurement stack can effectively answer clinical questions related to the disease. Returning again to the atopic dermatitis example, clinical validation can involve evaluating the therapeutic effect of nighttime scratching within an intervention patient population. Here, it is expected that the intervention reduces the amount of nighttime scratching. The specific procedures identified in the clinical validation can be performed during or after the clinical trial to ensure that changes in the patient population are accurately evaluated, which provides an accurate evaluation of the impact of the intervention. Further details of clinical validation are described in Goldsack, JC et al., Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs). npj Digit. Med., 3, 55 (2020), which is hereby incorporated by reference in its entirety.

[0116] In various instances, the measurement stack also includes a regulatory element for comparing regulatory experts with measurement endpoints. Generally speaking, the regulatory element may include regulatory guidelines. In various instances, the regulatory element includes scientific advice from a third party (such as a third-party regulator). For example, for the "nighttime scratching" measurement, it is necessary to standardize what "nighttime" refers to. Therefore, the regulatory element can identify a guideline for defining "nighttime" (e.g., the start time when a person tries to fall asleep), an example of which can be a measure defined as the percentage of time a patient is conscious during his or her total sleep opportunity (TSO) time. Additional instances of regulatory guidelines can be standard guidelines (e.g., guidelines issued by the U.S. Food and Drug Administration (FDA), such as FDA patient-reported outcomes (PRO) guidelines or FDA patient-centered drug development (PFDD) guidelines series).

[0117] In various instances, the regulatory element includes guidelines that help achieve regulatory acceptance. For example, different regulatory pathways involve different requirements for achieving regulatory acceptance. In some scenarios, a request can be submitted through an FDA CPIM meeting, within an IND, or through a formal qualification program. In a specific instance, the regulatory element evaluates one or more other elements of the measurement stack, such as elements of measurement definition resources and / or other elements of evidence resources. Therefore, the regulatory element can save resources by involving regulators as early as possible when the use environment (COU), digital measurements (medical devices, digital biomarkers, clinical outcome assessments) and all verifications (e.g., technical verification, analytical verification, and clinical verification) can be approved.

[0118] In various examples, the supervision element can be used by a third party (such as a regulator), and these third parties can further cooperate to jointly develop and / or propose improvements to standardized solutions. This makes it possible to dynamically supervise and evaluate standardized solutions. For example, regulators can provide new evidence requests and questions in more real time. In response, new evidence, comments and additional context can be provided to regulators. In various examples, dynamic supervision evaluation involves multiple stakeholders (for example, involving customers, resource developers, pharmaceutical companies, regulators, etc.), so that collaborative methods can achieve regulatory approval of standardized solutions. An example of dynamic supervision evaluation is described below in Example 5.

[0119] In various instances, regulators can evaluate the tolerance and / or deviation of a standardized solution (e.g., a DMS). Here, the regulatory element can provide guidance for understanding the expected size of the treatment effect. If the effect is large, the tolerance can be larger; if the effect is small, the measurement also needs to be more precise. In various instances, the regulatory element can involve regulatory recommendations given independently of the intervention (e.g., which measurements are meaningful).

[0120] Figure 2C is an example measurement stack of one or more elements forming a target solution profile (TSP), a target instrumentation profile (TIP), or a target component profile (TCP) according to the instructions of examples of the present disclosure. In various examples, the TSP covers a complete measurement stack (e.g., Figure 2C ). In contrast, TIP and TCP include fewer than all elements of the measurement stack. For example, TIP includes six elements (from the concept of interest to technical / analysis verification). TCP refers to individual elements of the measurement stack.

[0121] In general, TSPs are considered solution agnostic (e.g., no specific brand and version are specified). TIPs are instrument-centric, and some elements of measurement definition resources (e.g., symptoms, meaningful health aspects, hypotheses) and some elements of evidence resources (e.g., clinical validation and regulatory intelligence) are agnostic. In addition, TIPs are considered disease agnostic because no element of the TIP layer is associated with a specific disease, meaningful health aspects, and patient population. This adds new value to the available resources provided to stakeholders and is in line with these TIPs. For example, TIPs can be interchangeable between different TSPs designed for specific diseases. Therefore, developers (e.g., developers of various elements or resources) can develop functional resources covering a variety of diseases. This contrasts with having developers develop new resources and research designs for each specific disease. New development of separate resources may be a waste of material resources, because the required resources may often already have ready-made solutions available. In addition, the resources included in TIPs can be easily reused for a variety of diseases. By implementing clinical validation for specific diseases, TIPs can be used for a variety of diseases, thereby allowing the reusability and sustainability of available resources to be improved. For example, a class of actigraphy solutions for measuring daily living physical activity (DLPA) for a first condition of pulmonary hypertension has been validated. Analytical validation has validated the ability to measure DLPA parameters by the devices included in this TIP. In addition, DLPA may also be a concept of interest in a second condition of Parkinson's disease (PD). Therefore, a validated TIP for measuring DLPA can be reused for a second condition of PD without re-performing the technical and analytical validation steps, as these are already included in the TIP. In addition, TIPs provide a structure to available resources, thereby preventing stakeholders from being overwhelmed by the myriad TSPs, devices, and algorithms available for digital measurement. Therefore, solutions within one TIP can be easily compared to deliver the most fit-for-purpose solution for a new study design. This increases the likelihood of picking the best measurement instrument for a specific use case.

[0122] like Figure 2C As shown, TCP defines a single common element. Therefore, TCP includes a common description of the technical details of each element. For example, Figure 2CA TCP identified in a TIP can include a generic description of a measurement method (e.g., tri-axial accelerometer (wrist-worn), 10Hz to 100Hz, 18+ hour battery life, 500+MB data storage space). In general, a TCP allows resources from the same or similar TCP to be easily substituted into multiple TIPs and TSPs. For example, a TSP includes a measurement method with a battery life of 18+ hours (Apple Watch 6), but a specific study design requires a battery life of at least 96 hours. Therefore, the TCP describing 18+ hours can be replaced with a different TCP with a battery life of at least 96 hours. This excludes the availability of an Apple Watch 6 in a TIP / TSP, and only components that meet the requirements will be included. Therefore, TCPs allow researchers to more easily best meet their specific needs.

[0123] Example Target Solution Configuration File

[0124] As described herein, the TSP encompasses a complete measurement stack. The generation and maintenance of the TSP may be performed by the TSP module 145, as described above in Figure 1 B. The various elements of the TSP are described using a generic description (in contrast to the DMS that describes a specific solution), thereby providing a profile that is agnostic to device technology. Device technology agnostic means that not only the hardware is agnostic (e.g., a specific device is agnostic), but also the software is agnostic (e.g., the software version is agnostic, such as the software used for the application or the software used for the device). This prevents any association with a specific brand, model, or technology, and allows a specific DMS to be smoothly simulated as a generic TSP. Therefore, the TSP is a generic profile that defines a class of DMSs, each of which provides further specificity to the generic profile. Because the TSP is device technology agnostic, it is easier to provide novel solutions for the same or even different conditions. This makes it possible to improve the development of sustainable solutions that adapt to the future. In addition, the TSP provides coordination in the ecosystem and shows high potential for shortening the life cycle of solution development because evidence generated early can be reused. In summary, the TSP shows the potential to ultimately accelerate the adoption of digital metrics in clinical research.

[0125] The TSP provides a general description that covers multiple DMSs within the same solution category. DMSs that conform to the category represented by the TSP can be considered to meet the purpose of the same use case. Because the TSP is device technology agnostic, the included DMSs show improved reusability of resources. Instead of being used for just one study, the TSP accelerates the reuse of available resources and increases the value of all resources included in the DMS. Moreover, the TSP categories can be utilized to compare subtle differences between similar TSPs. This allows stakeholders to more easily compare multiple TSPs (and DMSs in the categories represented by the TSP) to select the best preference (e.g., cost, device weight, or battery life) for their specific use cases. In various instances, a DMS may also conform to a general description of various TSPs.

[0126] Furthermore, the TSP allows for version management (lifecycle management of digital measurement solutions). Over time, available devices, algorithms, and technologies evolve. In various instances, the TSP can be edited and modified, resulting in newer versions (which can coexist with older versions). This allows elements of the solution to be upgraded if the solution as a whole still meets the TSP criteria. The validation of the evolving TSP is evaluated through a Qualification Protocol (QP). The QP validates version management and ensures that the TSP is considered to be future-proof. In various instances, the QP allows for version management of specific resources and is able to validate comparability between multiple DMSs.

[0127] Figure 2D An example target solution profile according to an example of the present disclosure is shown. Here, the elements of the TSP are described in general terms. However, although Figure 2D The TSP shown in identifies a general disease or condition, but in various instances, the TSP identifies a specific disease or condition.

[0128] See the measurement method element, which describes a device-agnostic measurement method. For example, a measurement method element can specify a specific class of devices. Examples of a class of devices can include, but are not limited to, any of wearable devices, ingestible devices, image and voice-based devices, contactless sensors, and sensor-based smart devices (e.g., scales, thermometers, and respirators). The measurement method element can also include specifications for the measurement method, such as battery life, data storage space, and available measurement frequency. Therefore, developers can determine whether TSP is suitable for their digital solution based on the specifications of the measurement method (e.g., if the developer needs to capture data for at least 96 hours, but the TSP measurement method only specifies 18 hours of battery life, the developer determines that a different TSP is needed).

[0129] See the raw data element of TSP, which describes the raw file captured according to the measurement method. For example, raw data according to the measurement method specification. Therefore, if the measurement method indicates a measurement frequency of 100Hz, the raw data element describes the raw file including data captured at a measurement frequency of 100Hz. In various instances, the digital measurement results reported by the measurement method are exported through a data supply chain including hardware elements, firmware elements, and software elements. The term "raw data" is used to describe data that exists in the early stages of the data supply chain. Sample-level sensor output data (e.g., a 50Hz accelerometer signal or a 250Hz ECG signal) will be raw data. Although signal processing methods may have been applied to the data (e.g., downsampling, filtering, interpolation, smoothing, etc.), the data is still considered "raw" because it is a direct representation of the raw signal generated by the sensor.

[0130] See the algorithm element of the TSP, which identifies one or more algorithms that can appropriately transform raw data into meaningful health data. Here, the algorithm is designed based on the specific measurement method used to capture the raw data. As an example, if the measurement method indicates a measurement frequency of 100 Hz, the algorithm is designed to transform the data specifically captured at a frequency of Hz. In various instances, the algorithm element represents a series of data manipulation processes embedded in firmware and software, including but not limited to signal processing, data compression and decompression, artificial intelligence, and machine learning. Algorithms are calculations that transform data from sensors into meaningful information. The algorithm can be directly part of the sensor, or it can be operated by one party to perform additional data science operations to create derived metrics.

[0131] Refer to the analysis verification element, which enables the verification of the elements (e.g., measurement methods, raw data, algorithms and health data) of the measuring instrument resources to ensure that the raw data and / or health data can reliably, effectively and sensitively meet the appropriate standards. In various instances, analysis verification occurs at the intersection of engineering expertise and clinical expertise. Analysis verification involves evaluating processed data and requires testing of human subjects. After the verified sample level data has been generated by the measurement method, the algorithm is applied to these data to create behaviorally or physiologically meaningful indicators. The process begins at the point where the verified output data (sample level data) becomes the data input for algorithm processing. Therefore, the first step of analysis verification requires a defined data capture scheme and a specified test subject population. In the analysis verification process, the indicators generated by the algorithm are evaluated against appropriate reference standards.

[0132] In various examples, the TSP can be constructed using a disease-focused approach (e.g., a bottom-up approach). In various examples, the TSP can be constructed using a measurement instrument-focused approach (e.g., a top-down approach). Regardless of the approach (e.g., bottom-up or top-down), the final TSP and DMS for the category can be the same.

[0133] First, referring to the method of focusing on the disease, a specific disease is identified. Here, a measurement definition meaningful for patients with the disease is determined. This includes determining the concept of interest to be measured. Next, a suitable measuring instrument is developed, or if available, a ready-made solution can be selected. For example, the measuring instrument resources of different TSPs can be selected and reused for the current TSP. Assuming that the measuring instrument resources of the TSP are usually described in general terms, the reuse of the measuring instrument resources for the current TSP may require little additional work or no additional work at all. Next, an evidence resource is generated for the TSP. In various instances, generating evidence resources involves determining the technical and analytical verification of the measuring instrument suitable for the TSP. In various instances, the elements of the evidence resource (such as elements for performing technical and analytical verification) can be reused from another TSP. Assuming that the general measuring instrument resources of another TSP may have been previously performed on the technical and analytical verification, the current TSP does not need to re-perform the same technical and analytical verification again. In various instances, the elements of the evidence resource include clinical verification elements. Here, the clinical verification element verifies whether the result is clinically valid. Therefore, generating evidence resources includes generating clinical verification elements that are meaningful for ensuring that the results of the TSP have clinical value.

[0134] See the focused approach to measuring instruments, which starts with discovering available components. Components with similar measuring instruments are identified and then divided into several categories of solutions. These individual components are divided into separate component categories called TCPs (just a generally described layer of the measurement stack, such as a measurement method or algorithm). If the concepts of interest, measurement methods, raw data, algorithms, health data and technical verification / analysis validation are completed, multiple TCPs at different layers can be merged into TIPs. TSPs are built on top of these TIPs by completing them with disease-related layers with consistent definitions and verified measuring instruments. For example, TSPs are built by generating meaningful health aspects and clinical validation elements on top of the elements of these TIPs.

[0135] The instrumentation-focused approach makes it easy to develop TCPs, multiple TIPs, and many TSPs. In the instrumentation-focused approach, smaller stakeholders are able to contribute to resource development more easily, as they usually own resources as measurement methods and algorithms within their digital portfolio. Resource providers can now focus more on the development of one element, which can be useful for multiple studies.

[0136] In various examples, given a complete TSP, resources and elements of the TSP can be quickly drawn up. For example, TIP and TCP can be quickly drawn up from the complete TSP by excluding defining related layers or by selecting a separate layer respectively.

[0137] As described herein, resources and elements of a TSP may be interchangeable and may be replaced. For example, a TIP from a first TSP may replace a second TIP in a second TSP. Here, replacing a TIP may be beneficial because the validation required to take into account the replacement is minimized. For example, only the technical validation / analytical validation of a TSP (including the now replaced TSP) may be re-evaluated. As another example, only the clinical validation of a TSP (including the now replaced TSP) may be re-evaluated (e.g., no technical / analytical validation may need to be performed).

[0138] Overview of an apparatus for managing at least elements of a standardized DMS

[0139] See now Figure 3 , presents an apparatus 300 for managing at least elements of a standardized digital solution for characterizing a disease. The apparatus 300 may include a processor 304 and a memory coupled to and / or in communication with the processor 304. The memory may contain instructions for configuring the processor to perform various tasks. In some instances, the processor 304 may be configured to obtain a central data structure 312. The central data structure 312 may be obtained via a network connection, may be retrieved from the memory 308, and / or may be communicated with one or more external computing devices.

[0140] The central data structure 312 may include one or more databases. In some instances, the central data structure 312 may include compiled evidence 316 of multiple elements of previously approved standardized digital solutions. The standardized digital solution may include a digital measurement solution, a target solution profile, and / or its elements. For example, the compiled evidence 316 may include a standardized digital solution in the form of a DMS. The compiled evidence 316 may include data of one or more elements of the DMS, such as but not limited to raw sensor data, hardware data, software data, diagnostic data, algorithm data, disease data, patient data, and / or other types of data of one or more elements of the DMS. In some instances, the compiled evidence 316 may include a standardized digital solution in the form of a target solution profile. For example, the compiled evidence data 316 may include data of one or more elements of the TSP, such as but not limited to measurement data, instrument data, data of methods for capturing and transforming raw data, public category data of one or more DMSs, and the like. In some instances, the compiled evidence 316 may be pre-approved by one or more individuals, organizations, and the like. The compiled evidence 316 may include data, such as but not limited to medical data, diagnostic data, hardware data, software data, patient data, patient population data, and / or other data. Compiled evidence 316 and / or central data structure 312 may include DMS', TPS, measurement stack and / or elements thereof, as described above with reference to Figures 1 to 2D The above, but not limited to.

[0141] The central data structure 312 may include one or more layers. In some examples, the central data structure 312 may include one or more layers, as described above with reference to Figures 1 to 2D As described, but not limited to. Each element of the digital measurement solution and / or target solution profile can be located in one or more layers of the central data structure 312. For example, but not limited to, the central data structure 312 may include a validation layer, a measurement instrument layer, and / or a definition layer. The validation layer of the central data structure 312 may include data, such as, but not limited to, regulatory intelligence and / or acceptance, clinical validation and / or technical verification and / or analytical validation data. The validation layer of the central data structure 312 may include evidence supporting that the measurement stack is fit for purpose in a specific use environment. As a non-limiting example, the validation layer may include evidence supporting the use of the measurement stack in diagnosing Parkinson's disease. The validation layer of the central data structure 312 may be part of and / or combined with one or more other layers of the central data structure 312, such as, but not limited to, the measurement instrument and / or definition layer.

[0142] The instrumentation layer of the central data structure 312 may include data, such as, but not limited to, health data, algorithm data, raw data, measurement methods, and the like. Health data may include patient vital sign data, such as, but not limited to, heart rate, blood pressure, blood oxygen level, respiratory rate, and / or other biological indicators of an individual. Algorithm data may include information about one or more algorithms and / or processes used to calculate, determine, and / or generate health data and / or measurement data of a patient. As a non-limiting example, algorithm data may include data for a specific EKG algorithm. Raw data may include information generated by one or more sensors. For example, but not limited to, raw data may include raw sensor data of a heart rate monitor, a respiratory monitor, a blood oxygen monitor, and the like. Measurement methods may include one or more operations for obtaining health data and / or other data of an individual. Measurement methods may include, but are not limited to, one or more procedural steps used, hardware used, software used, clinical settings, patient population data, and the like.

[0143] In some instances, the instrumentation layer of the central data structure may be part of and / or combined with the validation layer of the central data structure 312. The instrumentation layer of the central data structure 312 may include user-centric factors, hardware component details, software component details, and / or other elements involved in the capture and processing of data related to the concept being measured. User-centric factors may include, but are not limited to, age, weight, height, body composition, race, blood type, etc. Hardware component details may include, but are not limited to, processor information, sensor information, power supply information, etc. for one or more sensors, medical devices, and / or other components having hardware. Software component details may include, but are not limited to, software version, software type, software used, etc. for one or more sensors and / or other measurement devices.

[0144] The definition layers of the central data structure 312 may include concepts of interest, meaningful aspects of health, aspects of conditions, and / or other parameters. Concepts of interest may include measurement areas of an individual, such as, but not limited to, reflexes, attention, language, executive function, muscle strength, muscle atrophy, sleep patterns, digestive conditions, exercise, and / or other measurement areas of an individual. Meaningful aspects of health (MAH) may include aspects of conditions that a patient would like to improve and / or prevent. Conditions may include any affliction that a patient or individual may experience, such as, but not limited to, digestive issues, sleep apnea, movement imbalances, and the like.

[0145] In some examples, the central data structure 312 may include one or more measurement stacks. The measurement stacks may be as described above, but are not limited to, with reference to Figure 1. In some instances, each measurement stack may include one or more validation, instrumentation, and / or definition layers. In some instances, each measurement stack may include additional layers, such as metadata or other layers. The metadata layer may include one or more layers of data, such as validation, instrumentation, and / or definition layers. In some instances, the central data structure 312 may act as an archive. For example, the central data structure 312 may be configured to store, update, and / or modify multiple measurement stacks, layers, DMS, TSP, etc. The central data structure 312 may include data, such as, but not limited to, input from one or more collaborators for one or more standardized digital solutions and / or multiple elements, feedback from one or more parties, DMS data, TSP data, etc.

[0146] Still see Figure 3 In some instances, the processor 304 may be configured to receive a user request 320. The user request 320 may be received via user input, an external computing device, and / or other communication paths. For example, but not limited to, the user request 320 may be received via a mobile application, a web browser, a web portal, and the like. The user request 320 may be a request for a new standardized digital solution 324. The new standardized digital solution 324 may be a standardized digital solution for a diagnostic method, a patient, a disease, and the like that has not been previously applied and / or recorded in the central data structure 312. The user request 320 may specify one or more diseases, conditions, patients, instruments, measurements, and / or other data. The processor 304 may query the compiled evidence 316 of the central data structure 312 for data applicable to the user request 320. As a non-limiting example, the user request 320 may be for night scratching, and the processor 304 may query the compiled evidence 316 of night scratching via the central data structure 312.

[0147] Still see Figure 3 In some instances, the processor 304 may be configured to receive additional data 328. The processor 304 may receive the additional data 328 via one or more external computing devices, user input at the apparatus 300, etc., but is not limited thereto. The additional data 328 may include new data for one or more elements of the DMS or TSP, such as, but not limited to, medical data, patient data, sensor data, disease data, etc. The additional data 328 may be generated by individuals and / or organizations executing one or more digital medical solutions. For example, the additional data 328 may include sensor data of a heart rate sensor collected during execution of the new standardized digital solution 124. In some instances, the additional data 328 may include, but is not limited to, sensor data, hardware data, software data, patient data, diagnostic data, method data, disease data, and / or other types of data.

[0148] The processor 304 may be configured to update the central data structure 312 to incorporate the additional data 328. In some instances, updating the central data structure 312 to incorporate the additional data 328 may include modifying, replacing, and / or adding to the compiled evidence 316. For example, but not limited to, the additional information 328 may include data from a previously approved DMS, but for a new patient population, such as a pediatric patient population. The processor 304 may update the central data structure 312 and / or the compiled evidence 316 to include data for the new patient population. In some instances, the processor 304 may modify one or more DMSs or TSPs based on the additional data 328 to generate a new standardized digital solution 324. For example, one or more elements of a DMS or TSP may be modified and / or exchanged with one or more other elements of other DMS or TSP elements. The new standardized digital solution 324 may be stored in the central data structure 312.

[0149] The processor 304 may be configured to utilize or otherwise use one or more elements of the compiled evidence 316 to generate a new standardized digital solution 324. For example, the processor 304 may determine compatible or otherwise usable data of the compiled evidence 316, such as, but not limited to, sensor data, hardware data, software data, patient data, disease data, etc. of a previously accepted standardized digital solution. For example, but not limited to, the processor 304 may retrieve a measurement stack, DMS, and / or TSP having data that is compatible with the requested data of the user request 320. The processor 304 may be configured to provide one or more measurement stacks from the central data structure 312 to the user in response to the user request 340.

[0150] In some examples, processor 304 may determine the acceptability of compiled evidence 316 based on an acceptability threshold or acceptability classification. An acceptability threshold refers to a value or range of values ​​that, if met, considers one or more aspects of a measurement stack to be acceptable.

[0151] The acceptability threshold may include one or more values ​​representing, but not limited to, the amount of data, the quality of data, the recency of data, and the like. For example, the acceptability threshold may include one or more standards (in some instances, analytical or technical standards) that have been defined and / or accepted by one or more regulators. The standards may include performance levels, algorithm accuracy, sensor capabilities, and the like. The acceptability threshold may be specific to one or more elements, sensors, algorithms, and the like. For example, in the case of an acceptability threshold for a sensor, the acceptability threshold may include an algorithm that requires raw data measured at a specific frequency. In an example, the acceptability threshold may include an interpretation requirement. For example, the acceptability threshold may include the ability of a sensor to measure sensitivity changes at a specific level (e.g., a quantitative level), such as a level that can assist in clinical decision making. As a non-limiting example, the acceptability threshold may include a performance level of approximately 85% of the ideal value of the algorithm. As another non-limiting example, the acceptability threshold may include 100,000 heart rate measurements using a specific heart rate sensor. In this example, if the measurement stack utilizes 100,000 heart rate measurements using a particular heart rate sensor, the particular heart rate sensor may be deemed acceptable for generating a new standardized digital solution 324. In an example, the acceptability threshold may be defined or determined at least in part by a qualification protocol for the DMS that describes the steps to be taken / qualifies the requirements for a particular solution.

[0152] The processor 304 may generate and / or retrieve one or more digital measurement templates, digital measurement solutions, etc. based on the user request 320. In some instances, one or more digital measurement templates may be pre-filled with data that has been previously accepted by one or more individuals, while having one or more blank fields for individuals to fill in. The processor 304 may determine the reusability of previously applied standardized digital solutions to the user request 320. Reusability may refer to one or more characteristics of data and / or objects that can be used in a variety of scenarios. As a non-limiting example, the processor 304 may determine that the reusability of a DMS for sleep apnea is about 80% of that for a DMS for night scratching. For example, in this example, a DMS for sleep apnea may have algorithms, sensors, methods, and / or measuring instruments similar to those for a DMS for night scratching. In some instances, the processor 304 may generate a new standardized digital solution 324 based on a reused DMS for a central data structure 312 for a previously unapplied purpose. For example, similar to the above-mentioned night scratching example.

[0153] In other instances, the processor 304 may generate a new standardized digital solution 324 by adding or modifying a previously applied DMS of the central data structure 312. The processor 304 may add the additional data 328 to a previously approved DMS of the central data structure 312 to generate an extended new standardized digital solution 324. The extended new standardized digital solution 324 may include a previously approved DMS combined with additional data, such as the additional data 328. As a non-limiting example, a previously approved DMS for insomnia may be extended with the additional data 328 including continuous air pressure access (CPAP) machine data, which may generate a new extended standardized digital solution 324 for sleep apnea.

[0154] Still see Figure 3 In some instances, the processor 304 may identify one or more portions of the new standardized digital solution 324 that may have been changed and / or modified with the additional data 328. For example, the processor 304 may display any data described throughout the present disclosure via a graphical user interface (GUI). The GUI may include one or more icons, text elements, user input fields, etc. The GUI may be displayed on a screen, such as, but not limited to, a smart phone, a tablet computer, a monitor, etc. In some instances, the processor 304 may highlight or otherwise mark the additional data 328 added to the new standardized digital solution 324. The processor 304 may generate and display one or more annotations of the additional data 328 added to the new standardized digital solution 324, such as, but not limited to, time, date, hardware data, software data, medical professional data, feedback from one or more medical professionals, etc. In some instances, the processor 304 may be configured to identify duplicate data, approved data, etc. Identification may include marking, highlighting, annotating, or otherwise marking data via a GUI display. For example, but not limited to, the processor 304 may be configured to identify the new standardized digital solution 324 as approved by one or more regulatory collaborators.

[0155] In some instances, before receiving the user request 320, the processor 304 may generate a digital environment that may include multiple elements of a standardized digital solution. The digital environment may be accessed by multiple collaborators and / or users. In some instances, the digital environment may include a virtual network that may provide access to one or more digital files. In some instances, before receiving the user request 320, the processor 304 may generate a task within the digital environment by establishing standardized roles for two or more collaborators. The task may include a goal and / or purpose for one or more people. The standardized role may include the work of one or more collaborators. In some instances, the processor 304 may be configured to assign a primary role to a first collaborator among two or more collaborators, and to assign a secondary role to a second collaborator among two or more collaborators. Compared to the secondary role, the primary role may have additional permissions. In some instances, the processor 304 may assign additional roles to the primary and secondary roles and / or secondary roles of multiple collaborators. The additional role may have fewer permissions than the primary role and / or the secondary role. In some instances, additional roles may be assigned to fund sponsors, observers, technology providers, data partners, etc., but are not limited thereto. The authority of one or more roles may include the authority to access and / or view the results generated from one or more tasks. In some instances, the authority of one or more roles may include the authority to use the results generated from one or more tasks. In some instances, the processor 304 may be configured to select two or more collaborators for inclusion in the task before generating the task. In some instances, generating the task may include generating a private task that can only be accessed by two or more collaborators with a defined role. In some instances, generating the task includes generating a public task that can be accessed by one or more additional collaborators (such as, but not limited to, two or more collaborators). In some instances, before receiving input from one or more collaborators for defining at least an element of a standardized new digital measurement solution 324, the processor 304 may be configured to receive an agreement on one or more standards of a task from two or more collaborators. The standard of the task may include the terms and conditions of the agreement. In some examples, prior to receiving the agreement from each of the two or more collaborators, the processor 304 can be configured to generate a predefined template identifying one or more criteria for the task and provide the predefined template to each of the two or more collaborators. In some examples, generating the predefined template can include receiving input from the one or more collaborators to establish characteristics of the second role, and incorporating the input from the one or more collaborators into the predefined template. In some examples,

[0156] In some instances, two or more collaborators may express a common interest, as an example of advancing irritable bowel syndrome treatment, but not limited thereto. In some instances, the processor 304 may be configured to receive input for elements of the new standardized digital solution 324 from one or more collaborators. The processor 304 may be configured to provide access to at least a portion of the central data structure 312 to one or more collaborators. In some instances, the processor 304 may be configured to receive feedback from a supervisor collaborator, which is dedicated to the input that contributes to at least an element of the new standardized digital solution 324 received. Feedback from the supervisory collaborator may include supervisory acceptance of at least an element of the new standardized digital solution 324. In some instances, feedback from the supervisory collaborator may include one or more questions or requests related to at least an element of the new standardized digital solution 324. In some instances, the processor 304 may be configured to receive additional input from two or more collaborators in response to feedback from the supervisory collaboration. The processor 304 may record additional input from two or more collaborators in a central data structure 312 accessible within a task and / or goal. In some examples, processor 304 may be configured to publish central data structure 312 outside of a first task for use in one or more subsequent tasks.

[0157] In some instances, the processor 304 may be configured to display a version history of one or more DMSs. The version history may include the most recent use date, the most recent update date, and / or other data of the DMS. The processor 304 may be configured to generate a collection of one or more measurement stacks for one or more DMSs. The collection of one or more measurement stacks may be generated based on symptoms, diseases, patient populations, etc. For example, but not limited to, the processor 304 may generate a collection of measurement stacks for adolescent asthma. A collection of measurement stacks may be generated for one or more environments of use (COUs).

[0158] Measuring the acceptability of the layers of the stack

[0159] Figure 4 is an illustrative example of a graphical user interface (GUI) that illustrates various acceptability levels for layers of a measurement stack. The measurement stack includes one or more layers, such as, but not limited to, verification, instrumentation, and / or definition layers, such as those described above with reference to Figures 1 to 3 A processor and / or computing device (such as the one described above) Figure 3The processor 304 is configured to calculate or otherwise determine the acceptability of one or more elements of each layer of the measurement stack. Acceptability may refer to reusability and one or more elements of one or more layers for various purposes. For example, one or more elements of one or more layers of the measurement stack may be reused for different diseases, conditions, patients, sensing devices, etc. For example, the verification layer may have, but is not limited to, elements of regulatory intelligence and / or acceptance, clinical validation, and / or technical verification / analysis validation. Each of the above-listed elements may have a corresponding acceptability score that may be determined based on an acceptability criterion (or acceptability threshold). Acceptability criteria may include, but are not limited to, data quality, data volume, data recency, and / or other criteria. For example, each element of a layer may have data for each element. As a non-limiting example, in the measurement instrument layer, elements of health data, algorithm data, raw data, and / or measurement method data may all have corresponding data.

[0160] In some instances, the acceptability criteria may include a threshold value, such as, but not limited to, 70%, 80%, 90%, and the like. In some instances, the acceptability criteria may include two or more acceptability criteria. For example, for a definition layer, the acceptability criteria may include the recency of the data and the amount of data relative to the conditional elements of the definition layer. In some instances, the acceptability criteria may be specific to one or more elements of one or more layers of the measurement stack. For example, in a verification layer, the acceptability criteria may include the recency of the data for one or more elements of the verification layer, while in a measurement instrument layer, the acceptability criteria may include the amount of data for one or more elements of the measurement instrument layer. In some instances, the acceptability criteria may be predefined by one or more user inputs. In other instances, a computing device and / or a processor (such as the processor 304 described above) may be configured to calculate the acceptability criteria based on one or more factors. For example, but not limited to, a computing device and / or a processor may be configured to calculate the acceptability criteria based on one or more factors. For example, but not limited to, a computing device and / or a processor may be configured to calculate the acceptability criteria based on one or more databases (such as the processor 304 described above). Figure 3 The data of the central data structure 312) is used to determine the acceptability criteria to include the amount of data from the wrist-worn heart rate sensor that measures the raw data elements of the instrument layer.

[0161] In some instances, one or more elements of one or more layers of the measurement stack may partially or completely meet the acceptability criteria. In some instances, one or more elements of one or more layers of the measurement stack may not meet the acceptability criteria. Each element of a layer of the measurement stack may meet the acceptability criteria completely, partially, and / or in combination. For example, but not limited to, a first element defining a layer may fully meet the acceptability criteria, while a second element defining a layer may only partially meet the acceptability criteria. The processor and / or computing device may classify the acceptability of one or more elements of one or more layers.

[0162] Classification may include categorizing and / or placing one or more elements into categories, such as, but not limited to, valid data, validation data, construct validity (e.g., the extent to which the solution measures the intended concept), sensitivity to change, etc. With respect to construct validity, for example, but not limited to, when measuring a patient's steps, acceleration and / or other measurements may be taken to determine the number of steps. Continuing with this example, the construct validity of the measure of steps may refer to the degree to which acceleration and / or other measurements reflect the number of steps taken. In some instances, classification may include content validity, such as whether a measure is appropriate for a given concept. With respect to content validity, for example, but not limited to, a concept may be physical activity and the measurement may be steps, and content validity may be the extent to which the measurement of steps reflects the patient's physical activity, as other measurements such as gait speed may be used, which may be appropriate for particular purposes.

[0163] Sensitivity to change refers to, for example, how a measurement can reflect potential changes in a concept. For example, asking a patient about their level of physical activity through a questionnaire may have poor sensitivity to change, because the change would have to be considerable for the patient to assess the change in the questionnaire, while direct measurement may be more sensitive. In some instances, each element of each layer of the measurement stack may be classified into evidence-based categories as "no additional evidence required", "less additional evidence required" and / or "complete additional evidence required". The "no additional evidence required" category may include a grouping of one or more data sets ready for reuse. For example, one or more data sets may meet one or more acceptability criteria, such as the quantity, recency and / or quality of the data. The "no additional evidence required" category may be set by user input and / or determined by one or more computing devices. In some instances, the "less additional evidence required" category may include a group of one or more data sets that partially meet one or more acceptability criteria. For example, one or more data sets of an element may partially meet one or more acceptability criteria, such as, but not limited to, the recency, quantity and / or quality of the data. In some examples, a "needs additional evidence to be complete" category may include a set of one or more data sets for an element that fails to meet one or more acceptability criteria, such as, but not limited to, recency, quantity, and / or quality of the data.

[0164] In some instances, the acceptability of one or more elements may be color-coded. For example, a measurement stack and each of its elements may be displayed by a GUI. A colored circle may be located on the left, right, top, and / or bottom of an element of the measurement stack of the measurement displayed by the GUI. The color of the color may represent the category of the data of the element. For example, a colored circle may correspond to a "no additional evidence required" category, a blue color may correspond to a "less additional evidence required" category, and an orange color may correspond to a "complete additional evidence required" category, but not limited thereto, and the color code may include any color, such as red, blue, green, yellow, and / or a combination thereof. In some instances, a color code may be used for each layer of the measurement stack. For example, a verification layer may be displayed with a green circle, a measurement instrument layer may be displayed with a blue circle, and a definition layer may be displayed with an orange circle, but not limited thereto. In other instances, each element of each layer of the measurement stack may be displayed with a corresponding colored circle. In some instances, one or more annotations of one or more elements may be displayed. Annotations may include, but are not limited to, text, icons, colors, and the like. In some instances, an annotation may include additional data on the acceptability of one or more elements of a layer of the measurement stack. For example, but not limited to, the clinical validation elements of the validation layer can be displayed with annotations, such as text boxes, which can show the various parts of the clinical validation elements that meet one or more acceptability criteria. Continuing with this non-limiting example, the annotations of the clinical validation elements can include a "content validity" text box with a green circle corresponding to the "no additional evidence required" category, a "construct validity" box with an orange circle corresponding to the "complete additional evidence required" category, and / or a "change sensitivity" text box with an orange circle corresponding to the "complete additional evidence required" category. The user can click on one or more elements of one or more layers of the measurement stack, which can cause the GUI to display one or more annotation boxes for one or more elements. Of course, the above content is merely exemplary, and different instances can use different colors, or other patterns, fonts, or other distinguishing visual features, layouts, etc., for different standards or acceptability indications.

[0165] In some instances, one or more layers may be reused and / or extended despite one or more elements of one or more layers failing to meet acceptability criteria. As a non-limiting example, the concepts of the elements of interest and conditional elements of a defined layer may partially or completely meet acceptability criteria, while the meaningful health aspects elements of a defined layer may fail to meet acceptability criteria. Continuing with this example, the reuse of defined layers for various purposes may still be acceptable. In some instances, a computing device and / or processor may determine the total acceptability of one or more layers. The total acceptability may include the sum of two or more elements of a layer. For example, in the example of defining layers described above, the defined layer may have a total acceptability of 70%.

[0166] Reusability and extensibility of flowcharts

[0167] See now Fig. 9 , presenting an exemplary example of a flowchart. Flowchart 900 may be implemented in any process, system and / or method as described throughout the present disclosure. A computing device may be configured to perform one or more steps of flowchart 900. In some instances, flowchart 900 may provide a comparison of one or more aspects of a DMS with one or more standards. Standards may include, but are not limited to, data age, data volume, data quality, algorithm similarity, measurement method similarity, condition similarity, etc., but are not limited thereto. Standards may be received by user input and / or by one or more external computing devices. Various aspects may include, but are not limited to, regulatory intelligence and / or acceptance of a DMS, clinical validation, technical verification / analysis validation, health data, algorithms, raw data, measurement methods, concepts of interest, meaningful health aspects, illnesses, etc. Flowchart 900 may allow determination of the reusability of one or more aspects of a DMS, the extension of one or more aspects of a DMS, etc. Reusability may include one or more parts of one or more layers of a DMS that may be used for different groups, illnesses, measurement methods, algorithms, etc. Extensions of one or more aspects of a DMS may include additional data that may be required by different patient groups, illnesses, measurement methods, etc.

[0168] In some instances, the flowchart 900 may be automatically executed by one or more computing devices. For example, the computing device may be configured to compare one or more aspects of the DMS and / or TSP with one or more criteria of the flowchart 900, and generate an output of the reusability, expansion, or other functions of one aspect of the DMS and / or TSP. In some instances, the computing device may utilize a machine learning model to generate the flowchart 900 and / or other comparison processes. In the flowchart 900, the machine learning model may determine which path to take. The machine learning model may be trained with training data that relates one or more aspects of the DMS to one or more criteria of the reusability, expansion, and / or other functions of one or more aspects of the DMS. Training data may be received through user input, external computing devices, and / or previous processing iterations. The machine learning model may be configured to input one or more verification, measuring instruments, and / or definition layer data, and output one or more determinations of the reusability, expansion, and / or other functions of one or more layers. Artificial intelligence, machine learning models, or other machine processes may be implemented by computing devices. In some instances, artificial intelligence and / or machine learning models may be configured to identify one or more elements that are acceptable for reuse for different interests. In an instance, an artificial intelligence and / or machine learning model may be configured to identify one or more matches between ongoing tasks and new interests / tasks of the DMS. For example, a clustering algorithm may be used to identify potential matches between ongoing tasks and new interests and / or tasks. The computing device may be configured to update standards for one or more verifications, measuring instruments, and / or definition layers for one or more DMSs. In some instances, the computing device may be configured to update the standards by utilizing a machine learning model. The standards or other determination processes of flowchart 900 may be specific to one or more aspects of the DMS, such as in a definition layer of the DMS. Aspects of the definition layer may include aspects of health, illness, populations, and the like.

[0169] At step 905, changes to elements of a standardized digital solution (e.g., DMS layer) are made. Changes may include any aspect of any layer of a standardized digital solution (e.g., TSP or DMS), such as, but not limited to, regulatory intelligence and / or acceptance of the TSP or DMS, clinical validation, technical verification / analytic validation, health data, algorithms, raw data, measurement methods, concepts of interest, meaningful health aspects, conditions, etc. For example, changes to a TSP or DMS layer may include changes to a disease or condition to be treated.

[0170] At step 910, the changes in the TSP or DMS layer are compared to previously used data. The previously used data may include evidence of one or more aspects of one or more DMS layers. In some instances, the computing device may utilize a loss function to determine a threshold difference between the changes in the DMS layer and the previously used evidence / data. In some instances, the threshold difference may include, but is not limited to, about 1% to about 100%. For example, a threshold difference of about 15% may be used to determine whether the previously used data is sufficient to cover the changes in the DMS layer.

[0171] As an example, assume a previously approved DMS. In various instances, changes to a previously approved DMS can be changes to a meaningful health aspect element (e.g., a new meaningful health aspect is introduced). Therefore, additional elements of the DMS (e.g., concepts of interest, measurement methods, raw data, algorithms, health data, technical verification / analysis verification, clinical verification and / or regulatory intelligence / acceptance) undergo evaluation to determine which elements can be reused and which elements will undergo full revalidation or partial revalidation. As an example, with reference to a measurement method element, the introduction of a new meaningful health aspect triggers an evaluation of the measurement method element to determine whether the measurement method has captured relevant data for the new meaningful health aspect. If the measurement method successfully captures relevant data about the new meaningful health aspect, the measurement method element can be reused without further evaluation (e.g., step 915). In a scenario, if the measurement method successfully captures relevant data about the new meaningful health aspect, the measurement method only undergoes partial verification without undergoing full revalidation. If the measurement method fails to capture relevant data for the new meaningful health aspect, the measurement method element undergoes full revalidation (e.g., step 920) by generating new evidence.

[0172] As another example, with reference to the technical verification / analysis validation element, the introduction of new meaningful aspects of health triggers the evaluation of the technical verification / analysis validation element. As described herein, in various instances, the analysis validation element may perform analytical verification of device specifications, algorithms, and health data outputs, and ensure that the data meets the necessary sensitivity, specificity, and / or reliability requirements. Therefore, the evaluation of the technical verification / analysis validation element may include determining whether the device specifications, algorithms, and health data outputs remain within the required sensitivity, specificity, and / or reliability requirements. In some instances, if it is determined that the required sensitivity, specificity, and / or reliability requirements are not met, the technical verification / analysis validation element will undergo a complete revalidation (e.g., step 920). In some instances, if it is determined that the required sensitivity, specificity, and / or reliability requirements are met, the technical verification / analysis validation element may undergo partial revalidation or may be automatically reused (e.g., step 915).

[0173] At step 915, if there is sufficient evidence to support the change in the DMS layer with previous data, the previous data may be reused for the change in the DMS layer. The reused data may include, but is not limited to, algorithms, sensors, measurements, concepts of interest, conditions, analytical validations, etc.

[0174] If there is insufficient evidence to support the change in the DMS layer, new evidence may be generated at step 920. The generation of new evidence may result in an extension of one or more layers, stacks, etc. of the DMS. The generation of new evidence may allow future changes to one or more DMS layers to reuse data generated as part of the extension of the DMS layer.

[0175] Diseases and conditions

[0176] Disclosed herein are TSPs and DMSs constructed and implemented for specific diseases or conditions. In various examples, the disease can be, for example, cancer, inflammatory diseases, neurodegenerative diseases, nervous system diseases, autoimmune diseases, neuromuscular diseases, metabolic disorders (e.g., diabetes), cardiac diseases, or fibrotic diseases.

[0177] In various examples, the cancer can be bronchioloalveolar carcinoma (BAC) of the lung, bladder cancer, female genital tract malignancies (e.g., uterine serous carcinoma, endometrial carcinoma, vulvar squamous cell carcinoma, and uterine sarcoma), ovarian surface epithelial cancer (e.g., ovarian clear cell carcinoma, epithelial ovarian carcinoma, fallopian tube carcinoma, and primary peritoneal carcinoma), breast cancer, non-small cell lung cancer (NSCLC), male genital tract malignancies (e.g., testicular cancer), retroperitoneal or peritoneal cancer, gastroesophageal adenocarcinoma, esophagogastric junction cancer, hepatocellular carcinoma, esophageal and esophagogastric junction cancer. In some embodiments, the cancer is any of: breast cancer, non-small cell lung cancer, bladder cancer, kidney cancer, colon cancer, and melanoma.

[0178] In various examples, the inflammatory disease can be any of the following: acute respiratory distress syndrome (ARDS), acute lung injury (ALI), alcoholic liver disease, allergic inflammation of the skin, lungs and gastrointestinal tract, allergic rhinitis, ankylosing spondylitis, asthma (allergic and non-allergic), atopic dermatitis (also known as atopic eczema), atherosclerosis, celiac disease, chronic obstructive pulmonary disease (COPD), pulmonary arterial hypertension (PAH), chronic respiratory distress syndrome (CRDS), colitis, dermatitis, diabetes, eczema, endocarditis, fatty liver disease, fibrosis (e.g., idiopathic pulmonary fibrosis, scleroderma, renal fibrosis and scarring), food allergies (e.g., allergies to peanuts, eggs, dairy, shellfish, tree nuts, etc.), gastritis, gout , liver steatosis, hepatitis, inflammation of body organs (including joint inflammation, including joints of the knees, limbs or hands), inflammatory bowel disease (IBD) (including Crohn's disease or ulcerative colitis), intestinal hyperplasia, irritable bowel syndrome, juvenile rheumatoid arthritis, liver disease, metabolic syndrome, multiple sclerosis, myasthenia gravis, neurogenic pulmonary edema, nephritis (e.g., glomerulonephritis), non-alcoholic fatty liver disease (NAFLD) (including non-alcoholic steatosis and non-alcoholic steatohepatitis (NASH)), obesity, prostatitis, psoriasis, psoriatic arthritis, rheumatoid arthritis (RA), sarcoidosis, sinusitis, splenitis, seasonal allergies, sepsis, systemic lupus erythematosus, uveitis and UV-induced skin inflammation.

[0179] In various examples, the neurodegenerative disease can be any of Alzheimer's disease, Parkinson's disease, traumatic CNS injury, Down syndrome (DS), glaucoma, amyotrophic lateral sclerosis (ALS), frontotemporal dementia (FTD), and Huntington's disease. In addition, neurodegenerative diseases may also include: septum pellucidum absence, acid lipase disease, acid maltase deficiency, acquired epileptic aphasia, acute disseminated encephalomyelitis, ADHD, Addison's pupil, Addison's syndrome, adrenoleukodystrophy, corpus callosum agenesis, agnosia, Icardi syndrome, AIDS, Alexander disease, Alpers disease, alternating hemiplegia, congenital anencephaly, aneurysm, Angelman syndrome, angiomatosis, hypoxia, antiphospholipid syndrome, aphasia, apraxia, arachnoid cyst, arachnoiditis, Arnold-Chiari malformation, arteriovenous malformation, Asperger's syndrome, ataxia, ataxia telangiectasia , ataxia and cerebellar or spinocerebellar degeneration, autism, autonomic dysfunction, Barth syndrome, Batten's disease, Becker's myotonia, Behcet's disease, Bell's palsy, benign essential blepharospasm, benign focal muscular atrophy, benign intracranial hypertension, Bernard-Rodriguez syndrome, Binswanger's disease, blepharospasm, Bloch-Sulzberger syndrome, brachial plexus injury, Bradbury-Egleston syndrome, brain or spinal tumor, cerebral aneurysm, brain injury, Brown-Segar syndrome, bulbospinal muscular atrophy, leukoencephalopathy, Canavan disease, causalgia, cavernous hemangioma, cavernous hemangioma, central cord syndrome, central pain syndrome, pons Central myelinolysis, head disease, ceramidase deficiency, cerebellar degeneration, cerebellar dysplasia, brain aneurysm, cerebral arteriosclerosis, brain atrophy, cerebral beriberi, cerebral gigantism, brain tissue hypoxia, cerebral palsy, cerebro-oculofacial-skeletal syndrome, Charcot-Marie-Toxoplasmosis, cerebellar tonsillar herniation, chorea, chronic inflammatory demyelinating polyneuropathy (CIDP), Coffin-Lowry syndrome, syrinx, congenital facial palsy, congenital myasthenia, congenital myopathy, cortical basal degeneration, cranial arteritis, craniosynostosis, Cruz-Freder-Jakob disease, cumulative injury disorder, Cushing syndrome, giant cell inclusion disease, eye-foot chorea syndrome, Dandy-Walker syndrome, Dawson disease, dementia disease, dementia with Lewy bodies, dentate-cerebellar ataxia, dentate-rubral atrophy, dermatomyositis, developmental dyspraxia, Devic's syndrome, diabetic neuropathy, diffuse sclerosis, Dravet's syndrome, autonomic dysfunction, dysgraphia, dyslexia, dysphagia, myoclonic cerebellar coordination disorder, dystonia, early epileptic encephalopathy in infancy, empty sella syndrome, encephalitis, epidemic encephalitis, encephalocele, encephalopathy, cerebral trigeminal angiomatosis, epilepsy, Erb-Duchenne's palsy and Degerin-Klupke's palsy, Erb's palsy, essential tremor, extrapontine myelinolysis, Fabry disease, Fahr's syndrome, syncope, familial autonomic dysfunction, familial angiomatosis,Familial periodic paralysis, familial spastic paralysis, Farber's disease, febrile seizures, fibromuscular dysplasia, Fisher's syndrome, infantile hypotonia syndrome, gait abnormalities, Friedreich's ataxia, frontotemporal dementia, gangliosidosis, Gaucher's disease, Gerstmann's syndrome, Gerstmann-Straussler-Schenck syndrome, giant cell arteritis, giant cell inclusion disease, globoid cell leukodystrophy, glossopharyngeal neuralgia, glycogen storage disease, Guillain-Barré syndrome, Hallervorden-Spatz disease, head injury, continuous migraine, hemifacial spasm, crossed hemiplegia, hereditary neuropathy, hereditary spastic paraplegia, hereditary movement disorder of the polyneuritis type, herpes zoster, herpes zoster oticus, Hirayama syndrome, Holmes-Ellison disease Dee syndrome, holoprosencephaly, HTLV-1-associated myelopathy, Hughes syndrome, Huntington's disease, hydrocephalus, hydrocephalus, hydromyelia, hyperrhythmia syndrome, narcolepsy, hypertonia, hypotonia, hypoxia, immune-mediated encephalomyelitis, inclusion body myositis, incontinence pigmenti, infantile hypotonia, neuroaxonal dystrophy of infants, phytanic acid storage disease of infants, infantile gastritis, infantile spasms, inflammatory myopathy, schizo-occipital exencephaly, enterogenic lipodystrophy, intracranial cysts, intracranial hypertension, Isaacs syndrome, Joubert syndrome, Kearns-Sayre syndrome, Kennedy disease, Kingsborne syndrome, Kleine-Levin syndrome, Klippel-Fehr syndrome, Klippel-Tenona syndrome (KTS) , Kroer-Bucy syndrome, Korsakoff amnestic syndrome, Krabbe disease, Kugelberg-Weiland disease, Kuru disease, Lambert-Eaton myasthenic syndrome, Landau-Klevner syndrome, dorsolateral medullary syndrome, learning disabilities, Ley's disease, Lennox-Gastaut syndrome, Lesch-Nyhan syndrome, leukodystrophy, Levine-Critchley syndrome, dementia with Lewy bodies, lipid storage disease, lipid protein deposition disease, lissencephaly, locked-in syndrome, Lou Gehrig's disease, lupus, Lyme disease, Machado-Joseph disease, megalencephaly, Melksson-Rosenthal syndrome, meningitis, Menkes disease, meralgia paresthesiae, metachromatic leukodystrophy, microcephaly, migraine, Miller Fisher syndrome, mini-stroke, mitochondrial muscle disease, motor neuron disease, moyamoya disease, mucolipidosis, mucopolysaccharidosis, multiple sclerosis (MS), multiple system atrophy, muscular dystrophy, myasthenia gravis, myoclonus, myopathy, myotonia, narcolepsy, neuroacanthocytosis, neurodegeneration with brain iron accumulation, neurofibromatosis, neuroleptic malignant syndrome, neurosarcoidosis, neurotoxicity, cavernous nevus, Niemann-Pick disease, non-24-hour sleep-wake disorder, normal pressure hydrocephalus, occipital neuralgia, latent spinal dysplasia sequence, Ohtawara syndrome, olivopontocerebellar atrophy, strabismus, myoclonus, orthostatic hypotension, O'Sullivan-McLeod syndrome, overuse syndrome, pantothenate kinase-related neurodegeneration, carcinoid syndrome, paresthesia,Parkinson's disease, paroxysmal choreoathetosis, paroxysmal migraine, Parry-Romberg syndrome, Pellizois-Merzbacher disease, perineural cysts, periodic paralysis, peripheral neuropathy, periventricular softening, combined mental development disorder, nerve compression, piriformis syndrome, plexopathy, polymyositis, Pompe disease, brain perforation malformation, postherpetic neuralgia, post-infectious encephalomyelitis, post-poliomyelitis, postural hypotension, postural tachycardia syndrome (POTS), primary lateral sclerosis, prion disease, progressive multifocal leukoencephalopathy, progressive sclerosis of the gray matter Malnutrition, gray matter dystrophy, progressive supranuclear palsy, prosopagnosia, pseudotumor cerebri, Ramsay Hunt syndrome I, Ramsay Hunt syndrome II, Rasmussen encephalitis, reflex sympathetic dystrophy syndrome, Refsum disease, Refsum disease, repetitive movement disorder, repetitive stress injury, restless legs syndrome, retrovirus-associated myelopathy, Rett syndrome, Reye's syndrome, rheumatic encephalitis, Riley-Day syndrome, St. Vitus chorea, Sandhoff disease, schizencephaly, septo-optic dysplasia, herpes zoster, Shy-Dreyfus syndrome, Sjogren syndrome, sleep apnea Apnea, sleeping sickness, Sotos syndrome, spasticity, spinal cord infarction, spinal cord injury, spinal cord tumor, spinocerebellar atrophy, spinocerebellar degeneration, stiff-person syndrome, striatonigral degeneration, stroke, Sturge-Weber syndrome, transient unilateral neuralgiform headache with conjunctival injection and tearing (SUNCT) headache, syncope, syphilitic myelosclerosis, syringomyelia, tabes dorsalis, tardive dyskinesia, Tarlov cyst, Tay-Sachs disease, temporal arteritis, tethered cord syndrome, Thomson's myotonia, thoracic outlet syndrome, thyrotoxic myopathy, tinnitus, Todd's palsy, Tourette syndrome , transient ischemic attack, transmissible spongiform encephalopathy, transverse myelitis, traumatic brain injury, tremor, trigeminal neuralgia, tropical spastic palsy, Troyer syndrome, tuberous sclerosis, vasculitis (including temporal arteritis), von Economou's disease, von Hippel-Lindau disease (VHL), von Recklinghausen's disease, Wallenberg's syndrome, Werdnig-Hoffmann disease, Wernicke-Korsakoff syndrome, Wester's syndrome, whiplash, Whipple's disease, Williams syndrome, Wilson's disease, Wolman's disease, X-linked spinal and bulbar muscular atrophy, and Zellweger syndrome.

[0180] In various examples, the autoimmune disease or disorder can be any of the following: arthritis, including rheumatoid arthritis, acute arthritis, chronic rheumatoid arthritis, gout or gouty arthritis, acute gouty arthritis, acute immune arthritis, chronic inflammatory arthritis, degenerative arthritis, type II collagen-induced arthritis, infectious arthritis, Lyme arthritis, proliferative arthritis, psoriatic arthritis, Still's disease, spondyloarthritis, juvenile-onset rheumatoid arthritis, osteoarthritis, deforming arthritis, chronic primary polyarthritis, reactive arthritis, and ankylosing spondylitis; inflammatory hyperproliferative skin diseases; psoriasis, such as plaque psoriasis, pustular psoriasis, and nail psoriasis atopic diseases, including atopic diseases such as hay fever and Job's syndrome; dermatitis, including contact dermatitis, chronic contact dermatitis, exfoliative dermatitis, allergic dermatitis, allergic contact dermatitis, dermatitis herpetiformis, nummular dermatitis, seborrheic dermatitis, nonspecific dermatitis, primary irritant contact dermatitis, and atopic dermatitis; X-linked hyper-IgM syndrome; allergic intraocular inflammatory diseases; urticaria, such as chronic allergic urticaria, chronic idiopathic urticaria, and chronic autoimmune urticaria; myositis; polymyositis / dermatomyositis; juvenile dermatomyositis; toxic epidermal necrolysis; scleroderma, including systemic sclerosis; sclerosis, such as systemic sclerosis, multiple sclerosis (MS), spinal-optic MS , primary progressive MS (PPMS), relapsing-remitting MS (RRMS), progressive systemic sclerosis, atherosclerosis, arteriosclerosis, disseminated sclerosis, and ataxia; neuromyelitis optica (NMO); inflammatory bowel disease (IBD), including Crohn's disease, autoimmune-mediated gastrointestinal diseases, colitis, ulcerative colitis, microscopic colitis, collagenous colitis, polypoidal colitis, necrotizing enterocolitis, transmural colitis, and autoimmune inflammatory bowel disease; intestinal inflammation; pyoderma gangrenosum; erythema nodosum; primary sclerosing cholangitis; respiratory distress syndrome, including acute respiratory distress syndrome (ARDS) in adults; meningitis; all or part of the uveal tract Inflammation; iritis; choroiditis; autoimmune blood disease; rheumatoid spondylitis; rheumatoid synovitis; hereditary angioedema; cranial nerve damage, such as in meningitis; herpes gestationis; pemphigoid gestationis; scrotal pruritus; autoimmune premature ovarian failure; sudden hearing loss due to autoimmune disorders; IgE-mediated diseases, such as allergies, and allergic and atopic rhinitis; encephalitis, such as Rasmussen's encephalitis and limbic and / or brainstem encephalitis; uveitis, such as anterior uveitis, acute anterior uveitis, granulomatous uveitis, non-granulomatous uveitis, lens antigenic uveitis, posterior uveitis, or autoimmune uveitis;Glomerulonephritis (GN) with and without nephrotic syndrome, such as chronic or acute glomerulonephritis, primary GN, immune-mediated GN, membranous GN (membranous nephropathy), idiopathic membranous GN or idiopathic membranous nephropathy, membranous or membranous proliferative GN (MPGN), including types I and II, and rapidly progressive GN; proliferative nephritis; autoimmune polyglandular endocrine failure; balanitis, including plasma cell localized balanitis; balanoposthitis; erythema annulare centrifugal; erythema dyschromicum perseverant; erythema multiforme; granuloma annulare; lichen rosacea; lichen sclerosus atrophicus; simple lichen chronicus; lichen spinulosus; lichen planus; lamellar ichthyosis; epidermolytic hyperkeratosis; precancerous keratosis; pyoderma gangrenosum; allergic conditions and responses; allergic reactions; eczema, including allergic or atopic eczema, sebaceous eczema, dyshidrotic eczema, and vesicular palmoplantar eczema; asthma, such as bronchial asthma and autoimmune asthma; conditions involving T cell infiltration and chronic inflammatory responses; immune responses to foreign antigens (such as fetal ABO blood type) during pregnancy; chronic inflammatory lung diseases; autoimmune myocarditis; leukocyte adhesion defects; lupus, including lupus nephritis, lupus encephalitis, pediatric lupus, nonrenal lupus, extrarenal lupus, discoid lupus and discoid lupus erythematosus, alopecia lupus, systemic lupus erythematosus (SLE), cutaneous SLE, subacute cutaneous SLE, neonatal lupus syndrome (NLE), and disseminated lupus erythematosus; juvenile Type I diabetes, including insulin-dependent diabetes mellitus in children (IDDM), adult-onset diabetes mellitus (Type II diabetes), autoimmune diabetes, idiopathic diabetes insipidus, diabetic retinopathy, diabetic nephropathy, and diabetic aortic disease; immune responses associated with acute and delayed hypersensitivity reactions mediated by cytokines and T lymphocytes; tuberculosis; sarcoidosis; granulomatous diseases, including lymphomatoid granulomatosis; Wegener's granulomatosis; agranulocytosis; vasculitis, including vasculitis, large vessel vasculitis, polymyalgia rheumatica, and giant cell (Takayasu's) arteritis, medium vessel vasculitis, Kawasaki disease, polyarteritis nodosa / periarteritis nodosa, microscopic polyarteritis, immune vasculitis, CNS blood vasculitis, cutaneous vasculitis, hypersensitivity vasculitis, necrotizing vasculitis, systemic necrotizing vasculitis, ANCA-associated vasculitis, Zag-Strauss vasculitis or syndrome (CSS), and ANCA-associated small vessel vasculitis; temporal arteritis; aplastic anemia; autoimmune aplastic anemia; Coombs-positive anemia; Diamond-Blackfein anemia; hemolytic anemia or immune hemolytic anemia, including autoimmune hemolytic anemia (AIHA), pernicious anemia; Addison's disease; pure red cell anemia or aplasia (PRCA); factor VIII deficiency; hemophilia A; autoimmune neutropenia; pancytopenia; leukopenia; diseases involving leukocytopenia; CNS inflammatory diseases;Multiple organ injury syndromes, such as those secondary to sepsis, trauma, or hemorrhage; antigen-antibody complex-mediated diseases; anti-glomerular basement membrane disease; antiphospholipid antibody syndrome; allergic neuritis; Behcet's disease / syndrome; Castleman's syndrome; Goodpasture's syndrome; Raynaud's syndrome; Sjögren's syndrome; Stevens-Johnson syndrome; pemphigoid, such as bullous pemphigoid and cutaneous pemphigoid, pemphigus, pemphigus vulgaris, pemphigus foliaceus, mucous membrane pemphigoid, and pemphigus erythematosus; autoimmune polyendocrine disease; Reiter's disease or syndrome; thermal injury; preeclampsia; immune complex disorders, such as immune complex nephritis and antibody-mediated nephritis; multiple Neuropathies; chronic neuropathies, such as IgM polyneuropathy and IgM-mediated neuropathy; thrombocytopenia (e.g., developing in patients with myocardial infarction), including thrombotic thrombocytopenic purpura (TTP), post-transfusion purpura (PTP), heparin-induced thrombocytopenia, autoimmune or immune-mediated thrombocytopenia, idiopathic thrombocytopenic purpura (ITP), and chronic or acute ITP; scleritis, such as idiopathic kerato-scleritis, and episcleritis; autoimmune diseases of the testicles and ovaries, including autoimmune orchitis and oophoritis; primary hypothyroidism; hypoparathyroidism; autoimmune endocrine diseases, including thyroiditis, autoimmune thyroiditis, bridging Ben's disease, chronic thyroiditis (Hashimoto's thyroiditis) or subacute thyroiditis, autoimmune thyroid disease, idiopathic hypothyroidism, Graves' disease, polyglandular syndrome, autoimmune polyglandular syndrome, and polyglandular endocrine syndrome; paraneoplastic syndromes, including neurological paraneoplastic syndromes; Lambert-Eaton myasthenic syndrome or Eaton-Lambert syndrome; stiff-man syndrome; encephalomyelitis, such as allergic encephalomyelitis and experimental allergic encephalomyelitis (EAE); myasthenia gravis, such as thymoma-associated myasthenia gravis; cerebellar degeneration; neuromyotonia; opsoclonus or opsoclonus-myoclonus syndrome (OMS); sensory neuropathy; multifocal motor neuropathy; Sheehan's syndrome; hepatitis, including including autoimmune hepatitis, chronic hepatitis, lupus hepatitis, giant cell hepatitis, chronic active hepatitis and autoimmune chronic active hepatitis; lymphocytic interstitial pneumonia (LIP); bronchiolitis obliterans (non-transplant) to NSIP; Guillain-Barre syndrome; Buerger's disease (IgA nephropathy); idiopathic IgA nephropathy; linear IgA dermatosis; acute febrile neutrophilic dermatosis; subcorneal pustular dermatosis; transient acantholytic dermatosis; cirrhosis, such as primary biliary cirrhosis and pulmonary cirrhosis; autoimmune enteropathy syndrome; celiac disease or celiac disease; sprue (gluten enteropathy); refractory steatorrhea; idiopathic steatorrhea; cryoglobulinemia; amyotrophic lateral sclerosis (ALS; Lou Gehrig's disease); coronary artery disease;Autoimmune ear diseases, such as autoimmune inner ear disease (AIED); autoimmune hearing loss; polychondritis, such as refractory or relapsing or relapsing polychondritis; pulmonary alveolar proteinosis; Cochrane syndrome / nonsyphilitic interstitial keratitis; Bell's palsy; Sweet disease / syndrome; autoimmune rosacea; herpes zoster-related pain; amyloidosis; noncancerous lymphocytosis; primary lymphocytosis, including monoclonal B-cell lymphocytosis (e.g., benign monoclonal gammopathy and monoclonal gammopathy of undetermined significance, MGUS); peripheral neuropathies; ion channelopathies, such as epilepsy, migraine, cardiac arrhythmias, muscle disorders, deafness, blindness, periodic paralysis, and ion channelopathies of the CNS disease; autism; inflammatory myopathies; focal or segmental glomerulosclerosis (FSGS); endocrine eye diseases; uveoretinitis; chorioretinitis; autoimmune liver dysfunction; fibromyalgia; multiple endocrine failure; Schmidt's syndrome; adrenalitis; gastric atrophy; Alzheimer's disease; demyelinating diseases, such as autoimmune demyelinating diseases and chronic inflammatory demyelinating polyneuropathy; Dressler syndrome; alopecia areata; alopecia totalis; CREST syndrome (calcification, Raynaud's phenomenon, esophageal dysmotility, sclerodactyly, and telangiectasia); male and female autoimmune infertility (e.g., due to antisperm antibodies); mixed connective tissue disease; Chagas' disease; rheumatic fever; recurrent abortion; farmer's lung; polymorphic erythropoiesis plaques; postcardiotomy syndrome; Cushing's syndrome; bird-feeder's lung; allergic granulomatous vasculitis; benign lymphocytic vasculitis; Alport's syndrome; alveolitis, such as allergic alveolitis and fibrosing alveolitis; interstitial lung disease; transfusion reaction; leprosy; malaria; Sumter's syndrome; Kaplan's syndrome; endocarditis; endomyocardial fibrosis; diffuse pulmonary interstitial fibrosis; interstitial pulmonary fibrosis; pulmonary fibrosis; idiopathic pulmonary fibrosis; cystic fibrosis; endophthalmitis; erythema protuberans; erythroblastosis fetalis; eosinophilic fasciitis; Schulman's syndrome; Felty's syndrome; filariasis; cyclitis, such as chronic cyclitis, metachronic cyclitis, iridocyclitis (acute or chronic), or Fuchs' cyclitis body inflammation; Henoch-Schwann purpura; sepsis; endotoxemia; pancreatitis; thyrotoxicosis; Evans syndrome; autoimmune gonadal failure; Sydenham's chorea; poststreptococcal nephritis; thromboangiitis obliterans; thyrotoxicosis; tabes dorsalis; choroiditis; giant cell polymyalgia; chronic hypersensitivity pneumonitis; keratoconjunctivitis sicca; epidemic keratoconjunctivitis; idiopathic nephrotic syndrome; minimal change nephropathy; benign familial and ischemia-reperfusion injury; reperfusion of transplanted organs; retinal autoimmunity; joint inflammation; bronchitis; chronic obstructive airway / lung disease; silicosis; aphthous stomatitis; arteriosclerotic disease; aspermiogenese; autoimmune hemolysis; Burke's disease;Cryoglobulinemia; Dupuytren's contracture; intraocular lens hypersensitivity; allergic enteritis; erythema nodosum leprosum; idiopathic facial palsy; febrile myalgia; Hammann-Ritchie disease; sensorineural hearing loss; paroxysmal hemoglobinuria; hypogonadism; regional ileitis; leukopenia; infectious mononucleosis; transverse myelitis; primary idiopathic myxedema; noninflammatory nephropathy; sympathetic ophthalmia; granulomatous orchitis; pancreatitis; acute polyradiculitis; pyoderma gangrenosum; Quifan's thyroiditis; acquired splenic atrophy; nonmalignant thymoma; vitiligo; toxic shock syndrome; food poisoning; disorders involving T-cell infiltration; leukocyte adhesion defects; and acute and delayed hypersensitivity mediated by cytokines and T lymphocytes immune response associated with the reaction; diseases involving leukocytic infiltration; multiple organ injury syndrome; antigen-antibody complex-mediated diseases; anti-glomerular basement membrane disease; allergic neuritis; autoimmune polyendocrine disease; oophoritis; primary myxedema; autoimmune atrophic gastritis; sympathetic ophthalmia; rheumatic diseases; mixed connective tissue disease; nephrotic syndrome; insulitis; polyendocrine failure; autoimmune polyglandular syndrome type I; adult-onset idiopathic hypoparathyroidism (AOIH); cardiomyopathy, such as dilated cardiomyopathy; acquired epidermolysis bullosa (EBA); hemochromatosis; myocarditis; nephrotic syndrome; primary sclerosing cholangitis; suppurative or non-suppurative sinusitis; acute or chronic sinusitis eosinophilic disease, such as eosinophilia, pulmonary infiltrative eosinophilia, eosinophilia-myalgia syndrome, Loeffler's syndrome, chronic eosinophilic pneumonia, tropical pulmonary eosinophilia, bronchopneumonia aspergillosis, aspergilloma, or granulomas containing eosinophils; allergic reactions; seronegative spondyloarthritis; polyendocrine autoimmune diseases; sclerosing cholangitis; chronic mucocutaneous candidiasis; Bruton's syndrome; transient hypogammaglobulinemia of infancy; Wiskott-Aldrich syndrome; ataxia-telangiectasia syndrome; vasodilation; and diseases related to collagen diseases, rheumatism, nervous system diseases, lymphatic Autoimmune diseases associated with inflammation, hypotensive response, vascular dysfunction, tissue damage, cardiovascular ischemia, hyperalgesia, renal ischemia, cerebral ischemia and diseases associated with angiogenesis; allergic hypersensitivity diseases; glomerulonephritis; reperfusion injury; ischemic reperfusion disorders; reperfusion injury of myocardial tissue or other tissues; lymphomatous tracheobronchitis; inflammatory skin diseases; skin diseases with acute inflammatory components; multiple organ failure; bullous diseases; renal cortical necrosis; acute purulent meningitis or other inflammatory diseases of the central nervous system; inflammatory diseases of the eye and orbit; granulocyte transfusion-related syndrome; cytokine-induced toxicity; narcolepsy; acute severe inflammation; chronic refractory inflammation; pyelitis; intraarterial hyperplasia; peptic ulcer;Valvulitis; and endometriosis. In a specific example, the subject's autoimmune disease may include one or more of the following: systemic lupus erythematosus (SLE), lupus nephritis, chronic graft-versus-host disease (cGVHD), rheumatoid arthritis (RA), Sjögren's syndrome, vitiligo, inflammatory bowel disease, and Crohn's disease. In a specific example, the autoimmune disease is systemic lupus erythematosus (SLE). In a specific example, the autoimmune disease is rheumatoid arthritis. ;

[0181] Exemplary metabolic disorders include, e.g., diabetes, insulin resistance, lysosomal storage diseases (e.g., Gaucher's disease, Krabbe's disease, Niemann-Pick disease types A and B, multiple sclerosis, Fabry's disease, Tay-Sachs disease, and Sandhoff disease variants A and B), obesity, cardiovascular disease, and dyslipidemia. Other exemplary metabolic diseases include, for example, 17-alpha-hydroxylase deficiency, 17-beta hydroxysteroid dehydrogenase 3 deficiency, 18-hydroxylase deficiency, 2-hydroxyglutaric aciduria, 2-methylbutyryl-CoA dehydrogenase deficiency, 3-alpha hydroxyacyl-CoA dehydrogenase deficiency, 3-hydroxyisobutyric aciduria, 3-methylcrotonyl-CoA carboxylase deficiency, 3-methylglutaryl-CoA hydratase deficiency (AUH deficiency), 5-hydroxyprolinase deficiency, 6-pyruvyltetrahydropterin synthase deficiency, abdominal obesity metabolic syndrome, abetalipoproteinemia, acatalase, ceruloplasmin deficiency, acetyl-CoA acetyltransferase 2 deficiency, acetylcarnitine deficiency acrodermatitis enteropathica, adenine phosphoribosyltransferase deficiency, adenosine deaminase deficiency, adenosine monophosphate deaminase 1 deficiency, adenylate succinate deficiency, adrenomedullary neuropathy, adult polyglucose body disease, albino-deafness syndrome, alkaptonuria, Alpers syndrome, alpha-1 antitrypsin deficiency, alpha-ketoglutarate dehydrogenase deficiency, alpha-mannosidosis, aminoacylase 1 deficiency, sideroblastic anemia and spinocerebellar ataxia, arginase deficiency, argininosuccinic aciduria, aromatic L-amino acid decarboxylase deficiency, joint contracture-renal dysfunction-cholestasis syndrome, Arts syndrome, aspartate Acylglucosaminuria, atypical Gaucher disease due to saposhin C deficiency, autoimmune polyglandular syndrome type 2, autosomal dominant optic atrophy and cataract, autosomal erythropoietic protoporphyria, autosomal recessive spastic ataxia type 4, Barth syndrome, Bartter syndrome, Bartter syndrome antenatal type 1, Bartter syndrome antenatal type 2, Bartter syndrome type 3, Bartter syndrome type 4, beta-ketothiolase deficiency, biotinidase deficiency, Bjornstad syndrome, carbamoyl phosphate synthetase 1 deficiency, carnitine palmitoyltransferase 1A deficiency, carnitine-acylcarnitine translocase deficiency, carnosinemia, central diabetes insipidus, cerebral folate deficiency, cerebrotendinosis Xanthomatosis, neuronal ceroid lipofuscinosis type 1, Chanarin-Dorfman syndrome, Chediak-Higashi syndrome, childhood hypophosphatase, cholesterol ester deposition disease, chondrocalcinosis, chylomicron retention disease, citrulline transport defect, congenital defect of bile acid synthesis type 2, Crigler-Najjar syndrome, cytochrome C oxidase deficiency, D-2-hydroxyglutaric aciduria, D-bifunctional protein deficiency, D-glyceric acidemia, Danon disease, dicarboxylic acid aminoaciduria, dihydropteridine reductase deficiency, dihydropyrimidinase deficiency, diabetes insipidus, dopamine beta-hydroxylase deficiency, Dowling-Degos disease, erythropoietic uroporphyria associated with myeloid malignancies,Familial chylomicronemia syndrome, familial HDL deficiency, familial hypocalcemic hypercalcemia type 1, familial hypocalcemic hypercalcemia type 2, familial hypocalcemic hypercalcemia type 3, familial LCAT deficiency, familial partial lipodystrophy type 2, Fanconi-Bickel syndrome, Farber disease, fructose-1,6-bisphosphatase deficiency, gamma-cystathionase deficiency, Gaucher disease, Gilbert syndrome, Gitelman syndrome, glucose transporter type 1 deficiency syndrome, glutamine deficiency, congenital glutaric acidemia, glutathione synthetase deficiency, glycine-N-methyltransferase deficiency, glycogen storage disease, hepatic lipase deficiency, hyperhomocysteinemia, Hurler syndrome, hyperglycerolemia, Immerslund-Grasbeck syndrome, iminoglycinuria, neuroaxonal dystrophy of the juveniles, Keane-Sell syndrome, Krabbe disease, lactate dehydrogenase deficiency disease, Leschnehan syndrome, Menkes disease, methionine adenosyltransferase deficiency, mitochondrial complex deficiency, muscle phosphorylase kinase deficiency, neuronal cerebrosidosis, Niemann-Pick disease type A, Niemann-Pick disease type B, Niemann-Pick disease type C1, Niemann-Pick disease type C2, ornithine transcarbamylase deficiency, Pearson syndrome, Perrett syndrome, phosphoribosylpyrophosphate synthetase hyperactivity, primary carnitine deficiency, hyperoxaluria, purine nucleoside phosphorylase deficiency, pyruvate carboxylase deficiency, pyruvate dehydrogenase complex deficiency, pyruvate dehydrogenase phosphatase deficiency, pyruvate kinase deficiency, Refsum disease, diabetes mellitus, Shea syndrome, Senger syndrome, sialidosis, Sjogren-Lasson syndrome, Tay-Sachs disease, cobalamin translocator 1 deficiency, trehalase deficiency, Walker-Warburg syndrome, Wilson disease, Wolfram syndrome, and Wolman disease. ,

[0182] Non-transitory computer readable medium

[0183] A computer-readable medium is also provided herein, comprising computer-executable instructions configured to implement any method described herein. In various instances, the computer-readable medium is a non-transitory computer-readable medium. In some instances, the computer-readable medium is a part of a computer system (e.g., a memory of a computer system). The computer-readable medium may include computer-executable instructions for performing the methods disclosed herein, such as methods for building, maintaining, implementing, and providing standardized solutions (e.g., DMS and TSP).

[0184] Computing equipment

[0185] In some examples, the above methods (including methods for building, maintaining, implementing, and providing standardized solutions (e.g., DMS and TSP)) are performed on a computing device. Examples of computing devices may include personal computers, desktop computers, laptop computers, server computers, computing nodes in a cluster, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, tablet computers, pagers, routers, switches, etc.

[0186] Fig.13 An example computing device 900 is shown for implementing the systems and methods described throughout this disclosure. In some instances, the computing device 1300 includes at least one processor 1302 coupled to a chipset 1304. The chipset 1304 includes a memory controller hub 1320 and an input / output (I / O) controller hub 1322. Memory 1306 and a graphics adapter 1312 are coupled to the memory controller hub 1320, and a display 1318 is coupled to the graphics adapter 1312. Storage devices 1308, input interfaces 1314, and network adapters 1316 are coupled to the I / O controller hub 1322. Other instances of the computing device 1300 have different architectures.

[0187] Storage device 1308 is a non-transitory computer-readable storage medium, such as a hard drive, a compact disk read-only memory (CD-ROM), a DVD, or a solid-state memory device. Memory 1306 stores instructions and data used by processor 1302. Input interface 1314 is a touch screen interface, a mouse, a trackball, or other type of input interface, a keyboard, or some combination thereof, for inputting data into computing device 1300. In some instances, computing device 1300 may be configured to receive input (e.g., commands) from input interface 1314 via gestures from a user. Graphics adapter 1312 displays images and other information on display 1318. As an example, display 1318 may show a directory of standardized solutions (e.g., DMS and / or TSP). Network adapter 1316 couples computing device 1300 to one or more computer networks.

[0188] The computing device 1300 is adapted to execute computer program modules to provide the functionality described herein. As used herein, the term "module" refers to computer program logic for providing a specified functionality. Thus, a module may be implemented in hardware, firmware, and / or software. In one example, a program module is stored on a storage device 1308, loaded into the memory 1306, and executed by the processor 1302.

[0189] The type of computing device 1300 may vary from the examples described herein. For example, computing device 1300 may lack some of the above elements, such as graphics adapter 1312, input interface 1314, and display 1318. In some examples, computing device 1300 may include a processor 1302 for executing instructions stored on memory 1306.

[0190] Example

[0191] Below are examples of specific embodiments for carrying out the present invention. These examples are provided for illustrative purposes only and are not intended to limit the scope of the present invention in any way. Efforts have been made to ensure accuracy with respect to the numbers used, but some experimental errors and deviations should be allowed.

[0192] Example 1: Measuring the acceptability of layers of a stack

[0193] Fig.14 The first example of the acceptability level of a digital measurement solution is depicted. Specifically, the first example is a use case that expands the use environment to new conditions in spinal muscular atrophy or night scratching. The case uses the SV95C clinical evaluation, which is a wearable-derived digital clinical outcome assessment defined by the European Medicines Agency in Duchenne muscular dystrophy (DMD) in night scratching. In other words, here, the digital measurement solution is to apply the SV95C clinical evaluation to new conditions / diseases. The digital measurement solution has a validation layer, a measuring instrument layer, and a definition layer. The validation layer includes regulatory intelligence / acceptance aspects, clinical validation aspects, and technical verification / analysis validation aspects. Each of these aspects is classified as not requiring additional evidence. The clinical validation aspect has three sub-layers: content validity, construct validity, and sensitivity to the change layer. These three sub-layers can be applied to other layers of DMS. The content validity layer is classified as having less required additional evidence. This classification comes from the data in the content validity covered by the concept derivation, but some content validity also needs to be re-conducted in the new study. The construct validity layer is classified as having complete additional evidence requirements. This classification comes from data from a clinical validation layer that is flagged as needing to be re-run, such as for a smaller study group. The sensitivity to change layer is classified as requiring partial additional evidence. This classification comes from data from a clinical validation layer that has one or more sections that sometimes need to be re-run, such as for a small study group.

[0194] The technical verification / analysis validation layer is marked as not requiring additional evidence. For example, the same algorithm can be used in the analysis validation layer. Sometimes, if the algorithm changes, the analysis validation layer can be recalculated. The measuring instrument layer has health data, algorithms, raw data, and measurement method layers. Each of these layers is classified as not requiring additional evidence. However, possible changes can be made to the algorithm of the algorithm layer. The definition layer has interesting concepts, meaningful health aspects, and condition layers. The interesting concepts and meaningful health aspects layers are classified as not requiring additional evidence. The condition layer is classified as requiring complete additional evidence. This can be attributed to the expansion of DMS from muscular dystrophy to night scratching. Complete additional evidence is needed to expand DMS for night scratching, while other layers of DMS can still be applied to night scratching.

[0195] Fig.15 A second example of the acceptability level of a digital measurement solution is depicted. Specifically, the second example is an expansion of the use environment in the case of different groups from adults to children. Here, from adults to children, there may be differences in the measuring instruments and algorithms in analytical validation. As described above, DMS has a validation layer, a measuring instrument layer, and a definition layer. The validation layer has a regulatory intelligence layer, a clinical validation layer, and a technical verification / analytic validation layer. In this example, each of the three layers is classified as requiring less additional evidence. For example, content validity, construct validity, and sensitivity to changes in each layer are all marked as requiring less additional evidence. The measuring instrument layer has a health data, algorithm, raw data, and measurement method layer. The health data, algorithm, and measurement method layers of the measuring instrument layer are classified as requiring less additional evidence. The raw data of the measuring instrument layer is classified as not requiring additional evidence. In the case where the performance requirements and / or the evidence of one or more sensors (such as accelerometers, gyroscopes, etc.) are sufficient, the raw data layer may not require additional evidence. However, algorithms, health data, and measurement methods require some additional evidence to illustrate different patient groups. The definition layer has interesting concepts, meaningful health aspects, and condition layers. The concepts of interest layer and the meaningful health aspects layer were classified as not requiring additional evidence, while the conditional layer was classified as requiring complete additional evidence. The conditional layer was classified as requiring complete additional evidence due to the new groups, such as considering disease severity, age group, population, etc.

[0196] Fig.16A third example of the acceptability level of a digital measurement solution is depicted. Specifically, the example depicts the reusability of evidence for future use. The DMS here is used for night scratching, and is reused and applied to atopic dermatitis. Here, the validation layer has each of the regulatory intelligence, clinical validation, and technical verification layers that are classified as requiring complete additional evidence. This classification may be due to the construct validity, sensitivity to change, and content validity of each layer of the validation layer. Each layer of the measurement instrument layer, such as health data, algorithms, raw data, and measurement method layers, is classified as requiring complete additional evidence. This is because each layer of the measurement instrument layer requires new data due to the content of night scratching in atopic dermatitis. Each layer of the definition layer, such as the concept of interest, meaningful health aspects, and condition layers are classified as not requiring additional evidence because night scratching is a disease within atopic dermatitis.

[0197] See now Fig.17 , presents a fourth example of acceptable levels for digital measurement solutions. Specifically, the example depicts a new measurement modality for an existing use environment, such as moving from a bed sensor to a wrist-worn device. Here, the Regulatory Intelligence, Clinical Validation, and Technical Verification layers of the Validation layer are classified as requiring complete additional evidence. The Health Data, Algorithms, Raw Data, and Measurement Methods layers of the Measurement Instruments layer are also classified as requiring complete additional evidence. Here, many layers of the DMS are classified as requiring some additional evidence because they are used for different purposes rather than a full transition to a new DMS. The Concepts of Interest, Meaningful Health Aspects, and Conditions layers of the Definition layer are classified as not requiring additional evidence.

[0198] Example 2: Targeted Solution Profile and Digital Measurement Solution for Atopic Dermatitis

[0199] The measurement stack is divided into nine separate layers. Each element includes unique information that can stand on its own and provide added value in combination with other elements. These elements have been classified into three sub-stacks (referred to as "resources" elsewhere): the definition sub-stack, the measurement instrument sub-stack, and the verification sub-stack. The definition sub-stack describes the disease, meaningful health aspects, and concepts of interest. Regulatory acceptance of this sub-stack is valuable for conducting research and should be adjusted as early as possible. The measurement instrument sub-stack includes elements for collecting, analyzing, and interpreting data. These layers include both specific solutions and generally described solution categories. Finally, the verification sub-stack describes the verification studies and regulatory approvals that complete these solutions. Although the stack can be built from each element, the process usually starts with an available measurement instrument or medical condition. Atopic dermatitis (AD) is shown here as a traversal of the measurement stack and an example target solution profile.

[0200] Figure 5ADepicted is an example target solution profile for atopic dermatitis. The condition describes a medical condition for a patient population. A meaningful aspect of health (MAH) defines an aspect of the condition for which patients have a desire to (1) not get worse, (2) improve, or (3) prevent. Here, the condition is atopic dermatitis and the meaningful aspect of health is nighttime itch. See Hypothesis Elements (e.g., Figure 5A 1), which describes a meaningful health-related goal. For example, here, the hypothesis is that in a population of adults with moderate to severe AD, nighttime pruritus is reduced due to drug X.

[0201] Element 2 in the measurement stack refers to the Concept of Interest. Here, the Concept of Interest (COI) describes how this specific meaningful aspect of health will be measured. For example, if the goal is to measure nighttime scratching, the Concept of Interest can be any of the total sleep time measure, scratching events per hour, or total scratching events. The COI is actually measured using the Outcome to be Measured (OTM). For example, the Outcome to be measured includes the evaluation of the condition, the specific, measurable characteristics of the MAH described by the COI. Therefore, the Outcome to be measured reveals whether the treatment of the MAH is beneficial. For example, after X weeks, the scratching events decreased. Although in Figure 5A It is not explicitly shown in , but a single condition may have multiple MAHs; a MAH may have multiple COIs; a COI may have multiple outcomes to be measured.

[0202] Element 3 in the measurement stack refers to the measurement method, which is part of the measurement instrument resource. The measurement method includes hardware, software or firmware solutions. Examples are wearable devices, mobile applications and sensors. In addition, a complete software solution can be a measurement method (for example, a voice battery) as long as the method can reliably measure OTM. As a specific example, the measurement method is a watch with a three-axis MEMS accelerometer (10Hz to 100Hz). The device captures acceleration 24 hours a day, 7 days a week for 7 to 45 days (depending on the set frequency).

[0203] Element 4 in the measurement stack refers to raw data. This element represents the raw data set as a result of the measurement method. Generally speaking, raw data does not yet provide meaningful health data. However, raw data is a separate layer because each of these data sets can be utilized by multiple algorithms for different purposes. Here, the raw data layer is introduced as a raw file that provides, for example, 10Hz to 100Hz acceleration. The data is captured in 3D SI units (XYZ gravity) and 28 days of data are collected continuously. In short, this example describes what kind of raw data is captured (acceleration in 3D SI units), its frequency (10Hz to 100Hz), and the amount of data captured (28 days).

[0204] Element 5 in the measurement stack refers to the algorithm. The algorithm represents the method of transforming the raw data into a meaningful health data set. Therefore, the meaningful health data set is easy to analyze and interpret. The algorithm can be integrated into a complete measurement instrument, or individual algorithms can be used to transform (individual) raw data sets. For example, the Philips Respironics RADA algorithm interprets the measurement device data into sleep events and scratching events.

[0205] Element 6 in the measurement stack refers to health data. The health data element describes a health data set generated by the algorithm based on the initial raw data set. The health data set can be evaluated as a separate data set for multiple purposes. Therefore, the health data set is considered an independent resource and is included in the stack as a separate layer. For example, a meaningful health data set provides total sleep time, scratching events per hour, and total number of scratching events.

[0206] Element 7 in the measurement stack refers to technical and analytical validation. Additionally, element 8 in the measurement stack refers to clinical validation. The different validations ensure that the digital measurements are valid and therefore can be identified as meeting the conditions for clinical trial approval. At the sample level, the measurement instrument output is evaluated both in silico and in vitro. Specifically, the technical validation of the digital resource (in vitro) Figure 5A Technical verification (referred to as V1 in this document) ensures that the measurement instrument captures basic analog data accordingly. Technical verification verifies that the captured data results in the generation of appropriate output data. For example, technical verification ensures that the measurement instrument matches the specifications of the device used to capture the data (e.g., battery life, data storage capacity, and available measurement frequency).

[0207] Analytical verification (in Figure 5A Analytical validation (referred to as V2 in Analytical Validation) is the evaluation of a complete measuring instrument, usually in vitro. Analytical validation includes verification of device details, processing algorithms, and reliable health data output. For example, analytical validation verifies the ability to measure, detect, and predict physiological or behavioral indicators according to appropriate measurement standards. Figure 5A The specific example in shows an analytical validation to establish the measurement properties between the measuring instrument and night scratching, including reliability, specificity and sensitivity. Comparison was made with reference measurement data of 8 nights to verify that ICC>85% (N=45). ICC verifies whether the novel digital measuring instrument is better than or at least comparable to the reference measure (e.g., infrared observations for monitoring night scratching).

[0208] Clinical validation (in Figure 5AV3 in the context of use) evaluates whether the solution identifies, measures and predicts meaningful clinical, biological, physical, functional states or experiences in a specific context of use (COU). This is stated through its study design. Clinical validation is in vivo validation, including specific target populations and study-specific numerical endpoints. Clinical validation allows for meaningful interpretation of the results and evaluates whether the solution is effective in answering clinical questions related to the measurement instrument substack and the definition substack. For example, clinical validation evaluates the treatment effect on night scratching, correlation with other measures of the disease (e.g., PROs, biomarkers), and the ability to measure intra-individual changes.

[0209] Element 9 of the measurement stack refers to regulatory validation. Here, regulatory qualification by health authorities is important for the acceptance and adoption of digital measurements. Regulatory validation assesses all layers of the measurement stack but primarily applies to the definition element and the verification element. For example, definitions, measurement instruments, and internal and external regulatory priorities for evidence.

[0210] like Figure 5A As shown, the Target Solution Profile for the adult atopic dermatitis population includes an actigraphy-based nighttime scratching endpoint. TSP represents a generic, solution-agnostic stack. Here, all layers of the measurement instrumentation resources are described generically. The measurement method is a tri-axis wrist-worn accelerometer with a range of specifications. In addition, the raw data layer, algorithm layer, and health data layer are generally described in terms of specific ranges (e.g., data is collected for 14 to 56 days, rather than a specific number of days). In addition, the COI is described generically as nighttime scratching. Many DMSs (such as Figure 5B The DMS shown in ) belongs to the category represented by this TSP.

[0211] Figure 5B An example digital measurement solution (DMS) for atopic dermatitis is depicted. Here, the DMS is populated with an actigraph-based nocturnal scratching endpoint. The meaningful health aspect is nocturnal scratching, and a total of 6 different concepts of interest have been identified (e.g., total sleep time, wake time after sleep onset, sleep efficiency, scratching events per hour, scratching duration per hour, and total number of scratching events). Each of the different concepts of interest can be selected for a specific DMS. Thus, here, 6 different concepts of interest correspond to 6 different DMS for atopic dermatitis.

[0212] Element 3 (measurement method) in DMS identifies the specific equipment used to capture the raw data. Figure 5BAs shown, the GENEActiv Original watch captures raw data. In addition, element 5 (algorithm) in the DMS identifies the specific algorithm used to transform the raw data (element 4) into health data (element 6). Here, the algorithm in the DMS is the Philips Respironics RADA algorithm, which interprets the device data into sleep and scratching events.

[0213] Figure 5C Depicting the resource interchangeability of different digital measurement solutions for atopic dermatitis. Here, the resource interchangeability enables the rapid development of many digital measurement solutions combining different resources. For example, Figure 5C The center measurement stack in Figure 5B . The GENEActiv Original Watch is included in Element 3 (Measurement Method). However, other devices such as Apple Watch (e.g., Apple Watch 6), Fitbit, and ActiGraph (e.g., ActiGraphGT9X Link) may alternatively be included in Element 3 in place of the GENEActiv Watch. These additional DMS also belong to the category represented by TSP. In addition, the Philips Respironics RADA algorithm may be interchangeable with other types of algorithms, such as the Koneska health algorithm or the Tudor-Locke 2014 algorithm. In addition, specific concepts of interest in the measurement stack (e.g., total sleep time) may be interchangeable with other concepts of interest (e.g., scratching events per hour or total scratching events).

[0214] Example 3: Target Solution Profile and Digital Measurement Solution for Pulmonary Hypertension

[0215] Fig. 6A Depicted is an example target solution profile for pulmonary hypertension. Here, the meaningful health aspect is the ability to perform activities of daily living, and the corresponding hypothesis is that a specific therapeutic intervention (e.g., drug X) improves the ability to perform physical activity. The concept of interest to be measured is the ability to perform daily activities while being affected by pulmonary hypertension.

[0216] In the measurement instrument resources, the measurement method generally identifies a wrist-worn device with device specifications. Here, the measurement method is device technology agnostic and neither identifies a specific device nor specific device software. The raw data element includes a raw data file captured using the measurement method (e.g., a wrist-worn device). The algorithm element identifies an algorithm that transforms the raw data file captured using the measurement method into meaningful health data. The health data element includes a meaningful set of health data transformed by the algorithm of the aforementioned element. Fig. 6AAs shown, examples of meaningful health data for pulmonary hypertension include measures of daily activity (such as number of steps, vacuum activity, etc.).

[0217] The analytical validation component ensures that the meaningful health data set is reliable, valid, and sensitive to the concept of interest. The clinical interpretation identifies that the daily performance of PAH patients significantly improved after the intervention of drug X.

[0218] Figure 6B A first example digital measurement solution for pulmonary hypertension is described. In addition, Figure 6C A second example digital measurement solution for pulmonary hypertension is depicted. Here, Figure 6B and Figure 6C The digital measurement solutions shown are each Fig. 6A The common categories represented by the target solution profile shown in .

[0219] See also Figure 6B In the DMS shown in Figure 1, the measurement method component specifies a specific wrist-worn device (e.g., ActiGraphGT9x Link). Therefore, the ActiGraph GT9X Link device captures the data represented in the raw data file. The algorithm component transforms the raw data file into a meaningful health data set. Figure 6B As shown, the ActiGraph deterministic algorithm (e.g., ActiLife 6) is implemented as an algorithm to transform the raw data files into meaningful health data sets. Fig. 6A Compared to the TSP shown, Figure 6B The DMS shown specifies specific equipment for the measurement method element and specific algorithms for the algorithm element.

[0220] See also Figure 6C The DMS shown in the measurement method element specifies the Figure 6B The device specified in the example is a specific wrist-worn device (e.g., Garmin Vivofit 4) that is different from the device specified in the example. Thus, the Garmin Vivofit 4 device captures the data represented in the raw data file. The algorithm component transforms the raw data file into a meaningful health data set. Figure 6C As shown, a machine learning algorithm is implemented that transforms the measurements captured by the Garmin Vivofit 4 into a meaningful health dataset. Fig. 6A Compared to the TSP shown, Figure 6C The DMS shown specifies specific equipment for the measurement method element and specific algorithms for the algorithm element.

[0221] Figure 6B and Figure 6CEach digital measurement solution shown in the is validated through a qualification protocol to ensure that the corresponding DMS produces similar results. An example qualification protocol includes the following:

[0222] 1) Participants wearing the DMS device, and a reference device that has been previously successfully validated for the DMS

[0223] 2) Participant walks X steps (e.g., 100 steps)

[0224] 3) Compare the total number of steps measured by the DMS's device with the total number of steps measured by the reference device. If the difference is less than a threshold number (eg, 10%), the DMS's device is successfully authenticated.

[0225] Fig.6D The reuse of at least measuring instrument resources in a digital measurement solution is depicted. Here, Fig.6D Shows Figure 6B and Figure 6C Target measurement instrument profiles (e.g., concept of interest elements, measurement instrument resources, and analysis validation elements) for each DMS are shown in . It is noteworthy that the target measurement instrument profiles for each DMS are interchangeable and feasible for other meaningful health aspects and disease aspects. Therefore, the stack of resources that make up the target measurement instrument profiles can be easily utilized and incorporated into different DMSs. As a specific example, a digital measurement solution is needed to measure concepts of interest related to Parkinson's disease. Therefore, these target measurement instrument profiles for pulmonary hypertension conditions are reused to measure Parkinson's disease.

[0226] Example 4: Dynamic digital document with intelligent sections

[0227] Figure 7 A dynamic digital document having an intelligent part is depicted. A dynamic digital document is a digital file. The dynamic digital document is located in one or more elements of one or more layers of a measurement stack. The dynamic digital document may represent an element of a layer of a measurement stack. For example, the dynamic digital document includes a concise manual of an element of a measurement instrument layer. The dynamic digital document includes one or more of text, icons, pictures, etc. Figure 7As shown, the dynamic digital document includes a header. The header includes one or more characters, words, phrases, etc. that can classify the text body. The dynamic digital document includes a version history representing the version of the dynamic digital document. The dynamic digital document displays its own version at the top, bottom, side, or other position of the dynamic digital document. The dynamic digital document includes an index. The index of the dynamic digital document includes one or more information lists that are part of the dynamic digital document. The user interacts with the index of the dynamic digital document, for example, by clicking on the table list with a mouse or other user input. The dynamic digital document responds to the user input. For example, the dynamic digital document displays a part of itself that the user may click on in the index table of the dynamic digital document. For example, the user clicks on the summary label of the dynamic digital document, and the dynamic digital document displays a summary to the summary label.

[0228] The dynamic digital document includes one or more intelligent parts. The intelligent part of the dynamic digital document may include a modifiable part of the dynamic digital document. For example, the intelligent part includes a text body that can be modified by multiple users or collaborators. The dynamic digital document includes one or more interactive parts. For example, the dynamic digital document includes a save button that saves the current version of the dynamic digital document when receiving a user interaction. The dynamic digital document includes a save as new version button that saves the current version of the dynamic digital document as a new version in a database or other file storage system. The dynamic digital document includes a lock button that locks the version of the dynamic digital document to prevent modification from one or more collaborators. The dynamic digital document includes a submit button that sends a copy, link, etc. of the dynamic digital document to one or more parties, such as via a network. The dynamic digital document includes a view product history button that allows one or more users to view the product usage history associated with the dynamic digital document.

[0229] The dynamic digital document includes a start conversation button. The start conversation button of the dynamic digital document allows two or more users to leave one or more comments, edits and / or other modifications in the dynamic digital document. The dynamic digital document includes a view conversation button, which allows one or more users to view the current and / or past conversations of one or more users of the dynamic digital document. The dynamic digital document includes a respond button. The respond button generates a text box for the user to enter text in response to one or more comments of one or more other users. The dynamic digital document also has an extract formal question list button. The extract formal question list button generates a list of formal questions left and / or answered by one or more users of the dynamic digital document. The extract list button generates a word, text or other file of one or more questions and / or answers left by one or more users.

[0230] The dynamic digital documents and / or other digital documents described throughout this disclosure show the probability of successful supervision. The probability of successful supervision includes the percentage of approval from one or more supervisors. The computing device uses a probability model, a machine learning model and / or other machine processes to determine the probability of successful supervision. The computing device inputs various data into the probability model and / or the machine learning model to determine the probability of successful supervision. The data may include but is not limited to algorithm data, measurement method data, condition data, patient population data, concept data of interest, analysis verification data and / or other aspects of one or more layers of DMS. The computing device is configured to predict that the DMS has an 85% probability of successful supervision based on the change of the patient population in the DMS from adults to children. The computing device continuously calculates or otherwise updates the probability of successful supervision, for example, when the dynamic digital document or other document is modified and / or updated. The computing device updates and / or calculates the probability of successful supervision based on data queries through one or more databases. For example, the computing device finds a DMS similar to the proposed DMS and updates and / or calculates the probability of successful supervision based on a similar DMS.

[0231] Example 5: Standardized solutions for improving regulatory acceptance

[0232] Fig. 8A A high-level overview of a collaborative effort for developing standardized solutions is depicted. Here, multiple parties participate in a collaborative effort to jointly develop standardized solutions. These parties participate in a standardized and structured approach, which results in reduced turnaround and development time. These standardized solutions undergo dynamic regulatory assessment by involving regulators at an early stage of development. The developed solutions are then delivered (e.g., to a client).

[0233] Figure 8B An example process involving various parties for implementing a dynamic regulatory assessment of a standardized solution is depicted. The process begins at step 1, which involves the Digital Endpoint Ecosystem and Protocol (DEEP) catalog. Here, additional digital metrics and / or digital solutions are developed, and further, these digital solutions can undergo dynamic regulatory acceptance. Dynamic Regulatory Acceptance (DRA) involves initiating a DEEP mission (step 2), which involves multiple stakeholders working together on a common mission. Specifically, at step 3, these stakeholders collaborate to develop a digital brief that includes digital solutions. Figure 8B As shown, various stakeholders provide various inputs to the brief manual creation process, such as clinical perspectives, patient perspectives, and / or technical feasibility. At step 4, the brief manual is submitted for regulatory approval, so that the regulator has access to the brief manual including the digital solution.

[0234] The interaction between the regulator and the stakeholders can be as follows: at step 5, the regulator logs in, browses the DEEP directory to further understand the digital solution with additional context, and views public issues and private background materials with private issues from two sponsoring companies. If necessary, the regulator can reengage with stakeholders to obtain additional clarity and information. The regulator sees that the input of patients and clinicians has been incorporated. However, the regulator still has doubts, for example, for the severe form of the disease, it seems important to fully evaluate the scratching activity. Therefore, the regulator requires additional background about the patient's behavior, such as the patient's scratching position, how to scratch, the number of hours of scratching, etc. The regulator also wants to ask patients whose disease is not too serious and understand whether their scratching activities are different. Stakeholders (e.g., patient representatives) provide this feedback. Here, patients with severe forms of the disease scratch all over their bodies, also scratch with both hands or even with both feet. Alternatively, patients with lighter forms of the disease report that usually their itching breaks out suddenly in one area, and they finally only scratch this one part.

[0235] The regulators then want to ask patients with severe forms of the disease about the camera solution. Given their disease, could they tolerate using such a solution for a period of time? The patient representatives responded that "yes" because their disease is already a heavy burden, so they would be willing to do so in order to help find a solution. However, they did express concerns about doing so for a long time. The regulators consider all of these inputs and then develop their responses to these questions. In an exemplary response, the regulators agree that scratching is meaningful and can be used as a key endpoint. Regarding the two solutions envisioned, the regulators see good applicability of both in different types of trials. Both sound plausible, but evidence of their performance in detecting scratching activity is needed, and the regulators also recommend that evidence be developed to better understand how much change in this measure will have a meaningful benefit to patients. In addition, the impact on sleep and next-day sleepiness should be explored.

[0236] Regarding the private question, the regulator recommends that Pharmaceutical Company A (Severe Disease) consider using both envisioned solutions in their trial. Studies could be designed utilizing camera observation periods as well as wearable devices. It could be investigated whether a wearable solution could be an appropriate alternative to obtain more robust video measures. Thus, if the added value of a video solution could be better understood, better guidance could be provided in the future. The ideal solution could be to use both in studies of severe forms of the disease, balancing scientific value with patient burden.

[0237] For Pharmaceutical Company B, the regulator foresees that a single wearable device solution could be sufficient to measure these isolated scratching episodes. However, the regulator advises the company to also work on understanding how well the video correlates with the wearable measures and then make an informed choice in their trial design. The regulator advises the sponsor to return for additional advice when more evidence becomes available and then discuss the specific trial design again.

[0238] Therefore, if the regulator sees enough evidence, then at step 6, the regulator provides regulatory acceptance of the digital solution. At step 7, the collaborative task involving multiple stakeholders is completed. The regulatory feedback is collated and connected with the catalog (MAH, COI and TSP). The stakeholders involved in this regulatory process now have clear guidance on the next steps. The two solutions have uses for different purposes, and the two sponsors have already started planning the solution development tasks. Pharma A wants to invest in both options, and Pharma B is interested in both options, but obviously wants to prioritize the development of the wearable solution.

[0239] Figure 8C An example process involving parties for implementing dynamic regulatory assessment of standardized solutions is depicted. The process begins at step 805, where the parties may access a digital environment, such as one configured for a DEEP catalog. In step 810, a central data structure is created using compiled evidence (in some instances, evidence of one or more aspects of one or more DMS layers) of previously approved digital solutions that have been received. In step 815, along the Figure 8BThe line of exemplary step 2 in starts the task. In this regard, in some instances, collaborators who are each independent types of stakeholders are assigned roles in the task. Some tasks may allow multiple collaborators with the same role, while some may not. Some tasks may specify primary roles / secondary roles for collaborators, thereby allowing different levels of rights or access to various elements of tasks and related elements. In step 820, one or more predefined templates may be generated / selected / used, in which various task identification standards are identified. For example, the template may reflect the agreement (e.g., contract) between various stakeholders on the task, thereby arranging the task standards. In step 825, an agreement on the task standards is received from the collaborator (in some instances, this may involve signing the agreement digitally or otherwise). In step 830, a request for a new digital solution is received, for example, from a collaborator, a regulator, or other party. In step 835, input is received from one or more collaborators to define one or more elements of a standardized digital solution. In step 840, a digital solution is generated based on the elements of a previously approved standardized digital solution. In step 845, additional data about the new digital solution is received from the collaborator, which is updated to the central data structure in step 850. In some instances, a dossier may be generated in step 855 for submission to a regulatory collaborator (step 860), but in some instances, the dossier may have already been created, such as as a central data structure or an element thereof. In step 860, the regulatory collaborator may provide feedback (e.g., regulatory acceptance) and / or a request for more information, as described above with reference to Figure 8B As described, any of these feedbacks may be recorded digitally, such as in conjunction with a central data structure (step 865).

[0240] Example 6: Example development of a standardized solution involving multiple stakeholders

[0241] Assume that Pharmaceutical Company A and Pharmaceutical Company B have generated their respective measurement definitions for atopic dermatitis and are interested in measuring the number of nighttime scratching events. The remaining question is how to capture these measurements. Pharmaceutical Company A wants to develop the right solution for measuring its endpoint. Therefore, Pharmaceutical Company A accesses and searches the DEEP catalog to identify solutions that already exist.

[0242] For example, Pharmaceutical Company A types "scratching" in the resource search box, which results in the discovery of sensor devices, algorithms, and relevant datasets for this use case. Pharmaceutical Company A can therefore find the building blocks it needs for its solution.

[0243] Here, Pharma A can create a new task to find a guardian service that can compile a digital solution and maintain it. Pharma A will fully fund this work, set up access rights to Pharma A, which will fully own the solution, but grant a 3-year operating license to the guardian, who will be responsible for maintaining the documentation of the solution, including any component upgrades during the license period.

[0244] The guardian assembles a solution from the components in the catalog and connects the solution to the established measurement definitions. Pharmaceutical Company A now has access to the solutions they need for their clinical trials.

[0245] In addition, Pharma Company B can now also see that the solution is available for licensing from the Guardian (they were notified that the DMS is available for the TSP they follow / subscribe to). The performance of the solution looks promising and the licensing conditions look fair. Pharma Company B also decides to license the solution from the Guardian.

[0246] In summary, multiple stakeholders can rapidly adopt solutions through a more efficient path provided by standardized solutions.

[0247] Example 7: Example Decision Tree for Use with Digital Measurement Solutions

[0248] Fig.10 An illustrative example of a decision tree that can be used with any system or process as described throughout this disclosure is shown. Fig.10 As shown, the health aspects in the definition layer of the DMS can be changed, updated and / or modified. If the change to the health aspect has not been captured by an algorithm, measurement method or other process, the health aspect requires new evidence for the new concept. The new evidence is received through user input, patient input, etc. Alternatively, if the current technology has already captured relevant data about the measurement of the health aspect, the decision tree moves to another standard. If there is analytical evidence about the technical ability to obtain new variables in the health change aspect, subsequent standards are included. If there is analytical evidence that the current technology does have the ability to capture new variables in the health change aspect, the analytical verification is reused. If there is no analytical evidence about the technical ability to capture new variables in the health change aspect, the analytical verification is re-performed.

[0249] Referring back to the decision tree, a change in disease (labeled "Disease Change") prompts a comparison to the criteria. As non-limiting examples, disease changes include spinal muscular atrophy or night scratching. For example, a change in disease at the definition layer of the DMS can be confirmed to be similar to one or more other diseases. Confirmation can come from one or more patient interviews and / or other sources. New concepts of disease change are extracted through one or more patient interviews. The definitions between diseases are compared for similarity. If the definitions between diseases are similar, bridges between diseases are created with new evidence and earlier evidence bases are reused. If the definitions between diseases are not similar, new definitions can be generated with new evidence from new disease areas. Techniques suitable for new diseases are evaluated that are similar to one or more changes in the definitions of the disease.

[0250] like Fig.10 As further shown in , the population of the DMS can be changed, which prompts a comparison of the new population to one or more standards. Population changes include, as a non-limiting example, an adult population to a pediatric population. Similar to disease changes, a comparison between the previously used population and the new population is performed. If the similarities between the populations are large enough, the analytical validation is reused. If the differences between the populations are large enough, new evidence is generated and a new analytical validation is performed.

[0251] Example 8: Stack navigation

[0252] Fig.11 is an exemplary example of a user interface for stack navigation. The user interface is displayed on, but not limited to, a laptop, a monitor, a smart phone, a tablet computer, etc. The user interface displays the workflow from the left side of the screen to the right side of the screen. Fig.11As shown, the user interface displays a tree diagram. For example, in the leftmost part of the screen, the text of the disease is displayed, an example of which is atopic dermatitis. The text of the disease is displayed next to an icon, such as a circle, an oval, etc. The user interface displays a list of disease symptoms next to the disease text, such as on the right. For example, the stack of symptoms next to the circle icon stack includes but is not limited to itching, skin conditions, sleep disorders, soreness / pain, mental health, burning / tingling sensation, pain, etc. Next to the stack of symptoms, such as on the right side of the stack of symptoms, a stack of additional symptoms specific to the symptoms of the previous stack is displayed next to the stack of circular icons. For example, a stack of additional symptoms includes EASI scores, scratching at night, scratching during the day, etc. Next to the stack of additional symptoms, such as on the right or left side of the stack of additional symptoms, a stack of TSPs is displayed. For example, a stack of TSPs for actigraphy, radio frequency, infrared and / or activity monitors is displayed. On the rightmost part of the user interface, one or more DMSs are displayed. For example, a stack of DMSs specific to the TSP of the previous stack is displayed. As non-limiting examples, the stack of TSPs may include an actigraph and the stack of DMSs include a DMS utilizing a Magic watch 5, magic band 2, Applewatch 7, actisleep, GENEactiv, and / or other measurement instruments and / or methods.

[0253] The user selects one or more icons of the user interface, which generates one or more stacks. For example, the user clicks on the circle icon next to the disease text, which generates a stack of symptoms of the disease. The user clicks on a symptom in the symptom stack of the disease, which generates additional symptoms specific to the first symptom. The user selects an additional symptom, which generates a stack of one or more TSPs for that symptom. The user selects a TSP that generates a stack of one or more DMSs. Each DMS can be interacted with, which results in a list of ongoing tasks, entries from an interest register, etc. being displayed.

[0254] Example 9: Cost Savings

[0255] Fig.12A screenshot of a user interface showing a chart of cost savings using DMS is depicted. By reusing and / or leveraging a previously applied DMS, the cost of one or more tasks is reduced. For example, the measurement definition task includes a literature review, which requires about 490 hours of work effort and about 70,000 euros in terms of third-party price considerations. However, by leveraging the DMS platform and reusing about 70% of the previously applied data, the cost of the literature review of about 49,000 euros is saved. Each element of each task type saves costs by leveraging various percentages of reused data. For example, the entire measurement definition task usually costs about 440,650 euros through a third party. However, by reusing the data, about 189,165 euros can be saved, thereby reducing the cost to about 251,485 euros. Similarly, reusing previously applied data can save the cost of regulatory tasks, target solution profiles, exploratory technology prototyping tasks, analytical validation tasks, clinical validation tasks, regulator tasks on DMS, etc. On average, reusing a total of approximately 42% of the data from previous applications reduces the combined total cost per task by approximately 7,507,040 EUR from approximately 13,044,900 EUR to a final cost of approximately 5,537,860 EUR.

[0256] Additional Instances

[0257] In various instances, the creation of standardized digital solutions (e.g., SMPS and DMS) and / or elements of standardized digital solutions involves collaborative efforts between two or more collaborators within a digital environment. Here, two or more collaborators can work on tasks generated within a digital environment to provide inputs about the individual elements of the standardized digital solution and / or the full stack of the standardized digital solution. For example, within a task, two or more collaborators can communicate and provide inputs for defining measurable concepts of interest. As another example, within a task, two or more collaborators can communicate and provide inputs for defining measurement methods. Two or more collaborators can communicate and provide inputs for any element of a digital measurement solution.

[0258] In various examples, to improve regulatory review and acceptance of components and / or standardized digital solutions, the methods disclosed herein involve generating a dossier that includes input from two or more collaborators. Here, the dossier may include input from multiple components and / or standardized digital solutions collaborators. Thus, the dossier may be provided to regulators to review and comment on multiple components and / or standardized digital solutions.

[0259] In various instances, archives are constructed to include separate elements, each of which is dedicated to a single element or to a single standardized digital solution. Therefore, input, communication and feedback (e.g., from collaborators and regulators) for a specific element or for a specific standardized digital solution can be compiled and included in a specific element of the archive. For example, a collaborator can provide input to define an element. A regulator can provide a question in response to the input of an element. Here, the input of the collaborator can be connected to an element in the archive dedicated to the element, and then catalog each response from the regulator for the same element in the archive. Therefore, when the regulator is presented as a complete archive, the single element of the archive can be decomposed into a single structural piece dedicated to a single element. This is beneficial because the regulatory feedback received for an element for a specific use case (e.g., for characterizing a first disease) can be reused in other use cases (e.g., for use in a standardized digital solution for characterizing other diseases). For example, if a specific device is used for a case and receives regulatory acceptance, another team can use the precedent in a different use environment without having to start from scratch. Therefore, this structural approach can achieve economies of scale across the entire digital ecosystem. In summary, by constructing a dossier out of individual elements, the dossier can be delivered to regulators as a single document, but interactions can be centered around these individual elements. For example, collaborators and regulators can communicate within a single element of the dossier centered around a specific issue or a specific location. Therefore, these communications can also be cataloged independently within the single element. Therefore, any suggestions received for resources or elements of standardized solutions can be cataloged appropriately. Finally, these elements of regulatory precedent can be reused throughout the ecosystem.

[0260] Additionally disclosed herein is a method for facilitating regulatory decision making for at least an element of a standardized digital solution for characterizing a disease, the method comprising: providing a digital environment comprising multiple elements of the standardized digital solution, wherein the digital environment is accessible to multiple collaborators; and wherein two or more of the multiple collaborators have expressed a common interest; generating tasks within the digital environment by establishing standardized roles for the two or more collaborators, wherein the tasks are created for the two or more collaborators to contribute to at least the element of the standardized digital solution in order to seek regulatory decision making; receiving input from one or more of the two or more collaborators, the input contributing to the element of the standardized digital solution in order to seek regulatory decision making; generating a dossier for at least the element of the standardized digital solution, the dossier including the received input contributing to at least the element of the standardized digital solution; and providing the regulator collaborator with access to at least a portion of the dossier within the task.

[0261] In various examples, the methods disclosed herein also include: receiving feedback from the regulator collaborator, the feedback being specific to the received input contributing to at least the element of the standardized digital solution; and recording the received feedback from the regulator collaborator in a profile. In various examples, the feedback from the regulatory collaborator includes regulatory acceptance of at least the element of the standardized digital solution. In various examples, the feedback from the regulatory collaborator includes one or more questions or requests related to at least the element of the standardized digital solution.

[0262] In various examples, the method disclosed herein also includes: receiving additional input from two or more collaborators in response to feedback from a supervisory collaborator; and recording additional input from two or more collaborators in an archive accessible within the task. In various examples, the method disclosed herein also includes: publishing archives outside the task for use in one or more subsequent tasks. In various examples, establishing roles for two or more collaborators includes assigning a primary role to a first collaborator in the two or more collaborators and assigning a secondary role to a second collaborator in the two or more collaborators. In various examples, the primary role has additional permissions compared to the secondary role. In various examples, establishing roles for two or more collaborators also includes assigning one or more additional roles. In various examples, the primary role and the secondary role each have additional permissions compared to the additional role. In various examples, the additional role is assigned to any one of a funding sponsor, an observer, a technology provider, or a data partner. In various examples, permissions include permissions to access or view the results generated from the task. In various examples, permissions include permissions to use the results generated from the task.

[0263] In various examples, before generating a task, two or more collaborators are selected to be included in the task. In various examples, the common interest expressed by the two or more collaborators is a common interest in contributing to the elements of a standardized digital solution. In various examples, generating a task includes generating a private task that can only be accessed by two or more collaborators with defined roles. In various examples, generating a task includes generating a public task that can be accessed by additional collaborators. In various examples, the two or more collaborators include two, three, four, five, six, seven, eight, nine, ten, eleven, twelve, thirteen, fourteen, fifteen, sixteen, seventeen, eighteen, nineteen or twenty collaborators.

[0264] In various examples, the method disclosed herein also includes: receiving an agreement on one or more standards for the task from each of the two or more collaborators before receiving input from one or more of the two or more collaborators to define at least the element of the standardized digital solution. In various examples, the one or more standards for the task include the terms and conditions of the agreement. In various examples, before receiving the agreement from each of the two or more collaborators, generating a predefined template identifying one or more standards for the task; and providing the predefined template to each of the two or more collaborators. In various examples, generating the predefined template includes: receiving input from one of the two or more collaborators for establishing the characteristics of the second role; and incorporating the input from one of the two or more collaborators into the predefined template.

[0265] In various instances, the standardized digital solution is one of a digital measurement solution or a target solution profile. In various instances, the target solution profile represents a common category of digital measurement solutions. In various instances, the digital measurement solution includes: a measurement definition that defines one or more concepts of interest related to a disease; a measurement instrument resource that is configured to transform the measurement results of interest captured according to the measurement definition into a data set that provides information for characterizing the disease, wherein the measurement instrument resource of the digital measurement solution is dedicated to the device used to capture the measurement results of interest. In various instances, the target solution profile includes: a measurement definition that defines one or more concepts of interest related to a disease; a measurement instrument resource that is configured to transform the data captured according to the measurement definition into a data set, which is device technology agnostic and therefore interchangeable between different target solution profiles. In various instances, the elements of the standardized digital solution include any of the following: a measurable concept of an element of interest; a measurement method element; a raw data element; an algorithm element; a data set element; a technical validation element; an analysis validation element; a clinical validation element; or a regulatory element.

[0266] In various examples, in addition to the elements of the standardized digital solution, a profile is further generated for one or more elements. In various examples, the profile is constructed to include a plurality of elements, wherein each of the plurality of elements of the profile is dedicated to a single element selected from the one or more elements or the elements of the standardized digital solution. In various examples, the element of the profile dedicated to a single element stores inputs contributing to the single element.

[0267] In various instances, the methods disclosed herein also include receiving feedback from a regulator collaborator that is specific to one of the one or more elements in the archive; and incorporating the feedback specific to one element into an element specific to one element. In various instances, in addition to the elements of the standardized digital solution, the archive is further generated for one or more standardized digital solutions. In various instances, the archive is constructed to include multiple elements, wherein each of the multiple elements of the archive is specific to a single standardized digital solution selected from one or more standardized digital solutions. In various instances, the methods disclosed herein also include receiving feedback from a regulator collaborator that is specific to one of the one or more standardized digital solutions in the archive; and incorporating the feedback specific to one standardized digital solution into an element specific to one standardized digital solution.

[0268] In various instances, the digital environment is a collaborative platform built based on a measurement stack model. In various instances, each of the two or more collaborators is an independent stakeholder. In various instances, the received input includes one or more of the following types of information: technical feasibility information, patient perspective information, and clinical information. In various instances, regulatory decision making includes regulatory acceptance.

[0269] In various instances, the methods disclosed herein also include: generating a legal agreement based on one or more of: common interests, regulator collaborators, regulatory decisions, and established roles of two or more collaborators, wherein the legal agreement includes legal terms that include one or more of: task rules and collaborator permissions; providing the legal agreement to each of the two or more collaborators via a digital environment; and providing a framework for one or more collaborators to negotiate and edit legal terms via the digital environment.

[0270] Each program can be implemented in a high-level procedural programming language or an object-oriented programming language to communicate with a computer system. However, these programs can be implemented in assembly language or machine language as needed. In any case, the language can be a compiled or interpreted language. Each such computer program is preferably stored on a storage medium or device (e.g., ROM or disk) readable by a general or special programmable computer to configure and run the computer to perform the process described herein when the computer reads the storage medium or device. The system can also be considered to be implemented as a computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner to perform the functions described herein.

[0271] Signature patterns and their databases may be provided in a variety of media to facilitate their use. "Medium" refers to an article containing the signature pattern information of the present invention. The database of the present invention may be recorded on a computer-readable medium (e.g., any medium that can be directly read and accessed by a computer). Such media include, but are not limited to: magnetic storage media, such as floppy disks, hard disk storage media, and magnetic tapes; optical storage media, such as CD-ROMs; electrical storage media, such as RAM and ROM; and hybrids of these categories, such as magnetic / optical storage media. Those skilled in the art can readily understand how any of the currently known computer-readable media can be used to create a recorded article containing the database information of the present invention. "Recorded" refers to a process for storing information on a computer-readable medium using any such method known in the art. Any convenient data storage structure may be selected based on the means for accessing the stored information. A variety of data processor programs and formats may be used for storage, such as word processing text files, database formats, and the like.

[0272] Some examples of the disclosed devices may be implemented, for example, using storage media, computer-readable media, or articles that can store instructions or instruction sets, which, when executed by a machine (i.e., a processor or microcontroller), can cause the machine to perform methods and / or operations according to examples of the present disclosure. Such machines may include, for example, any suitable processing platform, computing platform, computing device, processing device, computing system, processing system, computer, processor, etc., and may be implemented using any suitable combination of hardware and / or software. Computer-readable media or articles may include, for example, any suitable type of memory unit, memory, memory article, memory medium, storage device, storage article, storage medium and / or storage unit, such as memory (including non-temporary memory), removable or non-removable media, erasable or non-erasable media, writable or rewritable media, digital or analog media, hard disk, floppy disk, compact disk read-only memory (CD-ROM), recordable compact disk (CD-R), rewritable compact disk (CD-RW), optical disk, magnetic media, magneto-optical media, removable memory card or disk, various types of digital versatile disks (DVD), magnetic tape, cassette tape, etc. The instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, programming code, etc., implemented using any suitable high-level, low-level, object-oriented, visual, compiled and / or interpreted programming language. A non-transitory computer-readable medium embodying the programming code may cause a processor to perform functions such as those described herein when executing the programming code.

[0273] Additional Instances

[0274] The present invention discloses a method for managing at least elements of a standardized digital solution for characterizing a disease, comprising: obtaining a central data structure at a processor, wherein the central data structure includes compiled evidence of multiple elements of previously approved standardized digital solutions; receiving a user request for a new standardized digital solution at the processor; generating a new standardized digital solution at the processor by utilizing one or more elements of the previously approved standardized digital solutions in the central data structure; receiving additional data for the new standardized digital solution from a user at the processor; updating the central data structure at the processor to incorporate the additional data; receiving feedback from a regulatory collaborator, the feedback being specific to the received input contributing to at least the element of the new standardized digital solution; and recording the feedback received from the regulatory collaborator, wherein the received feedback includes regulatory acceptance of at least the element of the new standardized digital solution.

[0275] In various instances, the method further includes adding, by the processor, the additional data to the new standardized digital solution to create an extended standardized digital solution. In various instances, the processor is further configured to generate a plurality of new standardized digital solutions for different patient populations. In various instances, the different patient populations include one of pediatric, geriatric, or adult population groups.

[0276] In various instances, elements of the previously approved standardized digital solution are included in one or more of the validation layer, the instrumentation layer, or the definition layer of the central data structure. In various instances, the previously approved resource includes measurement data of the medical device.

[0277] In various examples, generating the new standardized digital solution includes comparing evidence data of previously approved resources and generating the new standardized digital solution based at least on an acceptability classification of the evidence data. In various examples, the acceptability classification is determined based on one or more of freshness of the evidence data, amount of the evidence data, patient population of the evidence data, or a combination thereof.

[0278] In various instances, the acceptability classification is determined based on one or more of the acceptance of regulatory intelligence / evidence data, clinical validation of evidence data, or technical verification of evidence data, or a combination thereof. In various instances, the acceptability classification is determined based on one or more of the content validity of the evidence data, the construct validity of the evidence data, the sensitivity to changes in the evidence data, or a combination thereof. In various instances, the acceptability classification is one of full acceptability, intermediate acceptability, or unacceptability. In various instances, full acceptability corresponds to no additional evidence required to generate a new standardized digital solution. In various instances, intermediate acceptability corresponds to less additional evidence required to generate a new standardized digital solution. In various instances, unacceptability corresponds to complete additional evidence required to generate a new standardized digital solution. In various instances, each element of a previously approved standardized digital solution undergoes a form of verification based on an acceptability threshold.

[0279] In various instances, at least one element of the instrumentation layer and at least one element of the definition layer are classified in a fully acceptability category. In various instances, at least one element of the definition layer is classified in a fully acceptability category. In various instances, at least one element of the instrumentation layer is classified in an unacceptability category, and at least one element of the definition layer is classified in a fully acceptability category. In various instances, each element of the instrumentation layer and at least two elements of the definition layer are classified as fully acceptable.

[0280] In various examples, one of the validation layers, instrumentation layers, definition layers, or a combination thereof of a previously approved new standardized digital solution undergoes a form of validation based on an acceptability threshold.

[0281] In various instances, generating a new standardized digital solution includes generating a new standardized digital solution based on a medical condition. In various instances, generating a new standardized digital solution includes generating a new standardized digital solution based on a diagnostic method.

[0282] In various examples, the method further includes displaying the updated central data structure at a display device via a graphical user interface (GUI), wherein the GUI visualizes the additional data added from the user via the new standardized digital solution.

[0283] In various instances, new standardized digital solutions include diagnostic-specific algorithms.

[0284] In various examples, the method further includes, prior to receiving the user request, providing a digital environment including a plurality of elements of the standardized digital solution, wherein the digital environment is accessible to a plurality of collaborators.

[0285] In various examples, the method further includes: generating a task within the digital environment by establishing standardized roles for two or more collaborators prior to receiving the user request. In various examples, the two or more collaborators express a common interest. In various examples, the method further includes: receiving input from one or more collaborators for elements of the new standardized digital solution.

[0286] In various examples, the method also includes providing the supervisor collaborator with access to at least a portion of the archive within the task.

[0287] In various instances, the feedback from the regulatory collaborators includes one or more questions or requests related to at least elements of the new standardized digital solution.

[0288] In various examples, the method further includes: receiving additional input from the two or more collaborators in response to the feedback from the supervisory collaborator; and recording the additional input from the two or more collaborators in a dossier accessible within the task.

[0289] In various examples, the method further includes publishing the archive outside of the task for use in one or more subsequent tasks.

[0290] In various instances, establishing roles for two or more collaborators includes assigning a primary role to a first collaborator of the two or more collaborators and assigning a secondary role to a second collaborator of the two or more collaborators. In various instances, the primary role has additional permissions compared to the secondary role. In various instances, establishing roles for two or more collaborators also includes assigning one or more additional roles. In various instances, the primary role and the secondary role each have additional permissions compared to the additional role. In various instances, the additional role is assigned to any one of a funder, an observer, a technology provider, or a data partner. In various instances, permissions include permissions to access or view results generated from a task. In various instances, permissions include permissions to use results generated from a task.

[0291] In various instances, the method also includes selecting two or more collaborators to include in the task before generating the task. In various instances, generating the task includes generating a private task that can only be accessed by two or more collaborators with defined roles. In various instances, generating the task includes generating a public task that can be accessed by additional collaborators. In various instances, the two or more collaborators include two, three, four, five, six, seven, eight, nine, ten, eleven, twelve, thirteen, fourteen, fifteen, sixteen, seventeen, eighteen, nineteen or twenty collaborators. In various instances, the method also includes: receiving an agreement on one or more standards about the task from each of the two or more collaborators before receiving input from one or more of the two or more collaborators to define at least the element of the standardized digital solution.

[0292] In various examples, the one or more criteria for the task include terms and conditions of the agreement. In various examples, prior to receiving the agreement from each of the two or more collaborators, generating a predefined template identifying the one or more criteria for the task; and providing the predefined template to each of the two or more collaborators. In various examples, generating the predefined template includes: receiving input from one of the two or more collaborators for establishing characteristics of the second role; and incorporating the input from the one of the two or more collaborators into the predefined template.

[0293] In various instances, the standardized digital solution is one of a digital measurement solution or a target solution profile. In various instances, the target solution profile represents a common category of digital measurement solutions. In various instances, the digital measurement solution includes: a measurement definition that defines one or more concepts of interest related to a disease; a measurement instrument resource that is configured to transform the measurement results of interest captured according to the measurement definition into a data set that provides information for characterizing the disease, wherein the measurement instrument resource of the digital measurement solution is dedicated to the device used to capture the measurement results of interest. In various instances, the target solution profile includes: a measurement definition that defines one or more concepts of interest related to a disease; a measurement instrument resource that is configured to transform the data captured according to the measurement definition into a data set, the measurement instrument resource is device technology agnostic, and is therefore interchangeable between different target solution profiles.

[0294] In various instances, elements of the standardized digital solution include any of: a measurable concept of an element of interest; a measurement method element; a raw data element; an algorithm element; a dataset element; a technical validation element; an analytical validation element; a clinical validation element; or a regulatory element.

[0295] In various examples, in addition to the elements of the standardized digital solution, an archive is further generated for one or more elements. In various examples, the archive is structured to include a plurality of elements, wherein each of the plurality of elements of the archive is dedicated to a single element selected from the one or more elements or the elements of the standardized digital solution. In various examples, the element of the archive dedicated to the single element stores inputs contributing to the single element. In various examples, in addition to the elements of the standardized digital solution, an archive is further generated for one or more standardized digital solutions. In various examples, the archive is structured to include a plurality of elements, wherein each of the plurality of elements of the archive is dedicated to a single standardized digital solution selected from the one or more standardized digital solutions.

[0296] In various examples, the method further includes: receiving feedback from a regulator collaborator specific to one of the one or more elements in the dossier; and incorporating the feedback specific to one element into the element specific to one element. In various examples, the method further includes: receiving feedback from a regulator collaborator specific to one of the one or more standardized digital solutions in the dossier; and incorporating the feedback specific to one standardized digital solution into the element specific to one standardized digital solution.

[0297] In various instances, the digital environment is a collaborative platform built on the measurement stack model.

[0298] In various examples, each of the two or more collaborators is an independent stakeholder.

[0299] In various examples, the received input includes one or more of the following types of information: technical feasibility information, patient perspective information, and clinical information.

[0300] In various examples, regulatory decision making includes regulatory acceptance.

[0301] In various instances, the method also includes: generating an agreement based on one or more of: common interests, regulator collaborators, regulatory decisions, and established roles of two or more collaborators, wherein the agreement includes terms that include one or more of: task rules and collaborator permissions; providing the agreement to each of the two or more collaborators via a digital environment; and providing a framework for editing the terms for one or more collaborators via the digital environment.

[0302] In various examples, the method also includes generating a dossier for seeking regulatory decision making for at least the element of the digital measurement solution.

[0303] In various examples, the common interest expressed by two or more collaborators is a common interest in contributing to elements of a standardized digital solution.

[0304] Also disclosed herein is an apparatus for managing at least elements of a standardized digital solution for characterizing a disease, the apparatus comprising: a processor; and a memory communicatively coupled to the processor, wherein the memory contains instructions that configure the processor to: obtain a central data structure, wherein the central data structure comprises compiled evidence of multiple elements of previously approved standardized digital solutions; receive a new request for a new standardized digital solution; generate the new standardized digital solution by utilizing one or more elements of the previously approved standardized digital solution in the central data structure; receive additional data for the new standardized digital solution from a user; and update the central data structure to incorporate the additional data.

[0305] In various instances, the processor is further configured to add additional data to the new standardized digital solution to create an extended standardized digital solution. In various instances, the processor is further configured to generate multiple new standardized digital solutions for different patient populations. In various instances, elements of previously approved standardized digital solutions are included in one or more of the validation layer, the measuring instrument layer, or the definition layer of the central data structure. In various instances, the processor is further configured to display the updated central data structure through a graphical user interface (GUI), wherein the GUI visualizes the additional data added from the user through the new standardized digital solution.

[0306] In various instances, a new standardized digital solution is generated based on an acceptability threshold of compiled evidence of previously approved standardized digital solutions. In various instances, the new standardized digital solution is based on a medical condition. In various instances, the new standardized digital solution is based on a diagnostic method. In various instances, the new standardized digital solution includes a diagnostic-specific algorithm.

[0307] In various instances, the compiled evidence includes measurement data of a medical device. In various instances, the additional data includes medical data. In various instances, the medical data includes one of heart rate data, blood oxygen data, breathing data, blood sample data, or a combination thereof.

[0308] Some examples of the present disclosure are described above. However, it should be clearly noted that the present disclosure is not limited to those examples, but is intended to be included in the scope of the disclosed examples to add and modify the contents clearly described herein. In addition, it should be understood that the features of the various examples described herein are not mutually exclusive, and may be present in various combinations and permutations, even if such combinations or permutations are not expressed herein, without departing from the spirit and scope of the disclosed examples. In fact, those of ordinary skill in the art will expect changes, modifications and other specific implementations of the contents described in the art without departing from the spirit and scope of the disclosed examples. Therefore, the disclosed examples are not limited only by the aforementioned illustrative descriptions.

[0309] It is emphasized that the description abstract is provided to allow the reader to quickly determine the nature of the technical disclosure. It should be understood that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the aforementioned specific embodiments, various features are grouped together in a single example to simplify the present disclosure. The method of the present disclosure should not be understood as embodying such an intention: the examples protected by the claims require more features than those explicitly listed in each claim. On the contrary, as reflected in the following claims, the subject matter of the invention lies in all the features of less than a single disclosed example. Therefore, the following claims are hereby incorporated into the specific embodiments, wherein each claim is independently used as a separate example. In the attached claims, the terms "including" and "in which" are used as the plain English equivalents of the corresponding terms "including" and "wherein", respectively. In addition, the terms "first", "second", "third", etc. are used only as markers and are not intended to impose numerical requirements on their objects.

[0310] The foregoing description of examples has been provided for purposes of illustration and description. This description is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and changes may be made in light of the disclosure. The scope of the disclosure is not intended to be limited by this detailed description, but rather by the appended claims. Future applications claiming priority to the present application may claim the disclosed subject matter in different ways, and may generally include any set of one or more limitations as variously disclosed or otherwise presented herein.

Claims

1. A method of digitally facilitating regulatory decision making for at least elements of a standardized digital solution for characterizing a disease, the method comprising: providing a digital environment comprising a plurality of elements of a standardized digital solution, wherein the digital environment is accessible to a plurality of collaborators; obtaining, at a processor, a central data structure, wherein the central data structure includes compiled evidence of a plurality of elements of one or more previously approved standardized digital solutions; receiving, at the processor, a request for a new standardized digital solution; generating, at the processor, the new standardized digital solution by utilizing one or more of the plurality of elements of the one or more previously approved standardized digital solutions in the central data structure; receiving, at the processor, additional data for the new standardized digital solution from a collaborator of the plurality of collaborators; updating, at the processor, the central data structure to incorporate the additional data; receiving, at the processor, feedback from a regulatory collaborator, wherein the received feedback includes regulatory acceptance of at least elements of the new standardized digital solution; and The received feedback from the supervisory collaborator is digitally recorded. 2 . The method of claim 1 , wherein the one or more previously approved standardized digital solutions include measurement data collected via a medical device.

3. The method of claim 1, wherein the processor is further configured to generate a plurality of new standardized digital solutions for different patient populations.

4. The method of claim 1, wherein the elements of the one or more previously approved standardized digital solutions are included in one or more of a validation layer, a measurement instrument layer, or a definition layer of the central data structure.

5. The method of claim 1, wherein generating the new standardized digital solution comprises analyzing evidence data of the one or more previously approved digital solutions and generating the new standardized digital solution based at least on an acceptability classification of the evidence data.

6. The method of claim 5, wherein the acceptability classification is determined based on one or more of: freshness of the evidence data, quantity of the evidence data, patient population of the evidence data, regulatory acceptance of the evidence data, clinical validation of the evidence data, or technical verification of the evidence data, content validity of the evidence data, construct validity of the evidence data, sensitivity of the evidence data to change, or a combination thereof.

7. The method of claim 1, wherein generating a new standardized digital solution is based on a medical condition or a diagnostic method.

8. The method of claim 1, wherein the new standardized digital solution comprises a diagnostic specific algorithm.

9. The method of claim 1, wherein the elements of the new standardized digital solution include any of the following: Measurable concepts of components of interest; Measurement method elements; Raw data element; Algorithm components; Dataset component; Technology verification components; Analyze and verify components; Clinically validated components; or Regulatory elements.

10. The method according to any one of claims 1 to 9, further comprising: prior to receiving the request, generating a task within the digital environment by establishing standardized roles for two or more of the plurality of collaborators, Wherein each of the two or more collaborators is an independent stakeholder in the task.

11. A method according to claim 10, wherein establishing roles for the two or more collaborators includes assigning a primary role to a first collaborator among the two or more collaborators and assigning a secondary role to a second collaborator among the two or more collaborators, and wherein the primary role has additional permissions compared to the secondary role, wherein the permissions include permissions to access, view and / or use results developed from the task.

12. The method according to claims 10 to 11, wherein a collaborator of the two or more collaborators is assigned to any of the following roles: funder, observer, technology provider or data partner.

13. The method according to any one of claims 10 to 2, further comprising: Prior to receiving input from one or more of the two or more collaborators to define at least the elements of the standardized digital solution, receiving an agreement on one or more standards for the task from each of the two or more collaborators, wherein the one or more standards for the task include terms and conditions of the agreement.

14. The method according to claim 13, further comprising: prior to receiving an agreement from each of the two or more collaborators, generating a predefined template identifying the one or more criteria for the task; and providing the predefined template to each of the two or more collaborators.

15. The method according to any one of claims 1 to 14, wherein the new standardized digital solution is a digital measurement solution, and wherein the digital measurement solution comprises: a measurement definition defining one or more concepts of interest associated with a disease; A measurement instrument resource configured to transform measurements of interest captured according to the measurement definition into a data set providing information for characterizing the disease, wherein the measurement instrument resource of the digital measurement solution is dedicated to a device used to capture the measurements of interest.

16. The method according to any one of claims 1 to 15, wherein the new standardized digital solution is a target solution profile representing a common class of digital measurement solutions; and The target solution configuration file includes: a measurement definition defining one or more concepts of interest associated with a disease; A measurement instrumentation resource configured to transform data captured according to the measurement definition into a data set, the measurement instrumentation resource being device technology agnostic and therefore interchangeable between different target solution profiles.

17. The method of claim 1, wherein the digital environment is a collaborative platform built based on a measurement stack model.

18. The method of claim 1, wherein the received input includes one or more of the following types of information: technical feasibility information, patient perspective information, and clinical information.

19. The method according to claim 10, wherein the method further comprises: generating an agreement based on one or more of: shared interests, regulator collaborators, regulatory decisions, and established roles of the two or more collaborators, wherein the agreement includes terms including one or more of: task rules and collaborator permissions; providing the agreement to each of the two or more collaborators via the digital environment; as well as A framework is provided via the digital environment for one or more collaborators to edit the terms.

20. The method of any one of claims 1 to 19, further comprising generating a dossier for use in seeking regulatory decision making, the dossier relating to at least an element of the new standardized digital solution.

Images