Electronic preference-based measurement with reduced communication overhead

Abstract

A system, method, and computer-readable medium storing instructions for conducting and analyzing results from a survey are disclosed. The survey permits respondents to take the survey and to view results with little to no communication with a remote server. This ability to obtain survey results is relevant to environments with little to no bandwidth connecting a user's device to a remote server. To take the survey, a respondent identifies his health state by modifying the severity levels of a predetermined quantity of fields. Next, the respondent orders the severity levels by comparing the levels to the levels to the remaining levels of the other fields. Using the combination of the original health state and the order in which the respondent quantified the levels against other levels, additional health states are postulated and a clearer identification of the state of health of the individual may be obtained.

Description

ELECTRONIC PREFERENCE-BASED MEASUREMENT WITH REDUCEDCOMMUNICATION OVERHEADTECHNICAL FIELD

[0001] Aspects of the disclosure relate generally to exchanging information between networked devices.BACKGROUND

[0002] With the advance of modem medicine, health care has evolved from physician-centered analyses to patients-centered analyses. Improving how medical providers, among other service providers, obtain information from their patients is a long-felt need in the medical field. Medical providers often obtain a majority of their information regarding a patient's condition directly from the patient in response to the medical providers' questions. While a discussion of the patient's condition with the patient would appear to be removed from bias, patients are prone to adapting their responses to new questions based on their previous responses. Similarly, patients often employ various coping strategies to alleviate feelings of discomfort or stress while discussing their medical condition (e.g., sometimes referred to as the "white lab coat syndrome"). Those adaptive responses and coping strategies often hamper the medical professional's task of obtaining a reliable and valid assessment of the patient's condition.Patient-Reported Outcome Measure (PROM)

[0003] A patient-reported outcome (PRO) or patient-reported outcome measure (PROM) is any assessment coming directly from patients, without interpretation by physicians or others, about how they function or feel in relation to their health condition. The term PROM encompasses a broad spectrum of outcomes that include the symptoms of a disease or the side effects of a treatment (e.g., fatigue, pain, or low mood), functions (e.g., social activities, cognitive functioning, or physical abilities), and even multidimensional constructs, including health-related quality of life (HRQoL) or perceived health status. Evidence shows that the use of information from PROMs contributes to better communication and decision-making between doctors and patients and improves patient satisfaction with health outcomes and care.

[0004] One tool medical professionals have started using to reduce the effects of adaptions and coping-related issues is to use machine -based questionnaires to obtain an assessment of the patient's condition independent of an actual interview with the patient. These machine -based questionnaires - also known as electronic Patient-Reported Outcome Measures (ePROMs) - provide additional benefits including, but not limited to, the following: monitoring patients with medium-to-chronic diseases from a distance; improving treatment to patients; permitting patients to become more actively involved in their treatments; and reducing costs by limiting in-person, telephone, and telepresence visits).

[0005] PROM and ePROMs can be developed within various measurement frameworks. One dominant framework is questionnaire-based, with these instruments measuring the intensity or level of specific health aspects through a bundle of items. However, when comparing health outcomes across different populations, conducting disease modeling studies, or performing economic evaluations of various health-care interventions, preference-based PROMs are more useful. Preference-based PROMs express outcomes in a single metric number (“value”). The PROMs' “value” is determined by incorporating weights that reflect the relative importance attached to health items. Based on these weights, an overall “value” for a health state can be generated.

[0006] Existing advanced measurement methods in the social sciences (e.g., psychology, economics, marketing) and health sciences (e.g., health economics, clinimetrics) are all preference-based and all based on one of the three fundamental measurement models for subjective phenomena: item response theory (1RT), discrete choice experiments (DCEs), and valuation techniques. Such measurement models are used to quantify phenomena such as attitudes, perceived health, intelligence, and consumer preferences. Preference values (variously called utilities, strengths of preferences, indices, or weights) that these methods generate are preferably assumed to be unidimensional on a linear scale so that differences between values of assessment of respondent states can be assumed to correspond to increments of difference in quality of these states, which implies that the values should be interval-level or cardinal data. Thus, the differences between values indicate true differences (e.g., if a patient’s value of his / her health status increases from 40 to 60, this increase is the same as an increase from 70 to 90). Apreference-based measurement can be very convenient because it produces one overall numerical value, which makes analyzing and interpreting results a straightforward procedure.

[0007] A special measurement framework is based on the MAPR measurement model (which belongs to the class of probabilistic choice models). These probabilistic choice models can establish the relative merit (value) of a subjective phenomenon. These models are indirect, producing measures using the metric scale (analogous to a yardstick). For all probabilistic choice models, respondents must perform preference-based tasks in a particular way to endorse a specific response. This then generates data for analysis in accordance with the measurement model. The core of a preference-based task in these probabilistic measurement frameworks consists of a response task that compares at least two objects intending to express which object is most preferred (is better). From a technical perspective, these models group ordinal data obtained from respondents. The grouped data are then aggregated to infer an interval scale (metric measure: value) that is based on a mathematical (measurement) model.

[0008] The probability of observing a specific ranking can be written as the product of such terms, representing a sequential decision interpretation, in which the respondent first chooses the most preferred alternative, and then the most preferred alternative among the rest, etc.

[0009] The methods of these models are all based on prespecified complex experimental research designs or complex mathematical computations controlled by a central server / computer. During the assessment tasks performed by individual respondents, connection to a central server / computer is required to transmit and receive data (new items, responses, additional new items, etc.) in consecutive steps. Based on each answer from a respondent, a central computer running a complex algorithm processes a received answer and determines which next question to ask. In short, individual responses are sent in real-time to the central server. The individual then waits until the central computer has determined a new question and waits until that question is received on the individual's device. This process continues until the end of the survey is reached. To prevent user frustration waiting for the next question to appear, a robust network connection (often an Internet connection) is required to interact with thecentral server / computer with no delay. In short, for a conventional survey to be effective, it must be supported by a system (hardware and network) with low network latency. An issue with conventional surveys is that the demanding bi-directional communications required to implement those surveys result in more communications being handled by networks. In the aggregate, the quantity of bi-directional communications for widespread questionnaires / instruments may result in increased network latency for the implementation of those questionnaires / instruments. Aspects of the disclosure are directed to improving how surveys are generated and processed to reduce reliance on computer networks, thereby reducing network requirements for an immediate survey as well as reducing latency for other network communications occurring at the same time a given survey is being conducted.

[0010] Surveys exist to obtain information from respondents. In a fully networked, high- bandwidth environment, processing for the surveys may be consolidated to one or more servers. However, in environments with little connectivity or in high network traffic environments, expecting fast and consistent communications with the servers is not possible. In some situations, a user may be unable to start a survey based on the inability to obtain survey information from one or more servers. In other situations, a user may be unable to complete the survey based on the lack of ability to communicate with the servers when needing to do so. In yet further situations, the user's results may not be readily accessible at the completion of the survey based on the lack of connectivity or exceedingly heavy network traffic.

[0011] In a general scenario in preparation for a physician visit, a patient may be provided with a computing device (e.g., a computing tablet executing computer instructions to display information on a screen and receive a user's responses) in the physician's waiting room. While a goal is for the patient to complete the survey in a short period of time and return the computing device to an attendant. Most patients may find completing a survey to be an easy task despite sitting in a public waiting room and completing the survey on a public computing device. However, other patients may become frustrated with needing to concentrate in a distracting environment or uncomfortable touching a computing device that was handled by others. Further, some physicians may be reluctant to suggest patients complete surveys on a computing tabletor other portable computing platform based on the nature of their practice including infectious disease specialists (where their patients may be harboring infectious diseases), oncologists and organ transplant specialists (where their patients likely have compromised immune systems), and orthopedists (where their patients may lack the dexterity or ambulatory ability to carry and / or manipulate a computing device without assistance). As such, some patients and / or physicians may feel more comfortable with the patients being able to complete surveys with their own devices in the waiting room or even before arriving at the physicians' offices.

[0012] To the extent that patients or physicians prefer to complete surveys on patient-supplied computing devices, a particular technical issue is the inability to know a priori the display size of any particular user's computing device. While some patients may prefer completing the survey on a full-size monitor (e.g., with a diagonal dimension of 36 cm or more), other patients may attempt to complete the surveys on smartphones having screen diagonals of less than 12 cm. Based on the limited screen diagonals, complex surveys are not practically conductible on smaller smartphones.SUMMARY

[0013] The following presents a simplified summary of various aspects described herein. This summary is not an extensive overview and is not intended to identify key or critical elements or to delineate the scope of the claims. The following summary merely presents some concepts in a simplified form as an introductory prelude to the more detailed description provided below.

[0014] Aspects described herein may address these and other problems, and generally improve how data may be provided to users and information from users provided to computing systems.

[0015] A system, method, and computer-readable medium storing instructions for conducting and analyzing results from a survey are disclosed. The survey permits respondents to take the survey and to view results with little to no communication with a remote server. This ability to obtain survey results is relevant to environments with little to no bandwidth connecting a user's device to a remote server. To take the survey, a respondent identifies his health state by adjusting the respective severity levels of a predetermined quantity of fields. Next, the respondent orders the severity levels bycomparing the levels to the levels to the remaining levels of the other fields. Using the combination of the original health state and the order in which the respondent quantified the levels against other levels, additional health states are postulated and a clearer identification of the state of health of the individual may be obtained.

[0016] In additional aspects, providing the analysis to the patient, without requiring additional processing by a remote server, improves the overall experience for the patient in as much as the patient will be able to see his answers.

[0017] In further aspects, an improved user interface is provided for display on computing devices having limited available screen real estate. As described herein, a process is disclosed for monitoring how a user interacts with specific functional user interface elements and, based on the content displayed in the specific functional user interface elements and the order of interactions with those specific functional user interface elements, predicting an order of severity of multiple conditions.

[0018] In one or more aspects, a computer-implemented method may comprise: generating a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the levels relate to different intensity levels; displaying, on a display of a user computing device, the first user interface; receiving, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; storing, based on the first user interactions, the first selected levels for the items; generating, without communicating with a remote server and based on the first selected levels, a second user interface; displaying, on the display, the second user interface; receiving, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculating, based on the order, additional second selected levels; generating, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generating, based on the ranking, a third user interface with the first selected levels storing the ranking.

[0019] The computer-implemented method may further comprise displaying, based on the ranking, the third user interface on the display of the user computing device. Thecomputer-implemented method may further comprise receiving, from a remote server, a plurality of items and available levels. The first user interface with the items may be based on the plurality of items. The two or more selectable levels may be based on the available levels.

[0020] The receiving of the first selected levels may comprise receiving, for a first item of the items, a first selected level. Calculating the additional second selected levels may comprise increasing, based on the order, the first selected level of the first item to a first higher level. The ranking may be further based on the first higher level. The first higher level may be two or more levels higher than the first selected level. The computer-implemented method may further comprise receiving, for a last item of the items, a first selected level. Calculating the additional second selected levels may comprise decreasing, based on the order, the first selected level of the last item to a first lower level. The ranking may be further based on the first lower level. The first lower level may be two or more levels lower than the first selected level.

[0021] In one or more examples, an apparatus according to the disclosed aspects may comprise a display; one or more processors; and a memory storing instructions that, when executed by the one or more processors, control the apparatus to: generate a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the levels relate to different intensity levels; display, on a display of a user computing device, the first user interface; receive, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; store, based on the first user interactions, the first selected levels for the items; generate, without communicating with a remote server and based on the first selected levels, a second user interface; display, on the display, the second user interface; receive, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculate, based on the order, additional second selected levels; generate, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generate, based on the ranking, a third user interface with the first selected levels storing the ranking.

[0022] In one or more aspects, a computer-readable medium may store instructions for implementing various methods described herein.

[0023] These and other aspects are described in relation to the attached figures and detailed description.BRIEF DESCRIPTION OF DRAWINGS

[0024] The present disclosure is illustrated by way of example and not limited to the accompanying figures in which like reference numerals indicate similar elements and in which:

[0025] Figure 1 depicts an example of a computing device and system architecture that may be used in implementing one or more aspects of the disclosure in accordance with one or more illustrative aspects discussed herein;

[0026] Figure 2 depicts a block diagram of an environment in which systems and / or methods described herein may be implemented;

[0027] Figure 3 shows an example user interface for a first task of a survey process;

[0028] Figure 4 shows an example user interface for a second task of a survey process;

[0029] Figure 5 shows a comparison between user interfaces between the first survey process and the second survey process;

[0030] Figure 6A shows a sample order of interactions with a user interface of the first survey process. Figure 6B shows an example of how a user's interactions with the first survey process are postulated;

[0031] Figure 7 shows an example of how a user's interactions with the first survey process are postulated in a sample order;

[0032] Figure 8 shows network communications associated with a first survey process;

[0033] Figure 9 shows network communications associated with a second survey process;

[0034] Figure 10 shows network communications associated with a third survey process;

[0035] Figure 11 shows network communication delays and other processing delays associated with the first survey process, the second survey process, and the third survey process;

[0036] Figure 12A shows sample network communication data size comparisons between the first survey process, the second survey process, and the third survey process. Figure 12B shows a comparison between the Drop-Down method and the Better / Worse method in the number of responses requested from the patient and the number ofelements changing across the displayed pages;

[0037] Figure 13 shows a graph displaying coefficients of the described survey processes;

[0038] Figure 14 shows a bar chart for the described survey process;

[0039] Figures 15A, 15B, 15C, and 15D show various user interfaces;

[0040] Figures 16A, 16B, and 16C show various user interfaces;

[0041] Figure 17 shows a user interface; and

[0042] Figure 18 shows another user interface.DETAILED DESCRIPTION

[0043] In the following description of the various embodiments, reference is made to the accompanying drawings, which form a part hereof, and in which are shown by way of illustration various embodiments in which aspects of the disclosure may be practiced. It is to be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope of the present disclosure. Aspects of the disclosure are capable of other embodiments and of being practiced or being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. Rather, the phrases and terms used herein are to be given their broadest interpretation and meaning. The use of “including” and “comprising” and variations thereof is meant to encompass the items listed thereafter and equivalents thereof as well as additional items and equivalents thereof. Any sequence of computer- implementable instructions described in this disclosure may be considered to be an "algorithm" as those instructions are intended to solve one or more classes of problems or to perform one or more computations. While various directional arrows are shown in the figures of this disclosure, the directional arrows are not intended to be limiting to the extent that bi-directional communications are excluded. Rather, the directional arrows are to show a general flow of steps and not the unidirectional movement of information. In the entire specification, when an element is referred to as "comprising" or "including" another element, the element should not be understood as excluding other elements so long as there is no special conflicting description, and the element may include at least one other element. In addition, the terms "unit" and "module", for example, may refer to a component that exerts at least one function or operation, andmay be realized in hardware or software, or may be realized by a combination of hardware and software executing on the hardware. In addition, terms such as " unit" and " module" described in the specification mean a unit for performing at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. Throughout the specification, the expression "at least one of a, b, and c" may include 'a only', 'b only', 'c only', 'a and b', 'a and c', 'b and c', and / or 'all of a, b, and c'.

[0044] By way of introduction, aspects discussed herein may relate to methods and techniques for collecting information from a user and analyzing the information to identify responses of higher significance than others. Further, the collection process reduces the quantity of information requested from the user, thereby making the collection process significantly easier to implement in a low bandwidth environment.

[0045] In general, a user is provided with two tasks: the first task is to identify the user's current state; and the second task is to order the selections of the first task in an order of prominence (e.g., severity). The system may use the results of these two tasks to postulate which of the selections of the first task are of higher importance than others.

[0046] Mobile devices, with their small screens, present challenges in displaying information typically suited for larger screens, like those of desktop computers. The smaller size of the screens of mobile devices necessitates segmenting information into smaller units for display. Too often, user experiences with their accompanying interfaces that were designed to be displayed only on larger screens are merely modified to fit on smaller screens, thus complicating effective organization and navigation. For example, what worked well with more screen real estate is frustratingly clunky and / or error-prone when minimized. The smaller screen sizes make it difficult for developers to effectively organize information and help users navigate to and from the information they want. As described here, functional user interface elements are identified and displayed. Based on user interaction with the functional user interface elements, the content associated with the functional user interface elements changes and modifies the selectable options on the user interface. For the purposes of explanation, the modifiable functional user interface components are described with respect to user interfaces for use in measuring subjective phenomena (such as health). Based on the order ofinteractions and how the regions of the functional user interface components change, new measurement methods using the disclosed user interface and order of interactions with the user interface provide permit measurement methods that require fewer interactions and simpler assessment tasks. One or more of the following may be used: A) Displaying all items on a single screen for patient self-assessment (Task 1) with interactive response categories. In contrast, conventional user interfaces forced separate items to be displayed on separate screens; B) Utilizing the MAPR measurement model, allowing patients to compare their health state with other health states on a single mobile device (Task 2). Previous approaches compared pairs of heath states; C) using common user interface components for Tasks 1 and 2. In contrast, previous approaches forced the users to interact with different types of functional components (thus requiring them to learn one user interface for Task 1 and another user interface for Task 2). Using common user interface components enhances user comprehension of the tasks and reduces response times; and D) selective highlighting of only active items permits a simpler and faster process. For ease of explanation, the disclosed functional user interface items and / or regions are described as displayed items, regions, buttons, and the like. While these brief descriptions are used to identify the appearances of the functional user interface items, these brief descriptions are not intended to diminish the capabilities of the functional user interface items or their uses.

[0047] Before discussing these concepts in greater detail, however, several examples of a computing device that may be used in implementing and / or otherwise providing various aspects of the disclosure will first be discussed with respect to Figure 1. Figure 1 illustrates one example of a computing device 101 that may be used to implement one or more illustrative aspects discussed herein. For example, the computing device 101 may, in some embodiments, implement one or more aspects of the disclosure by reading and / or executing instructions and performing one or more actions based on the instructions. In some embodiments, the computing device 101 may represent, be incorporated in, and / or include various devices such as a desktop computer, a computer server, a mobile device (e.g., a laptop computer, a tablet computer, a smart phone, any other types of mobile computing devices, and the like), and / or any other type of data processing device.

[0048] The computing device 101 may, in some embodiments, operate in a standalone environment. In others, the computing device 101 may operate in a networked environment. As shown in Figure 1, various network nodes / devices 101, 105, 107, and 109 may be interconnected via a network 103 , such as the Internet. Other networks may also or alternatively be used, including private intranets, corporate networks, LANs, wireless networks, personal networks (PAN), and the like. Network 103 is for illustration purposes and may be replaced with fewer or additional computer networks. A local area network (LAN) may have one or more of any known LAN topologies and may use one or more of a variety of different protocols, such as Ethernet. Computing devices 101, 105, 107, 109, and other devices (not shown) may be connected to one or more of the networks via twisted pair wires, coaxial cables, fiber optics, radio waves, or other communication media. Additionally or alternatively, the computing device 101 and / or the network nodes / devices 105, 107, and 109 may be a server hosting one or more databases.

[0049] As seen in Figure 1 , the computing device 101 may include a processor 111, RAM 113, ROM 115, network interface 117, input / output interfaces 119 (e.g., keyboard, mouse, display, printer, etc.), and memory 121. Processor 111 may include one or more computer processing units (CPUs), graphical processing units (GPUs), and / or other processing units such as a processor adapted to perform computations associated with database operations. Input / output interface 119 may include a variety of interface units and drives for reading, writing, displaying, and / or printing data or files. Input / output interface 119 may be coupled with a display such as display 120. Memory 121 may store software for configuring computing device 101 into a special-purpose computing device in order to perform one or more of the various functions discussed herein. Memory 121 may store operating system software 123 for controlling the overall operation of the computing device 101, control logic 125 for instructing the computing device 101 to perform aspects discussed herein, database creation and manipulation software 127, and other applications 129. Control logic 125 maybe incorporated in and may be part of database creation and manipulation software 127. An integrated display is provided as display 130 as part of computing device 101. In other embodiments, the computing device 101 may include two or more of any and / or all of these components(e.g., two or more processors, two or more memories, etc.) and / or other components and / or subsystems not illustrated here.

[0050] Devices 105, 107, and 109 may have similar or different architecture as described with respect to the computing device 101. Those of skill in the art will appreciate that the functionality of the computing device 101 (or device 105, 107, 109) as described herein may be spread across multiple data processing devices, for example, to distribute processing load across multiple computers, to segregate transactions based on geographic location, user access level, quality of service (QoS), etc. For example, computing devices 101, 105, 107, 109, and others may operate in concert to provide parallel computing features in support of the operation of control logic 125 and / or software 127.

[0051] One or more aspects discussed herein may be embodied in computer-usable or readable data and / or computer-executable instructions, such as in one or more program modules, executed by one or more computers or other devices as described herein. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types when executed by a processor in a computer or other device. The modules may be written in a source code programming language that is subsequently compiled for execution or may be written in a scripting language such as (but not limited to) Python or JavaScript. The computer-executable instructions may be stored on a computer-readable medium such as a hard disk, optical disk, removable storage media, solid-state memory, RAM, etc. As will be appreciated by one of skill in the art, the functionality of the program modules may be combined or distributed as desired in various embodiments. In addition, the functionality may be embodied in whole or in part in firmware or hardware equivalents such as integrated circuits, field programmable gate arrays (FPGA), and the like. Particular data structures may be used to more effectively implement one or more aspects discussed herein, and such data structures are contemplated within the scope of computer-executable instructions and computer-usable data described herein. Various aspects discussed herein may be embodied as a method, a computing device, a data processing system, or a computer program product. Having discussed several examples of computing devices that may be used to implement some aspects asdiscussed further below, the discussion will now turn to a method for conducting a survey with limited communications with a remote server.

[0052] Figure 2 is a block diagram of an environment in which systems and / or methods described herein may be implemented. As shown in Figure 2, the environment may include servers 201 and 202 and a computing device 203 connected by a network 204. The devices, servers, and network may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections. The server 201 may be directed toward receiving files relating to activities from computing device 203 and then sending the files to server 202 for processing. The server 201 may further include a processor 205 and storage 206. The server 202 may further include a processor 207 and storage 208. The computing device 203 may further include a processor 209 and storage 210.

[0053] The network 204 may include one or more wired and / or wireless networks. For example, network 204 may include a cellular network (e.g., a long-term evolution (LTE) network, a code division multiple access (CDMA) network, a 3G network, a 4G network, a 5G network, another type of next-generation network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., the Public Switched Telephone Network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber optic-based network, a cloud computing network, or the like, and / or a combination of these or other types of networks.

[0054] The number and arrangement of devices and networks shown in Figure 2 are provided as an example. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown in Figure 2. Furthermore, two or more servers shown in Figure 2 may be implemented within a single server, or a single server shown in Figure 2 may be implemented as multiple, distributed servers or in a cloudbased computing environment. Additionally, or alternatively, a set of devices (e.g., one or more devices) of the environment may perform one or more functions described as being performed by another set of devices of the environment. Network 204 may be represented as a single network but may comprise combinations of other networks orsubnetworks. In one or more examples, a data stream (not shown) may be received by server 201, where server 201 is a data store for received survey information. The data stream may also be received by server 202. Server 202 may also store the new survey information as well. The servers 201 and 202 may extract information from the data stream or streams, create survey items, and store one or more survey templates in their respective storages. The application on computer device 203 may request information survey information from one or more of the servers 201 and / or 202 and the survey started.Mobile Application

[0055] One or more aspects described herein relate to an electronic patient-reported outcome measure (ePROM) that uses software to run a mobile application to obtain user responses to a survey. The mobile application may provide immediate analyses of the user's results. As part of the measurement model (multi-attribute preference response: MAPR), users perform two distinct tasks in the mobile application. The first produces a description (health state) of the patient’s current health condition (health status), and the second elicits the patient’s preference responses concerning his / her health status.

[0056] The MAPR model is a generic statistical model that is based on the input from many respondents who estimated their relative positions in relationship to states in which other respondents are or may be. Preference responses (whether respondents consider their own situation / condition better or worse than alternative situations / conditions) are collected in interaction with a user interface controlled by the computer program product being executed and sent to the central server. On the central server, a computer program processes the received data in accordance with the MAPR model structure.

[0057] Preference values (variously called utilities, strengths of preferences, indices, or weights) that these methods generate are preferably assumed to be unidimensional on a linear scale so that differences between values of assessment of respondent states can be assumed to correspond to increments of difference in quality of these states, which implies that the values should be interval-level or cardinal data. Thus, the differences between values indicate true differences (e.g., if a patient’s value of his / her health status increases from 40 to 60, this increase is the same as an increase from 70 to 90). Preference-based measurement can be very convenient because it produces one overallnumerical value, which makes analyzing and interpreting results a straightforward procedure.MAPR Measurement Model: Response TasksTask 1: Base State

[0058] Figure 3 shows an example of user interfaces and underlying processes to establish a base state. A user operates a user interface (e.g., referred to herein as "CS-Base User Interface" or just "CS-Base") to establish an initial base state. The "CS-Base" user interface is described in this section as well as in additional sections below. For simplicity of explanation and for example, the base state being obtained is a health state. One of ordinary skill in the art would readily appreciate the base state may be directed to any base state and not specifically related to a patient's health. Alternatively, depending on the type of PROM and its content, particular base health states may be obtained. The CS-Base User Interface is one of a plurality of ePROMs that may be used. For instance, other ePROMs may be directed specifically to particular health areas including cardiovascular, pulmonary, transplantation, prosthetics, and the like. Where the CS-Base user interface is used for establishing a base health state and for each health item in the CS-Base user interface, the mobile application depicts an interactive box all together on the same screen. When the patient clicks on the box for a specific health item, it rotates, displaying the response options. For example, a user may start with an initial state (e.g., a first page of a user interface 301) as shown in Figure 3. The user may modify five of the options in page 302 (e.g., "some problems with mobility", "good vision", "poor hearing", "not anxious", and "a little pain"). The completed set of selections is shown, for instance in the modified user interface of page 303.

[0059] More particularly, Figure 3 shows an example user interface for task 1 of the survey process. A first displayed page (first page of the user interface) 301 includes a plurality of regions identifying items for which a user will be requested to specify a particular level of severity. As used herein, the severity level may also be expressed as a priority level, an intensity level, a burden, and / or items that hinder or disturb the patient the most. Examples of the regions may include items relating to mobility, vision, hearing,cognition, mood, anxiety, pain, fatigue, social functioning, daily activities, self-esteem, and independence. 12 regions are provided for selection in the user interface 301. Regions may be added or subtracted as desired. The user interacts with one or more of the regions of user interface 301 to specify a level of severity (e.g., in the case of CS- Base, four levels of intensity may be used. A different quantity of levels may be made available as desired). In the example of Figure 3, page 302 of the user interface shows that the user has modified the following regions: mobility (selecting "some problems with mobility"); vision (selecting "good vision"); hearing (selecting "poor hearing"); cognition (selecting "no cognitive problems"); and mood (selecting "good mood"). Here, "hearing" has been modified to a second level of severity (e.g., "poor hearing"). In one or more examples, the various selected levels may be identified as an intensity bar (e.g., in length, position, and / or color) associated with each of the selectable regions. With respect to page 303, all regions have been interacted with at least once. If at least some fields are interacted with, the user may be presented with the option to proceed to task 2 of the survey.

[0060] The actual interactions with the user interface may be visually emphasized to improve how the information is displayed on the screen. For instance, when a patient selects the “Hearing” box of user interface 301 of Figure 3, the displayed options may shift (e.g., rotate) to offer the response option “Some hearing problems” (Level 1). After selecting the box again, the display shifted to “Poor hearing” (Level 2). Other options may also be provided. For instance, for "Fatigue", the option may include "Not tired" (Level 1), "Little tired" (Level 2), “Quite tired” (Level 3), and “Very tired” (Level 4). Patients rate their current health status by rotating the boxes to show the best-fit descriptions in all boxes of page 303 of Figure 3. The CS-Base user interface may permit the local application to generate a description of a patient’s overall state of health expressed as 12 digits (e.g., 213111212221, or even worse 214111212331). Patients can also click information points beside the health items to access explanations.Task 2: Preferences

[0061] After Task 1, patients performed a second task (Task 2), based on their descriptions of their own health states (Task 1) according to the same CS-Base. Task 2 requires them to make trade-offs and to provide selections regarding which of the displayed intensitylevels causes the greatest burden to the patient. Two different methods for establishing preference are described herein: Better / Worse (BW) and Drop-Down (DD). For the Better / Worse method, a full description is provided in U.S. Patent No. 11,631,476 to Dr. Paul F. M. Krabbe.Drop-Down (DD) Method

[0062] For the Drop-Down method, Figure 4 shows an example user interface for task 2 of a survey process. Upon selecting a "next" functional user interface 410 from displayed page 401 (showing the result from task 1), the user is presented with various pages (including pages 402, 403, and 404 and other interstitial pages). On the various pages, the user is requested to select, of the identified items in the user interface, which health- related aspect hinders the patient the most. As shown on page 402, is requested to identify the item that is causing the most significant issue for them. The user selected "poor hearing" and that selection is then shown on page 403 as the highlighted user interface element of the third region 405 with a reduced intensity level. Here, the level "impaired hearing" is less intense than "poor hearing" as originally selected by the user during Task 1. In particular, the "poor hearing" content is replaced with a next lower level of that aspect and displayed, on page 403, as "impaired hearing" in the third region 405. The user continues identifying the most hindering of the remaining displayed aspects. To aid the user, the relevant selectable fields may be identified (e.g., highlighted) while the fields with no lower-level aspects may be grayed out (e.g., not selectable). As shown on page 404 (the final page of Task 2), the first region 406 "some problems with mobility" was deprecated to "no problems with mobility", the third region 405 "poor hearing" was deprecated to "impaired hearing" (carrying over the modification from page 403), and left at "impaired hearing" as shown on page 404. The seventh region 407 "a little pain" was deprecated to "no pain" as shown on page 404. The ninth region 408 "some problems with social functioning" was deprecated to "no problems with social functioning" as shown on page 404. The tenth region 409 "some problems with daily activities" was deprecated to "no problems with daily activities" as shown on page 404. The remaining regions, for instance, the "good self-esteem" region 414, were not modified between pages 402 and 404. The status of Task 2 is shown in the functional user interface regions 411, 412, and 413 (e.g., clickable buttonsthat change based on the status of Task 2).

[0063] Figure 5 shows a comparison between the Drop-Down (DD) method for performing Task 2 of Figure 4 and a Better / Worse method for performing Task 2. The patient’s own health state from Task 1 501 is presented on a user interface. For the DD method (Figure 5, pages 503A, 503B, and 503C), the patient is asked to select the item, of the displayed items (with a suboptimal level / intensity level: 2, 3, or 4), that hindered or disturbed them the most. By clicking or swiping (dropping down via user interaction with the displayed user interface), a selected item is shifting one level lower in intensity (e.g., better). Each drop-down of an item produced a health state ranked better than the initial health state from Task 1 501. (There had to be at least two items with levels > 1, otherwise the choice was predetermined. Items at Level 3 or higher could be dropped down more than once).

[0064] In the DD method, patients make trade-offs between the levels of multiple items (i.e., "is Level i of an item x worse than any level of another item?"). In contrast to the Better / Worse method, patients do not have to make trade-offs between their own and an alternative health state. Patients used the drop-down option up to a maximum quantity of levels, with each drop-down producing a different (better) ranked health state. In the example of Figures 4 and 5, five levels were useable. However, greater or fewer quantities of levels may be made available. For each patient, therefore, the ranking for the states could range from 1, 2, 3 (with at least 2 items having suboptimal levels: 2 drop downs) and 1 , 2, 3, 4, 5, 6 (in case of 5 drop-downs). The lowest or worst ranking was coded as 1 , representing the patient’s actual health state. With the new task 2, the DD method is able to determine, for an individual patient, which health aspects are most important for an individual patient (and correspondingly relevant information for medical doctors).Better / Worse (BW) Method

[0065] For the Better / Worse method for Task 2 as shown in pages 502A, 502B, and 502C of Figure 5, patients compared their own health states (Task 1, step 501) to computergenerated, slightly different alternative health states (shown in pages 502A, 502B, 502C) and determine whether their own health state is better or worse than the alternative health state. The alternative health states differed from their own healthstates by a predetermined and limited number of items (for example, two) that had been altered. Patients could thus regard these alternative states as hypothetical states. One of the items represented an improvement of one level compared to the patient’s actual health state (one level lower, depicted as a lightly shaded box). The other item represented a reduction of one level compared to the patient’s actual health state (one level higher, depicted as a darker shaded box). For example, on Task 1 (501), a patient- reported being “Not tired” for the “Fatigue” condition and having “Some problems with daily activities” for the “Daily activities” condition. These two health items were altered into “A little tired” and “No problems with daily activities” to construct an alternative health state in the subsequent Task 2 (See Figure 5, 502A, Better / Worse).

[0066] The assumption is that a one-level improvement on one item is not necessarily the same difference as a one-level decline on another item. For example, patients may have appraised differences between the levels of distinct items in different ways. The patients may have been asked, “Please indicate whether your health is better or worse than the health description below.” The generation of these alternative health states may have been based on a flexible randomization algorithm (number of alternative states, number of items to vary, colors) that was built into the software. The task essentially called for the patients to internally make a trade-off between their own health state and other alternative states in a paired comparison, and then to select either their own health state or the alternative health state as better (i.e., 1 if preferred and 0 if not).

[0067] In one example, patients compared five alternative health states with their own health state. For each patient, therefore, there were five sets of the most basic ranking, namely of two health states: 1,2 (coded as 1,0 for the statistical software).MAPR Measurement Model: Statistical Models

[0068] Like all probabilistic measurement models, the MAPR measurement model uses an indirect form of measurement. The data generated by the preference methods are not measures. Ordinal response data (ranks) obtained from the preference methods are aggregated to estimate coefficients based on a mathematical (measurement) model. Subsequently, the coefficients are used to compute values for the health states. These values are the measures. As used herein, the terms "coefficient" and "weight" may beused in place of each other or, as described in a particular section, refer to values that are applied as weighting factors to adjust outcomes.

[0069] The mathematical model consists of a latent (hidden) variable (the metric scale) and a set of manifest (observable) variables (i.e., the items of the CS-Base). Such models have a long history, commencing with the model developed by Louis Thurstone in 1927. Other researchers have introduced extensions to the basic Thurstonian model.

[0070] For all probabilistic measurement models, respondents must perform assessments (preliminary phase of information processing prior to making a judgment) and judgments (choice in favor of something) in particular ways to endorse specific responses. This then generates data for analysis in accordance with the measurement model. Within these probabilistic measurement frameworks, the assessment consists of comparing at least two objects (i.e., health states or set of health items), with the aim of expressing which object is preferred (i.e., better). Therefore, the Better / Worse and DD methods are developed in such a way that both will produce preference data that fit the measurement model described below.

[0071] The data generated when patients select one health state over another (Better / Worse method) are discrete choice data. The data generated when patients rank health states from most favorite to least favorite (DD) are rank-ordered data. To process the data generated by the Better / Worse and DD methods, two different but related statistical models are adequate: conditional logit and rank-ordered logit. These models differ in terms of the expected data structure of health-state preferences and estimation procedures. The distinction is that the dependent variable (preference: choice or rank) in the conditional logit records only the best state by a value not equal to zero (" 1 " if the best state and "0" for the other state)). In contrast, the ranked-ordered logit model marks the rankings of the states.Mathematics

[0072] The preference data of the Better / Worse and Drop-Down methods are processed in similar ways. The Drop-Down methods process preferences in the following way. The value of a health state j for an individual i is denoted by Vij. A respondent will rank state j higher than state k if Vij > Vik. The probability of choosing state j as the most preferred of the set of J states (Better / Worse: J= 2; Drop-Down: J> 2) can be writtenas follows:

[0073] The probability of observing a specific ranking among three or more health states (Drop-Down) can be written as the product of such terms, representing a sequential interpretation. In this sequence, the respondent first chooses the most preferred health state, followed by the most preferred of the remaining health states, and so forth. It was assumed that Vij is a linear combination of the levels of the health-state items plus an error term sij for the individual. The model is specified as follows:Vjy — PXj + 8 j (2) where / > represents a vector of regression coefficients. Further, xj is a vector of binary dummy explanatory variables (x5), with A indicating the levels of each of the items for a given health state. For example, in an example involving the CS-Base, x72represents the second level (“A little pain”) of the seventh item (Pain). Because a given health state has the same expected value across all respondents, x is indexed only by j. Although the estimation procedures for the two models differ, they will produce the same results if the rank-ordered logit model is used for data consisting exclusively of sets of two states.

[0074] The Drop-Down method provides an improvement in how the data is gathered as the Drop-Down method forces the patients to order their health states such that no two items (e.g., poor hearing / poor mobility each with the same intensity level) can have the same latent / intemal weight. For instance, where a patient's hearing was designated as Level 3: Poor Hearing and the patient's mobility was designated as Level 3: Poor Mobility, the Drop-Down method forces the patient to drop down each item (hearing or mobility) separately, e.g., forcing one to be dropped down before the other.Postulated States

[0075] For the Drop-Down method, additional health states may be created, for instance, on a server, according to the patient’s responses. The postulated states may be based on the ranked health states produced in the preference task (in which patients dropped down item levels to improve their actual health states), as the information derived from the Drop-Down task 1 is limited and may not be sufficient. The implementation of postulated states may significantly streamline the assessment process by achieving a greater quantity of useable states while requiring fewer interactions with the user. This approach reduces the total number of questions asked, yielding multiple benefits. For instance, batteries may be more efficiently used. Fewer questions translate to less time for the user to interact with the application, conserving the battery life of the mobile device. This is particularly important in health-related applications where different users might need to access a common device, running the same application, multiple times a day. The process for creating the first postulated health state involves adjusting two items based on their initial assessment levels: one item is raised (indicating a worsening condition) by one level, and another is lowered (indicating an improvement) by one level. The selection of the item to be raised in severity is determined by the item that was initially rated lower in the first drop-down. Conversely, the item to be lowered in severity is chosen based on the one that received a lower rating in the third dropdown. This method ensures that the adjustments in the postulated state are directly linked to the patient's initial responses, providing a tailored and accurate representation of potential health state changes.

[0076] Another advantage may include reducing the power consumption of the display screen by reducing the time spent interacting with the screen. For instance, minimizing the quantity of questions also means users spend less time staring at their device screens. This is beneficial for eye health and overall well-being, especially for patients who might already be experiencing discomfort or stress. Further, it reduces the power needed by the display by reducing the time the display is on.

[0077] A further advantage may include providing a stable response quality. With fewer questions, the likelihood of answers varying due to short-term changes or fluctuations is reduced. For instance, in pain assessments, a patient's pain perception might changewithin a brief period. Fewer, well-targeted questions can capture a more accurate snapshot of the patient's condition at a particular moment.

[0078] Yet another advantage may include convenience for patients. The streamlined process is less taxing for patients, especially those with health issues that make prolonged focus or interaction with devices challenging. This enhances the user experience and encourages consistent use of the application.

[0079] A further advantage may include an increased reliability of responses. With a reduced cognitive load from fewer questions, patients are more likely to provide accurate and thoughtful responses. This is crucial in health assessments where the precision of patient-reported data is essential.

[0080] Another advantage may include the avoidance of survey fatigue. Lengthy surveys often lead to decreased data quality as respondents become tired and less attentive, a phenomenon known as 'survey fatigue'. By curtailing the number of questions, the application ensures that the data collected is of higher quality, as patients are more likely to remain engaged and provide genuine responses throughout the assessment.

[0081] In summary, the use of postulated states in the application not only enhances user experience but also improves the quality and reliability of the data collected, which is crucial in health outcome measurements.

[0082] Postulated states are described with respect to Figures 6A and 6B. The postulated states are generated from the user interactions with the five drop-downs of Figure 5. For example, although the case of five drop-downs (Figure 5, pages 503 A, 503B, and 503C) generated six ranked health states, all the levels for these states are lower (i.e., better) than the patient’s actual health state. Moreover, for each of these ranked states, only one of the 12 items has a level that is lower.

[0083] The regression routine of the MAPR measurement model to estimate the weights for the levels of the items requires variation (i.e., more than one item must vary for each health state, and to both lower and higher levels) to determine a stable estimation (i.e., achieve convergence). For this reason, postulated states can be imputed based on information (Figure 6A) derived from the actual states (e.g., Task 1 from Figure 5, 501). This extends the ranking in the analyses to a maximum of 9 ranked states (Figure 6B). The additional postulated states may be created locally on the user's device and / orlater on a remote server.

[0084] As shown in Figure 6A, five states were modified (e.g., hearing 601, pain 602, daily activities 603, mobility 604, and social functioning 605). The order of the drop downs is shown as order 606. The levels of intensity (including baseline) are shown in group 607.

[0085] Figure 6B shows an example of creating postulated states. The postulated states are logical derivations from the combination of the patient's original health state and the order of the drop-down tasks performed on the original health state. In this example, the first drop-down (the health aspect considered as most disturbing) is item 3 (drop from level 3 to level 2). Increasing the actual health state for this item one level higher produces a health state that is by definition worse than the actual health state. Moreover, even if we reduce a particular item level from which it is known that this is considered less disturbing than the item from the first drop-down, we might assume that overall such a postulated state is still worse than the actual state. Here in this example, to increase the contrast (the right square bracket in Figure 6B) the third drop-down was selected as the item to decrease the level. The original order from Task 1 is shown as row 0 608 and represents the patients’ own health state. The Drop-Down health states are shown in the Drop-Down ordered levels of the items as rows 1-5. The original health state and the dropped-down intensity levels are shown in the matrix 614 as row 0 (the original health state) and rows 1-5 as health states corresponding to the modifications from each drop-down step.

[0086] Next, as shown in matrix 615, the patient's own health state (from row 0 of matrix 614) is replicated and modified based on the drop-down information. In this example, the patient's own health state is replicated three times. The patient's own health state may be replicated a greater number of times or a fewer number of times as desired. For instance, the user's health state may be replicated a number of times where the number of times is scaled to the quantity of drop-down steps. Where the quantity of drop-down steps is five (as shown in Figures 6A and 6B), the quantity of replicated original health states may be three (= quantity of drop-down steps - 2). Using this approach, where the quantity of drop-down steps is eight, the quantity of replicated original health states may be six (= 8 - 2). Other approaches may be available as well for quantifying healthstates (e.g., sliding scales on user interfaces and the like).

[0087] As a result, the quantity of health states is increased from the single health state provided by the patient to six health states (the original health state plus the five health states identified by dropping down one intensity level from the identified health items), and subsequently to nine health states (the previous six health states plus the three replicated and modified from the patient's actual health state). These additional health states, all centered on the patient, help to increase the precision of the estimated weights (regression coefficients) of the levels of the items. Consequently, these additional health states allow a survey to be conducted across fewer individuals to achieve analyzable results between individuals.

[0088] In this example, the first three items dropped down have one level added to each. The last three items dropped down have one level subtracted from each. Other examples exist including only adding level intensities to the levels of the items in the original health state, only subtracting level intensities from the levels of the items in the original health state, and / or a different combination of additions and subtractions.

[0089] In the current example, item 3 609 was dropped down first, item 7 610 was dropped down second, item 10 611 was dropped down third, item 1 612 was dropped down fourth, and item 9 613 was dropped down fifth. The first postulated state (row 1 of matrix 615) reflects modifications of the original health state (608 from matrix 614) as modified by the first drop-down. The first drop-down (609) is raised by one level (from 3 to 4). The second postulated state (row 2 of matrix 615) reflects modifications of the original health state as modified by the second drop-down. The second drop-down(610) is raised by one level (from 2 to 3). The third postulated state (row 3 of matrix 615) reflects modifications of the original health state as modified by the third dropdown. The third drop-down (611) is raised by one level (from 2 to 3).

[0090] Next, the third postulated state (row 3 of matrix 615) is further modified by the dropdowns. The fifth drop-down (613) is lowered by one level (from 2 to 1). The second postulated state (row 2 of matrix 615) is further modified by the dropdowns. The fourth drop-down (612) is lowered by one level (from 2 to 1). The first postulated state (row 1 of matrix 615) is further modified by the dropdowns. The third drop-down(611) is lowered by one level (from 2 to 1).

[0091] As shown in Figure 7, the ranks of drop downs are shown in matrix 701. The third item is raised in intensity as shown by arrow 702. The seventh item is raised in intensity as shown by arrow 703. The tenth item is raised in intensity as shown by arrow 704. The ninth item is lowered in intensity as shown by arrow 706. The first item is lowered in intensity as shown by arrow 705, and the tenth item (previously raised) is now lowered in intensity as shown by arrow 707. Alternatively, the number of additional rows added may be closer to the quantity of drop-down steps. In the example of Figure 7, the number of additional rows is three but the quantity of drop-down steps is five. Here, the level for item 10 is raised for one additional level (e.g., row 3) and dropped in another additional level (e.g., row 1). In another example, the number of additional rows may be three as well but the quantity of increased levels and decreased levels may be reduced to one change per row. In other words, for an even quantity of drop-down steps, the quantity of increased levels may be equal to the quantity of decreased levels. For an odd quantity of drop-down steps, the quantity of increased levels may be one greater (or one less than, as desired) than the quantity of decreased levels.

[0092] Further, in the example of Figure 7, the postulated states are modified to include the increase in levels per the first three drop-down steps. The first new row (row 1) reflects the increase in the item related to the first drop-down step (e.g., the previous level 3 being changed to level 4). The second new row (row 2) reflects the increase in the item relating to the second drop-down step (e.g. the previous level 2 for item seven being changed to level 3). The third new row (row 3) reflects the increase in the item relating to the third drop-down step (e.g., the previous level 2 of the tenth item being changed to level 3).

[0093] Next, for row 3, the level of the last selected drop-down item (item 9) is decreased from level 2 to level 1. For row 2, the level of the second to last drop-down item (item 1) is decreased from level 2 to level 1. Finally, for row 1 , the level of the third to last dropdown item (item 10) is decreased from level 2 to level 1. The result is three postulated health states now found in rows 1-3 of the matrix of Figure 7 where each of the new rows is different from the original health state (now row 4) and the related drop-down health states (rows 5-9).Differences in Communication with Remote Servers

[0094] Figures 8-12B show various examples of communications between a user's device and a remote server to highlight differences between network loads based on the various disclosed methods.

[0095] Figure 8 shows an example of the generation of alternative states by a server 802 and communicating those alternative states to a user's device 801 (for a survey using the Better / Worse method). The server 802 generates a selection set 803 and sends it to the user's device 801. The user inputs his health state in step 804. The results are sent to the server in step 805, an alternative state is generated in step 806, and the alternative state is sent to the local device in step 807. The user selects his preference in step 808 and the preference is sent to the server in step 809. The process is repeated by generating steps 810, 814, and 818, receiving the alternate states in steps 811, 815, and 819, receiving the user's preference in steps 812, 816, and 820, and then providing the user preference to the server in steps 813, 817, and 821. In step 822, the complete survey results are exchanged between the user's device 801 and the server 802. For comparison purposes here, a dotted box 823 is shown encompassing various communication steps.

[0096] Figure 9 shows an example of the generation of alternative states by a local device (for a survey using the Better / Worse method). A local device 901 receives a selection set 903 from a server 902. The user's state is determined in step 904. As shown in steps 905, 907, and 910, alternative states are generated. As shown in steps 906, 908, and 911, the alternative states are presented to the user, and the user's preferences are collected. The survey may be completed in step 912. The survey may be reset in step 913 for the next user. In step 914, the completed survey or surveys may be provided to the server for analysis. Step 909 represents additional combinations of the generating alternative states, the presenting of the alternative states to the user, and the collecting of the user's preferences on the alternative states. For comparison purposes here, a dotted box 915 is shown encompassing various communication steps.

[0097] Figure 10 shows an example of sequential level comparison by a local device (for a survey using the Drop-Down method). A local device 1001 receiving user selections is shown in relation to a server 1002. A selection set of options 1003 (e.g., a selection set)may be received from the server 1002. A user may designate his own health state (e.g., a user state) as a first task in step 1004. In steps 1005-1011, the user may identify an order in which the previous selections comprising the first task have priority. For instance, the user selects in step 1005 the item at the highest priority in this health state. In step 1006, the user interface is updated to reduce the intensity level of the item having the highest priority. In step 1007, the user selects the item having the highest priority in the updated user interface from step 1006. In step 1008, the user interface is updated to reduce the intensity level of the item having the highest priority selected in step 1007. The process continues in step 1009. In step 1010, the user selects the item having the highest priority in the most recent updated user interface. In step 1011, the user selects the item having the highest priority in the updated user interface from step 1010. In step 1012, the survey may be completed and, in step 1013, prepared for the next user. Additionally or alternatively, the system may provide the result of the analysis to the user in step 1014. Additionally or alternatively, the process may send the completed survey results to the server in step 1015. Based on the completed survey results being sent to the server 1002 in step 1015, the server may generate (step 1017) postulated states. The postulated states may be sent (in step 1018) to a physician and / or to the user. On the server, weights for the levels of the items may be estimated. For comparison purposes here, a dotted box 1016 is shown encompassing various communication steps.Comparison of Approaches

[0098] The following describes how the various processes described above compare for similar conducting a survey to obtain useable results. For comparison purposes of Figures 8, 9, and 10, the first device, shown in Figure 8, represents the remote generation, e.g., by a server, of additional queries or comparison states for the user. In the example of the first device, the additional queries are alternative states of the MAPR model. However, the additional queries may also be the next set of questions using a conventional survey (e.g., the next set of questions using the survey method of US 20020035486 to Huyn et al.). The second device, shown in Figure 9, represents the local generation of the alternative states of the MAPR model. The third device, shown in Figure 9, represents the user's selection of a state and then internally compares theitems and their respective levels of the user's state against each other. The operations within the dotted rectangles in each of Figures 8, 9, and 10 (e.g., 823, 915, and 1016) are compared. As described below, the faster operation of the device of Figure 9 shows how the improvement of the application provides a technical improvement to related computing systems. In the examples below, an initial user state is identified as 354 bytes (B), an alternative state is identified as 354 B, and the combination of the alternative state and the user preference (the preference associated with the MAPR model) is identified as 354-355 B. For the calculations shown below, the size of 355 B is used to differentiate from the size of the initial state and alternative states by themselves.

[0099] Figure 8 shows a local device and a server. The local device presents options to a user and the user, by selecting various ones of the presented options, identifies the initial user state. The local device transmits the user state of 354 B to the server. In response, the server generates a first alternative state (e.g., alternative state 1) and transmits it (at 354 B) to the local device. The local device displays the alternative state 1 to the user and receives the user's preference. The alternative state 1 and the user's preference are sent to the server (at 355 B) where the server generates the next alternative state 2. This process continues for as many alternative states to complete the survey. Shown outside the dotted rectangle, the survey is completed. At this point, the local device may be provisioned to begin another survey, for instance, for another user.

[0100] Figure 9 shows an example of a local device generating the initial states by itself instead of relying on the server to generate the alternative states. In Figure 9, the local device presents options to the user, and the user, by selecting various ones of the presented options, identifies the initial user state. A processor of the local device generates, based on the initial user state, alternate state 1. Alternate state 1 is displayed to the user and the user identifies a preference for the initial user state or alternative state 1. Based on the user's preference, the processor of the local device generates an alternative state 2. Alternative state 2 is displayed to the user and the user identifies a preference for the initial user state or alternative state 2. This process continues for the quantity of alternative states needed to complete the survey. As shown outside the dotted rectangle in Figure 9, the local device may be used to complete another survey by anotherindividual before eventually uploading the survey or surveys to the server.

[0101] In Figure 10, the Drop-Down method uses the ordering of the intensity levels previously assigned to each item in the user's health state to better quantify the relationships between the severities of the items. Based on the comparison between intensity levels, the tasks of generating alternative health states and the associated recordkeeping of those alternative health states with the user's Better / Worse preferences are not required.

[0102] For each of the examples of Figures 8 and 9, the initial set of total selectable options is shown as 4.8 kilobytes (KB). As that data may be downloaded or already resident on the local device, it is excluded from the following analysis.

[0103] High-speed wireless communication is not ubiquitous. Sometimes, wireless communications are restricted or the infrastructure supporting high-speed wireless communication (e.g., access points and wired networks and / or mesh networks) does not exist. Wireless communications are often prohibited in cardiac monitoring wards of hospitals. Rural environments may lack the infrastructure to guarantee high-speed wireless connections. Where communication with a server does not exist (e.g., in a cardiac monitoring ward or in a rural environment without connectivity), the local device of Figure 8 may never be able to complete the survey as the local device of Figure 8 relies on the server to generate the alternative states. Where some connectivity does exist, that connectivity may be slow. To show how the local device of Figure 9, representing the technical approach of the application, is a technical improvement to the operation of the local device of Figure 8, the following compares the operation of the local devices in an environment with a communication bandwidth of 100 kbits / second, e.g., the communication bandwidth specified in IEEE 802.15.4 "Low-rate wireless personal area network". The 100 kbits / second bandwidth described below is only an example as greater bandwidths may be used in practice. This bandwidth is used to explain where delays occur and how those delays differentiate the operations of the local devices of Figures 9 and 10.

[0104] Figure 11 shows a comparison between the local device relying on the server to generate the alternative states (e.g., Figure 8 above) and the local device generating the alternative states by itself (e.g., Figure 9 above and relating to a technical improvementover that of Figure 8 in the processing time to complete the survey). Figure 11 also shows the improvement of Figure 10 over each of the processes of Figures 8 and 9.

[0105] Figure 11 represents the processes for receiving a user state, generating an alternative state, obtaining a user preference between the user state and the alternative state, and getting ready to generate the next alternative state. Three types of delays are present in the operation of the server-side calculated alternative states (left side of Figure 11 pertaining to the process of Figure 8) that are not present in the operation of the locally generated alternative states (middle of Figure 11 pertaining to the process of Figure 9). The first delay is the transmission delay between the sending of the user state from the local device and when the user state is received at the server. The second delay is the transmission delay between the sending of the alternative state of the server to the local device. The third delay is the transmission delay between the sending of the alternative state and user preference from the local device to the server. These three types of delays are not present in the process (shown in Figure 9 and shown in the middle in Figure 11) where the local device generates the alternative states. The combined delay is shown in the curly bracket at the bottom center of Figure 11.

[0106] More particularly, Figure 11 shows relative bandwidth and processing comparisons between the server-generated alternative states (steps 1104-1109), the local device generating the alternative states (steps 1110-1112), and the local device using the Drop- Down method comparison (steps 1113-1115). In swim lane A, the user receives and completes their health state in step 1104 on their local device 1101. In step 1105, the user sends the health state to the server 1102 and waits for the server to receive the transmitted health state, generate an alternative state 1106, and receive the alternative state 1107. The user receives and responds to the alternate state in step 1108. The per- altemative state delay is shown in step 1109. In swim lane B, the local device 1103 displays the survey to the user and receives the user's health state in step 1110. In step 1111, the local device 1103 generates an alternative state. In step 1112, the local device 1103 displays the alternative health state and obtains the user preference. The results are sent to the server 1102. The time difference between the end of step 1112 and the end of step 1109 represents the combined initial transmission delay and subsequent delays per alternative state. In swim lane C, the local device 1117 performs the DDmethod (performing an ordered comparison of health item intensities). In step 1113, the user completes the survey (task 1), and the results are received by the local device 1117. In step 1114, the user drops down the health items (e.g., the local device 1117 receives the user priorities), and the results are made available afterward in step 1116. The time 1115 represents the time saved by the system not needing to generate alternative states (from swim lanes A and B). In short, the server-based generation of the alternative states is subject to the greatest quantity of network-based and processing-based delays. The local device-generated alternative states are preferable as fewer communications over a burdened network are needed to complete the survey. However, the Drop-Down method faces the fewest amount of potential network delays as no alternative states are generated and subsequently stored.

[0107] To compare the differences between the times to complete a survey using the servergenerated alternatives approach of the left side of Figure 11 and the locally-generated alternatives approach of the middle of Figure 11, the transmission delays are determined using a 100 kbits / second bandwidth with five alternative states. As used herein, a given transmission delay is determined as the transmission size divided by the available bandwidth. The total transmission delay may be determined as the {initial send delay} plus the per alternative delay (each alternative delay represented as the sum of the {alternative receive delay} + {alternative send delay}).Total delay = initial send delay + X(altemative receive delay + alternative send delay), where X = quantity of alternative states compared to initial user stateWhere bandwidth = 100 kbits / s (IEEE 802.15.4 Low-rate wireless personal area network) = 12.5 kB / s,For 5 alternative states, X =5,Total delay = 354 / 12500 + 5((354 / 12500) + (355 / 12500))0.0283 + 5(0.0283 + 0.0284)0.0283 + 5(0.0567)0.0283+0.28350.3118 sFor 10 alternative states, X =10,Total delay = 354 / 12500 + 5((354 / 12500) + (355 / 12500))0.0283 + 5(0.0283 + 0.0284)0.0283 + 5(0.0567) 0.0283+0.567 0.5953 s

[0108] Where the five alternative state survey takes an estimated 60 seconds (depending on the length of the survey) for a user to complete using a device locally generating alternative states, an equivalent survey using a device relying on the server to generate the alternative states and in the low bandwidth environment described above would take an estimated 60.3118 seconds, an increase of approximately 1% (=60.3118 / 60). Using a device that locally generates the alternatives, the local device may complete the survey faster and without reliance on communications with the remote server.

[0109] These examples show how, for limited bandwidth environments, the device that locally generates the alternative states requires less time to complete the survey. Where a device relies on the server to generate the alternative states, its processor continues to run (and consume power, drain battery, etc.) that is not required by the device that locally generates the alternative states. This reduction in the processing time of the device that locally generates the alternative states is a technical improvement to the operation of a device used to conduct a survey of an individual.

[0110] The right side of Figure 11 (pertaining to the process of Figure 10) shows a further improvement in that no generation of alternative states is needed but instead, the user internally ranks the levels that comprise his health state against each other. The process of Figure 10 and as shown on the right side of Figure 11 shows the computational savings of the processor and the subsequent data transmissions with the server. As such, the process of Figure 10 does not face the processing delays of the process of Figure 9 to generate the alternative states. Further, because the results of the process of Figure 9 are computable based on the postulated states described herein, the survey resultsmay be locally analyzed by the local device 1117 instead of requiring that the analysis be performed via the server 1102.

[0111] A conventional approach to generating and receiving survey results may generate questions and ask the user to generate responses to each question. US Patent 8340982 to Bjomer et al. is an example of the item response theory (IRT) method of conducting a survey. US Patent 7552104 to Hansen is an example of a discrete choice model (DCE) method of conducting a survey.

[0112] The MAPR model using Better / Worse alternatives is an improvement to the IRT and DCE examples of surveys. In general, the Drop-Down method is an improvement to the MAPR model, where both the Drop-Down method and the Better / Worse method are applicable. However, specific situations may exist where the Drop-Down method is not a feasible option. While the MAPR model may be used with a specific set of steps to significantly reduce network latency, the Drop-Down method further improves on reducing network latency. A goal of the Drop-Down method is to measure the perceived health of individuals while minimizing the load on a computer network. In short, the use of the Drop-Down method model simplifies the quantity of survey questions while ensuring adequate sampling of user responses.

[0113] This is possible because the initial response of the patient (describing his / her own health condition by selecting the levels of specific items) is obtained and the levels of intensity of the specific items ordered by the user based on the user's subjective view of the intensity of each. Accordingly, patients may use the application without an internet connection. Responses are stored in a local application executing on the patient's device and subsequently sent to the central server when the internet connection is available. In short, the user is being asked to rank each of his items that make up his health state with other items within his own health state, not comparing those results to alternative health states or answering another round of questions (i.e., generated by a remote server). One of ordinary skill in the art would appreciate that the application's ability to reduce reliance on a possibly intermittent network solves the technical issue of reducing reliance on an active network connection to complete a survey.

[0114] With respect to reduced data transmissions, both the number of transmissions and the quantity of data that is part of each transmission may be reduced. In existing softwareto measure subjective phenomena, such as health, the number of transmission occasions as well as the amount of transmitted data is higher per user. Respondents (patients or non-patients) are presented item by item (IRT) or pairs of health states (DCE). Each assessment requires data to be transmitted back and forth to the central server / computer. In the approach of the present application, the assessment is different and simpler. Therefore, less data transmission is needed: 1) After a patient has described his / her own health state, no further data transmission is needed to depict the alternate states in the assessment; and 2) as the patient ranks the levels of his items in his health state against the levels of other items in his health state, thus reducing the need to generate alternative health states.

[0115] In contrast, the conventional approach (IRT) or (DCE) requires repeated queries to the central computer for each of the items (e.g., sending, by the central computer, five options for each of five items equals 25 sent separate items and five returned items from the local device). As aspects of the present application compare the patient's own health state with alternate states, less data transmission is needed compared to the existing measurement methods.

[0116] One of ordinary skill in the art would have appreciated that the application's teachings of sending a respondent's state and the ordered level of the user's subjective comparisons between the items comprising the respondent's state solves the technical issue of reducing data transmissions not possible with the conventional question-based survey generating and data gathering techniques.

[0117] The improvement to the device administering the survey to the user is a technical improvement shown by how the device operates in low bandwidth environments. As described below, three devices operating in a low bandwidth environment are compared in how long each takes to complete a survey described in the application (e.g., where the user indicates a preference for remotely generated alternatives; where the user indicates a preference for locally generated alternatives; and where a user describes the user state and internally ranks items in the state against each other).

[0118] Figure 12A provides an illustration of why the different transmission volumes can be significant in low-bandwidth environments. Figure 12A shows a user device 1201 running a native application with various user interfaces and running a web-basedapplication with various user interfaces. The various methods described herein are represented as DD 1202 (the Drop-Down method), BW 1205 (the Better- Worse method), DCE 1208 (the Discrete Choice model), and IRT 1210 (the Item Response Theory model). In other words, Figure 12A shows examples of data transmissions for a native application (e.g., running on a local device and eventually communicating with a server) and a web-based application communicating with a server (e.g., a server-based application).

[0119] For the Drop-Down method DD 1202, two approaches are represented. Each represents 12 selections for Task 1 and 5 Drop-Down selections for Task 2. A first approach is a native application 1203 executing on the user device 1201 and interacting with a remote server storing information from the user device 1201. The user makes 12 selections for Task 1 and then 5 Drop-Down selections for Task 2. The result is a total of 17 user selections sent to the server. The selections may be transmitted to the server. In the Drop-Down method, each user selection may be represented by data packet. For purposes of explanation, the size of each data packet is identified as approximately 354 bytes (B). To provide the 17 selections to the server, the user device 1201 using the DD method 1202 as a native application 1203 sends 17 packets describing the 17 selections to the server for storage via one transmission. The one transmission is represented by the "lx" arrow from the user device 1201 to the data storage in Figure 12A. At a transmission size of 354 B per packet representing a selection, the resulting data transmitted is approximately 6 kB (= 17 * 354 B = 6018 B).In that the native application 1203 may have the original selectable content stored locally in the user device, no initial transmission of the selection options is required (e.g., represented by the "Ox" arrow from the data storage to the user device 1201.

[0120] The second approach for the DD 1202 method represents a web-based application 1204. Similar to native application 1203 approach, the user makes 12 selections for Task 1 and 5 Drop-Down selections for Task 2. The result is 17 user selections. A user device 1201 is shown obtaining an initial user state (Task 1). For Task 2, in the DD method 1202 the user interacts with a user interface using the Drop-Down method to identify and order the most significant selections of Task 1. For Task 1, 12 selections were received. For Task 2, 5 selections were received. The result is that 17 totalselections are pushed to the server. For each of the native application 1203 and the webbased app 1204, the 17 total selections are the same. At a transmission size of 354 B per transmission of a selection, the resulting data transmitted is approximately 6 kB (= 17 * 354 B = 6018 B).

[0121] In the web-based application 1204, the original selectable content is resident in the data storage and transmitted to the user device 1201. The revised user interfaces for each of Task 1 and Task 2 are to the user device 1201 as well. The quantity of transmissions from the server to the user device 1201 may comprise a variety of transmissions, depending on whether, for instance, the user selections transmitted to the server at the end of Task 1 are responded to by the server with new user interfaces (e.g., modifying the functional regions of the user interfaces) and / or whether the last step of Task 2 requires a separate transmission. For purposes of explanation, the transmissions from the server to the user device 1201 are represented by the arrow "6x" arrow from the data storage to the user device 1201. The transmissions from the user device to the server are represented by the arrow "6x" representing one transmission at the end of Task 1 (comprising 12 selections) and one transmission at the end of each step of Task 2 (5 more transmissions of 1 selection each).

[0122] For the Better-Worse method 1205, a native application 1206 executing on the user device 1201 relates to 12 selections for Task 1 and 5 selections for Task 2. To provide the 5 selections for Task 2 to the server, the BW method 1205 provides the 12 items displayed at each step in Task 2 with the accompanying better / worse selection (= 13 data packets to represent each Task 2 selection = 12 + 1). Performed five times, 65 data packets are transmitted to accurately describe the five Task 2 selections. In short, the 65 data packets are based on five (5) sets of alternative states at twelve (12) selections each plus the user preference decisions (e.g., better or worse) as one (1) selection. Accordingly, the total size for the native application BW 1206 process is 77 data packets = 12 original selections (described by 12 data packets) plus 5 alternate selections (described by 65 data packets). Similarly, for a web-based application using the BW process 1207, the total size to send to the user selections are 77 transmitted packets.

[0123] For the locally generated states (using a native application 1206) (using theBetter / Worse method) of BW method 1205, Task 1 results in 12 initial selections / 12 initial packet transmissions. Task 2 results in 5 selections represented by 65 packet transmissions. The total packets to describe the user selections in the BW 1206 method are 77 packets. (77 selections (=(12 alternatives + 1 selection (Better / Worse)) x 5 alternatives).

[0124] By comparison, other selections using a server-based alternatives generation (the IRT method 1210) result in 120 packets based on, for instance, five initial selections of 12 items and 5 subsequent selections of 12 items. Similarly, individual server-based / peritem selections (the DCE method 1208) result in 125 packets in the DCE web application method 1209. For the IRT method 1210 (e.g., the DCE web application method 1211) with an estimated 354 B per packet and 120 packets in method 1211, the resulting data transmitted is approximately 42 kB (= 120 * 354 B = 42480 B). For the DCE method 1208 with an estimated 354 B per packet, the resulting data transmitted is approximately 44 kB (= 125 * 354 B = 44250 B).

[0125] Where each transmission packet is estimated as 354 B per selection, the locally generated Better / Worse process 1205 results in approximately 27 kB as follows: Total subsequent transmitted packets equals 77 = (quantity of users = 1 )(initial state (12)+ 5(alternative state (12)+ preference(l))). At a transmission size of 354 B per transmission, the resulting data transmitted is approximately 27 kB (= 77 * 354 B = 27280 B). For a server-side generation of alternative states, the server-side-generated Better / Worse process 1205 is a combination of the total sent to the server and the total received from the server during the comparison window. The result is (12 initial responses x 5 alternatives) + (12 alternates x 5 preferences) = 120 transmitted packets.

[0126] The DD method 1202 results in the smallest quantity of transmitted packets to represent the selections, e.g., only 17 packets representing the Task 1 and Task 2 selections.

[0127] Figure 12B provides an illustration of how the DD method selections compare to those of the BW method. In the process 1212 relating to the DD method, 12 selections are made for Task 1 1213. Next, for Task 2 1214, five selections are made. In the process 1215 relating to the BW method, the same 12 selections are made for Task 1 1216. However, in contrast with the DD method for Task 2 1217 of the BW method, the quantity of selections is fewer (here, for instance, just one selection shown by theencircled numeral 1 ) as patients only select between the Better button A 1218 or Worse button B 1219. However, they repeat the selection four more times (e.g., deciding between their own health state and the generated health state of each new page). For the DD method, all five selections may occur on the same page. For instance, only drop-down items change so no Next button is required at the bottom of the page. While the quantities of selections between the BW method and the DD method are similar or identical, the BW method may require greater cognitive attention.Additional Patient-Centered Features

[0128] It is helpful to include items that are relevant to the target populations’ subjective health evaluation. In the development of PROMs, it is increasingly recognized that the item selection should be based on the patient’s input. However, many of the existing PROMs are not patient-centered in their development, but the health professionals ’ views are prioritized. This could result in either omitting health items that have high relevance to patients or accentuating irrelevant ones. For instance, the EQ-5D-5L measure (available from the EuroQol Research Foundation of Rotterdam, the Netherlands) comprises a self-administered survey across five dimensions ("5D") and with five levels ("5L") for each dimension. The EQ-5D-5L is relatively simple to administer (e.g., five dimensions / questions where the patient selects one of five levels of intensity for each of the five dimensions / questions). Despite its popularity, the EQ-5D-5L exhibits some degree of bias in that its content of the 5 dimensions (e.g., items) was not selected by patients but by health researchers. Questions regarding the usefulness of the EQ-5D-5L include whether its content accurately reflects what is important to patients and whether the five items are sufficient enough to assess the overall health of patients. CS-Base is also another PROM and is further described herein.

[0129] Besides health item selection, another crucial part of preference-based PROMs is the health valuation or preference method. This is used to generate weights for levels of items and can further provide a quantitative measure (value) of the overall health. The value allows comparison between many different disease groups and can be used for many areas such as calculating quality-adjusted life years, assessing the costeffectiveness of interventions, monitoring the health conditions of the population, and supporting clinical decision-making.

[0130] Preference-based methods are frequently used in valuation techniques. Conventional preference-based methods (e.g., Standard Gamble, Time Trade-Off) applied in the health setting were developed by health economists and mainly based on (pairs of) hypothetic health states assessed by a sample of the general population instead of patients. However, it is reasonable to assume that in many situations, a sample of unaffected respondents from the general population may be inadequately informed or lack good imagination to make an appropriate assessment of the impact of (severe) health states.

[0131] Besides, such conventional preference-based methods are, to some extent, too complicated to be easily understood by respondents. As such, well-trained interviewers are often needed to help respondents to complete the tasks. Preference-based methods used by health economists often use an iterative process to offer respondents different lengths of life. This may introduce anchoring bias, which means people tend to rely too heavily on the very first piece of information encountered. As Tversky and Kahneman explained, people make estimates by starting from an initial value that is adjusted to yield the final answer. One example given by Tversky and Kahneman is participants spun a wheel to select a number between 0 and 100. The volunteers were then asked to adjust that number up or down to indicate how many African countries were in the U.N. Those who spun a high number gave higher estimates while those who spun a low number gave lower estimates. In each case, the participants were using that initial number as their anchor point to base their decision. In general, these conventional preference-based tasks are complex and cognitively demanding, as a result, these tasks are likely to produce results that are less precise or that may even be biased.

[0132] It is beneficial to make the preference tasks as simple as possible. A novel preferencebased measurement framework has been recently introduced. This framework is known as the multi-attribute preference response (MAPR) model described above. In its general form, it is a probabilistic choice model that combines the Rasch model (item response theory) and the discrete choice model (i.e., discrete choice experiments). These types of choice models have a long history, commencing with Louis Thurstone’s model, which was developed in 1927. Other researchers have introduced extensions of the basic Thurstonian model. There are two assessment tasks within the MAPR model,the first is a descriptive task, patients (hence, not respondents from the general population) describe their health states in this task based on a set of health items. These health items are all selected by the patients themselves. The second is a preferencebased task, which generates ranked preference data that is used to estimate the overall weights of the levels of the items.

[0133] The Drop-Down method, described above, was used as the preference-based task in the MAPR model. In the Drop-Down method, respondents do not need to be confronted with hypothetic health states or make trade-offs between their own health and alternative, hypothetical health states. They only focus on their own health state and select health items that hinder them most. The Drop-Down method provides good results as described herein. An additional benefit of the MAPR measurement framework is that the assessment tasks (Task 1 : descriptive task, Task 2: preferencebased task) can be performed on smartphone screens which makes the PROM user- friendly, and attractive to the users (patients) and to researchers. In addition, all responses are automatically stored and processed.CS-Base

[0134] CS-Base is a generic health-outcome instrument. More particularly, CS-Base is an electronic patient-reported outcome measure (ePROM) that may run in a mobile application executing on a portable device or execute as a web-based application. For example, the mobile or web-based application (hereinafter, "app") may comprise 12 health items, each specified on four levels. All the 12 health items in the CS-Base were selected by patients. Specifically, this instrument is an electronic patient-reported outcome measure (ePROM) that uses special software. The CS-Base was developed for measuring HRQoL and comprises 12 health items, each specified on four levels: mobility, vision, hearing, cognition, mood, anxiety, pain, fatigue, social functioning, daily activities, self-esteem, and independence. The MAPR measurement model, described above, was used for the CS-Base.Mobile app

[0135] PROMs maybe executed in a mobile app (e.g., in the HealthSnApp from Chateau Sante of the Netherlands). This is a flexible tool, with interactive routines. It runs on variouselectronic devices (e.g., smartphones, tablets, laptops, and the like) and is highly configurable from a web-based console.Coefficients

[0136] An empirical study using the CS-Base has been conducted in which the Drop-Down was used. The coefficients estimation of the CS-Base was based on the outcomes of 2,534 respondents who did the Drop-Down tasks. For CS-Base, all coefficients revealed a logical order (all the coefficients are negative numbers). The more negative a coefficient is, the lower the coefficient is (indicating a higher impact) (See the Table below). All coefficients showed statistically significant differences (P<0.001). Clear differences in coefficients were observed between levels for all items 1301 of Figure 13.TableCoefficients of CS-Base (N=2534)Item levels _ Coefficient SE ZMobility (2) -3.22 0.13 -25.49Mobility (3) -8.95 0.19 -46.29Mobility (4) -15.40 0.35 -44.57Vision (2) -3.25 0.12 -26.06Vision (3) -8.24 0.19 -43.54Vision (4) -14.55 0.39 -37.67Hearing (2) -3.45 0.10 -35.59Hearing (3) -8.66 0.16 -53.91Hearing (4) -14.76 0.32 -46.31Cognition (2) -3.28 0.14 -23.43Cognition (3) -8.19 0.21 -37.77Cognition (4) -12.87 0.54 -23.81Mood (2) -3.30 0.10 -33.49Mood (3) -7.89 0.16 -50.51Mood (4) -13.19 0.27 -48.34Anxiety (2) -3.13 0.09 -34.28Anxiety (3) _7.44 0.14 -51.67Anxiety (4) -12.94 0.22 -57.65Pain (2) -3.23 0.09 -35.93Pain (3) -7.54 0.14 -53.28Pain (4) -13.14 0.22 -58.81Fatigue (2) -3.40 0.09 -39.08Fatigue (3) -7.65 0.14 -53.25Fatigue (4) -12.55 0.23 -55.68Social function (2) -3.44 0.10 -33.63Social function (3) -7.56 0.17 -45.59Social function (4) -12.71 0.29 -43.50Daily activity (2) -3.46 0.10 -34.14Daily activity (3) -7.65 0.17 -45.76Daily activity (4) -11.72 0.35 -33.20Self-esteem (2) -3.81 0.11 -35.80Self-esteem (3) -7.54 0.17 -45.19Self-esteem (4) -12.45 0.25 -50.58Independence (2) -3.83 0.13 -28.63Independence (3) -8.15 0.22 -36.41Independence (4) -12.50 0.42 -29.83All the p values <0.001Health States and Values

[0137] In one example, 1,988 respondents assessed their health states. The number of different health states, assessed in the CS-Base, was 1,472. The mean value of the health states reported in the CS-Base was -30.05. No respondent reported the worst health state in the CS-Base. The worst health state among the 1,988 respondents reported in the CS- Base is 342444443344 (value= -131.80).

[0138] Figures 13 and 14 show distributions of values (without perfect health) for the CS- Base. The values for perfect health state in the CS-Base are 0. The perfect health state was excluded from Figures 13 and 14, so there are no values (Figure 13) or bars 1401 (Figure 14) above the value “0”. The number of respondents (without those who reported perfect health) in the CS-Base PROM is identified in the top-left.

[0139] Compared to conventional preference-based methods, the most outstanding advantage of the Drop-Down method is that it is easy to perform. No alternative or hypothetic health states are included in this method, the patients only have to assess their ownhealth conditions. They just need to select and swipe away the items that hinder them most. Thus, the Drop-Down method is directed more accurately at the patients’ own experience and is easier to perform. Meanwhile, the Drop-Down method can also be administered on smartphones or other electronic devices, which makes the tasks more convenient and attractive to users.

[0140] Figure 15A shows an example bar chart of how results may be provided to a physician. Scores on each item for individual patients may be identified in item 1501 as the narrower bars. Scores on each item for a group may be identified as item 1502 as the wider bars. The most burdensome health items to an individual patient may be identified as item 1503 - the central narrow bar. In Figure 15B, the total values based on scores on items (e.g., based on an estimation at a server) are represented as values for individual patients as item 1504. Item 1505 pertains to values on a group level for a selected group of patients. Figure 15C compares subgroups of patients as items 1506 and 1507. Figure 15D shows how scores 1508, 1509, and 1510 may change for a given patient over time (from time T1 to time T2 to time T3).

[0141] Figure 16A shows that the ordered top number of items 1601 are considered as imparting the largest burden to an individual patient. Figure 16B shows bubble charts for males 1602 and females 1603 with different medians. Figure 16C shows a warning level graphic 1604 with an explanation 1605. Figure 17 shows a user interface for displaying how a median value 1701 for a user's responses compares to the median values 1702 of others. More particularly, Figure 17 shows an alternative version of Figure 16B where the median value result 1702 of the individual are highlighted in an overall plot of median values 1701 of all similarly situated individuals. With respect to Figures 15A, 15B, 15C, 15D, 16A, 16B, 16C, and 17, items 1503 and 1601 are only available using the Drop-Down method for the survey and not for the Better / Worse method for the survey.

[0142] Figure 18 shows another user interface 1801 that provides a rotating list of selectable options. The user interface may comprise a prompt 1802 to instruct a user on the type of information needed. User interface 1803 may comprise a region with multiple selection options that are displayed on subregions of the region where subregions are scrollable. A user may interact with any particular subregion and select it (e.g., via acheckbox, for instance). The selected subregions may then be finally accepted via clicking a "next" button 1804. The rolling function of the user interface 1801 may help alleviate the screen real estate limitations in user-interface design and provide rich contextual information to ease users' navigation tasks. For instance, the rolling display options may be used in Task 1 in case patients have to select from a large set of health items (e.g., more than 18) to identify which item they consider relevant to them.

[0143] In one example, the functional user interface elements may all start at a zero-intensity level for each state. In an alternative example, the elements may be prepopulated with states other than the zero intensity level. For instance, where a user is being monitored over time, the user interface may be prepopulated with the results of Task 1 from a previous session with the user. Alternatively, the user interface may be prepopulated with sample response levels based on responses from other individuals in a similar cohort.

[0144] Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0145] Many illustrative embodiments are listed below in accordance with one or more aspects disclosed herein. Although many of the embodiments listed below are described as depending from other embodiments, the dependencies are not so limited.

[0146] Embodiment 1 : A computer-implemented method comprising: generating a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the levels relate to different intensity levels; displaying, on a display of a user computing device, the first user interface; receiving, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; storing, based on the first user interactions, the first selected levels for the items; generating, without communicating with a remote server and based on the first selected levels, a second user interface; displaying, on the display, the second user interface; receiving, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculating, based on the order, additional second selected levels; generating,based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generating, based on the ranking, a third user interface with the first selected levels storing the ranking.

[0147] Embodiment 2: The computer-implemented method of Embodiment 1, further comprising: displaying, based on the ranking, the third user interface on the display of the user computing device.

[0148] Embodiment 3: The computer-implemented method of Embodiment 1, further comprising: receiving, from a remote server, a plurality of items and available levels, wherein the generating the first user interface with the items is based on the plurality of items, and wherein the two or more selectable levels are based on the available levels.

[0149] Embodiment 4: The computer-implemented method of Embodiment 1, wherein the receiving the first selected levels comprises receiving, for a first item of the items, a first selected level, wherein the calculating the additional second selected levels comprise increasing, based on the order, the first selected level of the first item to a first higher level, and wherein the generating the ranking is further based on the first higher level.

[0150] Embodiment 5: The computer-implemented method of Embodiment 4, wherein the first higher level is two or more levels higher than the first selected level.

[0151] Embodiment 6: The computer-implemented method of Embodiment 1, wherein the receiving the first selected levels comprises receiving, for a last item of the items, a first selected level, wherein the calculating the additional second selected levels comprise decreasing, based on the order, the first selected level of the last item to a first lower level, and wherein the generating the ranking is further based on the first lower level.

[0152] Embodiment 7: The computer-implemented method of Embodiment 6, wherein the first lower level is two or more levels lower than the first selected level.

[0153] Embodiment 8: An apparatus comprising: a display; one or more processors; and a memory storing instructions that, when executed by the one or more processors, control the apparatus to: generate a first user interface comprising functional regions associatedwith items, each functional region comprising two or more selectable levels, wherein the levels relate to different intensity levels; display, on a display of a user computing device, the first user interface; receive, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; store, based on the first user interactions, the first selected levels for the items; generate, without communicating with a remote server and based on the first selected levels, a second user interface; display, on the display, the second user interface; receive, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculate, based on the order, additional second selected levels; generate, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generate, based on the ranking, a third user interface with the first selected levels storing the ranking.

[0154] Embodiment 9: The apparatus of Embodiment 8, wherein the instructions further control the apparatus to: display, based on the ranking, the third user interface on the display of the user computing device.

[0155] Embodiment 10: The apparatus of Embodiment 8, wherein the instructions further control the apparatus to: receive, from a remote server, a plurality of items and available levels, wherein the generated first user interface with the items is based on the plurality of items, and wherein the two or more selectable levels are based on the available levels.

[0156] Embodiment 11 : The apparatus of Embodiment 8, wherein the instructions to receive the first selected levels further control the apparatus to receive, for a first item of the items, a first selected level, wherein the instructions to calculate the additional second selected levels further control the apparatus to increase, based on the order, the first selected level of the first item to a first higher level, and wherein the instructions to generate the ranking are further based on the first higher level.

[0157] Embodiment 12: The apparatus of Embodiment 11, wherein the first higher level is two or more levels higher than the first selected level.

[0158] Embodiment 13: The apparatus of Embodiment 8, wherein the instructions to receivethe first selected levels further control the apparatus to receive, for a last item of the items, a first selected level, wherein the instructions to calculate the additional second selected levels further control the apparatus to decrease, based on the order, the first selected level of the last item to a first lower level, and wherein the instructions to generate the ranking are further based on the first lower level.

[0159] Embodiment 14: The apparatus of Embodiment 13, wherein the first lower level is two or more levels lower than the first selected level.

[0160] Embodiment 15: One or more computer-readable media comprising instructions configured, when executed by one or more processors, to control the one or more processors to perform steps comprising: generating a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the levels relate to different intensity levels; displaying, on a display of a user computing device, the first user interface; receiving, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; storing, based on the first user interactions, the first selected levels for the items; generating, without communicating with a remote server and based on the first selected levels, a second user interface; displaying, on the display, the second user interface; receiving, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculating, based on the order, additional second selected levels; generating, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generating, based on the ranking, a third user interface with the first selected levels storing the ranking.

[0161] Embodiment 16: The one or more computer-readable media of Embodiment 15, wherein the instructions, when executed by one or more processors, control the one or more processors to perform further steps comprising: displaying, based on the ranking, the third user interface on the display of the user computing device.

[0162] Embodiment 17: The one or more computer-readable media of Embodiment 15, wherein the instructions, when executed by one or more processors, control the one ormore processors to perform further steps comprising: receiving, from a remote server, a plurality of items and available levels, wherein the instructions for generating the first user interface with the items are based on the plurality of items, and wherein the two or more selectable levels are based on the available levels.

[0163] Embodiment 18: The one or more computer-readable media of Embodiment 15, wherein the instructions for receiving the first selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising receiving, for a first item of the items, a first selected level, wherein the instructions for calculating the additional second selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising increasing, based on the order, the first selected level of the first item to a first higher level, and wherein the instructions for generating the ranking are further based on the first higher level.

[0164] Embodiment 19: The one or more computer-readable media of Embodiment 18, wherein the first higher level is two or more levels higher than the first selected level.

[0165] Embodiment 20: The one or more computer-readable media of Embodiment 15, wherein the instructions for receiving the first selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising receiving, for a last item of the items, a first selected level, wherein the instructions for calculating the additional second selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising decreasing, based on the order, the first selected level of the last item to a first lower level, and wherein the instructions for generating the ranking are further based on the first lower level.

[0166] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

CLAIMSWhat is claimed is:

1. A computer- implemented method comprising: generating a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the two or more selectable levels relate to different intensity levels; displaying, on a display of a user computing device, the first user interface; receiving, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; storing the first selected levels for the items; generating, without communicating with a remote server and based on the first selected levels, a second user interface; displaying, on the display, the second user interface; receiving, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculating, based on the order, additional second selected levels; generating, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generating, based on the ranking, a third user interface with the first selected levels storing the ranking.

2. The computer-implemented method of claim 1, further comprising: displaying, based on the ranking, the third user interface on the display of the user computing device.

3. The computer-implemented method of claim 1, further comprising: receiving, from a remote server, a plurality of items and available levels,wherein the generating the first user interface with the items is based on the plurality of items, and wherein the two or more selectable levels are based on the available levels.

4. The computer-implemented method of claim 1 , wherein the receiving the first selected levels comprises receiving, for a first item of the items, a first selected level, wherein the calculating the additional second selected levels comprise increasing, based on the order, the first selected level of the first item to a first higher level, and wherein the generating the ranking is further based on the first higher level.

5. The computer-implemented method of claim 4, wherein the first higher level is two or more levels higher than the first selected level.

6. The computer-implemented method of claim 1, wherein the receiving the first selected levels comprises receiving, for a last item of the items, a first selected level, wherein the calculating the additional second selected levels comprise decreasing, based on the order, the first selected level of the last item to a first lower level, and wherein the generating the ranking is further based on the first lower level.

7. The computer-implemented method of claim 6, wherein the first lower level is two or more levels lower than the first selected level.

8. An apparatus comprising: a display; one or more processors; and a memory storing instructions that, when executed by the one or more processors, control the apparatus to:generate a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the two or more selectable levels relate to different intensity levels; display, on a display of a user computing device, the first user interface; receive, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; store the first selected levels for the items; generate, without communicating with a remote server and based on the first selected levels, a second user interface; display, on the display, the second user interface; receive, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculate, based on the order, additional second selected levels; generate, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generate, based on the ranking, a third user interface with the first selected levels storing the ranking.

9. The apparatus of claim 8, wherein the instructions further control the apparatus to: display, based on the ranking, the third user interface on the display of the user computing device.

10. The apparatus of claim 8, wherein the instructions further control the apparatus to: receive, from a remote server, a plurality of items and available levels, wherein the generated first user interface with the items is based on the plurality of items, and wherein the two or more selectable levels are based on the available levels.

11. The apparatus of claim 8,wherein the instructions to receive the first selected levels further control the apparatus to receive, for a first item of the items, a first selected level, wherein the instructions to calculate the additional second selected levels further control the apparatus to increase, based on the order, the first selected level of the first item to a first higher level, and wherein the instructions to generate the ranking are further based on the first higher level.

12. The apparatus of claim 11, wherein the first higher level is two or more levels higher than the first selected level.

13. The apparatus of claim 8, wherein the instructions to receive the first selected levels further control the apparatus to receive, for a last item of the items, a first selected level, wherein the instructions to calculate the additional second selected levels further control the apparatus to decrease, based on the order, the first selected level of the last item to a first lower level, and wherein the instructions to generate the ranking are further based on the first lower level.

14. The apparatus of claim 13, wherein the first lower level is two or more levels lower than the first selected level.

15. One or more computer-readable media comprising instructions configured, when executed by one or more processors, to control the one or more processors to perform steps comprising: generating a first user interface comprising functional regions associated with items, each functional region comprising two or more selectable levels, wherein the two or more selectable levels relate to different intensity levels; displaying, on a display of a user computing device, the first user interface; receiving, based on first user interactions with the first user interface, first selected levels for the items associated with the functional regions; storing the first selected levels for the items;generating, without communicating with a remote server and based on the first selected levels, a second user interface; displaying, on the display, the second user interface; receiving, based on second user interactions with the second user interface, second selected levels, wherein the second selected levels include an order in which the second selected levels were selected; calculating, based on the order, additional second selected levels; generating, based on the first selected levels from the first user interface, the second selected levels, and the calculated additional second selected levels, a ranking of the first selected levels; and generating, based on the ranking, a third user interface with the first selected levels storing the ranking.

16. The one or more computer-readable media of claim 15, wherein the instructions, when executed by one or more processors, control the one or more processors to perform further steps comprising: displaying, based on the ranking, the third user interface on the display of the user computing device.

17. The one or more computer-readable media of claim 15, wherein the instructions, when executed by one or more processors, control the one or more processors to perform further steps comprising: receiving, from a remote server, a plurality of items and available levels, wherein the instructions for generating the first user interface with the items are based on the plurality of items, and wherein the two or more selectable levels are based on the available levels.

18. The one or more computer-readable media of claim 15, wherein the instructions for receiving the first selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising receiving, for a first item of the items, a first selected level,wherein the instructions for calculating the additional second selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising increasing, based on the order, the first selected level of the first item to a first higher level, and wherein the instructions for generating the ranking are further based on the first higher level.

19. The one or more computer-readable media of claim 18, wherein the first higher level is two or more levels higher than the first selected level.

20. The one or more computer-readable media of claim 15, wherein the instructions for receiving the first selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising receiving, for a last item of the items, a first selected level, wherein the instructions for calculating the additional second selected levels, when executed by one or more processors, control the one or more processors to perform further steps comprising decreasing, based on the order, the first selected level of the last item to a first lower level, and wherein the instructions for generating the ranking are further based on the first lower level.

Images