Engagement prediction machine learning models for respiratory therapy

Abstract

Techniques for improved machine learning are provided. Exposure scores for content assets are determined. A value for an exploration rate of a machine learning model is selected based on determining that a first content asset has an exposure score that fails to satisfy one or more criteria. Selection scores for the content assets are generated using the machine learning model. The first content asset is selected for exposure based on at least one of (i) the selection scores or (ii) the exploration rate, and a second value for the exploration rate is selected based on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, where the second value is lower than the first.

Description

ENGAGEMENT PREDICTION MACHINE LEARNING MODELS FOR RESPIRATORY THERAPYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 716,036, filed November 4, 2024, the content of which is hereby incorporated by reference in its entirety.TECHNICAL FIELD

[0002] The present disclosure relates generally to machine learning, and more particularly, to use of machine learning to predict content engagement and / or usage engagement with respect to respiratory therapy.

[0003] Many individuals suffer from sleep-related and / or respiratory -related disorders such as, for example, Periodic Limb Movement Disorder (PLMD), Restless Leg Syndrome (RLS), Sleep-Disordered Breathing (SDB) such as Obstructive Sleep Apnea (OSA) and Central Sleep Apnea (CSA), Cheyne-Stokes Respiration (CSR), respiratory insufficiency, Obesity Hyperventilation Syndrome (OHS), Chronic Obstructive Pulmonary Disease (COPD), Neuromuscular Disease (NMD), and chest wall disorders. These disorders are often treated using respiratory therapy systems.

[0004] Each respiratory therapy system generally has a respiratory therapy device connected to a user interface (e.g., a mask) via a conduit and optionally a connector. The user wears the user interface and is supplied a flow of pressurized air from the respiratory therapy device via the conduit. The user interface generally is a specific category and type of user interface for the user, such as direct or indirect connections for the category of user interface, and full face mask, a partial face mask, nasal mask, or nasal pillows for the type of user interface. In addition to the specific category and type, the user interface generally is a specific model made by a specific manufacturer, e.g., AirFit™ F20 manufactured by ResMed.

[0005] In some cases, patient usage of the therapy system can be collected or monitored in order to evaluate user compliance with the therapy. For example, the duration of usage (e.g., the length of time that a user wears the user interface on a given night) may be determined. Generally, a wide variety of factors may affect compliance and usage. Some efforts to improve usage have included use of targeted media content to users, but existing approaches are often generic and insufficient. Although many patients would benefit from increased therapy usage (e.g., using their mask for a longer period of time each night), it is generally difficult or impossible to effectively predict and improve usage.P2712WO1 (RSMD / 0130PC-157584)

[0006] Improved systems and techniques to predict engagement and thereby improve therapy and outcomes are needed.SUMMARY

[0007] According to some implementations of the present disclosure, a method includes: determining a first set of exposure scores for a plurality of content assets; selecting a first value for an exploration rate of a machine learning model based at least in part on determining that at least a first content asset of the plurality of content assets has a first exposure score that fails to satisfy one or more criteria; generating a first set of selection scores for the plurality of content assets using the machine learning model; selecting the first content asset for exposure based on at least one of (i) the first set of selection scores or (ii) the first value for the exploration rate of the machine learning model; and selecting a second value for the exploration rate of the machine learning model based at least in part on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, wherein the second value is lower than the first value.

[0008] According to some implementations of the present disclosure, a system includes a control system and a memory. The control system includes one or more processors. The memory has stored thereon machine readable instructions. The control system is coupled to the memory, and any one of the methods disclosed herein is implemented when the machine executable instructions in the memory are executed by at least one of the one or more processors of the control system.

[0009] Other aspects provide processing systems configured to perform the aforementioned method as well as those described herein; non-transitory, computer-readable media comprising instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the aforementioned methods as well as those described herein; a computer program product embodied on a computer-readable storage medium comprising code for performing the aforementioned methods as well as those further described herein; and a processing system comprising means for performing the aforementioned methods as well as those further described herein.

[0010] The above summary is not intended to represent each implementation or every aspect of the present disclosure. Additional features and benefits of the present disclosure are apparent from the detailed description and figures set forth below.P2712WO1 (RSMD / 0130PC-157584)BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG. 1 depicts an example environment for predicting therapy engagement and selecting content, according to some implementations of the present disclosure.

[0012] FIG. 2 depicts an example workflow for training machine learning models to predict engagement, according to some implementations of the present disclosure.

[0013] FIG. 3 depicts an example workflow for predicting engagement using machine learning models, according to some embodiments of the present disclosure.

[0014] FIG. 4 is a flow diagram depicting an example method for training machine learning models to predict engagement, according to some embodiments of the present disclosure.

[0015] FIG. 5 is a flow diagram depicting an example method for facilitating content delivery using machine learning, according to some embodiments of the present disclosure.

[0016] FIG. 6 is a flow diagram depicting an example method for selecting content using machine learning, according to some embodiments of the present disclosure.

[0017] FIG. 7 depicts an example computing device configured to perform various aspects of the present disclosure, according to some embodiments disclosed herein.

[0018] While the present disclosure is susceptible to various modifications and alternative forms, specific implementations and embodiments thereof have been shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that it is not intended to limit the present disclosure to the particular forms disclosed, but on the contrary, the present disclosure is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present disclosure as defined by the appended claims.DETAILED DESCRIPTION

[0019] Embodiments of the present disclosure generally provide techniques for using machine learning to predict engagement and improve therapy device usage.

[0020] Generally, respiratory therapy refers to the use of a flow generator and user interface to deliver air and / or oxygen to a user, such as during sleep. For example, respiratory therapy may include use of continuous positive airway pressure (CPAP) devices, bi-level positive airway pressure (BiPAP) devices, and the like. Respiratory therapy can significantly improve the lives of users who engage in it. However, many users do not use such respiratory therapy devices sufficiently (e.g., for sufficient durations and / or sufficiently often) to achieve optimal results. A wide variety of approaches may be used to improve therapy usage (e.g., to increase the length of time the user uses the flow generator each night). For example, content may beP2712WO1 (RSMD / 0130PC-157584)provided to attempt to guide the user, suggest modifications or other things to try if the user is uncomfortable, encourage the user to continue treatment, and the like. However, when multiple alternatives exist (e.g., multiple types or pieces of content), how the user will engage with each piece of content, as well as the impact that such alternatives will have on the patient’s usage, is difficult or impossible to determine. This results in sub-optimal guidance in conventional approaches.

[0021] Aspects of the present disclosure provide techniques and architectures to predict content engagement and / or therapy engagement. That is, in some embodiments, the actions or interactions that a patient will have with a piece of delivered content can be predicted, such that specific content can be selected for specific users in such a way that the probability of engagement and / or therapy usage increase is maximized for each user. In some aspects, models may be trained to directly predict therapy engagement, such as by predicting therapy device usage (e.g., the number of hours that the user will use the respiratory therapy system per day), application usage (e.g., the number of times the user will open or use a therapy -related application, such as one used to view their sleep statistics), and the like. In some aspects, the models may be trained to predict engagement with the content itself (e.g., the media provided to the user) which acts as a proxy to predict the user’s engagement with the therapy (e.g., because increased engagement with therapy content may itself serve or lead to increased engagement or usage of the respiratory therapy).

[0022] In some embodiments, each piece of alternative content may include metadata or tags indicating characteristics of the content. For example, the metadata may indicate whether the content relates to or uses education, persuasion, incentivizing, coercion, training, restriction, environmental restructuring, modelling, enablement, and the like. Similarly, content may be tagged based on its format (e.g., image, video, audio, text, and the like), length or duration (e.g., amount of text and / or length of a video or audio), contents (e.g., characteristics of the user(s) depicted in the content, such as their demographics, the number of such users, roles of such users, and the like), and the like. In some embodiments, such features are used as input (along with user-specific features, as discussed in more detail below) to predict how a user will engage with such content if it is provided to them.

[0023] Advantageously, by using machine learning to predict the patient actions in response to various content, embodiments of the present disclosure enable improved content delivery that is more tailored to the individual user at the individual time. These predictions are more accurate and reliable (as well as being objective), as compared to more conventional systems that rely on random delivery, simple heuristics, or manually-selected content (whichP2712WO1 (RSMD / 0130PC-157584)is subjective, as well as prone to error and bias). In this way, by predicting which content is most likely to result in engagement, which may therefore result in improved respiratory therapy engagement, embodiments of the present disclosure enable overall therapy outcomes to be improved substantially (e.g., users are more likely to remain on therapy and / or increase their usage, which improves their outcomes).Example Workflow for Predicting Therapy Engagement and Selecting Content

[0024] FIG. 1 depicts an example workflow 100 for predicting therapy engagement and selecting content, according to some implementations of the present disclosure.

[0025] In the illustrated example, a user 105 (also referred to in some embodiments as a patient) engaged in a respiratory therapy can use / interact with a flow generator 110 and a user device 130. The flow generator 110 generally corresponds to a respiratory therapy device or system, such as for CPAP therapy. That is, the flow generator 110 may correspond to a device that generates and provides air flow to the user 105 as part of a respiratory therapy, such as via a user interface or respiratory mask connected to the flow generator 110 via a conduit.

[0026] In some embodiments, the user device 130 can generally correspond to a computing device associated with and / or controlled / used by the user 105 in connection with the respiratory therapy. For example, the user device 130 may correspond to a smartphone, tablet, laptop computer, smart device (e.g., an loT device), and the like. For example, the user 105 may interact with an application (e.g., via a graphical user interface (GUI)) using the user device 130 in order to control the flow generator 110, review information collected by the flow generator 110, review content relating to their therapy, and the like.

[0027] Although not depicted in the illustrated workflow 100, in some aspects, the flow generator 110 can collect, generate, or otherwise provide data relevant to the respiratory therapy, which may be transmitted to various devices such as the user device 130, one or more remote systems (e.g., servers) used to facilitate therapy, and the like. For example, the flow generator 110 may collect or generate data for one or more usage / sleep sessions for a variety of variables or features, such as the usage duration (e.g., the length of time the user 105 used the flow generator 110 during the session and / or the total or average duration across multiple sessions). In some aspects, the flow generator 110 can additionally collect other information relevant to the therapy, such as the number of times the user put on (donned) and removed (doffed) the user interface during a period of time and / or during a sleep session, the amount of air or mask leak of the flow generator, air pressure and / or flow volume settings, and the like.

[0028] In the illustrated example, a repository of engagement data 115 may generallyP2712WO1 (RSMD / 0130PC-157584)correspond to a data store that can reside in any suitable location, such as in the cloud. The engagement data 115 may include data for any number of users 105 and / or any number of user devices 130 and / or flow generators 110. Although a single store of engagement data 115 is depicted for conceptual clarity, in embodiments, there may be any number of discrete data stores that store the engagement data 115. Generally, the granularity and contents of the engagement data 115 may differ depending on the particular implementation. For example, the engagement data 115 may store, for each user 105, information about whether (and how) the user(s) 105 interacted with provided content via their user devices 130. As another example, the engagement data 115 may store, for each user 105, information about how the user(s) 105 engaged with their therapy (e.g., with the flow generator 110).

[0029] As illustrated, an engagement prediction system 120 can generally access the engagement data 115 to predict how various factors (e.g., different pieces of content that can be delivered to the user) will impact the user’s actions (e.g., interactions with the content, interactions with the flow generator 110, and the like), as discussed in more detail below. As used herein, accessing data may generally refer to receiving, retrieving, requesting, acquiring, obtaining, or otherwise gaining access to the data. The engagement prediction system 120 may generally be implemented using hardware, software, or a combination of hardware and software, and may execute in any suitable location, including the cloud. In some embodiments, the engagement prediction system 120 trains and / or uses machine learning to predict the the user’s interactions.

[0030] In some aspects, as discussed in more detail below, the engagement prediction system 120 predicts, for each specific piece of content (e.g., from a library of content), predicted engagement or interactions of the user 105 if the content is delivered to the user 105 (e.g., during a current day). In some aspects, prior to generating such predictions, the engagement prediction system 120 may first update, re-train, or refine its machine learning model(s). That is, the engagement prediction system 120 may update its model each day when new engagement data 115 is available. For example, when new engagement data 115 is available (e.g., in the morning, when users 105 use their user devices 130 to review the night’s statistics from the flow generator 110 and / or view new content relating to therapy), the engagement prediction system 120 may use this updated engagement data 115 (e.g., indicating how the user engaged with the newly delivered content and / or how the user engaged with their flow generator 110 overnight, along with indications of the content that was delivered to the user 105) to refine the model(s), then use these refined model(s) to generate updated predictions for the subsequent content delivery.P2712WO1 (RSMD / 0130PC-157584)

[0031] Generally, the particular machine learning architecture used may vary depending on the particular implementation. In some embodiments, the engagement prediction system 120 uses a factorization machine (FM) architecture. FM models are a class of supervised machine learning model that are able to capture interactions between features within high dimensional sparse datasets efficiently. While some other architectures struggle to adequately learn on sparse data, FM models may be able to handle this sparsity effectively.

[0032] In the illustrated workflow 100, the predictions of the engagement prediction system 120 are used by a content system 125 to select and / or deliver specific content to the users 105. The content system 125 may generally be implemented using hardware, software, or a combination of hardware and software, and may execute in any suitable location, including the cloud. Although depicted as discrete components for conceptual clarity, in some embodiments, the content system 125 and engagement prediction system 120 may be implemented as components of a single system.

[0033] In some embodiments, the content system 125 uses the engagement predictions to select and / or deliver specific content to each user 105. For example, in the illustrated workflow 100, the content system 125 can transmit or otherwise cause the content to be delivered to the user device 130 associated with the user 105. Generally, the particular method of delivery for the content may vary depending on the particular content. For example, in some embodiments, the content is delivered via text message, email, and the like. In some aspects, the content is delivered as a card, notification, or snippet via an application executing on the user device 130. For example, the user 105 may use the application to monitor and / or control their respiratory therapy (e.g., to review sleep statistics from each night, to update settings of the flow generator 110, and the like). In some embodiments, the content may be included as a card or insert within such an application (e.g., on the main screen when the application is opened).

[0034] This can enable the content system 125 and engagement prediction system 120 to accurately and reliably predict user engagement and therapy usage changes in order to tailor content delivery specifically for each individual user, which may result in improved usage of the flow generator 110 (e.g., because users are more likely to engage with the content provided, and engagement with the content is likely to result in increased or better use of the flow generator 110) and improved therapy outcomes.Example Workflow for Training Machine Learning Models to Predict Engagement

[0035] FIG. 2 depicts an example workflow 200 for training machine learning models to predict engagement, according to some implementations of the present disclosure. In someP2712WO1 (RSMD / 0130PC-157584)embodiments, the workflow 200 is used by an engagement prediction system (e.g., a computing system), such as the engagement prediction system 120 of FIG. 1.

[0036] In the illustrated workflow 200, engagement data 115 is processed using a label component 205 to generate labeled exemplars including interaction exemplars 210, noninteraction exemplars 215, and non-display ed exemplars 220. As discussed above, the engagement data 115 may generally correspond to or comprise records or logs of previous content servicing events (e.g., when content is selected and / or provided for output to a user, such as the user 105 of a respiratory therapy system) and / or records of previous user engagement in the respiratory therapy (e.g., average daily hours of usage, engagement with the therapy application on their smartphone, and the like). For example, each record in the engagement data 115 may indicate information such as the date and / or timestamp when the content was selected and / or provided, what content item was provided, and the like. In some aspects, the engagement data 115 can indicate any relevant characteristics of the user and / or therapy, as discussed above, such as the user’s usage duration of their flow generator, the number of times the user donned and doffed their mask overnight, the amount of air or mask leak, the delivered air pressure and / or flow volume settings, and the like.

[0037] In some aspects, the engagement data 115 may indicate, for one or more records, relevant characteristics of the content itself to be used as model input, such as whether the user viewed and / or interacted with the content from the previous day or delivery, the frequency with which the user views and / or interacts with the content, and the like. As another example, the engagement data 115 may indicate, for one or more records, characteristics of the user’s engagement with therapy, such as the number of hours that the user used the therapy device after viewing the content. Generally, the engagement data 115 may include any relevant information used to drive content selection (e.g., any data that is evaluated using the model to select or suggest content).

[0038] In some embodiments, the engagement data 115 may have a substantial number of records (e.g., because the engagement prediction system is used to serve content to a large number of patients over a relatively long period of time). However, in many cases, a relatively large proportion of the records in the engagement data 115 may be considered “noise” or unusable for machine learning purposes. Specifically, records where the user engaged or interacted with the content (e.g., interaction exemplars 210) may be useful as positive exemplars indicating content selections that achieved the desired goal (e.g., caused the user to interact with the content and become more engaged with their therapy). Similarly, records where the user chose to ignore or close the content (e.g., non-interaction exemplars 215) mayP2712WO1 (RSMD / 0130PC-157584)be useful as negative exemplars indicating content selections that did not achieve the goal (e.g., where the user refused to interact with the content and may become further disengaged with their therapy).

[0039] However, in many realistic technical environments, a large portion (e.g., the substantial majority) of the records may correspond to times when the content was never actually delivered to the user (e.g., the non-displayed exemplars 220). That is, a large portion of the records may correspond to times when content was selected for delivery, but the user was never actually presented with the selected content. In many conventional systems, this type of record is generally regarded as useless noise, as they may not be indicative of the quality of the content selection. That is, in conventional systems, it is difficult or impossible to quantify the selection (and how the user would respond to the selection) if the user never actually saw the selection. In conventional approaches, therefore, these records are often discarded or ignored as noise, and are not used to train the models.

[0040] However, in some environments, such as the therapy setting discussed above with reference to FIG. 1, these non-displayed exemplars 220 may be useful. For example, suppose the content is provided as a card or insert in a therapy application (e.g., executing on the user device 130 of FIG. 1) used to control and / or monitor the patient’s respiratory therapy (e.g., to access data from and / or control the operations of the flow generator 110 of FIG. 1). If the user does not open or use the application on a given day, they will not receive the selected content and will not have a chance to interact or ignore it. While this lack of action is irrelevant for many conventional systems, it may provide useful information for the engagement prediction system in conjunction with the therapy. For example, the user refraining from opening the application may, itself, be useful information to predict the user’s interaction and engagement with the therapy and / or with future content related to the therapy.

[0041] In the illustrated example, therefore, the label component 205 may evaluate each record in the engagement data 115 to determine whether the content was actually delivered or displayed. For example, after content is selected and provided (e.g., by the content system 125 of FIG. 1), the system may monitor the delivery to determine when (and if) the user interacts with it (e.g., whether it is displayed to the user, and if so, when it was displayed and whether the user interacted with the content or ignored it). In the illustrated workflow 200, if a given content selection was never actually displayed or output, the label component 205 labels the record as a non-displayed exemplar 220. If the content was output and the user ignored it, closed it, or took some other action to decline engaging with the content, the label component 205 may label the record as a non-interaction exemplar 215 (e.g., labeling the record to indicateP2712WO1 (RSMD / 0130PC-157584)the specific action performed by the user). If the content was output and the user engaged with it, the label component 205 may label the record as an interaction exemplar 210 (e.g., labeling the record to indicate the particular action the user performed to engage with the content).

[0042] In this way, the label component 205 enables non-displayed exemplars 220, which conventional systems ignore or discard, to be used to train the machine learning models. That is, the engagement prediction system may be able to use substantially more data to train the model(s), as compared to conventional systems, by imparting useful meaning into the “noisy” samples. Further, in the respiratory therapy setting, there may be multiple records for the same user (e.g., over one or more days). Further, in some embodiments, by utilizing non-displayed exemplars 220, the engagement prediction system can prevent fragmentation of a give user’s journey through the therapy. That is, the engagement prediction system can better understand and evaluate user engagement (e.g., driving improved content selections) when these “gaps” in their timelines (corresponding to days they did not open the application or view the content) are included in the training data.

[0043] In the illustrated workflow 200, the interaction exemplars 210, non-interaction exemplars 215, and non-displayed exemplars 220 are accessed by a weighting component 225 to generate weighted exemplars 230. The weighting component 225 is generally used to encode the labeled exemplars to facilitate their use in training the machine learning model(s). For example, in some conventional approaches, “positive” exemplars (e.g., examples where the user engaged with the content or performed some other desirable action) are often given a positive weight (e.g., one) while “negative” exemplars (e.g., examples where the user did not engage with the content or performed some other non-desirable action) are given a negative weight (e.g., negative one). However, this simplistic labeling process does not enable deeper understanding of the user interactions based on the specific actions taken, nor does it enable understanding of the role that non-display events have. In some embodiments, a trinary labeling system (e.g., where interaction exemplars 210 may be encoded using a label value “one,” non-interaction exemplars 215 may be encoded using a label value of “negative one,” and non-displayed exemplars 220 may be encoded using a label value of “zero”) to help differentiate between choosing to ignore the content and never seeing the content.

[0044] Further, due to the fact that there may be many more non-displayed exemplars 220, the class imbalance may cause training difficulties if improper weights are used. In some aspects, the weighting component 225 may weight the labeled exemplars based on their relative importance to the system (e.g., weighting exemplars that indicate desirable behavior highly) and / or based on their relative frequency in the training data (e.g., where the weight may beP2712WO1 (RSMD / 0130PC-157584)inversely proportional to the frequency). For example, in some embodiments, the weighting component 225 may use a set of hyperparameters to weight the samples, such as by assigning interaction exemplars 210 with a high weight (e.g., ten or twenty), assigning non-interaction exemplars with a medium weight (e.g., three) and assigning non-displayed exemplars 220 with a low weight (e.g., one).

[0045] In the illustrated example, these weighted trinary exemplars can then be accessed by a training component 240 to train the machine learning model(s) 250. Generally, the training component 240 may use a variety of operations to train the machine learning model 250 depending on the particular implementation and architecture. For example, in some embodiments, the machine learning model 250 may comprise a FM, as discussed above. In some embodiments, the training component 240 may process a given weighted exemplar (e.g., the input features of a given exemplar) using the machine learning model 250 to generate a content selection (e.g., to generate selection scores for a set of content items). The training component 240 may then compare the generated score(s) of a given asset (e.g., each indicating the probability that the user will perform one or more interactions or actions with respect to the content, such as whether the will click it, ignore it, and the like) with the actual label of the exemplar (e.g., generated by the label component 205) to generate a loss. This loss may then be used to refine the parameters of the machine learning model 250 to provide improved and more accurate predictions regarding user engagement.

[0046] Generally, the training component 240 may use a variety of techniques and formulations to generate the loss used to train the machine learning model 250. For example, in some embodiments, the training component 240 may use a modified negative log loss function as the loss function for training the machine learning model 150. That is, while conventional negative log-loss may be useful for binary problems, in some embodiments, the training component 240 may use a modified formulation to account for the trinary nature of the problem.

[0047] For example, in some embodiments, the training component 240 may define the loss for positive exemplars (e.g., loss generated when an interaction exemplar 210 is being processed) using one formulation, while defining the loss for negative exemplars (e.g., the loss generated when non-interaction exemplars 215 and non-displayed exemplars 220 are being processed) using a second formulation. As another example, in some embodiments, the training component 240 may define the loss for “action” exemplars (e.g., records where the user took an affirmative action, such as the interaction exemplars 210 and the non-interaction exemplars 215) using one loss formulation, while using a second loss formulation for the non-P2712WO1 (RSMD / 0130PC-157584)displayed exemplars 220.

[0048] As one example, suppose the prediction generated by the machine learning model 250 for a given exemplar is represented by pred, the label of the exemplar is y, and the weight of the exemplar is w. In some embodiments, the training component 240 may define the loss for some exemplars (e.g., the non-displayed exemplars 220) as w * (ln(l + e~pred) +where e is Euler’s number (e.g., approximately 2.71828), while the loss for other exemplars (e.g., the interaction exemplars 210 and / or the non-interaction exemplars 215) may be defined as w1). As an additional example, in some embodiments, the loss may be defined using other formulations, such as a Bayesian Personalized Ranking (BPR) loss.

[0049] Regardless of the particular formulation used, in the illustrated example, the machine learning model 250 may then be deployed for runtime use. In some embodiments, the machine learning model 250 may be continuously or iteratively updated. For example, after using the machine learning model 250 to generate content selections for a given day or interval, the engagement prediction system may collect information (e.g., new engagement data 115) relating to the content and / or therapy, and may then use this updated information to further refine the machine learning model 250. In this way, the machine learning model 250 can be rapidly and frequently updated to learn how to best recommend content for individual users.Example Workflow for Predicting Engagement Using Machine Learning Models

[0050] FIG. 3 depicts an example workflow 300 for predicting engagement using machine learning models, according to some embodiments of the present disclosure. In some embodiments, the workflow 300 is used by an engagement prediction system (e.g., a computing system), such as the engagement prediction system 120 of FIG. 1 and / or the engagement prediction system discussed above with reference to FIG. 2.

[0051] In the illustrated example, a set of content assets 305 is evaluated using an exposure component 310 to generate a set of hyperparameters 315. Generally, as discussed above, the content assets 305 may correspond to one or more libraries or repositories of content, such as cards, media, articles, and the like, which may be provided to users (e.g., the user 105 of FIG. 1) such as patients engaging in respiratory therapy. For example, in some embodiments, the content assets 305 may be created and / or curated by healthcare entities involved in the respiratory therapy, such as manufacturers of therapy devices (e.g., creating tutorials and suggestions to help with common issues with the hardware), doctors and nurses (e.g., providing guidance with respect to therapy progression), and the like.P2712WO1 (RSMD / 0130PC-157584)

[0052] Generally, the content assets 305 may be modified (e.g., adding new assets, removing old assets, updating or modifying current assets, and the like) with little or no restriction (e.g., at any time). For example, designers may continuously work to generate new content assets 305 for distribution in an effort to make relevant and non-stale content that will improve patient engagement. In some conventional systems, such new content may be at a significant disadvantage in terms of recommendation. For example, because the machine learning model may have been trained on a given set of content, the new content may be poorly recommended by the model simply because it is new (e.g., it differs from what the model was trained using). While it is desirable for the model to converge quickly, it is also desirable to avoid excluding new content to continue relying on the old.

[0053] In the illustrated example, to mitigate such concerns, the exposure component 310 can be used to generate dynamic hyperparameters 315 for the model execution based on the content assets 305 in order to improve the adaptability of the system. In some embodiments, the machine learning model 250 may use an exploration-exploitation approach, where the system seeks to “exploit” good content (e.g., generating high scores for content assets that have previously generated good engagement) while also “exploring” new content (e.g., to evaluate how effective new assets are).

[0054] In some embodiments, this tradeoff between exploration and exploitation is defined by a hyperparameter which may be referred to in some aspects as an exploration rate (also referred to as the explore-exploit ratio in some aspects). For example, in some conventional approaches, a static exploration rate such as 0.1 may be used, indicating that the model has a 90% probability of exploiting existing assets (e.g., of a population of users to whom content is being distributed, 90% will be selected for exploiting known assets) and a 10% probability of testing new (unknown) assets (e.g., 10% of users will be provided new assets for exploration). Further, in some embodiments, the model may use weights for each content asset to affect the probability that the given asset will be selected during the exploration phase (e.g., where higher weight for a given asset may correspond to a higher probability that the given asset is selected, if the user has been selected for exploration of new content).

[0055] In some embodiments, rather than using fixed values for these parameters (as many conventional systems do), the exposure component 310 can generate dynamic hyperparameters 315 to improve the systems adaptability and ability to incorporate new assets. In some embodiments, the exposure component 310 can evaluate the exposure of each asset of the set of content assets 305 to define the hyperparameters 315. For example, the exposure component 310 may determine or generate a respective exposure score for each respective asset, where theP2712WO1 (RSMD / 0130PC-157584)exposure score indicates how much exposure the asset has received (e.g., based on the number of times the given asset was selected, the number of times the given asset was actually displayed or output, and / or the number of times the given asset was interacted with by one or more users). In some embodiments, the exposure component 310 may compare the exposure score (e.g., the number of impressions for each asset, where an impression corresponds to a time when the asset was actually delivered or output regardless of whether the user interacted with it) against one or more exposure criteria (e.g., minimum thresholds).

[0056] In some embodiments, any media assets with an exposure score below the threshold may be referred to as underexposed or inadequately explored, while media assets with an exposure score above the threshold may be referred to as sufficiently explored. In some embodiments, the particular definition or methodology used to generate the exposure score may vary depending on the particular implementation. For example, in some embodiments, the exposure score corresponds to the number of impressions (e.g.. the number of times the content was actually output to a user, regardless of how they responded) during a window or all-time (e.g., since the asset was introduced). Further, in some embodiments, the exposure criteria (e.g., the minimum threshold) may vary depending on the particular implementation.

[0057] In some embodiments, the exposure component 310 may define values for the exploration rate of the model and / or the weight of one or more of the content assets 305 based at least in part on the exposure scores for each asset. For example, the exposure component 310 may use a first value for the exploration rate if a threshold number or percentage of content assets 305 do not satisfy the exposure criteria (e.g., if at least one asset has an insufficient exposure), while using a second (lower) exploration rate when all of the content assets 305 have been adequately explored (e.g., when the exposure scores of each asset satisfy the criteria. In this way, the exposure component 310 may dynamically increase the exploration of the) model (e.g., increasing the exploration rate) to cause the machine learning model to be more likely to explore the content library. In some aspects, rather than using a bi-level exploration rate (e.g., high or standard), the exposure component 310 may use a decaying exploration rate (e.g., progressively reducing the exploration rate as the smallest exposure score increases until a threshold is reached).

[0058] As another example, the exposure component 310 may use a first weight for content assets 305 that do not satisfy the exposure criteria (e.g., for any assets having insufficient exposure), while using a second (lower) weight for any content assets 305 that have been adequately explored (e.g., for assets having exposure scores that satisfy the criteria). As discussed above, the weight of a given asset may affect how frequently it is selected in the caseP2712WO1 (RSMD / 0130PC-157584)of content exploration (e.g., if the system determines to explore new content for one or more users). In this way, the exposure component 310 may dynamically increase the probability that poorly explored (e.g., new) assets will be selected for exploration, as compared to well- explored assets. In some aspects, rather than using a bi-level weight (e.g., high or standard), the exposure component 310 may use a decaying weight approach (e.g., progressively reducing the weight of a given asset as the exposure score of the asset increases, until a threshold is reached).

[0059] In these ways, the exposure component 310 can dynamically modify multiple hyperparameters to control the operations of the machine learning model 250 in order to account for new or modified assets. In the illustrated example, the hyperparameters 315 are provided to a prediction component 325. The prediction component 325 also accesses the trained machine learning model 250, as well as sets of user data 320 to be used as input to the model. In some aspects, as discussed above, the user data 320 may generally include or indicate any relevant information used to drive content selection, such as the demographics of each user, respiratory therapy information for each user, and the like.

[0060] As illustrated, the prediction component 325 can process each sample of user data 320 (e.g., for a given user) using the machine learning model 250 and based on the generated hyperparameters 315 to generate a corresponding selection of content 330 for the user (based on associated selection scores 335). For example, as discussed above, the prediction component 325 may, for a given user and with respect to each respective content asset of the set of content assets 305, generate a respective selection score 335. Based on the selection scores, the prediction component 325 (or another system or component) can then select a respective piece of content 330 for each respective user (e.g., selecting the content having the highest selection score). This content may then be provided for the user (which, as discussed above, may include actually outputting the content to the user and / or attempting to provide the content but failing to do so, such as if the user does not open their application that day).

[0061] These dynamic selections using responsive or adaptive hyperparameters 315 can significantly improve the operations of the prediction system, as well as the engagement system and content delivery system overall.Example Method for Training Machine Learning Models to Predict Engagement

[0062] FIG. 4 is a flow diagram depicting an example method 400 for training machine learning models to predict engagement, according to some embodiments of the present disclosure. In some embodiments, the method 400 is performed by an engagement predictionP2712WO1 (RSMD / 0130PC-157584)system (e.g., a computing system), such as the engagement prediction system 120 of FIG. 1 and / or the engagement prediction system discussed above with reference to FIGS. 2-3.

[0063] At block 405, the engagement prediction system accesses a set of one or more engagement records (e.g., the engagement data 115 of FIGS. 1-2). In some embodiments, as discussed above, each engagement record generally corresponds to or indicates a prior time or event when content was selected and / or provided to a user and / or user device. For example, an engagement record may correspond to a time when a specific media asset was delivered to a specific user in the past. In some aspects, each engagement record includes one or more userspecific features (e.g., the user’s name, demographics, how long they have been engaged in the respiratory therapy, and the like). In some embodiments, the engagement records can further include information relating to the card(s) that was or were selected for the user, as well as how the user responded to the content (e.g., whether the user engaged with it, did not engage with it, or was never even provided with it, whether the user increased their usage of the therapy system, and the like).

[0064] At block 410, the engagement prediction system selects an engagement record from the set of records. In some aspects, the engagement prediction system may select the engagement record using a variety of criteria or techniques, including randomly or pseudo- randomly, as each record may be used to train the model.

[0065] At block 415, the engagement prediction system labels the selected record based on the user engagement with the content and / or the therapy, as indicated in the record. For example, as discussed above, the engagement prediction system may label the record to indicate that the user engaged with the content (e.g., as an interaction exemplar 210 of FIG. 2), to indicate that the user ignored or did not engage with the content (e.g., as a non-interaction exemplar 215 of FIG. 2), to indicate that the user did not see the content, such as if the user never triggered the content to be output and / or the content was never displayed (e.g., as a nondisplayed exemplar 220 of FIG. 2), to indicate the number of hours that the user used the respiratory therapy after viewing the content, and the like.

[0066] At block 420, the engagement prediction system may weight the selected record based at least in part on the engagement discussed above. For example, in some embodiments, the engagement prediction system may define the weight based on the label of the exemplar (e.g., giving higher weights to exemplars in the “interaction” class, as compared to exemplars in the “non-interaction” class, which in turn have higher weight than the “not displayed” class).

[0067] At block 425, the engagement prediction system determines whether there is at least one additional record remaining to train the model. Generally, at block 425, the engagementP2712WO1 (RSMD / 0130PC-157584)prediction system may evaluate a variety of termination criteria, such as to determine whether a defined amount of time or computing resources have been spent training, whether the model has reached a desired accuracy, whether sufficient time remains to keep training prior to the next content scoring deadline, whether there are any additional records available to use for training, and the like. Although the illustrated example depicts evaluation of each record sequentially for conceptual clarity, in some aspects, the engagement prediction system may evaluate some or all of the records entirely or partially in parallel.

[0068] If, at block 425, the engagement prediction system determines that the criteria are not met, the method 400 returns to block 410 to continue gathering data. If, at block 425, the engagement prediction system determines that one or more criteria are satisfied, the method 400 continues to block 430.

[0069] At block 430, the engagement prediction system trains the machine learning model based on the labeled and / or weighted exemplars, as discussed above. For example, in some embodiments, the engagement prediction system may process the input portion of a given record using the model in order to generate one or more selection scores for the data. In some embodiments, the engagement prediction system may then generate a loss using one or more loss formulations (e.g.,. where the formulation is defined based at least in part on the class of the input, in some aspects), and may update the parameters of the model using this loss (e.g., using backpropagation).

[0070] In this way, the engagement prediction system may train the model to predict user engagement based on content delivery, including use of a vast store of data that conventionally is ignored or discarded, resulting in substantially improved prediction models. In some embodiments, as discussed above, the method 400 may be performed repeatedly (e.g., daily) as new engagement records become available.Example Method for Facilitating Content Delivery using Machine Learning

[0071] FIG. 5 is a flow diagram depicting an example method 500 for facilitating content delivery using machine learning, according to some embodiments of the present disclosure. In some embodiments, the method 500 is performed by an engagement prediction system (e.g., a computing system), such as the engagement prediction system 120 of FIG. 1 and / or the engagement prediction system discussed above with reference to FIGS. 2-4.

[0072] At block 505, the engagement prediction system accesses user data (e.g., the user data 320 of FIG. 3). In some embodiments, as discussed above, the user data may generally correspond to any data used as input to the machine learning model to drive content scoring,P2712WO1 (RSMD / 0130PC-157584)such as characteristics of the user, previous interactions of the user, and the like.

[0073] At block 510, the engagement prediction system accesses a set of content assets (e.g., the library of content assets 305 of FIG. 3). As discussed above, the content assets may generally include any media content (e.g., articles, blogs, videos, images, and the like) that may be provided or suggested to users, such as in conjunction with a respiratory therapy system.

[0074] At block 515, the engagement prediction system selects a content asset for evaluation. In some aspects, the engagement prediction system may select the content asset using a variety of criteria or techniques, including randomly or pseudo-randomly, as each asset may be evaluated.

[0075] At block 520, the engagement prediction system determines an exposure score for the selected content asset. For example, as discussed above, the engagement prediction system may determine the number of times the asset has been output to one or more users (e.g., in total), and / or may determine whether the exposure satisfies one or more criteria (e.g., a minimum impression threshold).

[0076] At block 525, the engagement prediction system selects one or more asset-specific hyperparameters based on the exposure score. For example, as discussed above, the engagement prediction system may select a weight for the selected content asset. In some embodiments, as discussed above, the engagement prediction system may use a bi-level weight (e.g., either a high weight for under-explored assets, or a standard or default weight for all other assets), or may use a more dynamic weight (e.g., a decaying weight with three or more distinct weight values depending on the particular exposure score).

[0077] At block 530, the engagement prediction system determines whether there is at least one additional content asset that has not yet been evaluated. If so, the method 500 returns to block 515. If all of the content assets have been evaluated, the method 500 continues to block 535. Although the illustrated example depicts evaluation of each content asset sequentially for conceptual clarity, in some aspects, the engagement prediction system may evaluate some or all of the assets entirely or partially in parallel.

[0078] At block 535, the engagement prediction system generates a set of selection scores for the set of content assets using the machine learning model. In some aspects, as discussed above, the engagement prediction system may process the user data using the model to generate a respective selection score for each respective content asset, where the selection scores generally indicate the probability that the content should be selected and / or that the user will engage with the content and / or therapy after viewing the content. In some embodiments, as discussed above, the engagement prediction system can use an explore-exploit framework (e.g.,P2712WO1 (RSMD / 0130PC-157584)using a FM) where the exploration rate and / or the content-specific weights are dynamically determined based on the exposure of each piece of content, as discussed in more detail below.

[0079] At block 540, the engagement prediction system selects one or more hyperparameters for the machine learning model itself (e.g., for the exploration and exploitation framework) based on the exposure scores of the set of content assets. For example, as discussed above, the engagement prediction system may select an exploration rate for the model based on the exposures. In some embodiments, as discussed above, the engagement prediction system may use a bi-level exploration rate (e.g., either a high exploration rate if any or at least a threshold number of the assets are under-explored, or a standard or default exploration rate if all or a defined proportion of the assets are sufficiently explored), or may use a more dynamic rate (e.g., a decaying exploration rate with three or more distinct rate values depending on the particular exposure scores of the content assets).

[0080] At block 545, the engagement prediction system selects a content asset for the user based (at least in part) on the selection scores. In some embodiments, as discussed above, the engagement prediction system selects the content based on the exploration rate determined at block 540. For example, the engagement prediction system may probabilistically determine whether to “explore” content for the user or “exploit” known content for the user based on the exploration rate. That is, if the exploration rate (selected at block 540) is n, the engagement prediction system may select n percent of the total set of users (e.g., randomly) for exploration.

[0081] In some embodiments, if the user is not selected for exploration (e.g., the user is in the exploitation group), the engagement prediction system can select the content asset, at block 545, having the highest score. That is, the engagement prediction system may evaluate a ranked list of content assets (ranked based on the selection scores generated at block 535), selecting and returning the highest-ranked content for delivery to the user. Alternatively, in some embodiments, the engagement prediction system may probabilistically select a content asset from the library, where the probabilistic selection is weighted based on the selection scores (e.g., such that assets with higher scores are more likely to be selected).

[0082] Further, in the illustrated example, if the user is selected for exploration (rather than exploitation), the engagement prediction system may probabilistically select a content asset weighted based on the asset-specific weights determined at block 525 (e.g., where content having a higher weight is more likely to be selected). For example, the engagement prediction system may generate or access a quasi -randomly sorted list where the ordering is determined (at least in part) based on the content-specific weights. This may allow the engagement prediction system to probabilistically (e.g., somewhat randomly) select content for exploration,P2712WO1 (RSMD / 0130PC-157584)where content having a higher weight is more likely to be selected.

[0083] As discussed above, the engagement prediction system may then return the selected asset for the user. In some aspects, as discussed above, returning the selected asset may include selecting the content for exposure, such as determining to output or suggest the content via a GUI (e.g., the GUI of the user device 130 of FIG. 1). For example, this may include providing the asset to the user (e.g., via a user device), and / or instructing or facilitating another system to provide the asset (e.g., a content delivery system such as the content system 125 of FIG. 1).

[0084] In some embodiments, as discussed above, the content asset may be delivered in the form of a suggested or recommended article or media, such as when the user opens an application that they use to control or interface with their respiratory therapy system(s). In some embodiments, the engagement prediction system (or another system) may monitor the user’s interactions with this application in order to determine the engagement, such as whether the content was displayed or suggested at all (e.g., whether the user opened the application and / or navigated to the portion of the application where the content is suggested), whether the user clicked on or otherwise selected the content (e.g., to read more), whether the user ignored the content, whether the user subsequently engaged with their therapy system (e.g., based on hours used), and the liked.

[0085] At block 550, the engagement prediction system can optionally update the machine learning model based at least in part on the feedback determined above. For example, as discussed above, the engagement prediction system can generate a new engagement record based on how the user engaged (or did not engage) with the content and / or therapy, and may use this record to update the model in order to generate improved predictions. In some embodiments, as discussed above, the engagement prediction system can also optionally update the exposure score(s) of any asset(s) as appropriate based on the interactions.Example Method for Selecting Content Using Machine Learning

[0086] FIG. 6 is a flow diagram depicting an example method 600 for selecting content using machine learning, according to some embodiments of the present disclosure. In some embodiments, the method 600 is performed by an engagement prediction system (e.g., a computing system), such as the engagement prediction system 120 of FIG. 1 and / or the engagement prediction system discussed above with reference to FIGS. 2-5.

[0087] At block 605, a first set of exposure scores for a plurality of content assets (e.g., the content assets 305 of FIG. 3) is determined.

[0088] At block 610, a first value for an exploration rate (e.g., a hyperparameter 315 ofP2712WO1 (RSMD / 0130PC-157584)FIG. 3) of a machine learning model (e.g., the machine learning model 250 of FIG. 3) is selected based at least in part on determining that at least a first content asset of the plurality of content assets has a first exposure score that fails to satisfy one or more criteria.

[0089] At block 615, a first set of selection scores (e.g., the selection scores 335 of FIG. 3) for the plurality of content assets is generated using the machine learning model and based on the first value for the exploration rate of the machine learning model.

[0090] At block 620, the first content asset is selected for exposure (e.g., via the user device 130 of FIG. 1) based at least in part on the first set of selection scores.

[0091] At block 625, a second value for the exploration rate of the machine learning model is selected based at least in part on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, wherein the second value is lower than the first value.Example Processing System for Engagement Prediction Machine Learning

[0092] FIG. 7 depicts an example computing device 700 configured to perform various aspects of the present disclosure, according to some embodiments disclosed herein. Although depicted as a physical device, in embodiments, the computing device 700 may be implemented using virtual device(s), and / or across a number of devices (e.g., in a cloud environment). In one embodiment, the computing device 700 corresponds to any element or aspect of an engagement prediction system (e.g., a computing system), such as the engagement prediction system 120 of FIG. 1 and / or the engagement prediction system discussed above with reference to FIGS. 2-6

[0093] As illustrated, the computing device 700 includes a CPU 705, memory 710, storage 715, a network interface 725, and one or more input / output (I / O) interfaces 720. In the illustrated embodiment, the CPU 705 retrieves and executes programming instructions stored in memory 710, as well as stores and retrieves application data residing in storage 715. The CPU 705 is generally representative of a single CPU and / or GPU, multiple CPUs and / or GPUs, a single CPU and / or GPU having multiple processing cores, and the like. The memory 710 is generally included to be representative of a random access memory. Storage 715 may be any combination of disk drives, flash-based storage devices, and the like, and may include fixed and / or removable storage devices, such as fixed disk drives, removable memory cards, caches, optical storage, network attached storage (NAS), or storage area networks (SAN).

[0094] In some embodiments, I / O devices 735 (such as keyboards, monitors, etc.) are connected via the I / O interface(s) 720. Further, via the network interface 725, the computingP2712WO1 (RSMD / 0130PC-157584)1 device 700 can be communicatively coupled with one or more other devices and components (e.g., via a network, which may include the Internet, local network(s), and the like). As illustrated, the CPU 705, memory 710, storage 715, network interface(s) 725, and I / O interface(s) 720 are communicatively coupled by one or more buses 730.

[0095] In the illustrated embodiment, the memory 710 includes a label component 750, a weighting component 755, a training component 760, an exposure component 765, and a prediction component 770, which may perform one or more embodiments discussed above. Although depicted as discrete components for conceptual clarity, in embodiments, the operations of the depicted components (and others not illustrated) may be combined or distributed across any number of components. Further, although depicted as software residing in memory 710, in embodiments, the operations of the depicted components (and others not illustrated) may be implemented using hardware, software, or a combination of hardware and software.

[0096] In some embodiments, the label component 750 can be used to generate labels for engagement records (e.g., from the engagement data 115 of FIG. 2), as discussed above. For example, the label component 750 may correspond to the label component 205 of FIG. 2. In some embodiments, the label component 750 can generate labels indicating whether the user interacted with delivered content (e.g., to generate an interaction exemplar 210 of FIG. 2), did not interact with the content (e.g., to generate a non-interaction exemplar 215 of FIG. 2), and / or did not view the content at all (e.g., to generate a non-display ed exemplar 220 of FIG. 2).

[0097] In some embodiments, the weighting component 755 can be used to generate weights for each training exemplar, as discussed above. For example, the weighting component 755 may correspond to the weighting component 225 of FIG. 2. In some embodiments, the weighting component 755 can weight each exemplar based in part on what class the exemplar is in (e.g., weighting interaction exemplars higher than non-interaction exemplars, which are in turn weighted higher than non-displayed exemplars) to generate weighted exemplars (e.g., the weighted exemplars 230 of FIG. 2).

[0098] In some embodiments, the training component 760 can be used to train machine learning models to provide content scoring, as discussed above. For example, the training component 760 may correspond to the training component 240 of FIG. 2. In some embodiments, the training component 760 can train machine learning models (such as the machine learning model 250 of FIG. 2) based on labeled and / or weighted exemplars (e.g., the weighted exemplars 230 of FIG. 2).

[0099] In some embodiments, the exposure component 765 can be used to evaluate orP2712WO1 (RSMD / 0130PC-157584)quantify the exposure that each content asset has received, as discussed above. For example, the exposure component 765 may correspond to the exposure component 310 of FIG. 3. In some embodiments, the exposure component 765 can compare the exposure (e.g., number of impressions) for each content asset against one or more thresholds to generate dynamic hyperparameters (e.g., the hyperparameters 315 of FIG. 3), such as content-specific weights and / or model exploration rates, as discussed above.

[0100] In some embodiments, the prediction component 770 can be used to generate content selection scores (e.g., the selection scores 335 of FIG. 3) using machine learning, as discussed above. For example, the prediction component 770 may correspond to the prediction component 325 of FIG. 3. In some embodiments, the prediction component 770 can process user data using the model to generate scores for each asset, where the processing is based at least in part on the selected hyperparameters, as discussed above.

[0101] In the illustrated example, the storage 715 includes exemplars 775 (which may correspond to the engagement data 115 of FIGS. 1-2, and / or the interaction exemplars 210, non-interaction exemplars 215, non-display ed exemplars 220, and / or weighted exemplars of FIG. 2). The storage 715 also includes content assets 780 (which may correspond to the content assets 305 of FIG. 3) and exposure criteria 785 (which may correspond to the threshold(s) or other criteria used to determine whether a given content asset has been sufficiently explored).

[0102] The storage 715 further includes or more machine learning models 790 (e.g., the machine learning model 250 of FIGS. 2-3). Although depicted as residing in storage 715, the depicted data may be stored in any suitable location, including memory 710.

[0103] Generally, the depicted components (and others not depicted) in memory 710 may evaluate and / or use the depicted data (and others not depicted) in storage 715 to provide therapy data-based detection of user interface swaps or changes and / or image-based identification / classification of user interfaces, as discussed above.Example Clauses

[0104] Clause 1 : A method, comprising: determining a first set of exposure scores for a plurality of content assets; selecting a first value for an exploration rate of a machine learning model based at least in part on determining that at least a first content asset of the plurality of content assets has a first exposure score that fails to satisfy one or more criteria; generating a first set of selection scores for the plurality of content assets using the machine learning model; selecting the first content asset for exposure based on at least one of (i) the first set of selection scores or (ii) the first value for the exploration rate of the machine learning model; and selectingP2712WO1 (RSMD / 0130PC-157584)a second value for the exploration rate of the machine learning model based at least in part on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, wherein the second value is lower than the first value.

[0105] Clause 2: A method according to Clause 1, wherein selecting the first content asset for exposure comprises determining to output the first content asset via a graphical user interface (GUI) of a user device.

[0106] Clause 3: A method according to Clause 2, further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was not displayed via the GUI of the user device, refraining from increasing the first exposure score.

[0107] Clause 4: A method according to Clause 2, further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was displayed via the GUI of the user device, increasing the first exposure score.

[0108] Clause 5: A method according to any of Clauses 2 or 4, further comprising: determining that the first content asset was displayed via the GUI of the user device; determining whether a user of the user device interacted with the first content asset displayed via the GUI; and modifying one or more parameters of the machine learning model based on whether the user interacted with the first content asset.

[0109] Clause 6: A method according to any of Clauses 2 or 3, further comprising: determining that the first content asset was not displayed via the GUI of the user device; and modifying one or more parameters of the machine learning model based on the determination that the first content asset was not displayed.

[0110] Clause 7: A method according to any of Clauses 1-6, further comprising: selecting a first weight for the first content asset based at least in part on determining that the first exposure score fails to satisfy the one or more criteria; selecting the first content asset based further on the first weight; and selecting a second weight for the first content asset further based at least in part on determining that the updated exposure score satisfies the one or more criteria, wherein the second weight is lower than the first weight.[OHl] Clause 8: A method according to Clause 7, further comprising selecting a third weight for the first content asset based at least in part on determining that a second updated exposure score for the first content asset satisfies a second set of one or more criteria, wherein the third weight is lower than the second weight.

[0112] Clause 9: A method according to any of Clauses 1-8, further comprising selecting a third value for the exploration rate of the machine learning model based at least in part on determining that a second updated exposure score for the first content asset satisfies a secondP2712WO1 (RSMD / 0130PC-157584)set of one or more criteria, wherein the third value is lower than the second value.

[0113] Clause 10: A method, comprising: accessing a set of patient impression exemplars, each indicating (i) a respective content asset, and (ii) whether a respective patient selected the respective content asset, ignored the respective content asset, or did not view the respective content asset; labeling the exemplars, comprising for each respective exemplar: in response determining that the respective patient selected the content asset, assigning a first label to the exemplar; in response determining that the respective patient ignored the content asset, assigning a second label to the exemplar; and in response determining that the respective patient did not view the content asset, assigning a third label to the exemplar; training a machine learning model to predict patient impressions based on the labeled set of exemplars, comprising: assigning a high weight to exemplars having the first label; assigning a medium weight to exemplars having the second label; and assigning a low weight to exemplars having the third label.

[0114] Clause 11 : A system, comprising: a memory comprising computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform a method in accordance with any one of Clauses 1-10.

[0115] Clause 12: A system, comprising means for performing a method in accordance with any one of Clauses 1-10.

[0116] Clause 13: A non-transitory computer-readable medium comprising computerexecutable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform a method in accordance with any one of Clauses 1-10.

[0117] Clause 14: A computer program product embodied on a computer-readable storage medium comprising code for performing a method in accordance with any one of Clauses 1- 10.Additional Considerations

[0118] One or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of claims below can be combined with one or more elements or aspects or steps, or any portion(s) thereof, from one or more of any of the other claims below or combinations thereof, to form one or more additional implementations and / or claims of the present disclosure.

[0119] While the present disclosure has been described with reference to one or more particular embodiments or implementations, those skilled in the art will recognize that many changes may be made thereto without departing from the spirit and scope of the presentP2712WO1 (RSMD / 0130PC-157584)disclosure. Each of these implementations and obvious variations thereof is contemplated as falling within the spirit and scope of the present disclosure. It is also contemplated that additional implementations according to aspects of the present disclosure may combine any number of features from any of the implementations described herein.

[0120] The preceding description is provided to enable any person skilled in the art to practice the various embodiments described herein. The examples discussed herein are not limiting of the scope, applicability, or embodiments set forth in the claims. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. For example, changes may be made in the function and arrangement of elements discussed without departing from the scope of the disclosure. Various examples may omit, substitute, or add various procedures or components as appropriate. For instance, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some examples may be combined in some other examples. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects set forth herein. In addition, the scope of the disclosure is intended to cover such an apparatus or method that is practiced using other structure, functionality, or structure and functionality in addition to, or other than, the various aspects of the disclosure set forth herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0121] As used herein, the word “exemplary” means “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.

[0122] As used herein, a phrase referring to “at least one of’ a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a c c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).

[0123] As used herein, the term “determining” encompasses a wide variety of actions. For example, “determining” may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Also, “determining” may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, “determining” may include resolving, selecting, choosing, establishing and the like.P2712WO1 (RSMD / 0130PC-157584)

[0124] The methods disclosed herein comprise one or more steps or actions for achieving the methods. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Further, the various operations of methods described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and / or software component s) and / or module(s), including, but not limited to a circuit, an application specific integrated circuit (ASIC), or processor. Generally, where there are operations illustrated in figures, those operations may have corresponding counterpart means-plus-function components with similar numbering.

[0125] Embodiments of the invention may be provided to end users through a cloud computing infrastructure. Cloud computing generally refers to the provision of scalable computing resources as a service over a network. More formally, cloud computing may be defined as a computing capability that provides an abstraction between the computing resource and its underlying technical architecture (e.g., servers, storage, networks), enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. Thus, cloud computing allows a user to access virtual computing resources (e.g., storage, data, applications, and even complete virtualized computing systems) in “the cloud,” without regard for the underlying physical systems (or locations of those systems) used to provide the computing resources.

[0126] Typically, cloud computing resources are provided to a user on a pay-per-use basis, where users are charged only for the computing resources actually used (e.g., an amount of storage space consumed by a user or a number of virtualized systems instantiated by the user). A user can access any of the resources that reside in the cloud at any time, and from anywhere across the Internet. In context of the present invention, a user may access applications or systems (e.g., the engagement prediction system) or related data available in the cloud. For example, the engagement prediction system could execute on a computing system in the cloud and train and use machine learning models to predict user engagement with respect to media content and / or respiratory therapy, as discussed above. In such a case, the engagement prediction system could receive and process engagement data, and store the models and predictions at a storage location in the cloud. Doing so allows a user to access this information from any computing system attached to a network connected to the cloud (e.g., the Internet).P2712WO1 (RSMD / 0130PC-157584)

[0127] The following claims are not intended to be limited to the embodiments shown herein, but are to be accorded the full scope consistent with the language of the claims. Within a claim, reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No claim element is to be construed under the provisions of 35 U.S.C. §112(f) unless the element is expressly recited using the phrase “means for” or, in the case of a method claim, the element is recited using the phrase “step for.” All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public regardless of whether such disclosure is explicitly recited in the claims.P2712WO1 (RSMD / 0130PC-157584)

Claims

CLAIMSWHAT IS CLAIMED IS:

1. A method, comprising; determining a first set of exposure scores for a plurality of content assets; selecting a first value for an exploration rate of a machine learning model based at least in part on determining that at least a first content asset of the plurality of content assets has a first exposure score that fails to satisfy one or more criteria; generating a first set of selection scores for the plurality of content assets using the machine learning model; selecting the first content asset for exposure based on at least one of (i) the first set of selection scores or (ii) the first value for the exploration rate of the machine learning model; and selecting a second value for the exploration rate of the machine learning model based at least in part on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, wherein the second value is lower than the first value.

2. The method of claim 1, wherein selecting the first content asset for exposure comprises determining to output the first content asset via a graphical user interface (GUI) of a user device.

3. The method of claim 2, further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was not displayed via the GUI of the user device, refraining from increasing the first exposure score.

4. The method of claim 2, further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was displayed via the GUI of the user device, increasing the first exposure score.

5. The method of claim 2, further comprising: determining that the first content asset was displayed via the GUI of the user device; determining whether a user of the user device interacted with the first content asset displayed via the GUI; andP2712WO1 (RSMD / 0130PC-157584)modifying one or more parameters of the machine learning model based on whether the user interacted with the first content asset.

6. The method of claim 2, further comprising: determining that the first content asset was not displayed via the GUI of the user device; and modifying one or more parameters of the machine learning model based on the determination that the first content asset was not displayed.

7. The method of claim 1, further comprising: selecting a first weight for the first content asset based at least in part on determining that the first exposure score fails to satisfy the one or more criteria; selecting the first content asset based further on the first weight; and selecting a second weight for the first content asset further based at least in part on determining that the updated exposure score satisfies the one or more criteria, wherein the second weight is lower than the first weight.

8. The method of claim 7, further comprising selecting a third weight for the first content asset based at least in part on determining that a second updated exposure score for the first content asset satisfies a second set of one or more criteria, wherein the third weight is lower than the second weight.

9. The method of claim 1, further comprising selecting a third value for the exploration rate of the machine learning model based at least in part on determining that a second updated exposure score for the first content asset satisfies a second set of one or more criteria, wherein the third value is lower than the second value.

10. One or more non-transitory computer-readable media collectively or individually comprising computer-executable instructions that, when executed by one or more processors of one or more processing systems, cause the one or more processing systems to collectively or individually perform an operation comprising: determining a first set of exposure scores for a plurality of content assets;P2712WO1 (RSMD / 0130PC-157584)selecting a first value for an exploration rate of a machine learning model based at least in part on determining that at least a first content asset of the plurality of content assets has a first exposure score that fails to satisfy one or more criteria; generating a first set of selection scores for the plurality of content assets using the machine learning model; selecting the first content asset for exposure based on at least one of (i) the first set of selection scores or (ii) the first value for the exploration rate of the machine learning model; and selecting a second value for the exploration rate of the machine learning model based at least in part on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, wherein the second value is lower than the first value.

11. The one or more non-transitory computer-readable media of claim 10, wherein selecting the first content asset for exposure comprises determining to output the first content asset via a graphical user interface (GUI) of a user device.

12. The one or more non-transitory computer-readable media of claim 11, the operation further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was not displayed via the GUI of the user device, refraining from increasing the first exposure score.

13. The one or more non-transitory computer-readable media of claim 11, the operation further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was displayed via the GUI of the user device, increasing the first exposure score.

14. The one or more non-transitory computer-readable media of claim 11, the operation further comprising: determining that the first content asset was not displayed via the GUI of the user device; and modifying one or more parameters of the machine learning model based on the determination that the first content asset was not displayed.P2712WO1 (RSMD / 0130PC-157584)15. The one or more non-transitory computer-readable media of claim 10, the operation further comprising: selecting a first weight for the first content asset based at least in part on determining that the first exposure score fails to satisfy the one or more criteria; selecting the first content asset based further on the first weight; and selecting a second weight for the first content asset further based at least in part on determining that the updated exposure score satisfies the one or more criteria, wherein the second weight is lower than the first weight.

16. A system, comprising: one or more memories collectively or individually comprising computer-executable instructions; and one or more processors configured to, individually or collectively, execute the computer-executable instructions and cause the system to perform an operation comprising: determining a first set of exposure scores for a plurality of content assets; selecting a first value for an exploration rate of a machine learning model based at least in part on determining that at least a first content asset of the plurality of content assets has a first exposure score that fails to satisfy one or more criteria; generating a first set of selection scores for the plurality of content assets using the machine learning model; selecting the first content asset for exposure based on at least one of (i) the first set of selection scores or (ii) the first value for the exploration rate of the machine learning model; and selecting a second value for the exploration rate of the machine learning model based at least in part on determining that the first content asset has an updated exposure score that satisfies the one or more criteria, wherein the second value is lower than the first value.

17. The system of claim 16, wherein selecting the first content asset for exposure comprises determining to output the first content asset via a graphical user interface (GUI) of a user device, the operation further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was not displayed via the GUI of the user device, refraining from increasing the first exposure score.P2712WO1 (RSMD / 0130PC-157584)18. The system of claim 16, wherein selecting the first content asset for exposure comprises determining to output the first content asset via a graphical user interface (GUI) of a user device, the operation further comprising, in response to determining, after selecting the first content asset for exposure, that the first content asset was displayed via the GUI of the user device, increasing the first exposure score.

19. The system of claim 16, wherein selecting the first content asset for exposure comprises determining to output the first content asset via a graphical user interface (GUI) of a user device, the operation further comprising: determining that the first content asset was not displayed via the GUI of the user device; and modifying one or more parameters of the machine learning model based on the determination that the first content asset was not displayed.

20. The system of claim 16, the operation further comprising: selecting a first weight for the first content asset based at least in part on determining that the first exposure score fails to satisfy the one or more criteria; selecting the first content asset based further on the first weight; and selecting a second weight for the first content asset further based at least in part on determining that the updated exposure score satisfies the one or more criteria, wherein the second weight is lower than the first weight.P2712WO1 (RSMD / 0130PC-157584)

Images