System and method for generating output data using generative models

Abstract

Mechanisms for generating output data using a generative model. The generative model includes a plurality of score-based neural networks, each configured to generate a processable initial score using a sampling technique to produce an instance of the example data. The initial scores are combined to define a combined score. The combined score is processed using a sampling technique to generate the output data. Each score-based neural network is trained using a different training data set.

Description

Technical Field

[0001] This invention relates to the field of score-based generative models. Background Technology

[0002] There is a growing interest in developing and using generative models to generate output data from some input data. Generative models have a wide variety of uses, such as those designed to perform one or more medical or clinical tasks (e.g., sleep analysis, image reconstruction, image segmentation, etc.).

[0003] In score-based generative models, input data is typically processed to produce scores, sometimes denoted as score functions, which define one or more features of the probability distribution of the output data. Appropriate sampling techniques are then used to process the scores to generate or predict the output data. Specifically, sampling techniques are used to sample the probability distribution defined by the scores to generate some output data. The input and output data can, for example, be multidimensional.

[0004] The current goal is to improve the flexibility and accuracy of generative models. Summary of the Invention

[0005] This invention is defined by the claims.

[0006] According to an example of one aspect of the invention, a processing system is provided, the processing system being configured to: receive a plurality of initial scores, wherein each initial score is the output of a different score-based neural network based on input data associated with a subject, wherein the plurality of initial scores are used for a generative model, wherein each initial score defines a probability distribution for sampling output data, wherein each score-based neural network has been independently trained using a different training dataset; generate a combined score using the generative model by performing a vector derivative with respect to the output data on the plurality of initial scores; and process the combined score using a sampling technique to produce output data of the generative model, wherein the output data represents the subject's medical condition.

[0007] This disclosure proposes a method for generating a combined score from multiple initial scores. In the context of this disclosure, the term "score" has a meaning used in the field of generative models, such as including information representing the distribution of possible values ​​of the output. Each initial score is generated by a different score-based neural network, for example, which processes a corresponding input dataset to produce the initial score. Each input dataset belongs to or relates to the same entity, such as the same subject or individual. The input datasets processed by each score-based neural network can be the same input dataset, or preferably different input datasets. For example, each input dataset can include data from different sets or groupings of features for the entity (e.g., subject or individual).

[0008] By combining scores from independent score-based neural networks, each trained on different input types, flexibility in the types of inputs used is achieved. The inventors have recognized that this provides outputs of at least comparable quality to generative models using score-based neural networks trained simultaneously on all input types.

[0009] This allows adding or removing input types from a generative model without retraining the entire generative model. Instead, adding an input type requires only an additional score-based neural network trained independently on the new input type (i.e., one), while removing an input type only requires not using the score-based neural network trained for the removed input type.

[0010] Each initial score can be a vector differential operator.

[0011] In one embodiment, the input data includes two or more sets of physiological signals known to be associated with a subject's sleep stage or sleep-disordered breathing.

[0012] In one embodiment, the physiological signal includes one or more of the following: electroencephalogram (EEG) signal, electrooculogram (EOG) signal, electromyogram (EMG) signal, electrocardiogram (ECG) signal, cardiac impactogram (CPA) signal, cardiac oscillation (CAOS) signal, pulse oximetry signal, and / or respiratory signal.

[0013] In one embodiment, the processing system is configured to use a generative model to process the input data using different score-based neural networks to generate multiple initial scores.

[0014] In some examples, each training dataset includes training data for a corresponding data type among a plurality of different data types; and the processing system is further configured to: receive a plurality of input datasets, each input dataset including input data for a corresponding data type among the plurality of different data types; and to provide each input dataset to a score-based neural network trained using the same type of training dataset to generate a plurality of initial scores.

[0015] Generative models can be clinical assessment tools. In other words, generative models can be configured to generate output data to aid clinical decision-making, such as helping to understand, analyze, and / or assess the condition of a medical subject (e.g., a patient). Examples of clinical assessment tools include tools designed to assist in performing any appropriate medical task. Example clinical assessment tools include models for determining sleep patterns by processing physiological signals; predicting the presence of one or more pathologies (e.g., heart defects, breathing problems, etc.) by processing physiological signals; identifying one or more target anatomical components (e.g., tumors, growths, cancer, etc.) in one or more medical images; segmenting anatomical components in one or more medical images; denoising medical images; predicting and generating medical image data from a portion of medical image data; and so on. This list of examples is not exhaustive, and those skilled in the art can readily identify additional functions and / or tasks that will be performed by generative models serving as clinical assessment tools.

[0016] In one embodiment, the generative model is a clinical assessment tool, wherein the input data includes physiological signals of the subjects, and the clinical assessment tool includes a model for predicting the presence of one or more pathologies by processing the physiological signals.

[0017] The output data of a generative model can be a time series, that is, a sequence or vector of data values ​​representing different values ​​of the output data over time.

[0018] The training dataset includes physiological signals from one or more subjects during one or more sleep periods. In other words, a generative model can be trained to analyze the sleep of subjects.

[0019] The physiological signals may include one or more of the following: electroencephalogram (EEG) signals, electrooculogram (EOG) signals, electromyogram (EMG) signals, electrocardiogram (ECG) signals, cardiac impaction signal, cardiac oscillation signal, pulse oximetry signal, photoplethysmography (PPG) signal, snoring microphone audio signal, environmental microphone audio signal, body movement signal, respiratory movement signal, respiratory effort signal, and / or respiratory airflow signal.

[0020] In some examples, the time series is a sleep graph or sleep density graph (e.g., the probability of sleep stages over time). A sleep graph is a time series of estimated sleep stages of a subject during sleep. In other examples, the time series represents the probability of sleep events over time, such as the probability of sleep-disordered breathing (SDB) events over time. In some examples, the time series may represent the probability of an SDB event (e.g., obstructive or central apnea, or insufficiency) occurring at that time for each sample. Other examples of sleep events include cortical or spontaneous arousal, periodic limb movements, or any other cortical or spontaneous event that occurs during sleep and is sleep-related (normal or disordered).

[0021] In some approaches, each training dataset includes different types of physiological signals. By combining initial scores from score-based neural networks trained independently on different types of physiological signals, generative models can be readily adapted to different combinations of physiological signals. This allows the use of generative models regardless of the specific physiological signals available in a given situation.

[0022] In one embodiment, the input data represents signals in response to the subject's sleep stages during a sleep period.

[0023] In one embodiment, the output data of the generative model is a time series, where the time series is a sleep map or a sleep density map.

[0024] In one embodiment, the input data represents the subject's signaling response to breathing difficulties during sleep.

[0025] In one embodiment, the output data of the generative model is a time series, where the time series represents the probability of sleep events over time.

[0026] Each training dataset can include data from different groups of subjects. In other words, the processing system can use joint learning techniques.

[0027] Different subject groups can be grouped based on the clinical settings associated with each subject. This allows generative models to utilize information from a large number of clinics without requiring clinics to share confidential patient data. Furthermore, clinics can be added to or removed from generative models without retraining the entire model.

[0028] Different subject groups can be grouped according to one or more demographic criteria. For example, subjects can be grouped according to one or more of the following: age, comorbidities, severity of sleep-disordered breathing, etc. Generative models can then be customized for the subjects being analyzed using only initial scores from a score-based neural network trained for the demographic group to which the subjects belong.

[0029] In some examples, each training dataset includes data from different subjects. This allows individual subjects to choose whether to enter or exit the generative model without retraining the entire generative model.

[0030] The processing system can also be configured, for each initial score, to: generate multiple samples by iteratively sampling the initial score; and process the multiple samples to generate a measure of uncertainty for the initial score. Uncertainty can be any statistically discrete measure, such as variance, standard deviation, interquartile range, etc. The processing system can be configured to process the initial scores to generate a combined score by performing a process that includes combining only those initial scores whose measure of uncertainty satisfies one or more predetermined conditions.

[0031] In some examples, the processing system can be configured to produce a combined measure of uncertainty by combining measures of uncertainty generated for each initial score. The combined measure of uncertainty can be output (e.g., displayed) along with the output data of the generative model to indicate a measure of general (stochastic) uncertainty of the entire generative model.

[0032] In alternative methods, a combined measure of uncertainty is generated using combined scores. Specifically, the method may include generating a plurality of second samples by iteratively sampling the combined scores; and processing the plurality of second samples to generate a combined measure of uncertainty.

[0033] A computer-implemented method is also proposed, comprising: receiving a plurality of initial scores, each initial score being the output of a different score-based neural network based on input data associated with a subject, wherein the plurality of initial scores are used for a generative model, wherein each initial score defines a probability distribution for sampling output data, and wherein each score-based neural network has been independently trained for a different training dataset; generating a combined score using the generative model by performing a vector derivative with respect to the output data on the plurality of initial scores; and processing the combined score using a sampling technique to produce output data of the generative model, wherein the output data represents the subject's medical condition.

[0034] The computer-implemented methods can be configured to perform the functions of any processing system disclosed herein, and vice versa.

[0035] A computer program product comprising computer program code components is also proposed, which, when executed on a computing device having a processing system, cause the processing system to perform all the steps of any method disclosed herein.

[0036] A respiratory support system for providing airflow to a subject is also proposed. The respiratory support system includes a processing system, at least two sensors, and an airflow control system. The processing system is as described above and is configured to generate (as output data) an output signal in response to a predicted occurrence of respiratory distress in the subject. Each sensor is adapted to generate a different physiological signal from the subject. Each score-based neural network used by the processing system is configured to generate an initial score by processing the corresponding different physiological signal. Thus, each score-based neural network is trained using a corresponding instance of training data with the same type of data as the corresponding physiological signal in the physiological signals. The processing system is configured to process the initial score to generate a combined score; the combined score is processed using sampling techniques to produce output data for a generative model. The airflow control system is configured to control the characteristics of the airflow to the subject based on the output data of the generative model, such as based on a predicted occurrence of any respiratory distress in the subject.

[0037] This method provides a mechanism for actively managing airflow to a subject using directly monitored physiological signals. Specifically, each physiological signal is processed independently using a separate or dedicated score-based neural network to generate a corresponding initial score. The initial scores are then combined to predict or generate an output signal in response to a predicted occurrence of dyspnea. This output signal is then used to control airflow to the subject. Occasionally, the subject removes the sensor while using a respiratory support system, such as when moving unintentionally during sleep or when intentionally experiencing discomfort while trying to fall asleep. The processing system is able to generate a combined score based on the available initial scores without requiring specialized training for each specific combination of sensors. In another scenario, the subject replaces a sensor with another. For example, the subject has been using a mask with sensors for some time. Due to the mask's wear, the subject replaces the mask with one from a different brand or model. The processing device is able to generate a new initial score based on physiological signals from the sensors in the new mask.

[0038] A sleep stage determination system for determining the sleep stages of a subject is also proposed. The sleep stage determination system includes a processing system and at least two sensors. Each sensor is adapted to generate different physiological signals from the subject. The processing system is implemented as described above, wherein each score-based neural network is configured to generate an initial score by processing different physiological signals from the physiological signals generated by the at least two sensors. Thus, each score-based neural network is trained using a corresponding instance of training data that is the same type of data as the corresponding physiological signal in the physiological signals. The processing system is configured to process the initial scores to generate a combined score; and to process the combined score using a sampling technique to produce output data for a generative model. The output data represents one or more sleep stages of the subject.

[0039] These and other aspects of the invention will become apparent from the embodiments described below. Attached Figure Description

[0040] To better understand the invention and to more clearly illustrate how to implement it, reference will now be made to the accompanying drawings by way of example only, wherein:

[0041] Figure 1 The workflow of the proposed method is shown;

[0042] Figure 2 This is a flowchart illustrating the proposed method;

[0043] Figure 3 This is a flowchart illustrating variations of the proposed method; and

[0044] Figure 4 This is a flowchart illustrating yet another proposed method. Detailed Implementation

[0045] The invention will be described with reference to the accompanying drawings.

[0046] It should be understood that the detailed description and specific examples, while indicating exemplary embodiments of the apparatuses, systems, and methods, are for illustrative purposes only and are not intended to limit the scope of the invention. The described and other features, aspects, and advantages of the apparatuses, systems, and methods of the present invention will become more readily apparent from the following description, the appended claims, and the accompanying drawings. It should be understood that the drawings are merely schematic and not drawn to scale. It should also be understood that the same reference numerals are used throughout the drawings to denote the same or similar parts.

[0047] This invention provides a mechanism for generating output data using a generative model. The generative model includes multiple score-based neural networks, each configured to generate a processable initial score using a sampling technique to produce instances of example data. The initial scores are combined to define a combined score. The combined score is processed using the sampling technique to produce the output data. Each score-based neural network is trained using a different training dataset.

[0048] This disclosure provides the concept of using multiple score-based neural networks to generate individual scores, which are then combined to define a score from which output data is generated.

[0049] In some examples, different score-based neural networks can be trained to process input data from different sets of input (data) types. In practice, this means that if a certain input (data) type becomes unavailable, the score-based neural network for that type can be deactivated, while the remaining score-based neural networks generate initial scores for combination. This increases the flexibility of using the proposed generative model.

[0050] In the context of this disclosure, input (data) type refers to input data for a specific variable. Therefore, instances of input data of different input (data) types are instances of input data for different variables. The terms input (data) type (i.e., input type or input data type) and variable are thus considered semantically interchangeable. Therefore, different score-based neural networks can be trained to process input data from different sets or combinations of one or more input variables.

[0051] In some examples, different score-based neural networks can be trained using training data from different groups or cohorts of one or more subjects. This effectively introduces joint learning components into the generative model, improving its accuracy by reducing specificity to a particular group or cohort of subjects (e.g., reducing unconscious bias in the generative model).

[0052] This disclosure thus effectively defines or provides a new architecture for generative models.

[0053] Typically, score-based generative models receive some input data, process it, and produce predictions of some output data. A characteristic of score-based generative models (also known as diffusion models) is the use of scores derived from the input data, which define or characterize a probability distribution. Sampling techniques are used to process the scores (specifically the probability distribution) to generate or predict one or more values ​​for the output data. In other words, the input data is used to define the possible values ​​of the output data (e.g., in the form of scores from a probability distribution). The actual output data is produced by sampling the possible values ​​using sampling techniques to guess or estimate the output data.

[0054] This invention proposes the concept of generating multiple initial scores using different score-based neural networks (i.e., separate or individual diffusion models). Each score-based neural network is trained using a different training dataset. For example, different training datasets can be represented as training data generated from input data of different populations and / or different types / variables. The multiple initial scores are then combined or fused to produce a combined score, which is used to define or derive the output data.

[0055] The following defines some notations to aid in understanding the context of the proposed method. However, it should be noted that the general principles of score-based neural networks used to generate scores for use in generative models (particularly diffusion models) are well known in the art, for example, in Karras, Tero et al., “Elucidating the design space of diffusion-based generative models,” Neural Information Processing Systems 35 (2022): 26565-26577; and Song, Yang et al., “Score-based generative modeling by stochastic differential equations,” ArXiv preprint arXiv: 2011.13456 (2020).

[0056] symbol x i This represents the input signal or channel with index i, i.e., an instance of input data. The total input data for a generative model can be formed by multiple input signals, such as up to N input signals. For example, each input signal can represent a different variable, i.e., different types or categories of input data. Alternatively, different input signals can represent different subsets of the input data. In some examples, one or more input signals are identical.

[0057] The symbol y represents output data, such as an output signal.

[0058] As a working example, in the context of sleep stage prediction, the input signal x i This can represent any signal that responds to a subject's sleep stage during a sleep period, such as an EEG signal; an electrocardiogram signal; a PPG signal; an audio signal (e.g., measured by a microphone); and / or a motion signal (e.g., in response to a subject's respiratory movements). In this working example, the output data y can be a sleep graph describing the subject's predicted sleep stage during the sleep period.

[0059] As another example of work, in the context of sleep apnea prediction, the input signal x i This can represent any signal that responds to a subject's respiratory disturbance during sleep, such as an EEG signal; an electrocardiogram signal; a PPG signal; an audio signal (e.g., measured by a microphone); and / or a motion signal (e.g., in response to the subject's respiratory movements). In this working example, the output data y could be an output signal describing the probability of an SDB event occurring over time.

[0060] As mentioned earlier, the score representation or definition derives the probability distribution of the output data, for example, by sampling from a probability distribution. In the working example, ('del' or 'nabla') indicates that the function is relative to... The vector derivative. and This can represent different types of scores, in this case, the previous score (e.g., baseline score) and the initial score. That is, the score is relative to... The logarithm of the probability distribution The vector derivative.

[0061] One purpose of the proposed public content is to calculate the combined score. Output data can be generated from this combined score. A combined score is generated by combining two or more initial scores and optionally / preferred previous scores. This combined score... It effectively considers multiple input signals or subsets of input data. This efficiently generates scores given all available or existing measurement modes.

[0062] The combined score can be effectively a vector differential operator that defines or characterizes a probability distribution from which the predicted output can be derived.

[0063] In reality, it is not functionally possible to accurately capture or express previous scores. and / or initial score Instead, this disclosure proposes to learn / approximate the score by applying a neural network. The neural network can consist of learnable parameters. Any parameterized function with learnable parameters Score and (used to generate initial scores) As input and output. In the equation, this can be expressed as: (1) (2)

[0064] in This represents a neural network. Each initial score is generated by a different trained neural network. Specifically, each neural network is trained using a different training dataset. Therefore, different training datasets are used to learn the parameters θ of different neural networks. In some examples, parameters can be learned for each signal source (e.g., index j can be equal to index i). Furthermore, previous scores can be considered as a special case of the initial scores, where... and .

[0065] Several known types of neural networks exist capable of learning how to approximate scores, often referred to as score matching techniques. Known examples include slice score matching and denoised score matching. The invention presented herein is agnostic regarding the score matching technique used. Therefore, the type of score matching technique employed is not important to the proposed embodiments.

[0066] As mentioned earlier, the goal of the proposed method is to generate a combined score by combining initial scores. An exemplary technique for estimating a combined score from an initial score (and optionally previous scores) is defined by the following equation:

[0067] (3)

[0068] The score on the right-hand side of the equation can be estimated using neural networks as shown in (1) and (2). In other words, the combined score is generated or inferred only during deployment and does not require training of the combined signal model.

[0069] It is worth noting that, due to the vanishing of the normalization constant, the gradient relative to the source is used. Therefore, using score-based neural networks instead of more classic deep learning methods (such as cross-entropy) is a fundamental requirement.

[0070] Figure 1 An example workflow 100 of the proposed method is conceptually illustrated.

[0071] In particular, Figure 1 This demonstrates how initial scores are combined to generate a combined score. Specifically, multiple corresponding neural networks N1, N2, ..., NN are used to process multiple input signals I1, I2, ..., IN or input data portions to generate multiple corresponding initial scores IS1, IS2, ..., ISN.

[0072] Each neural network N1, N2, ... N3 can also process the output data y to generate a corresponding initial score. Thus, each neural network can operate efficiently using, for example, the principles of denoised score matching explained or proposed by: Vincent, Pascal, “Connection between score matching and denoised autoencoders,” Neural Computation 23.7 (2011): 1661-1674; Karras, Tero et al., “Elucidating the design space of diffusion-based generative models,” Neural Information Processing Systems 35 (2022): 26565-26577, etc.

[0073] Multiple initial scores are combined (through the combination process CP) to produce a combined score CS.

[0074] The combined score CS is then processed using the sampling technique ST to produce the output data y. In a specific example, the combined score can define the probability distribution (or multiple probability distributions) of the output data y. The probability distribution is appropriately sampled using the sampling technique, and then the output data is produced.

[0075] Many possible sampling techniques can perform this procedure, including: Langevin dynamics, denoised diffusion probabilistic model (DDPM), denoised diffusion implicit model (DDIM), and ordinary differential equation (ODE) solvers such as the Euler method or higher-order Runge-Kutta method. The proposed method is agnostic to the exact sampling used. Therefore, the type of sampling technique employed is not critical to the proposed embodiment.

[0076] In some instances, a prior score PS is also generated by processing the previous prediction of the output data y using a properly trained neural network NPS. The generation of the combined score CS can also use the prior score (as well as the initial score) to produce the combined score, for example by employing the method described by equation (3).

[0077] Thus, the generation of output data can be an iterative method that uses input data I1, I2, ..., IN to iteratively update the predicted output y. Therefore, the proposed method can effectively address the question: "Given the input, what is a better estimate of the output based on the current version of the output?".

[0078] We have already discussed how to use score-based neural networks to process input data to generate initial scores. In particular, different score-based neural networks produce different initial scores, which are then combined when generating the combined score.

[0079] Artificial neural networks (or simply neural networks) are inspired by the human brain in their structure. A neural network consists of multiple layers, each containing multiple neurons. Each neuron performs a mathematical operation. Specifically, each neuron can include different weighted combinations of a single type of transformation (e.g., the same type of transformation, a sigmoid transform, etc., but with different weights). In processing the input data, the mathematical operation of each neuron is performed on the input data to produce a digital output, and the outputs of each layer in the neural network are sequentially fed to the next layer. The final layer provides the output.

[0080] Methods for training neural networks to perform score generation functions are well known. As a general overview, such a neural network can be trained by adding noise to the training data and training the neural network to denoise the training data, thereby defining a (trained) score-based neural network.

[0081] Example methods for appropriately training score-based neural networks are described below: Song, Yang et al., “Score-based generative modeling by stochastic differential equations,” arXiv preprint arXiv: 2011.13456 (2020); Karras, Tero et al., “Elucidating the design space of diffusion-based generative models,” Neural Information Processing Systems 35 (2022): 26565-26577; Ho, Jonathan, Ajay Jain, and Peter Abbeel, “Denoising diffusion probabilistic models,” Neural Information Processing Systems 33 (2020): 6840-6851; and Pang, Tianyu et al., “Efficiently learning generative models by finite difference score matching,” Neural Information Processing Systems 33 (2020): 19175-19188. This list of exemplary methods is non-exhaustive, and those skilled in the art will readily understand various other methods and / or techniques for appropriately training neural networks to perform score generation functions.

[0082] As mentioned earlier, different score-based neural networks are trained using different training datasets. Therefore, the training data used to train the nth score-based neural network may include the nth training dataset.

[0083] Those skilled in the art will understand that there is a direct connection between the training data used to train a particular score-based neural network and the input data processed by the neural network during subsequent inference (to generate an initial score). Specifically, the input data processed during inference includes data of the same type as the example input data in the training data used to train the score-based neural network (e.g., representing the same or comparable (e.g., alternative versions of attributes) or variables).

[0084] Figure 2 This is a flowchart illustrating the proposed method 200. Method 200 is a computer-implemented method that can be executed by a processing system, and suitable embodiments thereof will be described later. Method 200 describes an inference process utilizing a generative model having the architecture proposed herein, for example, using multiple different score-based neural networks.

[0085] The method includes a step 210 of receiving multiple initial scores, wherein each initial score is the output of a different score-based neural network of a generative model and defining a probability distribution for sampling the output data, wherein each score-based neural network has been independently trained for a different training dataset.

[0086] Step 210 may include a sub-step 211, for example, receiving multiple instances of input data. For example, each instance of input data may represent an input signal generated by a different sensor, or an input signal defined in a different field or entry of a data record for the subject.

[0087] Then, in substep 212, two or more sets of one or more instances of the input data are processed by the respective score-based neural networks to generate an initial score. Thus, each score-based neural network processes one or more instances of the input data to produce an initial score. Different score-based neural networks process different sets of one or more instances of the input data. As previously described, each score-based neural network has been independently trained against and / or using different training datasets, e.g., it has been designed to process a specific set (e.g., a combination) of one or more instances of the input data and / or training datasets. Suitable examples of instances of the input data and / or training datasets will be provided later in this disclosure.

[0088] Method 200 also includes a step 220 of processing the initial scores to generate a combined score. Appropriate methods for performing step 220 have already been described above.

[0089] Method 200 further includes a step 230 of processing the combined scores using sampling techniques to produce output data for a generative model. Specifically, step 230 may include a sub-step 231 of defining a probability distribution using the combined scores. Step 230 may then perform a sub-step 232 of processing the probability distribution using sampling techniques to produce output data. Examples of suitable sampling techniques have been described above and will be apparent to those skilled in the art.

[0090] We have previously described how to train different score-based neural networks using different training datasets.

[0091] In the first working example, each score-based neural network is trained to process input data of one or more input (data) types (representing one or more different variables) from different groups to produce corresponding initial scores. Therefore, during the training process, one score-based neural network will be trained using a training dataset (including example input data for the first group of input (data) types), and another score-based neural network will be trained using a different training dataset (including example input data for a different second group of input (data) types).

[0092] The input (data) type defines the expected source of the corresponding instance of the input data. Thus, for example, first type of input data is generated or otherwise provided by a first source (e.g., a first sensor), and second type of input data is generated or otherwise provided by a different second source (e.g., a second sensor).

[0093] In this first working example, during inference using trained score-based neural networks, each score-based neural network processes input data from a different set of input sources to generate an initial score. This produces different initial scores for different sets or combinations of input sources, which are then combined to produce a combined score.

[0094] In the second working example, each score-based neural network is trained to process the same set of one or more input (data) types to produce a corresponding score. However, during training, different training datasets are used to train different neural networks.

[0095] As an example, different training datasets can represent data from different subject groups (e.g., different demographics, locations, etc.). Therefore, each training dataset can include data from different subject groups. As a working example, each individual training dataset can belong to a specific demographic of the subjects, such as age group, presence of comorbidities, or severity of sleep-disordered breathing.

[0096] As another example, different training datasets can represent data from different individual subjects or subjects. Therefore, each training dataset can include data from different subjects. Thus, score-based neural networks built for different subjects / individuals can be efficiently aggregated. Individual subjects can easily choose to enter and exit the score-based neural network they want to use at any time without needing to retrain the entire algorithm. Conversely, individual score-based neural networks can simply omit the combination process.

[0097] In this second working example, during inference using trained score-based neural networks, each score-based neural network processes input data from the same set of input sources to generate an initial score. Since each score-based neural network has been trained with different datasets, it is obvious that different initial scores will be produced.

[0098] This second working example effectively provides a federated learning technique to generate combined scores.

[0099] Of course, combinations of these two working examples can also be implemented. For example, each score-based neural network can be trained and / or processed using datasets for different subject populations, handling one or more different sets of input (data) types. Thus, in some examples, there can be at least three score-based neural networks, where at least two score-based neural networks can be trained using datasets for different subject populations, and at least two score-based neural networks can be trained to handle different sets of input (data) types. As another example, there can be multiple score-based neural networks, where one score-based neural network is trained using a first subject population and processes a first set of input (data) types, while another score-based neural network is trained using a second (different) subject population and a second (different) set of input (data) types.

[0100] Method 200 may also include step 240 of outputting or otherwise providing output data (generated in step 230). For example, step 240 may include controlling a user interface to provide a visual representation of the output data. In an example, step 240 may include storing the output data in memory or a storage system. Step 240 may also / or otherwise include processing the output data (e.g., processing it using an additional algorithm). For example, step 240 may be performed by a separate processing system. Step 240 may include any combination of performing one or more of the examples described above.

[0101] Figure 3 This is a flowchart illustrating method 300, which is previously referenced. Figure 2 A variation of the described method 200. In this variation, it is assumed that at least two score-based neural networks are configured to process different sets of one or more input (data) types.

[0102] Method 300 includes a modified version of step 210. Specifically, step 210 further includes step 310 of identifying one or more available types of input data used to determine output data. Step 210 also includes step 320 of identifying any score-based neural networks that require unavailable types of input data to generate an initial score. In response to a positive determination / identification in step 320, step 210 executes step 330, in which any identified score-based neural network is prevented from contributing to the combined score. Specifically, step 330 may include preventing any identified score-based neural network from generating an initial score as described in step 320.

[0103] Of course, step 220 may appropriately include preventing any initial scores generated by a score-based neural network that requires input data of unavailable types from contributing to the combined score.

[0104] This method can effectively adjust the generation of combined scores during runtime (e.g., per subject) based on the availability of input data used to generate initial scores. This advantageously provides a generative model architecture that adapts to the environment in which it is deployed.

[0105] In the context of this disclosure, if corresponding input data is not obtained or is obtained with insufficient quality, the type of input data can be considered unusable. Methods for determining the quality of input data are well known in the art, including methods for monitoring the signal-to-noise ratio of the input data. This advantageously avoids the need to repeatedly sample or record the input data, for example, if the generation of a particular type of input fails.

[0106] Figure 4 This is a flowchart illustrating method 400, which provides additional optional steps or processes for any of the previously described methods 200, 300 for generating output for a generative model.

[0107] Method 400 is configured to allow only initial scores that satisfy one or more predetermined quality conditions to contribute to the combined score. Specifically, method 400 is configured such that only those initial scores that can produce output data with relatively high determinism are allowed to contribute to the combined score.

[0108] Method 400 further includes performing step 410: generating multiple samples for each initial score by iteratively sampling the initial scores. Any suitable sampling technique can be used in step 410. Preferably, the same sampling technique used in step 230 is employed in step 410 to ensure that each initial score is identical when generating multiple samples.

[0109] Method 400 also includes a step 420 of processing multiple samples to generate a measure of uncertainty for the initial scores. In step 420, any known measure of uncertainty can be determined, examples of which include any statistical dispersion measure, such as variance, standard deviation, interquartile range, etc.

[0110] Step 220 may be adapted to include only those initial scores that combine uncertainty measures to satisfy one or more predetermined conditions. Therefore, refer to... Figure 1 Only those initial scores that meet one or more predetermined conditions can form part of the initial scores IS1, IS2, ..., ISN. The initial scores IS1, IS2, ..., ISN are processed to produce combined scores.

[0111] Examples of suitable predetermined conditions include measures of uncertainty below a predetermined value, such as the standard deviation below a predetermined value. Other example conditions will be apparent to those skilled in the art.

[0112] The proposed method can be used or applied in a wide variety of environments and / or situations. Many working embodiments are described below.

[0113] In one working example, the proposed method is used to classify sleep stages during a subject's sleep period.

[0114] The input data to be processed by the score-based neural network (to generate an initial score) may include two or more sets of physiological signals known to be related to or otherwise responsive to the subject's sleep stages. Suitable examples include neural signals such as EEG, EOG, EMG, but alternative sensor forms that characterize sleep from the perspective of autonomic nervous system activity may also be included, such as cardiac signals (e.g., ECG, PPG, BCG, SCG) or respiratory signals (e.g., those measured with chest and / or abdominal breathing belts, bed sensors, Doppler radar, chest-mounted accelerometers, etc.).

[0115] Each score-based neural network can be configured (i.e., appropriately trained) to process different sets of one or more physiological signals to generate corresponding initial scores. Thus, each set of physiological signals can be processed by a different score-based neural network to generate multiple initial scores. The initial scores are then combined using the previously described method to generate a combined score. The combined score is then processed using a sampling algorithm and the previously described method to predict a time series of data representing the predicted sleep stages of a subject through sleep periods.

[0116] In another working example, the proposed method is used to detect sleep apnea (SDB).

[0117] The input data to be processed by the score-based neural network (to generate an initial score) may include two or more sets of physiological signals known to be associated with or otherwise responsive to sleep apnea. Examples include the same physiological signals stated in the examples above for sleep stage prediction. Other examples include physiological signals generated by sensors that measure the direct effects of SDB events, such as airflow sensors (e.g., nasal pressure cannulas or oronasal thermistors) or oxygen saturation sensors (e.g., finger-mounted transmissive SpO2 or transmissive SpO2 mounted on other parts of the body).

[0118] As previously described, each score-based neural network can be configured (i.e., appropriately trained) to process a different set of one or more physiological signals to generate corresponding initial scores. Therefore, each set of physiological signals can be processed by a different score-based neural network to generate multiple initial scores. The initial scores are then combined using the previously described method to generate a combined score. The combined score is then processed using a sampling algorithm and the previously described method to predict a time series of data representing the predicted occurrence (or non-occurrence) of sleep apnea in subjects during sleep periods.

[0119] The output data of generative models can be used to control the operation of respiratory support devices for subjects, such as controlling the characteristics of the airflow supplied to the subjects. This facilitates direct control of respiratory support device operation based on the predicted occurrence of sleep apnea, thereby improving the control of mechanically assisted breathing for subjects.

[0120] Therefore, a respiratory support device for providing airflow to a subject is provided. The respiratory support device includes a processing system configured to perform the previously described method for generating output using a generative model. The respiratory support device also includes at least two sensors, each adapted to generate physiological signals of the subject. Each score-based neural network is trained to process different sets of physiological signals to generate a corresponding initial score. Thus, each score-based neural network is trained independently using a different training dataset, wherein the different training datasets include data of the same type or attributes as each set of physiological signals. The processing system is configured to generate an initial score for each set of physiological signals generated by the at least two sensors and to control the characteristics of the airflow based on the output data generated by the generative model. Examples of suitable properties include pressure, pressure distribution, flow rate, temperature, and / or humidity.

[0121] In another working example, the proposed method is used to segment anatomical elements, such as tumors or growths, in image data.

[0122] In this working example, the input data to be processed by the score-based neural network (to generate an initial score) may include two or more instances of image data from different imaging modalities, such as those representing the same region of interest. Instances include MRI images, CT images, PET images, ultrasound images, etc.

[0123] Each score-based neural network can be configured (i.e., appropriately trained) to process different image modalities to generate corresponding initial scores. Therefore, each instance of image data in the input data can be processed by a different score-based neural network to generate multiple initial scores. The initial scores are then combined using the previously described method to generate a combined score. The combined score is then processed using a sampling algorithm and the previously described method to predict the segmentation of at least one anatomical component in the instance of image data. Thus, the output data of the generative model is the segmentation of the anatomical components.

[0124] The proposed methods or techniques can be used to perform a variety of technical tasks. The proposed methods are particularly advantageous when used to process or analyze medical data for medical analysis tasks.

[0125] Those skilled in the art can readily develop processing systems for performing any of the methods described herein. Therefore, each step of the flowchart can represent a different action performed by the processing system and can be executed by the corresponding module of the processing system.

[0126] Therefore, embodiments can utilize processing systems. Processing systems can be implemented in various ways, using software and / or hardware, to perform a variety of desired functions. A processor is one example of a processing system employing one or more microprocessors, which can be programmed using software (e.g., microcode) to perform desired functions. However, processing systems can be implemented with or without a processor, and can also be implemented as a combination of dedicated hardware for performing certain functions and processors (e.g., one or more programmable microprocessors and associated circuitry) for performing other functions.

[0127] Examples of processing system components that may be used in various embodiments of this disclosure include, but are not limited to, conventional microprocessors, application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

[0128] In various implementations, a processor or processing system may be associated with one or more storage media, such as volatile and non-volatile computer memories, like RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors and / or processing systems, perform the required functions. The various storage media may be fixed within the processor or processing system, or may be transferable, allowing one or more programs stored thereon to be loaded into the processor or processing system.

[0129] It should be understood that the disclosed methods are preferably computer-implemented methods. Thus, the concept of a computer program is also proposed, which includes code components for implementing any of the methods when the program is run on a processing system such as a computer. Therefore, different portions, lines, or blocks of code of the computer program according to embodiments can be executed by a processing system or computer to perform any of the methods described herein.

[0130] A non-transitory storage medium for storing or carrying a computer program or computer code is also proposed, which, when executed by a processing system, causes the processing system to perform any of the methods described herein.

[0131] In some alternative implementations, the functions indicated in the block diagram or flowchart may not occur in the order shown in the diagram. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order.

[0132] By studying the accompanying drawings, the disclosure, and the appended claims, those skilled in the art can understand and implement variations of the disclosed embodiments in practicing the claimed invention. The fact that certain measures are recited in mutually different dependent claims does not imply that combinations of said measures cannot be used advantageously.

[0133] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. If the term "suitable" is used in the claims or description, it should be noted that the term "suitable" is intended to be equivalent to the term "configured as." If the term "arranged" is used in the claims or description, it should be noted that the term "arranged" is intended to be equivalent to the term "system," and vice versa.

[0134] A single processor or other unit can implement the functions of several of the claims. If a computer program has been discussed above, it can be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state medium provided with or as part of other hardware, but it can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0135] Any reference numerals in the claims should not be construed as limiting the scope.

Claims

1. A processing system configured to: Receive multiple initial scores (IS1-ISN). Each initial score (IS1-ISN) is the output of a different score-based neural network (N1-NN) based on the subject-related input data (I1-IN). The multiple initial scores (IS1-ISN) are used in the generative model. Each initial score (IS1-ISN) defines the probability distribution used to sample the output data (y). Each score-based neural network (N1-NN) has been trained independently using a different training dataset; The generative model is used to generate a combined score (CS) by performing a vector derivative with respect to the output data (y) on the plurality of initial scores (IS1-ISN); and The combined score (CS) is processed using a sampling technique (ST) to produce the output data (y) of the generative model. The output data (y) represents the subject's medical condition.

2. The processing system according to claim 1, wherein the input data includes: Two or more sets of physiological signals known to be associated with one or more sleep stages or sleep-disordered breathing in the subject.

3. The processing system according to claim 2, wherein the physiological signal includes one or more of the following: electroencephalogram (EEG) signal, electrooculogram (EOG) signal, electromyogram (EMG) signal, electrocardiogram (ECG) signal, cardiac impaction signal, cardiac oscillation signal, pulse oximetry signal, and / or respiratory signal.

4. The processing system according to any one of claims 1-3, configured as follows: The input data (I1-IN) is processed using the generative model and the different score-based neural networks (N1-NN) to generate the multiple initial scores (IS1-ISN).

5. The processing system according to claim 4, wherein: Each training dataset includes training data for the corresponding data type across multiple different data types; and The processing system is also configured to: Receive multiple input datasets, each input dataset including input data for a corresponding data type among the multiple different data types; as well as Each input dataset is fed into a score-based neural network trained using the same type of training dataset to generate the multiple initial scores.

6. The processing system according to any one of claims 1 to 5, wherein the generative model is a clinical assessment tool. The input data includes the subject's physiological signals. The clinical assessment tools mentioned therein include models for predicting the presence of one or more pathologies by processing the physiological signals.

7. The processing system according to any one of the preceding claims, wherein the input data (x) i () indicates a signal in response to the sleep stage of the subject during the sleep period.

8. The processing system according to claim 7, wherein the output data of the generative model is a time series. The time series mentioned therein is a sleep graph or sleep density graph.

9. The processing system according to any one of the preceding claims, wherein the input data (x) i () indicates the signal response to respiratory disturbances in the subject during sleep.

10. The processing system of claim 9, wherein the output data of the generative model is a time series, wherein the time series represents the probability of sleep events over time.

11. The processing system according to any one of claims 1 to 10, further configured to, for each initial score: Multiple samples are generated by iteratively sampling the initial score; and A measure of the uncertainty of processing the multiple samples to generate the initial score. The initial scores are processed to generate combined scores by performing a process that combines only those initial scores that satisfy one or more predetermined conditions for measuring uncertainty.

12. A computer-implemented method, comprising: Receive multiple initial scores (IS1-ISN). Each initial score (IS1-ISN) is the output of a different score-based neural network (N1-NN) based on the subject-related input data (I1-IN). The multiple initial scores (IS1-ISN) are used in the generative model. Each initial score (IS1-ISN) defines the probability distribution used to sample the output data (y). Each score-based neural network (N1-NN) has been trained independently on a different training dataset; The generative model is used to generate a combined score (CS) by performing a vector derivative with respect to the output data (y) on the plurality of initial scores (IS1-ISN); and The combined score (CS) is processed using a sampling technique (ST) to produce the output data (y) of the generative model. The output data (y) represents the subject's medical condition.

13. A computer program product comprising computer program code components that, when executed on a computing device having a processing system, cause the processing system to perform all the steps of the method according to claim 12.

14. A respiratory support system for providing airflow to a subject, the respiratory support system comprising: The processing system according to any one of claims 1-11; Two sensors; as well as Airflow control system Each of the two sensors is adapted to generate different physiological signals for the subject. The input data mentioned above includes the different physiological signals of the subject. The generative model mentioned therein includes the plurality of score-based neural networks. Each score-based neural network is configured to generate an initial score by processing the corresponding physiological signal among the different physiological signals. Each score-based neural network is trained using corresponding instances of training data that are the same type of data as the corresponding physiological signals in the different physiological signals. The airflow control system is configured to control the characteristics of the airflow toward the subject based on the output data of the generative model.

15. A sleep stage determination system for determining sleep stages of a subject, the sleep stage determination system comprising: The processing system according to any one of claims 1-11; Two sensors; as well as Each of the two sensors is adapted to generate different physiological signals for the subject. The input data mentioned above includes the different physiological signals of the subject. The generative model mentioned therein includes the plurality of score-based neural networks. Each score-based neural network is configured to generate an initial score by processing the corresponding physiological signal among the different physiological signals. Each score-based neural network is trained using a corresponding instance of training data on data of the same type as the corresponding physiological signal in the different physiological signals. The output data therein represents one or more sleep stages of the subject.

Images