Selection of speech features for building a model to detect medical conditions

Abstract

Providing a selection of speech features for building a model to detect medical conditions. [Solution] A mathematical model can be trained to diagnose a person's medical condition by processing the acoustic features 1021 and linguistic features 1022 of a person's voice. The performance of the mathematical model can be improved by appropriately selecting the features 1021 and 1022 used with the mathematical model. Features 1021 and 1022 can be selected by calculating a feature selection score 1031 for each acoustic feature 1021 and each linguistic feature 1022, and then using the score 1031 to select the feature 1021 or 1022 that gave the highest score 1031.

Description

Technical Field

[0001] The present invention relates to the selection of voice features used to construct a mathematical model for detecting a medical condition in order to improve the performance of the model.

Background Art

[0002] Early diagnosis of medical conditions, such as Alzheimer's disease or concussion, can be said to be carried out in consideration of improving the treatment for the person who has developed the medical condition and improving the quality of life. One method that can be used to detect a medical condition is to process a person's voice. This is because a person's voice or the sound of the words a person uses can provide useful information for medical diagnosis.

[0003] To detect a medical condition from a person's voice, features can be extracted from the voice, and these features can also be processed by a mathematical model. The type and number of features extracted from the voice can affect the performance of the model, especially when the amount of training data for training the model is limited. Therefore, if suitable features are selected, the performance of the model can be improved.

Summary of the Invention

Means for Solving the Problems

[0004] What is described herein is a technique for selecting voice features used to construct or train a mathematical model for detecting or diagnosing a medical condition. The techniques described herein can be used for any suitable medical condition, but for the sake of clarity, concussion and Alzheimer's disease are used as examples of medical conditions. However, the techniques described herein are not limited to any particular medical condition. The present invention provides, for example, the following items. (Item 1) A system for training a mathematical model for detecting a medical condition, the system comprising A training corpus containing audio data items is obtained, and each audio data item is accompanied by a diagnostic value. Using automatic speech recognition, obtain speech recognition results for each audio data item, and if the speech recognition result for an audio data item includes transcription of the audio data item, Multiple acoustic features are calculated for each speech data item in the training corpus, and these multiple acoustic features are calculated from the speech data item, and the calculation of these multiple acoustic features does not use the speech recognition results of the speech data item. By processing the speech recognition results, multiple linguistic features are calculated for each speech data item in the training corpus. A feature selection score is calculated for each of the multiple acoustic features and for each of the multiple linguistic features. The feature selection score for a feature indicates its usefulness in detecting the disease state of that feature. The feature selection score is calculated for each audio data item using the value of the feature and the diagnostic value corresponding to the audio data item. Using the feature selection score, select multiple features from the multiple acoustic features and the multiple linguistic features, For each audio data item of the training corpus, the mathematical model for detecting the disease state is trained using the selected features. A system comprising at least one computer configured in such a way. (Item 2) In the system described in item 1, the at least one computer is Using the aforementioned mathematical model, a product or service for detecting the aforementioned medical condition is deployed. Receives audio data items corresponding to human speech, By processing the audio data items using the aforementioned mathematical model, a medical diagnostic score is calculated. A system configured in such a way. (Item 3) In the system described in item 1, each audio data item of the training corpus corresponds to one of a plurality of prompts, and the at least one computer, By processing the audio data items using the mathematical model described above, a medical diagnostic score is calculated for each audio data item in the training corpus. Using the aforementioned medical diagnostic score, a prompt selection score is calculated for each prompt in the plurality of prompts. Using the prompt selection score, a subset of prompts is selected from the plurality of prompts. Using the mathematical model and a subset of the prompts, a product or service for detecting the medical condition is deployed. For each prompt in the subset of the aforementioned prompts, receive an audio data item corresponding to a human voice. By processing the voice data items using the mathematical model, a medical diagnostic score for the person is calculated. A system configured in such a way. (Item 4) In the system described in item 1, the at least one computer is The aforementioned training corpus is divided into multiple folds, Statistics are calculated for each feature and for each fold of the plurality of folds. A system configured in such a way. (Item 5) In the system described in item 4, the at least one computer is Using the statistics for each feature and each of the multiple folds, a stability determination is calculated for each of the multiple acoustic features and the multiple linguistic features. Using the stability determination, select the multiple features. A system configured in such a way. (Item 6) A system in which the mathematical model described in item 1 includes a neural network or a support vector machine. (Item 7) A system in which the plurality of acoustic features include at least one of spectral features, prosodic features, or vocal characteristics, as described in item 1. (Item 8) A computer implementation method for training a mathematical model for detecting disease symptoms, A step of obtaining a training corpus containing audio data items, wherein each audio data item is accompanied by a diagnostic value. A step of obtaining speech recognition results for each voice data item using automatic speech recognition, wherein the speech recognition results for a voice data item include the transcription of the voice data item. A step of calculating multiple acoustic features for each speech data item in the training corpus, wherein the multiple acoustic features are calculated from the speech data item, and the calculation of the multiple acoustic features does not use the speech recognition result of the speech data item. The steps include: processing the speech recognition results to calculate multiple linguistic features for each speech data item in the training corpus; A step of calculating a feature selection score for each of the multiple acoustic features and each of the multiple linguistic features, The feature selection score for a feature indicates its usefulness in detecting the disease state of that feature. The feature selection score is calculated for each audio data item using the value of the feature and the diagnostic value corresponding to the audio data item, in the following steps: The steps include selecting multiple features from the multiple acoustic features and the multiple linguistic features using the feature selection score, The steps include training the mathematical model for detecting the disease state using the selected features for each audio data item of the training corpus, Computer implementation methods, including those mentioned above. (Item 9) A computer implementation method according to item 8, wherein the medical condition is a concussion or Alzheimer's disease. (Item 10) A computer implementation method according to item 8, wherein the plurality of linguistic features include one or more of the following: the number of connecting words over a certain period of time, the number of connecting words over a certain number of words, the difficulty of the words, or the speaking speed. (Item 11) In the computer-implemented method according to item 8, the step of calculating a feature selection score for a feature includes the step of generating a pair of numerical values for each voice data item in the training corpus, wherein the first numerical value in the pair corresponds to a feature value and the second numerical value in the pair corresponds to a diagnostic value. A computer-implemented method. (Item 12) A computer-implemented method according to item 8, The step of dividing the training corpus into a plurality of folds, The step of calculating statistics for each feature and for each fold of the plurality of folds, A computer-implemented method including (Item 13) A computer-implemented method according to item 12, Using the statistics for each feature and for each fold of the plurality of folds, calculating a stability determination for each feature of the plurality of acoustic features and the plurality of linguistic features, Using the stability determination to select the plurality of features, A computer-implemented method including (Item 14) A computer-implemented method according to item 8, The step of selecting a plurality of prompts using the mathematical model, The step of training a second mathematical model using the selected plurality of prompts and the voice data items of the training corpus, A computer-implemented method including (Item 15) One or more non-transitory computer-readable media including computer-executable instructions, which when the computer-executable instructions are executed, An action of obtaining a training corpus including voice data items, wherein each voice data item is accompanied by a diagnostic value, an action, An action that uses automatic speech recognition to obtain speech recognition results for each voice data item, wherein the speech recognition result for a voice data item includes the transcription of the voice data item, An action to calculate multiple acoustic features for each speech data item in the training corpus, wherein the multiple acoustic features are calculated from the speech data item, and the calculation of the multiple acoustic features does not use the speech recognition result of the speech data item. The action of processing the speech recognition results to calculate multiple linguistic features for each speech data item in the training corpus, An action to calculate a feature selection score for each of the multiple acoustic features and each of the multiple linguistic features, The feature selection score for a feature indicates its usefulness in detecting the disease state of that feature. The aforementioned feature selection score is calculated for each audio data item using the value of the feature and the diagnostic value corresponding to the audio data item, and the action is as follows: An action to select multiple features from the multiple acoustic features and the multiple linguistic features using the feature selection score, For each audio data item of the training corpus, an action is taken to train the mathematical model for detecting the disease state using the selected features, One or more non-transient computer-readable media that cause at least one processor to perform an action including the following. (Item 16) In one or more non-temporary computer-readable media as described in item 15, the action of calculating the first acoustic feature among the plurality of acoustic features is: To obtain multiple values, an action is taken to calculate a value for each short-time segment of the audio signal, An action to calculate the first acoustic feature using the aforementioned multiple values, One or more non-temporary computer-readable media, including [the specified format]. (Item 17) One or more non-temporary computer-readable media described in item 15, wherein the feature selection score includes the modulated Rand index, modulated mutual information, absolute Pearson correlation, or absolute Spearman correlation. (Item 18) In one or more non-temporary computer-readable media as described in item 15, the action is: An action to calculate a stability determination for each of the aforementioned multiple acoustic features and the aforementioned multiple linguistic features, Using the stability determination, an action is taken to select the multiple features, One or more non-temporary computer-readable media, including [the specified format]. (Item 19) In one or more non-temporary computer-readable media as described in item 15, each audio data item of the training corpus corresponds to one of a plurality of prompts, and the action is, The action of calculating a medical diagnostic score for the speech data items of the training corpus by processing the speech data items using the mathematical model, An action to calculate a prompt selection score for each prompt in the plurality of prompts using the aforementioned medical diagnostic score, An action to select a subset of prompts from the plurality of prompts using the prompt selection score, Using the mathematical model and a subset of the prompts, an action is taken to deploy a product or service that detects the medical condition, One or more non-temporary computer-readable media, including [the specified format]. (Item 20) In one or more non-temporary computer-readable media as described in item 15, the action is: An action to obtain multiple non-speech features for each speech data item in the aforementioned training corpus, An action to calculate a feature selection score for each of the aforementioned multiple non-speech features, An action to select the plurality of features using the feature selection score for the plurality of non-speech features, One or more non-temporary computer-readable media, including [the specified format]. [Brief explanation of the drawing]

[0005] The present invention and a detailed description of its specific embodiments that follow can be understood by referring to the following figures. [Figure 1] This is a schematic block diagram illustrating another embodiment of a voice-based medical evaluation system. [Figure 2] This is a schematic block diagram illustrating one embodiment of a system that processes speech data using a mathematical model to perform medical diagnoses. [Figure 3] This is a schematic block diagram showing one embodiment of a training corpus of speech data. [Figure 4] This is a schematic block diagram showing one embodiment of a list of prompts to be used when diagnosing a medical condition. [Figure 5] This is a schematic block diagram illustrating one embodiment of a system for selecting features to train mathematical models for diagnosing medical conditions. [Figure 6A] This is a schematic block diagram showing one embodiment in which feature values ​​and diagnostic values ​​are represented graphically. [Figure 6B] This schematic block diagram shows another embodiment in which feature values ​​and diagnostic values ​​are represented graphically. [Figure 7] This is a schematic flowchart illustrating one embodiment of a method for selecting functions to train a mathematical model for diagnosing a medical condition. [Figure 8] This is a schematic flowchart illustrating one embodiment of a method for selecting prompts to be used in conjunction with a mathematical model for diagnosing a medical condition. [Figure 9] This is a schematic flowchart illustrating one embodiment of a method for training a mathematical model to diagnose a medical condition, appropriate to a set of selected prompts. [Figure 10]This is a schematic block diagram showing one embodiment of a computing device that can be used to train and deploy mathematical models for diagnosing medical conditions. [Modes for carrying out the invention]

[0006] Figure 1 shows an example system 100 for diagnosing medical conditions using a person's voice. Figure 1 includes a medical condition diagnosis service 140 that receives a person's voice data, processes this voice data, and determines whether or not the person has a medical condition. For example, the medical condition diagnosis service 140 can process the voice data to calculate a "yes" or "no" determination regarding whether or not the person has a medical condition, or to calculate a score indicating the probability or likelihood that the person has a medical condition, and / or the severity of that condition.

[0007] As used herein, diagnosis relates to any determination of whether a person may have a medical condition, or to any determination of the possible severity of a medical condition. A diagnosis may include any form of assessment, conclusion, opinion, or determination regarding a medical condition. In some cases, a diagnosis may be inaccurate, and a person diagnosed with a medical condition may not actually have one.

[0008] The medical diagnosis service 140 can receive human voice data using any suitable technique. For example, a person may speak into a mobile device 110, which can record the voice and transmit the recorded voice data to the medical diagnosis service 140 via the network 130. Any suitable technique and any suitable network can be used to transmit the voice data recorded by the mobile device 110 to the medical diagnosis service 140. For example, an application or "app" may be installed on the mobile device 110, and the voice data may be transmitted via the internet or a mobile telephone network using a REST (representational state transfer) API (Application Programming Interface) call. In another example, a healthcare provider may have a healthcare provider computer 120 which can be used to record a person's voice and transmit the voice data to the medical diagnosis service 140.

[0009] In one embodiment, the medical condition diagnosis service 140 may be installed on a mobile device 110 or a healthcare provider computer 120, eliminating the need to transmit voice data over a network. The example in Figure 1B is not limiting, and any suitable technique can be used to transmit voice data for processing by mathematical models.

[0010] Subsequently, the output of the medical condition diagnosis service 140 can be used for any appropriate purpose. For example, the information can be presented to the person who provided the audio data, or to the medical professional treating that person.

[0011] Figure 2 shows an example system 200 for processing speech data using a mathematical model to perform medical diagnoses. When processing speech data, features can be calculated from the speech data, and then these features can be processed by the mathematical model. Any suitable type of features can be used.

[0012] Features may include acoustic features, which are any features computed from speech data without performing or relying on speech recognition on the speech data (for example, acoustic features do not use information about the uttered data in the speech data). For example, acoustic features may include mel-frequency cepstral coefficients, perceptual linear prediction features, jitter, or shimmer.

[0013] Features can include linguistic features, which are calculated using the results of speech recognition. For example, linguistic features may include speech rate (e.g., number of vowels or syllables per second), the number of pause fillers (e.g., "um" and "uh"), word difficulty (e.g., less commonly used words), or the phonetic portion of the word following a pause filler.

[0014] In Figure 2, the audio data is processed by the acoustic feature calculation component 210 and the speech recognition component 220. The acoustic feature calculation component 210 can calculate acoustic features from the audio data, such as any of the acoustic features described herein. The speech recognition component 220 can perform automatic speech recognition on the audio data using any suitable technique (e.g., Gaussian mixture models, acoustic modeling, language modeling, and neural networks).

[0015] Since the speech recognition component 220 may use acoustic features when performing speech recognition, there may be some overlap in the processing of these two components, meaning other configurations are also possible. For example, the acoustic feature component 210 could also compute the acoustic features required by the speech recognition component 220, thus eliminating the need for the speech recognition component 220 to compute acoustic features at all.

[0016] The language feature computation component 230 receives speech recognition results from the speech recognition component 220 and processes the speech recognition results to determine language features, such as any of the language features described herein. Speech recognition features can be in any suitable format and may include any suitable information. For example, speech recognition results may include a sequence of multiple possible words, information about connective words, and a word lattice containing the timing of words, syllables, vowels, connective words, or any other unit of speech.

[0017] The disease classifier 240 processes acoustic and linguistic features using a mathematical model and can output one or more diagnostic scores indicating whether a person has a disease, such as a score indicating the probability or likelihood of a person having a disease, and / or a score indicating the severity of the disease. The disease classifier 240 can use any suitable technique, such as a classifier that implements a neural network like a support vector machine or a multilayer perceptron.

[0018] The performance of the disease classifier 240 may depend on the features computed by the acoustic feature computation component 210 and the linguistic feature computation component 230. Furthermore, a set of features that correctly processes one disease may not correctly process another. For example, speech difficulty is an important feature for diagnosing Alzheimer's disease, but it may not be useful in determining whether a person has a concussion. Another example is that features related to vowel, syllable, or word pronunciation may be important for Parkinson's disease, but not so much for other diseases. Therefore, a technique is needed to determine a first set of features that correctly processes a first disease, and this process may need to be repeated to determine a second set of features that correctly processes a second disease.

[0019] In one embodiment, the disease classifier 240 may also use other features, which may be called non-vocal features, in addition to acoustic and linguistic features. For example, features may be derived from or calculated from a person's demographic information (e.g., sex, age, place of residence), medical history (e.g., weight, most recent blood pressure reading, or previous diagnosis), or any other suitable information.

[0020] The selection of features for diagnosing a medical condition becomes all the more important when the amount of training data for training a mathematical model is relatively small. For example, training a mathematical model to diagnose a concussion may require training data that includes voice data from a large number of individuals immediately after experiencing a concussion. Such data may exist in small quantities, and obtaining further examples of such data may require a very long period of time.

[0021] When training a mathematical model, a smaller amount of training data increases the risk of overfitting. In this case, the mathematical model may adapt well to specific training data, but due to the small amount of training data, it may not be able to process new data correctly. For example, a model that can detect all concussions in the training data may have a high error rate when processing production data of people at risk of concussions.

[0022] One technique to prevent overfitting when training mathematical models is to reduce the number of features used to train the model. The amount of training data required to train a model without overfitting increases as the number of features increases. Therefore, by reducing the number of features used, it becomes possible to build a model with less training data.

[0023] When training a model with a limited number of features, it becomes increasingly important to select features that enable the model to function correctly. For example, when a large amount of training data is available, it's possible to train a model using hundreds of features, increasing the likelihood that suitable features will be used. Conversely, when only a small amount of training data is available, a model might be trained using only around 10 features, making it increasingly important to select the features most crucial for diagnosing a medical condition.

[0024] The following are examples of features that can be used to diagnose a medical condition. Acoustic features can be calculated using short-time segment features. When processing audio data, the duration of this audio data may vary. For example, some audio may be 1 or 2 seconds long, while others may be several minutes or longer. For consistency when processing audio data, it is best to process it in units of short segments (sometimes called frames). For example, each short segment could be 25 milliseconds, with segments advancing in 10-millisecond increments, allowing for a 15-millisecond overlap between two consecutive segments.

[0025] The following are non-restrictive examples of short-time segmental features: spectral features (such as Mel-frequency cepstral coefficients or perceptual linear predictions), prosodic features (such as tone, energy, and probability of utterance), speech quality features (such as jitter, jitter of jitter, fluctuation, or harmonic-to-noise ratio), and entropy (if entropy can be calculated from the posterior of an acoustic model trained on natural speech data, for example, to capture how accurately utterances were pronounced).

[0026] By combining short-time segmental features, it is possible to compute acoustic features for speech. For example, a 2-second speech sample can generate 200 short-time segmental features about pitch, and by combining these, one or more acoustic features can be computed about pitch.

[0027] Using any suitable technique, short-time segmented features can be combined to compute acoustic features for speech samples. In one embodiment, acoustic features can be calculated using statistics of short-time segmental features (e.g., arithmetic mean, standard deviation, skewness, kurtosis, first quartile, second quartile, third quartile, second quartile minus first quartile, third quartile minus first quartile, third quartile minus second quartile, 0.01 percentile, 0.99 percentile, 0.99 percentile minus 0.01 percentile), the percentage of short-time segmental features whose value is above a threshold (e.g., the threshold is 75% of the range plus the minimum value), the percentage of segmental features whose value is above a threshold (e.g., the threshold is 90% of the range plus the minimum value), the slope of the linear approximation of the value, the offset of the linear approximation of the value, the linear error calculated as the difference between the linear approximation and the actual value, or the quadratic error calculated as the difference between the linear approximation and the actual value). In one embodiment, acoustic features can also be computed as i-vectors or identity vectors of short-time segmental features. The identity vectors can be computed using any suitable technique, such as performing row-to-vector transformations using factor analysis techniques and Gaussian mixture models.

[0028] The following are non-restrictive examples of linguistic features: Speech rate, or any other suitable measure of speech rate, such as by dividing the duration of all uttered words by the number of vowels; the number of connective words, which can be said to indicate hesitation in speech, determined by (1) dividing the number of connective words by the duration of uttered words, or (2) dividing the number of connective words by the number of uttered words; a measure of word difficulty or the use of uncommon words. For example, word difficulty can be calculated using statistics of 1-gram probabilities of uttered words, such as classifying words according to their frequency percentiles (e.g., 5%, 10%, 15%, 20%, 30%, or 40%). (1) The value obtained by dividing the number of each part-of-speech class by the number of words spoken, or (2) The value obtained by dividing the number of each part-of-speech class by the sum of the total number of all part-of-speech components, which is the part of the word following a connecting word.

[0029] In one embodiment, the linguistic features may include determining whether a person answered a question correctly. For example, a person might be asked what year it is or who the president of the United States is. By processing this person's voice, it is possible to determine what the person said in response to the question and further determine whether the person answered the question correctly.

[0030] To train a model for diagnosing medical conditions, one should collect a corpus of training data. This training corpus should include examples of voices that reveal human diagnoses. For example, it should indicate whether a person has no concussion, a mild concussion, a moderate concussion, or a severe concussion.

[0031] Figure 3 shows an example of a training corpus containing audio data for training a model to diagnose concussions. For example, in the table in Figure 3, rows may correspond to entries in a database. In this example, each entry includes a person identifier, a known diagnosis about that person (e.g., not concussed, mild, moderate, or severe concussion), an identifier of a prompt or question presented to the person (e.g., "How are you feeling today?"), and the filename of the file containing the audio data. The training data can be stored in any suitable format using any suitable storage technique.

[0032] A training corpus can store representations of human speech using any suitable format. For example, a speech data entry in a training corpus may include a digital sample of the audio signal received by a microphone, or it may include a processed version of the audio signal, such as a Mel-frequency cepstral coefficient.

[0033] A single training corpus may contain speech data for multiple medical conditions, or separate training corpora may be used for each medical condition (e.g., a first training corpus for concussions and a second training corpus for Alzheimer's disease). A separate training corpus may be used to store speech data for individuals whose medical condition is unknown or undiagnosed, because this training corpus can be used to train models for multiple medical conditions.

[0034] Figure 4 shows an example of storing prompts that can be used to diagnose a medical condition. Each prompt can be presented to a person (e.g., a medical professional) or a computer to obtain a human voice in response to that prompt. Each prompt may have a prompt identifier so that it can be cross-referenced with prompt identifiers in a training corpus. The prompts in Figure 4 can be stored using any suitable storage technique, such as a database.

[0035] Figure 5 shows an example system 500 that can be used to select features for training a mathematical model to diagnose a medical condition, and then train the mathematical model using the selected features. System 500 can be used multiple times to select features for each different medical condition. For example, the first use of system 500 may be to select features for diagnosing a concussion, and the second use of system 500 may be to select features for diagnosing Alzheimer's disease.

[0036] Figure 5 includes a training corpus 510 of speech data items for training a mathematical model to diagnose medical conditions. The training corpus 510 may include any suitable information, such as speech data of multiple people with and without medical conditions, labels indicating whether or not a person has a medical condition, and any other information as described herein.

[0037] The acoustic feature computation component 210, the speech recognition component 220, and the language feature computation component 230 can be implemented as described above to compute acoustic and language features for speech data in a training corpus. The acoustic feature computation component 210 and the language feature computation component 230 can compute a large number of features so that the best-performing features can be determined. This is in contrast to the case in Figure 2 where these components are used in a production system and therefore only need to compute previously selected features.

[0038] The feature selection score calculation component 520 can calculate a selection score for each feature (which may be an acoustic feature, a linguistic feature, or any other feature described herein). To calculate a selection score for a feature, a pair of numbers can be created for each speech data item in the training corpus. The first number in this pair is the feature value, and the second number in this pair is the symptom diagnosis index. The symptom diagnosis index may have two values ​​(for example, 0 if the person has no symptoms, and 1 if the person has symptoms), or it may have more than one number (for example, a real number between 0 and 1, or a number of integers indicating the likelihood or severity of the symptoms).

[0039] Therefore, for each feature, a pair of numerical values ​​can be obtained for each audio data item in the training corpus. Figures 6A and 6B show two conceptual plots of numerical pairs for the first and second features. In Figure 6A, there does not appear to be a pattern or correlation between the values ​​of the first feature and the corresponding diagnostic values, but in Figure 6B, there appears to be a pattern or correlation between the values ​​of the second feature and the diagnostic values. Therefore, we can conclude that the second feature is likely to be a useful feature for determining whether or not a person has a medical condition, while the first feature is not.

[0040] The feature selection score calculation component 520 can calculate a selection score for a feature using pairs of feature values ​​and diagnostic values. The feature selection score calculation component 520 can calculate any suitable score that shows a pattern or correlation between feature values ​​and diagnostic values. For example, the feature selection score calculation component 520 can calculate the Rand index, modulated Rand index, mutual information, modulated mutual information, Pearson correlation, absolute Pearson correlation, Spearman correlation, or absolute Spearman correlation.

[0041] The selection score can indicate the usefulness of a feature in detecting a disease state. For example, a high selection score may indicate that a certain feature should be used when training a mathematical model, while a low selection score may indicate that that feature should not be used when training a mathematical model.

[0042] The feature stability determination component 530 can determine whether a feature (which may be an acoustic feature, a linguistic feature, or any other feature described herein) is stable or unstable. To perform stability determination, the speech data item can be divided into several groups. These groups may also be called folds. For example, the speech data item may be divided into five folds. In one embodiment, the speech data item may be divided into folds such that each fold has approximately equal numbers of speech data items for different gender and age groups.

[0043] The statistics of each fold can be compared with those of other folds. For example, for the first fold, the median (or mean, or any other statistic relating to the center or middle of the distribution) feature value (indicated as M1) can be determined. Statistics can also be calculated for combinations of other folds. For example, for combinations of several other folds, the median of the feature value (indicated as M0) and a statistical measure of the variability of the feature value (indicated as V0), such as the interquartile range, variance, or standard deviation, can be calculated. If the median of the first fold differs significantly from the median of the second fold, the feature can be determined to be unstable. For example,

[0044]

number

[0045] If this is the case, the feature can be determined to be unstable. Here, C is the multiplier. This process can then be repeated for each other fold. For example, as mentioned above, the median of the second fold may be compared with the medians and variability of the other folds.

[0046] In one embodiment, after comparing each fold with the others, a feature can be determined to be stable if the median of each fold is not too far from the median of the other folds. Conversely, a feature can be determined to be unstable if the median of any fold is too far from the median of the other folds.

[0047] In one embodiment, the feature stability determination component 530 may output a Boolean value for each feature to indicate whether the feature is stable or not. In another embodiment, the stability determination component 530 may output a stability score for each feature. For example, the stability score may be calculated as the largest distance (e.g., Mahalanobis distance) between the medians of one fold and another.

[0048] The feature selection calculation component 540 receives a selection score from the feature selection score calculation component 520 and a stability determination from the feature stability determination component 530, and can select a subset of features to be used to train the mathematical model. The feature selection component 540 can select multiple features that have the highest selection score and are also sufficiently stable.

[0049] In one embodiment, the number of features to be selected (or the maximum number of features to be selected) may be predetermined. For example, the number N may be determined based on the amount of training data, and N features may be selected. Feature selection may be determined by removing unstable features (e.g., features determined to be unstable, or features with a stability score below a threshold), and then selecting the N features with the highest selection scores.

[0050] In one embodiment, the number of features selected may be based on selection scores and stability criteria. For example, feature selection may be determined by removing unstable features and then selecting all features whose selection scores are above a threshold.

[0051] In one embodiment, when selecting features, the selection score and stability score may be combined. For example, a combined score may be calculated for each feature (by adding or multiplying the selection score and stability score for that feature), and this combined score may be used to select features.

[0052] Next, the model training component 550 can train a mathematical model using the selected features. For example, the model training component 550 can iterate through speech data items of the training corpus, obtain selected features for the speech data items, and then train a mathematical model using the selected features. In one embodiment, as part of model training, a dimensionality reduction technique such as principal component analysis or linear discriminant analysis may be applied to the selected features. Any suitable mathematical model can be trained, such as any of the mathematical models described herein.

[0053] In some embodiments, other techniques, such as the wrapper method, may be used for feature selection, or in combination with the feature selection techniques described above. The wrapper method can select a set of features, train a mathematical model using this selected set of features, and then evaluate the performance of the set of features using the trained model. When the number of possible features is relatively small and / or the training time is relatively short, all possible sets of features may be evaluated and the best-performing set may be selected. When the number of possible features is relatively large and / or training time is an important factor, optimization techniques may be used to iteratively discover a set of features that performs well. In some embodiments, a set of features may be selected using system 500, and then a subset of these features may be selected using the wrapper method as the final set of features.

[0054] Figure 7 is a flowchart of an example embodiment for selecting features to train a mathematical model for diagnosing a medical condition. In Figure 7 and other flowcharts herein, the order of steps is illustrative and other orders are possible; not all steps are necessary; steps can be combined (whole or partially) or subdivided; and in some embodiments, some steps may be omitted or other steps may be added. Any method described by the flowcharts herein can be implemented, for example, by any of the computers or systems described herein.

[0055] In step 710, a training corpus of audio data items is obtained. The training corpus may include any other suitable information, such as an audio signal representation of a person's voice, a medical diagnosis of the person from whom the voice was obtained, and any of the information described herein.

[0056] In step 720, speech recognition results are obtained for each audio data item of the training corpus. The speech recognition results may be pre-calculated and stored with the training corpus, or stored elsewhere. The speech recognition results may include arbitrary suitable information such as the written transcript, a list of the highest-scoring written transcripts (e.g., N best lists), a lattice of possible transcriptions, and timing information such as the start and end times of words, connectives, or other audio units.

[0057] In step 730, acoustic features are calculated for each speech data item of the training corpus. Acoustic features may include any features calculated without using the speech recognition results of the speech data item, such as any of the acoustic features described herein. Acoustic features may include or be calculated from data used in the speech recognition process (e.g., Mel frequency cepstral coefficients or perceptual linear predictors), but acoustic features do not use speech recognition results, such as information about words or connective words present in the speech data item.

[0058] In step 740, a linguistic feature is calculated for each speech data item of the training corpus. The linguistic feature may include any feature calculated using the speech recognition result, such as any of the linguistic features described herein.

[0059] In step 750, a feature selection score is calculated for each acoustic feature and each linguistic feature. To calculate the feature selection score for a feature, the feature value for each speech data item in the training corpus may be used together with other information, such as known diagnostic values ​​corresponding to the speech data item. The feature selection score may be calculated using any of the techniques described herein, such as by calculating the absolute Pearson correlation. In one embodiment, the feature selection score may be similarly calculated for other features, such as features relating to a person's demographic information.

[0060] In step 760, multiple features are selected using feature selection scores. For example, a number of features with the highest selection scores may be selected. In one embodiment, a stability determination may be calculated for each feature, and multiple features may be selected using both feature selection scores and stability determinations, such as by using any of the techniques described herein.

[0061] In step 770, a mathematical model is trained using the selected features. Any suitable mathematical model can be trained, such as a neural network or a support vector machine. After training the mathematical model, it can be deployed in a production system, such as the voice module 104 and system 109 in Figure 1B, to perform disease diagnosis.

[0062] The steps in Figure 7 can be performed in various ways. For example, in one embodiment, steps 730 and 740 may be performed in a loop, repeatedly for each speech data item in the training corpus. In the first iteration, acoustic and linguistic features may be calculated for the first speech data item, and in the second iteration, acoustic and linguistic features may be calculated for the second speech data item, and so on.

[0063] When using a model deployed to diagnose a medical condition, a series of prompts or questions can be issued to the person being diagnosed in order to obtain their voice. Any suitable prompts can be used, such as any of the prompts shown in Figure 4. After features have been selected as described above, prompts can be selected so that the selected prompts provide useful information about the selected features.

[0064] For example, suppose the selected feature is pitch. Pitch is determined to be a useful feature for diagnosing a medical condition, but in some cases, a particular prompt may be better than others for obtaining a useful pitch feature. Very short utterances (e.g., yes / no answers) may not provide enough data to accurately calculate pitch, and therefore, prompts that generate longer responses can be more useful in obtaining information about pitch.

[0065] To give another example, let's assume the selected feature is word difficulty. While word difficulty is considered a useful feature for diagnosing a medical condition, some prompts may be better than others for obtaining useful word difficulty features. A prompt asking the user to read a given passage generally results in the words in that passage being pronounced, and therefore the word difficulty feature will have the same value each time the prompt is presented. In other words, this prompt is not useful for obtaining information about word difficulty. In contrast, an open-ended question such as "Tell me about your day" results in a wider variety of vocabulary in the responses, and therefore can provide more useful information about word difficulty.

[0066] Furthermore, selecting a set of prompts can improve the performance of the system for diagnosing medical conditions and provide a better experience for those being evaluated. By using the same set of prompts for each person being evaluated, the system can obtain more accurate results because data collected from multiple people is easier to compare than data collected from each individual using different prompts. In addition, using a predetermined set of prompts makes it easier to predict a person's evaluation and to predict the desired duration of evaluation that is appropriate for assessing a medical condition. For example, to evaluate whether or not someone has Alzheimer's disease, it is acceptable to use more prompts to collect more data, but to evaluate whether or not someone has suffered a concussion at a sporting event, it is reasonable to reduce the number of prompts used to obtain results more quickly.

[0067] In one embodiment, prompts may be selected by calculating a prompt selection score. The training corpus may have multiple speech data items for a single prompt, or even a large number of speech data items. For example, the training corpus may include examples of prompts used by different people, or the same prompt may be used multiple times by the same person.

[0068] Figure 8 is a flowchart of an example embodiment in which prompts are selected for use with a deployed model to diagnose a medical condition. Steps 810 through 840 may be performed for each prompt (or subset of prompts) in the training corpus to calculate a prompt selection score for each prompt.

[0069] In step 810, a prompt is obtained, and in step 820, an audio data item corresponding to this prompt is obtained from the training corpus. In step 830, a medical diagnosis score is calculated for each voice data item corresponding to this prompt. For example, the medical diagnosis score for a voice data item may be a numerical value output by a mathematical model (e.g., the mathematical model trained in Figure 7) that indicates the likelihood that a person has a medical condition and / or the severity of that condition.

[0070] In step 840, the calculated medical diagnostic score is used to calculate a prompt selection score for the prompt. The calculation of the prompt selection score may be similar to the calculation of the feature selection score as described earlier. For each voice data item corresponding to a prompt, a pair of numbers can be obtained. For each pair, the first number in the pair may be the medical diagnostic score calculated from the voice data item, and the second number in the pair may be a known medical diagnosis for the person (e.g., it is known that this person has a medical condition, or that the severity of this condition is indicated). Plotting these numerical pairs yields a plot similar to Figure 6A or Figure 6B, and for some prompts, there may or may not be a pattern or correlation in the numerical pairs.

[0071] The prompt selection score for a prompt may include any score that shows a pattern or correlation between the calculated medical diagnostic score and known medical condition diagnoses. For example, the prompt selection score may include the Rand index, modulated Rand index, mutual information, modulated mutual information, Pearson correlation, absolute Pearson correlation, Spearman correlation, or absolute Spearman correlation.

[0072] In step 850, it is determined whether there are any other prompts that need to be processed. If there are any prompts that need to be processed, the process can proceed to step 810 to process the additional prompts. If all prompts have been processed, the process can proceed to step 860.

[0073] In step 860, multiple prompts are selected using the prompt selection score. For example, multiple prompts having the highest prompt selection score may be selected. In one embodiment, a stability determination may be calculated for each prompt, and multiple prompts may be selected using both the prompt selection score and the prompt stability score, by using any of the techniques described herein.

[0074] In step 870, the selected prompts are used in conjunction with the deployed medical diagnosis service. For example, when diagnosing a person, the selected prompts are presented to the person, and the person's voice can be obtained in response to each prompt.

[0075] In some embodiments, other techniques, such as the wrapper method, may be used for prompt selection, or in combination with the prompt selection techniques described above. In some embodiments, a set of prompts may be selected using the process shown in Figure 8, and then a subset of these prompts may be selected as a final set of features using the wrapper method.

[0076] In one embodiment, a person involved in creating the medical diagnosis service may assist in the selection of prompts. This person can use their knowledge or experience to select prompts based on the selected features. For example, if the selected feature is word difficulty, this person can review the prompts and select those that are most likely to provide useful information about word difficulty. This person can select one or more prompts that are most likely to provide useful information for each of the selected features.

[0077] In one embodiment, this person can review the prompts selected by the process in Figure 8 and add or remove prompts to improve the performance of the disease diagnosis system. For example, two prompts may each provide useful information about the difficulty of a word, but the information provided by these two prompts may be highly redundant, and using both prompts may not provide a significant benefit compared to using only one of them.

[0078] In one embodiment, after prompt selection, a second mathematical model appropriate to the selected prompt can also be trained. The mathematical model trained in Figure 7 can process one utterance (responding to a prompt) to generate a medical diagnostic score. The process of performing a diagnosis may include a step of processing multiple utterances corresponding to multiple prompts, and then the mathematical model in Figure 7 can process each utterance to generate multiple medical diagnostic scores. In some cases, it may be necessary to combine multiple medical diagnostic scores in some way to make a determination about an overall medical diagnosis. Therefore, the mathematical model trained in Figure 7 may not be appropriate to a selected set of prompts.

[0079] When selected prompts are used in a session to diagnose a person, each prompt can be presented to the person, and a corresponding vocalization can be obtained for each prompt. Instead of processing the vocalizations separately, the model can also process the vocalizations simultaneously to generate a medical diagnostic score. Thus, the model can be adapted to the selected prompts, as it is trained to process the corresponding vocalizations for each selected prompt simultaneously.

[0080] Figure 9 is a flowchart of an example embodiment for training a mathematical model appropriate to a set of selected prompts. In step 910, a first mathematical model is obtained by using the process in Figure 7. In step 920, a set of prompts is selected using the first mathematical model by the process in Figure 8.

[0081] In step 930, a second mathematical model is trained to generate a medical diagnostic score by simultaneously processing multiple audio data items corresponding to multiple selected prompts. When training the second mathematical model, a training corpus can be used that includes sessions with audio data items corresponding to each of the multiple selected prompts. When training this mathematical model, the input to the mathematical model may be fixed to the audio data items from the sessions and corresponding to each of the selected prompts. The output of the mathematical model may be fixed to known medical diagnoses.

[0082] Next, the parameters of this model can be trained to optimally process the voice data items to simultaneously generate medical diagnostic scores. Any suitable training technique, such as stochastic gradient descent, can be used.

[0083] Next, the second mathematical model can be deployed as part of the medical condition diagnosis service, as shown in the voice module 104 and the service in Figure 1. Since the second mathematical model is trained to process utterances simultaneously rather than individually, it can perform better than the first mathematical model. In other words, training allows for the generation of more accurate medical condition diagnosis scores by combining information from all utterances.

[0084] Figure 10 shows components of one embodiment of a computing device 1000 for implementing any of the techniques described above. In Figure 10, the components are shown as being on a single computing device, but the components can also be distributed across multiple computing devices, such as a system of computing devices including, for example, end-user computing devices (e.g., smartphones or tablets) and / or server computing devices (e.g., cloud computing).

[0085] The computing device 1000 may include any components typical of a computing device, such as volatile or non-volatile memory 1010, one or more processors 1011, and one or more network interfaces 1012. The computing device 1000 may also include any input and output components, such as a display, keyboard, and touchscreen. Furthermore, the computing device 1000 may include various components or modules that provide specific functions, and these components or modules may be implemented in software, hardware, or a combination thereof. Various examples of components are described below as an example of an implementation, but other implementations may include additional components or omit some of the components described below.

[0086] The computing device 1000 may have an acoustic feature calculation component 1021 that can calculate acoustic features for speech data items as previously described. The computing device 1000 may have a language feature calculation component 1022 that can calculate language features for speech data items as previously described. The computing device 1000 may have a speech recognition component 1023 that can generate speech recognition results for speech data items as previously described. The computing device 1000 may have a feature selection score calculation component 1031 that can calculate selection scores for features as previously described. The computing device 1000 may have a feature stability score calculation component 1032 that can perform stability determinations or calculate stability scores as previously described. The computing device 1000 may have a feature selection component 1033 that can select features using selection scores and / or stability determinations as previously described. The computing device 1000 may have a prompt selection score calculation component 1041 that can calculate selection scores for prompts as previously described. The computing device 1000 may have a prompt stability score calculation component 1042 that can perform a stability determination or calculate a stability score, as described earlier. The computing device 1000 may have a prompt selection component 1043 that can select a prompt using a selection score and / or stability determination, as described earlier. The computing device 1000 may have a model training component 1050 that can train a mathematical model, as described earlier. The computing device 1000 may have a medical condition diagnosis component 1060 that can process voice data items to determine a medical diagnosis score, as described earlier.

[0087] The computing device 1000 may include, or be able to access, various data stores, such as the training corpus data store 1070. The data stores may use any well-known storage technique, such as files, relational or non-relational databases, or any non-temporary computer-readable media.

[0088] The methods and systems described herein can also be deployed, in part or in whole, by machines that execute computer software, program code, and / or instructions on a processor. “Processor” as used herein means including at least one processor, and the plural and singular should be understood to be mutually interchangeable unless the context clearly indicates otherwise. Any aspect of this disclosure involves a method on a machine, a system or apparatus as part of or relating to a machine, or a computer readout executed on one or more machines. It can be realized as a computer program product embodied in an available medium. The processor may be a server, client, network infrastructure, mobile computing platform, stationary computing platform, or part of another computing platform. The processor may be any type of computing or processing device capable of executing program instructions, code, binary instructions, etc. The processor may be a single processor, a digital processor, an embedded processor, a microprocessor, or any variant such as a coprocessor (mass coprocessor, graphics coprocessor, communications coprocessor, etc.) that can directly or indirectly facilitate the execution of stored program code or program instructions. In addition, the processor may enable the execution of multiple programs, threads, and code. Multiple threads may be executed simultaneously to improve processor performance and to facilitate simultaneous processing of applications. In one embodiment, the methods, program code, program instructions, etc. described herein may be implemented in one or more threads. Threads may spawn other threads and assign priorities to them in association, and the processor may execute these threads based on priority or on any other order based on instructions given in the program code. The processor may include memory for storing methods, code, instructions, and programs as described herein and elsewhere. The processor may have an interface to access storage media capable of storing methods, code, and instructions as described herein and elsewhere.Storage media accompanying a processor for storing methods, programs, code, program instructions, or other types of instructions that can be executed by a computing device or processing device may include, but are not limited to, one or more of the following: CD-ROM, DVD, memory, hard disk, flash drive, RAM, ROM, cache, etc.

[0089] The processor may include one or more cores that can improve the speed and performance of the multiprocessor. In embodiments, the process may be a dual-core processor, a quad-core processor, or other chip-level multiprocessor that combines two or more independent cores (referred to as dies).

[0090] The methods and systems described herein can be deployed, in part or in whole, by servers, clients, firewalls, gateways, hubs, routers, or other machines running computer software on such computers and / or networking hardware. Software programs can be associated with servers, which may include other variants such as file servers, print servers, domain servers, internet servers, intranet servers, and secondary servers, host servers, distributed servers, etc. A server may include one or more of the following: memory, processors, computer-readable media, storage media, ports (physical and virtual), communication devices, and interfaces that can access other servers, clients, machines, and devices via wired or wireless media. Methods, programs, or code described herein and elsewhere may be executed by a server. In addition, other devices required for the execution of methods described herein may be considered part of the infrastructure associated with the server.

[0091] The server can provide an interface to other devices. These other devices include, but are not limited to, clients, other servers, printers, database servers, print servers, file servers, communication servers, distributed servers, and the like. In addition, this coupling and / or connection can facilitate the remote execution of programs across networks. By networking some or all of these devices, parallel processing of programs or methods in one or more locations can be facilitated without departing from the scope of this disclosure. In addition, any device attached to the server via the interface may include at least one storage medium capable of storing methods, programs, code, and / or instructions. A central repository may provide program instructions to be executed on different devices. In this embodiment, a remote repository can act as a storage medium for program code, instructions, and programs.

[0092] Software programs can also be associated with clients. Clients may include file clients, print clients, domain clients, internet clients, intranet clients, and other variants such as secondary clients, host clients, and distributed clients. A client may include one or more of the following: memory, processors, computer-readable media, storage media, ports (physical and virtual), communication devices, and interfaces that can access other clients, servers, machines, and devices via wired or wireless media. Methods, programs, or code as described herein and elsewhere may be executed by a client. In addition, other devices required to perform methods as described herein may be considered part of the infrastructure associated with the client.

[0093] A client can provide an interface to other devices. These other devices include, but are not limited to, servers, other clients, printers, database servers, print servers, file servers, communication servers, distributed servers, and the like. In addition, this coupling and / or connection can facilitate the remote execution of programs across networks. By networking some or all of these devices, parallel processing of programs or methods in one or more locations can be facilitated without departing from the scope of this disclosure. In addition, any device attached to a client via the interface may include at least one storage medium capable of storing methods, programs, applications, code, and / or instructions. A central repository may provide program instructions to be executed on different devices. In this embodiment, a remote repository can act as a storage medium for program code, instructions, and programs.

[0094] The methods and systems described herein can also be deployed, in part or in whole, via a network infrastructure. The network infrastructure may include elements such as computing devices, servers, routers, hubs, firewalls, clients, personal computers, communication devices, routing devices, and other active and passive devices, modules, and / or components known in the art. One or more computing and / or non-computing devices associated with the network infrastructure may include storage media such as flash memory, buffers, stacks, RAM, ROM, etc., in addition to other components. The processes, methods, program code, and instructions described herein and elsewhere may be executed by one or more of the network infrastructure elements.

[0095] The methods, program code, and instructions described herein and elsewhere may also be implemented on a cellular network having multiple cells. The cellular network may be either a frequency division multiple access (FDMA) network or a code division multiple access (CDMA) network. The cellular network may include mobile devices, cell sites, base stations, repeaters, antennas, towers, etc. The cellular network may be GSM®, GPRS, 3G, EVDO, mesh, or other network types.

[0096] The methods, program code, and instructions described herein and elsewhere may also be implemented on or through mobile devices. Mobile devices may include navigation devices, cell phones, mobile telephones, mobile personal digital assistants, laptops, palmtops, netbooks, pagers, e-readers, music players, and the like. These devices may include, in addition to other components, storage media such as flash memory, buffers, RAM, ROM, and one or more computing devices. A computing device associated with a mobile device may be enabled to execute the program code, methods, and instructions stored therein. Alternatively, a mobile device may be configured to execute instructions in cooperation with other devices. A mobile device may be configured to communicate with a base station interfaced with a server and execute program code. A mobile device may also communicate over a peer-to-peer network, a mesh network, or other communication network. The program code may be stored on storage media attached to the server and executed by a computing device embedded within the server. The base station may include computing devices and storage media. The storage device can store program code and instructions executed by computing devices associated with the base station.

[0097] Computer software, program code, and / or instructions are stored on and / or accessible on machine-readable media. Machine-readable media include computer components, devices, and recording media that hold digital data used to perform calculations over time; semiconductor storage known as random-access memory (RAM); mass storage, typically for more persistent storage, such as optical discs, hard disks, tapes, drums, cards, and other types of magnetic storage; processor registers, cache memory, volatile memory, non-volatile memory; optical storage such as CDs and DVDs; removable media such as flash memory (e.g., USB sticks or keys), floppy disks, magnetic tape, paper tape, punch cards, standalone RAM disks, Zip drives, removable mass storage, offline storage, etc.; and other computer memories such as dynamic memory, static memory, read / write storage, variable storage, read-only, random access, sequential access, location-addressable, file-addressable, content-addressable, network-mounted storage, storage area networks, barcodes, magnetic ink, etc.

[0098] The methods and systems described herein can convert physical items and / or intangible items from one state to another. Furthermore, the methods and systems described herein can convert data representing physical items and / or intangible items from one state to another.

[0099] Throughout the diagrams, the elements included in the flowcharts and block diagrams, and described and illustrated herein, imply logical boundaries between them. However, according to the practice of software and hardware design, the illustrated elements and their functions may be implemented on a machine having a processor via a computer executable medium. The processor can execute program instructions stored on the medium as a monolithic software structure, as a standalone software module, as a module employing external routines, code, services, etc., or any combination thereof, and all such embodiments fall within the scope of this disclosure. Examples of such machines include, but are not limited to, personal digital assistants, laptops, personal computers, mobile phones, other handheld computing devices, medical devices, wired or wireless communication devices, converters, chips, calculators, satellites, tablet PCs, ebooks, gadgets, electronic devices, devices with artificial intelligence, computing devices, networking equipment, servers, routers, etc. Furthermore, the elements shown in the flowcharts and block diagrams, or any other logical components, may be implemented on a machine capable of executing program instructions. In other words, while the above drawings and descriptions illustrate the functional aspects of the disclosed system, specific software configurations for implementing these functional aspects do not need to be inferred from these descriptions unless explicitly stated, or at least evident from the context. Similarly, it will be acknowledged that the various steps confirmed and described above may be modified in various ways, and the order of the steps may be altered to suit specific uses of the techniques disclosed herein. All such variations and modifications are intended to fall within the scope of this disclosure.Therefore, illustrations and / or descriptions of the order of various steps should not be interpreted as requiring a specific order of execution unless necessary for a particular use, or unless explicitly stated, or otherwise evident from the context.

[0100] The methods and / or processes described above, as well as their steps, can be implemented with hardware, software, or any combination of hardware and software suitable for a particular application. Hardware may include general-purpose computers and / or dedicated computing devices, or specific computing devices, or specific embodiments or components of specific computing devices. Processes can be implemented in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices, together with internal and / or external memory. Furthermore, or alternatively, processes can be embodied in application-specific integrated circuits, programmable gate arrays, programmable array logic, or any other device or combination of devices that can be configured to process electronic signals. Moreover, it will be acknowledged that one or more of the above processes can be implemented as computer executable code that can run on machine-readable media.

[0101] Computer executable code can be written using structured programming languages ​​such as C, object-oriented programming languages ​​such as C++, or any other high- or low-level programming language (including assembly language, hardware description language, and database programming languages ​​and techniques), and can be stored, compiled, or interpreted for execution on any of the above devices, or on a processor, processor architecture, or a heterogeneous combination of different hardware and software, or any other machine capable of executing program instructions.

[0102] Accordingly, in one embodiment, each of the methods and combinations described above can be embodied in computer executable code, and when the computer executable code is executed on one or more computing devices, the steps are performed. In other embodiments, these methods can be embodied in a system that performs the steps, and can be distributed across multiple devices in a number of ways, or all of the functionality can be integrated into a dedicated standalone device or other hardware. In other embodiments, the means for performing the steps related to the processes described above can include any of the hardware and / or software described above. All such permutations and combinations are intended to fall within the scope of this disclosure.

[0103] Although the present invention has been disclosed in relation to preferred embodiments described in detail above, various modifications and improvements thereto will be readily apparent to those skilled in the art. Therefore, the spirit and scope of the present invention should not be limited by the foregoing description, but should be understood in the broadest sense permitted by law.

[0104] All documents cited herein are deemed to have their contents included in this application as a result of the citation.

Claims

1. A system for detecting a medical condition, The system comprises at least one computer, The aforementioned at least one computer is The system presents a prompt to a person, the prompt being selected from a plurality of prompts based on a plurality of prompt selection scores calculated for the plurality of prompts, the plurality of prompt selection scores being based on a plurality of medical diagnostic values ​​associated with each audio data item in a training corpus, the plurality of medical diagnostic values ​​being generated for each audio data item by a mathematical model trained to detect the medical condition using a plurality of features selected from the plurality of audio data items, the plurality of features being selected based on the plurality of medical diagnostic values ​​of the plurality of audio data items and their usefulness in detecting the medical condition, and each audio data item in the training corpus corresponding to one of the plurality of prompts. The system receives a voice data item corresponding to a human voice in response to the prompt, The system derives a medical diagnosis by processing the received audio data items using the mathematical model, The system displays the medical diagnosis. A system configured to perform the following actions.

2. The system according to claim 1, wherein the plurality of features include language features, the language features are calculated for each speech data item in the training corpus by processing a plurality of speech recognition results, each speech data item in the training corpus corresponds to the plurality of speech recognition results, and each speech data item includes a transcription of the speech data item.

3. The at least one computer is For each presented prompt, receive an audio data item corresponding to the person's voice, By processing the multiple received audio data items using the mathematical model, the medical diagnosis for the person is derived. The system according to claim 1, configured to perform the following:

4. The system according to claim 1, wherein the plurality of features include voice features, the voice features are calculated from the voice data items, and the calculation of the voice features does not use a plurality of voice recognition results of the voice data items.

5. The system according to claim 4, wherein the plurality of features include language features calculated from a plurality of speech recognition results of speech data items.

6. The system according to claim 1, wherein the mathematical model includes a neural network or a support vector machine.

7. The system according to claim 1, wherein the plurality of features include at least one of spectral features, prosodic features, or vocal features.

8. The at least one computer is The method involves obtaining a training corpus containing multiple audio data items, each audio data item being labeled with a diagnostic value, and each audio data item in the training corpus corresponding to one of the multiple prompts. By processing the multiple audio data items using the aforementioned mathematical model, a medical diagnostic score is calculated for each audio data item in the training corpus. Using the aforementioned multiple medical diagnostic values, a prompt selection score is calculated for each of the aforementioned multiple prompts. Using the plurality of prompt selection scores, select a subset of the plurality of prompts, wherein the prompt is selected from the subset of the plurality of prompts. The system according to claim 1, configured to perform the following:

9. A computer implementation method for detecting a medical condition, The aforementioned method, The presenting of a prompt to a person, wherein the prompt is selected from a plurality of prompts based on a plurality of prompt selection scores calculated for the plurality of prompts, wherein the plurality of prompt selection scores are based on a plurality of medical diagnostic values ​​associated with each audio data item in a training corpus, wherein the plurality of medical diagnostic values ​​are generated for each audio data item by a mathematical model trained to detect the medical condition using a plurality of features selected from the plurality of audio data items, wherein the plurality of features are selected based on the plurality of medical diagnostic values ​​of the plurality of audio data items and their usefulness in detecting the medical condition, wherein each audio data item in the training corpus corresponds to one of the plurality of prompts. In response to the aforementioned prompt, the system receives an audio data item corresponding to a human voice, By processing the received audio data items using the aforementioned mathematical model, a medical diagnosis can be derived. Displaying the aforementioned medical diagnosis Computer implementation methods, including those mentioned above.

10. The computer implementation method according to claim 9, wherein the medical condition is a concussion or Alzheimer's disease.

11. The computer implementation method according to claim 9, wherein the plurality of features include one or more of the following: the number of connecting words over a certain period of time, the number of connecting words over a certain number of words, the difficulty of the words, or the speaking speed.

12. The computer implementation method according to claim 9, wherein the feature value and diagnostic value are a pair of values ​​associated with an audio data item from the training corpus, the pair of values ​​is generated for each audio data item of the training corpus, and the plurality of pairs of values ​​are used to calculate a feature selection score for a feature.

13. The plurality of features are, The aforementioned training corpus is divided into multiple folds, To calculate statistics for each feature and for each of the multiple folds mentioned above. A computer implementation method according to claim 9, further selected based on the above.

14. The plurality of features are, Multiple stability determinations for each feature, calculated using the statistics for each feature and each of the multiple folds. A computer implementation method according to claim 13, further selected based on the above.

15. One or more non-temporary computer-readable media containing a plurality of computer-executable instructions, The aforementioned computer-executable instructions, when executed, cause at least one processor to perform multiple actions. The aforementioned multiple Actions are, The presenting of a prompt to a person, wherein the prompt is selected from a plurality of prompts based on a plurality of prompt selection scores calculated for the plurality of prompts, wherein the plurality of prompt selection scores are based on a plurality of medical diagnostic values ​​associated with each audio data item in a training corpus, wherein the plurality of medical diagnostic values ​​are generated for each audio data item by a mathematical model trained to detect a medical condition using a plurality of features selected from the plurality of audio data items, wherein the plurality of features are selected based on the plurality of medical diagnostic values ​​of the plurality of audio data items and their usefulness in detecting the medical condition, wherein each audio data item in the training corpus corresponds to one of the plurality of prompts. In response to the aforementioned prompt, the system receives an audio data item corresponding to a human voice, By processing the received audio data items using the aforementioned mathematical model, a medical diagnosis can be derived. Displaying the aforementioned medical diagnosis One or more non-temporary computer-readable media, including [the specified text].

16. One or more non-temporal computer-readable media according to claim 15, characterized in that they are calculated using a plurality of values, the plurality of values ​​being calculated by calculating a value for each short-time segment of an audio signal.

17. One or more non-temporary computer-readable media according to claim 15, wherein each feature is associated with a feature selection score, the feature selection score includes a modulated Rand index, modulated mutual information, absolute Pearson correlation, or absolute Spearman correlation.

18. One or more non-transient computer-readable media according to claim 15, wherein the plurality of features are further selected based on a plurality of stability determinations calculated for each feature.

19. Each audio data item of the training corpus corresponds to one of a plurality of prompts, the plurality of prompts includes the presented prompt, The aforementioned multiple Actions are, By processing the multiple audio data items using the aforementioned mathematical model, a medical diagnostic score is calculated for each audio data item in the training corpus. Using the aforementioned medical diagnostic scores, a prompt selection score is calculated for each of the aforementioned prompts. Using the plurality of prompt selection scores, select a subset of the plurality of prompts, wherein the subset of the plurality of prompts includes the presented prompt. Using the mathematical model and the subset of the multiple prompts, deploy a computer program product or computer service for detecting the medical condition. One or more non-temporary computer-readable media according to claim 15, including:

20. One or more non-transient computer-readable media according to claim 15, wherein the plurality of features include non-speech features.

Images